2018-01-05T11:28:26,533 INFO [main] control.CoreCliDriver: Starting org.apache.hadoop.hive.cli.control.CoreCliDriver run at 1515180506529 2018-01-05T11:28:26,545 INFO [main] QTestUtil: Setting up QTestUtil with outDir=/Users/djaiswal/hive/hive/ql/src/test/results/clientpositive/llap, logDir=/Users/djaiswal/hive/hive/itests/qtest/target/qfile-results/clientpositive, clusterType=llap_local, confDir=/Users/djaiswal/hive/hive/data/conf/llap, hadoopVer=null, initScript=q_test_init.sql, cleanupScript=q_test_cleanup.sql, withLlapIo=true, fsType=local 2018-01-05T11:28:26,573 INFO [main] conf.HiveConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hive-site.xml 2018-01-05T11:28:26,896 DEBUG [main] util.Shell: Failed to detect a valid hadoop home directory java.io.FileNotFoundException: HADOOP_HOME and hadoop.home.dir are unset. at org.apache.hadoop.util.Shell.checkHadoopHomeInner(Shell.java:454) ~[hadoop-common-3.0.0-beta1.jar:?] at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:425) ~[hadoop-common-3.0.0-beta1.jar:?] at org.apache.hadoop.util.Shell.(Shell.java:502) [hadoop-common-3.0.0-beta1.jar:?] at org.apache.hadoop.util.StringUtils.(StringUtils.java:78) [hadoop-common-3.0.0-beta1.jar:?] at org.apache.hadoop.conf.Configuration.getBoolean(Configuration.java:1555) [hadoop-common-3.0.0-beta1.jar:?] at org.apache.hadoop.hive.conf.HiveConf.getBoolVar(HiveConf.java:4193) [hive-common-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.conf.HiveConf.getBoolVar(HiveConf.java:4220) [hive-common-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:4427) [hive-common-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.conf.HiveConf.(HiveConf.java:4338) [hive-common-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.QTestUtil.(QTestUtil.java:561) [classes/:?] at org.apache.hadoop.hive.cli.control.CoreCliDriver$1.invokeInternal(CoreCliDriver.java:65) [classes/:?] at org.apache.hadoop.hive.cli.control.CoreCliDriver$1.invokeInternal(CoreCliDriver.java:61) [classes/:?] at org.apache.hadoop.hive.util.ElapsedTimeLoggingWrapper.invoke(ElapsedTimeLoggingWrapper.java:33) [classes/:?] at org.apache.hadoop.hive.cli.control.CoreCliDriver.beforeClass(CoreCliDriver.java:67) [classes/:?] at org.apache.hadoop.hive.cli.control.CliAdapter$1$1.evaluate(CliAdapter.java:71) [classes/:?] at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.11.jar:?] at org.junit.runners.ParentRunner.run(ParentRunner.java:309) [junit-4.11.jar:?] at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:369) [surefire-junit4-2.20.1.jar:2.20.1] at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:275) [surefire-junit4-2.20.1.jar:2.20.1] at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:239) [surefire-junit4-2.20.1.jar:2.20.1] at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:160) [surefire-junit4-2.20.1.jar:2.20.1] at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:373) [surefire-booter-2.20.1.jar:2.20.1] at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:334) [surefire-booter-2.20.1.jar:2.20.1] at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:119) [surefire-booter-2.20.1.jar:2.20.1] at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:407) [surefire-booter-2.20.1.jar:2.20.1] 2018-01-05T11:28:26,908 DEBUG [main] util.Shell: setsid is not available on this machine. So not using it. 2018-01-05T11:28:26,908 DEBUG [main] util.Shell: setsid exited with exit code 0 2018-01-05T11:28:26,909 WARN [main] conf.HiveConf: HiveConf of name hive.dummyparam.test.server.specific.config.override does not exist 2018-01-05T11:28:26,909 WARN [main] conf.HiveConf: HiveConf of name hive.metastore.metadb.dir does not exist 2018-01-05T11:28:26,910 WARN [main] conf.HiveConf: HiveConf of name hive.dummyparam.test.server.specific.config.metastoresite does not exist 2018-01-05T11:28:26,910 WARN [main] conf.HiveConf: HiveConf of name hive.llap.daemon.service.port does not exist 2018-01-05T11:28:27,010 DEBUG [main] util.NativeCodeLoader: Trying to load the custom-built native-hadoop library... 2018-01-05T11:28:27,011 DEBUG [main] util.NativeCodeLoader: Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path 2018-01-05T11:28:27,011 DEBUG [main] util.NativeCodeLoader: java.library.path=/Users/djaiswal/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:. 2018-01-05T11:28:27,011 WARN [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2018-01-05T11:28:27,012 DEBUG [main] util.PerformanceAdvisory: Falling back to shell based 2018-01-05T11:28:27,056 DEBUG [main] core.Tracer: sampler.classes = ; loaded no samplers 2018-01-05T11:28:27,061 DEBUG [main] core.Tracer: span.receiver.classes = ; loaded no span receivers 2018-01-05T11:28:27,061 DEBUG [main] fs.FileSystem: Loading filesystems 2018-01-05T11:28:27,063 DEBUG [main] fs.FileSystem: nullscan:// = class org.apache.hadoop.hive.ql.io.NullScanFileSystem from /Users/djaiswal/.m2/repository/org/apache/hive/hive-exec/3.0.0-SNAPSHOT/hive-exec-3.0.0-SNAPSHOT.jar 2018-01-05T11:28:27,063 DEBUG [main] fs.FileSystem: file:// = class org.apache.hadoop.fs.LocalFileSystem from /Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-common/3.0.0-beta1/hadoop-common-3.0.0-beta1.jar 2018-01-05T11:28:27,064 DEBUG [main] fs.FileSystem: file:// = class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem from /Users/djaiswal/.m2/repository/org/apache/hive/hive-exec/3.0.0-SNAPSHOT/hive-exec-3.0.0-SNAPSHOT.jar 2018-01-05T11:28:27,067 DEBUG [main] fs.FileSystem: s3:// = class org.apache.hadoop.fs.s3.S3FileSystem from /Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-aws/2.7.3/hadoop-aws-2.7.3.jar 2018-01-05T11:28:27,071 DEBUG [main] fs.FileSystem: s3n:// = class org.apache.hadoop.fs.s3native.NativeS3FileSystem from /Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-aws/2.7.3/hadoop-aws-2.7.3.jar 2018-01-05T11:28:27,078 DEBUG [main] fs.FileSystem: s3a:// = class org.apache.hadoop.fs.s3a.S3AFileSystem from /Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-aws/2.7.3/hadoop-aws-2.7.3.jar 2018-01-05T11:28:27,083 DEBUG [main] fs.FileSystem: viewfs:// = class org.apache.hadoop.fs.viewfs.ViewFileSystem from /Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-common/3.0.0-beta1/hadoop-common-3.0.0-beta1.jar 2018-01-05T11:28:27,086 DEBUG [main] fs.FileSystem: ftp:// = class org.apache.hadoop.fs.ftp.FTPFileSystem from /Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-common/3.0.0-beta1/hadoop-common-3.0.0-beta1.jar 2018-01-05T11:28:27,088 DEBUG [main] fs.FileSystem: har:// = class org.apache.hadoop.fs.HarFileSystem from /Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-common/3.0.0-beta1/hadoop-common-3.0.0-beta1.jar 2018-01-05T11:28:27,090 DEBUG [main] fs.FileSystem: http:// = class org.apache.hadoop.fs.http.HttpFileSystem from /Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-common/3.0.0-beta1/hadoop-common-3.0.0-beta1.jar 2018-01-05T11:28:27,090 DEBUG [main] fs.FileSystem: https:// = class org.apache.hadoop.fs.http.HttpsFileSystem from /Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-common/3.0.0-beta1/hadoop-common-3.0.0-beta1.jar 2018-01-05T11:28:27,096 DEBUG [main] fs.FileSystem: hdfs:// = class org.apache.hadoop.hdfs.DistributedFileSystem from /Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-hdfs-client/3.0.0-beta1/hadoop-hdfs-client-3.0.0-beta1.jar 2018-01-05T11:28:27,275 DEBUG [main] fs.FileSystem: webhdfs:// = class org.apache.hadoop.hdfs.web.WebHdfsFileSystem from /Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-hdfs-client/3.0.0-beta1/hadoop-hdfs-client-3.0.0-beta1.jar 2018-01-05T11:28:27,281 DEBUG [main] fs.FileSystem: swebhdfs:// = class org.apache.hadoop.hdfs.web.SWebHdfsFileSystem from /Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-hdfs-client/3.0.0-beta1/hadoop-hdfs-client-3.0.0-beta1.jar 2018-01-05T11:28:27,281 DEBUG [main] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:27,281 DEBUG [main] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:27,358 DEBUG [main] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:27,358 DEBUG [main] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:27,440 INFO [main] server.ZooKeeperServer: Server environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT 2018-01-05T11:28:27,441 INFO [main] server.ZooKeeperServer: Server environment:host.name=10.22.8.182 2018-01-05T11:28:27,441 INFO [main] server.ZooKeeperServer: Server environment:java.version=1.8.0_101 2018-01-05T11:28:27,441 INFO [main] server.ZooKeeperServer: Server environment:java.vendor=Oracle Corporation 2018-01-05T11:28:27,441 INFO [main] server.ZooKeeperServer: Server environment:java.home=/Library/Java/JavaVirtualMachines/jdk1.8.0_101.jdk/Contents/Home/jre 2018-01-05T11:28:27,441 INFO [main] server.ZooKeeperServer: Server environment:java.class.path=/Users/djaiswal/hive/hive/itests/qtest/target/test-classes:/Users/djaiswal/hive/hive/itests/qtest/target/classes:/Users/djaiswal/.m2/repository/org/apache/hive/hive-common/3.0.0-SNAPSHOT/hive-common-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-classification/3.0.0-SNAPSHOT/hive-classification-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-shims/3.0.0-SNAPSHOT/hive-shims-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/shims/hive-shims-common/3.0.0-SNAPSHOT/hive-shims-common-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/shims/hive-shims-0.23/3.0.0-SNAPSHOT/hive-shims-0.23-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/shims/hive-shims-scheduler/3.0.0-SNAPSHOT/hive-shims-scheduler-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-storage-api/3.0.0-SNAPSHOT/hive-storage-api-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/commons-cli/commons-cli/1.2/commons-cli-1.2.jar:/Users/djaiswal/.m2/repository/commons-lang/commons-lang/2.6/commons-lang-2.6.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-lang3/3.2/commons-lang3-3.2.jar:/Users/djaiswal/.m2/repository/org/apache/orc/orc-core/1.4.1/orc-core-1.4.1.jar:/Users/djaiswal/.m2/repository/io/airlift/aircompressor/0.8/aircompressor-0.8.jar:/Users/djaiswal/.m2/repository/io/airlift/slice/0.29/slice-0.29.jar:/Users/djaiswal/.m2/repository/org/openjdk/jol/jol-core/0.2/jol-core-0.2.jar:/Users/djaiswal/.m2/repository/jline/jline/2.12/jline-2.12.jar:/Users/djaiswal/.m2/repository/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-rewrite/9.3.8.v20160314/jetty-rewrite-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-client/9.3.8.v20160314/jetty-client-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-server/9.3.8.v20160314/jetty-server-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-http/9.3.8.v20160314/jetty-http-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-io/9.3.8.v20160314/jetty-io-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-servlet/9.3.8.v20160314/jetty-servlet-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-webapp/9.3.8.v20160314/jetty-webapp-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-xml/9.3.8.v20160314/jetty-xml-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/joda-time/joda-time/2.8.1/joda-time-2.8.1.jar:/Users/djaiswal/.m2/repository/org/apache/logging/log4j/log4j-1.2-api/2.8.2/log4j-1.2-api-2.8.2.jar:/Users/djaiswal/.m2/repository/org/apache/logging/log4j/log4j-web/2.8.2/log4j-web-2.8.2.jar:/Users/djaiswal/.m2/repository/org/apache/logging/log4j/log4j-slf4j-impl/2.8.2/log4j-slf4j-impl-2.8.2.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-compress/1.9/commons-compress-1.9.jar:/Users/djaiswal/.m2/repository/org/apache/ant/ant/1.9.1/ant-1.9.1.jar:/Users/djaiswal/.m2/repository/org/apache/ant/ant-launcher/1.9.1/ant-launcher-1.9.1.jar:/Users/djaiswal/.m2/repository/com/tdunning/json/1.8/json-1.8.jar:/Users/djaiswal/.m2/repository/io/dropwizard/metrics/metrics-core/3.1.0/metrics-core-3.1.0.jar:/Users/djaiswal/.m2/repository/io/dropwizard/metrics/metrics-jvm/3.1.0/metrics-jvm-3.1.0.jar:/Users/djaiswal/.m2/repository/io/dropwizard/metrics/metrics-json/3.1.0/metrics-json-3.1.0.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/core/jackson-databind/2.6.5/jackson-databind-2.6.5.jar:/Users/djaiswal/.m2/repository/com/github/joshelser/dropwizard-metrics-hadoop-metrics2-reporter/0.1.2/dropwizard-metrics-hadoop-metrics2-reporter-0.1.2.jar:/Users/djaiswal/.m2/repository/javolution/javolution/5.5.1/javolution-5.5.1.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-contrib/3.0.0-SNAPSHOT/hive-contrib-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/commons-codec/commons-codec/1.4/commons-codec-1.4.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-standalone-metastore/3.0.0-SNAPSHOT/hive-standalone-metastore-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/com/google/guava/guava/19.0/guava-19.0.jar:/Users/djaiswal/.m2/repository/com/google/protobuf/protobuf-java/2.5.0/protobuf-java-2.5.0.jar:/Users/djaiswal/.m2/repository/com/jolbox/bonecp/0.8.0.RELEASE/bonecp-0.8.0.RELEASE.jar:/Users/djaiswal/.m2/repository/com/zaxxer/HikariCP/2.6.1/HikariCP-2.6.1.jar:/Users/djaiswal/.m2/repository/commons-dbcp/commons-dbcp/1.4/commons-dbcp-1.4.jar:/Users/djaiswal/.m2/repository/commons-pool/commons-pool/1.5.4/commons-pool-1.5.4.jar:/Users/djaiswal/.m2/repository/org/antlr/antlr-runtime/3.5.2/antlr-runtime-3.5.2.jar:/Users/djaiswal/.m2/repository/org/apache/derby/derby/10.11.1.1/derby-10.11.1.1.jar:/Users/djaiswal/.m2/repository/org/apache/thrift/libfb303/0.9.3/libfb303-0.9.3.jar:/Users/djaiswal/.m2/repository/org/apache/thrift/libthrift/0.9.3/libthrift-0.9.3.jar:/Users/djaiswal/.m2/repository/org/apache/httpcomponents/httpcore/4.4.4/httpcore-4.4.4.jar:/Users/djaiswal/.m2/repository/org/datanucleus/datanucleus-api-jdo/4.2.4/datanucleus-api-jdo-4.2.4.jar:/Users/djaiswal/.m2/repository/org/datanucleus/datanucleus-core/4.1.17/datanucleus-core-4.1.17.jar:/Users/djaiswal/.m2/repository/org/datanucleus/datanucleus-rdbms/4.1.19/datanucleus-rdbms-4.1.19.jar:/Users/djaiswal/.m2/repository/org/datanucleus/javax.jdo/3.2.0-m3/javax.jdo-3.2.0-m3.jar:/Users/djaiswal/.m2/repository/javax/transaction/transaction-api/1.1/transaction-api-1.1.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-standalone-metastore/3.0.0-SNAPSHOT/hive-standalone-metastore-3.0.0-SNAPSHOT-tests.jar:/Users/djaiswal/hive/hive/itests/custom-serde/target/classes:/Users/djaiswal/hive/hive/itests/hive-unit/target/test-classes:/Users/djaiswal/.m2/repository/org/apache/hive/hive-jdbc/3.0.0-SNAPSHOT/hive-jdbc-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-service/3.0.0-SNAPSHOT/hive-service-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/javax/servlet/jsp/javax.servlet.jsp-api/2.3.1/javax.servlet.jsp-api-2.3.1.jar:/Users/djaiswal/.m2/repository/net/sf/jpam/jpam/1.1/jpam-1.1.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-runner/9.3.8.v20160314/jetty-runner-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-plus/9.3.8.v20160314/jetty-plus-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-annotations/9.3.8.v20160314/jetty-annotations-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-jaas/9.3.8.v20160314/jetty-jaas-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/websocket/websocket-server/9.3.8.v20160314/websocket-server-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/websocket/websocket-common/9.3.8.v20160314/websocket-common-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/websocket/websocket-api/9.3.8.v20160314/websocket-api-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/websocket/websocket-client/9.3.8.v20160314/websocket-client-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/websocket/websocket-servlet/9.3.8.v20160314/websocket-servlet-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-jndi/9.3.8.v20160314/jetty-jndi-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/apache-jsp/9.3.8.v20160314/apache-jsp-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/toolchain/jetty-schemas/3.1/jetty-schemas-3.1.jar:/Users/djaiswal/.m2/repository/org/eclipse/jdt/core/compiler/ecj/4.4.2/ecj-4.4.2.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/apache-jstl/9.3.8.v20160314/apache-jstl-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/apache/taglibs/taglibs-standard-spec/1.2.5/taglibs-standard-spec-1.2.5.jar:/Users/djaiswal/.m2/repository/org/apache/taglibs/taglibs-standard-impl/1.2.5/taglibs-standard-impl-1.2.5.jar:/Users/djaiswal/.m2/repository/tomcat/jasper-compiler/5.5.23/jasper-compiler-5.5.23.jar:/Users/djaiswal/.m2/repository/javax/servlet/jsp-api/2.0/jsp-api-2.0.jar:/Users/djaiswal/.m2/repository/ant/ant/1.6.5/ant-1.6.5.jar:/Users/djaiswal/.m2/repository/tomcat/jasper-runtime/5.5.23/jasper-runtime-5.5.23.jar:/Users/djaiswal/.m2/repository/commons-el/commons-el/1.0/commons-el-1.0.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-llap-ext-client/3.0.0-SNAPSHOT/hive-llap-ext-client-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hcatalog/hive-hcatalog-core/3.0.0-SNAPSHOT/hive-hcatalog-core-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.9.13/jackson-mapper-asl-1.9.13.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hcatalog/hive-hcatalog-streaming/3.0.0-SNAPSHOT/hive-hcatalog-streaming-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hcatalog/hive-hcatalog-server-extensions/3.0.0-SNAPSHOT/hive-hcatalog-server-extensions-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/javax/jms/jms/1.1/jms-1.1.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hcatalog/hive-webhcat-java-client/3.0.0-SNAPSHOT/hive-webhcat-java-client-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-service/3.0.0-SNAPSHOT/hive-service-3.0.0-SNAPSHOT-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-exec/3.0.0-SNAPSHOT/hive-exec-3.0.0-SNAPSHOT-tests.jar:/Users/djaiswal/hive/hive/itests/util/target/classes:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-minicluster/1.7.3/accumulo-minicluster-1.7.3.jar:/Users/djaiswal/.m2/repository/com/beust/jcommander/1.32/jcommander-1.32.jar:/Users/djaiswal/.m2/repository/commons-configuration/commons-configuration/1.6/commons-configuration-1.6.jar:/Users/djaiswal/.m2/repository/commons-digester/commons-digester/1.8/commons-digester-1.8.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-core/1.7.3/accumulo-core-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-math/2.1/commons-math-2.1.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-gc/1.7.3/accumulo-gc-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-master/1.7.3/accumulo-master-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-monitor/1.7.3/accumulo-monitor-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-shell/1.7.3/accumulo-shell-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-server-base/1.7.3/accumulo-server-base-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-start/1.7.3/accumulo-start-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-tracer/1.7.3/accumulo-tracer-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-tserver/1.7.3/accumulo-tserver-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-vfs2/2.1/commons-vfs2-2.1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-minicluster/3.0.0-beta1/hadoop-minicluster-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-accumulo-handler/3.0.0-SNAPSHOT/hive-accumulo-handler-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-fate/1.7.3/accumulo-fate-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-trace/1.7.3/accumulo-trace-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-accumulo-handler/3.0.0-SNAPSHOT/hive-accumulo-handler-3.0.0-SNAPSHOT-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-cli/3.0.0-SNAPSHOT/hive-cli-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-beeline/3.0.0-SNAPSHOT/hive-beeline-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/net/sf/supercsv/super-csv/2.2.0/super-csv-2.2.0.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-hbase-handler/3.0.0-SNAPSHOT/hive-hbase-handler-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-hbase-handler/3.0.0-SNAPSHOT/hive-hbase-handler-3.0.0-SNAPSHOT-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-metastore/3.0.0-SNAPSHOT/hive-metastore-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/javax/jdo/jdo-api/3.0.1/jdo-api-3.0.1.jar:/Users/djaiswal/.m2/repository/javax/transaction/jta/1.1/jta-1.1.jar:/Users/djaiswal/.m2/repository/co/cask/tephra/tephra-api/0.6.0/tephra-api-0.6.0.jar:/Users/djaiswal/.m2/repository/co/cask/tephra/tephra-core/0.6.0/tephra-core-0.6.0.jar:/Users/djaiswal/.m2/repository/com/google/inject/extensions/guice-assistedinject/3.0/guice-assistedinject-3.0.jar:/Users/djaiswal/.m2/repository/org/apache/twill/twill-common/0.6.0-incubating/twill-common-0.6.0-incubating.jar:/Users/djaiswal/.m2/repository/org/apache/twill/twill-core/0.6.0-incubating/twill-core-0.6.0-incubating.jar:/Users/djaiswal/.m2/repository/org/apache/twill/twill-api/0.6.0-incubating/twill-api-0.6.0-incubating.jar:/Users/djaiswal/.m2/repository/org/apache/twill/twill-discovery-api/0.6.0-incubating/twill-discovery-api-0.6.0-incubating.jar:/Users/djaiswal/.m2/repository/org/apache/twill/twill-discovery-core/0.6.0-incubating/twill-discovery-core-0.6.0-incubating.jar:/Users/djaiswal/.m2/repository/org/apache/twill/twill-zookeeper/0.6.0-incubating/twill-zookeeper-0.6.0-incubating.jar:/Users/djaiswal/.m2/repository/co/cask/tephra/tephra-hbase-compat-1.0/0.6.0/tephra-hbase-compat-1.0-0.6.0.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-client/2.0.0-alpha4/hbase-client-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/jruby/jcodings/jcodings/1.0.18/jcodings-1.0.18.jar:/Users/djaiswal/.m2/repository/org/jruby/joni/joni/2.1.11/joni-2.1.11.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-mapreduce/2.0.0-alpha4/hbase-mapreduce-2.0.0-alpha4-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-serde/3.0.0-SNAPSHOT/hive-serde-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-service-rpc/3.0.0-SNAPSHOT/hive-service-rpc-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/com/google/code/findbugs/jsr305/3.0.0/jsr305-3.0.0.jar:/Users/djaiswal/.m2/repository/org/apache/avro/avro/1.7.7/avro-1.7.7.jar:/Users/djaiswal/.m2/repository/org/codehaus/jackson/jackson-core-asl/1.9.13/jackson-core-asl-1.9.13.jar:/Users/djaiswal/.m2/repository/com/thoughtworks/paranamer/paranamer/2.3/paranamer-2.3.jar:/Users/djaiswal/.m2/repository/org/xerial/snappy/snappy-java/1.1.4/snappy-java-1.1.4.jar:/Users/djaiswal/.m2/repository/net/sf/opencsv/opencsv/2.3/opencsv-2.3.jar:/Users/djaiswal/.m2/repository/org/apache/parquet/parquet-hadoop-bundle/1.9.0/parquet-hadoop-bundle-1.9.0.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-exec/3.0.0-SNAPSHOT/hive-exec-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-vector-code-gen/3.0.0-SNAPSHOT/hive-vector-code-gen-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/velocity/velocity/1.5/velocity-1.5.jar:/Users/djaiswal/.m2/repository/oro/oro/2.0.8/oro-2.0.8.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-llap-tez/3.0.0-SNAPSHOT/hive-llap-tez-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/commons-httpclient/commons-httpclient/3.0.1/commons-httpclient-3.0.1.jar:/Users/djaiswal/.m2/repository/commons-io/commons-io/2.4/commons-io-2.4.jar:/Users/djaiswal/.m2/repository/org/antlr/ST4/4.0.4/ST4-4.0.4.jar:/Users/djaiswal/.m2/repository/org/apache/ivy/ivy/2.4.0/ivy-2.4.0.jar:/Users/djaiswal/.m2/repository/org/apache/zookeeper/zookeeper/3.4.6/zookeeper-3.4.6.jar:/Users/djaiswal/.m2/repository/org/apache/curator/curator-framework/2.12.0/curator-framework-2.12.0.jar:/Users/djaiswal/.m2/repository/org/codehaus/groovy/groovy-all/2.4.11/groovy-all-2.4.11.jar:/Users/djaiswal/.m2/repository/org/apache/calcite/calcite-core/1.15.0/calcite-core-1.15.0.jar:/Users/djaiswal/.m2/repository/org/apache/calcite/calcite-linq4j/1.15.0/calcite-linq4j-1.15.0.jar:/Users/djaiswal/.m2/repository/com/esri/geometry/esri-geometry-api/2.0.0/esri-geometry-api-2.0.0.jar:/Users/djaiswal/.m2/repository/com/yahoo/datasketches/sketches-core/0.9.0/sketches-core-0.9.0.jar:/Users/djaiswal/.m2/repository/com/yahoo/datasketches/memory/0.9.0/memory-0.9.0.jar:/Users/djaiswal/.m2/repository/org/codehaus/janino/janino/2.7.6/janino-2.7.6.jar:/Users/djaiswal/.m2/repository/org/codehaus/janino/commons-compiler/2.7.6/commons-compiler-2.7.6.jar:/Users/djaiswal/.m2/repository/org/apache/calcite/calcite-druid/1.15.0/calcite-druid-1.15.0.jar:/Users/djaiswal/.m2/repository/org/apache/calcite/avatica/avatica/1.10.0/avatica-1.10.0.jar:/Users/djaiswal/.m2/repository/com/google/code/gson/gson/2.2.4/gson-2.2.4.jar:/Users/djaiswal/.m2/repository/stax/stax-api/1.0.1/stax-api-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-registry/3.0.0-beta1/hadoop-yarn-registry-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-druid-handler/3.0.0-SNAPSHOT/hive-druid-handler-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/com/metamx/java-util/0.27.10/java-util-0.27.10.jar:/Users/djaiswal/.m2/repository/org/skife/config/config-magic/0.9/config-magic-0.9.jar:/Users/djaiswal/.m2/repository/org/mozilla/rhino/1.7R5/rhino-1.7R5.jar:/Users/djaiswal/.m2/repository/com/jayway/jsonpath/json-path/2.1.0/json-path-2.1.0.jar:/Users/djaiswal/.m2/repository/io/druid/druid-server/0.10.1/druid-server-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/druid-console/0.0.2/druid-console-0.0.2.jar:/Users/djaiswal/.m2/repository/com/metamx/http-client/1.1.0/http-client-1.1.0.jar:/Users/djaiswal/.m2/repository/com/metamx/server-metrics/0.2.8/server-metrics-0.2.8.jar:/Users/djaiswal/.m2/repository/javax/inject/javax.inject/1/javax.inject-1.jar:/Users/djaiswal/.m2/repository/org/glassfish/javax.el/3.0.0/javax.el-3.0.0.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/jaxrs/jackson-jaxrs-smile-provider/2.4.6/jackson-jaxrs-smile-provider-2.4.6.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/dataformat/jackson-dataformat-smile/2.4.6/jackson-dataformat-smile-2.4.6.jar:/Users/djaiswal/.m2/repository/io/tesla/aether/tesla-aether/0.0.5/tesla-aether-0.0.5.jar:/Users/djaiswal/.m2/repository/org/eclipse/aether/aether-spi/0.9.0.M2/aether-spi-0.9.0.M2.jar:/Users/djaiswal/.m2/repository/org/eclipse/aether/aether-util/0.9.0.M2/aether-util-0.9.0.M2.jar:/Users/djaiswal/.m2/repository/org/eclipse/aether/aether-impl/0.9.0.M2/aether-impl-0.9.0.M2.jar:/Users/djaiswal/.m2/repository/org/eclipse/aether/aether-connector-file/0.9.0.M2/aether-connector-file-0.9.0.M2.jar:/Users/djaiswal/.m2/repository/io/tesla/aether/aether-connector-okhttp/0.0.9/aether-connector-okhttp-0.0.9.jar:/Users/djaiswal/.m2/repository/org/apache/maven/wagon/wagon-provider-api/2.4/wagon-provider-api-2.4.jar:/Users/djaiswal/.m2/repository/org/apache/maven/maven-aether-provider/3.1.1/maven-aether-provider-3.1.1.jar:/Users/djaiswal/.m2/repository/org/apache/maven/maven-model/3.1.1/maven-model-3.1.1.jar:/Users/djaiswal/.m2/repository/org/apache/maven/maven-model-builder/3.1.1/maven-model-builder-3.1.1.jar:/Users/djaiswal/.m2/repository/org/apache/maven/maven-repository-metadata/3.1.1/maven-repository-metadata-3.1.1.jar:/Users/djaiswal/.m2/repository/org/codehaus/plexus/plexus-utils/3.0.15/plexus-utils-3.0.15.jar:/Users/djaiswal/.m2/repository/org/apache/maven/maven-settings-builder/3.1.1/maven-settings-builder-3.1.1.jar:/Users/djaiswal/.m2/repository/org/codehaus/plexus/plexus-interpolation/1.19/plexus-interpolation-1.19.jar:/Users/djaiswal/.m2/repository/org/apache/maven/maven-settings/3.1.1/maven-settings-3.1.1.jar:/Users/djaiswal/.m2/repository/org/eclipse/aether/aether-api/0.9.0.M2/aether-api-0.9.0.M2.jar:/Users/djaiswal/.m2/repository/net/spy/spymemcached/2.12.3/spymemcached-2.12.3.jar:/Users/djaiswal/.m2/repository/net/jpountz/lz4/lz4/1.3.0/lz4-1.3.0.jar:/Users/djaiswal/.m2/repository/com/ircclouds/irc/irc-api/1.0-0014/irc-api-1.0-0014.jar:/Users/djaiswal/.m2/repository/com/maxmind/geoip2/geoip2/0.4.0/geoip2-0.4.0.jar:/Users/djaiswal/.m2/repository/com/maxmind/maxminddb/maxminddb/0.2.0/maxminddb-0.2.0.jar:/Users/djaiswal/.m2/repository/org/apache/derby/derbynet/10.11.1.1/derbynet-10.11.1.1.jar:/Users/djaiswal/.m2/repository/org/apache/derby/derbyclient/10.11.1.1/derbyclient-10.11.1.1.jar:/Users/djaiswal/.m2/repository/it/unimi/dsi/fastutil/7.2.0/fastutil-7.2.0.jar:/Users/djaiswal/.m2/repository/io/druid/druid-processing/0.10.1/druid-processing-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/druid-common/0.10.1/druid-common-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/java-util/0.10.1/java-util-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/druid-api/0.10.1/druid-api-0.10.1.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-dbcp2/2.0.1/commons-dbcp2-2.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-pool2/2.2/commons-pool2-2.2.jar:/Users/djaiswal/.m2/repository/org/hibernate/hibernate-validator/5.1.3.Final/hibernate-validator-5.1.3.Final.jar:/Users/djaiswal/.m2/repository/org/jboss/logging/jboss-logging/3.1.3.GA/jboss-logging-3.1.3.GA.jar:/Users/djaiswal/.m2/repository/com/fasterxml/classmate/1.0.0/classmate-1.0.0.jar:/Users/djaiswal/.m2/repository/javax/el/javax.el-api/3.0.0/javax.el-api-3.0.0.jar:/Users/djaiswal/.m2/repository/javax/validation/validation-api/1.1.0.Final/validation-api-1.1.0.Final.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/datatype/jackson-datatype-guava/2.4.6/jackson-datatype-guava-2.4.6.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/datatype/jackson-datatype-joda/2.4.6/jackson-datatype-joda-2.4.6.jar:/Users/djaiswal/.m2/repository/com/google/inject/extensions/guice-multibindings/4.1.0/guice-multibindings-4.1.0.jar:/Users/djaiswal/.m2/repository/org/jdbi/jdbi/2.63.1/jdbi-2.63.1.jar:/Users/djaiswal/.m2/repository/org/apache/logging/log4j/log4j-jul/2.5/log4j-jul-2.5.jar:/Users/djaiswal/.m2/repository/org/slf4j/jcl-over-slf4j/1.7.12/jcl-over-slf4j-1.7.12.jar:/Users/djaiswal/.m2/repository/net/java/dev/jets3t/jets3t/0.9.4/jets3t-0.9.4.jar:/Users/djaiswal/.m2/repository/javax/activation/activation/1.1.1/activation-1.1.1.jar:/Users/djaiswal/.m2/repository/org/bouncycastle/bcprov-jdk15on/1.52/bcprov-jdk15on-1.52.jar:/Users/djaiswal/.m2/repository/com/jamesmurty/utils/java-xmlbuilder/1.1/java-xmlbuilder-1.1.jar:/Users/djaiswal/.m2/repository/net/iharder/base64/2.3.8/base64-2.3.8.jar:/Users/djaiswal/.m2/repository/org/antlr/antlr4-runtime/4.5.1/antlr4-runtime-4.5.1.jar:/Users/djaiswal/.m2/repository/io/druid/druid-hll/0.10.1/druid-hll-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/bytebuffer-collections/0.10.1/bytebuffer-collections-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/extendedset/0.10.1/extendedset-0.10.1.jar:/Users/djaiswal/.m2/repository/com/metamx/emitter/0.4.5/emitter-0.4.5.jar:/Users/djaiswal/.m2/repository/com/ning/compress-lzf/1.0.3/compress-lzf-1.0.3.jar:/Users/djaiswal/.m2/repository/com/ibm/icu/icu4j/54.1.1/icu4j-54.1.1.jar:/Users/djaiswal/.m2/repository/org/mapdb/mapdb/1.0.8/mapdb-1.0.8.jar:/Users/djaiswal/.m2/repository/org/ow2/asm/asm/5.2/asm-5.2.jar:/Users/djaiswal/.m2/repository/org/ow2/asm/asm-commons/5.2/asm-commons-5.2.jar:/Users/djaiswal/.m2/repository/org/ow2/asm/asm-tree/5.2/asm-tree-5.2.jar:/Users/djaiswal/.m2/repository/io/druid/extensions/druid-hdfs-storage/0.10.1/druid-hdfs-storage-0.10.1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-aws/2.7.3/hadoop-aws-2.7.3.jar:/Users/djaiswal/.m2/repository/com/amazonaws/aws-java-sdk-s3/1.10.77/aws-java-sdk-s3-1.10.77.jar:/Users/djaiswal/.m2/repository/com/amazonaws/aws-java-sdk-kms/1.10.77/aws-java-sdk-kms-1.10.77.jar:/Users/djaiswal/.m2/repository/com/amazonaws/aws-java-sdk-core/1.10.77/aws-java-sdk-core-1.10.77.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/dataformat/jackson-dataformat-cbor/2.5.3/jackson-dataformat-cbor-2.5.3.jar:/Users/djaiswal/.m2/repository/io/druid/extensions/mysql-metadata-storage/0.10.1/mysql-metadata-storage-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/extensions/postgresql-metadata-storage/0.10.1/postgresql-metadata-storage-0.10.1.jar:/Users/djaiswal/.m2/repository/org/postgresql/postgresql/9.4.1208.jre7/postgresql-9.4.1208.jre7.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-druid-handler/3.0.0-SNAPSHOT/hive-druid-handler-3.0.0-SNAPSHOT-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-jdbc-handler/3.0.0-SNAPSHOT/hive-jdbc-handler-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/junit/junit/4.11/junit-4.11.jar:/Users/djaiswal/.m2/repository/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar:/Users/djaiswal/.m2/repository/com/sun/jersey/jersey-servlet/1.19/jersey-servlet-1.19.jar:/Users/djaiswal/.m2/repository/com/sun/jersey/jersey-server/1.19/jersey-server-1.19.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-archives/3.0.0-beta1/hadoop-archives-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-common/3.0.0-beta1/hadoop-common-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-annotations/3.0.0-beta1/hadoop-annotations-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-math3/3.1.1/commons-math3-3.1.1.jar:/Users/djaiswal/.m2/repository/commons-net/commons-net/3.1/commons-net-3.1.jar:/Users/djaiswal/.m2/repository/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-util/9.3.19.v20170502/jetty-util-9.3.19.v20170502.jar:/Users/djaiswal/.m2/repository/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.jar:/Users/djaiswal/.m2/repository/com/sun/jersey/jersey-core/1.19/jersey-core-1.19.jar:/Users/djaiswal/.m2/repository/javax/ws/rs/jsr311-api/1.1.1/jsr311-api-1.1.1.jar:/Users/djaiswal/.m2/repository/com/sun/jersey/jersey-json/1.19/jersey-json-1.19.jar:/Users/djaiswal/.m2/repository/com/sun/xml/bind/jaxb-impl/2.2.3-1/jaxb-impl-2.2.3-1.jar:/Users/djaiswal/.m2/repository/org/codehaus/jackson/jackson-jaxrs/1.9.13/jackson-jaxrs-1.9.13.jar:/Users/djaiswal/.m2/repository/org/codehaus/jackson/jackson-xc/1.9.13/jackson-xc-1.9.13.jar:/Users/djaiswal/.m2/repository/commons-logging/commons-logging/1.1.3/commons-logging-1.1.3.jar:/Users/djaiswal/.m2/repository/log4j/log4j/1.2.17/log4j-1.2.17.jar:/Users/djaiswal/.m2/repository/commons-beanutils/commons-beanutils/1.9.3/commons-beanutils-1.9.3.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-configuration2/2.1.1/commons-configuration2-2.1.1.jar:/Users/djaiswal/.m2/repository/com/google/re2j/re2j/1.1/re2j-1.1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-auth/3.0.0-beta1/hadoop-auth-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/com/nimbusds/nimbus-jose-jwt/4.41.1/nimbus-jose-jwt-4.41.1.jar:/Users/djaiswal/.m2/repository/com/github/stephenc/jcip/jcip-annotations/1.0-1/jcip-annotations-1.0-1.jar:/Users/djaiswal/.m2/repository/net/minidev/json-smart/2.3/json-smart-2.3.jar:/Users/djaiswal/.m2/repository/net/minidev/accessors-smart/1.2/accessors-smart-1.2.jar:/Users/djaiswal/.m2/repository/com/jcraft/jsch/0.1.54/jsch-0.1.54.jar:/Users/djaiswal/.m2/repository/org/apache/curator/curator-client/2.12.0/curator-client-2.12.0.jar:/Users/djaiswal/.m2/repository/org/apache/curator/curator-recipes/2.12.0/curator-recipes-2.12.0.jar:/Users/djaiswal/.m2/repository/org/apache/htrace/htrace-core4/4.1.0-incubating/htrace-core4-4.1.0-incubating.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerb-simplekdc/1.0.1/kerb-simplekdc-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerb-client/1.0.1/kerb-client-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerby-config/1.0.1/kerby-config-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerb-core/1.0.1/kerb-core-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerby-pkix/1.0.1/kerby-pkix-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerby-asn1/1.0.1/kerby-asn1-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerby-util/1.0.1/kerby-util-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerb-common/1.0.1/kerb-common-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerb-crypto/1.0.1/kerb-crypto-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerb-util/1.0.1/kerb-util-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/token-provider/1.0.1/token-provider-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerb-admin/1.0.1/kerb-admin-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerb-server/1.0.1/kerb-server-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerb-identity/1.0.1/kerb-identity-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerby-xdr/1.0.1/kerby-xdr-1.0.1.jar:/Users/djaiswal/.m2/repository/org/codehaus/woodstox/stax2-api/3.1.4/stax2-api-3.1.4.jar:/Users/djaiswal/.m2/repository/com/fasterxml/woodstox/woodstox-core/5.0.3/woodstox-core-5.0.3.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-common/3.0.0-beta1/hadoop-common-3.0.0-beta1-tests.jar:/Users/djaiswal/.m2/repository/org/apache/httpcomponents/httpclient/4.5.2/httpclient-4.5.2.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-distcp/3.0.0-beta1/hadoop-distcp-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-hdfs/3.0.0-beta1/hadoop-hdfs-3.0.0-beta1-tests.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-util-ajax/9.3.19.v20170502/jetty-util-ajax-9.3.19.v20170502.jar:/Users/djaiswal/.m2/repository/commons-daemon/commons-daemon/1.0.13/commons-daemon-1.0.13.jar:/Users/djaiswal/.m2/repository/io/netty/netty/3.10.5.Final/netty-3.10.5.Final.jar:/Users/djaiswal/.m2/repository/io/netty/netty-all/4.0.52.Final/netty-all-4.0.52.Final.jar:/Users/djaiswal/.m2/repository/org/fusesource/leveldbjni/leveldbjni-all/1.8/leveldbjni-all-1.8.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-hdfs/3.0.0-beta1/hadoop-hdfs-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-jobclient/3.0.0-beta1/hadoop-mapreduce-client-jobclient-3.0.0-beta1-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-common/3.0.0-beta1/hadoop-mapreduce-client-common-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/com/google/inject/extensions/guice-servlet/4.0/guice-servlet-4.0.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-hs/3.0.0-beta1/hadoop-mapreduce-client-hs-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-hdfs-client/3.0.0-beta1/hadoop-hdfs-client-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/com/squareup/okhttp/okhttp/2.4.0/okhttp-2.4.0.jar:/Users/djaiswal/.m2/repository/com/squareup/okio/okio/1.4.0/okio-1.4.0.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-app/3.0.0-beta1/hadoop-mapreduce-client-app-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-shuffle/3.0.0-beta1/hadoop-mapreduce-client-shuffle-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-core/3.0.0-beta1/hadoop-mapreduce-client-core-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-llap-server/3.0.0-SNAPSHOT/hive-llap-server-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-llap-common/3.0.0-SNAPSHOT/hive-llap-common-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-llap-client/3.0.0-SNAPSHOT/hive-llap-client-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/slider/slider-core/0.92.0-incubating/slider-core-0.92.0-incubating.jar:/Users/djaiswal/.m2/repository/org/yaml/snakeyaml/1.16/snakeyaml-1.16.jar:/Users/djaiswal/.m2/repository/org/codehaus/jettison/jettison/1.1/jettison-1.1.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-llap-common/3.0.0-SNAPSHOT/hive-llap-common-3.0.0-SNAPSHOT-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-llap-server/3.0.0-SNAPSHOT/hive-llap-server-3.0.0-SNAPSHOT-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-server-tests/3.0.0-beta1/hadoop-yarn-server-tests-3.0.0-beta1-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-server-common/3.0.0-beta1/hadoop-yarn-server-common-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/geronimo/specs/geronimo-jcache_1.0_spec/1.0-alpha-1/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/Users/djaiswal/.m2/repository/org/ehcache/ehcache/3.3.1/ehcache-3.3.1.jar:/Users/djaiswal/.m2/repository/com/zaxxer/HikariCP-java7/2.4.12/HikariCP-java7-2.4.12.jar:/Users/djaiswal/.m2/repository/com/microsoft/sqlserver/mssql-jdbc/6.2.1.jre7/mssql-jdbc-6.2.1.jre7.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-server-nodemanager/3.0.0-beta1/hadoop-yarn-server-nodemanager-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/com/codahale/metrics/metrics-core/3.0.1/metrics-core-3.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-server-resourcemanager/3.0.0-beta1/hadoop-yarn-server-resourcemanager-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-server-applicationhistoryservice/3.0.0-beta1/hadoop-yarn-server-applicationhistoryservice-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/de/ruedigermoeller/fst/2.50/fst-2.50.jar:/Users/djaiswal/.m2/repository/com/cedarsoftware/java-util/1.9.0/java-util-1.9.0.jar:/Users/djaiswal/.m2/repository/com/cedarsoftware/json-io/2.5.1/json-io-2.5.1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-server-timelineservice/3.0.0-beta1/hadoop-yarn-server-timelineservice-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-csv/1.0/commons-csv-1.0.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-client/3.0.0-beta1/hadoop-yarn-client-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-common/3.0.0-beta1/hadoop-yarn-common-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/javax/xml/bind/jaxb-api/2.2.11/jaxb-api-2.2.11.jar:/Users/djaiswal/.m2/repository/com/sun/jersey/jersey-client/1.19/jersey-client-1.19.jar:/Users/djaiswal/.m2/repository/com/google/inject/guice/4.0/guice-4.0.jar:/Users/djaiswal/.m2/repository/aopalliance/aopalliance/1.0/aopalliance-1.0.jar:/Users/djaiswal/.m2/repository/com/sun/jersey/contribs/jersey-guice/1.19/jersey-guice-1.19.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/core/jackson-core/2.7.8/jackson-core-2.7.8.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/module/jackson-module-jaxb-annotations/2.7.8/jackson-module-jaxb-annotations-2.7.8.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/jaxrs/jackson-jaxrs-json-provider/2.7.8/jackson-jaxrs-json-provider-2.7.8.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/jaxrs/jackson-jaxrs-base/2.7.8/jackson-jaxrs-base-2.7.8.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-api/3.0.0-beta1/hadoop-yarn-api-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/core/jackson-annotations/2.7.8/jackson-annotations-2.7.8.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-common/2.0.0-alpha4/hbase-common-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/thirdparty/hbase-shaded-miscellaneous/1.0.1/hbase-shaded-miscellaneous-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-collections4/4.1/commons-collections4-4.1.jar:/Users/djaiswal/.m2/repository/org/apache/htrace/htrace-core/3.2.0-incubating/htrace-core-3.2.0-incubating.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-crypto/1.0.0/commons-crypto-1.0.0.jar:/Users/djaiswal/.m2/repository/com/github/stephenc/findbugs/findbugs-annotations/1.3.9-1/findbugs-annotations-1.3.9-1.jar:/Users/djaiswal/.m2/repository/org/apache/yetus/audience-annotations/0.5.0/audience-annotations-0.5.0.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-common/2.0.0-alpha4/hbase-common-2.0.0-alpha4-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-hadoop-compat/2.0.0-alpha4/hbase-hadoop-compat-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-metrics-api/2.0.0-alpha4/hbase-metrics-api-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-hadoop-compat/2.0.0-alpha4/hbase-hadoop-compat-2.0.0-alpha4-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-hadoop2-compat/2.0.0-alpha4/hbase-hadoop2-compat-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-metrics/2.0.0-alpha4/hbase-metrics-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-hadoop2-compat/2.0.0-alpha4/hbase-hadoop2-compat-2.0.0-alpha4-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-server/2.0.0-alpha4/hbase-server-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/thirdparty/hbase-shaded-protobuf/1.0.1/hbase-shaded-protobuf-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/thirdparty/hbase-shaded-netty/1.0.1/hbase-shaded-netty-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-http/2.0.0-alpha4/hbase-http-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/glassfish/jersey/core/jersey-server/2.25.1/jersey-server-2.25.1.jar:/Users/djaiswal/.m2/repository/org/glassfish/jersey/core/jersey-common/2.25.1/jersey-common-2.25.1.jar:/Users/djaiswal/.m2/repository/org/glassfish/jersey/bundles/repackaged/jersey-guava/2.25.1/jersey-guava-2.25.1.jar:/Users/djaiswal/.m2/repository/org/glassfish/hk2/osgi-resource-locator/1.0.1/osgi-resource-locator-1.0.1.jar:/Users/djaiswal/.m2/repository/org/glassfish/jersey/core/jersey-client/2.25.1/jersey-client-2.25.1.jar:/Users/djaiswal/.m2/repository/org/glassfish/jersey/media/jersey-media-jaxb/2.25.1/jersey-media-jaxb-2.25.1.jar:/Users/djaiswal/.m2/repository/javax/annotation/javax.annotation-api/1.2/javax.annotation-api-1.2.jar:/Users/djaiswal/.m2/repository/org/glassfish/hk2/hk2-api/2.5.0-b32/hk2-api-2.5.0-b32.jar:/Users/djaiswal/.m2/repository/org/glassfish/hk2/hk2-utils/2.5.0-b32/hk2-utils-2.5.0-b32.jar:/Users/djaiswal/.m2/repository/org/glassfish/hk2/external/aopalliance-repackaged/2.5.0-b32/aopalliance-repackaged-2.5.0-b32.jar:/Users/djaiswal/.m2/repository/org/glassfish/hk2/external/javax.inject/2.5.0-b32/javax.inject-2.5.0-b32.jar:/Users/djaiswal/.m2/repository/org/glassfish/hk2/hk2-locator/2.5.0-b32/hk2-locator-2.5.0-b32.jar:/Users/djaiswal/.m2/repository/org/javassist/javassist/3.20.0-GA/javassist-3.20.0-GA.jar:/Users/djaiswal/.m2/repository/org/glassfish/jersey/containers/jersey-container-servlet-core/2.25.1/jersey-container-servlet-core-2.25.1.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-protocol/2.0.0-alpha4/hbase-protocol-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-protocol-shaded/2.0.0-alpha4/hbase-protocol-shaded-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-procedure/2.0.0-alpha4/hbase-procedure-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-replication/2.0.0-alpha4/hbase-replication-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-prefix-tree/2.0.0-alpha4/hbase-prefix-tree-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/glassfish/web/javax.servlet.jsp/2.3.2/javax.servlet.jsp-2.3.2.jar:/Users/djaiswal/.m2/repository/org/jamon/jamon-runtime/2.3.1/jamon-runtime-2.3.1.jar:/Users/djaiswal/.m2/repository/javax/ws/rs/javax.ws.rs-api/2.0.1/javax.ws.rs-api-2.0.1.jar:/Users/djaiswal/.m2/repository/com/lmax/disruptor/3.3.6/disruptor-3.3.6.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-client/3.0.0-beta1/hadoop-client-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-jobclient/3.0.0-beta1/hadoop-mapreduce-client-jobclient-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-server/2.0.0-alpha4/hbase-server-2.0.0-alpha4-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-mapreduce/2.0.0-alpha4/hbase-mapreduce-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/tez/tez-tests/0.9.1-SNAPSHOT/tez-tests-0.9.1-SNAPSHOT-tests.jar:/Users/djaiswal/.m2/repository/org/apache/tez/tez-common/0.9.1-SNAPSHOT/tez-common-0.9.1-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/slf4j/slf4j-log4j12/1.7.10/slf4j-log4j12-1.7.10.jar:/Users/djaiswal/.m2/repository/org/apache/tez/tez-examples/0.9.1-SNAPSHOT/tez-examples-0.9.1-SNAPSHOT.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_101.jdk/Contents/Home/jre/../lib/tools.jar:/Users/djaiswal/.m2/repository/org/apache/tez/tez-api/0.9.1-SNAPSHOT/tez-api-0.9.1-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/tez/tez-runtime-library/0.9.1-SNAPSHOT/tez-runtime-library-0.9.1-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/roaringbitmap/RoaringBitmap/0.4.9/RoaringBitmap-0.4.9.jar:/Users/djaiswal/.m2/repository/com/ning/async-http-client/1.8.16/async-http-client-1.8.16.jar:/Users/djaiswal/.m2/repository/org/apache/tez/tez-mapreduce/0.9.1-SNAPSHOT/tez-mapreduce-0.9.1-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/tez/tez-dag/0.9.1-SNAPSHOT/tez-dag-0.9.1-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/tez/hadoop-shim/0.9.1-SNAPSHOT/hadoop-shim-0.9.1-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/tez/tez-runtime-internals/0.9.1-SNAPSHOT/tez-runtime-internals-0.9.1-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-server-web-proxy/3.0.0-beta1/hadoop-yarn-server-web-proxy-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/mortbay/jetty/jetty/6.1.26/jetty-6.1.26.jar:/Users/djaiswal/.m2/repository/org/mortbay/jetty/servlet-api/2.5-20081211/servlet-api-2.5-20081211.jar:/Users/djaiswal/.m2/repository/org/mortbay/jetty/jetty-util/6.1.26/jetty-util-6.1.26.jar:/Users/djaiswal/.m2/repository/javax/servlet/servlet-api/2.5/servlet-api-2.5.jar:/Users/djaiswal/hive/hive/itests/qtest-druid/target/classes:/Users/djaiswal/.m2/repository/io/druid/druid-services/0.10.1/druid-services-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/druid-indexing-hadoop/0.10.1/druid-indexing-hadoop-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/druid-indexing-service/0.10.1/druid-indexing-service-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/druid-sql/0.10.1/druid-sql-0.10.1.jar:/Users/djaiswal/.m2/repository/org/apache/calcite/avatica/avatica-core/1.9.0/avatica-core-1.9.0.jar:/Users/djaiswal/.m2/repository/org/apache/calcite/avatica/avatica-metrics/1.9.0/avatica-metrics-1.9.0.jar:/Users/djaiswal/.m2/repository/org/apache/calcite/avatica/avatica-server/1.9.0/avatica-server-1.9.0.jar:/Users/djaiswal/.m2/repository/io/airlift/airline/0.7/airline-0.7.jar:/Users/djaiswal/.m2/repository/org/apache/logging/log4j/log4j-api/2.8.2/log4j-api-2.8.2.jar:/Users/djaiswal/.m2/repository/org/apache/logging/log4j/log4j-core/2.8.2/log4j-core-2.8.2.jar:/Users/djaiswal/.m2/repository/org/apache/curator/curator-x-discovery/2.11.0/curator-x-discovery-2.11.0.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-servlets/9.3.19.v20170502/jetty-servlets-9.3.19.v20170502.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-continuation/9.3.19.v20170502/jetty-continuation-9.3.19.v20170502.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-proxy/9.3.19.v20170502/jetty-proxy-9.3.19.v20170502.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-security/9.3.19.v20170502/jetty-security-9.3.19.v20170502.jar:/Users/djaiswal/.m2/repository/org/slf4j/slf4j-api/1.7.10/slf4j-api-1.7.10.jar:/Users/djaiswal/.m2/repository/org/mockito/mockito-all/1.10.19/mockito-all-1.10.19.jar:/Users/djaiswal/hive/hive/itests/qtest/target/testconf:/Users/djaiswal/hive/hive/itests/qtest/../../conf: 2018-01-05T11:28:27,443 INFO [main] server.ZooKeeperServer: Server environment:java.library.path=/Users/djaiswal/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:. 2018-01-05T11:28:27,443 INFO [main] server.ZooKeeperServer: Server environment:java.io.tmpdir=/Users/djaiswal/hive/hive/itests/qtest/target/tmp 2018-01-05T11:28:27,443 INFO [main] server.ZooKeeperServer: Server environment:java.compiler= 2018-01-05T11:28:27,443 INFO [main] server.ZooKeeperServer: Server environment:os.name=Mac OS X 2018-01-05T11:28:27,443 INFO [main] server.ZooKeeperServer: Server environment:os.arch=x86_64 2018-01-05T11:28:27,443 INFO [main] server.ZooKeeperServer: Server environment:os.version=10.12.6 2018-01-05T11:28:27,443 INFO [main] server.ZooKeeperServer: Server environment:user.name=djaiswal 2018-01-05T11:28:27,443 INFO [main] server.ZooKeeperServer: Server environment:user.home=/Users/djaiswal 2018-01-05T11:28:27,443 INFO [main] server.ZooKeeperServer: Server environment:user.dir=/Users/djaiswal/hive/hive/itests/qtest 2018-01-05T11:28:27,458 INFO [main] server.ZooKeeperServer: Created server with tickTime 2000 minSessionTimeout 4000 maxSessionTimeout 40000 datadir /Users/djaiswal/hive/hive/itests/qtest/target/tmp/tmp_55056/zookeeper_0/version-2 snapdir /Users/djaiswal/hive/hive/itests/qtest/target/tmp/tmp_55056/zookeeper_0/version-2 2018-01-05T11:28:27,458 INFO [main] server.ZooKeeperServer: minSessionTimeout set to -1 2018-01-05T11:28:27,458 INFO [main] server.ZooKeeperServer: maxSessionTimeout set to -1 2018-01-05T11:28:27,480 INFO [main] server.NIOServerCnxnFactory: binding to port 0.0.0.0/0.0.0.0:49384 2018-01-05T11:28:27,508 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:49384] server.NIOServerCnxnFactory: Accepted socket connection from /127.0.0.1:49814 2018-01-05T11:28:27,519 INFO [main] zookeeper.MiniZooKeeperCluster: Started MiniZooKeeperCluster and ran successful 'stat' on client port=49384 2018-01-05T11:28:27,524 INFO [main] zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT 2018-01-05T11:28:27,524 INFO [main] zookeeper.ZooKeeper: Client environment:host.name=10.22.8.182 2018-01-05T11:28:27,524 INFO [main] zookeeper.ZooKeeper: Client environment:java.version=1.8.0_101 2018-01-05T11:28:27,524 INFO [main] zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation 2018-01-05T11:28:27,524 INFO [main] zookeeper.ZooKeeper: Client environment:java.home=/Library/Java/JavaVirtualMachines/jdk1.8.0_101.jdk/Contents/Home/jre 2018-01-05T11:28:27,524 INFO [main] zookeeper.ZooKeeper: Client environment:java.class.path=/Users/djaiswal/hive/hive/itests/qtest/target/test-classes:/Users/djaiswal/hive/hive/itests/qtest/target/classes:/Users/djaiswal/.m2/repository/org/apache/hive/hive-common/3.0.0-SNAPSHOT/hive-common-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-classification/3.0.0-SNAPSHOT/hive-classification-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-shims/3.0.0-SNAPSHOT/hive-shims-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/shims/hive-shims-common/3.0.0-SNAPSHOT/hive-shims-common-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/shims/hive-shims-0.23/3.0.0-SNAPSHOT/hive-shims-0.23-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/shims/hive-shims-scheduler/3.0.0-SNAPSHOT/hive-shims-scheduler-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-storage-api/3.0.0-SNAPSHOT/hive-storage-api-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/commons-cli/commons-cli/1.2/commons-cli-1.2.jar:/Users/djaiswal/.m2/repository/commons-lang/commons-lang/2.6/commons-lang-2.6.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-lang3/3.2/commons-lang3-3.2.jar:/Users/djaiswal/.m2/repository/org/apache/orc/orc-core/1.4.1/orc-core-1.4.1.jar:/Users/djaiswal/.m2/repository/io/airlift/aircompressor/0.8/aircompressor-0.8.jar:/Users/djaiswal/.m2/repository/io/airlift/slice/0.29/slice-0.29.jar:/Users/djaiswal/.m2/repository/org/openjdk/jol/jol-core/0.2/jol-core-0.2.jar:/Users/djaiswal/.m2/repository/jline/jline/2.12/jline-2.12.jar:/Users/djaiswal/.m2/repository/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-rewrite/9.3.8.v20160314/jetty-rewrite-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-client/9.3.8.v20160314/jetty-client-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-server/9.3.8.v20160314/jetty-server-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-http/9.3.8.v20160314/jetty-http-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-io/9.3.8.v20160314/jetty-io-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-servlet/9.3.8.v20160314/jetty-servlet-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-webapp/9.3.8.v20160314/jetty-webapp-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-xml/9.3.8.v20160314/jetty-xml-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/joda-time/joda-time/2.8.1/joda-time-2.8.1.jar:/Users/djaiswal/.m2/repository/org/apache/logging/log4j/log4j-1.2-api/2.8.2/log4j-1.2-api-2.8.2.jar:/Users/djaiswal/.m2/repository/org/apache/logging/log4j/log4j-web/2.8.2/log4j-web-2.8.2.jar:/Users/djaiswal/.m2/repository/org/apache/logging/log4j/log4j-slf4j-impl/2.8.2/log4j-slf4j-impl-2.8.2.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-compress/1.9/commons-compress-1.9.jar:/Users/djaiswal/.m2/repository/org/apache/ant/ant/1.9.1/ant-1.9.1.jar:/Users/djaiswal/.m2/repository/org/apache/ant/ant-launcher/1.9.1/ant-launcher-1.9.1.jar:/Users/djaiswal/.m2/repository/com/tdunning/json/1.8/json-1.8.jar:/Users/djaiswal/.m2/repository/io/dropwizard/metrics/metrics-core/3.1.0/metrics-core-3.1.0.jar:/Users/djaiswal/.m2/repository/io/dropwizard/metrics/metrics-jvm/3.1.0/metrics-jvm-3.1.0.jar:/Users/djaiswal/.m2/repository/io/dropwizard/metrics/metrics-json/3.1.0/metrics-json-3.1.0.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/core/jackson-databind/2.6.5/jackson-databind-2.6.5.jar:/Users/djaiswal/.m2/repository/com/github/joshelser/dropwizard-metrics-hadoop-metrics2-reporter/0.1.2/dropwizard-metrics-hadoop-metrics2-reporter-0.1.2.jar:/Users/djaiswal/.m2/repository/javolution/javolution/5.5.1/javolution-5.5.1.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-contrib/3.0.0-SNAPSHOT/hive-contrib-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/commons-codec/commons-codec/1.4/commons-codec-1.4.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-standalone-metastore/3.0.0-SNAPSHOT/hive-standalone-metastore-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/com/google/guava/guava/19.0/guava-19.0.jar:/Users/djaiswal/.m2/repository/com/google/protobuf/protobuf-java/2.5.0/protobuf-java-2.5.0.jar:/Users/djaiswal/.m2/repository/com/jolbox/bonecp/0.8.0.RELEASE/bonecp-0.8.0.RELEASE.jar:/Users/djaiswal/.m2/repository/com/zaxxer/HikariCP/2.6.1/HikariCP-2.6.1.jar:/Users/djaiswal/.m2/repository/commons-dbcp/commons-dbcp/1.4/commons-dbcp-1.4.jar:/Users/djaiswal/.m2/repository/commons-pool/commons-pool/1.5.4/commons-pool-1.5.4.jar:/Users/djaiswal/.m2/repository/org/antlr/antlr-runtime/3.5.2/antlr-runtime-3.5.2.jar:/Users/djaiswal/.m2/repository/org/apache/derby/derby/10.11.1.1/derby-10.11.1.1.jar:/Users/djaiswal/.m2/repository/org/apache/thrift/libfb303/0.9.3/libfb303-0.9.3.jar:/Users/djaiswal/.m2/repository/org/apache/thrift/libthrift/0.9.3/libthrift-0.9.3.jar:/Users/djaiswal/.m2/repository/org/apache/httpcomponents/httpcore/4.4.4/httpcore-4.4.4.jar:/Users/djaiswal/.m2/repository/org/datanucleus/datanucleus-api-jdo/4.2.4/datanucleus-api-jdo-4.2.4.jar:/Users/djaiswal/.m2/repository/org/datanucleus/datanucleus-core/4.1.17/datanucleus-core-4.1.17.jar:/Users/djaiswal/.m2/repository/org/datanucleus/datanucleus-rdbms/4.1.19/datanucleus-rdbms-4.1.19.jar:/Users/djaiswal/.m2/repository/org/datanucleus/javax.jdo/3.2.0-m3/javax.jdo-3.2.0-m3.jar:/Users/djaiswal/.m2/repository/javax/transaction/transaction-api/1.1/transaction-api-1.1.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-standalone-metastore/3.0.0-SNAPSHOT/hive-standalone-metastore-3.0.0-SNAPSHOT-tests.jar:/Users/djaiswal/hive/hive/itests/custom-serde/target/classes:/Users/djaiswal/hive/hive/itests/hive-unit/target/test-classes:/Users/djaiswal/.m2/repository/org/apache/hive/hive-jdbc/3.0.0-SNAPSHOT/hive-jdbc-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-service/3.0.0-SNAPSHOT/hive-service-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/javax/servlet/jsp/javax.servlet.jsp-api/2.3.1/javax.servlet.jsp-api-2.3.1.jar:/Users/djaiswal/.m2/repository/net/sf/jpam/jpam/1.1/jpam-1.1.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-runner/9.3.8.v20160314/jetty-runner-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-plus/9.3.8.v20160314/jetty-plus-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-annotations/9.3.8.v20160314/jetty-annotations-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-jaas/9.3.8.v20160314/jetty-jaas-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/websocket/websocket-server/9.3.8.v20160314/websocket-server-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/websocket/websocket-common/9.3.8.v20160314/websocket-common-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/websocket/websocket-api/9.3.8.v20160314/websocket-api-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/websocket/websocket-client/9.3.8.v20160314/websocket-client-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/websocket/websocket-servlet/9.3.8.v20160314/websocket-servlet-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-jndi/9.3.8.v20160314/jetty-jndi-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/apache-jsp/9.3.8.v20160314/apache-jsp-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/toolchain/jetty-schemas/3.1/jetty-schemas-3.1.jar:/Users/djaiswal/.m2/repository/org/eclipse/jdt/core/compiler/ecj/4.4.2/ecj-4.4.2.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/apache-jstl/9.3.8.v20160314/apache-jstl-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/apache/taglibs/taglibs-standard-spec/1.2.5/taglibs-standard-spec-1.2.5.jar:/Users/djaiswal/.m2/repository/org/apache/taglibs/taglibs-standard-impl/1.2.5/taglibs-standard-impl-1.2.5.jar:/Users/djaiswal/.m2/repository/tomcat/jasper-compiler/5.5.23/jasper-compiler-5.5.23.jar:/Users/djaiswal/.m2/repository/javax/servlet/jsp-api/2.0/jsp-api-2.0.jar:/Users/djaiswal/.m2/repository/ant/ant/1.6.5/ant-1.6.5.jar:/Users/djaiswal/.m2/repository/tomcat/jasper-runtime/5.5.23/jasper-runtime-5.5.23.jar:/Users/djaiswal/.m2/repository/commons-el/commons-el/1.0/commons-el-1.0.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-llap-ext-client/3.0.0-SNAPSHOT/hive-llap-ext-client-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hcatalog/hive-hcatalog-core/3.0.0-SNAPSHOT/hive-hcatalog-core-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.9.13/jackson-mapper-asl-1.9.13.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hcatalog/hive-hcatalog-streaming/3.0.0-SNAPSHOT/hive-hcatalog-streaming-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hcatalog/hive-hcatalog-server-extensions/3.0.0-SNAPSHOT/hive-hcatalog-server-extensions-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/javax/jms/jms/1.1/jms-1.1.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hcatalog/hive-webhcat-java-client/3.0.0-SNAPSHOT/hive-webhcat-java-client-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-service/3.0.0-SNAPSHOT/hive-service-3.0.0-SNAPSHOT-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-exec/3.0.0-SNAPSHOT/hive-exec-3.0.0-SNAPSHOT-tests.jar:/Users/djaiswal/hive/hive/itests/util/target/classes:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-minicluster/1.7.3/accumulo-minicluster-1.7.3.jar:/Users/djaiswal/.m2/repository/com/beust/jcommander/1.32/jcommander-1.32.jar:/Users/djaiswal/.m2/repository/commons-configuration/commons-configuration/1.6/commons-configuration-1.6.jar:/Users/djaiswal/.m2/repository/commons-digester/commons-digester/1.8/commons-digester-1.8.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-core/1.7.3/accumulo-core-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-math/2.1/commons-math-2.1.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-gc/1.7.3/accumulo-gc-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-master/1.7.3/accumulo-master-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-monitor/1.7.3/accumulo-monitor-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-shell/1.7.3/accumulo-shell-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-server-base/1.7.3/accumulo-server-base-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-start/1.7.3/accumulo-start-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-tracer/1.7.3/accumulo-tracer-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-tserver/1.7.3/accumulo-tserver-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-vfs2/2.1/commons-vfs2-2.1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-minicluster/3.0.0-beta1/hadoop-minicluster-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-accumulo-handler/3.0.0-SNAPSHOT/hive-accumulo-handler-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-fate/1.7.3/accumulo-fate-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-trace/1.7.3/accumulo-trace-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-accumulo-handler/3.0.0-SNAPSHOT/hive-accumulo-handler-3.0.0-SNAPSHOT-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-cli/3.0.0-SNAPSHOT/hive-cli-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-beeline/3.0.0-SNAPSHOT/hive-beeline-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/net/sf/supercsv/super-csv/2.2.0/super-csv-2.2.0.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-hbase-handler/3.0.0-SNAPSHOT/hive-hbase-handler-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-hbase-handler/3.0.0-SNAPSHOT/hive-hbase-handler-3.0.0-SNAPSHOT-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-metastore/3.0.0-SNAPSHOT/hive-metastore-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/javax/jdo/jdo-api/3.0.1/jdo-api-3.0.1.jar:/Users/djaiswal/.m2/repository/javax/transaction/jta/1.1/jta-1.1.jar:/Users/djaiswal/.m2/repository/co/cask/tephra/tephra-api/0.6.0/tephra-api-0.6.0.jar:/Users/djaiswal/.m2/repository/co/cask/tephra/tephra-core/0.6.0/tephra-core-0.6.0.jar:/Users/djaiswal/.m2/repository/com/google/inject/extensions/guice-assistedinject/3.0/guice-assistedinject-3.0.jar:/Users/djaiswal/.m2/repository/org/apache/twill/twill-common/0.6.0-incubating/twill-common-0.6.0-incubating.jar:/Users/djaiswal/.m2/repository/org/apache/twill/twill-core/0.6.0-incubating/twill-core-0.6.0-incubating.jar:/Users/djaiswal/.m2/repository/org/apache/twill/twill-api/0.6.0-incubating/twill-api-0.6.0-incubating.jar:/Users/djaiswal/.m2/repository/org/apache/twill/twill-discovery-api/0.6.0-incubating/twill-discovery-api-0.6.0-incubating.jar:/Users/djaiswal/.m2/repository/org/apache/twill/twill-discovery-core/0.6.0-incubating/twill-discovery-core-0.6.0-incubating.jar:/Users/djaiswal/.m2/repository/org/apache/twill/twill-zookeeper/0.6.0-incubating/twill-zookeeper-0.6.0-incubating.jar:/Users/djaiswal/.m2/repository/co/cask/tephra/tephra-hbase-compat-1.0/0.6.0/tephra-hbase-compat-1.0-0.6.0.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-client/2.0.0-alpha4/hbase-client-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/jruby/jcodings/jcodings/1.0.18/jcodings-1.0.18.jar:/Users/djaiswal/.m2/repository/org/jruby/joni/joni/2.1.11/joni-2.1.11.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-mapreduce/2.0.0-alpha4/hbase-mapreduce-2.0.0-alpha4-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-serde/3.0.0-SNAPSHOT/hive-serde-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-service-rpc/3.0.0-SNAPSHOT/hive-service-rpc-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/com/google/code/findbugs/jsr305/3.0.0/jsr305-3.0.0.jar:/Users/djaiswal/.m2/repository/org/apache/avro/avro/1.7.7/avro-1.7.7.jar:/Users/djaiswal/.m2/repository/org/codehaus/jackson/jackson-core-asl/1.9.13/jackson-core-asl-1.9.13.jar:/Users/djaiswal/.m2/repository/com/thoughtworks/paranamer/paranamer/2.3/paranamer-2.3.jar:/Users/djaiswal/.m2/repository/org/xerial/snappy/snappy-java/1.1.4/snappy-java-1.1.4.jar:/Users/djaiswal/.m2/repository/net/sf/opencsv/opencsv/2.3/opencsv-2.3.jar:/Users/djaiswal/.m2/repository/org/apache/parquet/parquet-hadoop-bundle/1.9.0/parquet-hadoop-bundle-1.9.0.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-exec/3.0.0-SNAPSHOT/hive-exec-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-vector-code-gen/3.0.0-SNAPSHOT/hive-vector-code-gen-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/velocity/velocity/1.5/velocity-1.5.jar:/Users/djaiswal/.m2/repository/oro/oro/2.0.8/oro-2.0.8.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-llap-tez/3.0.0-SNAPSHOT/hive-llap-tez-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/commons-httpclient/commons-httpclient/3.0.1/commons-httpclient-3.0.1.jar:/Users/djaiswal/.m2/repository/commons-io/commons-io/2.4/commons-io-2.4.jar:/Users/djaiswal/.m2/repository/org/antlr/ST4/4.0.4/ST4-4.0.4.jar:/Users/djaiswal/.m2/repository/org/apache/ivy/ivy/2.4.0/ivy-2.4.0.jar:/Users/djaiswal/.m2/repository/org/apache/zookeeper/zookeeper/3.4.6/zookeeper-3.4.6.jar:/Users/djaiswal/.m2/repository/org/apache/curator/curator-framework/2.12.0/curator-framework-2.12.0.jar:/Users/djaiswal/.m2/repository/org/codehaus/groovy/groovy-all/2.4.11/groovy-all-2.4.11.jar:/Users/djaiswal/.m2/repository/org/apache/calcite/calcite-core/1.15.0/calcite-core-1.15.0.jar:/Users/djaiswal/.m2/repository/org/apache/calcite/calcite-linq4j/1.15.0/calcite-linq4j-1.15.0.jar:/Users/djaiswal/.m2/repository/com/esri/geometry/esri-geometry-api/2.0.0/esri-geometry-api-2.0.0.jar:/Users/djaiswal/.m2/repository/com/yahoo/datasketches/sketches-core/0.9.0/sketches-core-0.9.0.jar:/Users/djaiswal/.m2/repository/com/yahoo/datasketches/memory/0.9.0/memory-0.9.0.jar:/Users/djaiswal/.m2/repository/org/codehaus/janino/janino/2.7.6/janino-2.7.6.jar:/Users/djaiswal/.m2/repository/org/codehaus/janino/commons-compiler/2.7.6/commons-compiler-2.7.6.jar:/Users/djaiswal/.m2/repository/org/apache/calcite/calcite-druid/1.15.0/calcite-druid-1.15.0.jar:/Users/djaiswal/.m2/repository/org/apache/calcite/avatica/avatica/1.10.0/avatica-1.10.0.jar:/Users/djaiswal/.m2/repository/com/google/code/gson/gson/2.2.4/gson-2.2.4.jar:/Users/djaiswal/.m2/repository/stax/stax-api/1.0.1/stax-api-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-registry/3.0.0-beta1/hadoop-yarn-registry-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-druid-handler/3.0.0-SNAPSHOT/hive-druid-handler-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/com/metamx/java-util/0.27.10/java-util-0.27.10.jar:/Users/djaiswal/.m2/repository/org/skife/config/config-magic/0.9/config-magic-0.9.jar:/Users/djaiswal/.m2/repository/org/mozilla/rhino/1.7R5/rhino-1.7R5.jar:/Users/djaiswal/.m2/repository/com/jayway/jsonpath/json-path/2.1.0/json-path-2.1.0.jar:/Users/djaiswal/.m2/repository/io/druid/druid-server/0.10.1/druid-server-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/druid-console/0.0.2/druid-console-0.0.2.jar:/Users/djaiswal/.m2/repository/com/metamx/http-client/1.1.0/http-client-1.1.0.jar:/Users/djaiswal/.m2/repository/com/metamx/server-metrics/0.2.8/server-metrics-0.2.8.jar:/Users/djaiswal/.m2/repository/javax/inject/javax.inject/1/javax.inject-1.jar:/Users/djaiswal/.m2/repository/org/glassfish/javax.el/3.0.0/javax.el-3.0.0.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/jaxrs/jackson-jaxrs-smile-provider/2.4.6/jackson-jaxrs-smile-provider-2.4.6.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/dataformat/jackson-dataformat-smile/2.4.6/jackson-dataformat-smile-2.4.6.jar:/Users/djaiswal/.m2/repository/io/tesla/aether/tesla-aether/0.0.5/tesla-aether-0.0.5.jar:/Users/djaiswal/.m2/repository/org/eclipse/aether/aether-spi/0.9.0.M2/aether-spi-0.9.0.M2.jar:/Users/djaiswal/.m2/repository/org/eclipse/aether/aether-util/0.9.0.M2/aether-util-0.9.0.M2.jar:/Users/djaiswal/.m2/repository/org/eclipse/aether/aether-impl/0.9.0.M2/aether-impl-0.9.0.M2.jar:/Users/djaiswal/.m2/repository/org/eclipse/aether/aether-connector-file/0.9.0.M2/aether-connector-file-0.9.0.M2.jar:/Users/djaiswal/.m2/repository/io/tesla/aether/aether-connector-okhttp/0.0.9/aether-connector-okhttp-0.0.9.jar:/Users/djaiswal/.m2/repository/org/apache/maven/wagon/wagon-provider-api/2.4/wagon-provider-api-2.4.jar:/Users/djaiswal/.m2/repository/org/apache/maven/maven-aether-provider/3.1.1/maven-aether-provider-3.1.1.jar:/Users/djaiswal/.m2/repository/org/apache/maven/maven-model/3.1.1/maven-model-3.1.1.jar:/Users/djaiswal/.m2/repository/org/apache/maven/maven-model-builder/3.1.1/maven-model-builder-3.1.1.jar:/Users/djaiswal/.m2/repository/org/apache/maven/maven-repository-metadata/3.1.1/maven-repository-metadata-3.1.1.jar:/Users/djaiswal/.m2/repository/org/codehaus/plexus/plexus-utils/3.0.15/plexus-utils-3.0.15.jar:/Users/djaiswal/.m2/repository/org/apache/maven/maven-settings-builder/3.1.1/maven-settings-builder-3.1.1.jar:/Users/djaiswal/.m2/repository/org/codehaus/plexus/plexus-interpolation/1.19/plexus-interpolation-1.19.jar:/Users/djaiswal/.m2/repository/org/apache/maven/maven-settings/3.1.1/maven-settings-3.1.1.jar:/Users/djaiswal/.m2/repository/org/eclipse/aether/aether-api/0.9.0.M2/aether-api-0.9.0.M2.jar:/Users/djaiswal/.m2/repository/net/spy/spymemcached/2.12.3/spymemcached-2.12.3.jar:/Users/djaiswal/.m2/repository/net/jpountz/lz4/lz4/1.3.0/lz4-1.3.0.jar:/Users/djaiswal/.m2/repository/com/ircclouds/irc/irc-api/1.0-0014/irc-api-1.0-0014.jar:/Users/djaiswal/.m2/repository/com/maxmind/geoip2/geoip2/0.4.0/geoip2-0.4.0.jar:/Users/djaiswal/.m2/repository/com/maxmind/maxminddb/maxminddb/0.2.0/maxminddb-0.2.0.jar:/Users/djaiswal/.m2/repository/org/apache/derby/derbynet/10.11.1.1/derbynet-10.11.1.1.jar:/Users/djaiswal/.m2/repository/org/apache/derby/derbyclient/10.11.1.1/derbyclient-10.11.1.1.jar:/Users/djaiswal/.m2/repository/it/unimi/dsi/fastutil/7.2.0/fastutil-7.2.0.jar:/Users/djaiswal/.m2/repository/io/druid/druid-processing/0.10.1/druid-processing-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/druid-common/0.10.1/druid-common-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/java-util/0.10.1/java-util-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/druid-api/0.10.1/druid-api-0.10.1.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-dbcp2/2.0.1/commons-dbcp2-2.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-pool2/2.2/commons-pool2-2.2.jar:/Users/djaiswal/.m2/repository/org/hibernate/hibernate-validator/5.1.3.Final/hibernate-validator-5.1.3.Final.jar:/Users/djaiswal/.m2/repository/org/jboss/logging/jboss-logging/3.1.3.GA/jboss-logging-3.1.3.GA.jar:/Users/djaiswal/.m2/repository/com/fasterxml/classmate/1.0.0/classmate-1.0.0.jar:/Users/djaiswal/.m2/repository/javax/el/javax.el-api/3.0.0/javax.el-api-3.0.0.jar:/Users/djaiswal/.m2/repository/javax/validation/validation-api/1.1.0.Final/validation-api-1.1.0.Final.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/datatype/jackson-datatype-guava/2.4.6/jackson-datatype-guava-2.4.6.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/datatype/jackson-datatype-joda/2.4.6/jackson-datatype-joda-2.4.6.jar:/Users/djaiswal/.m2/repository/com/google/inject/extensions/guice-multibindings/4.1.0/guice-multibindings-4.1.0.jar:/Users/djaiswal/.m2/repository/org/jdbi/jdbi/2.63.1/jdbi-2.63.1.jar:/Users/djaiswal/.m2/repository/org/apache/logging/log4j/log4j-jul/2.5/log4j-jul-2.5.jar:/Users/djaiswal/.m2/repository/org/slf4j/jcl-over-slf4j/1.7.12/jcl-over-slf4j-1.7.12.jar:/Users/djaiswal/.m2/repository/net/java/dev/jets3t/jets3t/0.9.4/jets3t-0.9.4.jar:/Users/djaiswal/.m2/repository/javax/activation/activation/1.1.1/activation-1.1.1.jar:/Users/djaiswal/.m2/repository/org/bouncycastle/bcprov-jdk15on/1.52/bcprov-jdk15on-1.52.jar:/Users/djaiswal/.m2/repository/com/jamesmurty/utils/java-xmlbuilder/1.1/java-xmlbuilder-1.1.jar:/Users/djaiswal/.m2/repository/net/iharder/base64/2.3.8/base64-2.3.8.jar:/Users/djaiswal/.m2/repository/org/antlr/antlr4-runtime/4.5.1/antlr4-runtime-4.5.1.jar:/Users/djaiswal/.m2/repository/io/druid/druid-hll/0.10.1/druid-hll-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/bytebuffer-collections/0.10.1/bytebuffer-collections-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/extendedset/0.10.1/extendedset-0.10.1.jar:/Users/djaiswal/.m2/repository/com/metamx/emitter/0.4.5/emitter-0.4.5.jar:/Users/djaiswal/.m2/repository/com/ning/compress-lzf/1.0.3/compress-lzf-1.0.3.jar:/Users/djaiswal/.m2/repository/com/ibm/icu/icu4j/54.1.1/icu4j-54.1.1.jar:/Users/djaiswal/.m2/repository/org/mapdb/mapdb/1.0.8/mapdb-1.0.8.jar:/Users/djaiswal/.m2/repository/org/ow2/asm/asm/5.2/asm-5.2.jar:/Users/djaiswal/.m2/repository/org/ow2/asm/asm-commons/5.2/asm-commons-5.2.jar:/Users/djaiswal/.m2/repository/org/ow2/asm/asm-tree/5.2/asm-tree-5.2.jar:/Users/djaiswal/.m2/repository/io/druid/extensions/druid-hdfs-storage/0.10.1/druid-hdfs-storage-0.10.1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-aws/2.7.3/hadoop-aws-2.7.3.jar:/Users/djaiswal/.m2/repository/com/amazonaws/aws-java-sdk-s3/1.10.77/aws-java-sdk-s3-1.10.77.jar:/Users/djaiswal/.m2/repository/com/amazonaws/aws-java-sdk-kms/1.10.77/aws-java-sdk-kms-1.10.77.jar:/Users/djaiswal/.m2/repository/com/amazonaws/aws-java-sdk-core/1.10.77/aws-java-sdk-core-1.10.77.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/dataformat/jackson-dataformat-cbor/2.5.3/jackson-dataformat-cbor-2.5.3.jar:/Users/djaiswal/.m2/repository/io/druid/extensions/mysql-metadata-storage/0.10.1/mysql-metadata-storage-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/extensions/postgresql-metadata-storage/0.10.1/postgresql-metadata-storage-0.10.1.jar:/Users/djaiswal/.m2/repository/org/postgresql/postgresql/9.4.1208.jre7/postgresql-9.4.1208.jre7.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-druid-handler/3.0.0-SNAPSHOT/hive-druid-handler-3.0.0-SNAPSHOT-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-jdbc-handler/3.0.0-SNAPSHOT/hive-jdbc-handler-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/junit/junit/4.11/junit-4.11.jar:/Users/djaiswal/.m2/repository/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar:/Users/djaiswal/.m2/repository/com/sun/jersey/jersey-servlet/1.19/jersey-servlet-1.19.jar:/Users/djaiswal/.m2/repository/com/sun/jersey/jersey-server/1.19/jersey-server-1.19.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-archives/3.0.0-beta1/hadoop-archives-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-common/3.0.0-beta1/hadoop-common-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-annotations/3.0.0-beta1/hadoop-annotations-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-math3/3.1.1/commons-math3-3.1.1.jar:/Users/djaiswal/.m2/repository/commons-net/commons-net/3.1/commons-net-3.1.jar:/Users/djaiswal/.m2/repository/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-util/9.3.19.v20170502/jetty-util-9.3.19.v20170502.jar:/Users/djaiswal/.m2/repository/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.jar:/Users/djaiswal/.m2/repository/com/sun/jersey/jersey-core/1.19/jersey-core-1.19.jar:/Users/djaiswal/.m2/repository/javax/ws/rs/jsr311-api/1.1.1/jsr311-api-1.1.1.jar:/Users/djaiswal/.m2/repository/com/sun/jersey/jersey-json/1.19/jersey-json-1.19.jar:/Users/djaiswal/.m2/repository/com/sun/xml/bind/jaxb-impl/2.2.3-1/jaxb-impl-2.2.3-1.jar:/Users/djaiswal/.m2/repository/org/codehaus/jackson/jackson-jaxrs/1.9.13/jackson-jaxrs-1.9.13.jar:/Users/djaiswal/.m2/repository/org/codehaus/jackson/jackson-xc/1.9.13/jackson-xc-1.9.13.jar:/Users/djaiswal/.m2/repository/commons-logging/commons-logging/1.1.3/commons-logging-1.1.3.jar:/Users/djaiswal/.m2/repository/log4j/log4j/1.2.17/log4j-1.2.17.jar:/Users/djaiswal/.m2/repository/commons-beanutils/commons-beanutils/1.9.3/commons-beanutils-1.9.3.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-configuration2/2.1.1/commons-configuration2-2.1.1.jar:/Users/djaiswal/.m2/repository/com/google/re2j/re2j/1.1/re2j-1.1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-auth/3.0.0-beta1/hadoop-auth-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/com/nimbusds/nimbus-jose-jwt/4.41.1/nimbus-jose-jwt-4.41.1.jar:/Users/djaiswal/.m2/repository/com/github/stephenc/jcip/jcip-annotations/1.0-1/jcip-annotations-1.0-1.jar:/Users/djaiswal/.m2/repository/net/minidev/json-smart/2.3/json-smart-2.3.jar:/Users/djaiswal/.m2/repository/net/minidev/accessors-smart/1.2/accessors-smart-1.2.jar:/Users/djaiswal/.m2/repository/com/jcraft/jsch/0.1.54/jsch-0.1.54.jar:/Users/djaiswal/.m2/repository/org/apache/curator/curator-client/2.12.0/curator-client-2.12.0.jar:/Users/djaiswal/.m2/repository/org/apache/curator/curator-recipes/2.12.0/curator-recipes-2.12.0.jar:/Users/djaiswal/.m2/repository/org/apache/htrace/htrace-core4/4.1.0-incubating/htrace-core4-4.1.0-incubating.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerb-simplekdc/1.0.1/kerb-simplekdc-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerb-client/1.0.1/kerb-client-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerby-config/1.0.1/kerby-config-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerb-core/1.0.1/kerb-core-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerby-pkix/1.0.1/kerby-pkix-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerby-asn1/1.0.1/kerby-asn1-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerby-util/1.0.1/kerby-util-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerb-common/1.0.1/kerb-common-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerb-crypto/1.0.1/kerb-crypto-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerb-util/1.0.1/kerb-util-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/token-provider/1.0.1/token-provider-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerb-admin/1.0.1/kerb-admin-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerb-server/1.0.1/kerb-server-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerb-identity/1.0.1/kerb-identity-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerby-xdr/1.0.1/kerby-xdr-1.0.1.jar:/Users/djaiswal/.m2/repository/org/codehaus/woodstox/stax2-api/3.1.4/stax2-api-3.1.4.jar:/Users/djaiswal/.m2/repository/com/fasterxml/woodstox/woodstox-core/5.0.3/woodstox-core-5.0.3.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-common/3.0.0-beta1/hadoop-common-3.0.0-beta1-tests.jar:/Users/djaiswal/.m2/repository/org/apache/httpcomponents/httpclient/4.5.2/httpclient-4.5.2.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-distcp/3.0.0-beta1/hadoop-distcp-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-hdfs/3.0.0-beta1/hadoop-hdfs-3.0.0-beta1-tests.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-util-ajax/9.3.19.v20170502/jetty-util-ajax-9.3.19.v20170502.jar:/Users/djaiswal/.m2/repository/commons-daemon/commons-daemon/1.0.13/commons-daemon-1.0.13.jar:/Users/djaiswal/.m2/repository/io/netty/netty/3.10.5.Final/netty-3.10.5.Final.jar:/Users/djaiswal/.m2/repository/io/netty/netty-all/4.0.52.Final/netty-all-4.0.52.Final.jar:/Users/djaiswal/.m2/repository/org/fusesource/leveldbjni/leveldbjni-all/1.8/leveldbjni-all-1.8.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-hdfs/3.0.0-beta1/hadoop-hdfs-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-jobclient/3.0.0-beta1/hadoop-mapreduce-client-jobclient-3.0.0-beta1-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-common/3.0.0-beta1/hadoop-mapreduce-client-common-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/com/google/inject/extensions/guice-servlet/4.0/guice-servlet-4.0.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-hs/3.0.0-beta1/hadoop-mapreduce-client-hs-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-hdfs-client/3.0.0-beta1/hadoop-hdfs-client-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/com/squareup/okhttp/okhttp/2.4.0/okhttp-2.4.0.jar:/Users/djaiswal/.m2/repository/com/squareup/okio/okio/1.4.0/okio-1.4.0.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-app/3.0.0-beta1/hadoop-mapreduce-client-app-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-shuffle/3.0.0-beta1/hadoop-mapreduce-client-shuffle-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-core/3.0.0-beta1/hadoop-mapreduce-client-core-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-llap-server/3.0.0-SNAPSHOT/hive-llap-server-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-llap-common/3.0.0-SNAPSHOT/hive-llap-common-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-llap-client/3.0.0-SNAPSHOT/hive-llap-client-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/slider/slider-core/0.92.0-incubating/slider-core-0.92.0-incubating.jar:/Users/djaiswal/.m2/repository/org/yaml/snakeyaml/1.16/snakeyaml-1.16.jar:/Users/djaiswal/.m2/repository/org/codehaus/jettison/jettison/1.1/jettison-1.1.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-llap-common/3.0.0-SNAPSHOT/hive-llap-common-3.0.0-SNAPSHOT-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-llap-server/3.0.0-SNAPSHOT/hive-llap-server-3.0.0-SNAPSHOT-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-server-tests/3.0.0-beta1/hadoop-yarn-server-tests-3.0.0-beta1-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-server-common/3.0.0-beta1/hadoop-yarn-server-common-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/geronimo/specs/geronimo-jcache_1.0_spec/1.0-alpha-1/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/Users/djaiswal/.m2/repository/org/ehcache/ehcache/3.3.1/ehcache-3.3.1.jar:/Users/djaiswal/.m2/repository/com/zaxxer/HikariCP-java7/2.4.12/HikariCP-java7-2.4.12.jar:/Users/djaiswal/.m2/repository/com/microsoft/sqlserver/mssql-jdbc/6.2.1.jre7/mssql-jdbc-6.2.1.jre7.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-server-nodemanager/3.0.0-beta1/hadoop-yarn-server-nodemanager-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/com/codahale/metrics/metrics-core/3.0.1/metrics-core-3.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-server-resourcemanager/3.0.0-beta1/hadoop-yarn-server-resourcemanager-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-server-applicationhistoryservice/3.0.0-beta1/hadoop-yarn-server-applicationhistoryservice-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/de/ruedigermoeller/fst/2.50/fst-2.50.jar:/Users/djaiswal/.m2/repository/com/cedarsoftware/java-util/1.9.0/java-util-1.9.0.jar:/Users/djaiswal/.m2/repository/com/cedarsoftware/json-io/2.5.1/json-io-2.5.1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-server-timelineservice/3.0.0-beta1/hadoop-yarn-server-timelineservice-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-csv/1.0/commons-csv-1.0.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-client/3.0.0-beta1/hadoop-yarn-client-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-common/3.0.0-beta1/hadoop-yarn-common-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/javax/xml/bind/jaxb-api/2.2.11/jaxb-api-2.2.11.jar:/Users/djaiswal/.m2/repository/com/sun/jersey/jersey-client/1.19/jersey-client-1.19.jar:/Users/djaiswal/.m2/repository/com/google/inject/guice/4.0/guice-4.0.jar:/Users/djaiswal/.m2/repository/aopalliance/aopalliance/1.0/aopalliance-1.0.jar:/Users/djaiswal/.m2/repository/com/sun/jersey/contribs/jersey-guice/1.19/jersey-guice-1.19.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/core/jackson-core/2.7.8/jackson-core-2.7.8.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/module/jackson-module-jaxb-annotations/2.7.8/jackson-module-jaxb-annotations-2.7.8.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/jaxrs/jackson-jaxrs-json-provider/2.7.8/jackson-jaxrs-json-provider-2.7.8.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/jaxrs/jackson-jaxrs-base/2.7.8/jackson-jaxrs-base-2.7.8.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-api/3.0.0-beta1/hadoop-yarn-api-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/core/jackson-annotations/2.7.8/jackson-annotations-2.7.8.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-common/2.0.0-alpha4/hbase-common-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/thirdparty/hbase-shaded-miscellaneous/1.0.1/hbase-shaded-miscellaneous-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-collections4/4.1/commons-collections4-4.1.jar:/Users/djaiswal/.m2/repository/org/apache/htrace/htrace-core/3.2.0-incubating/htrace-core-3.2.0-incubating.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-crypto/1.0.0/commons-crypto-1.0.0.jar:/Users/djaiswal/.m2/repository/com/github/stephenc/findbugs/findbugs-annotations/1.3.9-1/findbugs-annotations-1.3.9-1.jar:/Users/djaiswal/.m2/repository/org/apache/yetus/audience-annotations/0.5.0/audience-annotations-0.5.0.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-common/2.0.0-alpha4/hbase-common-2.0.0-alpha4-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-hadoop-compat/2.0.0-alpha4/hbase-hadoop-compat-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-metrics-api/2.0.0-alpha4/hbase-metrics-api-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-hadoop-compat/2.0.0-alpha4/hbase-hadoop-compat-2.0.0-alpha4-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-hadoop2-compat/2.0.0-alpha4/hbase-hadoop2-compat-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-metrics/2.0.0-alpha4/hbase-metrics-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-hadoop2-compat/2.0.0-alpha4/hbase-hadoop2-compat-2.0.0-alpha4-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-server/2.0.0-alpha4/hbase-server-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/thirdparty/hbase-shaded-protobuf/1.0.1/hbase-shaded-protobuf-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/thirdparty/hbase-shaded-netty/1.0.1/hbase-shaded-netty-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-http/2.0.0-alpha4/hbase-http-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/glassfish/jersey/core/jersey-server/2.25.1/jersey-server-2.25.1.jar:/Users/djaiswal/.m2/repository/org/glassfish/jersey/core/jersey-common/2.25.1/jersey-common-2.25.1.jar:/Users/djaiswal/.m2/repository/org/glassfish/jersey/bundles/repackaged/jersey-guava/2.25.1/jersey-guava-2.25.1.jar:/Users/djaiswal/.m2/repository/org/glassfish/hk2/osgi-resource-locator/1.0.1/osgi-resource-locator-1.0.1.jar:/Users/djaiswal/.m2/repository/org/glassfish/jersey/core/jersey-client/2.25.1/jersey-client-2.25.1.jar:/Users/djaiswal/.m2/repository/org/glassfish/jersey/media/jersey-media-jaxb/2.25.1/jersey-media-jaxb-2.25.1.jar:/Users/djaiswal/.m2/repository/javax/annotation/javax.annotation-api/1.2/javax.annotation-api-1.2.jar:/Users/djaiswal/.m2/repository/org/glassfish/hk2/hk2-api/2.5.0-b32/hk2-api-2.5.0-b32.jar:/Users/djaiswal/.m2/repository/org/glassfish/hk2/hk2-utils/2.5.0-b32/hk2-utils-2.5.0-b32.jar:/Users/djaiswal/.m2/repository/org/glassfish/hk2/external/aopalliance-repackaged/2.5.0-b32/aopalliance-repackaged-2.5.0-b32.jar:/Users/djaiswal/.m2/repository/org/glassfish/hk2/external/javax.inject/2.5.0-b32/javax.inject-2.5.0-b32.jar:/Users/djaiswal/.m2/repository/org/glassfish/hk2/hk2-locator/2.5.0-b32/hk2-locator-2.5.0-b32.jar:/Users/djaiswal/.m2/repository/org/javassist/javassist/3.20.0-GA/javassist-3.20.0-GA.jar:/Users/djaiswal/.m2/repository/org/glassfish/jersey/containers/jersey-container-servlet-core/2.25.1/jersey-container-servlet-core-2.25.1.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-protocol/2.0.0-alpha4/hbase-protocol-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-protocol-shaded/2.0.0-alpha4/hbase-protocol-shaded-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-procedure/2.0.0-alpha4/hbase-procedure-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-replication/2.0.0-alpha4/hbase-replication-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-prefix-tree/2.0.0-alpha4/hbase-prefix-tree-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/glassfish/web/javax.servlet.jsp/2.3.2/javax.servlet.jsp-2.3.2.jar:/Users/djaiswal/.m2/repository/org/jamon/jamon-runtime/2.3.1/jamon-runtime-2.3.1.jar:/Users/djaiswal/.m2/repository/javax/ws/rs/javax.ws.rs-api/2.0.1/javax.ws.rs-api-2.0.1.jar:/Users/djaiswal/.m2/repository/com/lmax/disruptor/3.3.6/disruptor-3.3.6.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-client/3.0.0-beta1/hadoop-client-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-jobclient/3.0.0-beta1/hadoop-mapreduce-client-jobclient-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-server/2.0.0-alpha4/hbase-server-2.0.0-alpha4-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-mapreduce/2.0.0-alpha4/hbase-mapreduce-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/tez/tez-tests/0.9.1-SNAPSHOT/tez-tests-0.9.1-SNAPSHOT-tests.jar:/Users/djaiswal/.m2/repository/org/apache/tez/tez-common/0.9.1-SNAPSHOT/tez-common-0.9.1-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/slf4j/slf4j-log4j12/1.7.10/slf4j-log4j12-1.7.10.jar:/Users/djaiswal/.m2/repository/org/apache/tez/tez-examples/0.9.1-SNAPSHOT/tez-examples-0.9.1-SNAPSHOT.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_101.jdk/Contents/Home/jre/../lib/tools.jar:/Users/djaiswal/.m2/repository/org/apache/tez/tez-api/0.9.1-SNAPSHOT/tez-api-0.9.1-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/tez/tez-runtime-library/0.9.1-SNAPSHOT/tez-runtime-library-0.9.1-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/roaringbitmap/RoaringBitmap/0.4.9/RoaringBitmap-0.4.9.jar:/Users/djaiswal/.m2/repository/com/ning/async-http-client/1.8.16/async-http-client-1.8.16.jar:/Users/djaiswal/.m2/repository/org/apache/tez/tez-mapreduce/0.9.1-SNAPSHOT/tez-mapreduce-0.9.1-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/tez/tez-dag/0.9.1-SNAPSHOT/tez-dag-0.9.1-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/tez/hadoop-shim/0.9.1-SNAPSHOT/hadoop-shim-0.9.1-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/tez/tez-runtime-internals/0.9.1-SNAPSHOT/tez-runtime-internals-0.9.1-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-server-web-proxy/3.0.0-beta1/hadoop-yarn-server-web-proxy-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/mortbay/jetty/jetty/6.1.26/jetty-6.1.26.jar:/Users/djaiswal/.m2/repository/org/mortbay/jetty/servlet-api/2.5-20081211/servlet-api-2.5-20081211.jar:/Users/djaiswal/.m2/repository/org/mortbay/jetty/jetty-util/6.1.26/jetty-util-6.1.26.jar:/Users/djaiswal/.m2/repository/javax/servlet/servlet-api/2.5/servlet-api-2.5.jar:/Users/djaiswal/hive/hive/itests/qtest-druid/target/classes:/Users/djaiswal/.m2/repository/io/druid/druid-services/0.10.1/druid-services-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/druid-indexing-hadoop/0.10.1/druid-indexing-hadoop-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/druid-indexing-service/0.10.1/druid-indexing-service-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/druid-sql/0.10.1/druid-sql-0.10.1.jar:/Users/djaiswal/.m2/repository/org/apache/calcite/avatica/avatica-core/1.9.0/avatica-core-1.9.0.jar:/Users/djaiswal/.m2/repository/org/apache/calcite/avatica/avatica-metrics/1.9.0/avatica-metrics-1.9.0.jar:/Users/djaiswal/.m2/repository/org/apache/calcite/avatica/avatica-server/1.9.0/avatica-server-1.9.0.jar:/Users/djaiswal/.m2/repository/io/airlift/airline/0.7/airline-0.7.jar:/Users/djaiswal/.m2/repository/org/apache/logging/log4j/log4j-api/2.8.2/log4j-api-2.8.2.jar:/Users/djaiswal/.m2/repository/org/apache/logging/log4j/log4j-core/2.8.2/log4j-core-2.8.2.jar:/Users/djaiswal/.m2/repository/org/apache/curator/curator-x-discovery/2.11.0/curator-x-discovery-2.11.0.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-servlets/9.3.19.v20170502/jetty-servlets-9.3.19.v20170502.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-continuation/9.3.19.v20170502/jetty-continuation-9.3.19.v20170502.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-proxy/9.3.19.v20170502/jetty-proxy-9.3.19.v20170502.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-security/9.3.19.v20170502/jetty-security-9.3.19.v20170502.jar:/Users/djaiswal/.m2/repository/org/slf4j/slf4j-api/1.7.10/slf4j-api-1.7.10.jar:/Users/djaiswal/.m2/repository/org/mockito/mockito-all/1.10.19/mockito-all-1.10.19.jar:/Users/djaiswal/hive/hive/itests/qtest/target/testconf:/Users/djaiswal/hive/hive/itests/qtest/../../conf: 2018-01-05T11:28:27,526 INFO [main] zookeeper.ZooKeeper: Client environment:java.library.path=/Users/djaiswal/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:. 2018-01-05T11:28:27,526 INFO [main] zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/Users/djaiswal/hive/hive/itests/qtest/target/tmp 2018-01-05T11:28:27,526 INFO [main] zookeeper.ZooKeeper: Client environment:java.compiler= 2018-01-05T11:28:27,526 INFO [main] zookeeper.ZooKeeper: Client environment:os.name=Mac OS X 2018-01-05T11:28:27,526 INFO [main] zookeeper.ZooKeeper: Client environment:os.arch=x86_64 2018-01-05T11:28:27,526 INFO [main] zookeeper.ZooKeeper: Client environment:os.version=10.12.6 2018-01-05T11:28:27,527 INFO [main] zookeeper.ZooKeeper: Client environment:user.name=djaiswal 2018-01-05T11:28:27,527 INFO [main] zookeeper.ZooKeeper: Client environment:user.home=/Users/djaiswal 2018-01-05T11:28:27,527 INFO [main] zookeeper.ZooKeeper: Client environment:user.dir=/Users/djaiswal/hive/hive/itests/qtest 2018-01-05T11:28:27,527 INFO [main] zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:49384 sessionTimeout=1200000 watcher=org.apache.hadoop.hive.ql.QTestUtil$QTestSetup$1@5c87bfe2 2018-01-05T11:28:27,541 INFO [main] llap.LlapItUtils: Using conf dir: /Users/djaiswal/hive/hive/data/conf/llap 2018-01-05T11:28:27,543 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:49384] server.NIOServerCnxnFactory: Accepted socket connection from /127.0.0.1:49815 2018-01-05T11:28:27,546 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:49384] server.ZooKeeperServer: Client attempting to establish new session at /127.0.0.1:49815 2018-01-05T11:28:27,548 INFO [SyncThread:0] persistence.FileTxnLog: Creating new log file: log.1 2018-01-05T11:28:27,559 INFO [SyncThread:0] server.ZooKeeperServer: Established session 0x160c7cbc1600000 with negotiated timeout 40000 for client /127.0.0.1:49815 2018-01-05T11:28:27,564 INFO [main] llap.LlapItUtils: MiniLlap Configs - maxMemory: 1072693248 memoryForCache: 160904000 totalExecutorMemory: 804519936 numExecutors: 4 asyncIOEnabled: true directMemoryEnabled: false numLocalDirs: 1 2018-01-05T11:28:27,596 INFO [main] daemon.MiniLlapCluster: Created localDir: /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir 2018-01-05T11:28:27,596 DEBUG [main] service.AbstractService: Service: llap_MiniLlapCluster entered state INITED 2018-01-05T11:28:27,607 INFO [main] daemon.MiniLlapCluster: MiniLlap configured to use ports from conf: false 2018-01-05T11:28:27,607 INFO [main] daemon.MiniLlapCluster: Initializing 1 llap instances for MiniLlapCluster with name=llap_MiniLlapCluster 2018-01-05T11:28:27,607 INFO [main] impl.LlapDaemon: $$\ $$\ $$$$$$\ $$$$$$$\ $$ | $$ | $$ __$$\ $$ __$$\ $$ | $$ | $$ / $$ |$$ | $$ | $$ | $$ | $$$$$$$$ |$$$$$$$ | $$ | $$ | $$ __$$ |$$ ____/ $$ | $$ | $$ | $$ |$$ | $$$$$$$$\ $$$$$$$$\ $$ | $$ |$$ | \________|\________|\__| \__|\__| 2018-01-05T11:28:27,607 INFO [main] impl.LlapDaemon: Zookeeper Quorum: localhost 2018-01-05T11:28:27,608 INFO [main] impl.LlapDaemon: Starting daemon as user: djaiswal 2018-01-05T11:28:27,620 INFO [main] impl.LlapDaemon: Attempting to start LlapDaemon with the following configuration: maxJvmMemory=1072693248 (1023.00MB), requestedExecutorMemory=804519936 (767.25MB), llapIoCacheSize=160904000 (153.45MB), xmxHeadRoomMemory=53634662 (51.15MB), adjustedExecutorMemory=750885274 (716.10MB), numExecutors=4, llapIoEnabled=true, llapIoCacheIsDirect=false, rpcListenerPort=0, mngListenerPort=0, webPort=0, outputFormatSvcPort=0, workDirs=[/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir], shufflePort=0, waitQueueSize= 4, enablePreemption= true, versionInfo= (3.0.0-SNAPSHOT from 3f5148d6aae94f2ae9db2aacccb302211834c699 by djaiswal source checksum 66ed139dcfe9fdcd9eb4e029f37bbaf2) 2018-01-05T11:28:27,624 WARN [main] impl.StaticPermanentFunctionChecker: Could not find UDF whitelist in configuration: llap-udfs.lst 2018-01-05T11:28:27,686 DEBUG [main] converters.BooleanConverter: Setting default value: false 2018-01-05T11:28:27,686 DEBUG [main] converters.BooleanConverter: Converting 'Boolean' value 'false' to type 'Boolean' 2018-01-05T11:28:27,686 DEBUG [main] converters.BooleanConverter: No conversion required, value is already a Boolean 2018-01-05T11:28:27,689 DEBUG [main] converters.ByteConverter: Setting default value: 0 2018-01-05T11:28:27,689 DEBUG [main] converters.ByteConverter: Converting 'Integer' value '0' to type 'Byte' 2018-01-05T11:28:27,689 DEBUG [main] converters.ByteConverter: Converted to Byte value '0' 2018-01-05T11:28:27,691 DEBUG [main] converters.CharacterConverter: Setting default value: 2018-01-05T11:28:27,691 DEBUG [main] converters.CharacterConverter: Converting 'Character' value ' ' to type 'Character' 2018-01-05T11:28:27,691 DEBUG [main] converters.CharacterConverter: No conversion required, value is already a Character 2018-01-05T11:28:27,692 DEBUG [main] converters.DoubleConverter: Setting default value: 0 2018-01-05T11:28:27,692 DEBUG [main] converters.DoubleConverter: Converting 'Integer' value '0' to type 'Double' 2018-01-05T11:28:27,692 DEBUG [main] converters.DoubleConverter: Converted to Double value '0.0' 2018-01-05T11:28:27,693 DEBUG [main] converters.FloatConverter: Setting default value: 0 2018-01-05T11:28:27,693 DEBUG [main] converters.FloatConverter: Converting 'Integer' value '0' to type 'Float' 2018-01-05T11:28:27,693 DEBUG [main] converters.FloatConverter: Converted to Float value '0.0' 2018-01-05T11:28:27,694 DEBUG [main] converters.IntegerConverter: Setting default value: 0 2018-01-05T11:28:27,694 DEBUG [main] converters.IntegerConverter: Converting 'Integer' value '0' to type 'Integer' 2018-01-05T11:28:27,694 DEBUG [main] converters.IntegerConverter: No conversion required, value is already a Integer 2018-01-05T11:28:27,696 DEBUG [main] converters.LongConverter: Setting default value: 0 2018-01-05T11:28:27,696 DEBUG [main] converters.LongConverter: Converting 'Integer' value '0' to type 'Long' 2018-01-05T11:28:27,696 DEBUG [main] converters.LongConverter: Converted to Long value '0' 2018-01-05T11:28:27,697 DEBUG [main] converters.ShortConverter: Setting default value: 0 2018-01-05T11:28:27,697 DEBUG [main] converters.ShortConverter: Converting 'Integer' value '0' to type 'Short' 2018-01-05T11:28:27,697 DEBUG [main] converters.ShortConverter: Converted to Short value '0' 2018-01-05T11:28:27,700 DEBUG [main] converters.BigDecimalConverter: Setting default value: 0.0 2018-01-05T11:28:27,700 DEBUG [main] converters.BigDecimalConverter: Converting 'BigDecimal' value '0.0' to type 'BigDecimal' 2018-01-05T11:28:27,700 DEBUG [main] converters.BigDecimalConverter: No conversion required, value is already a BigDecimal 2018-01-05T11:28:27,701 DEBUG [main] converters.BigIntegerConverter: Setting default value: 0 2018-01-05T11:28:27,701 DEBUG [main] converters.BigIntegerConverter: Converting 'BigInteger' value '0' to type 'BigInteger' 2018-01-05T11:28:27,701 DEBUG [main] converters.BigIntegerConverter: No conversion required, value is already a BigInteger 2018-01-05T11:28:27,701 DEBUG [main] converters.BooleanConverter: Setting default value: false 2018-01-05T11:28:27,701 DEBUG [main] converters.BooleanConverter: Converting 'Boolean' value 'false' to type 'Boolean' 2018-01-05T11:28:27,701 DEBUG [main] converters.BooleanConverter: No conversion required, value is already a Boolean 2018-01-05T11:28:27,701 DEBUG [main] converters.ByteConverter: Setting default value: 0 2018-01-05T11:28:27,701 DEBUG [main] converters.ByteConverter: Converting 'Integer' value '0' to type 'Byte' 2018-01-05T11:28:27,701 DEBUG [main] converters.ByteConverter: Converted to Byte value '0' 2018-01-05T11:28:27,701 DEBUG [main] converters.CharacterConverter: Setting default value: 2018-01-05T11:28:27,702 DEBUG [main] converters.CharacterConverter: Converting 'Character' value ' ' to type 'Character' 2018-01-05T11:28:27,702 DEBUG [main] converters.CharacterConverter: No conversion required, value is already a Character 2018-01-05T11:28:27,702 DEBUG [main] converters.DoubleConverter: Setting default value: 0 2018-01-05T11:28:27,702 DEBUG [main] converters.DoubleConverter: Converting 'Integer' value '0' to type 'Double' 2018-01-05T11:28:27,702 DEBUG [main] converters.DoubleConverter: Converted to Double value '0.0' 2018-01-05T11:28:27,702 DEBUG [main] converters.FloatConverter: Setting default value: 0 2018-01-05T11:28:27,702 DEBUG [main] converters.FloatConverter: Converting 'Integer' value '0' to type 'Float' 2018-01-05T11:28:27,702 DEBUG [main] converters.FloatConverter: Converted to Float value '0.0' 2018-01-05T11:28:27,702 DEBUG [main] converters.IntegerConverter: Setting default value: 0 2018-01-05T11:28:27,702 DEBUG [main] converters.IntegerConverter: Converting 'Integer' value '0' to type 'Integer' 2018-01-05T11:28:27,702 DEBUG [main] converters.IntegerConverter: No conversion required, value is already a Integer 2018-01-05T11:28:27,702 DEBUG [main] converters.LongConverter: Setting default value: 0 2018-01-05T11:28:27,702 DEBUG [main] converters.LongConverter: Converting 'Integer' value '0' to type 'Long' 2018-01-05T11:28:27,702 DEBUG [main] converters.LongConverter: Converted to Long value '0' 2018-01-05T11:28:27,702 DEBUG [main] converters.ShortConverter: Setting default value: 0 2018-01-05T11:28:27,702 DEBUG [main] converters.ShortConverter: Converting 'Integer' value '0' to type 'Short' 2018-01-05T11:28:27,702 DEBUG [main] converters.ShortConverter: Converted to Short value '0' 2018-01-05T11:28:27,703 DEBUG [main] converters.StringConverter: Setting default value: 2018-01-05T11:28:27,703 DEBUG [main] converters.StringConverter: Converting 'String' value '' to type 'String' 2018-01-05T11:28:27,709 DEBUG [main] converters.ArrayConverter: Setting default value: [Z@3a4e343 2018-01-05T11:28:27,710 DEBUG [main] converters.ArrayConverter: Converting 'boolean[]' value '[Z@3a4e343' to type 'boolean[]' 2018-01-05T11:28:27,710 DEBUG [main] converters.ArrayConverter: No conversion required, value is already a boolean[] 2018-01-05T11:28:27,710 DEBUG [main] converters.ArrayConverter: Setting default value: [B@6a1d204a 2018-01-05T11:28:27,710 DEBUG [main] converters.ArrayConverter: Converting 'byte[]' value '[B@6a1d204a' to type 'byte[]' 2018-01-05T11:28:27,710 DEBUG [main] converters.ArrayConverter: No conversion required, value is already a byte[] 2018-01-05T11:28:27,710 DEBUG [main] converters.ArrayConverter: Setting default value: [C@62dae245 2018-01-05T11:28:27,710 DEBUG [main] converters.ArrayConverter: Converting 'char[]' value '[C@62dae245' to type 'char[]' 2018-01-05T11:28:27,710 DEBUG [main] converters.ArrayConverter: No conversion required, value is already a char[] 2018-01-05T11:28:27,710 DEBUG [main] converters.ArrayConverter: Setting default value: [D@4b6579e8 2018-01-05T11:28:27,710 DEBUG [main] converters.ArrayConverter: Converting 'double[]' value '[D@4b6579e8' to type 'double[]' 2018-01-05T11:28:27,710 DEBUG [main] converters.ArrayConverter: No conversion required, value is already a double[] 2018-01-05T11:28:27,710 DEBUG [main] converters.ArrayConverter: Setting default value: [F@6fff253c 2018-01-05T11:28:27,710 DEBUG [main] converters.ArrayConverter: Converting 'float[]' value '[F@6fff253c' to type 'float[]' 2018-01-05T11:28:27,710 DEBUG [main] converters.ArrayConverter: No conversion required, value is already a float[] 2018-01-05T11:28:27,710 DEBUG [main] converters.ArrayConverter: Setting default value: [I@6c6357f9 2018-01-05T11:28:27,710 DEBUG [main] converters.ArrayConverter: Converting 'int[]' value '[I@6c6357f9' to type 'int[]' 2018-01-05T11:28:27,710 DEBUG [main] converters.ArrayConverter: No conversion required, value is already a int[] 2018-01-05T11:28:27,710 DEBUG [main] converters.ArrayConverter: Setting default value: [J@591e58fa 2018-01-05T11:28:27,710 DEBUG [main] converters.ArrayConverter: Converting 'long[]' value '[J@591e58fa' to type 'long[]' 2018-01-05T11:28:27,710 DEBUG [main] converters.ArrayConverter: No conversion required, value is already a long[] 2018-01-05T11:28:27,710 DEBUG [main] converters.ArrayConverter: Setting default value: [S@3954d008 2018-01-05T11:28:27,710 DEBUG [main] converters.ArrayConverter: Converting 'short[]' value '[S@3954d008' to type 'short[]' 2018-01-05T11:28:27,710 DEBUG [main] converters.ArrayConverter: No conversion required, value is already a short[] 2018-01-05T11:28:27,710 DEBUG [main] converters.ArrayConverter: Setting default value: [Ljava.math.BigDecimal;@2f94c4db 2018-01-05T11:28:27,710 DEBUG [main] converters.ArrayConverter: Converting 'BigDecimal[]' value '[Ljava.math.BigDecimal;@2f94c4db' to type 'BigDecimal[]' 2018-01-05T11:28:27,710 DEBUG [main] converters.ArrayConverter: No conversion required, value is already a BigDecimal[] 2018-01-05T11:28:27,710 DEBUG [main] converters.ArrayConverter: Setting default value: [Ljava.math.BigInteger;@72ccd81a 2018-01-05T11:28:27,711 DEBUG [main] converters.ArrayConverter: Converting 'BigInteger[]' value '[Ljava.math.BigInteger;@72ccd81a' to type 'BigInteger[]' 2018-01-05T11:28:27,711 DEBUG [main] converters.ArrayConverter: No conversion required, value is already a BigInteger[] 2018-01-05T11:28:27,711 DEBUG [main] converters.ArrayConverter: Setting default value: [Ljava.lang.Boolean;@64bc21ac 2018-01-05T11:28:27,711 DEBUG [main] converters.ArrayConverter: Converting 'Boolean[]' value '[Ljava.lang.Boolean;@64bc21ac' to type 'Boolean[]' 2018-01-05T11:28:27,711 DEBUG [main] converters.ArrayConverter: No conversion required, value is already a Boolean[] 2018-01-05T11:28:27,711 DEBUG [main] converters.ArrayConverter: Setting default value: [Ljava.lang.Byte;@5d25e6bb 2018-01-05T11:28:27,711 DEBUG [main] converters.ArrayConverter: Converting 'Byte[]' value '[Ljava.lang.Byte;@5d25e6bb' to type 'Byte[]' 2018-01-05T11:28:27,711 DEBUG [main] converters.ArrayConverter: No conversion required, value is already a Byte[] 2018-01-05T11:28:27,711 DEBUG [main] converters.ArrayConverter: Setting default value: [Ljava.lang.Character;@9d157ff 2018-01-05T11:28:27,711 DEBUG [main] converters.ArrayConverter: Converting 'Character[]' value '[Ljava.lang.Character;@9d157ff' to type 'Character[]' 2018-01-05T11:28:27,711 DEBUG [main] converters.ArrayConverter: No conversion required, value is already a Character[] 2018-01-05T11:28:27,711 DEBUG [main] converters.ArrayConverter: Setting default value: [Ljava.lang.Double;@5df417a7 2018-01-05T11:28:27,711 DEBUG [main] converters.ArrayConverter: Converting 'Double[]' value '[Ljava.lang.Double;@5df417a7' to type 'Double[]' 2018-01-05T11:28:27,711 DEBUG [main] converters.ArrayConverter: No conversion required, value is already a Double[] 2018-01-05T11:28:27,711 DEBUG [main] converters.ArrayConverter: Setting default value: [Ljava.lang.Float;@7f69d591 2018-01-05T11:28:27,711 DEBUG [main] converters.ArrayConverter: Converting 'Float[]' value '[Ljava.lang.Float;@7f69d591' to type 'Float[]' 2018-01-05T11:28:27,711 DEBUG [main] converters.ArrayConverter: No conversion required, value is already a Float[] 2018-01-05T11:28:27,711 DEBUG [main] converters.ArrayConverter: Setting default value: [Ljava.lang.Integer;@1cb3ec38 2018-01-05T11:28:27,711 DEBUG [main] converters.ArrayConverter: Converting 'Integer[]' value '[Ljava.lang.Integer;@1cb3ec38' to type 'Integer[]' 2018-01-05T11:28:27,711 DEBUG [main] converters.ArrayConverter: No conversion required, value is already a Integer[] 2018-01-05T11:28:27,711 DEBUG [main] converters.ArrayConverter: Setting default value: [Ljava.lang.Long;@71c5b236 2018-01-05T11:28:27,711 DEBUG [main] converters.ArrayConverter: Converting 'Long[]' value '[Ljava.lang.Long;@71c5b236' to type 'Long[]' 2018-01-05T11:28:27,711 DEBUG [main] converters.ArrayConverter: No conversion required, value is already a Long[] 2018-01-05T11:28:27,711 DEBUG [main] converters.ArrayConverter: Setting default value: [Ljava.lang.Short;@2f7a7219 2018-01-05T11:28:27,711 DEBUG [main] converters.ArrayConverter: Converting 'Short[]' value '[Ljava.lang.Short;@2f7a7219' to type 'Short[]' 2018-01-05T11:28:27,711 DEBUG [main] converters.ArrayConverter: No conversion required, value is already a Short[] 2018-01-05T11:28:27,711 DEBUG [main] converters.ArrayConverter: Setting default value: [Ljava.lang.String;@3a1d593e 2018-01-05T11:28:27,711 DEBUG [main] converters.ArrayConverter: Converting 'String[]' value '[Ljava.lang.String;@3a1d593e' to type 'String[]' 2018-01-05T11:28:27,711 DEBUG [main] converters.ArrayConverter: No conversion required, value is already a String[] 2018-01-05T11:28:27,712 DEBUG [main] converters.ArrayConverter: Setting default value: [Ljava.lang.Class;@4a8a60bc 2018-01-05T11:28:27,712 DEBUG [main] converters.ArrayConverter: Converting 'Class[]' value '[Ljava.lang.Class;@4a8a60bc' to type 'Class[]' 2018-01-05T11:28:27,712 DEBUG [main] converters.ArrayConverter: No conversion required, value is already a Class[] 2018-01-05T11:28:27,712 DEBUG [main] converters.ArrayConverter: Setting default value: [Ljava.util.Date;@361c294e 2018-01-05T11:28:27,712 DEBUG [main] converters.ArrayConverter: Converting 'Date[]' value '[Ljava.util.Date;@361c294e' to type 'Date[]' 2018-01-05T11:28:27,712 DEBUG [main] converters.ArrayConverter: No conversion required, value is already a Date[] 2018-01-05T11:28:27,712 DEBUG [main] converters.ArrayConverter: Setting default value: [Ljava.util.Calendar;@285d851a 2018-01-05T11:28:27,712 DEBUG [main] converters.ArrayConverter: Converting 'Calendar[]' value '[Ljava.util.Calendar;@285d851a' to type 'Calendar[]' 2018-01-05T11:28:27,712 DEBUG [main] converters.ArrayConverter: No conversion required, value is already a Calendar[] 2018-01-05T11:28:27,712 DEBUG [main] converters.ArrayConverter: Setting default value: [Ljava.io.File;@664a9613 2018-01-05T11:28:27,712 DEBUG [main] converters.ArrayConverter: Converting 'java.io.File[]' value '[Ljava.io.File;@664a9613' to type 'java.io.File[]' 2018-01-05T11:28:27,712 DEBUG [main] converters.ArrayConverter: No conversion required, value is already a java.io.File[] 2018-01-05T11:28:27,712 DEBUG [main] converters.ArrayConverter: Setting default value: [Ljava.sql.Date;@15a902e7 2018-01-05T11:28:27,712 DEBUG [main] converters.ArrayConverter: Converting 'java.sql.Date[]' value '[Ljava.sql.Date;@15a902e7' to type 'java.sql.Date[]' 2018-01-05T11:28:27,712 DEBUG [main] converters.ArrayConverter: No conversion required, value is already a java.sql.Date[] 2018-01-05T11:28:27,712 DEBUG [main] converters.ArrayConverter: Setting default value: [Ljava.sql.Time;@4a3e3e8b 2018-01-05T11:28:27,712 DEBUG [main] converters.ArrayConverter: Converting 'java.sql.Time[]' value '[Ljava.sql.Time;@4a3e3e8b' to type 'java.sql.Time[]' 2018-01-05T11:28:27,712 DEBUG [main] converters.ArrayConverter: No conversion required, value is already a java.sql.Time[] 2018-01-05T11:28:27,712 DEBUG [main] converters.ArrayConverter: Setting default value: [Ljava.sql.Timestamp;@71104a4 2018-01-05T11:28:27,712 DEBUG [main] converters.ArrayConverter: Converting 'java.sql.Timestamp[]' value '[Ljava.sql.Timestamp;@71104a4' to type 'java.sql.Timestamp[]' 2018-01-05T11:28:27,712 DEBUG [main] converters.ArrayConverter: No conversion required, value is already a java.sql.Timestamp[] 2018-01-05T11:28:27,712 DEBUG [main] converters.ArrayConverter: Setting default value: [Ljava.net.URL;@72f46e16 2018-01-05T11:28:27,712 DEBUG [main] converters.ArrayConverter: Converting 'java.net.URL[]' value '[Ljava.net.URL;@72f46e16' to type 'java.net.URL[]' 2018-01-05T11:28:27,712 DEBUG [main] converters.ArrayConverter: No conversion required, value is already a java.net.URL[] 2018-01-05T11:28:27,752 INFO [main] beanutils.FluentPropertyBeanIntrospector: Error when creating PropertyDescriptor for public final void org.apache.commons.configuration2.AbstractConfiguration.setProperty(java.lang.String,java.lang.Object)! Ignoring this property. 2018-01-05T11:28:27,752 DEBUG [main] beanutils.FluentPropertyBeanIntrospector: Exception is: java.beans.IntrospectionException: bad write method arg count: public final void org.apache.commons.configuration2.AbstractConfiguration.setProperty(java.lang.String,java.lang.Object) at java.beans.PropertyDescriptor.findPropertyType(PropertyDescriptor.java:657) ~[?:1.8.0_101] at java.beans.PropertyDescriptor.setWriteMethod(PropertyDescriptor.java:327) ~[?:1.8.0_101] at java.beans.PropertyDescriptor.(PropertyDescriptor.java:139) ~[?:1.8.0_101] at org.apache.commons.beanutils.FluentPropertyBeanIntrospector.createFluentPropertyDescritor(FluentPropertyBeanIntrospector.java:178) ~[commons-beanutils-1.9.3.jar:1.9.3] at org.apache.commons.beanutils.FluentPropertyBeanIntrospector.introspect(FluentPropertyBeanIntrospector.java:141) [commons-beanutils-1.9.3.jar:1.9.3] at org.apache.commons.beanutils.PropertyUtilsBean.fetchIntrospectionData(PropertyUtilsBean.java:2245) [commons-beanutils-1.9.3.jar:1.9.3] at org.apache.commons.beanutils.PropertyUtilsBean.getIntrospectionData(PropertyUtilsBean.java:2226) [commons-beanutils-1.9.3.jar:1.9.3] at org.apache.commons.beanutils.PropertyUtilsBean.getPropertyDescriptor(PropertyUtilsBean.java:954) [commons-beanutils-1.9.3.jar:1.9.3] at org.apache.commons.beanutils.PropertyUtilsBean.isWriteable(PropertyUtilsBean.java:1478) [commons-beanutils-1.9.3.jar:1.9.3] at org.apache.commons.configuration2.beanutils.BeanHelper.isPropertyWriteable(BeanHelper.java:521) [commons-configuration2-2.1.1.jar:2.1.1] at org.apache.commons.configuration2.beanutils.BeanHelper.initProperty(BeanHelper.java:357) [commons-configuration2-2.1.1.jar:2.1.1] at org.apache.commons.configuration2.beanutils.BeanHelper.initBeanProperties(BeanHelper.java:273) [commons-configuration2-2.1.1.jar:2.1.1] at org.apache.commons.configuration2.beanutils.BeanHelper.initBean(BeanHelper.java:192) [commons-configuration2-2.1.1.jar:2.1.1] at org.apache.commons.configuration2.beanutils.BeanHelper$BeanCreationContextImpl.initBean(BeanHelper.java:669) [commons-configuration2-2.1.1.jar:2.1.1] at org.apache.commons.configuration2.beanutils.DefaultBeanFactory.initBeanInstance(DefaultBeanFactory.java:162) [commons-configuration2-2.1.1.jar:2.1.1] at org.apache.commons.configuration2.beanutils.DefaultBeanFactory.createBean(DefaultBeanFactory.java:116) [commons-configuration2-2.1.1.jar:2.1.1] at org.apache.commons.configuration2.beanutils.BeanHelper.createBean(BeanHelper.java:459) [commons-configuration2-2.1.1.jar:2.1.1] at org.apache.commons.configuration2.beanutils.BeanHelper.createBean(BeanHelper.java:479) [commons-configuration2-2.1.1.jar:2.1.1] at org.apache.commons.configuration2.beanutils.BeanHelper.createBean(BeanHelper.java:492) [commons-configuration2-2.1.1.jar:2.1.1] at org.apache.commons.configuration2.builder.BasicConfigurationBuilder.createResultInstance(BasicConfigurationBuilder.java:447) [commons-configuration2-2.1.1.jar:2.1.1] at org.apache.commons.configuration2.builder.BasicConfigurationBuilder.createResult(BasicConfigurationBuilder.java:417) [commons-configuration2-2.1.1.jar:2.1.1] at org.apache.commons.configuration2.builder.BasicConfigurationBuilder.getConfiguration(BasicConfigurationBuilder.java:285) [commons-configuration2-2.1.1.jar:2.1.1] at org.apache.hadoop.metrics2.impl.MetricsConfig.loadFirst(MetricsConfig.java:119) [hadoop-common-3.0.0-beta1.jar:?] at org.apache.hadoop.metrics2.impl.MetricsConfig.create(MetricsConfig.java:98) [hadoop-common-3.0.0-beta1.jar:?] at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.configure(MetricsSystemImpl.java:478) [hadoop-common-3.0.0-beta1.jar:?] at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.start(MetricsSystemImpl.java:188) [hadoop-common-3.0.0-beta1.jar:?] at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.init(MetricsSystemImpl.java:163) [hadoop-common-3.0.0-beta1.jar:?] at org.apache.hadoop.hive.llap.metrics.LlapMetricsSystem.initialize(LlapMetricsSystem.java:41) [hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon.(LlapDaemon.java:243) [hive-llap-server-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.llap.daemon.MiniLlapCluster.serviceInit(MiniLlapCluster.java:171) [hive-llap-server-3.0.0-SNAPSHOT-tests.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) [hadoop-common-3.0.0-beta1.jar:?] at org.apache.hadoop.hive.llap.LlapItUtils.startAndGetMiniLlapCluster(LlapItUtils.java:77) [classes/:?] at org.apache.hadoop.hive.ql.QTestUtil.setupMiniCluster(QTestUtil.java:653) [classes/:?] at org.apache.hadoop.hive.ql.QTestUtil.(QTestUtil.java:581) [classes/:?] at org.apache.hadoop.hive.cli.control.CoreCliDriver$1.invokeInternal(CoreCliDriver.java:65) [classes/:?] at org.apache.hadoop.hive.cli.control.CoreCliDriver$1.invokeInternal(CoreCliDriver.java:61) [classes/:?] at org.apache.hadoop.hive.util.ElapsedTimeLoggingWrapper.invoke(ElapsedTimeLoggingWrapper.java:33) [classes/:?] at org.apache.hadoop.hive.cli.control.CoreCliDriver.beforeClass(CoreCliDriver.java:67) [classes/:?] at org.apache.hadoop.hive.cli.control.CliAdapter$1$1.evaluate(CliAdapter.java:71) [classes/:?] at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.11.jar:?] at org.junit.runners.ParentRunner.run(ParentRunner.java:309) [junit-4.11.jar:?] at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:369) [surefire-junit4-2.20.1.jar:2.20.1] at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:275) [surefire-junit4-2.20.1.jar:2.20.1] at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:239) [surefire-junit4-2.20.1.jar:2.20.1] at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:160) [surefire-junit4-2.20.1.jar:2.20.1] at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:373) [surefire-booter-2.20.1.jar:2.20.1] at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:334) [surefire-booter-2.20.1.jar:2.20.1] at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:119) [surefire-booter-2.20.1.jar:2.20.1] at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:407) [surefire-booter-2.20.1.jar:2.20.1] 2018-01-05T11:28:27,770 WARN [main] impl.MetricsConfig: Cannot locate configuration: tried hadoop-metrics2-llapdaemon.properties,hadoop-metrics2.properties 2018-01-05T11:28:27,786 INFO [main] impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s). 2018-01-05T11:28:27,786 INFO [main] impl.MetricsSystemImpl: LlapDaemon metrics system started 2018-01-05T11:28:27,801 INFO [main] util.ProcfsBasedProcessTree: ProcfsBasedProcessTree currently is supported only on Linux. 2018-01-05T11:28:27,823 INFO [main] impl.LlapDaemon: Started LlapMetricsSystem with displayName: LlapDaemonExecutorMetrics-hw13500.local sessionId: 08b8d4bf-4067-4799-a62f-002ef4e0fd8f 2018-01-05T11:28:27,837 INFO [main] impl.AMReporter: Setting up AMReporter with heartbeatInterval(ms)=10000, retryTime(ms)=10000, retrySleep(ms)=2000 2018-01-05T11:28:27,845 INFO [main] impl.LlapProtocolServerImpl: Creating: LlapProtocolServerImpl with port configured to: 0 2018-01-05T11:28:27,856 INFO [main] impl.QueryTracker: QueryTracker setup with numCleanerThreads=1, defaultCleanupDelay(s)=300, routeBasedLogging=true 2018-01-05T11:28:27,856 DEBUG [main] service.CompositeService: Adding service QueryTracker 2018-01-05T11:28:27,863 INFO [main] impl.TaskExecutorService: TaskExecutorService is being setup with parameters: numExecutors=4, waitQueueSize=4, waitQueueComparatorClassName=org.apache.hadoop.hive.llap.daemon.impl.comparator.ShortestJobFirstComparator, enablePreemption=true 2018-01-05T11:28:27,918 DEBUG [main] service.CompositeService: Adding service TaskExecutorService 2018-01-05T11:28:27,925 INFO [main] shim.HadoopShimsLoader: Trying to locate HadoopShimProvider for hadoopVersion=3.0.0-beta1, majorVersion=3, minorVersion=0 2018-01-05T11:28:27,926 INFO [main] shim.HadoopShimsLoader: Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=3.0.0-beta1, majorVersion=3, minorVersion=0 2018-01-05T11:28:27,926 INFO [main] impl.ContainerRunnerImpl: ContainerRunnerImpl config: memoryPerExecutorDerviced=187721312 2018-01-05T11:28:27,926 DEBUG [main] service.CompositeService: Adding service ContainerRunnerImpl 2018-01-05T11:28:27,929 DEBUG [main] service.CompositeService: Adding service LlapDaemonProtocolServerImpl 2018-01-05T11:28:27,929 DEBUG [main] service.CompositeService: Adding service org.apache.hadoop.hive.llap.daemon.impl.AMReporter 2018-01-05T11:28:27,930 DEBUG [main] service.AbstractService: Service: LlapDaemon entered state INITED 2018-01-05T11:28:27,930 DEBUG [main] service.CompositeService: LlapDaemon: initing services, size=3 2018-01-05T11:28:27,930 DEBUG [main] service.AbstractService: Service: ContainerRunnerImpl entered state INITED 2018-01-05T11:28:27,930 DEBUG [main] service.CompositeService: ContainerRunnerImpl: initing services, size=2 2018-01-05T11:28:27,930 DEBUG [main] service.AbstractService: Service: QueryTracker entered state INITED 2018-01-05T11:28:27,939 DEBUG [main] service.AbstractService: Service: TaskExecutorService entered state INITED 2018-01-05T11:28:27,939 DEBUG [main] service.AbstractService: Service: LlapDaemonProtocolServerImpl entered state INITED 2018-01-05T11:28:27,939 DEBUG [main] service.AbstractService: Service: org.apache.hadoop.hive.llap.daemon.impl.AMReporter entered state INITED 2018-01-05T11:28:27,958 INFO [main] LlapIoImpl: Initializing LLAP IO in cache mode 2018-01-05T11:28:27,965 INFO [main] metrics.LlapDaemonIOMetrics: Created interval PercentileDecodingTime_30s 2018-01-05T11:28:27,967 INFO [main] LlapIoImpl: Started llap daemon metrics with displayName: LlapDaemonIOMetrics-hw13500.local sessionId: 08b8d4bf-4067-4799-a62f-002ef4e0fd8f 2018-01-05T11:28:27,968 INFO [main] LlapIoImpl: LRFU cache policy with min buffer size 16384 and lambda 0.009999999776482582 (heap size 717) 2018-01-05T11:28:27,970 INFO [main] LlapIoImpl: Memory manager initialized with max size 1073741824 and ability to evict blocks 2018-01-05T11:28:27,975 INFO [main] LlapIoImpl: Buddy allocator with byte buffers; allocation sizes 16384 - 16777216, arena size 134217728, total size 1073741824 2018-01-05T11:28:27,976 INFO [main] LlapIoImpl: Leaving 1048576 of defragmentation headroom 2018-01-05T11:28:28,049 INFO [main] LlapIoImpl: Low level cache; cleanup interval 600 sec 2018-01-05T11:28:28,060 INFO [main] LlapIoImpl: SerDe low-level level cache; cleanup interval 600 sec 2018-01-05T11:28:28,069 INFO [main] LlapIoImpl: Initializing ORC column vector producer 2018-01-05T11:28:28,071 INFO [main] LlapIoImpl: Initializing ORC column vector producer 2018-01-05T11:28:28,071 INFO [main] LlapIoImpl: LLAP IO initialized 2018-01-05T11:28:28,072 INFO [main] daemon.MiniLlapCluster: Initialized 1 llap instances for MiniLlapCluster with name=llap_MiniLlapCluster 2018-01-05T11:28:28,072 INFO [main] daemon.MiniLlapCluster: Starting 1 llap instances for MiniLlapCluster with name=llap_MiniLlapCluster 2018-01-05T11:28:28,242 INFO [main] shufflehandler.IndexCache: IndexCache created with max memory = 10485760 2018-01-05T11:28:28,242 INFO [main] shufflehandler.ShuffleHandler: DirWatcher disabled by config 2018-01-05T11:28:28,242 INFO [main] shufflehandler.ShuffleHandler: manageOsCache:true, shouldAlwaysEvictOsCache:false, readaheadLength:4194304, maxShuffleConnections:0, localDirs:[/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir], shuffleBufferSize:131072, shuffleTransferToAllowed:true, connectionKeepAliveEnabled:true, connectionKeepAliveTimeOut:5, mapOutputMetaInfoCacheSize:10000, sslFileBufferSize:61440 2018-01-05T11:28:28,283 DEBUG [main] logging.InternalLoggerFactory: Using SLF4J as the default logging framework 2018-01-05T11:28:28,288 DEBUG [main] util.NetUtil: Loopback interface: lo0 (lo0, 127.0.0.1) 2018-01-05T11:28:28,353 DEBUG [main] internal.PlatformDependent: Platform: MacOS 2018-01-05T11:28:28,357 DEBUG [main] internal.PlatformDependent0: -Dio.netty.noUnsafe: false 2018-01-05T11:28:28,357 DEBUG [main] internal.PlatformDependent0: Java version: 8 2018-01-05T11:28:28,359 DEBUG [main] internal.PlatformDependent0: sun.misc.Unsafe.theUnsafe: available 2018-01-05T11:28:28,360 DEBUG [main] internal.PlatformDependent0: sun.misc.Unsafe.copyMemory: available 2018-01-05T11:28:28,360 DEBUG [main] internal.PlatformDependent0: java.nio.Buffer.address: available 2018-01-05T11:28:28,361 DEBUG [main] internal.PlatformDependent0: direct buffer constructor: available 2018-01-05T11:28:28,362 DEBUG [main] internal.PlatformDependent0: java.nio.Bits.unaligned: available, true 2018-01-05T11:28:28,362 DEBUG [main] internal.PlatformDependent0: jdk.internal.misc.Unsafe.allocateUninitializedArray(int): unavailable prior to Java9 2018-01-05T11:28:28,362 DEBUG [main] internal.PlatformDependent0: java.nio.DirectByteBuffer.(long, int): available 2018-01-05T11:28:28,362 DEBUG [main] internal.PlatformDependent: sun.misc.Unsafe: available 2018-01-05T11:28:28,363 DEBUG [main] internal.PlatformDependent: -Dio.netty.tmpdir: /Users/djaiswal/hive/hive/itests/qtest/target/tmp (java.io.tmpdir) 2018-01-05T11:28:28,363 DEBUG [main] internal.PlatformDependent: -Dio.netty.bitMode: 64 (sun.arch.data.model) 2018-01-05T11:28:28,364 DEBUG [main] internal.PlatformDependent: -Dio.netty.noPreferDirect: false 2018-01-05T11:28:28,365 DEBUG [main] internal.PlatformDependent: -Dio.netty.maxDirectMemory: 954728448 bytes 2018-01-05T11:28:28,365 DEBUG [main] internal.PlatformDependent: -Dio.netty.uninitializedArrayAllocationThreshold: -1 2018-01-05T11:28:28,367 DEBUG [main] internal.CleanerJava6: java.nio.ByteBuffer.cleaner(): available 2018-01-05T11:28:28,367 DEBUG [main] util.NetUtil: /proc/sys/net/core/somaxconn: 128 (non-existent) 2018-01-05T11:28:28,388 INFO [main] shufflehandler.ShuffleHandler: LlapShuffleHandler listening on port 49816 (SOMAXCONN: 128) 2018-01-05T11:28:28,388 INFO [main] impl.LlapDaemon: Setting shuffle port to: 49816 2018-01-05T11:28:28,394 INFO [main] llap.LlapOutputFormat: Starting LlapOutputFormatService 2018-01-05T11:28:28,404 DEBUG [main] channel.MultithreadEventLoopGroup: -Dio.netty.eventLoopThreads: 16 2018-01-05T11:28:28,444 DEBUG [main] nio.NioEventLoop: -Dio.netty.noKeySetOptimization: false 2018-01-05T11:28:28,444 DEBUG [main] nio.NioEventLoop: -Dio.netty.selectorAutoRebuildThreshold: 512 2018-01-05T11:28:28,462 DEBUG [main] internal.PlatformDependent: org.jctools-core.MpscChunkedArrayQueue: available 2018-01-05T11:28:28,499 DEBUG [main] internal.InternalThreadLocalMap: -Dio.netty.threadLocalMap.stringBuilder.initialSize: 1024 2018-01-05T11:28:28,499 DEBUG [main] internal.InternalThreadLocalMap: -Dio.netty.threadLocalMap.stringBuilder.maxSize: 4096 2018-01-05T11:28:28,531 DEBUG [main] buffer.ByteBufUtil: -Dio.netty.allocator.type: unpooled 2018-01-05T11:28:28,531 DEBUG [main] buffer.ByteBufUtil: -Dio.netty.threadLocalDirectBufferSize: 65536 2018-01-05T11:28:28,531 DEBUG [main] buffer.ByteBufUtil: -Dio.netty.maxThreadLocalCharBufferSize: 16384 2018-01-05T11:28:28,540 INFO [main] llap.LlapOutputFormat: LlapOutputFormatService: Binding to port: 49817 with send buffer size: 131072 2018-01-05T11:28:28,540 DEBUG [main] service.CompositeService: LlapDaemon: starting services, size=3 2018-01-05T11:28:28,540 INFO [main] impl.ContainerRunnerImpl: Using ShufflePort: 49816 2018-01-05T11:28:28,543 DEBUG [main] service.CompositeService: ContainerRunnerImpl: starting services, size=2 2018-01-05T11:28:28,543 INFO [main] impl.QueryTracker: QueryTracker started 2018-01-05T11:28:28,543 DEBUG [main] service.AbstractService: Service QueryTracker is started 2018-01-05T11:28:28,543 DEBUG [main] service.AbstractService: Service TaskExecutorService is started 2018-01-05T11:28:28,543 DEBUG [main] service.AbstractService: Service ContainerRunnerImpl is started 2018-01-05T11:28:28,597 INFO [main] llap.LlapUtil: Instantiated LlapProtocolBlockingPB at /10.22.8.182:49818 2018-01-05T11:28:28,600 INFO [main] llap.LlapUtil: Instantiated LlapManagementProtocolPB at /10.22.8.182:49819 2018-01-05T11:28:28,600 DEBUG [main] service.AbstractService: Service LlapDaemonProtocolServerImpl is started 2018-01-05T11:28:28,608 INFO [main] impl.AMReporter: AMReporter running with DaemonId: org.apache.hadoop.hive.llap.DaemonId@7d755813, NodeId: 10.22.8.182:49818 2018-01-05T11:28:28,609 DEBUG [main] service.AbstractService: Service org.apache.hadoop.hive.llap.daemon.impl.AMReporter is started 2018-01-05T11:28:28,609 DEBUG [main] service.AbstractService: Service: LlapRegistryService entered state INITED 2018-01-05T11:28:28,717 INFO [main] impl.LlapZookeeperRegistryImpl: Llap Zookeeper Registry is enabled with registryid: llap_MiniLlapCluster 2018-01-05T11:28:28,717 INFO [main] impl.LlapRegistryService: Using LLAP registry type org.apache.hadoop.hive.llap.registry.impl.LlapZookeeperRegistryImpl@5177e388 2018-01-05T11:28:28,718 INFO [main] impl.ZookeeperUtils: UGI security is not enabled, or no SASL context name. Skipping setting up ZK auth. 2018-01-05T11:28:28,718 INFO [main] imps.CuratorFrameworkImpl: Starting 2018-01-05T11:28:28,720 DEBUG [main] curator.CuratorZookeeperClient: Starting 2018-01-05T11:28:28,720 DEBUG [main] curator.ConnectionState: Starting 2018-01-05T11:28:28,720 DEBUG [main] curator.ConnectionState: reset 2018-01-05T11:28:28,720 INFO [main] zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:49384 sessionTimeout=1200000 watcher=org.apache.curator.ConnectionState@5e1218b4 2018-01-05T11:28:28,722 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:49384] server.NIOServerCnxnFactory: Accepted socket connection from /127.0.0.1:49820 2018-01-05T11:28:28,722 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:49384] server.ZooKeeperServer: Client attempting to establish new session at /127.0.0.1:49820 2018-01-05T11:28:28,724 INFO [SyncThread:0] server.ZooKeeperServer: Established session 0x160c7cbc1600001 with negotiated timeout 40000 for client /127.0.0.1:49820 2018-01-05T11:28:28,729 INFO [main-EventThread] state.ConnectionStateManager: State change: CONNECTED 2018-01-05T11:28:28,807 WARN [main] utils.ZKPaths: The version of ZooKeeper being used doesn't support Container nodes. CreateMode.PERSISTENT will be used instead. 2018-01-05T11:28:28,812 INFO [ProcessThread(sid:0 cport:-1):] server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x160c7cbc1600001 type:create cxid:0x3 zxid:0x4 txntype:-1 reqpath:n/a Error Path:/llap-unsecure/user-djaiswal/llap_MiniLlapCluster/workers Error:KeeperErrorCode = NoNode for /llap-unsecure/user-djaiswal/llap_MiniLlapCluster/workers 2018-01-05T11:28:28,829 INFO [main] impl.SlotZnode: Will attempt to take slot 0 2018-01-05T11:28:28,829 INFO [main] impl.SlotZnode: Attempting to create /user-djaiswal/llap_MiniLlapCluster/workers/slot-0000000000 2018-01-05T11:28:28,831 INFO [main-EventThread] impl.SlotZnode: Acquired the slot znode /user-djaiswal/llap_MiniLlapCluster/workers/slot-0000000000; this is the initial assignment 2018-01-05T11:28:28,831 INFO [main] impl.LlapZookeeperRegistryImpl: Registered node. Created a znode on ZooKeeper for LLAP instance: rpc: { "api" : "llap", "addressType" : "host/port", "protocolType" : "hadoop/IPC", "addresses" : [ { "host" : "10.22.8.182", "port" : "49818" } ] }, shuffle: { "api" : "shuffle", "addressType" : "host/port", "protocolType" : "tcp", "addresses" : [ { "host" : "10.22.8.182", "port" : "49816" } ] }, webui: { "api" : "services", "addressType" : "uri", "protocolType" : "webui", "addresses" : [ { "uri" : "http://10.22.8.182:15002" } ] }, mgmt: { "api" : "llapmng", "addressType" : "host/port", "protocolType" : "hadoop/IPC", "addresses" : [ { "host" : "10.22.8.182", "port" : "49819" } ] }, znodePath: /user-djaiswal/llap_MiniLlapCluster/workers/worker-0000000000 2018-01-05T11:28:28,833 DEBUG [main] service.AbstractService: Service LlapRegistryService is started 2018-01-05T11:28:28,833 INFO [main] impl.LlapDaemon: LlapDaemon serviceStart complete. RPC Port=49818, ManagementPort=49819, ShuflePort=49816, WebPort= 2018-01-05T11:28:28,833 DEBUG [main] service.AbstractService: Service LlapDaemon is started 2018-01-05T11:28:28,833 INFO [main] daemon.MiniLlapCluster: Started 1 llap instances for MiniLlapCluster with name=llap_MiniLlapCluster 2018-01-05T11:28:28,833 DEBUG [main] service.AbstractService: Service llap_MiniLlapCluster is started 2018-01-05T11:28:28,896 DEBUG [main] session.SessionState: SessionState user: null 2018-01-05T11:28:28,900 INFO [main] DependencyResolver: ivysettings.xml file not found in HIVE_HOME or HIVE_CONF_DIR,/Users/djaiswal/hive/hive/conf/ivysettings.xml will be used 2018-01-05T11:28:28,970 DEBUG [main] exec.Utilities: Create dirs /Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir with permission rwx-wx-wx recursive true 2018-01-05T11:28:28,970 DEBUG [main] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:28,970 DEBUG [main] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:28,970 DEBUG [main] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:28,970 DEBUG [main] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:29,005 DEBUG [main] session.SessionState: HDFS root scratch dir: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir with schema null, permission: rwx-wx-wx 2018-01-05T11:28:29,017 INFO [main] session.SessionState: Created HDFS directory: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal 2018-01-05T11:28:29,028 INFO [main] session.SessionState: Created local directory: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir 2018-01-05T11:28:29,040 INFO [main] session.SessionState: Created HDFS directory: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:29,052 INFO [main] session.SessionState: Created local directory: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:29,065 INFO [main] session.SessionState: Created HDFS directory: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/53565c10-a9a1-4685-81b4-afcf82ce77fb/_tmp_space.db 2018-01-05T11:28:29,737 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:28:29,756 INFO [main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:29,785 WARN [main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:29,785 WARN [main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:29,790 INFO [main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:29,791 INFO [main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hive-site.xml 2018-01-05T11:28:29,791 INFO [main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hivemetastore-site.xml 2018-01-05T11:28:29,792 INFO [main] conf.MetastoreConf: Unable to find config file metastore-site.xml 2018-01-05T11:28:29,792 INFO [main] conf.MetastoreConf: Found configuration file null 2018-01-05T11:28:29,804 INFO [main] conf.MetastoreConf: MetastoreConf object: Used hive-site file: file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hive-site.xml Used hivemetastore-site file: file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hivemetastore-site.xml Key: old hive key: value: <> Key: old hive key: value: <0.8> Key: old hive key: value: Key: old hive key: value: <0.01> Key: old hive key: value: <0.9> Key: old hive key: value: <10000> Key: old hive key: value: <1000ms> Key: old hive key: value: <0.01> Key: old hive key: value: <5000ms> Key: old hive key: value: <10000> Key: old hive key: value: <600s> Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: <300> Key: old hive key: value: <1000> Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: <60s> Key: old hive key: value: <.*> Key: old hive key: value: <> Key: old hive key: value: Key: old hive key: value: <1s> Key: old hive key: value: <> Key: old hive key: value: <0s> Key: old hive key: value: <600s> Key: old hive key: value: <2m> Key: old hive key: value: <2> Key: old hive key: value: <3> Key: old hive key: value: <3> Key: old hive key: value: <2> Key: old hive key: value: Key: old hive key: value: <0> Key: old hive key: value: Key: old hive key: value: <10> Key: old hive key: value: <> Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: <1s> Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: <> Key: old hive key: value: <__HIVE_DEFAULT_PARTITION__> Key: old hive key: value: <1d> Key: old hive key: value: <1h> Key: old hive key: value: <7d> Key: old hive key: value: <1d> Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: <1000> Key: old hive key: value: <1000> Key: old hive key: value: <100> Key: old hive key: value: <0> Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: <> Key: old hive key: value: <0s> Key: old hive key: value: <0s> Key: old hive key: value: <> Key: old hive key: value: Key: old hive key: value: <86400s> Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: <1> Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: <15> Key: old hive key: value: <10> Key: old hive key: value: Key: old hive key: value: <2000ms> Key: old hive key: value: Key: old hive key: value: <> Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: <> Key: old hive key: value: Key: old hive key: value: <-1> Key: old hive key: value: <> Key: old hive key: value: Key: old hive key: value: <> Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: <1m> Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: <100000> Key: old hive key: value: Key: old hive key: value: <5> Key: old hive key: value: <500ms> Key: old hive key: value: Key: old hive key: value: <> Key: old hive key: value: <> Key: old hive key: value: <> Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: <24h> Key: old hive key: value: <3600s> Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: <1> Key: old hive key: value: <33554432> Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: <104857600> Key: old hive key: value: <1000> Key: old hive key: value: <200> Key: old hive key: value: <9083> Key: old hive key: value: <> Key: old hive key: value: Key: old hive key: value: <> Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: <0.0> Key: old hive key: value: Key: old hive key: value: <> Key: old hive key: value: <> Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: <10> Key: old hive key: value: <3> Key: old hive key: value: <1> Key: old hive key: value: <> Key: old hive key: value: <100s> Key: old hive key: value: <180s> Key: old hive key: value: <> Key: old hive key: value: <> Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: <1000> Key: old hive key: value: <> Key: old hive key: value: Key: old hive key: value: <300s> Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: <60s> Key: old hive key: value: <4> Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: <> Key: old hive key: value: <> Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: Key: old hive key: value: <42> Key: old hive key: value: <3.141592654> Key: old hive key: value: <1s> Key: old hive key: value: <1s> Key: old hive key: value: <1s> Key: old hive key: value: Key: old hive key: value: <> Finished MetastoreConf object. 2018-01-05T11:28:29,814 DEBUG [main] datasource.HikariCPDataSourceProvider: Configuration requested hikaricp pooling, HikariCpDSProvider exiting 2018-01-05T11:28:29,814 DEBUG [main] datasource.BoneCPDataSourceProvider: Configuration requested hikaricp pooling, BoneCpDSProvider exiting 2018-01-05T11:28:30,228 DEBUG [main] hikari.HikariConfig: HikariPool-1 - configuration: 2018-01-05T11:28:30,232 DEBUG [main] hikari.HikariConfig: allowPoolSuspension.............false 2018-01-05T11:28:30,232 DEBUG [main] hikari.HikariConfig: autoCommit......................true 2018-01-05T11:28:30,232 DEBUG [main] hikari.HikariConfig: catalog.........................none 2018-01-05T11:28:30,232 DEBUG [main] hikari.HikariConfig: connectionInitSql...............none 2018-01-05T11:28:30,232 DEBUG [main] hikari.HikariConfig: connectionTestQuery.............none 2018-01-05T11:28:30,232 DEBUG [main] hikari.HikariConfig: connectionTimeout...............30000 2018-01-05T11:28:30,232 DEBUG [main] hikari.HikariConfig: dataSource......................none 2018-01-05T11:28:30,232 DEBUG [main] hikari.HikariConfig: dataSourceClassName.............none 2018-01-05T11:28:30,232 DEBUG [main] hikari.HikariConfig: dataSourceJNDI..................none 2018-01-05T11:28:30,233 DEBUG [main] hikari.HikariConfig: dataSourceProperties............{password=} 2018-01-05T11:28:30,233 DEBUG [main] hikari.HikariConfig: driverClassName................."org.apache.derby.jdbc.EmbeddedDriver" 2018-01-05T11:28:30,233 DEBUG [main] hikari.HikariConfig: healthCheckProperties...........{} 2018-01-05T11:28:30,233 DEBUG [main] hikari.HikariConfig: healthCheckRegistry.............none 2018-01-05T11:28:30,233 DEBUG [main] hikari.HikariConfig: idleTimeout.....................600000 2018-01-05T11:28:30,234 DEBUG [main] hikari.HikariConfig: initializationFailFast..........true 2018-01-05T11:28:30,234 DEBUG [main] hikari.HikariConfig: initializationFailTimeout.......1 2018-01-05T11:28:30,234 DEBUG [main] hikari.HikariConfig: isolateInternalQueries..........false 2018-01-05T11:28:30,234 DEBUG [main] hikari.HikariConfig: jdbc4ConnectionTest.............false 2018-01-05T11:28:30,234 DEBUG [main] hikari.HikariConfig: jdbcUrl........................."jdbc:derby:;databaseName=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db;create=true" 2018-01-05T11:28:30,234 DEBUG [main] hikari.HikariConfig: leakDetectionThreshold..........0 2018-01-05T11:28:30,234 DEBUG [main] hikari.HikariConfig: maxLifetime.....................1800000 2018-01-05T11:28:30,234 DEBUG [main] hikari.HikariConfig: maximumPoolSize.................10 2018-01-05T11:28:30,234 DEBUG [main] hikari.HikariConfig: metricRegistry..................none 2018-01-05T11:28:30,234 DEBUG [main] hikari.HikariConfig: metricsTrackerFactory...........none 2018-01-05T11:28:30,234 DEBUG [main] hikari.HikariConfig: minimumIdle.....................10 2018-01-05T11:28:30,234 DEBUG [main] hikari.HikariConfig: password........................ 2018-01-05T11:28:30,234 DEBUG [main] hikari.HikariConfig: poolName........................"HikariPool-1" 2018-01-05T11:28:30,234 DEBUG [main] hikari.HikariConfig: readOnly........................false 2018-01-05T11:28:30,234 DEBUG [main] hikari.HikariConfig: registerMbeans..................false 2018-01-05T11:28:30,234 DEBUG [main] hikari.HikariConfig: scheduledExecutor...............none 2018-01-05T11:28:30,234 DEBUG [main] hikari.HikariConfig: scheduledExecutorService........internal 2018-01-05T11:28:30,235 DEBUG [main] hikari.HikariConfig: threadFactory...................internal 2018-01-05T11:28:30,235 DEBUG [main] hikari.HikariConfig: transactionIsolation............default 2018-01-05T11:28:30,235 DEBUG [main] hikari.HikariConfig: username........................"APP" 2018-01-05T11:28:30,235 DEBUG [main] hikari.HikariConfig: validationTimeout...............5000 2018-01-05T11:28:30,235 INFO [main] hikari.HikariDataSource: HikariPool-1 - Starting... 2018-01-05T11:28:30,241 WARN [main] util.DriverDataSource: Registered driver with driverClassName=org.apache.derby.jdbc.EmbeddedDriver was not found, trying direct instantiation. 2018-01-05T11:28:30,688 INFO [main] pool.PoolBase: HikariPool-1 - Driver does not support get/set network timeout for connections. (Feature not implemented: No details.) 2018-01-05T11:28:30,689 DEBUG [main] pool.HikariPool: HikariPool-1 - Added connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:30,693 INFO [main] hikari.HikariDataSource: HikariPool-1 - Start completed. 2018-01-05T11:28:30,778 DEBUG [main] hikari.HikariConfig: HikariPool-2 - configuration: 2018-01-05T11:28:30,779 DEBUG [main] hikari.HikariConfig: allowPoolSuspension.............false 2018-01-05T11:28:30,779 DEBUG [main] hikari.HikariConfig: autoCommit......................true 2018-01-05T11:28:30,779 DEBUG [main] hikari.HikariConfig: catalog.........................none 2018-01-05T11:28:30,779 DEBUG [main] hikari.HikariConfig: connectionInitSql...............none 2018-01-05T11:28:30,779 DEBUG [main] hikari.HikariConfig: connectionTestQuery.............none 2018-01-05T11:28:30,779 DEBUG [main] hikari.HikariConfig: connectionTimeout...............30000 2018-01-05T11:28:30,779 DEBUG [main] hikari.HikariConfig: dataSource......................none 2018-01-05T11:28:30,779 DEBUG [main] hikari.HikariConfig: dataSourceClassName.............none 2018-01-05T11:28:30,779 DEBUG [main] hikari.HikariConfig: dataSourceJNDI..................none 2018-01-05T11:28:30,779 DEBUG [main] hikari.HikariConfig: dataSourceProperties............{password=} 2018-01-05T11:28:30,779 DEBUG [main] hikari.HikariConfig: driverClassName................."org.apache.derby.jdbc.EmbeddedDriver" 2018-01-05T11:28:30,779 DEBUG [main] hikari.HikariConfig: healthCheckProperties...........{} 2018-01-05T11:28:30,779 DEBUG [main] hikari.HikariConfig: healthCheckRegistry.............none 2018-01-05T11:28:30,780 DEBUG [main] hikari.HikariConfig: idleTimeout.....................600000 2018-01-05T11:28:30,780 DEBUG [main] hikari.HikariConfig: initializationFailFast..........true 2018-01-05T11:28:30,780 DEBUG [main] hikari.HikariConfig: initializationFailTimeout.......1 2018-01-05T11:28:30,780 DEBUG [main] hikari.HikariConfig: isolateInternalQueries..........false 2018-01-05T11:28:30,780 DEBUG [main] hikari.HikariConfig: jdbc4ConnectionTest.............false 2018-01-05T11:28:30,780 DEBUG [main] hikari.HikariConfig: jdbcUrl........................."jdbc:derby:;databaseName=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db;create=true" 2018-01-05T11:28:30,780 DEBUG [main] hikari.HikariConfig: leakDetectionThreshold..........0 2018-01-05T11:28:30,780 DEBUG [main] hikari.HikariConfig: maxLifetime.....................1800000 2018-01-05T11:28:30,780 DEBUG [main] hikari.HikariConfig: maximumPoolSize.................10 2018-01-05T11:28:30,780 DEBUG [main] hikari.HikariConfig: metricRegistry..................none 2018-01-05T11:28:30,780 DEBUG [main] hikari.HikariConfig: metricsTrackerFactory...........none 2018-01-05T11:28:30,780 DEBUG [main] hikari.HikariConfig: minimumIdle.....................10 2018-01-05T11:28:30,780 DEBUG [main] hikari.HikariConfig: password........................ 2018-01-05T11:28:30,780 DEBUG [main] hikari.HikariConfig: poolName........................"HikariPool-2" 2018-01-05T11:28:30,780 DEBUG [main] hikari.HikariConfig: readOnly........................false 2018-01-05T11:28:30,781 DEBUG [main] hikari.HikariConfig: registerMbeans..................false 2018-01-05T11:28:30,781 DEBUG [main] hikari.HikariConfig: scheduledExecutor...............none 2018-01-05T11:28:30,781 DEBUG [main] hikari.HikariConfig: scheduledExecutorService........internal 2018-01-05T11:28:30,781 DEBUG [main] hikari.HikariConfig: threadFactory...................internal 2018-01-05T11:28:30,781 DEBUG [main] hikari.HikariConfig: transactionIsolation............default 2018-01-05T11:28:30,781 DEBUG [main] hikari.HikariConfig: username........................"APP" 2018-01-05T11:28:30,781 DEBUG [main] hikari.HikariConfig: validationTimeout...............5000 2018-01-05T11:28:30,781 INFO [main] hikari.HikariDataSource: HikariPool-2 - Starting... 2018-01-05T11:28:30,781 WARN [main] util.DriverDataSource: Registered driver with driverClassName=org.apache.derby.jdbc.EmbeddedDriver was not found, trying direct instantiation. 2018-01-05T11:28:30,786 INFO [main] pool.PoolBase: HikariPool-2 - Driver does not support get/set network timeout for connections. (Feature not implemented: No details.) 2018-01-05T11:28:30,787 DEBUG [main] pool.HikariPool: HikariPool-2 - Added connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:30,787 INFO [main] hikari.HikariDataSource: HikariPool-2 - Start completed. 2018-01-05T11:28:30,795 DEBUG [HikariPool-1 housekeeper] pool.HikariPool: HikariPool-1 - Pool stats (total=1, active=0, idle=1, waiting=0) 2018-01-05T11:28:30,799 DEBUG [HikariPool-1 connection adder] pool.HikariPool: HikariPool-1 - Added connection org.apache.derby.impl.jdbc.EmbedConnection@1266554026 (XID = 171), (SESSIONID = 5), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:30,801 DEBUG [HikariPool-1 connection adder] pool.HikariPool: HikariPool-1 - Added connection org.apache.derby.impl.jdbc.EmbedConnection@172170897 (XID = 173), (SESSIONID = 7), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:30,803 DEBUG [HikariPool-1 connection adder] pool.HikariPool: HikariPool-1 - Added connection org.apache.derby.impl.jdbc.EmbedConnection@990064844 (XID = 175), (SESSIONID = 9), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:30,806 DEBUG [HikariPool-1 connection adder] pool.HikariPool: HikariPool-1 - Added connection org.apache.derby.impl.jdbc.EmbedConnection@1785520606 (XID = 177), (SESSIONID = 11), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:30,808 DEBUG [HikariPool-1 connection adder] pool.HikariPool: HikariPool-1 - Added connection org.apache.derby.impl.jdbc.EmbedConnection@1431106067 (XID = 179), (SESSIONID = 13), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:30,809 DEBUG [HikariPool-1 connection adder] pool.HikariPool: HikariPool-1 - Added connection org.apache.derby.impl.jdbc.EmbedConnection@1847754515 (XID = 181), (SESSIONID = 15), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:30,811 DEBUG [HikariPool-1 connection adder] pool.HikariPool: HikariPool-1 - Added connection org.apache.derby.impl.jdbc.EmbedConnection@1925338592 (XID = 183), (SESSIONID = 17), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:30,813 DEBUG [HikariPool-1 connection adder] pool.HikariPool: HikariPool-1 - Added connection org.apache.derby.impl.jdbc.EmbedConnection@77990385 (XID = 185), (SESSIONID = 19), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:30,814 DEBUG [HikariPool-1 connection adder] pool.HikariPool: HikariPool-1 - Added connection org.apache.derby.impl.jdbc.EmbedConnection@1132252713 (XID = 187), (SESSIONID = 21), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:30,814 DEBUG [HikariPool-1 connection adder] pool.HikariPool: HikariPool-1 - After adding stats (total=10, active=0, idle=10, waiting=0) 2018-01-05T11:28:30,889 DEBUG [HikariPool-2 housekeeper] pool.HikariPool: HikariPool-2 - Pool stats (total=1, active=1, idle=0, waiting=0) 2018-01-05T11:28:30,890 DEBUG [HikariPool-2 connection adder] pool.HikariPool: HikariPool-2 - Added connection org.apache.derby.impl.jdbc.EmbedConnection@106487650 (XID = 190), (SESSIONID = 23), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:30,892 DEBUG [HikariPool-2 connection adder] pool.HikariPool: HikariPool-2 - Added connection org.apache.derby.impl.jdbc.EmbedConnection@1197856089 (XID = 192), (SESSIONID = 25), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:30,893 DEBUG [HikariPool-2 connection adder] pool.HikariPool: HikariPool-2 - Added connection org.apache.derby.impl.jdbc.EmbedConnection@1986564320 (XID = 194), (SESSIONID = 27), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:30,894 DEBUG [HikariPool-2 connection adder] pool.HikariPool: HikariPool-2 - Added connection org.apache.derby.impl.jdbc.EmbedConnection@659171862 (XID = 196), (SESSIONID = 29), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:30,895 DEBUG [HikariPool-2 connection adder] pool.HikariPool: HikariPool-2 - Added connection org.apache.derby.impl.jdbc.EmbedConnection@2090933574 (XID = 198), (SESSIONID = 31), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:30,896 DEBUG [HikariPool-2 connection adder] pool.HikariPool: HikariPool-2 - Added connection org.apache.derby.impl.jdbc.EmbedConnection@587526645 (XID = 200), (SESSIONID = 33), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:30,897 DEBUG [HikariPool-2 connection adder] pool.HikariPool: HikariPool-2 - Added connection org.apache.derby.impl.jdbc.EmbedConnection@2050327269 (XID = 202), (SESSIONID = 35), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:30,898 DEBUG [HikariPool-2 connection adder] pool.HikariPool: HikariPool-2 - Added connection org.apache.derby.impl.jdbc.EmbedConnection@1754971906 (XID = 204), (SESSIONID = 37), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:30,900 DEBUG [HikariPool-2 connection adder] pool.HikariPool: HikariPool-2 - Added connection org.apache.derby.impl.jdbc.EmbedConnection@1729593776 (XID = 206), (SESSIONID = 39), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:30,900 DEBUG [HikariPool-2 connection adder] pool.HikariPool: HikariPool-2 - After adding stats (total=10, active=1, idle=9, waiting=0) 2018-01-05T11:28:31,111 INFO [main] metastore.ObjectStore: Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order" 2018-01-05T11:28:31,144 DEBUG [main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:31,146 DEBUG [main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:31,149 DEBUG [main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:34,295 DEBUG [main] pool.PoolBase: HikariPool-2 - Reset (autoCommit, isolation) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:34,984 DEBUG [main] pool.PoolBase: HikariPool-2 - Reset (autoCommit, isolation) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:35,121 DEBUG [main] pool.PoolBase: HikariPool-2 - Reset (autoCommit, isolation) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:35,200 DEBUG [main] pool.PoolBase: HikariPool-2 - Reset (autoCommit, isolation) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:35,226 DEBUG [main] pool.PoolBase: HikariPool-2 - Reset (autoCommit, isolation) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:35,248 DEBUG [main] pool.PoolBase: HikariPool-2 - Reset (autoCommit, isolation) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:35,401 DEBUG [main] pool.PoolBase: HikariPool-2 - Reset (autoCommit, isolation) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:35,405 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:35,414 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:35,415 INFO [main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:35,417 INFO [main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:35,484 DEBUG [main] pool.PoolBase: HikariPool-2 - Reset (autoCommit, isolation) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:35,487 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:35,490 DEBUG [main] metastore.MetaStoreSchemaInfoFactory: HIVE_HOME is not set. Using current directory instead 2018-01-05T11:28:35,495 WARN [main] metastore.ObjectStore: Version information not found in metastore. metastore.schema.verification is not enabled so recording the schema version 3.0.0 2018-01-05T11:28:35,495 WARN [main] metastore.ObjectStore: setMetaStoreSchemaVersion called but recording version is disabled: version = 3.0.0, comment = Set by MetaStore djaiswal@10.22.8.182 2018-01-05T11:28:35,499 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:35,499 WARN [main] metastore.ObjectStore: Failed to get database default, returning NoSuchObjectException 2018-01-05T11:28:35,562 DEBUG [main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:35,579 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:35,589 DEBUG [main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:35,617 DEBUG [main] pool.PoolBase: HikariPool-2 - Reset (autoCommit, isolation) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:35,633 DEBUG [main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:35,636 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:35,636 INFO [main] metastore.HiveMetaStore: Added admin role in metastore 2018-01-05T11:28:35,637 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:35,637 INFO [main] metastore.HiveMetaStore: Added public role in metastore 2018-01-05T11:28:35,669 DEBUG [main] pool.PoolBase: HikariPool-2 - Reset (autoCommit, isolation) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:35,684 DEBUG [main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:35,688 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:35,688 INFO [main] metastore.HiveMetaStore: No user is added in admin role, since config is empty 2018-01-05T11:28:35,693 DEBUG [main] metastore.ThreadPool: ThreadPool initialized 2018-01-05T11:28:35,701 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:28:35,802 INFO [main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:35,815 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:28:35,817 INFO [main] metastore.HiveMetaStore: 0: get_all_functions 2018-01-05T11:28:35,819 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_all_functions 2018-01-05T11:28:35,875 DEBUG [main] pool.PoolBase: HikariPool-2 - Reset (autoCommit, isolation) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:35,877 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:35,878 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:28:35,926 INFO [main] control.CoreCliDriver: QtestUtil instance created ElapsedTime(ms)=9385 2018-01-05T11:28:35,927 DEBUG [main] session.SessionState: SessionState user: null 2018-01-05T11:28:35,928 INFO [main] DependencyResolver: ivysettings.xml file not found in HIVE_HOME or HIVE_CONF_DIR,/Users/djaiswal/hive/hive/conf/ivysettings.xml will be used 2018-01-05T11:28:35,930 DEBUG [main] session.SessionState: Removing resource dir /Users/djaiswal/hive/hive/itests/qtest/target/tmp/53565c10-a9a1-4685-81b4-afcf82ce77fb_resources 2018-01-05T11:28:35,931 INFO [main] session.SessionState: Deleted directory: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/53565c10-a9a1-4685-81b4-afcf82ce77fb on fs with scheme file 2018-01-05T11:28:35,931 INFO [main] session.SessionState: Deleted directory: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb on fs with scheme file 2018-01-05T11:28:35,934 DEBUG [main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:35,935 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:28:35,935 INFO [main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:35,935 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:35,935 INFO [main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:35,935 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:35,935 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:28:35,952 DEBUG [main] session.SessionState: HDFS root scratch dir: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir with schema null, permission: rwx-wx-wx 2018-01-05T11:28:35,965 INFO [main] session.SessionState: Created local directory: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/53565c10-a9a1-4685-81b4-afcf82ce77fb_resources 2018-01-05T11:28:35,981 INFO [main] session.SessionState: Created HDFS directory: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:35,995 INFO [main] session.SessionState: Created local directory: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:36,007 INFO [main] session.SessionState: Created HDFS directory: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/53565c10-a9a1-4685-81b4-afcf82ce77fb/_tmp_space.db 2018-01-05T11:28:36,009 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:28:36,010 INFO [main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:36,011 WARN [main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:36,011 WARN [main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:36,011 INFO [main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:36,011 DEBUG [main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:36,011 DEBUG [main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:36,012 DEBUG [main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:36,017 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:36,019 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:36,019 INFO [main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:36,019 INFO [main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:36,020 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:28:36,020 INFO [main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:36,020 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:28:36,020 INFO [main] metastore.HiveMetaStore: 0: get_all_databases 2018-01-05T11:28:36,020 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_all_databases 2018-01-05T11:28:36,024 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:36,024 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:28:36,024 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:28:36,024 INFO [main] metastore.HiveMetaStore: 0: get_tables: db=default pat=.* 2018-01-05T11:28:36,024 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_tables: db=default pat=.* 2018-01-05T11:28:36,032 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:36,032 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:28:36,032 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:28:36,035 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:36,036 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:28:36,036 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:28:36,036 INFO [main] metastore.HiveMetaStore: 0: get_functions: db=default pat=.* 2018-01-05T11:28:36,036 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_functions: db=default pat=.* 2018-01-05T11:28:36,050 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:36,050 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:28:36,052 INFO [main] QTestUtil: Cleanup (/Users/djaiswal/hive/hive/itests/qtest/../..//data/scripts/q_test_cleanup.sql): DROP TABLE IF EXISTS src; DROP TABLE IF EXISTS src1; DROP TABLE IF EXISTS src_json; DROP TABLE IF EXISTS src_sequencefile; DROP TABLE IF EXISTS src_thrift; DROP TABLE IF EXISTS srcbucket; DROP TABLE IF EXISTS srcbucket2; DROP TABLE IF EXISTS srcpart; DROP TABLE IF EXISTS primitives; DROP TABLE IF EXISTS dest1; DROP TABLE IF EXISTS dest2; DROP TABLE IF EXISTS dest3; DROP TABLE IF EXISTS dest4; DROP TABLE IF EXISTS dest4_sequencefile; DROP TABLE IF EXISTS dest_j1; DROP TABLE IF EXISTS dest_g1; DROP TABLE IF EXISTS dest_g2; DROP TABLE IF EXISTS fetchtask_ioexception; DROP TABLE IF EXISTS alltypesorc; DROP TABLE IF EXISTS alltypesparquet; DROP TABLE IF EXISTS cbo_t1; DROP TABLE IF EXISTS cbo_t2; DROP TABLE IF EXISTS cbo_t3; DROP TABLE IF EXISTS src_cbo; DROP TABLE IF EXISTS part; DROP TABLE IF EXISTS lineitem; 2018-01-05T11:28:36,056 DEBUG [main] CliDriver: CliDriver inited with classpath /Users/djaiswal/hive/hive/itests/qtest/target/test-classes:/Users/djaiswal/hive/hive/itests/qtest/target/classes:/Users/djaiswal/.m2/repository/org/apache/hive/hive-common/3.0.0-SNAPSHOT/hive-common-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-classification/3.0.0-SNAPSHOT/hive-classification-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-shims/3.0.0-SNAPSHOT/hive-shims-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/shims/hive-shims-common/3.0.0-SNAPSHOT/hive-shims-common-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/shims/hive-shims-0.23/3.0.0-SNAPSHOT/hive-shims-0.23-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/shims/hive-shims-scheduler/3.0.0-SNAPSHOT/hive-shims-scheduler-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-storage-api/3.0.0-SNAPSHOT/hive-storage-api-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/commons-cli/commons-cli/1.2/commons-cli-1.2.jar:/Users/djaiswal/.m2/repository/commons-lang/commons-lang/2.6/commons-lang-2.6.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-lang3/3.2/commons-lang3-3.2.jar:/Users/djaiswal/.m2/repository/org/apache/orc/orc-core/1.4.1/orc-core-1.4.1.jar:/Users/djaiswal/.m2/repository/io/airlift/aircompressor/0.8/aircompressor-0.8.jar:/Users/djaiswal/.m2/repository/io/airlift/slice/0.29/slice-0.29.jar:/Users/djaiswal/.m2/repository/org/openjdk/jol/jol-core/0.2/jol-core-0.2.jar:/Users/djaiswal/.m2/repository/jline/jline/2.12/jline-2.12.jar:/Users/djaiswal/.m2/repository/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-rewrite/9.3.8.v20160314/jetty-rewrite-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-client/9.3.8.v20160314/jetty-client-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-server/9.3.8.v20160314/jetty-server-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-http/9.3.8.v20160314/jetty-http-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-io/9.3.8.v20160314/jetty-io-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-servlet/9.3.8.v20160314/jetty-servlet-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-webapp/9.3.8.v20160314/jetty-webapp-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-xml/9.3.8.v20160314/jetty-xml-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/joda-time/joda-time/2.8.1/joda-time-2.8.1.jar:/Users/djaiswal/.m2/repository/org/apache/logging/log4j/log4j-1.2-api/2.8.2/log4j-1.2-api-2.8.2.jar:/Users/djaiswal/.m2/repository/org/apache/logging/log4j/log4j-web/2.8.2/log4j-web-2.8.2.jar:/Users/djaiswal/.m2/repository/org/apache/logging/log4j/log4j-slf4j-impl/2.8.2/log4j-slf4j-impl-2.8.2.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-compress/1.9/commons-compress-1.9.jar:/Users/djaiswal/.m2/repository/org/apache/ant/ant/1.9.1/ant-1.9.1.jar:/Users/djaiswal/.m2/repository/org/apache/ant/ant-launcher/1.9.1/ant-launcher-1.9.1.jar:/Users/djaiswal/.m2/repository/com/tdunning/json/1.8/json-1.8.jar:/Users/djaiswal/.m2/repository/io/dropwizard/metrics/metrics-core/3.1.0/metrics-core-3.1.0.jar:/Users/djaiswal/.m2/repository/io/dropwizard/metrics/metrics-jvm/3.1.0/metrics-jvm-3.1.0.jar:/Users/djaiswal/.m2/repository/io/dropwizard/metrics/metrics-json/3.1.0/metrics-json-3.1.0.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/core/jackson-databind/2.6.5/jackson-databind-2.6.5.jar:/Users/djaiswal/.m2/repository/com/github/joshelser/dropwizard-metrics-hadoop-metrics2-reporter/0.1.2/dropwizard-metrics-hadoop-metrics2-reporter-0.1.2.jar:/Users/djaiswal/.m2/repository/javolution/javolution/5.5.1/javolution-5.5.1.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-contrib/3.0.0-SNAPSHOT/hive-contrib-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/commons-codec/commons-codec/1.4/commons-codec-1.4.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-standalone-metastore/3.0.0-SNAPSHOT/hive-standalone-metastore-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/com/google/guava/guava/19.0/guava-19.0.jar:/Users/djaiswal/.m2/repository/com/google/protobuf/protobuf-java/2.5.0/protobuf-java-2.5.0.jar:/Users/djaiswal/.m2/repository/com/jolbox/bonecp/0.8.0.RELEASE/bonecp-0.8.0.RELEASE.jar:/Users/djaiswal/.m2/repository/com/zaxxer/HikariCP/2.6.1/HikariCP-2.6.1.jar:/Users/djaiswal/.m2/repository/commons-dbcp/commons-dbcp/1.4/commons-dbcp-1.4.jar:/Users/djaiswal/.m2/repository/commons-pool/commons-pool/1.5.4/commons-pool-1.5.4.jar:/Users/djaiswal/.m2/repository/org/antlr/antlr-runtime/3.5.2/antlr-runtime-3.5.2.jar:/Users/djaiswal/.m2/repository/org/apache/derby/derby/10.11.1.1/derby-10.11.1.1.jar:/Users/djaiswal/.m2/repository/org/apache/thrift/libfb303/0.9.3/libfb303-0.9.3.jar:/Users/djaiswal/.m2/repository/org/apache/thrift/libthrift/0.9.3/libthrift-0.9.3.jar:/Users/djaiswal/.m2/repository/org/apache/httpcomponents/httpcore/4.4.4/httpcore-4.4.4.jar:/Users/djaiswal/.m2/repository/org/datanucleus/datanucleus-api-jdo/4.2.4/datanucleus-api-jdo-4.2.4.jar:/Users/djaiswal/.m2/repository/org/datanucleus/datanucleus-core/4.1.17/datanucleus-core-4.1.17.jar:/Users/djaiswal/.m2/repository/org/datanucleus/datanucleus-rdbms/4.1.19/datanucleus-rdbms-4.1.19.jar:/Users/djaiswal/.m2/repository/org/datanucleus/javax.jdo/3.2.0-m3/javax.jdo-3.2.0-m3.jar:/Users/djaiswal/.m2/repository/javax/transaction/transaction-api/1.1/transaction-api-1.1.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-standalone-metastore/3.0.0-SNAPSHOT/hive-standalone-metastore-3.0.0-SNAPSHOT-tests.jar:/Users/djaiswal/hive/hive/itests/custom-serde/target/classes:/Users/djaiswal/hive/hive/itests/hive-unit/target/test-classes:/Users/djaiswal/.m2/repository/org/apache/hive/hive-jdbc/3.0.0-SNAPSHOT/hive-jdbc-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-service/3.0.0-SNAPSHOT/hive-service-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/javax/servlet/jsp/javax.servlet.jsp-api/2.3.1/javax.servlet.jsp-api-2.3.1.jar:/Users/djaiswal/.m2/repository/net/sf/jpam/jpam/1.1/jpam-1.1.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-runner/9.3.8.v20160314/jetty-runner-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-plus/9.3.8.v20160314/jetty-plus-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-annotations/9.3.8.v20160314/jetty-annotations-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-jaas/9.3.8.v20160314/jetty-jaas-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/websocket/websocket-server/9.3.8.v20160314/websocket-server-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/websocket/websocket-common/9.3.8.v20160314/websocket-common-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/websocket/websocket-api/9.3.8.v20160314/websocket-api-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/websocket/websocket-client/9.3.8.v20160314/websocket-client-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/websocket/websocket-servlet/9.3.8.v20160314/websocket-servlet-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-jndi/9.3.8.v20160314/jetty-jndi-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/apache-jsp/9.3.8.v20160314/apache-jsp-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/toolchain/jetty-schemas/3.1/jetty-schemas-3.1.jar:/Users/djaiswal/.m2/repository/org/eclipse/jdt/core/compiler/ecj/4.4.2/ecj-4.4.2.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/apache-jstl/9.3.8.v20160314/apache-jstl-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/apache/taglibs/taglibs-standard-spec/1.2.5/taglibs-standard-spec-1.2.5.jar:/Users/djaiswal/.m2/repository/org/apache/taglibs/taglibs-standard-impl/1.2.5/taglibs-standard-impl-1.2.5.jar:/Users/djaiswal/.m2/repository/tomcat/jasper-compiler/5.5.23/jasper-compiler-5.5.23.jar:/Users/djaiswal/.m2/repository/javax/servlet/jsp-api/2.0/jsp-api-2.0.jar:/Users/djaiswal/.m2/repository/ant/ant/1.6.5/ant-1.6.5.jar:/Users/djaiswal/.m2/repository/tomcat/jasper-runtime/5.5.23/jasper-runtime-5.5.23.jar:/Users/djaiswal/.m2/repository/commons-el/commons-el/1.0/commons-el-1.0.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-llap-ext-client/3.0.0-SNAPSHOT/hive-llap-ext-client-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hcatalog/hive-hcatalog-core/3.0.0-SNAPSHOT/hive-hcatalog-core-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.9.13/jackson-mapper-asl-1.9.13.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hcatalog/hive-hcatalog-streaming/3.0.0-SNAPSHOT/hive-hcatalog-streaming-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hcatalog/hive-hcatalog-server-extensions/3.0.0-SNAPSHOT/hive-hcatalog-server-extensions-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/javax/jms/jms/1.1/jms-1.1.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hcatalog/hive-webhcat-java-client/3.0.0-SNAPSHOT/hive-webhcat-java-client-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-service/3.0.0-SNAPSHOT/hive-service-3.0.0-SNAPSHOT-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-exec/3.0.0-SNAPSHOT/hive-exec-3.0.0-SNAPSHOT-tests.jar:/Users/djaiswal/hive/hive/itests/util/target/classes:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-minicluster/1.7.3/accumulo-minicluster-1.7.3.jar:/Users/djaiswal/.m2/repository/com/beust/jcommander/1.32/jcommander-1.32.jar:/Users/djaiswal/.m2/repository/commons-configuration/commons-configuration/1.6/commons-configuration-1.6.jar:/Users/djaiswal/.m2/repository/commons-digester/commons-digester/1.8/commons-digester-1.8.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-core/1.7.3/accumulo-core-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-math/2.1/commons-math-2.1.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-gc/1.7.3/accumulo-gc-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-master/1.7.3/accumulo-master-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-monitor/1.7.3/accumulo-monitor-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-shell/1.7.3/accumulo-shell-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-server-base/1.7.3/accumulo-server-base-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-start/1.7.3/accumulo-start-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-tracer/1.7.3/accumulo-tracer-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-tserver/1.7.3/accumulo-tserver-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-vfs2/2.1/commons-vfs2-2.1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-minicluster/3.0.0-beta1/hadoop-minicluster-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-accumulo-handler/3.0.0-SNAPSHOT/hive-accumulo-handler-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-fate/1.7.3/accumulo-fate-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-trace/1.7.3/accumulo-trace-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-accumulo-handler/3.0.0-SNAPSHOT/hive-accumulo-handler-3.0.0-SNAPSHOT-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-cli/3.0.0-SNAPSHOT/hive-cli-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-beeline/3.0.0-SNAPSHOT/hive-beeline-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/net/sf/supercsv/super-csv/2.2.0/super-csv-2.2.0.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-hbase-handler/3.0.0-SNAPSHOT/hive-hbase-handler-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-hbase-handler/3.0.0-SNAPSHOT/hive-hbase-handler-3.0.0-SNAPSHOT-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-metastore/3.0.0-SNAPSHOT/hive-metastore-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/javax/jdo/jdo-api/3.0.1/jdo-api-3.0.1.jar:/Users/djaiswal/.m2/repository/javax/transaction/jta/1.1/jta-1.1.jar:/Users/djaiswal/.m2/repository/co/cask/tephra/tephra-api/0.6.0/tephra-api-0.6.0.jar:/Users/djaiswal/.m2/repository/co/cask/tephra/tephra-core/0.6.0/tephra-core-0.6.0.jar:/Users/djaiswal/.m2/repository/com/google/inject/extensions/guice-assistedinject/3.0/guice-assistedinject-3.0.jar:/Users/djaiswal/.m2/repository/org/apache/twill/twill-common/0.6.0-incubating/twill-common-0.6.0-incubating.jar:/Users/djaiswal/.m2/repository/org/apache/twill/twill-core/0.6.0-incubating/twill-core-0.6.0-incubating.jar:/Users/djaiswal/.m2/repository/org/apache/twill/twill-api/0.6.0-incubating/twill-api-0.6.0-incubating.jar:/Users/djaiswal/.m2/repository/org/apache/twill/twill-discovery-api/0.6.0-incubating/twill-discovery-api-0.6.0-incubating.jar:/Users/djaiswal/.m2/repository/org/apache/twill/twill-discovery-core/0.6.0-incubating/twill-discovery-core-0.6.0-incubating.jar:/Users/djaiswal/.m2/repository/org/apache/twill/twill-zookeeper/0.6.0-incubating/twill-zookeeper-0.6.0-incubating.jar:/Users/djaiswal/.m2/repository/co/cask/tephra/tephra-hbase-compat-1.0/0.6.0/tephra-hbase-compat-1.0-0.6.0.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-client/2.0.0-alpha4/hbase-client-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/jruby/jcodings/jcodings/1.0.18/jcodings-1.0.18.jar:/Users/djaiswal/.m2/repository/org/jruby/joni/joni/2.1.11/joni-2.1.11.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-mapreduce/2.0.0-alpha4/hbase-mapreduce-2.0.0-alpha4-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-serde/3.0.0-SNAPSHOT/hive-serde-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-service-rpc/3.0.0-SNAPSHOT/hive-service-rpc-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/com/google/code/findbugs/jsr305/3.0.0/jsr305-3.0.0.jar:/Users/djaiswal/.m2/repository/org/apache/avro/avro/1.7.7/avro-1.7.7.jar:/Users/djaiswal/.m2/repository/org/codehaus/jackson/jackson-core-asl/1.9.13/jackson-core-asl-1.9.13.jar:/Users/djaiswal/.m2/repository/com/thoughtworks/paranamer/paranamer/2.3/paranamer-2.3.jar:/Users/djaiswal/.m2/repository/org/xerial/snappy/snappy-java/1.1.4/snappy-java-1.1.4.jar:/Users/djaiswal/.m2/repository/net/sf/opencsv/opencsv/2.3/opencsv-2.3.jar:/Users/djaiswal/.m2/repository/org/apache/parquet/parquet-hadoop-bundle/1.9.0/parquet-hadoop-bundle-1.9.0.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-exec/3.0.0-SNAPSHOT/hive-exec-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-vector-code-gen/3.0.0-SNAPSHOT/hive-vector-code-gen-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/velocity/velocity/1.5/velocity-1.5.jar:/Users/djaiswal/.m2/repository/oro/oro/2.0.8/oro-2.0.8.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-llap-tez/3.0.0-SNAPSHOT/hive-llap-tez-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/commons-httpclient/commons-httpclient/3.0.1/commons-httpclient-3.0.1.jar:/Users/djaiswal/.m2/repository/commons-io/commons-io/2.4/commons-io-2.4.jar:/Users/djaiswal/.m2/repository/org/antlr/ST4/4.0.4/ST4-4.0.4.jar:/Users/djaiswal/.m2/repository/org/apache/ivy/ivy/2.4.0/ivy-2.4.0.jar:/Users/djaiswal/.m2/repository/org/apache/zookeeper/zookeeper/3.4.6/zookeeper-3.4.6.jar:/Users/djaiswal/.m2/repository/org/apache/curator/curator-framework/2.12.0/curator-framework-2.12.0.jar:/Users/djaiswal/.m2/repository/org/codehaus/groovy/groovy-all/2.4.11/groovy-all-2.4.11.jar:/Users/djaiswal/.m2/repository/org/apache/calcite/calcite-core/1.15.0/calcite-core-1.15.0.jar:/Users/djaiswal/.m2/repository/org/apache/calcite/calcite-linq4j/1.15.0/calcite-linq4j-1.15.0.jar:/Users/djaiswal/.m2/repository/com/esri/geometry/esri-geometry-api/2.0.0/esri-geometry-api-2.0.0.jar:/Users/djaiswal/.m2/repository/com/yahoo/datasketches/sketches-core/0.9.0/sketches-core-0.9.0.jar:/Users/djaiswal/.m2/repository/com/yahoo/datasketches/memory/0.9.0/memory-0.9.0.jar:/Users/djaiswal/.m2/repository/org/codehaus/janino/janino/2.7.6/janino-2.7.6.jar:/Users/djaiswal/.m2/repository/org/codehaus/janino/commons-compiler/2.7.6/commons-compiler-2.7.6.jar:/Users/djaiswal/.m2/repository/org/apache/calcite/calcite-druid/1.15.0/calcite-druid-1.15.0.jar:/Users/djaiswal/.m2/repository/org/apache/calcite/avatica/avatica/1.10.0/avatica-1.10.0.jar:/Users/djaiswal/.m2/repository/com/google/code/gson/gson/2.2.4/gson-2.2.4.jar:/Users/djaiswal/.m2/repository/stax/stax-api/1.0.1/stax-api-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-registry/3.0.0-beta1/hadoop-yarn-registry-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-druid-handler/3.0.0-SNAPSHOT/hive-druid-handler-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/com/metamx/java-util/0.27.10/java-util-0.27.10.jar:/Users/djaiswal/.m2/repository/org/skife/config/config-magic/0.9/config-magic-0.9.jar:/Users/djaiswal/.m2/repository/org/mozilla/rhino/1.7R5/rhino-1.7R5.jar:/Users/djaiswal/.m2/repository/com/jayway/jsonpath/json-path/2.1.0/json-path-2.1.0.jar:/Users/djaiswal/.m2/repository/io/druid/druid-server/0.10.1/druid-server-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/druid-console/0.0.2/druid-console-0.0.2.jar:/Users/djaiswal/.m2/repository/com/metamx/http-client/1.1.0/http-client-1.1.0.jar:/Users/djaiswal/.m2/repository/com/metamx/server-metrics/0.2.8/server-metrics-0.2.8.jar:/Users/djaiswal/.m2/repository/javax/inject/javax.inject/1/javax.inject-1.jar:/Users/djaiswal/.m2/repository/org/glassfish/javax.el/3.0.0/javax.el-3.0.0.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/jaxrs/jackson-jaxrs-smile-provider/2.4.6/jackson-jaxrs-smile-provider-2.4.6.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/dataformat/jackson-dataformat-smile/2.4.6/jackson-dataformat-smile-2.4.6.jar:/Users/djaiswal/.m2/repository/io/tesla/aether/tesla-aether/0.0.5/tesla-aether-0.0.5.jar:/Users/djaiswal/.m2/repository/org/eclipse/aether/aether-spi/0.9.0.M2/aether-spi-0.9.0.M2.jar:/Users/djaiswal/.m2/repository/org/eclipse/aether/aether-util/0.9.0.M2/aether-util-0.9.0.M2.jar:/Users/djaiswal/.m2/repository/org/eclipse/aether/aether-impl/0.9.0.M2/aether-impl-0.9.0.M2.jar:/Users/djaiswal/.m2/repository/org/eclipse/aether/aether-connector-file/0.9.0.M2/aether-connector-file-0.9.0.M2.jar:/Users/djaiswal/.m2/repository/io/tesla/aether/aether-connector-okhttp/0.0.9/aether-connector-okhttp-0.0.9.jar:/Users/djaiswal/.m2/repository/org/apache/maven/wagon/wagon-provider-api/2.4/wagon-provider-api-2.4.jar:/Users/djaiswal/.m2/repository/org/apache/maven/maven-aether-provider/3.1.1/maven-aether-provider-3.1.1.jar:/Users/djaiswal/.m2/repository/org/apache/maven/maven-model/3.1.1/maven-model-3.1.1.jar:/Users/djaiswal/.m2/repository/org/apache/maven/maven-model-builder/3.1.1/maven-model-builder-3.1.1.jar:/Users/djaiswal/.m2/repository/org/apache/maven/maven-repository-metadata/3.1.1/maven-repository-metadata-3.1.1.jar:/Users/djaiswal/.m2/repository/org/codehaus/plexus/plexus-utils/3.0.15/plexus-utils-3.0.15.jar:/Users/djaiswal/.m2/repository/org/apache/maven/maven-settings-builder/3.1.1/maven-settings-builder-3.1.1.jar:/Users/djaiswal/.m2/repository/org/codehaus/plexus/plexus-interpolation/1.19/plexus-interpolation-1.19.jar:/Users/djaiswal/.m2/repository/org/apache/maven/maven-settings/3.1.1/maven-settings-3.1.1.jar:/Users/djaiswal/.m2/repository/org/eclipse/aether/aether-api/0.9.0.M2/aether-api-0.9.0.M2.jar:/Users/djaiswal/.m2/repository/net/spy/spymemcached/2.12.3/spymemcached-2.12.3.jar:/Users/djaiswal/.m2/repository/net/jpountz/lz4/lz4/1.3.0/lz4-1.3.0.jar:/Users/djaiswal/.m2/repository/com/ircclouds/irc/irc-api/1.0-0014/irc-api-1.0-0014.jar:/Users/djaiswal/.m2/repository/com/maxmind/geoip2/geoip2/0.4.0/geoip2-0.4.0.jar:/Users/djaiswal/.m2/repository/com/maxmind/maxminddb/maxminddb/0.2.0/maxminddb-0.2.0.jar:/Users/djaiswal/.m2/repository/org/apache/derby/derbynet/10.11.1.1/derbynet-10.11.1.1.jar:/Users/djaiswal/.m2/repository/org/apache/derby/derbyclient/10.11.1.1/derbyclient-10.11.1.1.jar:/Users/djaiswal/.m2/repository/it/unimi/dsi/fastutil/7.2.0/fastutil-7.2.0.jar:/Users/djaiswal/.m2/repository/io/druid/druid-processing/0.10.1/druid-processing-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/druid-common/0.10.1/druid-common-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/java-util/0.10.1/java-util-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/druid-api/0.10.1/druid-api-0.10.1.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-dbcp2/2.0.1/commons-dbcp2-2.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-pool2/2.2/commons-pool2-2.2.jar:/Users/djaiswal/.m2/repository/org/hibernate/hibernate-validator/5.1.3.Final/hibernate-validator-5.1.3.Final.jar:/Users/djaiswal/.m2/repository/org/jboss/logging/jboss-logging/3.1.3.GA/jboss-logging-3.1.3.GA.jar:/Users/djaiswal/.m2/repository/com/fasterxml/classmate/1.0.0/classmate-1.0.0.jar:/Users/djaiswal/.m2/repository/javax/el/javax.el-api/3.0.0/javax.el-api-3.0.0.jar:/Users/djaiswal/.m2/repository/javax/validation/validation-api/1.1.0.Final/validation-api-1.1.0.Final.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/datatype/jackson-datatype-guava/2.4.6/jackson-datatype-guava-2.4.6.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/datatype/jackson-datatype-joda/2.4.6/jackson-datatype-joda-2.4.6.jar:/Users/djaiswal/.m2/repository/com/google/inject/extensions/guice-multibindings/4.1.0/guice-multibindings-4.1.0.jar:/Users/djaiswal/.m2/repository/org/jdbi/jdbi/2.63.1/jdbi-2.63.1.jar:/Users/djaiswal/.m2/repository/org/apache/logging/log4j/log4j-jul/2.5/log4j-jul-2.5.jar:/Users/djaiswal/.m2/repository/org/slf4j/jcl-over-slf4j/1.7.12/jcl-over-slf4j-1.7.12.jar:/Users/djaiswal/.m2/repository/net/java/dev/jets3t/jets3t/0.9.4/jets3t-0.9.4.jar:/Users/djaiswal/.m2/repository/javax/activation/activation/1.1.1/activation-1.1.1.jar:/Users/djaiswal/.m2/repository/org/bouncycastle/bcprov-jdk15on/1.52/bcprov-jdk15on-1.52.jar:/Users/djaiswal/.m2/repository/com/jamesmurty/utils/java-xmlbuilder/1.1/java-xmlbuilder-1.1.jar:/Users/djaiswal/.m2/repository/net/iharder/base64/2.3.8/base64-2.3.8.jar:/Users/djaiswal/.m2/repository/org/antlr/antlr4-runtime/4.5.1/antlr4-runtime-4.5.1.jar:/Users/djaiswal/.m2/repository/io/druid/druid-hll/0.10.1/druid-hll-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/bytebuffer-collections/0.10.1/bytebuffer-collections-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/extendedset/0.10.1/extendedset-0.10.1.jar:/Users/djaiswal/.m2/repository/com/metamx/emitter/0.4.5/emitter-0.4.5.jar:/Users/djaiswal/.m2/repository/com/ning/compress-lzf/1.0.3/compress-lzf-1.0.3.jar:/Users/djaiswal/.m2/repository/com/ibm/icu/icu4j/54.1.1/icu4j-54.1.1.jar:/Users/djaiswal/.m2/repository/org/mapdb/mapdb/1.0.8/mapdb-1.0.8.jar:/Users/djaiswal/.m2/repository/org/ow2/asm/asm/5.2/asm-5.2.jar:/Users/djaiswal/.m2/repository/org/ow2/asm/asm-commons/5.2/asm-commons-5.2.jar:/Users/djaiswal/.m2/repository/org/ow2/asm/asm-tree/5.2/asm-tree-5.2.jar:/Users/djaiswal/.m2/repository/io/druid/extensions/druid-hdfs-storage/0.10.1/druid-hdfs-storage-0.10.1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-aws/2.7.3/hadoop-aws-2.7.3.jar:/Users/djaiswal/.m2/repository/com/amazonaws/aws-java-sdk-s3/1.10.77/aws-java-sdk-s3-1.10.77.jar:/Users/djaiswal/.m2/repository/com/amazonaws/aws-java-sdk-kms/1.10.77/aws-java-sdk-kms-1.10.77.jar:/Users/djaiswal/.m2/repository/com/amazonaws/aws-java-sdk-core/1.10.77/aws-java-sdk-core-1.10.77.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/dataformat/jackson-dataformat-cbor/2.5.3/jackson-dataformat-cbor-2.5.3.jar:/Users/djaiswal/.m2/repository/io/druid/extensions/mysql-metadata-storage/0.10.1/mysql-metadata-storage-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/extensions/postgresql-metadata-storage/0.10.1/postgresql-metadata-storage-0.10.1.jar:/Users/djaiswal/.m2/repository/org/postgresql/postgresql/9.4.1208.jre7/postgresql-9.4.1208.jre7.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-druid-handler/3.0.0-SNAPSHOT/hive-druid-handler-3.0.0-SNAPSHOT-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-jdbc-handler/3.0.0-SNAPSHOT/hive-jdbc-handler-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/junit/junit/4.11/junit-4.11.jar:/Users/djaiswal/.m2/repository/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar:/Users/djaiswal/.m2/repository/com/sun/jersey/jersey-servlet/1.19/jersey-servlet-1.19.jar:/Users/djaiswal/.m2/repository/com/sun/jersey/jersey-server/1.19/jersey-server-1.19.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-archives/3.0.0-beta1/hadoop-archives-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-common/3.0.0-beta1/hadoop-common-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-annotations/3.0.0-beta1/hadoop-annotations-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-math3/3.1.1/commons-math3-3.1.1.jar:/Users/djaiswal/.m2/repository/commons-net/commons-net/3.1/commons-net-3.1.jar:/Users/djaiswal/.m2/repository/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-util/9.3.19.v20170502/jetty-util-9.3.19.v20170502.jar:/Users/djaiswal/.m2/repository/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.jar:/Users/djaiswal/.m2/repository/com/sun/jersey/jersey-core/1.19/jersey-core-1.19.jar:/Users/djaiswal/.m2/repository/javax/ws/rs/jsr311-api/1.1.1/jsr311-api-1.1.1.jar:/Users/djaiswal/.m2/repository/com/sun/jersey/jersey-json/1.19/jersey-json-1.19.jar:/Users/djaiswal/.m2/repository/com/sun/xml/bind/jaxb-impl/2.2.3-1/jaxb-impl-2.2.3-1.jar:/Users/djaiswal/.m2/repository/org/codehaus/jackson/jackson-jaxrs/1.9.13/jackson-jaxrs-1.9.13.jar:/Users/djaiswal/.m2/repository/org/codehaus/jackson/jackson-xc/1.9.13/jackson-xc-1.9.13.jar:/Users/djaiswal/.m2/repository/commons-logging/commons-logging/1.1.3/commons-logging-1.1.3.jar:/Users/djaiswal/.m2/repository/log4j/log4j/1.2.17/log4j-1.2.17.jar:/Users/djaiswal/.m2/repository/commons-beanutils/commons-beanutils/1.9.3/commons-beanutils-1.9.3.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-configuration2/2.1.1/commons-configuration2-2.1.1.jar:/Users/djaiswal/.m2/repository/com/google/re2j/re2j/1.1/re2j-1.1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-auth/3.0.0-beta1/hadoop-auth-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/com/nimbusds/nimbus-jose-jwt/4.41.1/nimbus-jose-jwt-4.41.1.jar:/Users/djaiswal/.m2/repository/com/github/stephenc/jcip/jcip-annotations/1.0-1/jcip-annotations-1.0-1.jar:/Users/djaiswal/.m2/repository/net/minidev/json-smart/2.3/json-smart-2.3.jar:/Users/djaiswal/.m2/repository/net/minidev/accessors-smart/1.2/accessors-smart-1.2.jar:/Users/djaiswal/.m2/repository/com/jcraft/jsch/0.1.54/jsch-0.1.54.jar:/Users/djaiswal/.m2/repository/org/apache/curator/curator-client/2.12.0/curator-client-2.12.0.jar:/Users/djaiswal/.m2/repository/org/apache/curator/curator-recipes/2.12.0/curator-recipes-2.12.0.jar:/Users/djaiswal/.m2/repository/org/apache/htrace/htrace-core4/4.1.0-incubating/htrace-core4-4.1.0-incubating.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerb-simplekdc/1.0.1/kerb-simplekdc-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerb-client/1.0.1/kerb-client-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerby-config/1.0.1/kerby-config-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerb-core/1.0.1/kerb-core-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerby-pkix/1.0.1/kerby-pkix-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerby-asn1/1.0.1/kerby-asn1-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerby-util/1.0.1/kerby-util-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerb-common/1.0.1/kerb-common-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerb-crypto/1.0.1/kerb-crypto-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerb-util/1.0.1/kerb-util-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/token-provider/1.0.1/token-provider-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerb-admin/1.0.1/kerb-admin-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerb-server/1.0.1/kerb-server-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerb-identity/1.0.1/kerb-identity-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerby-xdr/1.0.1/kerby-xdr-1.0.1.jar:/Users/djaiswal/.m2/repository/org/codehaus/woodstox/stax2-api/3.1.4/stax2-api-3.1.4.jar:/Users/djaiswal/.m2/repository/com/fasterxml/woodstox/woodstox-core/5.0.3/woodstox-core-5.0.3.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-common/3.0.0-beta1/hadoop-common-3.0.0-beta1-tests.jar:/Users/djaiswal/.m2/repository/org/apache/httpcomponents/httpclient/4.5.2/httpclient-4.5.2.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-distcp/3.0.0-beta1/hadoop-distcp-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-hdfs/3.0.0-beta1/hadoop-hdfs-3.0.0-beta1-tests.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-util-ajax/9.3.19.v20170502/jetty-util-ajax-9.3.19.v20170502.jar:/Users/djaiswal/.m2/repository/commons-daemon/commons-daemon/1.0.13/commons-daemon-1.0.13.jar:/Users/djaiswal/.m2/repository/io/netty/netty/3.10.5.Final/netty-3.10.5.Final.jar:/Users/djaiswal/.m2/repository/io/netty/netty-all/4.0.52.Final/netty-all-4.0.52.Final.jar:/Users/djaiswal/.m2/repository/org/fusesource/leveldbjni/leveldbjni-all/1.8/leveldbjni-all-1.8.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-hdfs/3.0.0-beta1/hadoop-hdfs-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-jobclient/3.0.0-beta1/hadoop-mapreduce-client-jobclient-3.0.0-beta1-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-common/3.0.0-beta1/hadoop-mapreduce-client-common-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/com/google/inject/extensions/guice-servlet/4.0/guice-servlet-4.0.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-hs/3.0.0-beta1/hadoop-mapreduce-client-hs-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-hdfs-client/3.0.0-beta1/hadoop-hdfs-client-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/com/squareup/okhttp/okhttp/2.4.0/okhttp-2.4.0.jar:/Users/djaiswal/.m2/repository/com/squareup/okio/okio/1.4.0/okio-1.4.0.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-app/3.0.0-beta1/hadoop-mapreduce-client-app-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-shuffle/3.0.0-beta1/hadoop-mapreduce-client-shuffle-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-core/3.0.0-beta1/hadoop-mapreduce-client-core-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-llap-server/3.0.0-SNAPSHOT/hive-llap-server-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-llap-common/3.0.0-SNAPSHOT/hive-llap-common-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-llap-client/3.0.0-SNAPSHOT/hive-llap-client-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/slider/slider-core/0.92.0-incubating/slider-core-0.92.0-incubating.jar:/Users/djaiswal/.m2/repository/org/yaml/snakeyaml/1.16/snakeyaml-1.16.jar:/Users/djaiswal/.m2/repository/org/codehaus/jettison/jettison/1.1/jettison-1.1.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-llap-common/3.0.0-SNAPSHOT/hive-llap-common-3.0.0-SNAPSHOT-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-llap-server/3.0.0-SNAPSHOT/hive-llap-server-3.0.0-SNAPSHOT-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-server-tests/3.0.0-beta1/hadoop-yarn-server-tests-3.0.0-beta1-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-server-common/3.0.0-beta1/hadoop-yarn-server-common-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/geronimo/specs/geronimo-jcache_1.0_spec/1.0-alpha-1/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/Users/djaiswal/.m2/repository/org/ehcache/ehcache/3.3.1/ehcache-3.3.1.jar:/Users/djaiswal/.m2/repository/com/zaxxer/HikariCP-java7/2.4.12/HikariCP-java7-2.4.12.jar:/Users/djaiswal/.m2/repository/com/microsoft/sqlserver/mssql-jdbc/6.2.1.jre7/mssql-jdbc-6.2.1.jre7.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-server-nodemanager/3.0.0-beta1/hadoop-yarn-server-nodemanager-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/com/codahale/metrics/metrics-core/3.0.1/metrics-core-3.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-server-resourcemanager/3.0.0-beta1/hadoop-yarn-server-resourcemanager-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-server-applicationhistoryservice/3.0.0-beta1/hadoop-yarn-server-applicationhistoryservice-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/de/ruedigermoeller/fst/2.50/fst-2.50.jar:/Users/djaiswal/.m2/repository/com/cedarsoftware/java-util/1.9.0/java-util-1.9.0.jar:/Users/djaiswal/.m2/repository/com/cedarsoftware/json-io/2.5.1/json-io-2.5.1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-server-timelineservice/3.0.0-beta1/hadoop-yarn-server-timelineservice-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-csv/1.0/commons-csv-1.0.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-client/3.0.0-beta1/hadoop-yarn-client-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-common/3.0.0-beta1/hadoop-yarn-common-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/javax/xml/bind/jaxb-api/2.2.11/jaxb-api-2.2.11.jar:/Users/djaiswal/.m2/repository/com/sun/jersey/jersey-client/1.19/jersey-client-1.19.jar:/Users/djaiswal/.m2/repository/com/google/inject/guice/4.0/guice-4.0.jar:/Users/djaiswal/.m2/repository/aopalliance/aopalliance/1.0/aopalliance-1.0.jar:/Users/djaiswal/.m2/repository/com/sun/jersey/contribs/jersey-guice/1.19/jersey-guice-1.19.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/core/jackson-core/2.7.8/jackson-core-2.7.8.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/module/jackson-module-jaxb-annotations/2.7.8/jackson-module-jaxb-annotations-2.7.8.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/jaxrs/jackson-jaxrs-json-provider/2.7.8/jackson-jaxrs-json-provider-2.7.8.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/jaxrs/jackson-jaxrs-base/2.7.8/jackson-jaxrs-base-2.7.8.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-api/3.0.0-beta1/hadoop-yarn-api-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/core/jackson-annotations/2.7.8/jackson-annotations-2.7.8.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-common/2.0.0-alpha4/hbase-common-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/thirdparty/hbase-shaded-miscellaneous/1.0.1/hbase-shaded-miscellaneous-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-collections4/4.1/commons-collections4-4.1.jar:/Users/djaiswal/.m2/repository/org/apache/htrace/htrace-core/3.2.0-incubating/htrace-core-3.2.0-incubating.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-crypto/1.0.0/commons-crypto-1.0.0.jar:/Users/djaiswal/.m2/repository/com/github/stephenc/findbugs/findbugs-annotations/1.3.9-1/findbugs-annotations-1.3.9-1.jar:/Users/djaiswal/.m2/repository/org/apache/yetus/audience-annotations/0.5.0/audience-annotations-0.5.0.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-common/2.0.0-alpha4/hbase-common-2.0.0-alpha4-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-hadoop-compat/2.0.0-alpha4/hbase-hadoop-compat-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-metrics-api/2.0.0-alpha4/hbase-metrics-api-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-hadoop-compat/2.0.0-alpha4/hbase-hadoop-compat-2.0.0-alpha4-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-hadoop2-compat/2.0.0-alpha4/hbase-hadoop2-compat-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-metrics/2.0.0-alpha4/hbase-metrics-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-hadoop2-compat/2.0.0-alpha4/hbase-hadoop2-compat-2.0.0-alpha4-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-server/2.0.0-alpha4/hbase-server-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/thirdparty/hbase-shaded-protobuf/1.0.1/hbase-shaded-protobuf-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/thirdparty/hbase-shaded-netty/1.0.1/hbase-shaded-netty-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-http/2.0.0-alpha4/hbase-http-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/glassfish/jersey/core/jersey-server/2.25.1/jersey-server-2.25.1.jar:/Users/djaiswal/.m2/repository/org/glassfish/jersey/core/jersey-common/2.25.1/jersey-common-2.25.1.jar:/Users/djaiswal/.m2/repository/org/glassfish/jersey/bundles/repackaged/jersey-guava/2.25.1/jersey-guava-2.25.1.jar:/Users/djaiswal/.m2/repository/org/glassfish/hk2/osgi-resource-locator/1.0.1/osgi-resource-locator-1.0.1.jar:/Users/djaiswal/.m2/repository/org/glassfish/jersey/core/jersey-client/2.25.1/jersey-client-2.25.1.jar:/Users/djaiswal/.m2/repository/org/glassfish/jersey/media/jersey-media-jaxb/2.25.1/jersey-media-jaxb-2.25.1.jar:/Users/djaiswal/.m2/repository/javax/annotation/javax.annotation-api/1.2/javax.annotation-api-1.2.jar:/Users/djaiswal/.m2/repository/org/glassfish/hk2/hk2-api/2.5.0-b32/hk2-api-2.5.0-b32.jar:/Users/djaiswal/.m2/repository/org/glassfish/hk2/hk2-utils/2.5.0-b32/hk2-utils-2.5.0-b32.jar:/Users/djaiswal/.m2/repository/org/glassfish/hk2/external/aopalliance-repackaged/2.5.0-b32/aopalliance-repackaged-2.5.0-b32.jar:/Users/djaiswal/.m2/repository/org/glassfish/hk2/external/javax.inject/2.5.0-b32/javax.inject-2.5.0-b32.jar:/Users/djaiswal/.m2/repository/org/glassfish/hk2/hk2-locator/2.5.0-b32/hk2-locator-2.5.0-b32.jar:/Users/djaiswal/.m2/repository/org/javassist/javassist/3.20.0-GA/javassist-3.20.0-GA.jar:/Users/djaiswal/.m2/repository/org/glassfish/jersey/containers/jersey-container-servlet-core/2.25.1/jersey-container-servlet-core-2.25.1.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-protocol/2.0.0-alpha4/hbase-protocol-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-protocol-shaded/2.0.0-alpha4/hbase-protocol-shaded-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-procedure/2.0.0-alpha4/hbase-procedure-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-replication/2.0.0-alpha4/hbase-replication-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-prefix-tree/2.0.0-alpha4/hbase-prefix-tree-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/glassfish/web/javax.servlet.jsp/2.3.2/javax.servlet.jsp-2.3.2.jar:/Users/djaiswal/.m2/repository/org/jamon/jamon-runtime/2.3.1/jamon-runtime-2.3.1.jar:/Users/djaiswal/.m2/repository/javax/ws/rs/javax.ws.rs-api/2.0.1/javax.ws.rs-api-2.0.1.jar:/Users/djaiswal/.m2/repository/com/lmax/disruptor/3.3.6/disruptor-3.3.6.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-client/3.0.0-beta1/hadoop-client-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-jobclient/3.0.0-beta1/hadoop-mapreduce-client-jobclient-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-server/2.0.0-alpha4/hbase-server-2.0.0-alpha4-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-mapreduce/2.0.0-alpha4/hbase-mapreduce-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/tez/tez-tests/0.9.1-SNAPSHOT/tez-tests-0.9.1-SNAPSHOT-tests.jar:/Users/djaiswal/.m2/repository/org/apache/tez/tez-common/0.9.1-SNAPSHOT/tez-common-0.9.1-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/slf4j/slf4j-log4j12/1.7.10/slf4j-log4j12-1.7.10.jar:/Users/djaiswal/.m2/repository/org/apache/tez/tez-examples/0.9.1-SNAPSHOT/tez-examples-0.9.1-SNAPSHOT.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_101.jdk/Contents/Home/jre/../lib/tools.jar:/Users/djaiswal/.m2/repository/org/apache/tez/tez-api/0.9.1-SNAPSHOT/tez-api-0.9.1-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/tez/tez-runtime-library/0.9.1-SNAPSHOT/tez-runtime-library-0.9.1-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/roaringbitmap/RoaringBitmap/0.4.9/RoaringBitmap-0.4.9.jar:/Users/djaiswal/.m2/repository/com/ning/async-http-client/1.8.16/async-http-client-1.8.16.jar:/Users/djaiswal/.m2/repository/org/apache/tez/tez-mapreduce/0.9.1-SNAPSHOT/tez-mapreduce-0.9.1-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/tez/tez-dag/0.9.1-SNAPSHOT/tez-dag-0.9.1-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/tez/hadoop-shim/0.9.1-SNAPSHOT/hadoop-shim-0.9.1-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/tez/tez-runtime-internals/0.9.1-SNAPSHOT/tez-runtime-internals-0.9.1-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-server-web-proxy/3.0.0-beta1/hadoop-yarn-server-web-proxy-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/mortbay/jetty/jetty/6.1.26/jetty-6.1.26.jar:/Users/djaiswal/.m2/repository/org/mortbay/jetty/servlet-api/2.5-20081211/servlet-api-2.5-20081211.jar:/Users/djaiswal/.m2/repository/org/mortbay/jetty/jetty-util/6.1.26/jetty-util-6.1.26.jar:/Users/djaiswal/.m2/repository/javax/servlet/servlet-api/2.5/servlet-api-2.5.jar:/Users/djaiswal/hive/hive/itests/qtest-druid/target/classes:/Users/djaiswal/.m2/repository/io/druid/druid-services/0.10.1/druid-services-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/druid-indexing-hadoop/0.10.1/druid-indexing-hadoop-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/druid-indexing-service/0.10.1/druid-indexing-service-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/druid-sql/0.10.1/druid-sql-0.10.1.jar:/Users/djaiswal/.m2/repository/org/apache/calcite/avatica/avatica-core/1.9.0/avatica-core-1.9.0.jar:/Users/djaiswal/.m2/repository/org/apache/calcite/avatica/avatica-metrics/1.9.0/avatica-metrics-1.9.0.jar:/Users/djaiswal/.m2/repository/org/apache/calcite/avatica/avatica-server/1.9.0/avatica-server-1.9.0.jar:/Users/djaiswal/.m2/repository/io/airlift/airline/0.7/airline-0.7.jar:/Users/djaiswal/.m2/repository/org/apache/logging/log4j/log4j-api/2.8.2/log4j-api-2.8.2.jar:/Users/djaiswal/.m2/repository/org/apache/logging/log4j/log4j-core/2.8.2/log4j-core-2.8.2.jar:/Users/djaiswal/.m2/repository/org/apache/curator/curator-x-discovery/2.11.0/curator-x-discovery-2.11.0.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-servlets/9.3.19.v20170502/jetty-servlets-9.3.19.v20170502.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-continuation/9.3.19.v20170502/jetty-continuation-9.3.19.v20170502.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-proxy/9.3.19.v20170502/jetty-proxy-9.3.19.v20170502.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-security/9.3.19.v20170502/jetty-security-9.3.19.v20170502.jar:/Users/djaiswal/.m2/repository/org/slf4j/slf4j-api/1.7.10/slf4j-api-1.7.10.jar:/Users/djaiswal/.m2/repository/org/mockito/mockito-all/1.10.19/mockito-all-1.10.19.jar:/Users/djaiswal/hive/hive/itests/qtest/target/testconf:/Users/djaiswal/hive/hive/itests/qtest/../../conf: 2018-01-05T11:28:36,057 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:36,057 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:36,066 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,066 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:36,066 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,066 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,066 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,069 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS src 2018-01-05T11:28:36,096 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112836_0d700b48-2684-426a-9388-871b75cbfa3f): DROP TABLE IF EXISTS src 2018-01-05T11:28:36,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS src 2018-01-05T11:28:36,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:36,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,898 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:36,899 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:36,899 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:36,900 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:36,901 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src 2018-01-05T11:28:36,901 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src 2018-01-05T11:28:36,912 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:36,912 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:36,926 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:36,926 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,928 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:36,932 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,933 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:36,933 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=1, getTables_(String, String, )=8, getAllDatabases_()=4, flushCache_()=1, listRoleNames_()=4, getFunctions_(String, String, )=14} 2018-01-05T11:28:36,933 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112836_0d700b48-2684-426a-9388-871b75cbfa3f); Time taken: 0.866 seconds 2018-01-05T11:28:36,933 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:36,933 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,933 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112836_0d700b48-2684-426a-9388-871b75cbfa3f): DROP TABLE IF EXISTS src 2018-01-05T11:28:36,938 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,939 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS src 2018-01-05T11:28:36,939 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:36,939 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,939 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,939 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,943 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,949 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:36,950 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:36,950 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src 2018-01-05T11:28:36,950 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src 2018-01-05T11:28:36,952 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:36,952 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:36,952 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table src not found: default.src table not found 2018-01-05T11:28:36,968 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:36,968 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src 2018-01-05T11:28:36,969 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src 2018-01-05T11:28:36,969 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:36,970 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:36,970 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,971 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,971 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS src 2018-01-05T11:28:36,971 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:36,972 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,972 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,972 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:36,972 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=2} 2018-01-05T11:28:36,972 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112836_0d700b48-2684-426a-9388-871b75cbfa3f); Time taken: 0.039 seconds 2018-01-05T11:28:36,972 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:36,972 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,972 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:36,972 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,973 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query DROP TABLE IF EXISTS src 2018-01-05T11:28:36,973 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.908 seconds 2018-01-05T11:28:36,973 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:36,973 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:36,973 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:36,973 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:36,974 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,974 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:36,974 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,974 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,974 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,974 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS src1 2018-01-05T11:28:36,974 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112836_e82c0410-b089-4f47-bcf5-0bc2343889ab): DROP TABLE IF EXISTS src1 2018-01-05T11:28:36,974 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,974 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS src1 2018-01-05T11:28:36,975 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:36,975 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,975 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,975 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:36,975 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:36,975 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:36,975 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:36,975 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src1 2018-01-05T11:28:36,976 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src1 2018-01-05T11:28:36,976 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:36,977 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:36,977 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:36,977 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,977 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:36,977 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,977 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:36,977 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:28:36,977 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112836_e82c0410-b089-4f47-bcf5-0bc2343889ab); Time taken: 0.003 seconds 2018-01-05T11:28:36,977 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:36,977 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,977 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112836_e82c0410-b089-4f47-bcf5-0bc2343889ab): DROP TABLE IF EXISTS src1 2018-01-05T11:28:36,978 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,978 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS src1 2018-01-05T11:28:36,978 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:36,978 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,978 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,978 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,978 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,979 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:36,979 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:36,979 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src1 2018-01-05T11:28:36,979 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src1 2018-01-05T11:28:36,979 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:36,980 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:36,980 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table src1 not found: default.src1 table not found 2018-01-05T11:28:36,980 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:36,980 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src1 2018-01-05T11:28:36,980 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src1 2018-01-05T11:28:36,980 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:36,980 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:36,980 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,980 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,981 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS src1 2018-01-05T11:28:36,981 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:36,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,981 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:36,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=0} 2018-01-05T11:28:36,981 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112836_e82c0410-b089-4f47-bcf5-0bc2343889ab); Time taken: 0.004 seconds 2018-01-05T11:28:36,981 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:36,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,981 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:36,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query DROP TABLE IF EXISTS src1 2018-01-05T11:28:36,981 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.007 seconds 2018-01-05T11:28:36,981 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:36,981 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:36,981 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:36,981 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:36,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:36,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS src_json 2018-01-05T11:28:36,981 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112836_79630187-23f9-40f5-82c6-711d7e596314): DROP TABLE IF EXISTS src_json 2018-01-05T11:28:36,982 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,982 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS src_json 2018-01-05T11:28:36,982 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:36,982 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,982 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,984 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:36,984 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:36,984 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:36,984 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:36,984 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_json 2018-01-05T11:28:36,984 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_json 2018-01-05T11:28:36,984 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:36,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:36,985 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:36,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,985 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:36,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,985 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:36,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=1, flushCache_()=1} 2018-01-05T11:28:36,985 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112836_79630187-23f9-40f5-82c6-711d7e596314); Time taken: 0.004 seconds 2018-01-05T11:28:36,985 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:36,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,985 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112836_79630187-23f9-40f5-82c6-711d7e596314): DROP TABLE IF EXISTS src_json 2018-01-05T11:28:36,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,986 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS src_json 2018-01-05T11:28:36,986 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:36,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,986 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:36,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:36,986 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_json 2018-01-05T11:28:36,986 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_json 2018-01-05T11:28:36,987 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:36,987 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:36,987 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table src_json not found: default.src_json table not found 2018-01-05T11:28:36,987 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:36,987 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_json 2018-01-05T11:28:36,987 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_json 2018-01-05T11:28:36,988 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:36,988 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:36,988 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,988 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,988 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS src_json 2018-01-05T11:28:36,988 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:36,988 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,988 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,988 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:36,988 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=1} 2018-01-05T11:28:36,988 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112836_79630187-23f9-40f5-82c6-711d7e596314); Time taken: 0.003 seconds 2018-01-05T11:28:36,988 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:36,988 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,988 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:36,988 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,988 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query DROP TABLE IF EXISTS src_json 2018-01-05T11:28:36,988 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.007 seconds 2018-01-05T11:28:36,988 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:36,988 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:36,989 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:36,989 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:36,989 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,989 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:36,989 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,989 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,989 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,989 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS src_sequencefile 2018-01-05T11:28:36,989 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112836_2b272a50-794e-476d-b248-abc0fe828d2f): DROP TABLE IF EXISTS src_sequencefile 2018-01-05T11:28:36,989 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,989 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS src_sequencefile 2018-01-05T11:28:36,989 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:36,990 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,990 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,990 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:36,990 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:36,990 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:36,990 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:36,990 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_sequencefile 2018-01-05T11:28:36,990 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_sequencefile 2018-01-05T11:28:36,992 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:36,992 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:36,992 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:36,992 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,992 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:36,992 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,992 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:36,992 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:28:36,992 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112836_2b272a50-794e-476d-b248-abc0fe828d2f); Time taken: 0.003 seconds 2018-01-05T11:28:36,992 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:36,992 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,992 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112836_2b272a50-794e-476d-b248-abc0fe828d2f): DROP TABLE IF EXISTS src_sequencefile 2018-01-05T11:28:36,993 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,993 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS src_sequencefile 2018-01-05T11:28:36,993 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:36,994 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,994 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,994 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,994 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,994 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:36,994 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:36,994 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_sequencefile 2018-01-05T11:28:36,994 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_sequencefile 2018-01-05T11:28:36,995 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:36,995 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:36,995 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table src_sequencefile not found: default.src_sequencefile table not found 2018-01-05T11:28:36,995 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:36,995 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_sequencefile 2018-01-05T11:28:36,995 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_sequencefile 2018-01-05T11:28:36,996 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:36,996 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:36,996 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,996 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,996 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS src_sequencefile 2018-01-05T11:28:36,996 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:36,996 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,996 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,996 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:36,996 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=1} 2018-01-05T11:28:36,996 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112836_2b272a50-794e-476d-b248-abc0fe828d2f); Time taken: 0.004 seconds 2018-01-05T11:28:36,996 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:36,996 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,996 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:36,996 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,996 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query DROP TABLE IF EXISTS src_sequencefile 2018-01-05T11:28:36,996 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.007 seconds 2018-01-05T11:28:36,996 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:36,997 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:36,997 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:36,997 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:36,997 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,997 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:36,997 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,997 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,997 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,997 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS src_thrift 2018-01-05T11:28:36,997 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112836_fc7ed713-7b38-4503-9356-32035f74dc22): DROP TABLE IF EXISTS src_thrift 2018-01-05T11:28:36,997 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,997 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS src_thrift 2018-01-05T11:28:36,998 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:36,998 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,998 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,998 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:36,998 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:36,998 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:36,998 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:36,998 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_thrift 2018-01-05T11:28:36,998 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_thrift 2018-01-05T11:28:36,999 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:36,999 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:36,999 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:36,999 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,999 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:36,999 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,999 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:36,999 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:28:36,999 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112836_fc7ed713-7b38-4503-9356-32035f74dc22); Time taken: 0.002 seconds 2018-01-05T11:28:36,999 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:36,999 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:36,999 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112836_fc7ed713-7b38-4503-9356-32035f74dc22): DROP TABLE IF EXISTS src_thrift 2018-01-05T11:28:37,000 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,000 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS src_thrift 2018-01-05T11:28:37,000 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:37,000 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,000 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,000 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,001 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,001 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:37,001 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,001 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_thrift 2018-01-05T11:28:37,001 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_thrift 2018-01-05T11:28:37,002 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,002 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,002 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table src_thrift not found: default.src_thrift table not found 2018-01-05T11:28:37,002 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,002 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_thrift 2018-01-05T11:28:37,002 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_thrift 2018-01-05T11:28:37,002 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,003 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,003 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,003 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,003 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS src_thrift 2018-01-05T11:28:37,003 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:37,003 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,003 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,003 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:37,003 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=1} 2018-01-05T11:28:37,003 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112836_fc7ed713-7b38-4503-9356-32035f74dc22); Time taken: 0.004 seconds 2018-01-05T11:28:37,003 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:37,003 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,003 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,003 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,003 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query DROP TABLE IF EXISTS src_thrift 2018-01-05T11:28:37,003 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.006 seconds 2018-01-05T11:28:37,003 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,003 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:37,003 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,003 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:37,003 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,003 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:37,003 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,003 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,003 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,003 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS srcbucket 2018-01-05T11:28:37,003 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112837_57cee015-b87f-4c2b-baa8-5121a55aba1e): DROP TABLE IF EXISTS srcbucket 2018-01-05T11:28:37,004 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,004 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS srcbucket 2018-01-05T11:28:37,004 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:37,004 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,004 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,004 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,004 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,004 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,004 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,005 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket 2018-01-05T11:28:37,005 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket 2018-01-05T11:28:37,006 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,006 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,006 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:37,006 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,006 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:37,006 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,006 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:37,006 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:28:37,006 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112837_57cee015-b87f-4c2b-baa8-5121a55aba1e); Time taken: 0.003 seconds 2018-01-05T11:28:37,006 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,006 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,006 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112837_57cee015-b87f-4c2b-baa8-5121a55aba1e): DROP TABLE IF EXISTS srcbucket 2018-01-05T11:28:37,007 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,007 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS srcbucket 2018-01-05T11:28:37,007 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:37,007 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,007 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,007 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,007 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,007 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:37,008 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,008 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket 2018-01-05T11:28:37,008 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket 2018-01-05T11:28:37,008 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,008 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,008 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table srcbucket not found: default.srcbucket table not found 2018-01-05T11:28:37,009 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,009 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket 2018-01-05T11:28:37,009 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket 2018-01-05T11:28:37,009 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,009 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,009 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,009 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,009 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS srcbucket 2018-01-05T11:28:37,009 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:37,009 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,009 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,009 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:37,009 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=1, dropTable_(String, String, boolean, boolean, boolean, )=0} 2018-01-05T11:28:37,009 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112837_57cee015-b87f-4c2b-baa8-5121a55aba1e); Time taken: 0.003 seconds 2018-01-05T11:28:37,009 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:37,009 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,010 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,010 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,010 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query DROP TABLE IF EXISTS srcbucket 2018-01-05T11:28:37,010 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.007 seconds 2018-01-05T11:28:37,010 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,010 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:37,010 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,010 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:37,010 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,010 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:37,010 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,010 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,010 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,010 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS srcbucket2 2018-01-05T11:28:37,010 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112837_fd435309-c548-49db-9d6f-b378c5b259de): DROP TABLE IF EXISTS srcbucket2 2018-01-05T11:28:37,010 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,010 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS srcbucket2 2018-01-05T11:28:37,010 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:37,010 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,010 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,011 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,011 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,011 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,011 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,011 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket2 2018-01-05T11:28:37,011 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket2 2018-01-05T11:28:37,011 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,011 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,011 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:37,012 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,012 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:37,012 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,012 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:37,012 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=1, flushCache_()=0} 2018-01-05T11:28:37,012 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112837_fd435309-c548-49db-9d6f-b378c5b259de); Time taken: 0.002 seconds 2018-01-05T11:28:37,012 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,012 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,012 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112837_fd435309-c548-49db-9d6f-b378c5b259de): DROP TABLE IF EXISTS srcbucket2 2018-01-05T11:28:37,013 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,013 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS srcbucket2 2018-01-05T11:28:37,013 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:37,013 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,013 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,013 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,013 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,013 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:37,014 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,014 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket2 2018-01-05T11:28:37,014 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket2 2018-01-05T11:28:37,015 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,015 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,015 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table srcbucket2 not found: default.srcbucket2 table not found 2018-01-05T11:28:37,015 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,015 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket2 2018-01-05T11:28:37,015 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket2 2018-01-05T11:28:37,016 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,016 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,016 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,016 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,017 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS srcbucket2 2018-01-05T11:28:37,017 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:37,017 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,017 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,017 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:37,017 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=1, dropTable_(String, String, boolean, boolean, boolean, )=1} 2018-01-05T11:28:37,017 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112837_fd435309-c548-49db-9d6f-b378c5b259de); Time taken: 0.005 seconds 2018-01-05T11:28:37,017 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:37,017 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,017 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,017 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,017 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query DROP TABLE IF EXISTS srcbucket2 2018-01-05T11:28:37,017 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.007 seconds 2018-01-05T11:28:37,017 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,017 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:37,017 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,017 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:37,017 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,017 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:37,017 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,017 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,017 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,017 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS srcpart 2018-01-05T11:28:37,017 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112837_72c78dfc-e3e6-4ed6-b0c9-2088588a4992): DROP TABLE IF EXISTS srcpart 2018-01-05T11:28:37,018 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,018 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS srcpart 2018-01-05T11:28:37,018 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:37,018 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,018 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,019 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,019 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,019 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,019 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,019 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcpart 2018-01-05T11:28:37,019 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcpart 2018-01-05T11:28:37,019 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,020 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,020 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:37,020 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,020 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:37,020 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,020 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:37,020 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=1} 2018-01-05T11:28:37,020 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112837_72c78dfc-e3e6-4ed6-b0c9-2088588a4992); Time taken: 0.003 seconds 2018-01-05T11:28:37,020 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,020 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,020 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112837_72c78dfc-e3e6-4ed6-b0c9-2088588a4992): DROP TABLE IF EXISTS srcpart 2018-01-05T11:28:37,021 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,021 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS srcpart 2018-01-05T11:28:37,021 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:37,021 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,021 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,021 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,021 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,021 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:37,022 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,022 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcpart 2018-01-05T11:28:37,022 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcpart 2018-01-05T11:28:37,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,023 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table srcpart not found: default.srcpart table not found 2018-01-05T11:28:37,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,023 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcpart 2018-01-05T11:28:37,023 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcpart 2018-01-05T11:28:37,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,024 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS srcpart 2018-01-05T11:28:37,024 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:37,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,024 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:37,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=1, dropTable_(String, String, boolean, boolean, boolean, )=1} 2018-01-05T11:28:37,024 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112837_72c78dfc-e3e6-4ed6-b0c9-2088588a4992); Time taken: 0.004 seconds 2018-01-05T11:28:37,024 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:37,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,024 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query DROP TABLE IF EXISTS srcpart 2018-01-05T11:28:37,024 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.007 seconds 2018-01-05T11:28:37,024 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,024 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:37,024 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,024 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:37,025 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,025 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:37,025 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,025 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,025 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,025 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS primitives 2018-01-05T11:28:37,025 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112837_74b50792-5cb8-440f-b021-30c0b298f5e4): DROP TABLE IF EXISTS primitives 2018-01-05T11:28:37,025 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,025 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS primitives 2018-01-05T11:28:37,025 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:37,025 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,025 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,026 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,026 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,026 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,026 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,026 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=primitives 2018-01-05T11:28:37,026 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=primitives 2018-01-05T11:28:37,027 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,027 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,027 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:37,027 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,027 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:37,027 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,027 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:37,027 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=1, flushCache_()=0} 2018-01-05T11:28:37,027 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112837_74b50792-5cb8-440f-b021-30c0b298f5e4); Time taken: 0.002 seconds 2018-01-05T11:28:37,027 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,027 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,027 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112837_74b50792-5cb8-440f-b021-30c0b298f5e4): DROP TABLE IF EXISTS primitives 2018-01-05T11:28:37,028 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,028 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS primitives 2018-01-05T11:28:37,028 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:37,028 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,028 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,028 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,028 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,028 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:37,029 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,029 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=primitives 2018-01-05T11:28:37,029 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=primitives 2018-01-05T11:28:37,029 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,029 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,030 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table primitives not found: default.primitives table not found 2018-01-05T11:28:37,030 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,030 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=primitives 2018-01-05T11:28:37,030 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=primitives 2018-01-05T11:28:37,031 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,031 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,031 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,031 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,031 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS primitives 2018-01-05T11:28:37,031 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:37,031 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,031 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,031 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:37,031 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=1} 2018-01-05T11:28:37,031 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112837_74b50792-5cb8-440f-b021-30c0b298f5e4); Time taken: 0.004 seconds 2018-01-05T11:28:37,031 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:37,031 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,031 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,031 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,031 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query DROP TABLE IF EXISTS primitives 2018-01-05T11:28:37,031 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.006 seconds 2018-01-05T11:28:37,031 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,031 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:37,031 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,031 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:37,032 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,032 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:37,032 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,032 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,032 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,032 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS dest1 2018-01-05T11:28:37,032 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112837_4cf1c61b-0377-4030-b7d1-16c854d5f50f): DROP TABLE IF EXISTS dest1 2018-01-05T11:28:37,032 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,032 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS dest1 2018-01-05T11:28:37,032 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:37,032 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,032 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,033 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,033 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,033 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,033 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,033 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest1 2018-01-05T11:28:37,033 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest1 2018-01-05T11:28:37,033 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,033 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,033 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:37,033 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,033 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:37,034 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,034 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:37,034 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:28:37,034 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112837_4cf1c61b-0377-4030-b7d1-16c854d5f50f); Time taken: 0.002 seconds 2018-01-05T11:28:37,034 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,034 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,034 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112837_4cf1c61b-0377-4030-b7d1-16c854d5f50f): DROP TABLE IF EXISTS dest1 2018-01-05T11:28:37,035 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,035 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS dest1 2018-01-05T11:28:37,035 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:37,035 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,035 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,035 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,035 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,035 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:37,035 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,035 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest1 2018-01-05T11:28:37,035 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest1 2018-01-05T11:28:37,036 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,036 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,036 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table dest1 not found: default.dest1 table not found 2018-01-05T11:28:37,036 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,036 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest1 2018-01-05T11:28:37,036 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest1 2018-01-05T11:28:37,037 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,037 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,037 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,037 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,037 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS dest1 2018-01-05T11:28:37,037 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:37,037 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,037 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,037 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:37,037 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=1} 2018-01-05T11:28:37,038 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112837_4cf1c61b-0377-4030-b7d1-16c854d5f50f); Time taken: 0.003 seconds 2018-01-05T11:28:37,038 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:37,038 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,038 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,038 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,038 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query DROP TABLE IF EXISTS dest1 2018-01-05T11:28:37,038 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.006 seconds 2018-01-05T11:28:37,038 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,038 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:37,038 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,038 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:37,038 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,038 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:37,038 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,038 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,039 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,039 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS dest2 2018-01-05T11:28:37,039 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112837_fc140fb6-eb7f-4abf-86b8-6f986005b03c): DROP TABLE IF EXISTS dest2 2018-01-05T11:28:37,039 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,039 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS dest2 2018-01-05T11:28:37,039 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:37,039 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,039 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,040 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,040 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,040 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,040 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,040 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest2 2018-01-05T11:28:37,040 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest2 2018-01-05T11:28:37,041 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,041 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,041 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:37,041 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,041 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:37,041 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,041 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:37,041 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=1, flushCache_()=0} 2018-01-05T11:28:37,041 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112837_fc140fb6-eb7f-4abf-86b8-6f986005b03c); Time taken: 0.002 seconds 2018-01-05T11:28:37,041 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,041 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,041 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112837_fc140fb6-eb7f-4abf-86b8-6f986005b03c): DROP TABLE IF EXISTS dest2 2018-01-05T11:28:37,042 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,042 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS dest2 2018-01-05T11:28:37,042 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:37,042 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,042 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,042 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,042 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,042 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:37,042 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,042 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest2 2018-01-05T11:28:37,042 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest2 2018-01-05T11:28:37,043 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,043 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,043 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table dest2 not found: default.dest2 table not found 2018-01-05T11:28:37,043 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,043 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest2 2018-01-05T11:28:37,043 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest2 2018-01-05T11:28:37,044 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,044 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,044 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,044 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,044 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS dest2 2018-01-05T11:28:37,044 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:37,044 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,044 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,044 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:37,044 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=1} 2018-01-05T11:28:37,044 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112837_fc140fb6-eb7f-4abf-86b8-6f986005b03c); Time taken: 0.003 seconds 2018-01-05T11:28:37,045 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:37,045 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,045 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,045 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,045 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query DROP TABLE IF EXISTS dest2 2018-01-05T11:28:37,045 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.007 seconds 2018-01-05T11:28:37,045 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,045 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:37,045 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,045 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:37,045 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,045 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:37,045 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,045 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,045 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,045 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS dest3 2018-01-05T11:28:37,045 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112837_05865ca1-aead-4729-975e-fb3d527f2879): DROP TABLE IF EXISTS dest3 2018-01-05T11:28:37,046 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,046 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS dest3 2018-01-05T11:28:37,046 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:37,046 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,046 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,046 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,046 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,046 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,046 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,046 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest3 2018-01-05T11:28:37,046 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest3 2018-01-05T11:28:37,047 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,047 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,047 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:37,047 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,047 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:37,048 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,048 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:37,048 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:28:37,048 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112837_05865ca1-aead-4729-975e-fb3d527f2879); Time taken: 0.003 seconds 2018-01-05T11:28:37,048 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,048 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,048 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112837_05865ca1-aead-4729-975e-fb3d527f2879): DROP TABLE IF EXISTS dest3 2018-01-05T11:28:37,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,049 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS dest3 2018-01-05T11:28:37,049 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:37,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,049 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:37,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,049 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest3 2018-01-05T11:28:37,049 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest3 2018-01-05T11:28:37,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,050 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table dest3 not found: default.dest3 table not found 2018-01-05T11:28:37,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,050 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest3 2018-01-05T11:28:37,050 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest3 2018-01-05T11:28:37,051 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,051 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,051 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,051 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,051 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS dest3 2018-01-05T11:28:37,051 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:37,051 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,051 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,051 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:37,051 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=1} 2018-01-05T11:28:37,051 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112837_05865ca1-aead-4729-975e-fb3d527f2879); Time taken: 0.003 seconds 2018-01-05T11:28:37,051 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:37,052 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,052 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,052 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,052 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query DROP TABLE IF EXISTS dest3 2018-01-05T11:28:37,052 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.007 seconds 2018-01-05T11:28:37,052 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,052 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:37,052 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,052 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:37,052 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,052 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:37,052 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,052 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,052 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,052 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS dest4 2018-01-05T11:28:37,052 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112837_05965ca9-5551-404b-9096-30e8ad574796): DROP TABLE IF EXISTS dest4 2018-01-05T11:28:37,052 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,052 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS dest4 2018-01-05T11:28:37,053 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:37,053 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,053 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,053 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,053 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,053 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,053 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,053 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest4 2018-01-05T11:28:37,053 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest4 2018-01-05T11:28:37,054 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,055 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,055 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:37,055 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,055 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:37,055 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,055 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:37,055 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:28:37,055 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112837_05965ca9-5551-404b-9096-30e8ad574796); Time taken: 0.003 seconds 2018-01-05T11:28:37,055 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,055 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,055 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112837_05965ca9-5551-404b-9096-30e8ad574796): DROP TABLE IF EXISTS dest4 2018-01-05T11:28:37,056 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,056 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS dest4 2018-01-05T11:28:37,056 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:37,056 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,056 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,056 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,056 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,057 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:37,057 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,057 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest4 2018-01-05T11:28:37,057 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest4 2018-01-05T11:28:37,058 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,058 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,058 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table dest4 not found: default.dest4 table not found 2018-01-05T11:28:37,058 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,058 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest4 2018-01-05T11:28:37,058 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest4 2018-01-05T11:28:37,059 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,059 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,059 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,059 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,059 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS dest4 2018-01-05T11:28:37,059 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:37,059 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,059 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,059 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:37,059 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=1} 2018-01-05T11:28:37,059 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112837_05965ca9-5551-404b-9096-30e8ad574796); Time taken: 0.004 seconds 2018-01-05T11:28:37,059 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:37,059 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,059 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,059 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,059 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query DROP TABLE IF EXISTS dest4 2018-01-05T11:28:37,059 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.007 seconds 2018-01-05T11:28:37,059 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,059 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:37,060 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,060 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:37,060 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,060 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:37,060 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,060 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,060 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,060 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS dest4_sequencefile 2018-01-05T11:28:37,060 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112837_3fa9ab5a-9461-4846-be29-a1b973ac37a8): DROP TABLE IF EXISTS dest4_sequencefile 2018-01-05T11:28:37,060 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,060 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS dest4_sequencefile 2018-01-05T11:28:37,060 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:37,060 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,060 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,060 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,060 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,061 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,061 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,061 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest4_sequencefile 2018-01-05T11:28:37,061 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest4_sequencefile 2018-01-05T11:28:37,061 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,062 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,062 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:37,062 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,062 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:37,062 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,062 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:37,062 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=1} 2018-01-05T11:28:37,062 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112837_3fa9ab5a-9461-4846-be29-a1b973ac37a8); Time taken: 0.002 seconds 2018-01-05T11:28:37,062 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,062 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,062 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112837_3fa9ab5a-9461-4846-be29-a1b973ac37a8): DROP TABLE IF EXISTS dest4_sequencefile 2018-01-05T11:28:37,063 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,063 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS dest4_sequencefile 2018-01-05T11:28:37,063 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:37,063 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,063 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,063 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,063 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,064 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:37,064 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,064 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest4_sequencefile 2018-01-05T11:28:37,064 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest4_sequencefile 2018-01-05T11:28:37,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,065 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table dest4_sequencefile not found: default.dest4_sequencefile table not found 2018-01-05T11:28:37,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,065 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest4_sequencefile 2018-01-05T11:28:37,065 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest4_sequencefile 2018-01-05T11:28:37,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,066 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS dest4_sequencefile 2018-01-05T11:28:37,066 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:37,066 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,066 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,066 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:37,066 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=0} 2018-01-05T11:28:37,066 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112837_3fa9ab5a-9461-4846-be29-a1b973ac37a8); Time taken: 0.004 seconds 2018-01-05T11:28:37,066 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:37,066 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,066 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,066 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,066 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query DROP TABLE IF EXISTS dest4_sequencefile 2018-01-05T11:28:37,066 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.006 seconds 2018-01-05T11:28:37,066 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,066 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:37,066 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,066 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:37,066 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,066 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:37,066 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,067 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,067 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,067 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS dest_j1 2018-01-05T11:28:37,067 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112837_ac271151-efb3-4118-a1cc-9214b8b3f538): DROP TABLE IF EXISTS dest_j1 2018-01-05T11:28:37,067 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,067 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS dest_j1 2018-01-05T11:28:37,067 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:37,067 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,067 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,068 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,068 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,068 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,068 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,068 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest_j1 2018-01-05T11:28:37,068 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest_j1 2018-01-05T11:28:37,068 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,069 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,069 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:37,069 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,069 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:37,069 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,069 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:37,069 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:28:37,069 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112837_ac271151-efb3-4118-a1cc-9214b8b3f538); Time taken: 0.002 seconds 2018-01-05T11:28:37,069 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,069 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,069 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112837_ac271151-efb3-4118-a1cc-9214b8b3f538): DROP TABLE IF EXISTS dest_j1 2018-01-05T11:28:37,070 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,070 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS dest_j1 2018-01-05T11:28:37,070 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:37,070 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,070 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,070 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,070 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,070 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:37,071 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,071 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest_j1 2018-01-05T11:28:37,071 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest_j1 2018-01-05T11:28:37,072 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,072 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,072 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table dest_j1 not found: default.dest_j1 table not found 2018-01-05T11:28:37,072 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,072 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest_j1 2018-01-05T11:28:37,072 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest_j1 2018-01-05T11:28:37,072 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,072 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,073 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,073 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,073 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS dest_j1 2018-01-05T11:28:37,073 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:37,073 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,073 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,073 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:37,073 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=1, dropTable_(String, String, boolean, boolean, boolean, )=1} 2018-01-05T11:28:37,073 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112837_ac271151-efb3-4118-a1cc-9214b8b3f538); Time taken: 0.004 seconds 2018-01-05T11:28:37,073 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:37,073 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,073 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,073 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,073 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query DROP TABLE IF EXISTS dest_j1 2018-01-05T11:28:37,073 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.007 seconds 2018-01-05T11:28:37,073 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,073 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:37,073 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,073 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:37,073 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,073 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:37,073 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,073 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,073 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,073 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS dest_g1 2018-01-05T11:28:37,073 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112837_32cac2f7-255d-4ac2-9f71-886acab4dc00): DROP TABLE IF EXISTS dest_g1 2018-01-05T11:28:37,074 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,074 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS dest_g1 2018-01-05T11:28:37,074 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:37,074 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,074 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,074 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,075 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,075 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest_g1 2018-01-05T11:28:37,075 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest_g1 2018-01-05T11:28:37,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,076 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:37,076 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,076 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:37,076 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,076 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:37,076 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=1} 2018-01-05T11:28:37,076 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112837_32cac2f7-255d-4ac2-9f71-886acab4dc00); Time taken: 0.003 seconds 2018-01-05T11:28:37,076 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,076 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,076 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112837_32cac2f7-255d-4ac2-9f71-886acab4dc00): DROP TABLE IF EXISTS dest_g1 2018-01-05T11:28:37,077 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,077 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS dest_g1 2018-01-05T11:28:37,077 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:37,077 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,077 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,077 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,077 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,077 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:37,078 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,078 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest_g1 2018-01-05T11:28:37,078 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest_g1 2018-01-05T11:28:37,078 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,079 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,079 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table dest_g1 not found: default.dest_g1 table not found 2018-01-05T11:28:37,079 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,079 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest_g1 2018-01-05T11:28:37,079 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest_g1 2018-01-05T11:28:37,079 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,080 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,080 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,080 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,080 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS dest_g1 2018-01-05T11:28:37,080 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:37,080 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,080 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,080 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:37,080 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=1, dropTable_(String, String, boolean, boolean, boolean, )=1} 2018-01-05T11:28:37,080 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112837_32cac2f7-255d-4ac2-9f71-886acab4dc00); Time taken: 0.004 seconds 2018-01-05T11:28:37,080 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:37,080 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,080 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,080 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,080 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query DROP TABLE IF EXISTS dest_g1 2018-01-05T11:28:37,080 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.007 seconds 2018-01-05T11:28:37,080 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,080 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:37,080 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,080 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:37,081 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,081 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:37,081 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,081 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,081 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,081 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS dest_g2 2018-01-05T11:28:37,081 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112837_4ea14c73-582f-45d8-ba4c-bf9073514300): DROP TABLE IF EXISTS dest_g2 2018-01-05T11:28:37,081 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,081 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS dest_g2 2018-01-05T11:28:37,081 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:37,081 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,081 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,081 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,081 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,081 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,081 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,081 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest_g2 2018-01-05T11:28:37,081 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest_g2 2018-01-05T11:28:37,082 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,082 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,082 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:37,082 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,082 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:37,082 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,082 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:37,082 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:28:37,082 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112837_4ea14c73-582f-45d8-ba4c-bf9073514300); Time taken: 0.001 seconds 2018-01-05T11:28:37,082 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,082 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,082 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112837_4ea14c73-582f-45d8-ba4c-bf9073514300): DROP TABLE IF EXISTS dest_g2 2018-01-05T11:28:37,083 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,083 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS dest_g2 2018-01-05T11:28:37,083 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:37,083 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,083 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,083 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,083 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,083 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:37,084 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,084 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest_g2 2018-01-05T11:28:37,084 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest_g2 2018-01-05T11:28:37,084 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,084 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,084 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table dest_g2 not found: default.dest_g2 table not found 2018-01-05T11:28:37,085 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,085 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest_g2 2018-01-05T11:28:37,085 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest_g2 2018-01-05T11:28:37,086 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,086 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,086 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,086 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,086 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS dest_g2 2018-01-05T11:28:37,086 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:37,086 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,086 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,086 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:37,086 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=1, dropTable_(String, String, boolean, boolean, boolean, )=1} 2018-01-05T11:28:37,086 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112837_4ea14c73-582f-45d8-ba4c-bf9073514300); Time taken: 0.004 seconds 2018-01-05T11:28:37,086 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:37,086 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,086 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,087 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,087 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query DROP TABLE IF EXISTS dest_g2 2018-01-05T11:28:37,087 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.006 seconds 2018-01-05T11:28:37,087 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,087 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:37,087 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,087 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:37,087 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,087 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:37,087 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,087 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,087 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,087 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS fetchtask_ioexception 2018-01-05T11:28:37,087 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112837_20de1342-1661-4e33-a206-27893e05909c): DROP TABLE IF EXISTS fetchtask_ioexception 2018-01-05T11:28:37,087 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,087 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS fetchtask_ioexception 2018-01-05T11:28:37,088 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:37,088 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,088 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,088 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,088 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,088 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,088 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,088 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=fetchtask_ioexception 2018-01-05T11:28:37,088 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=fetchtask_ioexception 2018-01-05T11:28:37,089 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,089 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,089 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:37,089 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,089 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:37,089 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,089 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:37,089 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:28:37,089 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112837_20de1342-1661-4e33-a206-27893e05909c); Time taken: 0.002 seconds 2018-01-05T11:28:37,089 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,089 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,089 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112837_20de1342-1661-4e33-a206-27893e05909c): DROP TABLE IF EXISTS fetchtask_ioexception 2018-01-05T11:28:37,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,090 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS fetchtask_ioexception 2018-01-05T11:28:37,090 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:37,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,090 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:37,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,090 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=fetchtask_ioexception 2018-01-05T11:28:37,090 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=fetchtask_ioexception 2018-01-05T11:28:37,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,091 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table fetchtask_ioexception not found: default.fetchtask_ioexception table not found 2018-01-05T11:28:37,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,091 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=fetchtask_ioexception 2018-01-05T11:28:37,091 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=fetchtask_ioexception 2018-01-05T11:28:37,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,092 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS fetchtask_ioexception 2018-01-05T11:28:37,092 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:37,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,092 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:37,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=1} 2018-01-05T11:28:37,092 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112837_20de1342-1661-4e33-a206-27893e05909c); Time taken: 0.003 seconds 2018-01-05T11:28:37,092 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:37,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,092 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query DROP TABLE IF EXISTS fetchtask_ioexception 2018-01-05T11:28:37,092 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.005 seconds 2018-01-05T11:28:37,092 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,092 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:37,092 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,092 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:37,093 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,093 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:37,093 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,093 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,093 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,093 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS alltypesorc 2018-01-05T11:28:37,093 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112837_4d33cb4b-fc84-40ec-86e7-abe041888a04): DROP TABLE IF EXISTS alltypesorc 2018-01-05T11:28:37,093 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,093 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS alltypesorc 2018-01-05T11:28:37,093 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:37,093 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,093 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,094 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,094 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesorc 2018-01-05T11:28:37,094 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesorc 2018-01-05T11:28:37,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,095 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:37,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,095 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:37,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,095 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:37,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:28:37,095 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112837_4d33cb4b-fc84-40ec-86e7-abe041888a04); Time taken: 0.002 seconds 2018-01-05T11:28:37,095 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,095 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112837_4d33cb4b-fc84-40ec-86e7-abe041888a04): DROP TABLE IF EXISTS alltypesorc 2018-01-05T11:28:37,096 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,096 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS alltypesorc 2018-01-05T11:28:37,096 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:37,096 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,096 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,096 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,096 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,096 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:37,097 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,097 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesorc 2018-01-05T11:28:37,097 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesorc 2018-01-05T11:28:37,097 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,097 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,097 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table alltypesorc not found: default.alltypesorc table not found 2018-01-05T11:28:37,097 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,097 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesorc 2018-01-05T11:28:37,098 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesorc 2018-01-05T11:28:37,098 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,098 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,098 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,098 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,098 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS alltypesorc 2018-01-05T11:28:37,098 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:37,098 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,098 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,099 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:37,099 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=1, dropTable_(String, String, boolean, boolean, boolean, )=1} 2018-01-05T11:28:37,099 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112837_4d33cb4b-fc84-40ec-86e7-abe041888a04); Time taken: 0.003 seconds 2018-01-05T11:28:37,099 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:37,099 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,099 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,099 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,099 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query DROP TABLE IF EXISTS alltypesorc 2018-01-05T11:28:37,099 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.006 seconds 2018-01-05T11:28:37,099 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,099 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:37,099 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,099 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:37,099 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,099 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:37,099 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,099 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,099 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,099 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS alltypesparquet 2018-01-05T11:28:37,099 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112837_6c8ffef2-6246-4259-9131-e85258a1d2fd): DROP TABLE IF EXISTS alltypesparquet 2018-01-05T11:28:37,099 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,099 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS alltypesparquet 2018-01-05T11:28:37,100 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:37,100 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,100 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,100 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,100 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,100 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,100 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,100 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesparquet 2018-01-05T11:28:37,100 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesparquet 2018-01-05T11:28:37,101 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,101 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,101 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:37,101 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,101 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:37,101 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,101 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:37,101 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:28:37,101 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112837_6c8ffef2-6246-4259-9131-e85258a1d2fd); Time taken: 0.002 seconds 2018-01-05T11:28:37,101 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,101 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,101 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112837_6c8ffef2-6246-4259-9131-e85258a1d2fd): DROP TABLE IF EXISTS alltypesparquet 2018-01-05T11:28:37,102 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,102 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS alltypesparquet 2018-01-05T11:28:37,102 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:37,102 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,102 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,102 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,102 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,102 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:37,102 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,102 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesparquet 2018-01-05T11:28:37,102 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesparquet 2018-01-05T11:28:37,103 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,103 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,103 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table alltypesparquet not found: default.alltypesparquet table not found 2018-01-05T11:28:37,103 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,103 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesparquet 2018-01-05T11:28:37,103 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesparquet 2018-01-05T11:28:37,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,104 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS alltypesparquet 2018-01-05T11:28:37,104 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:37,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,104 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:37,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=1} 2018-01-05T11:28:37,104 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112837_6c8ffef2-6246-4259-9131-e85258a1d2fd); Time taken: 0.003 seconds 2018-01-05T11:28:37,104 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:37,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,104 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query DROP TABLE IF EXISTS alltypesparquet 2018-01-05T11:28:37,104 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.005 seconds 2018-01-05T11:28:37,104 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,104 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:37,104 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,104 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:37,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:37,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS cbo_t1 2018-01-05T11:28:37,104 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112837_0b357d58-eda9-4554-96e9-7a17cde1a3b6): DROP TABLE IF EXISTS cbo_t1 2018-01-05T11:28:37,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS cbo_t1 2018-01-05T11:28:37,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:37,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,105 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,105 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t1 2018-01-05T11:28:37,105 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t1 2018-01-05T11:28:37,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,106 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:37,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,106 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:37,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,106 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:37,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:28:37,106 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112837_0b357d58-eda9-4554-96e9-7a17cde1a3b6); Time taken: 0.002 seconds 2018-01-05T11:28:37,106 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,106 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112837_0b357d58-eda9-4554-96e9-7a17cde1a3b6): DROP TABLE IF EXISTS cbo_t1 2018-01-05T11:28:37,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,107 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS cbo_t1 2018-01-05T11:28:37,107 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:37,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,107 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:37,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,107 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t1 2018-01-05T11:28:37,107 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t1 2018-01-05T11:28:37,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,108 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table cbo_t1 not found: default.cbo_t1 table not found 2018-01-05T11:28:37,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,108 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t1 2018-01-05T11:28:37,108 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t1 2018-01-05T11:28:37,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS cbo_t1 2018-01-05T11:28:37,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:37,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:37,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=0} 2018-01-05T11:28:37,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112837_0b357d58-eda9-4554-96e9-7a17cde1a3b6); Time taken: 0.003 seconds 2018-01-05T11:28:37,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:37,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query DROP TABLE IF EXISTS cbo_t1 2018-01-05T11:28:37,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.005 seconds 2018-01-05T11:28:37,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:37,109 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,109 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:37,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:37,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS cbo_t2 2018-01-05T11:28:37,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112837_45772aae-54cd-42d5-892e-e37473b8adfa): DROP TABLE IF EXISTS cbo_t2 2018-01-05T11:28:37,110 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,110 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS cbo_t2 2018-01-05T11:28:37,110 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:37,110 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,110 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,110 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,110 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,110 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,110 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,110 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t2 2018-01-05T11:28:37,110 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t2 2018-01-05T11:28:37,111 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,111 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,111 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:37,111 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,111 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:37,111 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,111 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:37,111 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:28:37,111 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112837_45772aae-54cd-42d5-892e-e37473b8adfa); Time taken: 0.002 seconds 2018-01-05T11:28:37,111 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,111 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,111 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112837_45772aae-54cd-42d5-892e-e37473b8adfa): DROP TABLE IF EXISTS cbo_t2 2018-01-05T11:28:37,112 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,112 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS cbo_t2 2018-01-05T11:28:37,112 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:37,112 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,112 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,112 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,112 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,112 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:37,113 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,113 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t2 2018-01-05T11:28:37,113 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t2 2018-01-05T11:28:37,113 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,113 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,113 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table cbo_t2 not found: default.cbo_t2 table not found 2018-01-05T11:28:37,113 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,113 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t2 2018-01-05T11:28:37,113 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t2 2018-01-05T11:28:37,114 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,114 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,114 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,114 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,114 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS cbo_t2 2018-01-05T11:28:37,114 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:37,114 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,114 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,114 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:37,114 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=1, dropTable_(String, String, boolean, boolean, boolean, )=1} 2018-01-05T11:28:37,114 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112837_45772aae-54cd-42d5-892e-e37473b8adfa); Time taken: 0.003 seconds 2018-01-05T11:28:37,114 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:37,114 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,114 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,114 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,114 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query DROP TABLE IF EXISTS cbo_t2 2018-01-05T11:28:37,114 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.005 seconds 2018-01-05T11:28:37,114 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,114 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:37,115 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,115 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:37,115 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,115 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:37,115 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,115 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,115 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,115 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS cbo_t3 2018-01-05T11:28:37,115 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112837_571511c2-3ff2-41de-b635-634249dda74b): DROP TABLE IF EXISTS cbo_t3 2018-01-05T11:28:37,115 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,115 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS cbo_t3 2018-01-05T11:28:37,115 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:37,115 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,115 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,116 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,116 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t3 2018-01-05T11:28:37,116 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t3 2018-01-05T11:28:37,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,116 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:37,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,116 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:37,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,116 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:37,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=1, flushCache_()=0} 2018-01-05T11:28:37,117 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112837_571511c2-3ff2-41de-b635-634249dda74b); Time taken: 0.001 seconds 2018-01-05T11:28:37,117 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,117 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112837_571511c2-3ff2-41de-b635-634249dda74b): DROP TABLE IF EXISTS cbo_t3 2018-01-05T11:28:37,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,118 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS cbo_t3 2018-01-05T11:28:37,118 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:37,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,118 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:37,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,118 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t3 2018-01-05T11:28:37,118 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t3 2018-01-05T11:28:37,119 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,119 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,119 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table cbo_t3 not found: default.cbo_t3 table not found 2018-01-05T11:28:37,119 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,119 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t3 2018-01-05T11:28:37,119 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t3 2018-01-05T11:28:37,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,120 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS cbo_t3 2018-01-05T11:28:37,120 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:37,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,120 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:37,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=1} 2018-01-05T11:28:37,120 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112837_571511c2-3ff2-41de-b635-634249dda74b); Time taken: 0.003 seconds 2018-01-05T11:28:37,120 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:37,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,120 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query DROP TABLE IF EXISTS cbo_t3 2018-01-05T11:28:37,120 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.005 seconds 2018-01-05T11:28:37,120 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,120 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:37,120 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,120 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:37,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:37,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS src_cbo 2018-01-05T11:28:37,120 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112837_aa59fb4e-d233-46df-8a88-1f2880ddfba3): DROP TABLE IF EXISTS src_cbo 2018-01-05T11:28:37,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS src_cbo 2018-01-05T11:28:37,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:37,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,121 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,121 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_cbo 2018-01-05T11:28:37,121 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_cbo 2018-01-05T11:28:37,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,122 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:37,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,122 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:37,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,122 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:37,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:28:37,122 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112837_aa59fb4e-d233-46df-8a88-1f2880ddfba3); Time taken: 0.002 seconds 2018-01-05T11:28:37,122 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,122 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112837_aa59fb4e-d233-46df-8a88-1f2880ddfba3): DROP TABLE IF EXISTS src_cbo 2018-01-05T11:28:37,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,123 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS src_cbo 2018-01-05T11:28:37,123 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:37,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,123 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:37,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,123 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_cbo 2018-01-05T11:28:37,123 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_cbo 2018-01-05T11:28:37,124 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,124 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,124 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table src_cbo not found: default.src_cbo table not found 2018-01-05T11:28:37,124 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,124 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_cbo 2018-01-05T11:28:37,124 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_cbo 2018-01-05T11:28:37,125 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,125 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,125 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,125 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,125 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS src_cbo 2018-01-05T11:28:37,125 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:37,125 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,125 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,125 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:37,125 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=1} 2018-01-05T11:28:37,125 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112837_aa59fb4e-d233-46df-8a88-1f2880ddfba3); Time taken: 0.003 seconds 2018-01-05T11:28:37,125 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:37,125 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,125 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,125 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,125 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query DROP TABLE IF EXISTS src_cbo 2018-01-05T11:28:37,125 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.005 seconds 2018-01-05T11:28:37,125 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,125 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:37,125 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,125 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:37,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:37,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS part 2018-01-05T11:28:37,126 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112837_698d4fa6-67b6-48c3-94df-3a93757f9364): DROP TABLE IF EXISTS part 2018-01-05T11:28:37,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS part 2018-01-05T11:28:37,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:37,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,127 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,127 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=part 2018-01-05T11:28:37,127 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=part 2018-01-05T11:28:37,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,128 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:37,128 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,128 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:37,128 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,128 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:37,128 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=1, flushCache_()=0} 2018-01-05T11:28:37,128 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112837_698d4fa6-67b6-48c3-94df-3a93757f9364); Time taken: 0.002 seconds 2018-01-05T11:28:37,128 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,128 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,128 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112837_698d4fa6-67b6-48c3-94df-3a93757f9364): DROP TABLE IF EXISTS part 2018-01-05T11:28:37,128 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,129 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS part 2018-01-05T11:28:37,129 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:37,129 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,129 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,129 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,129 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,129 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:37,129 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,129 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=part 2018-01-05T11:28:37,129 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=part 2018-01-05T11:28:37,130 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,130 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,130 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table part not found: default.part table not found 2018-01-05T11:28:37,130 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,130 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=part 2018-01-05T11:28:37,130 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=part 2018-01-05T11:28:37,130 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,131 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,131 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,131 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,131 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS part 2018-01-05T11:28:37,131 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:37,131 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,131 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,131 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:37,131 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=1} 2018-01-05T11:28:37,131 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112837_698d4fa6-67b6-48c3-94df-3a93757f9364); Time taken: 0.003 seconds 2018-01-05T11:28:37,131 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:37,131 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,131 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,131 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,131 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query DROP TABLE IF EXISTS part 2018-01-05T11:28:37,131 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.005 seconds 2018-01-05T11:28:37,131 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,131 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:37,131 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,131 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:37,131 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,131 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:37,131 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,131 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,131 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,131 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS lineitem 2018-01-05T11:28:37,131 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112837_3e81e013-ad14-4d2a-9efd-5a15fbd69c59): DROP TABLE IF EXISTS lineitem 2018-01-05T11:28:37,132 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,132 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS lineitem 2018-01-05T11:28:37,132 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:37,132 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,132 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,132 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,132 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,132 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,132 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,132 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=lineitem 2018-01-05T11:28:37,132 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=lineitem 2018-01-05T11:28:37,133 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,133 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,133 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:37,133 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,133 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:37,133 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,133 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:37,133 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:28:37,133 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112837_3e81e013-ad14-4d2a-9efd-5a15fbd69c59); Time taken: 0.002 seconds 2018-01-05T11:28:37,133 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,133 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,133 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112837_3e81e013-ad14-4d2a-9efd-5a15fbd69c59): DROP TABLE IF EXISTS lineitem 2018-01-05T11:28:37,134 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,134 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS lineitem 2018-01-05T11:28:37,134 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:37,134 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,134 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,134 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,134 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,134 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:37,135 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,135 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=lineitem 2018-01-05T11:28:37,135 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=lineitem 2018-01-05T11:28:37,135 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,135 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,135 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table lineitem not found: default.lineitem table not found 2018-01-05T11:28:37,135 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,135 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=lineitem 2018-01-05T11:28:37,136 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=lineitem 2018-01-05T11:28:37,136 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,136 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,136 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,136 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,136 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS lineitem 2018-01-05T11:28:37,136 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:37,136 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,136 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,136 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:37,136 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=1, dropTable_(String, String, boolean, boolean, boolean, )=1} 2018-01-05T11:28:37,136 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112837_3e81e013-ad14-4d2a-9efd-5a15fbd69c59); Time taken: 0.003 seconds 2018-01-05T11:28:37,136 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:37,136 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,136 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,136 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,136 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query DROP TABLE IF EXISTS lineitem 2018-01-05T11:28:37,136 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.005 seconds 2018-01-05T11:28:37,136 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,136 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:37,137 INFO [main] control.CoreCliDriver: Initialization cleanup done. ElapsedTime(ms)=1209 2018-01-05T11:28:37,139 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,139 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:37,142 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: /Users/djaiswal/hive/hive/itests/qtest/../..//data/files 2018-01-05T11:28:37,146 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,146 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:37,147 INFO [main] QTestUtil: Initial setup (/Users/djaiswal/hive/hive/itests/qtest/../..//data/scripts/q_test_init.sql): set hive.stats.dbclass=fs; -- -- Table src -- DROP TABLE IF EXISTS src; CREATE TABLE src (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS TEXTFILE; LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/kv1.txt" INTO TABLE src; ANALYZE TABLE src COMPUTE STATISTICS; ANALYZE TABLE src COMPUTE STATISTICS FOR COLUMNS key,value; -- -- Table src1 -- DROP TABLE IF EXISTS src1; CREATE TABLE src1 (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS TEXTFILE; LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/kv3.txt" INTO TABLE src1; ANALYZE TABLE src1 COMPUTE STATISTICS; ANALYZE TABLE src1 COMPUTE STATISTICS FOR COLUMNS key,value; -- -- Table src_json -- DROP TABLE IF EXISTS src_json; CREATE TABLE src_json (json STRING COMMENT 'default') STORED AS TEXTFILE; LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/json.txt" INTO TABLE src_json; ANALYZE TABLE src_json COMPUTE STATISTICS; ANALYZE TABLE src_json COMPUTE STATISTICS FOR COLUMNS json; -- -- Table src_sequencefile -- DROP TABLE IF EXISTS src_sequencefile; CREATE TABLE src_sequencefile (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS SEQUENCEFILE; LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/kv1.seq" INTO TABLE src_sequencefile; ANALYZE TABLE src_sequencefile COMPUTE STATISTICS; ANALYZE TABLE src_sequencefile COMPUTE STATISTICS FOR COLUMNS key,value; -- -- Table src_thrift -- DROP TABLE IF EXISTS src_thrift; CREATE TABLE src_thrift ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.thrift.ThriftDeserializer' WITH SERDEPROPERTIES ( 'serialization.class' = 'org.apache.hadoop.hive.serde2.thrift.test.Complex', 'serialization.format' = 'org.apache.thrift.protocol.TBinaryProtocol') STORED AS SEQUENCEFILE; LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/complex.seq" INTO TABLE src_thrift; ANALYZE TABLE src_thrift COMPUTE STATISTICS; -- -- Table srcbucket -- DROP TABLE IF EXISTS srcbucket_tmp; DROP TABLE IF EXISTS srcbucket; CREATE TABLE srcbucket_tmp (key INT, value STRING) STORED AS TEXTFILE; CREATE TABLE srcbucket (key INT, value STRING) CLUSTERED BY (key) INTO 2 BUCKETS STORED AS TEXTFILE; LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/srcbucket0.txt" INTO TABLE srcbucket_tmp; LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/srcbucket1.txt" INTO TABLE srcbucket_tmp; INSERT INTO srcbucket SELECT * FROM srcbucket_tmp; DROP TABLE srcbucket_tmp; ANALYZE TABLE srcbucket COMPUTE STATISTICS; ANALYZE TABLE srcbucket COMPUTE STATISTICS FOR COLUMNS key,value; -- -- Table srcbucket2 -- DROP TABLE IF EXISTS srcbucket_tmp; DROP TABLE IF EXISTS srcbucket2; CREATE TABLE srcbucket_tmp (key INT, value STRING); CREATE TABLE srcbucket2 (key INT, value STRING) CLUSTERED BY (key) INTO 4 BUCKETS STORED AS TEXTFILE; LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/srcbucket20.txt" INTO TABLE srcbucket_tmp; LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/srcbucket21.txt" INTO TABLE srcbucket_tmp; LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/srcbucket22.txt" INTO TABLE srcbucket_tmp; LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/srcbucket23.txt" INTO TABLE srcbucket_tmp; INSERT INTO srcbucket2 SELECT * FROM srcbucket_tmp; DROP TABLE srcbucket_tmp; ANALYZE TABLE srcbucket2 COMPUTE STATISTICS; ANALYZE TABLE srcbucket2 COMPUTE STATISTICS FOR COLUMNS key,value; -- -- Table srcpart -- DROP TABLE IF EXISTS srcpart; CREATE TABLE srcpart (key STRING COMMENT 'default', value STRING COMMENT 'default') PARTITIONED BY (ds STRING, hr STRING) STORED AS TEXTFILE; LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/kv1.txt" OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-08", hr="11"); LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/kv1.txt" OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-08", hr="12"); LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/kv1.txt" OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-09", hr="11"); LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/kv1.txt" OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-09", hr="12"); ANALYZE TABLE srcpart PARTITION(ds, hr) COMPUTE STATISTICS; ANALYZE TABLE srcpart PARTITION(ds, hr) COMPUTE STATISTICS FOR COLUMNS key,value; -- -- Table alltypesorc -- DROP TABLE IF EXISTS alltypesorc; CREATE TABLE alltypesorc( ctinyint TINYINT, csmallint SMALLINT, cint INT, cbigint BIGINT, cfloat FLOAT, cdouble DOUBLE, cstring1 STRING, cstring2 STRING, ctimestamp1 TIMESTAMP, ctimestamp2 TIMESTAMP, cboolean1 BOOLEAN, cboolean2 BOOLEAN) STORED AS ORC; LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/alltypesorc" OVERWRITE INTO TABLE alltypesorc; ANALYZE TABLE alltypesorc COMPUTE STATISTICS; ANALYZE TABLE alltypesorc COMPUTE STATISTICS FOR COLUMNS ctinyint,csmallint,cint,cbigint,cfloat,cdouble,cstring1,cstring2,ctimestamp1,ctimestamp2,cboolean1,cboolean2; -- -- Table alltypesparquet -- DROP TABLE IF EXISTS alltypesparquet; CREATE TABLE alltypesparquet( ctinyint TINYINT, csmallint SMALLINT, cint INT, cbigint BIGINT, cfloat FLOAT, cdouble DOUBLE, cstring1 STRING, cstring2 STRING, ctimestamp1 TIMESTAMP, ctimestamp2 TIMESTAMP, cboolean1 BOOLEAN, cboolean2 BOOLEAN) STORED AS PARQUET; INSERT OVERWRITE TABLE alltypesparquet SELECT * FROM alltypesorc; ANALYZE TABLE alltypesparquet COMPUTE STATISTICS; ANALYZE TABLE alltypesparquet COMPUTE STATISTICS FOR COLUMNS ctinyint,csmallint,cint,cbigint,cfloat,cdouble,cstring1,cstring2,ctimestamp1,ctimestamp2,cboolean1,cboolean2; -- -- Table primitives -- DROP TABLE IF EXISTS primitives; CREATE TABLE primitives ( id INT COMMENT 'default', bool_col BOOLEAN COMMENT 'default', tinyint_col TINYINT COMMENT 'default', smallint_col SMALLINT COMMENT 'default', int_col INT COMMENT 'default', bigint_col BIGINT COMMENT 'default', float_col FLOAT COMMENT 'default', double_col DOUBLE COMMENT 'default', date_string_col STRING COMMENT 'default', string_col STRING COMMENT 'default', timestamp_col TIMESTAMP COMMENT 'default') PARTITIONED BY (year INT COMMENT 'default', month INT COMMENT 'default') ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' ESCAPED BY '\\' STORED AS TEXTFILE; LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/types/primitives/090101.txt" OVERWRITE INTO TABLE primitives PARTITION(year=2009, month=1); LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/types/primitives/090201.txt" OVERWRITE INTO TABLE primitives PARTITION(year=2009, month=2); LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/types/primitives/090301.txt" OVERWRITE INTO TABLE primitives PARTITION(year=2009, month=3); LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/types/primitives/090401.txt" OVERWRITE INTO TABLE primitives PARTITION(year=2009, month=4); -- -- Function qtest_get_java_boolean -- DROP FUNCTION IF EXISTS qtest_get_java_boolean; CREATE FUNCTION qtest_get_java_boolean AS 'org.apache.hadoop.hive.ql.udf.generic.GenericUDFTestGetJavaBoolean'; -- -- Table dest1 -- DROP TABLE IF EXISTS dest1; CREATE TABLE dest1 (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'; -- -- Table dest2 -- DROP TABLE IF EXISTS dest2; CREATE TABLE dest2 (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'; -- -- Table dest3 -- DROP TABLE IF EXISTS dest3; CREATE TABLE dest3 (key STRING COMMENT 'default', value STRING COMMENT 'default') PARTITIONED BY (ds STRING, hr STRING) STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'; ALTER TABLE dest3 ADD PARTITION (ds='2008-04-08',hr='12'); -- -- Table dest4 -- DROP TABLE IF EXISTS dest4; CREATE TABLE dest4 (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'; -- -- Table dest4_sequencefile -- DROP TABLE IF EXISTS dest4_sequencefile; CREATE TABLE dest4_sequencefile (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.SequenceFileInputFormat' OUTPUTFORMAT 'org.apache.hadoop.mapred.SequenceFileOutputFormat'; -- -- CBO tables -- drop table if exists cbo_t1; drop table if exists cbo_t2; drop table if exists cbo_t3; drop table if exists src_cbo; drop table if exists part; drop table if exists lineitem; set hive.cbo.enable=true; create table cbo_t1(key string, value string, c_int int, c_float float, c_boolean boolean) partitioned by (dt string) row format delimited fields terminated by ',' STORED AS TEXTFILE; create table cbo_t2(key string, value string, c_int int, c_float float, c_boolean boolean) partitioned by (dt string) row format delimited fields terminated by ',' STORED AS TEXTFILE; create table cbo_t3(key string, value string, c_int int, c_float float, c_boolean boolean) row format delimited fields terminated by ',' STORED AS TEXTFILE; load data local inpath '${hiveconf:test.data.dir}/cbo_t1.txt' into table cbo_t1 partition (dt='2014'); load data local inpath '${hiveconf:test.data.dir}/cbo_t2.txt' into table cbo_t2 partition (dt='2014'); load data local inpath '${hiveconf:test.data.dir}/cbo_t3.txt' into table cbo_t3; CREATE TABLE part( p_partkey INT, p_name STRING, p_mfgr STRING, p_brand STRING, p_type STRING, p_size INT, p_container STRING, p_retailprice DOUBLE, p_comment STRING ); LOAD DATA LOCAL INPATH '${hiveconf:test.data.dir}/part_tiny.txt' overwrite into table part; CREATE TABLE lineitem (L_ORDERKEY INT, L_PARTKEY INT, L_SUPPKEY INT, L_LINENUMBER INT, L_QUANTITY DOUBLE, L_EXTENDEDPRICE DOUBLE, L_DISCOUNT DOUBLE, L_TAX DOUBLE, L_RETURNFLAG STRING, L_LINESTATUS STRING, l_shipdate STRING, L_COMMITDATE STRING, L_RECEIPTDATE STRING, L_SHIPINSTRUCT STRING, L_SHIPMODE STRING, L_COMMENT STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY '|'; LOAD DATA LOCAL INPATH '${hiveconf:test.data.dir}/lineitem.txt' OVERWRITE INTO TABLE lineitem; create table src_cbo as select * from src; analyze table cbo_t1 partition (dt) compute statistics; analyze table cbo_t1 compute statistics for columns key, value, c_int, c_float, c_boolean; analyze table cbo_t2 partition (dt) compute statistics; analyze table cbo_t2 compute statistics for columns key, value, c_int, c_float, c_boolean; analyze table cbo_t3 compute statistics; analyze table cbo_t3 compute statistics for columns key, value, c_int, c_float, c_boolean; analyze table src_cbo compute statistics; analyze table src_cbo compute statistics for columns; analyze table part compute statistics; analyze table part compute statistics for columns; analyze table lineitem compute statistics; analyze table lineitem compute statistics for columns; reset; set hive.stats.dbclass=fs; 2018-01-05T11:28:37,161 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,161 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:37,162 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: fs 2018-01-05T11:28:37,162 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,162 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:37,162 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,162 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:37,162 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,162 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:37,162 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,162 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,162 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,162 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: -- -- Table src -- DROP TABLE IF EXISTS src 2018-01-05T11:28:37,162 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112837_bbf22a4b-6f6e-4303-9534-7fb8564b05e4): -- -- Table src -- DROP TABLE IF EXISTS src 2018-01-05T11:28:37,162 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,162 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: -- -- Table src -- DROP TABLE IF EXISTS src 2018-01-05T11:28:37,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:37,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,163 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,163 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src 2018-01-05T11:28:37,163 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src 2018-01-05T11:28:37,164 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,164 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,164 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:37,164 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,164 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:37,164 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,164 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:37,164 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:28:37,164 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112837_bbf22a4b-6f6e-4303-9534-7fb8564b05e4); Time taken: 0.002 seconds 2018-01-05T11:28:37,164 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,164 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,164 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112837_bbf22a4b-6f6e-4303-9534-7fb8564b05e4): -- -- Table src -- DROP TABLE IF EXISTS src 2018-01-05T11:28:37,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,166 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: -- -- Table src -- DROP TABLE IF EXISTS src 2018-01-05T11:28:37,166 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:37,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,166 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:37,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,166 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src 2018-01-05T11:28:37,166 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src 2018-01-05T11:28:37,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,167 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table src not found: default.src table not found 2018-01-05T11:28:37,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,167 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src 2018-01-05T11:28:37,167 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src 2018-01-05T11:28:37,168 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,168 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,168 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,168 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,168 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: -- -- Table src -- DROP TABLE IF EXISTS src 2018-01-05T11:28:37,168 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:37,168 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,168 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,168 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:37,168 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=1} 2018-01-05T11:28:37,168 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112837_bbf22a4b-6f6e-4303-9534-7fb8564b05e4); Time taken: 0.004 seconds 2018-01-05T11:28:37,168 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:37,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,169 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query -- -- Table src -- DROP TABLE IF EXISTS src 2018-01-05T11:28:37,169 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.007 seconds 2018-01-05T11:28:37,169 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,169 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:37,169 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,169 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:37,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:37,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: CREATE TABLE src (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS TEXTFILE 2018-01-05T11:28:37,169 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112837_7c03afdc-3faf-49ac-aae6-a51772778e4f): CREATE TABLE src (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS TEXTFILE 2018-01-05T11:28:37,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: CREATE TABLE src (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS TEXTFILE 2018-01-05T11:28:37,171 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:37,171 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,171 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,176 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,176 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,176 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,181 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:28:37,187 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Creating table default.src position=13 2018-01-05T11:28:37,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,203 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_database: default 2018-01-05T11:28:37,203 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_database: default 2018-01-05T11:28:37,216 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:37,217 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,217 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,242 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] sqlstd.SQLStdHiveAccessController: Created SQLStdHiveAccessController for session context : HiveAuthzSessionContext [sessionString=53565c10-a9a1-4685-81b4-afcf82ce77fb, clientType=HIVECLI] 2018-01-05T11:28:37,244 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: METASTORE_FILTER_HOOK will be ignored, since hive.security.authorization.manager is set to instance of HiveAuthorizerFactory. 2018-01-05T11:28:37,244 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStoreClient: Mestastore configuration metastore.filter.hook changed from org.apache.hadoop.hive.metastore.DefaultMetaStoreFilterHookImpl to org.apache.hadoop.hive.ql.security.authorization.plugin.AuthorizationMetaStoreFilterHook 2018-01-05T11:28:37,244 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@50ac1249, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:37,244 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:37,244 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,244 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:37,244 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:37,245 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:37,245 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:37,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:37,247 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,248 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:37,248 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:37,248 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:37,248 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:37,248 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,248 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,248 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,250 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,251 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,251 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:37,251 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:37,251 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,251 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:37,253 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Session is using authorization class class org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAuthorizerImpl 2018-01-05T11:28:37,256 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:37,256 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: validation start 2018-01-05T11:28:37,256 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:28:37,256 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Not a partition. 2018-01-05T11:28:37,261 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,261 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:37,261 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,261 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:37,261 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {} 2018-01-05T11:28:37,262 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112837_7c03afdc-3faf-49ac-aae6-a51772778e4f); Time taken: 0.092 seconds 2018-01-05T11:28:37,262 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,262 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,262 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112837_7c03afdc-3faf-49ac-aae6-a51772778e4f): CREATE TABLE src (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS TEXTFILE 2018-01-05T11:28:37,263 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,264 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: CREATE TABLE src (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS TEXTFILE 2018-01-05T11:28:37,264 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: CREATETABLE 2018-01-05T11:28:37,264 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: database:default 2018-01-05T11:28:37,264 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@src 2018-01-05T11:28:37,264 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,264 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,264 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,265 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,265 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:37,265 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Found class for org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe 2018-01-05T11:28:37,294 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: creating table default.src on file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src 2018-01-05T11:28:37,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,339 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: create_table: Table(tableName:src, dbName:default, owner:hive_test_user, createTime:1515180517, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:string, comment:default), FieldSchema(name:value, type:string, comment:default)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, numFiles=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:37,339 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=create_table: Table(tableName:src, dbName:default, owner:hive_test_user, createTime:1515180517, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:string, comment:default), FieldSchema(name:value, type:string, comment:default)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, numFiles=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:37,340 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hive-site.xml 2018-01-05T11:28:37,341 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hivemetastore-site.xml 2018-01-05T11:28:37,341 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Unable to find config file metastore-site.xml 2018-01-05T11:28:37,341 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file null 2018-01-05T11:28:37,359 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:37,360 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: Location: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src specified for non-external table:src 2018-01-05T11:28:37,363 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src 2018-01-05T11:28:37,389 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,399 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,405 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,418 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,459 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit, isolation) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,467 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,477 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,484 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,484 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@src) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@src) Type=TABLE WriteType=DDL_NO_LOCK is present 2018-01-05T11:28:37,484 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,484 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,484 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: CREATE TABLE src (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS TEXTFILE 2018-01-05T11:28:37,484 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: CREATETABLE 2018-01-05T11:28:37,484 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: database:default 2018-01-05T11:28:37,484 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@src 2018-01-05T11:28:37,484 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,484 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,484 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:37,484 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {createTable_(Table, )=145, isCompatibleWith_(Configuration, )=0} 2018-01-05T11:28:37,484 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112837_7c03afdc-3faf-49ac-aae6-a51772778e4f); Time taken: 0.222 seconds 2018-01-05T11:28:37,484 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:37,484 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,484 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query CREATE TABLE src (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS TEXTFILE 2018-01-05T11:28:37,485 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.316 seconds 2018-01-05T11:28:37,485 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,485 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:37,485 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,485 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:37,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:37,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/kv1.txt" INTO TABLE src 2018-01-05T11:28:37,485 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112837_1384ed17-085c-42a7-9722-eee56223293c): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv1.txt" INTO TABLE src 2018-01-05T11:28:37,486 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,486 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv1.txt" INTO TABLE src 2018-01-05T11:28:37,487 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:37,487 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,487 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,488 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,488 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,488 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,491 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.LoadSemanticAnalyzer: file@null@/Users/djaiswal/hive/hive/data/files/kv1.txt 2018-01-05T11:28:37,492 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,492 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src 2018-01-05T11:28:37,492 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src 2018-01-05T11:28:37,588 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,589 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,671 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:37,671 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,671 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:37,671 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,671 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:37,671 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=98, flushCache_()=0} 2018-01-05T11:28:37,671 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112837_1384ed17-085c-42a7-9722-eee56223293c); Time taken: 0.186 seconds 2018-01-05T11:28:37,672 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,672 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,672 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112837_1384ed17-085c-42a7-9722-eee56223293c): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv1.txt" INTO TABLE src 2018-01-05T11:28:37,673 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,673 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv1.txt" INTO TABLE src 2018-01-05T11:28:37,673 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: LOAD 2018-01-05T11:28:37,673 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/kv1.txt 2018-01-05T11:28:37,673 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@src 2018-01-05T11:28:37,673 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,673 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,673 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,673 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,673 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MOVE] in serial mode 2018-01-05T11:28:37,673 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@7582a16b, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:37,673 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:37,673 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,673 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:37,673 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:37,673 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:37,673 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:37,673 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,673 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:37,673 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Loading data to table default.src from file:/Users/djaiswal/hive/hive/data/files/kv1.txt 2018-01-05T11:28:37,674 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,675 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:37,675 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:37,675 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:37,675 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:37,675 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,675 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,675 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,679 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,680 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,680 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:37,680 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:37,680 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,680 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:37,680 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,680 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src 2018-01-05T11:28:37,680 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src 2018-01-05T11:28:37,689 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,690 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,693 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,693 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src 2018-01-05T11:28:37,693 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src 2018-01-05T11:28:37,700 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,701 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,701 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] FileOperations: moving file:/Users/djaiswal/hive/hive/data/files/kv1.txt to file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src (replace = KEEP_EXISTING) 2018-01-05T11:28:37,703 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:37,703 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:37,732 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Moved src: file:/Users/djaiswal/hive/hive/data/files/kv1.txt, to dest: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src/kv1.txt 2018-01-05T11:28:37,732 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,732 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=src newtbl=src 2018-01-05T11:28:37,732 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=src newtbl=src 2018-01-05T11:28:37,741 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,765 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,765 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@src) Type=TABLE WriteType=INSERT because WriteEntity(default@src) Type=TABLE WriteType=INSERT is present 2018-01-05T11:28:37,769 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-1:STATS] in serial mode 2018-01-05T11:28:37,769 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@628bcf2c, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:37,769 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:37,769 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,769 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:37,769 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:37,769 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:37,769 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:37,769 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,769 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:37,769 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:37,770 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,771 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:37,771 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:37,771 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:37,771 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:37,771 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,771 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,771 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,773 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,773 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,773 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:37,773 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:37,773 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,773 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:37,773 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,773 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src 2018-01-05T11:28:37,773 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src 2018-01-05T11:28:37,781 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,782 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,784 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=src newtbl=src 2018-01-05T11:28:37,784 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=src newtbl=src 2018-01-05T11:28:37,790 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,813 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,813 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,814 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.src stats: [numFiles=1, numRows=0, totalSize=5812, rawDataSize=0] 2018-01-05T11:28:37,814 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,814 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,814 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv1.txt" INTO TABLE src 2018-01-05T11:28:37,814 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: LOAD 2018-01-05T11:28:37,814 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/kv1.txt 2018-01-05T11:28:37,814 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@src 2018-01-05T11:28:37,814 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,814 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,814 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:37,814 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=30, getTable_(String, String, )=9} 2018-01-05T11:28:37,814 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112837_1384ed17-085c-42a7-9722-eee56223293c); Time taken: 0.142 seconds 2018-01-05T11:28:37,814 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:37,814 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,814 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,814 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,814 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv1.txt" INTO TABLE src 2018-01-05T11:28:37,814 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.329 seconds 2018-01-05T11:28:37,814 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,814 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:37,814 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:37,814 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:37,814 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,815 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:37,815 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,815 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,815 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,815 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: ANALYZE TABLE src COMPUTE STATISTICS 2018-01-05T11:28:37,815 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112837_e75d6d24-220d-47b6-b23e-4cad8b21dd5d): ANALYZE TABLE src COMPUTE STATISTICS 2018-01-05T11:28:37,815 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,815 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: ANALYZE TABLE src COMPUTE STATISTICS 2018-01-05T11:28:37,816 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:37,816 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,816 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,817 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,817 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,817 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:37,818 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Invoking analyze on original query 2018-01-05T11:28:37,818 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Starting Semantic Analysis 2018-01-05T11:28:37,822 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed phase 1 of Semantic Analysis 2018-01-05T11:28:37,822 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for source tables 2018-01-05T11:28:37,822 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,822 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src 2018-01-05T11:28:37,822 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src 2018-01-05T11:28:37,831 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,832 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,832 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,832 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src 2018-01-05T11:28:37,832 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src 2018-01-05T11:28:37,841 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:37,842 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:37,842 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for subqueries 2018-01-05T11:28:37,842 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for destination tables 2018-01-05T11:28:37,842 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed getting MetaData in Semantic Analysis 2018-01-05T11:28:37,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-37_815_7793031023986305103-1/-mr-10000 2018-01-05T11:28:37,895 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Table Plan for src TS[0] 2018-01-05T11:28:37,896 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Body Plan for Query Block null 2018-01-05T11:28:37,896 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Plan for Query Block null 2018-01-05T11:28:37,897 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Before logical optimization TS[0] 2018-01-05T11:28:37,934 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,934 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,934 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,950 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] lineage.Generator: Time taken for lineage transform=16 2018-01-05T11:28:37,950 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,950 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,951 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,951 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,952 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,952 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,953 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,953 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,965 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{src}key,value: string|{src}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src}block__offset__inside__file,INPUT__FILE__NAME: string|{src}input__file__name,ROW__ID: struct|{src}row__id,RAW__DATA__SIZE: bigint|{src}raw__data__size) 2018-01-05T11:28:37,965 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,965 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,967 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,967 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,974 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:28:37,974 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.PredicatePushDown: After PPD: TS[0] 2018-01-05T11:28:37,974 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,974 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,974 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{src}key,value: string|{src}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src}block__offset__inside__file,INPUT__FILE__NAME: string|{src}input__file__name,ROW__ID: struct|{src}row__id,RAW__DATA__SIZE: bigint|{src}raw__data__size) 2018-01-05T11:28:37,974 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,974 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,976 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,976 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,979 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,979 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{src}key,value: string|{src}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src}block__offset__inside__file,INPUT__FILE__NAME: string|{src}input__file__name,ROW__ID: struct|{src}row__id,RAW__DATA__SIZE: bigint|{src}raw__data__size) 2018-01-05T11:28:37,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,983 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,983 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,995 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,995 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,996 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,996 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,997 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,997 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,999 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:37,999 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,001 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,001 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,004 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,004 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,004 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,004 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,008 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,008 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,009 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,009 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,009 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,009 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,011 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,011 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,011 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,011 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,011 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,011 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: After logical optimization TS[0] 2018-01-05T11:28:38,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,026 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,026 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,033 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,034 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,038 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:38,038 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=src 2018-01-05T11:28:38,038 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=src 2018-01-05T11:28:38,046 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:38,053 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:38,056 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:38,058 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting stats (Num rows: 1 Data size: 58120 Basic stats: COMPLETE Column stats: COMPLETE) on TS[0] 2018-01-05T11:28:38,058 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (src): numRows: 1 dataSize: 58120 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:28:38,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on TS[0] 2018-01-05T11:28:38,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,069 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,069 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,072 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,072 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,076 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,076 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,076 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:38,076 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:28:38,076 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Cycle free: true 2018-01-05T11:28:38,076 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,076 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,079 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,079 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{src}key,value: string|{src}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src}block__offset__inside__file,INPUT__FILE__NAME: string|{src}input__file__name,ROW__ID: struct|{src}row__id,RAW__DATA__SIZE: bigint|{src}raw__data__size) 2018-01-05T11:28:38,079 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:28:38,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src of table src 2018-01-05T11:28:38,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src 2018-01-05T11:28:38,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:28:38,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,132 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:38,132 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:38,132 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:38,133 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:38,133 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:38,133 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:38,133 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:28:38,139 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping vectorization 2018-01-05T11:28:38,139 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:28:38,144 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapClusterStateForCompile: Creating cluster info for djaiswal:@llap_MiniLlapCluster 2018-01-05T11:28:38,145 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: llap mode: only 2018-01-05T11:28:38,149 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:28:38,149 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:38,149 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@2de96eba can be done in LLAP 2018-01-05T11:28:38,151 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,158 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed plan generation 2018-01-05T11:28:38,158 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:38,158 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validation start 2018-01-05T11:28:38,158 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Not a partition. 2018-01-05T11:28:38,158 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,158 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:src.key, type:string, comment:null), FieldSchema(name:src.value, type:string, comment:null)], properties:null) 2018-01-05T11:28:38,158 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,158 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:38,158 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=20, flushCache_()=0, getTableColumnStatistics_(String, String, List, )=18} 2018-01-05T11:28:38,158 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112837_e75d6d24-220d-47b6-b23e-4cad8b21dd5d); Time taken: 0.343 seconds 2018-01-05T11:28:38,158 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:38,158 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,158 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112837_e75d6d24-220d-47b6-b23e-4cad8b21dd5d): ANALYZE TABLE src COMPUTE STATISTICS 2018-01-05T11:28:38,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,160 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: ANALYZE TABLE src COMPUTE STATISTICS 2018-01-05T11:28:38,160 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:28:38,160 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@src 2018-01-05T11:28:38,160 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@src 2018-01-05T11:28:38,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,160 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Query ID = djaiswal_20180105112837_e75d6d24-220d-47b6-b23e-4cad8b21dd5d 2018-01-05T11:28:38,160 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Total jobs = 1 2018-01-05T11:28:38,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:38,172 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:28:38,172 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MAPRED] in serial mode 2018-01-05T11:28:38,211 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-37_815_7793031023986305103-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:28:38,257 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:28:38,260 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: QueueName: null nonDefaultUser: true defaultQueuePool: null hasInitialSessions: false 2018-01-05T11:28:38,270 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: Created new tez session for queue: null with session id: 9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b 2018-01-05T11:28:38,270 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112837_e75d6d24-220d-47b6-b23e-4cad8b21dd5d 2018-01-05T11:28:38,270 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Tez session hasn't been created yet. Opening session 2018-01-05T11:28:38,271 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionState: User of session id 9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b is djaiswal 2018-01-05T11:28:38,307 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Hive jar directory is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar 2018-01-05T11:28:38,560 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionState: Computed sha: ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4 for file: file:/Users/djaiswal/.m2/repository/org/apache/hive/hive-exec/3.0.0-SNAPSHOT/hive-exec-3.0.0-SNAPSHOT.jar of length: 35.98MB in 252 ms 2018-01-05T11:28:38,560 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionState: The destination file name for [file:/Users/djaiswal/.m2/repository/org/apache/hive/hive-exec/3.0.0-SNAPSHOT/hive-exec-3.0.0-SNAPSHOT.jar] is hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar 2018-01-05T11:28:38,560 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Localizing resource because it does not exist: file:/Users/djaiswal/.m2/repository/org/apache/hive/hive-exec/3.0.0-SNAPSHOT/hive-exec-3.0.0-SNAPSHOT.jar to dest: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar 2018-01-05T11:28:38,676 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Resource modification time: 1515180518000 for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar 2018-01-05T11:28:38,691 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Hive jar directory is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar 2018-01-05T11:28:38,693 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionState: Computed sha: 7d8409a01f976cf89949edfc540edc89a2cb77c63c7a4af2bb79cbbdccde8c1e for file: file:/Users/djaiswal/.m2/repository/org/apache/hive/hive-llap-tez/3.0.0-SNAPSHOT/hive-llap-tez-3.0.0-SNAPSHOT.jar of length: 120.85KB in 1 ms 2018-01-05T11:28:38,693 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionState: The destination file name for [file:/Users/djaiswal/.m2/repository/org/apache/hive/hive-llap-tez/3.0.0-SNAPSHOT/hive-llap-tez-3.0.0-SNAPSHOT.jar] is hive-llap-tez-3.0.0-SNAPSHOT-7d8409a01f976cf89949edfc540edc89a2cb77c63c7a4af2bb79cbbdccde8c1e.jar 2018-01-05T11:28:38,693 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Localizing resource because it does not exist: file:/Users/djaiswal/.m2/repository/org/apache/hive/hive-llap-tez/3.0.0-SNAPSHOT/hive-llap-tez-3.0.0-SNAPSHOT.jar to dest: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-llap-tez-3.0.0-SNAPSHOT-7d8409a01f976cf89949edfc540edc89a2cb77c63c7a4af2bb79cbbdccde8c1e.jar 2018-01-05T11:28:38,707 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Resource modification time: 1515180518000 for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-llap-tez-3.0.0-SNAPSHOT-7d8409a01f976cf89949edfc540edc89a2cb77c63c7a4af2bb79cbbdccde8c1e.jar 2018-01-05T11:28:38,708 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Hive jar directory is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar 2018-01-05T11:28:38,708 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionState: The destination file name for [file:/Users/djaiswal/.m2/repository/org/apache/hive/hive-exec/3.0.0-SNAPSHOT/hive-exec-3.0.0-SNAPSHOT.jar] is hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar 2018-01-05T11:28:38,708 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Resource modification time: 1515180518000 for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar 2018-01-05T11:28:38,710 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Hive jar directory is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar 2018-01-05T11:28:38,710 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionState: The destination file name for [file:/Users/djaiswal/.m2/repository/org/apache/hive/hive-exec/3.0.0-SNAPSHOT/hive-exec-3.0.0-SNAPSHOT.jar] is hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar 2018-01-05T11:28:38,710 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Resource modification time: 1515180518000 for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar 2018-01-05T11:28:38,711 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Hive jar directory is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar 2018-01-05T11:28:38,712 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionState: Computed sha: 2645a277e1b05589cf0b6485454ab80c967932d8086e06025633ed70aa42b206 for file: file:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-registry/3.0.0-beta1/hadoop-yarn-registry-3.0.0-beta1.jar of length: 135.94KB in 1 ms 2018-01-05T11:28:38,713 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionState: The destination file name for [file:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-registry/3.0.0-beta1/hadoop-yarn-registry-3.0.0-beta1.jar] is hadoop-yarn-registry-3.0.0-beta1-2645a277e1b05589cf0b6485454ab80c967932d8086e06025633ed70aa42b206.jar 2018-01-05T11:28:38,713 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Localizing resource because it does not exist: file:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-registry/3.0.0-beta1/hadoop-yarn-registry-3.0.0-beta1.jar to dest: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hadoop-yarn-registry-3.0.0-beta1-2645a277e1b05589cf0b6485454ab80c967932d8086e06025633ed70aa42b206.jar 2018-01-05T11:28:38,725 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Resource modification time: 1515180518000 for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hadoop-yarn-registry-3.0.0-beta1-2645a277e1b05589cf0b6485454ab80c967932d8086e06025633ed70aa42b206.jar 2018-01-05T11:28:38,741 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionState: Config: mr(unset):mapreduce.task.io.sort.mb, mr initial value=10, tez(original):tez.runtime.io.sort.mb=null, tez(final):tez.runtime.io.sort.mb=10 2018-01-05T11:28:38,741 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionState: Config: mr(unset):mapreduce.reduce.shuffle.read.timeout, mr initial value=180000, tez(original):tez.runtime.shuffle.read.timeout=null, tez(final):tez.runtime.shuffle.read.timeout=180000 2018-01-05T11:28:38,741 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionState: Config: mr(unset):mapreduce.ifile.readahead.bytes, mr initial value=4194304, tez(original):tez.runtime.ifile.readahead.bytes=null, tez(final):tez.runtime.ifile.readahead.bytes=4194304 2018-01-05T11:28:38,741 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionState: Config: mr(unset):mapreduce.shuffle.ssl.enabled, mr initial value=false, tez(original):tez.runtime.shuffle.ssl.enable=null, tez(final):tez.runtime.shuffle.ssl.enable=false 2018-01-05T11:28:38,741 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionState: Config: mr(unset):mapreduce.map.sort.spill.percent, mr initial value=0.80, tez(original):tez.runtime.sort.spill.percent=null, tez(final):tez.runtime.sort.spill.percent=0.80 2018-01-05T11:28:38,741 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionState: Config: mr(unset):mapreduce.ifile.readahead, mr initial value=true, tez(original):tez.runtime.ifile.readahead=null, tez(final):tez.runtime.ifile.readahead=true 2018-01-05T11:28:38,741 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionState: Config: mr(unset):mapreduce.reduce.shuffle.merge.percent, mr initial value=0.66, tez(original):tez.runtime.shuffle.merge.percent=null, tez(final):tez.runtime.shuffle.merge.percent=0.66 2018-01-05T11:28:38,741 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionState: Config: mr(unset):mapreduce.reduce.shuffle.parallelcopies, mr initial value=5, tez(original):tez.runtime.shuffle.parallel.copies=null, tez(final):tez.runtime.shuffle.parallel.copies=5 2018-01-05T11:28:38,741 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionState: Config: mr(unset):mapreduce.job.reduce.slowstart.completedmaps, mr initial value=0.05, tez(original):tez.shuffle-vertex-manager.min-src-fraction=null, tez(final):tez.shuffle-vertex-manager.min-src-fraction=0.05 2018-01-05T11:28:38,741 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionState: Config: mr(unset):mapreduce.reduce.shuffle.memory.limit.percent, mr initial value=0.25, tez(original):tez.runtime.shuffle.memory.limit.percent=null, tez(final):tez.runtime.shuffle.memory.limit.percent=0.25 2018-01-05T11:28:38,741 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionState: Config: mr(unset):mapreduce.task.io.sort.factor, mr initial value=10, tez(original):tez.runtime.io.sort.factor=null, tez(final):tez.runtime.io.sort.factor=10 2018-01-05T11:28:38,741 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionState: Config: mr(unset):mapreduce.map.output.compress, mr initial value=false, tez(original):tez.runtime.compress=null, tez(final):tez.runtime.compress=false 2018-01-05T11:28:38,741 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionState: Config: mr(unset):mapreduce.reduce.shuffle.connect.timeout, mr initial value=180000, tez(original):tez.runtime.shuffle.connect.timeout=20000, tez(final):tez.runtime.shuffle.connect.timeout=20000 2018-01-05T11:28:38,741 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionState: Config: mr(unset):mapreduce.reduce.input.buffer.percent, mr initial value=0.0, tez(original):tez.runtime.task.input.post-merge.buffer.percent=null, tez(final):tez.runtime.task.input.post-merge.buffer.percent=0.0 2018-01-05T11:28:38,741 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionState: Config: mr(unset):mapreduce.map.output.compress.codec, mr initial value=org.apache.hadoop.io.compress.DefaultCodec, tez(original):tez.runtime.compress.codec=null, tez(final):tez.runtime.compress.codec=org.apache.hadoop.io.compress.DefaultCodec 2018-01-05T11:28:38,741 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionState: Config: mr(unset):mapreduce.task.merge.progress.records, mr initial value=10000, tez(original):tez.runtime.merge.progress.records=null, tez(final):tez.runtime.merge.progress.records=10000 2018-01-05T11:28:38,741 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionState: Config: mr(unset):map.sort.class, mr initial value=org.apache.hadoop.util.QuickSort, tez(original):tez.runtime.internal.sorter.class=null, tez(final):tez.runtime.internal.sorter.class=org.apache.hadoop.util.QuickSort 2018-01-05T11:28:38,741 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionState: Config: mr(unset):mapreduce.reduce.shuffle.input.buffer.percent, mr initial value=0.70, tez(original):tez.runtime.shuffle.fetch.buffer.percent=null, tez(final):tez.runtime.shuffle.fetch.buffer.percent=0.70 2018-01-05T11:28:38,741 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionState: Config: mr(unset):mapreduce.job.maxtaskfailures.per.tracker, mr initial value=3, tez(original):tez.am.maxtaskfailures.per.node=null, tez(final):tez.am.maxtaskfailures.per.node=3 2018-01-05T11:28:38,741 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionState: Config: mr(unset):mapreduce.task.timeout, mr initial value=600000, tez(original):tez.task.timeout-ms=null, tez(final):tez.task.timeout-ms=600000 2018-01-05T11:28:38,741 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionState: Config: mr(unset):yarn.app.mapreduce.am.job.node-blacklisting.enable, mr initial value=false, tez(original):tez.am.node-blacklisting.enabled=false, tez(final):tez.am.node-blacklisting.enabled=false 2018-01-05T11:28:38,741 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionState: Config: mr(unset):mapreduce.job.counters.max, mr initial value=120, tez(original):tez.counters.max=null, tez(final):tez.counters.max=120 2018-01-05T11:28:38,741 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionState: Config: mr(unset):mapreduce.job.queuename, mr initial value=default, tez(original):tez.queue.name=default, tez(final):tez.queue.name=default 2018-01-05T11:28:38,741 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionState: Config: mr(unset):yarn.app.mapreduce.am.job.task.listener.thread-count, mr initial value=30, tez(original):tez.am.task.listener.thread-count=null, tez(final):tez.am.task.listener.thread-count=30 2018-01-05T11:28:38,782 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionState: Setting Tez Session access for sessionId=53565c10-a9a1-4685-81b4-afcf82ce77fb with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:28:38,803 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Tez Client Version: [ component=tez-api, version=0.9.1-SNAPSHOT, revision=cfede26b054a8a0243e969ee0e62b25a6f542f10, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=2017-12-03T22:05:27Z ] 2018-01-05T11:28:38,803 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionState: Opening new Tez Session (id: 9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b) 2018-01-05T11:28:38,815 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Session mode. Starting session. 2018-01-05T11:28:38,818 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClientUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:38,898 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Tez system stage directory file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001 doesn't exist and is created 2018-01-05T11:28:38,918 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:39,080 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.LocalClient: DAGAppMaster thread has been created 2018-01-05T11:28:39,080 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.LocalClient: DAGAppMaster is not created wait for 100ms... 2018-01-05T11:28:39,081 INFO [DAGAppMaster Thread] client.LocalClient: Using working directory: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd 2018-01-05T11:28:39,184 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.LocalClient: DAGAppMaster is not created wait for 100ms... 2018-01-05T11:28:39,219 INFO [DAGAppMaster Thread] app.DAGAppMaster: Created DAGAppMaster for application appattempt_1515180518813_0001_000000, versionInfo=[ component=tez-dag, version=0.9.1-SNAPSHOT, revision=4c4bf99eb749e2db0f211710d3a625dc91588d14, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=2018-01-04T22:48:16Z ] 2018-01-05T11:28:39,221 DEBUG [DAGAppMaster Thread] service.AbstractService: Service: org.apache.tez.dag.app.DAGAppMaster entered state INITED 2018-01-05T11:28:39,225 INFO [DAGAppMaster Thread] shim.HadoopShimsLoader: Trying to locate HadoopShimProvider for hadoopVersion=3.0.0-beta1, majorVersion=3, minorVersion=0 2018-01-05T11:28:39,225 INFO [DAGAppMaster Thread] shim.HadoopShimsLoader: Picked HadoopShim org.apache.tez.hadoop.shim.DefaultHadoopShim, providerName=null, overrideProviderViaConfig=null, hadoopVersion=3.0.0-beta1, majorVersion=3, minorVersion=0 2018-01-05T11:28:39,232 INFO [DAGAppMaster Thread] app.DAGAppMaster: AM Level configured TaskSchedulers: [0:TezUber:null],[1:LLAP:org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService] 2018-01-05T11:28:39,232 INFO [DAGAppMaster Thread] app.DAGAppMaster: AM Level configured ContainerLaunchers: [0:TezUber:null],[1:LLAP:org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher] 2018-01-05T11:28:39,232 INFO [DAGAppMaster Thread] app.DAGAppMaster: AM Level configured TaskCommunicators: [0:TezUber:null],[1:LLAP:org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator] 2018-01-05T11:28:39,232 INFO [DAGAppMaster Thread] app.DAGAppMaster: Comparing client version with AM version, clientVersion=0.9.1-SNAPSHOT, AMVersion=0.9.1-SNAPSHOT 2018-01-05T11:28:39,238 DEBUG [DAGAppMaster Thread] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:39,238 DEBUG [DAGAppMaster Thread] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:39,238 DEBUG [DAGAppMaster Thread] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:39,238 DEBUG [DAGAppMaster Thread] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:39,264 INFO [DAGAppMaster Thread] app.TaskCommunicatorManager: Creating Default Local Task Communicator 2018-01-05T11:28:39,277 INFO [DAGAppMaster Thread] app.TaskCommunicatorManager: Creating TaskCommunicator org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator:{} LLAP 2018-01-05T11:28:39,287 INFO [DAGAppMaster Thread] tezplugins.LlapTaskCommunicator: Task communicator with a token null 2018-01-05T11:28:39,288 DEBUG [DAGAppMaster Thread] service.AbstractService: Service: LlapRegistryService entered state INITED 2018-01-05T11:28:39,289 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.LocalClient: DAGAppMaster state: NEW 2018-01-05T11:28:39,289 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.LocalClient: DAGAppMaster is not started wait for 100ms... 2018-01-05T11:28:39,289 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.LocalClient: DAGAppMaster is not created wait for 100ms... 2018-01-05T11:28:39,290 INFO [DAGAppMaster Thread] impl.LlapZookeeperRegistryImpl: Llap Zookeeper Registry is enabled with registryid: llap_MiniLlapCluster 2018-01-05T11:28:39,290 INFO [DAGAppMaster Thread] impl.LlapRegistryService: Using LLAP registry type org.apache.hadoop.hive.llap.registry.impl.LlapZookeeperRegistryImpl@32c1b9bb 2018-01-05T11:28:39,290 INFO [DAGAppMaster Thread] impl.ZookeeperUtils: UGI security is not enabled, or no SASL context name. Skipping setting up ZK auth. 2018-01-05T11:28:39,290 INFO [DAGAppMaster Thread] imps.CuratorFrameworkImpl: Starting 2018-01-05T11:28:39,290 DEBUG [DAGAppMaster Thread] curator.CuratorZookeeperClient: Starting 2018-01-05T11:28:39,290 DEBUG [DAGAppMaster Thread] curator.ConnectionState: Starting 2018-01-05T11:28:39,290 DEBUG [DAGAppMaster Thread] curator.ConnectionState: reset 2018-01-05T11:28:39,290 INFO [DAGAppMaster Thread] zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:49384 sessionTimeout=1200000 watcher=org.apache.curator.ConnectionState@1b8f34fc 2018-01-05T11:28:39,291 DEBUG [DAGAppMaster Thread] service.AbstractService: Service LlapRegistryService is started 2018-01-05T11:28:39,291 INFO [DAGAppMaster Thread] impl.LlapRegistryService: Using LLAP registry (client) type: Service LlapRegistryService in state LlapRegistryService: STARTED 2018-01-05T11:28:39,292 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:49384] server.NIOServerCnxnFactory: Accepted socket connection from /127.0.0.1:49822 2018-01-05T11:28:39,292 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:49384] server.ZooKeeperServer: Client attempting to establish new session at /127.0.0.1:49822 2018-01-05T11:28:39,293 INFO [SyncThread:0] server.ZooKeeperServer: Established session 0x160c7cbc1600002 with negotiated timeout 40000 for client /127.0.0.1:49822 2018-01-05T11:28:39,293 INFO [DAGAppMaster Thread-EventThread] state.ConnectionStateManager: State change: CONNECTED 2018-01-05T11:28:39,304 INFO [DAGAppMaster Thread] common.AsyncDispatcher: Registering class org.apache.tez.dag.app.rm.container.AMContainerEventType for class org.apache.tez.dag.app.rm.container.AMContainerMap 2018-01-05T11:28:39,306 INFO [DAGAppMaster Thread] common.AsyncDispatcher: Registering class org.apache.tez.dag.app.rm.node.AMNodeEventType for class org.apache.tez.dag.app.rm.node.AMNodeTracker 2018-01-05T11:28:39,308 INFO [DAGAppMaster Thread] common.AsyncDispatcher: Registering class org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler 2018-01-05T11:28:39,309 INFO [DAGAppMaster Thread] common.AsyncDispatcher: Registering class org.apache.tez.dag.app.dag.event.DAGEventType for class org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher 2018-01-05T11:28:39,309 INFO [DAGAppMaster Thread] common.AsyncDispatcher: Registering class org.apache.tez.dag.app.dag.event.VertexEventType for class org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher 2018-01-05T11:28:39,309 INFO [DAGAppMaster Thread] app.DAGAppMaster: Using concurrent dispatcher: false 2018-01-05T11:28:39,310 INFO [DAGAppMaster Thread] common.AsyncDispatcher: Registering class org.apache.tez.dag.app.dag.event.TaskEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher 2018-01-05T11:28:39,312 INFO [DAGAppMaster Thread] common.AsyncDispatcher: Registering class org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher 2018-01-05T11:28:39,313 INFO [DAGAppMaster Thread] common.AsyncDispatcher: Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for independent dispatch using: class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler 2018-01-05T11:28:39,313 INFO [DAGAppMaster Thread] common.AsyncDispatcher: Registering class org.apache.tez.dag.app.dag.event.SpeculatorEventType for class org.apache.tez.dag.app.DAGAppMaster$SpeculatorEventHandler 2018-01-05T11:28:39,313 DEBUG [DAGAppMaster Thread] service.CompositeService: Adding service Speculator 2018-01-05T11:28:39,323 INFO [DAGAppMaster Thread] common.AsyncDispatcher: Registering class org.apache.tez.dag.app.rm.AMSchedulerEventType for class org.apache.tez.dag.app.rm.TaskSchedulerManager 2018-01-05T11:28:39,330 INFO [DAGAppMaster Thread] launcher.TezContainerLauncherImpl: Creating LocalContainerLauncher 2018-01-05T11:28:39,341 INFO [DAGAppMaster Thread] launcher.TezContainerLauncherImpl: Creating container launcher LLAP:org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher 2018-01-05T11:28:39,343 INFO [DAGAppMaster Thread] common.AsyncDispatcher: Registering class org.apache.tez.dag.app.rm.ContainerLauncherEventType for class org.apache.tez.dag.app.launcher.ContainerLauncherManager 2018-01-05T11:28:39,346 DEBUG [DAGAppMaster Thread] service.AbstractService: Service: Central entered state INITED 2018-01-05T11:28:39,346 DEBUG [DAGAppMaster Thread] service.CompositeService: Central: initing services, size=1 2018-01-05T11:28:39,346 DEBUG [DAGAppMaster Thread] service.AbstractService: Service: Speculator entered state INITED 2018-01-05T11:28:39,346 DEBUG [DAGAppMaster Thread] service.CompositeService: Speculator: initing services, size=0 2018-01-05T11:28:39,347 DEBUG [DAGAppMaster Thread] service.AbstractService: Service: DAGClientRPCServer entered state INITED 2018-01-05T11:28:39,347 DEBUG [DAGAppMaster Thread] service.AbstractService: Service: TaskHeartbeatHandler entered state INITED 2018-01-05T11:28:39,347 DEBUG [DAGAppMaster Thread] service.AbstractService: Service: ContainerHeartbeatHandler entered state INITED 2018-01-05T11:28:39,347 DEBUG [DAGAppMaster Thread] service.AbstractService: Service: org.apache.tez.dag.app.TaskCommunicatorManager entered state INITED 2018-01-05T11:28:39,347 DEBUG [DAGAppMaster Thread] service.AbstractService: Service: AMContainerMaps entered state INITED 2018-01-05T11:28:39,347 DEBUG [DAGAppMaster Thread] service.AbstractService: Service: AMNodeTracker entered state INITED 2018-01-05T11:28:39,347 INFO [DAGAppMaster Thread] node.AMNodeTracker: blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 3, nodeUpdatesRescheduleEnabled: false 2018-01-05T11:28:39,347 DEBUG [DAGAppMaster Thread] service.AbstractService: Service: org.apache.tez.dag.app.rm.TaskSchedulerManager entered state INITED 2018-01-05T11:28:39,347 DEBUG [DAGAppMaster Thread] service.AbstractService: Service: org.apache.tez.dag.app.launcher.ContainerLauncherManager entered state INITED 2018-01-05T11:28:39,347 DEBUG [DAGAppMaster Thread] service.AbstractService: Service: org.apache.tez.dag.app.launcher.LocalContainerLauncher entered state INITED 2018-01-05T11:28:39,347 DEBUG [DAGAppMaster Thread] service.AbstractService: Service: org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher entered state INITED 2018-01-05T11:28:39,347 DEBUG [DAGAppMaster Thread] service.AbstractService: Service: org.apache.tez.dag.history.HistoryEventHandler entered state INITED 2018-01-05T11:28:39,347 INFO [DAGAppMaster Thread] history.HistoryEventHandler: Initializing HistoryEventHandler withrecoveryEnabled=true, historyServiceClassName=org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService 2018-01-05T11:28:39,350 DEBUG [DAGAppMaster Thread] service.CompositeService: Adding service org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService 2018-01-05T11:28:39,353 DEBUG [DAGAppMaster Thread] service.CompositeService: Adding service org.apache.tez.dag.history.recovery.RecoveryService 2018-01-05T11:28:39,354 DEBUG [DAGAppMaster Thread] service.CompositeService: org.apache.tez.dag.history.HistoryEventHandler: initing services, size=2 2018-01-05T11:28:39,354 DEBUG [DAGAppMaster Thread] service.AbstractService: Service: org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService entered state INITED 2018-01-05T11:28:39,354 INFO [DAGAppMaster Thread] impl.SimpleHistoryLoggingService: Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir 2018-01-05T11:28:39,355 INFO [DAGAppMaster Thread] impl.SimpleHistoryLoggingService: Initializing SimpleHistoryLoggingService, logFileLocation=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/history.txt.appattempt_1515180518813_0001_000000, maxErrors=10 2018-01-05T11:28:39,355 DEBUG [DAGAppMaster Thread] service.AbstractService: Service: org.apache.tez.dag.history.recovery.RecoveryService entered state INITED 2018-01-05T11:28:39,355 INFO [DAGAppMaster Thread] recovery.RecoveryService: RecoveryService initialized with recoveryPath=file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/recovery/0, bufferSize(bytes)=8192, flushInterval(s)=30, maxUnflushedEvents=100 2018-01-05T11:28:39,357 INFO [DAGAppMaster Thread] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1515180518813_0001_000000, appSubmitTime=1515180519189, launchTime=1515180519218 2018-01-05T11:28:39,360 DEBUG [ServiceThread:Central] service.CompositeService: Central: starting services, size=1 2018-01-05T11:28:39,360 DEBUG [ServiceThread:Central] service.CompositeService: Speculator: starting services, size=0 2018-01-05T11:28:39,360 DEBUG [ServiceThread:Central] service.AbstractService: Service Speculator is started 2018-01-05T11:28:39,360 DEBUG [ServiceThread:Central] service.AbstractService: Service Central is started 2018-01-05T11:28:39,361 DEBUG [ServiceThread:org.apache.tez.dag.app.TaskCommunicatorManager] service.AbstractService: Service: org.apache.tez.dag.app.TezLocalTaskCommunicatorImpl entered state INITED 2018-01-05T11:28:39,361 DEBUG [ServiceThread:AMNodeTracker] service.AbstractService: Service AMNodeTracker is started 2018-01-05T11:28:39,361 DEBUG [ServiceThread:AMContainerMaps] service.AbstractService: Service AMContainerMaps is started 2018-01-05T11:28:39,361 DEBUG [ServiceThread:org.apache.tez.dag.history.HistoryEventHandler] service.CompositeService: org.apache.tez.dag.history.HistoryEventHandler: starting services, size=2 2018-01-05T11:28:39,362 DEBUG [ServiceThread:org.apache.tez.dag.app.TaskCommunicatorManager] service.AbstractService: Service org.apache.tez.dag.app.TezLocalTaskCommunicatorImpl is started 2018-01-05T11:28:39,362 DEBUG [ServiceThread:org.apache.tez.dag.app.TaskCommunicatorManager] service.AbstractService: Service: org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator entered state INITED 2018-01-05T11:28:39,362 INFO [ServiceThread:org.apache.tez.dag.history.HistoryEventHandler] impl.SimpleHistoryLoggingService: Starting SimpleHistoryLoggingService 2018-01-05T11:28:39,363 DEBUG [ServiceThread:TaskHeartbeatHandler] service.AbstractService: Service TaskHeartbeatHandler is started 2018-01-05T11:28:39,363 DEBUG [ServiceThread:org.apache.tez.dag.app.launcher.ContainerLauncherManager] service.AbstractService: Service org.apache.tez.dag.app.launcher.LocalContainerLauncher is started 2018-01-05T11:28:39,364 DEBUG [ServiceThread:ContainerHeartbeatHandler] service.AbstractService: Service ContainerHeartbeatHandler is started 2018-01-05T11:28:39,364 DEBUG [ServiceThread:org.apache.tez.dag.app.launcher.ContainerLauncherManager] service.AbstractService: Service org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher is started 2018-01-05T11:28:39,365 DEBUG [ServiceThread:org.apache.tez.dag.app.launcher.ContainerLauncherManager] service.AbstractService: Service org.apache.tez.dag.app.launcher.ContainerLauncherManager is started 2018-01-05T11:28:39,371 INFO [ServiceThread:org.apache.tez.dag.app.TaskCommunicatorManager] llap.AsyncPbRpcProxy: Setting up AsyncPbRpcProxy withnumThreads=10retryTime(millis)=16000retrySleep(millis)=2000 2018-01-05T11:28:39,371 INFO [ServiceThread:org.apache.tez.dag.app.TaskCommunicatorManager] tezplugins.LlapTaskCommunicator: Running LlapTaskCommunicator with fileCleanupDelay=300, numCommunicatorThreads=10 2018-01-05T11:28:39,371 DEBUG [ServiceThread:org.apache.tez.dag.app.TaskCommunicatorManager] service.AbstractService: Service: LlapProtocolClientProxy entered state INITED 2018-01-05T11:28:39,385 DEBUG [ServiceThread:org.apache.tez.dag.history.HistoryEventHandler] service.AbstractService: Service org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService is started 2018-01-05T11:28:39,388 DEBUG [ServiceThread:org.apache.tez.dag.history.HistoryEventHandler] service.AbstractService: Service org.apache.tez.dag.history.recovery.RecoveryService is started 2018-01-05T11:28:39,389 DEBUG [ServiceThread:org.apache.tez.dag.history.HistoryEventHandler] service.AbstractService: Service org.apache.tez.dag.history.HistoryEventHandler is started 2018-01-05T11:28:39,393 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.LocalClient: DAGAppMaster state: INITED 2018-01-05T11:28:39,393 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.LocalClient: DAGAppMaster is not startetd wait for 100ms... 2018-01-05T11:28:39,393 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.LocalClient: DAGAppMaster is not created wait for 100ms... 2018-01-05T11:28:39,397 INFO [ServiceThread:DAGClientRPCServer] client.DAGClientServer: Instantiated DAGClientRPCServer at /10.22.8.182:49823 2018-01-05T11:28:39,397 DEBUG [ServiceThread:DAGClientRPCServer] service.AbstractService: Service DAGClientRPCServer is started 2018-01-05T11:28:39,401 INFO [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager] rm.TaskSchedulerManager: Creating TaskScheduler: Local TaskScheduler with clusterIdentifier=1515180518813 2018-01-05T11:28:39,402 INFO [ServiceThread:org.apache.tez.dag.app.TaskCommunicatorManager] tezplugins.LlapTaskCommunicator: Started LlapUmbilical: org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator$LlapTaskUmbilicalProtocolImpl at address: hw13500.local/10.22.8.182:49824 with numHandlers=30 using the host name 10.22.8.182 2018-01-05T11:28:39,402 DEBUG [ServiceThread:org.apache.tez.dag.app.TaskCommunicatorManager] service.AbstractService: Service LlapProtocolClientProxy is started 2018-01-05T11:28:39,403 DEBUG [ServiceThread:org.apache.tez.dag.app.TaskCommunicatorManager] service.AbstractService: Service org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator is started 2018-01-05T11:28:39,403 DEBUG [ServiceThread:org.apache.tez.dag.app.TaskCommunicatorManager] service.AbstractService: Service org.apache.tez.dag.app.TaskCommunicatorManager is started 2018-01-05T11:28:39,408 INFO [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager] rm.TaskSchedulerManager: Creating custom TaskScheduler LLAP:org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService with clusterIdentifier=111101111 2018-01-05T11:28:39,416 INFO [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager] tezplugins.LlapTaskSchedulerService: Running with configuration: hosts=@llap_MiniLlapCluster, numSchedulableTasksPerNode=0, nodeBlacklistConf=NodeBlacklistConf{minDelay=200, maxDelay=10000, backoffFactor=1.5}, localityConf=LocalityDelayConf{nodeLocalityDelay=0} 2018-01-05T11:28:39,420 INFO [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager] impl.TezAmRegistryImpl: AM Zookeeper Registry is enabled with registryid: llap 2018-01-05T11:28:39,420 DEBUG [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager] service.AbstractService: Service: org.apache.tez.dag.app.rm.LocalTaskSchedulerService entered state INITED 2018-01-05T11:28:39,422 DEBUG [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager] service.AbstractService: Service org.apache.tez.dag.app.rm.LocalTaskSchedulerService is started 2018-01-05T11:28:39,422 DEBUG [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager] service.AbstractService: Service: org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService entered state INITED 2018-01-05T11:28:39,422 DEBUG [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager] service.AbstractService: Service: LlapRegistryService entered state INITED 2018-01-05T11:28:39,424 INFO [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager] impl.LlapZookeeperRegistryImpl: Llap Zookeeper Registry is enabled with registryid: llap_MiniLlapCluster 2018-01-05T11:28:39,424 INFO [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager] impl.LlapRegistryService: Using LLAP registry type org.apache.hadoop.hive.llap.registry.impl.LlapZookeeperRegistryImpl@37b97e21 2018-01-05T11:28:39,426 INFO [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager] impl.ZookeeperUtils: UGI security is not enabled, or no SASL context name. Skipping setting up ZK auth. 2018-01-05T11:28:39,426 INFO [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager] imps.CuratorFrameworkImpl: Starting 2018-01-05T11:28:39,426 DEBUG [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager] curator.CuratorZookeeperClient: Starting 2018-01-05T11:28:39,426 DEBUG [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager] curator.ConnectionState: Starting 2018-01-05T11:28:39,426 DEBUG [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager] curator.ConnectionState: reset 2018-01-05T11:28:39,426 INFO [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager] zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:49384 sessionTimeout=1200000 watcher=org.apache.curator.ConnectionState@5a3b169f 2018-01-05T11:28:39,426 DEBUG [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager] service.AbstractService: Service LlapRegistryService is started 2018-01-05T11:28:39,427 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:49384] server.NIOServerCnxnFactory: Accepted socket connection from /127.0.0.1:49825 2018-01-05T11:28:39,427 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:49384] server.ZooKeeperServer: Client attempting to establish new session at /127.0.0.1:49825 2018-01-05T11:28:39,428 INFO [SyncThread:0] server.ZooKeeperServer: Established session 0x160c7cbc1600003 with negotiated timeout 40000 for client /127.0.0.1:49825 2018-01-05T11:28:39,428 INFO [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager-EventThread] state.ConnectionStateManager: State change: CONNECTED 2018-01-05T11:28:39,458 DEBUG [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager] impl.ZkRegistryBase: Added path=/user-djaiswal/llap_MiniLlapCluster/workers/worker-0000000000, host=10.22.8.182 instance=DynamicServiceInstance [id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, host=10.22.8.182:49818 with resources=, shufflePort=49816, servicesAddress=http://10.22.8.182:15002, mgmtPort=49819] to cache. pathToInstanceCache:size=1, nodeToInstanceCache:size=1 2018-01-05T11:28:39,459 INFO [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager] tezplugins.LlapTaskSchedulerService: Setting up node: DynamicServiceInstance [id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, host=10.22.8.182:49818 with resources=, shufflePort=49816, servicesAddress=http://10.22.8.182:15002, mgmtPort=49819] with available capacity=4, pendingQueueSize=4, memory=4096 2018-01-05T11:28:39,473 INFO [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager] tezplugins.LlapTaskSchedulerService: Adding new node: NodeInfo{instance=DynamicServiceInstance [id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, host=10.22.8.182:49818 with resources=, shufflePort=49816, servicesAddress=http://10.22.8.182:15002, mgmtPort=49819], expireTimeMillis=-1, numSuccessfulTasks=0, numSuccessfulTasksAtLastBlacklist=-1, cumulativeBackoffFactor=1.0, numSchedulableTasks=8, numScheduledTasks=0, disabled=false, commFailures=false}. TotalNodeCount=1. activeInstances.size=1 2018-01-05T11:28:39,473 INFO [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager] impl.ZookeeperUtils: UGI security is not enabled, or no SASL context name. Skipping setting up ZK auth. 2018-01-05T11:28:39,473 INFO [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager] imps.CuratorFrameworkImpl: Starting 2018-01-05T11:28:39,473 DEBUG [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager] curator.CuratorZookeeperClient: Starting 2018-01-05T11:28:39,473 DEBUG [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager] curator.ConnectionState: Starting 2018-01-05T11:28:39,473 DEBUG [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager] curator.ConnectionState: reset 2018-01-05T11:28:39,473 INFO [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager] zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:49384 sessionTimeout=1200000 watcher=org.apache.curator.ConnectionState@513c0b25 2018-01-05T11:28:39,475 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:49384] server.NIOServerCnxnFactory: Accepted socket connection from /127.0.0.1:49826 2018-01-05T11:28:39,475 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:49384] server.ZooKeeperServer: Client attempting to establish new session at /127.0.0.1:49826 2018-01-05T11:28:39,475 INFO [SyncThread:0] server.ZooKeeperServer: Established session 0x160c7cbc1600004 with negotiated timeout 40000 for client /127.0.0.1:49826 2018-01-05T11:28:39,475 INFO [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager-EventThread] state.ConnectionStateManager: State change: CONNECTED 2018-01-05T11:28:39,478 INFO [ProcessThread(sid:0 cport:-1):] server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x160c7cbc1600004 type:create cxid:0x3 zxid:0xe txntype:-1 reqpath:n/a Error Path:/tez-am-unsecure/user-djaiswal/llap/workers Error:KeeperErrorCode = NoNode for /tez-am-unsecure/user-djaiswal/llap/workers 2018-01-05T11:28:39,485 INFO [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager] impl.TezAmRegistryImpl: Registered this AM: rpc: { "api" : "tez-client", "addressType" : "host/port", "protocolType" : "hadoop/IPC", "addresses" : [ { "host" : "10.22.8.182", "port" : "49823" } ] }, plugin: null, sessionId: 53565c10-a9a1-4685-81b4-afcf82ce77fb, token: false, znodePath: /user-djaiswal/llap/workers/worker-0000000000 2018-01-05T11:28:39,485 DEBUG [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager] service.AbstractService: Service org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService is started 2018-01-05T11:28:39,485 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:39,485 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:39,485 DEBUG [ServiceThread:org.apache.tez.dag.app.rm.TaskSchedulerManager] service.AbstractService: Service org.apache.tez.dag.app.rm.TaskSchedulerManager is started 2018-01-05T11:28:39,486 INFO [DAGAppMaster Thread] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1515180518813_0001_000000, startTime=1515180519485 2018-01-05T11:28:39,487 INFO [DAGAppMaster Thread] app.DAGAppMaster: In Session mode. Waiting for DAG over RPC 2018-01-05T11:28:39,487 DEBUG [DAGAppMaster Thread] service.AbstractService: Service org.apache.tez.dag.app.DAGAppMaster is started 2018-01-05T11:28:39,490 INFO [Dispatcher thread {Central}] node.PerSourceNodeTracker: Ignoring RM Health Update for unknown node: 10.22.8.182:49818 2018-01-05T11:28:39,493 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.LocalClient: DAGAppMaster state: IDLE 2018-01-05T11:28:39,510 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: The url to track the Tez Session: N/A 2018-01-05T11:28:39,647 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:28:39,647 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:39,647 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Dag name: ANALYZE TABLE src COMPUTE STATISTICS (Stage-0) 2018-01-05T11:28:39,659 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: DagInfo: {"context":"Hive","description":"\n\nANALYZE TABLE src COMPUTE STATISTICS"} 2018-01-05T11:28:39,659 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112837_e75d6d24-220d-47b6-b23e-4cad8b21dd5d with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:28:39,660 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:39,680 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:28:39,745 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:39,745 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:28:39,756 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:39,756 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 1.73KB 2018-01-05T11:28:39,800 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-37_815_7793031023986305103-1/-mr-10000 2018-01-05T11:28:39,811 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-37_815_7793031023986305103-1/-mr-10000 2018-01-05T11:28:39,820 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src 2018-01-05T11:28:39,820 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:39,820 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:39,820 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:39,820 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=ANALYZE TABLE src COMPUTE STATISTICS (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112837_e75d6d24-220d-47b6-b23e-4cad8b21dd5d } 2018-01-05T11:28:39,853 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:39,861 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: ANALYZE TABLE src COMPUTE STATISTICS (Stage-0) 2018-01-05T11:28:39,864 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:28:39,916 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_1, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_1.dot 2018-01-05T11:28:39,918 INFO [IPC Server handler 1 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_1 2018-01-05T11:28:39,918 WARN [IPC Server handler 1 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:39,918 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Running DAG: ANALYZE TABLE src COMPUTE STATISTICS (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112837_e75d6d24-220d-47b6-b23e-4cad8b21dd5d } 2018-01-05T11:28:40,004 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_1, submitTime=1515180519861, queueName=null 2018-01-05T11:28:40,063 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_1_00 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_1_00 [Map 1], Default Resources= 2018-01-05T11:28:40,063 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_1_00 [Map 1] 2018-01-05T11:28:40,063 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_1_00 [Map 1] 2018-01-05T11:28:40,064 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:28:40,065 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_1_00 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:40,065 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:28:40,068 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_1, initTime=1515180520068 2018-01-05T11:28:40,069 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: dag_1515180518813_0001_1 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:28:40,071 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:28:40,073 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_1, dagName=ANALYZE TABLE src COMPUTE STATISTICS (Stage-0) 2018-01-05T11:28:40,074 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_1][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_1, startTime=1515180520071 2018-01-05T11:28:40,074 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_1 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:28:40,075 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {src={InputName=src}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:28:40,075 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: src, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:28:40,075 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_1_00 [Map 1] 2018-01-05T11:28:40,081 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:40,098 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_1_00 [Map 1] 2018-01-05T11:28:40,098 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_1_00 [Map 1] 2018-01-05T11:28:40,098 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:40,100 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:40,100 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:40,101 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_1_00 [Map 1] 2018-01-05T11:28:40,122 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:28:40,122 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@41bc7274 2018-01-05T11:28:40,123 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-37_815_7793031023986305103-1/djaiswal/_tez_scratch_dir/4d6cc739-6302-4cba-a598-1f650f9c1410/map.xml 2018-01-05T11:28:40,123 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-37_815_7793031023986305103-1/djaiswal/_tez_scratch_dir/4d6cc739-6302-4cba-a598-1f650f9c1410/map.xml 2018-01-05T11:28:40,123 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-37_815_7793031023986305103-1/djaiswal/_tez_scratch_dir/4d6cc739-6302-4cba-a598-1f650f9c1410/map.xml 2018-01-05T11:28:40,123 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:40,123 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:40,125 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:40,156 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:40,156 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 1.73KB 2018-01-05T11:28:40,163 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: src on vertex vertex_1515180518813_0001_1_00 [Map 1] 2018-01-05T11:28:40,164 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:28:40,164 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:40,169 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_1_00 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:28:40,169 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:28:40,169 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:40,169 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-37_815_7793031023986305103-1/djaiswal/_tez_scratch_dir/4d6cc739-6302-4cba-a598-1f650f9c1410/map.xml 2018-01-05T11:28:40,169 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:40,170 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias src 2018-01-05T11:28:40,170 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src 2018-01-05T11:28:40,170 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src 2018-01-05T11:28:40,173 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 2018-01-05T11:28:40,173 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = 2018-01-05T11:28:40,173 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src 2018-01-05T11:28:40,189 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:40,189 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:40,189 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:40,189 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:40,190 INFO [IPC Server handler 0 on 49823] counters.Limits: Counter limits initialized with parameters: GROUP_NAME_MAX=256, MAX_GROUPS=500, COUNTER_NAME_MAX=64, MAX_COUNTERS=120 2018-01-05T11:28:40,209 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Time taken to get FileStatuses: 16 2018-01-05T11:28:40,209 INFO [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:28:40,215 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 25 2018-01-05T11:28:40,215 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 1 2018-01-05T11:28:40,215 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:40,216 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 1. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:28:40,216 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-37_815_7793031023986305103-1/djaiswal/_tez_scratch_dir/4d6cc739-6302-4cba-a598-1f650f9c1410/map.xml 2018-01-05T11:28:40,216 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-37_815_7793031023986305103-1/djaiswal/_tez_scratch_dir/4d6cc739-6302-4cba-a598-1f650f9c1410/map.xml 2018-01-05T11:28:40,216 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-37_815_7793031023986305103-1/djaiswal/_tez_scratch_dir/4d6cc739-6302-4cba-a598-1f650f9c1410/map.xml 2018-01-05T11:28:40,216 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:40,216 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:40,216 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:40,217 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:40,218 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 1.73KB 2018-01-05T11:28:40,218 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src/kv1.txt to src new group? true 2018-01-05T11:28:40,219 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:28:40,220 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:28:40,223 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:28:40,226 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 107 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 5812 Original splits: 1 2018-01-05T11:28:40,226 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Using original number of splits: 1 desired splits: 1 2018-01-05T11:28:40,229 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 1 grouped split count is 1, for bucket: 1 2018-01-05T11:28:40,230 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:28:40,239 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: src on vertex vertex_1515180518813_0001_1_00 [Map 1] 2018-01-05T11:28:40,243 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_1_00 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:40,243 INFO [App Shared Pool - #0] impl.VertexImpl: Got updated RootInputsSpecs: {src=forAllWorkUnits=true, update=[1]} 2018-01-05T11:28:40,257 INFO [App Shared Pool - #0] impl.VertexImpl: Vertex vertex_1515180518813_0001_1_00 [Map 1] parallelism set to 1 2018-01-05T11:28:40,257 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_1_00 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:40,298 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_1_00 [Map 1], numAdditionalOutputs=1 2018-01-05T11:28:40,299 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_1][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_1_00, initRequestedTime=1515180520075, initedTime=1515180520298, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:40,299 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_1_00 [Map 1] with distanceFromRoot: 0 2018-01-05T11:28:40,299 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_1_00 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:28:40,300 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_1_00, startRequestedTime=1515180520169, startedTime=1515180520299 2018-01-05T11:28:40,302 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_1][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_1_00, reconfigureDoneTime=1515180520302, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:28:40,302 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_1_00 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:40,303 INFO [App Shared Pool - #0] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:28:40,303 INFO [App Shared Pool - #0] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:28:40,304 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_1_00 [Map 1] 2018-01-05T11:28:40,308 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_1][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_1_00_000000, scheduledTime=1515180520307, launchTime=1515180520307 2018-01-05T11:28:40,338 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_1_00 [Map 1] 2018-01-05T11:28:40,342 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_1_00_000000_0, priority=2, capability=, hosts=null 2018-01-05T11:28:40,342 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=2,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:40,342 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=2,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:40,343 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:40,343 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_1_00_000000_0 on hosts=null 2018-01-05T11:28:40,343 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_1_00_000000_0 on hosts=null 2018-01-05T11:28:40,343 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_1_00_000000_0 2018-01-05T11:28:40,343 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:40,351 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #1, task=TaskInfo{task=attempt_1515180518813_0001_1_00_000000_0, priority=2, startTime=0, containerId=null, uniqueId=0, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000001 2018-01-05T11:28:40,352 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_1_00_000000_0; false 2018-01-05T11:28:40,352 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_1_00_000000_0, priority=2, startTime=311540860, containerId=container_111101111_0001_00_000001assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=0, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:40,385 INFO [TaskSchedulerAppCallbackExecutor #0] node.PerSourceNodeTracker: Adding new node 10.22.8.182:49818 to nodeTracker 1 2018-01-05T11:28:40,393 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000001 for node: 10.22.8.182:49818 2018-01-05T11:28:40,401 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 1, name=ANALYZE TABLE src COMPUTE STATISTICS (Stage-0), queryId=djaiswal_20180105112837_e75d6d24-220d-47b6-b23e-4cad8b21dd5d 2018-01-05T11:28:40,401 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Added new known node: 10.22.8.182:49818 2018-01-05T11:28:40,401 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000001, attempt_1515180518813_0001_1_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:40,401 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000001 for node: 10.22.8.182:49818 2018-01-05T11:28:40,414 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:40,414 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: Status: Running (Executing on YARN cluster with App id application_1515180518813_0001) 2018-01-05T11:28:40,426 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 0/1 2018-01-05T11:28:40,436 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000001 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:40,436 DEBUG [TaskCommunicator # 0] llap.AsyncPbRpcProxy: Creating a client without a token for 10.22.8.182:49818 2018-01-05T11:28:40,438 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000001, launchTime=1515180520438 2018-01-05T11:28:40,438 INFO [TaskCommunicator # 0] impl.LlapProtocolClientImpl: Creating protocol proxy as null 2018-01-05T11:28:40,441 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_1_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000001] on NM: [10.22.8.182:49818] 2018-01-05T11:28:40,450 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Added path=/user-djaiswal/llap_MiniLlapCluster/workers/worker-0000000000, host=10.22.8.182 instance=DynamicServiceInstance [id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, host=10.22.8.182:49818 with resources=, shufflePort=49816, servicesAddress=http://10.22.8.182:15002, mgmtPort=49819] to cache. pathToInstanceCache:size=1, nodeToInstanceCache:size=1 2018-01-05T11:28:40,450 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:40,450 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:40,451 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_1_00_000000_0, startTime=1515180520438, containerId=container_111101111_0001_00_000001, nodeId=10.22.8.182:49818 2018-01-05T11:28:40,465 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_1_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112837_e75d6d24-220d-47b6-b23e-4cad8b21dd5d, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000001, dagName=ANALYZE TABLE src COMPUTE STATISTICS (Stage-0), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{src,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{out_Map 1,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180520071, firstAttemptStartTime=1515180520338, currentAttemptStartTime=1515180520408} 2018-01-05T11:28:40,466 INFO [IPC Server handler 0 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000001, QueryId=djaiswal_20180105112837_e75d6d24-220d-47b6-b23e-4cad8b21dd5d-1, DagId=1, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180520466 2018-01-05T11:28:40,475 DEBUG [IPC Server handler 0 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=1} with the ShuffleHandler 2018-01-05T11:28:40,530 INFO [IPC Server handler 0 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:28:40,532 DEBUG [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/1] 2018-01-05T11:28:40,544 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:40,544 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_1_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:40,545 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_1_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180520338, dagStartTime=1515180520071, withinDagPriority=2, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:28:40,545 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_1_00_000000_0 : ACCEPTED 2018-01-05T11:28:40,545 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_1_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:40,545 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_1_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180520338, dagStartTime=1515180520071, withinDagPriority=2, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:28:40,547 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_1_00_000000_0: true 2018-01-05T11:28:40,547 INFO [TaskCommunicator # 0] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_1_00_000000_0 2018-01-05T11:28:40,547 INFO [TaskCommunicator # 0] GuaranteedTasks: Received response for attempt_1515180518813_0001_1_00_000000_0, true 2018-01-05T11:28:40,548 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_1_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:40,548 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_1_00_000000_0 to pre-emption queue 2018-01-05T11:28:40,549 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:2008235852 2018-01-05T11:28:40,576 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : ANALYZE TABLE src COMPUTE STATISTICS (Stage-0), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_1_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=src, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=out_Map 1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:28:40,578 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=954728448, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:40,582 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_1_00_000000_0 vertexName: Map 1 2018-01-05T11:28:40,584 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000001, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_1_00_000000_0, eventCount=1 } 2018-01-05T11:28:40,587 DEBUG [TezTR-518813_1_1_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_1 queryId: djaiswal_20180105112837_e75d6d24-220d-47b6-b23e-4cad8b21dd5d instanceType: TaskRunner2Callable 2018-01-05T11:28:40,588 INFO [TezTR-518813_1_1_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_1_00_000000_0 2018-01-05T11:28:40,593 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:40,747 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:40,749 DEBUG [TezTR-518813_1_1_0_0_0] log.PerfLogger: 2018-01-05T11:28:40,754 DEBUG [TezTR-518813_1_1_0_0_0] log.PerfLogger: 2018-01-05T11:28:40,754 INFO [TezTR-518813_1_1_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:40,754 INFO [TezTR-518813_1_1_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:40,764 INFO [IPC Server handler 0 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_1_00_000000_0 sent events: (0-1). 2018-01-05T11:28:40,766 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:28:40,766 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_1_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:28:40,770 INFO [I/O Setup 0 Initialize: {src}] input.MRInput: src using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:28:40,770 INFO [I/O Setup 0 Initialize: {src}] input.MRInputLegacy: src MRInputLegacy deferring initialization 2018-01-05T11:28:40,770 INFO [TezTR-518813_1_1_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:40,773 INFO [I/O Setup 1 Initialize: {out_Map 1}] output.MROutput: out_Map 1: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:28:40,774 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:40,774 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:40,774 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:40,774 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:40,774 INFO [TezTR-518813_1_1_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:40,774 INFO [TezTR-518813_1_1_0_0_0] resources.MemoryDistributor: InitialRequests=[src:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [out_Map 1:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput] 2018-01-05T11:28:40,778 INFO [TezTR-518813_1_1_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:40,783 INFO [TezTR-518813_1_1_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:40,783 INFO [TezTR-518813_1_1_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 954728448, TotalAvailable: 639668060, TotalRequested/TotalJVMHeap:0.00 2018-01-05T11:28:40,783 INFO [TezTR-518813_1_1_0_0_0] resources.MemoryDistributor: Allocations=[src:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [out_Map 1:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0] 2018-01-05T11:28:40,783 INFO [TezTR-518813_1_1_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:40,784 INFO [TezTR-518813_1_1_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: src being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:28:40,785 INFO [TezTR-518813_1_1_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:28:40,785 INFO [TezTR-518813_1_1_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:28:40,785 INFO [I/O Setup 0 Start: {src}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: src 2018-01-05T11:28:40,785 INFO [TezTR-518813_1_1_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:40,786 INFO [TezTR-518813_1_1_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_1_00_000000_0 2018-01-05T11:28:40,786 DEBUG [TezTR-518813_1_1_0_0_0] log.PerfLogger: 2018-01-05T11:28:40,786 DEBUG [TezTR-518813_1_1_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_1_00_000000_0_10001 2018-01-05T11:28:40,789 DEBUG [TezTR-518813_1_1_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:40,796 DEBUG [TezTR-518813_1_1_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_1_00_000000_0 2018-01-05T11:28:40,799 INFO [TezTR-518813_1_1_0_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112837_e75d6d24-220d-47b6-b23e-4cad8b21dd5d 2018-01-05T11:28:40,800 DEBUG [TezTR-518813_1_1_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:40,800 DEBUG [TezTR-518813_1_1_0_0_0] log.PerfLogger: 2018-01-05T11:28:40,801 DEBUG [TezTR-518813_1_1_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:40,801 DEBUG [TezTR-518813_1_1_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:40,801 DEBUG [TezTR-518813_1_1_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112837_e75d6d24-220d-47b6-b23e-4cad8b21dd5d_Map 1__MAP_PLAN__ 2018-01-05T11:28:40,801 INFO [TezTR-518813_1_1_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-37_815_7793031023986305103-1/djaiswal/_tez_scratch_dir/4d6cc739-6302-4cba-a598-1f650f9c1410/map.xml 2018-01-05T11:28:40,801 DEBUG [TezTR-518813_1_1_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-37_815_7793031023986305103-1/djaiswal/_tez_scratch_dir/4d6cc739-6302-4cba-a598-1f650f9c1410/map.xml 2018-01-05T11:28:40,801 DEBUG [TezTR-518813_1_1_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-37_815_7793031023986305103-1/djaiswal/_tez_scratch_dir/4d6cc739-6302-4cba-a598-1f650f9c1410/map.xml 2018-01-05T11:28:40,801 DEBUG [TezTR-518813_1_1_0_0_0] log.PerfLogger: 2018-01-05T11:28:40,801 INFO [TezTR-518813_1_1_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:40,801 DEBUG [TezTR-518813_1_1_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:40,803 DEBUG [TezTR-518813_1_1_0_0_0] log.PerfLogger: 2018-01-05T11:28:40,803 INFO [TezTR-518813_1_1_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 1.73KB 2018-01-05T11:28:40,807 DEBUG [TezTR-518813_1_1_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:40,808 INFO [TezTR-518813_1_1_0_0_0] tez.MapRecordProcessor: The input names are: [src] 2018-01-05T11:28:40,810 INFO [TezTR-518813_1_1_0_0_0] input.MRInput: Initialized MRInput: src 2018-01-05T11:28:40,819 DEBUG [TezTR-518813_1_1_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 1 2018-01-05T11:28:40,819 INFO [TezTR-518813_1_1_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-37_815_7793031023986305103-1/djaiswal/_tez_scratch_dir/4d6cc739-6302-4cba-a598-1f650f9c1410/map.xml 2018-01-05T11:28:40,819 DEBUG [TezTR-518813_1_1_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:40,819 DEBUG [TezTR-518813_1_1_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src/kv1.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.src, numFiles=1, columns.types=string:string, serialization.format=1, serialization.ddl=struct src { string key, string value}, columns=key,value, columns.comments=defaultdefault, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, totalSize=5812, column.name.delimiter=,, transient_lastDdlTime=1515180517}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.src, numFiles=1, columns.types=string:string, columns=key,value, serialization.format=1, serialization.ddl=struct src { string key, string value}, columns.comments=defaultdefault, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, totalSize=5812, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src, column.name.delimiter=,, transient_lastDdlTime=1515180517}, baseFileName=src, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.src, numFiles=1, columns.types=string:string, serialization.format=1, serialization.ddl=struct src { string key, string value}, columns=key,value, columns.comments=defaultdefault, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, totalSize=5812, column.name.delimiter=,, transient_lastDdlTime=1515180517}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.src, numFiles=1, columns.types=string:string, columns=key,value, serialization.format=1, serialization.ddl=struct src { string key, string value}, columns.comments=defaultdefault, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, totalSize=5812, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src, column.name.delimiter=,, transient_lastDdlTime=1515180517}, baseFileName=src, vectorPartitionDesc=null]} 2018-01-05T11:28:40,820 DEBUG [TezTR-518813_1_1_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:40,820 INFO [TezTR-518813_1_1_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:40,820 DEBUG [TezTR-518813_1_1_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:40,826 DEBUG [TezTR-518813_1_1_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:40,826 INFO [TezTR-518813_1_1_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-37_815_7793031023986305103-1/djaiswal/_tez_scratch_dir/4d6cc739-6302-4cba-a598-1f650f9c1410/map.xml 2018-01-05T11:28:40,826 DEBUG [TezTR-518813_1_1_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:40,830 INFO [TezTR-518813_1_1_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_1_00_000000_0 2018-01-05T11:28:40,839 INFO [TezTR-518813_1_1_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [string, string] (isAcidRead false) 2018-01-05T11:28:40,868 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000001, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_1_00_000000_0, eventCount=1 } 2018-01-05T11:28:40,869 WARN [TezTR-518813_1_1_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src/kv1.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:28:40,871 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:40,875 DEBUG [TezTR-518813_1_1_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:28:40,875 WARN [TezTR-518813_1_1_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src/kv1.txt:0+5812 2018-01-05T11:28:40,883 DEBUG [TezTR-518813_1_1_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src/kv1.txt 2018-01-05T11:28:40,883 INFO [TezTR-518813_1_1_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src/kv1.txt:0+5812], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=5812} 2018-01-05T11:28:40,883 INFO [TezTR-518813_1_1_0_0_0] input.MRInput: src initialized RecordReader from event 2018-01-05T11:28:40,884 DEBUG [TezTR-518813_1_1_0_0_0] tez.MapRecordProcessor: Starting Output: out_Map 1 2018-01-05T11:28:40,886 INFO [TezTR-518813_1_1_0_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:28:40,886 INFO [TezTR-518813_1_1_0_0_0] exec.MapOperator: Initializing operator MAP[0] 2018-01-05T11:28:40,886 DEBUG [TezTR-518813_1_1_0_0_0] exec.MapOperator: Initialization Done 0 MAP 2018-01-05T11:28:40,886 DEBUG [TezTR-518813_1_1_0_0_0] exec.MapOperator: Operator 0 MAP initialized 2018-01-05T11:28:40,886 DEBUG [TezTR-518813_1_1_0_0_0] exec.MapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:28:40,887 DEBUG [TezTR-518813_1_1_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:40,888 DEBUG [TezTR-518813_1_1_0_0_0] exec.MapOperator: Adding alias src to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src 2018-01-05T11:28:40,892 DEBUG [TezTR-518813_1_1_0_0_0] exec.MapOperator: dump TS[0] struct 2018-01-05T11:28:40,893 INFO [TezTR-518813_1_1_0_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:28:40,893 INFO [TezTR-518813_1_1_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:40,894 DEBUG [TezTR-518813_1_1_0_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:28:40,896 INFO [TezTR-518813_1_1_0_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:28:40,896 DEBUG [TezTR-518813_1_1_0_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:28:40,896 DEBUG [TezTR-518813_1_1_0_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:28:40,896 DEBUG [TezTR-518813_1_1_0_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:28:40,896 DEBUG [TezTR-518813_1_1_0_0_0] log.PerfLogger: 2018-01-05T11:28:40,897 INFO [TezTR-518813_1_1_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-37_815_7793031023986305103-1/djaiswal/_tez_scratch_dir/4d6cc739-6302-4cba-a598-1f650f9c1410/map.xml 2018-01-05T11:28:40,897 DEBUG [TezTR-518813_1_1_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:40,898 DEBUG [TezTR-518813_1_1_0_0_0] exec.MapOperator: Processing alias(es) src for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src/kv1.txt 2018-01-05T11:28:40,900 INFO [TezTR-518813_1_1_0_0_0] exec.MapOperator: MAP[0]: records read - 1 2018-01-05T11:28:40,900 INFO [TezTR-518813_1_1_0_0_0] exec.MapOperator: MAP[0]: records read - 10 2018-01-05T11:28:40,902 INFO [TezTR-518813_1_1_0_0_0] exec.MapOperator: MAP[0]: records read - 100 2018-01-05T11:28:40,907 DEBUG [TezTR-518813_1_1_0_0_0] log.PerfLogger: 2018-01-05T11:28:40,907 DEBUG [TezTR-518813_1_1_0_0_0] mr.ObjectCache: djaiswal_20180105112837_e75d6d24-220d-47b6-b23e-4cad8b21dd5d_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:28:40,907 DEBUG [TezTR-518813_1_1_0_0_0] exec.MapOperator: close called for operator MAP[0] 2018-01-05T11:28:40,907 INFO [TezTR-518813_1_1_0_0_0] exec.MapOperator: Closing operator MAP[0] 2018-01-05T11:28:40,907 INFO [TezTR-518813_1_1_0_0_0] exec.MapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:500, 2018-01-05T11:28:40,907 DEBUG [TezTR-518813_1_1_0_0_0] exec.MapOperator: Closing child = TS[0] 2018-01-05T11:28:40,907 DEBUG [TezTR-518813_1_1_0_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:28:40,907 DEBUG [TezTR-518813_1_1_0_0_0] exec.TableScanOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:40,907 INFO [TezTR-518813_1_1_0_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:28:40,908 DEBUG [TezTR-518813_1_1_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:40,908 DEBUG [TezTR-518813_1_1_0_0_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-37_815_7793031023986305103-1/-mr-10000 2018-01-05T11:28:40,908 INFO [TezTR-518813_1_1_0_0_0] exec.TableScanOperator: publishing : default.src/ : {numRows=500, rawDataSize=5312} 2018-01-05T11:28:40,929 DEBUG [TezTR-518813_1_1_0_0_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-37_815_7793031023986305103-1/-mr-10000/tmpstats-0_0 2018-01-05T11:28:40,930 DEBUG [TezTR-518813_1_1_0_0_0] fs.FSStatsPublisher: Writing stats in it : {default.src/={numRows=500, rawDataSize=5312}} 2018-01-05T11:28:40,930 DEBUG [TezTR-518813_1_1_0_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:28:40,930 DEBUG [TezTR-518813_1_1_0_0_0] exec.MapOperator: 0 Close done 2018-01-05T11:28:40,935 INFO [TezTR-518813_1_1_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_1_00_000000_0 2018-01-05T11:28:40,935 INFO [TezTR-518813_1_1_0_0_0] output.MROutput: out_Map 1 closed 2018-01-05T11:28:40,936 DEBUG [TezTR-518813_1_1_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:40,936 INFO [TezTR-518813_1_1_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:40,937 INFO [TezTR-518813_1_1_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_1_00_000000_0, askedToStop=false 2018-01-05T11:28:40,937 INFO [TezTR-518813_1_1_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_1_00_000000_0, stopRequested=false 2018-01-05T11:28:40,938 INFO [TezTR-518813_1_1_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_1_00_000000_0: Counters: 5 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=500, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=0][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500]] 2018-01-05T11:28:40,941 DEBUG [TezTR-518813_1_1_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_1_0_0_0 thread id: 302 scheme: file bytesRead: 5872 bytesWritten: 82 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:40,943 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_1_00_000000_0, isTaskDone=true 2018-01-05T11:28:40,945 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000001, requestId=3, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_1_00_000000_0, eventCount=2 } 2018-01-05T11:28:40,950 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=3, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:40,950 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_1_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:40,950 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_1_00_000000_0 2018-01-05T11:28:40,951 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000001=403 2018-01-05T11:28:40,951 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_1_00_000000_0: true 2018-01-05T11:28:40,951 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_1_00_000000_0 2018-01-05T11:28:40,952 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000001, QueryId=djaiswal_20180105112837_e75d6d24-220d-47b6-b23e-4cad8b21dd5d-1, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:28:40,952 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_1_00_000000_0, creationTime=1515180520330, allocationTime=1515180520394, startTime=1515180520438, finishTime=1515180520949, timeTaken=511, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:40,952 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_1_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:40,954 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_1_00_000000_0 2018-01-05T11:28:40,956 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_1_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:40,956 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_1_00_000000_0 2018-01-05T11:28:40,956 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_1_00_000000_0; false 2018-01-05T11:28:40,956 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_1_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:40,956 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:40,956 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:40,959 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_1][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_1_00_000000, startTime=1515180520438, finishTime=1515180520956, timeTaken=518, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_1_00_000000_0, diagnostics=, counters=Counters: 10, File System Counters, FILE_BYTES_READ=5872, FILE_BYTES_WRITTEN=82, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=500, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500 2018-01-05T11:28:40,962 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_1_00 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:40,962 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_1_00 [Map 1] 2018-01-05T11:28:40,971 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_1][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_1_00, initRequestedTime=1515180520075, initedTime=1515180520298, startRequestedTime=1515180520169, startedTime=1515180520299, finishTime=1515180520962, timeTaken=663, status=SUCCEEDED, diagnostics=, counters=Counters: 10, File System Counters, FILE_BYTES_READ=5872, FILE_BYTES_WRITTEN=82, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=500, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500, vertexStats=firstTaskStartTime=1515180520438, firstTasksToStart=[ task_1515180518813_0001_1_00_000000 ], lastTaskFinishTime=1515180520949, lastTasksToFinish=[ task_1515180518813_0001_1_00_000000 ], minTaskDuration=511, maxTaskDuration=511, avgTaskDuration=511.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_1_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_1_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:40,971 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_1_00 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:40,973 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_1][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000001, stoppedTime=1515180520973, exitStatus=0 2018-01-05T11:28:40,974 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000001 exited with diagnostics set to Container completed. 2018-01-05T11:28:40,974 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_1_00 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1 2018-01-05T11:28:40,974 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null 2018-01-05T11:28:40,974 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_1_00 [Map 1] 2018-01-05T11:28:40,982 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_1, queueSize=0 2018-01-05T11:28:40,983 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_1][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_1, startTime=1515180520071, finishTime=1515180520974, timeTaken=903, status=SUCCEEDED, diagnostics=, counters=Counters: 14, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, AM_CPU_MILLISECONDS=2047, AM_GC_TIME_MILLIS=157, File System Counters, FILE_BYTES_READ=5872, FILE_BYTES_WRITTEN=82, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=500, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500 2018-01-05T11:28:40,984 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_1 finished with state: SUCCEEDED 2018-01-05T11:28:40,984 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:28:40,984 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000001 2018-01-05T11:28:40,985 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_1, dagState=SUCCEEDED 2018-01-05T11:28:40,985 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_1_post 2018-01-05T11:28:40,985 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:40,985 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:28:40,985 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:28:40,986 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 1 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=1, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=1,NumRequestsWithoutLocation=1, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=1} 2018-01-05T11:28:40,986 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:28:40,986 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=ANALYZE TABLE src COMPUTE STATISTICS (Stage-0), with id=dag_1515180518813_0001_1 2018-01-05T11:28:40,987 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:40,987 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:40,987 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 2018-01-05T11:28:40,987 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:40,987 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 1, to 10.22.8.182:49818 2018-01-05T11:28:40,988 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=1, Remaining=0 2018-01-05T11:28:40,988 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=ANALYZE TABLE src COMPUTE STATISTICS (Stage-0), with id=dag_1515180518813_0001_1 2018-01-05T11:28:40,989 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=1} 2018-01-05T11:28:40,989 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-2:STATS] in serial mode 2018-01-05T11:28:40,989 INFO [IPC Server handler 0 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=1}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:28:40,989 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@22c4354d, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:40,989 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:40,989 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:40,989 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:40,989 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:40,989 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:40,989 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:40,989 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:40,989 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:40,989 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:40,989 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-37_815_7793031023986305103-1/-mr-10000 2018-01-05T11:28:40,989 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-37_815_7793031023986305103-1/-mr-10000 2018-01-05T11:28:40,989 INFO [IPC Server handler 0 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/1 after 300 seconds 2018-01-05T11:28:40,990 ERROR [IPC Server handler 0 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112837_e75d6d24-220d-47b6-b23e-4cad8b21dd5d, dag_1515180518813_0001_1 2018-01-05T11:28:40,990 INFO [IPC Server handler 0 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112837_e75d6d24-220d-47b6-b23e-4cad8b21dd5d 2018-01-05T11:28:40,990 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=1} = 0 2018-01-05T11:28:40,990 DEBUG [IPC Server handler 0 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=1} 2018-01-05T11:28:40,990 DEBUG [IPC Server handler 0 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:28:40,991 DEBUG [IPC Server handler 0 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=1} 2018-01-05T11:28:40,993 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.src/ numRows 2018-01-05T11:28:40,993 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.src/ numRows 500 2018-01-05T11:28:40,993 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.src/ rawDataSize 2018-01-05T11:28:40,993 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.src/ rawDataSize 5312 2018-01-05T11:28:40,993 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:40,994 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:40,994 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:40,994 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:40,994 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:40,994 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:40,994 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:40,994 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:40,996 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:40,996 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:40,997 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:40,997 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:40,997 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:40,997 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:40,997 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:40,997 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=src newtbl=src 2018-01-05T11:28:40,997 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=src newtbl=src 2018-01-05T11:28:41,004 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:41,017 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:41,017 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:41,017 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.src stats: [numFiles=1, numRows=500, totalSize=5812, rawDataSize=5312] 2018-01-05T11:28:41,017 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: About to delete stats tmp dir :file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-37_815_7793031023986305103-1/-mr-10000 2018-01-05T11:28:41,018 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,018 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,018 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: ANALYZE TABLE src COMPUTE STATISTICS 2018-01-05T11:28:41,018 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:28:41,018 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@src 2018-01-05T11:28:41,018 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@src 2018-01-05T11:28:41,018 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,018 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,018 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:41,018 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=20} 2018-01-05T11:28:41,018 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112837_e75d6d24-220d-47b6-b23e-4cad8b21dd5d); Time taken: 2.86 seconds 2018-01-05T11:28:41,018 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:41,018 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,018 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:41,018 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,018 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query ANALYZE TABLE src COMPUTE STATISTICS 2018-01-05T11:28:41,019 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-37_815_7793031023986305103-1 2018-01-05T11:28:41,019 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 3.204 seconds 2018-01-05T11:28:41,019 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:41,019 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:41,019 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:41,019 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:41,019 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,020 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:41,020 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,020 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,020 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,020 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: ANALYZE TABLE src COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:41,020 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112841_c0c75619-7aaf-4c02-a457-8aa38d666b4f): ANALYZE TABLE src COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:41,020 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,020 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: ANALYZE TABLE src COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:41,020 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:41,020 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,020 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,021 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:41,021 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:41,021 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:41,021 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:41,021 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src 2018-01-05T11:28:41,021 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src 2018-01-05T11:28:41,027 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:41,028 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:41,029 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: select compute_stats(`key`, 'hll') , compute_stats(`value`, 'hll') from `default`.`src` 2018-01-05T11:28:41,029 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: select compute_stats(`key`, 'hll') , compute_stats(`value`, 'hll') from `default`.`src` 2018-01-05T11:28:41,033 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:41,033 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Invoking analyze on rewritten query 2018-01-05T11:28:41,033 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Starting Semantic Analysis 2018-01-05T11:28:41,034 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:41,034 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:41,034 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed phase 1 of Semantic Analysis 2018-01-05T11:28:41,034 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for source tables 2018-01-05T11:28:41,034 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:41,034 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src 2018-01-05T11:28:41,034 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src 2018-01-05T11:28:41,045 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:41,046 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:41,046 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for subqueries 2018-01-05T11:28:41,046 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for destination tables 2018-01-05T11:28:41,057 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed getting MetaData in Semantic Analysis 2018-01-05T11:28:41,058 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-41_029_8026269352618129281-1/-mr-10001 2018-01-05T11:28:41,058 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Table Plan for src TS[0] 2018-01-05T11:28:41,059 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000001, QueryId=djaiswal_20180105112837_e75d6d24-220d-47b6-b23e-4cad8b21dd5d-1, DagId=1, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180520547, EndTime=1515180521059 2018-01-05T11:28:41,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:41,077 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:41,084 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: RR before GB src{(key,key: string)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)(RAW__DATA__SIZE,RAW__DATA__SIZE: bigint)} after GB {((tok_function compute_stats (tok_table_or_col key) 'hll'),_col0: struct)((tok_function compute_stats (tok_table_or_col value) 'hll'),_col1: struct)} 2018-01-05T11:28:41,084 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: tree: (tok_select (tok_selexpr (tok_function compute_stats (tok_table_or_col key) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col value) 'hll'))) 2018-01-05T11:28:41,084 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: genSelectPlan: input = {((tok_function compute_stats (tok_table_or_col key) 'hll'),_col0: struct)((tok_function compute_stats (tok_table_or_col value) 'hll'),_col1: struct)} starRr = src{(key,key: string)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)(RAW__DATA__SIZE,RAW__DATA__SIZE: bigint)} 2018-01-05T11:28:41,086 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Select Plan row schema: null{(_c0,_col0: struct)(_c1,_col1: struct)} 2018-01-05T11:28:41,086 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Select Plan for clause: insclause-0 2018-01-05T11:28:41,087 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-41_029_8026269352618129281-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-41_029_8026269352618129281-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-41_029_8026269352618129281-1/-mr-10000 2018-01-05T11:28:41,087 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-41_029_8026269352618129281-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-41_029_8026269352618129281-1 2018-01-05T11:28:41,112 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-41_029_8026269352618129281-1/-mr-10003 2018-01-05T11:28:41,113 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-41_029_8026269352618129281-1/-mr-10000 row schema: null{(_c0,_col0: struct)(_c1,_col1: struct)} 2018-01-05T11:28:41,113 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Body Plan for Query Block null 2018-01-05T11:28:41,113 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Plan for Query Block null 2018-01-05T11:28:41,113 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Before logical optimization TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-SEL[5]-FS[6] 2018-01-05T11:28:41,113 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,113 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,113 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] lineage.Generator: Time taken for lineage transform=7 2018-01-05T11:28:41,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{src}key,value: string|{src}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src}block__offset__inside__file,INPUT__FILE__NAME: string|{src}input__file__name,ROW__ID: struct|{src}row__id,RAW__DATA__SIZE: bigint|{src}raw__data__size) 2018-01-05T11:28:41,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: string|{src}key,value: string|{src}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src}block__offset__inside__file,INPUT__FILE__NAME: string|{src}input__file__name,ROW__ID: struct|{src}row__id,RAW__DATA__SIZE: bigint|{src}raw__data__size) 2018-01-05T11:28:41,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:41,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:41,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:28:41,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:41,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:41,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:41,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:41,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:41,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:41,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:41,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:41,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:41,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1) 2018-01-05T11:28:41,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:41,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:41,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0] Column[_col1]) 2018-01-05T11:28:41,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{}) 2018-01-05T11:28:41,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:41,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:41,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,121 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for FS(6) 2018-01-05T11:28:41,121 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(5) 2018-01-05T11:28:41,121 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for GBY(4) 2018-01-05T11:28:41,121 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for RS(3) 2018-01-05T11:28:41,121 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for GBY(2) 2018-01-05T11:28:41,121 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(1) 2018-01-05T11:28:41,121 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:28:41,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.PredicatePushDown: After PPD: TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-SEL[5]-FS[6] 2018-01-05T11:28:41,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{src}key,value: string|{src}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src}block__offset__inside__file,INPUT__FILE__NAME: string|{src}input__file__name,ROW__ID: struct|{src}row__id,RAW__DATA__SIZE: bigint|{src}raw__data__size) 2018-01-05T11:28:41,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: string|{src}key,value: string|{src}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src}block__offset__inside__file,INPUT__FILE__NAME: string|{src}input__file__name,ROW__ID: struct|{src}row__id,RAW__DATA__SIZE: bigint|{src}raw__data__size) 2018-01-05T11:28:41,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:41,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:41,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:28:41,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:41,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:41,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:41,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:41,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:41,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:41,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:41,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:41,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:41,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1) 2018-01-05T11:28:41,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:41,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:41,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0] Column[_col1]) 2018-01-05T11:28:41,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{}) 2018-01-05T11:28:41,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:41,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:41,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{src}key,value: string|{src}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src}block__offset__inside__file,INPUT__FILE__NAME: string|{src}input__file__name,ROW__ID: struct|{src}row__id,RAW__DATA__SIZE: bigint|{src}raw__data__size) 2018-01-05T11:28:41,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: string|{src}key,value: string|{src}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src}block__offset__inside__file,INPUT__FILE__NAME: string|{src}input__file__name,ROW__ID: struct|{src}row__id,RAW__DATA__SIZE: bigint|{src}raw__data__size) 2018-01-05T11:28:41,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:41,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:41,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:28:41,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:41,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:41,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:41,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:41,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:41,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:41,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:41,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:41,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:41,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1) 2018-01-05T11:28:41,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:41,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:41,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0] Column[_col1]) 2018-01-05T11:28:41,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{}) 2018-01-05T11:28:41,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:41,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:41,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,124 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,124 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,125 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 3 key:[] 2018-01-05T11:28:41,125 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 3 oldColExprMap: {VALUE._col0=Column[_col0], VALUE._col1=Column[_col1]} 2018-01-05T11:28:41,125 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 3 newColExprMap: {VALUE._col0=Column[_col0], VALUE._col1=Column[_col1]} 2018-01-05T11:28:41,125 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,125 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,125 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,125 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.IdentityProjectRemover: Identity project remover optimization removed : SEL[5] 2018-01-05T11:28:41,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,128 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,128 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: After logical optimization TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-FS[6] 2018-01-05T11:28:41,128 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,128 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,128 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,128 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,128 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,129 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:41,129 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=src 2018-01-05T11:28:41,129 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=src 2018-01-05T11:28:41,141 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 5.601355ms + 0.365283ms, the query is [select "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "TAB_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (...)] 2018-01-05T11:28:41,142 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:41,153 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:41,154 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:41,158 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting stats (Num rows: 500 Data size: 175168 Basic stats: COMPLETE Column stats: NONE) on TS[0] 2018-01-05T11:28:41,158 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (src): numRows: 500 dataSize: 175168 basicStatsState: COMPLETE colStatsState: NONE colStats: {value= colName: value colType: string countDistincts: 100 numNulls: 25 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: string countDistincts: 100 numNulls: 25 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true} 2018-01-05T11:28:41,158 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting stats (Num rows: 500 Data size: 175168 Basic stats: COMPLETE Column stats: NONE) on SEL[1] 2018-01-05T11:28:41,158 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [1] STATS-SEL[1]: numRows: 500 dataSize: 175168 basicStatsState: COMPLETE colStatsState: NONE colStats: {value= colName: value colType: string countDistincts: 100 numNulls: 25 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: string countDistincts: 100 numNulls: 25 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true} 2018-01-05T11:28:41,158 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[2]: inputSize: 175168 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:28:41,158 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [Case 1] STATS-GBY[2]: cardinality: 500 2018-01-05T11:28:41,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 1248 Basic stats: COMPLETE Column stats: NONE) on GBY[2] 2018-01-05T11:28:41,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-GBY[2]: numRows: 1 dataSize: 1248 basicStatsState: COMPLETE colStatsState: NONE colStats: {_col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true} 2018-01-05T11:28:41,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 1248 Basic stats: COMPLETE Column stats: NONE) on RS[3] 2018-01-05T11:28:41,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-RS[3]: numRows: 1 dataSize: 1248 basicStatsState: COMPLETE colStatsState: NONE colStats: {_col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true} 2018-01-05T11:28:41,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[4]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:28:41,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [Case 7] STATS-GBY[4]: cardinality: 0 2018-01-05T11:28:41,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[4]: Equals 0 in number of rows. 0 rows will be set to 1 2018-01-05T11:28:41,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 1248 Basic stats: COMPLETE Column stats: NONE) on GBY[4] 2018-01-05T11:28:41,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-GBY[4]: numRows: 1 dataSize: 1248 basicStatsState: COMPLETE colStatsState: NONE colStats: {_col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true} 2018-01-05T11:28:41,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-FS[6]: numRows: 1 dataSize: 1248 basicStatsState: COMPLETE colStatsState: NONE colStats: {_col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true} 2018-01-05T11:28:41,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on TS[0] 2018-01-05T11:28:41,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on SEL[1] 2018-01-05T11:28:41,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[2] 2018-01-05T11:28:41,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[3] 2018-01-05T11:28:41,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[4] 2018-01-05T11:28:41,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FileSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on FS[6] 2018-01-05T11:28:41,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,160 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:28:41,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:41,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: FS, 6 2018-01-05T11:28:41,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:41,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: GBY, 4 2018-01-05T11:28:41,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:41,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: RS, 3 2018-01-05T11:28:41,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:41,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: GBY, 2 2018-01-05T11:28:41,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:41,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: SEL, 1 2018-01-05T11:28:41,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:41,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:28:41,161 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Cycle free: true 2018-01-05T11:28:41,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{src}key,value: string|{src}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src}block__offset__inside__file,INPUT__FILE__NAME: string|{src}input__file__name,ROW__ID: struct|{src}row__id,RAW__DATA__SIZE: bigint|{src}raw__data__size) 2018-01-05T11:28:41,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: string|{src}key,value: string|{src}value) 2018-01-05T11:28:41,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:41,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:41,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value]) 2018-01-05T11:28:41,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:41,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:41,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:41,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:41,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:41,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:41,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:41,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:41,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:41,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{}) 2018-01-05T11:28:41,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:41,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:41,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: TS[0] 2018-01-05T11:28:41,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: RS[3] 2018-01-05T11:28:41,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:28:41,162 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src of table src 2018-01-05T11:28:41,162 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src 2018-01-05T11:28:41,162 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: RS[3] 2018-01-05T11:28:41,162 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: GBY[4] 2018-01-05T11:28:41,162 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: FS[6] 2018-01-05T11:28:41,162 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding reduce work (Reducer 2) for GBY[4] 2018-01-05T11:28:41,162 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Setting up reduce sink: RS[3] with following reduce work: Reducer 2 2018-01-05T11:28:41,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Removing RS[3] as parent from GBY[4] 2018-01-05T11:28:41,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: FS[6] 2018-01-05T11:28:41,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:28:41,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:41,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:41,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:41,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:41,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:41,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:41,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:28:41,164 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping vectorization 2018-01-05T11:28:41,164 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:28:41,164 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: llap mode: only 2018-01-05T11:28:41,164 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:28:41,164 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(key, 'hll')' 2018-01-05T11:28:41,164 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'key' 2018-01-05T11:28:41,164 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:41,164 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(value, 'hll')' 2018-01-05T11:28:41,164 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'value' 2018-01-05T11:28:41,164 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:41,164 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'key' 2018-01-05T11:28:41,164 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'value' 2018-01-05T11:28:41,164 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:41,164 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@3a7d6976 can be done in LLAP 2018-01-05T11:28:41,164 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Reducer 2 2018-01-05T11:28:41,164 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col0)' 2018-01-05T11:28:41,164 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:28:41,164 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col1)' 2018-01-05T11:28:41,164 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col1' 2018-01-05T11:28:41,164 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:41,164 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@37c45154 can be done in LLAP 2018-01-05T11:28:41,165 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,166 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed plan generation 2018-01-05T11:28:41,166 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:41,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validation start 2018-01-05T11:28:41,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Not a partition. 2018-01-05T11:28:41,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:28:41,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,166 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0, type:struct, comment:null), FieldSchema(name:_c1, type:struct, comment:null)], properties:null) 2018-01-05T11:28:41,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,166 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:41,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=19, flushCache_()=0, getTableColumnStatistics_(String, String, List, )=25} 2018-01-05T11:28:41,166 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112841_c0c75619-7aaf-4c02-a457-8aa38d666b4f); Time taken: 0.146 seconds 2018-01-05T11:28:41,166 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:41,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,166 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112841_c0c75619-7aaf-4c02-a457-8aa38d666b4f): ANALYZE TABLE src COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:41,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,167 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: ANALYZE TABLE src COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:41,167 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:28:41,167 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@src 2018-01-05T11:28:41,167 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@src 2018-01-05T11:28:41,167 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-41_029_8026269352618129281-1/-mr-10000 2018-01-05T11:28:41,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,167 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Query ID = djaiswal_20180105112841_c0c75619-7aaf-4c02-a457-8aa38d666b4f 2018-01-05T11:28:41,167 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Total jobs = 1 2018-01-05T11:28:41,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,167 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:28:41,167 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MAPRED] in serial mode 2018-01-05T11:28:41,194 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-41_020_1421415792491703556-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:28:41,215 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:28:41,216 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:28:41,216 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:28:41,216 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112841_c0c75619-7aaf-4c02-a457-8aa38d666b4f 2018-01-05T11:28:41,216 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Session is already open 2018-01-05T11:28:41,216 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:28:41,216 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,216 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Dag name: ANALYZE TABLE src COMPUTE STATIS...key,value (Stage-0) 2018-01-05T11:28:41,216 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: DagInfo: {"context":"Hive","description":"\n\nANALYZE TABLE src COMPUTE STATISTICS FOR COLUMNS key,value"} 2018-01-05T11:28:41,216 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112841_c0c75619-7aaf-4c02-a457-8aa38d666b4f with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:28:41,216 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,228 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,228 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:28:41,234 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,235 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 2 size: 2.01KB 2018-01-05T11:28:41,253 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,253 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,264 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:28:41,264 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,264 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:28:41,268 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,269 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 3.20KB 2018-01-05T11:28:41,277 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-41_029_8026269352618129281-1/-mr-10001 2018-01-05T11:28:41,288 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-41_029_8026269352618129281-1/-mr-10001 2018-01-05T11:28:41,288 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src 2018-01-05T11:28:41,288 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,325 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,325 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,325 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=ANALYZE TABLE src COMPUTE STATIS...key,value (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112841_c0c75619-7aaf-4c02-a457-8aa38d666b4f } 2018-01-05T11:28:41,325 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:41,326 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:41,333 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: ANALYZE TABLE src COMPUTE STATIS...key,value (Stage-0) 2018-01-05T11:28:41,333 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:28:41,334 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_2, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_2.dot 2018-01-05T11:28:41,334 INFO [IPC Server handler 0 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_2 2018-01-05T11:28:41,334 WARN [IPC Server handler 0 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:41,334 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Running DAG: ANALYZE TABLE src COMPUTE STATIS...key,value (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112841_c0c75619-7aaf-4c02-a457-8aa38d666b4f } 2018-01-05T11:28:41,359 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_2][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_2, submitTime=1515180521333, queueName=null 2018-01-05T11:28:41,361 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_2_00 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_2_00 [Map 1], Default Resources= 2018-01-05T11:28:41,361 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_2_00 [Map 1] 2018-01-05T11:28:41,361 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:28:41,361 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_2_00 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:41,362 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_2_01 [Reducer 2]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_2_01 [Reducer 2], Default Resources= 2018-01-05T11:28:41,362 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_2_01 [Reducer 2] 2018-01-05T11:28:41,362 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 2 2018-01-05T11:28:41,362 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_2_01 [Reducer 2] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:41,370 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:28:41,370 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_2][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_2, initTime=1515180521370 2018-01-05T11:28:41,370 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: dag_1515180518813_0001_2 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:28:41,371 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:28:41,371 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_2][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_2, startTime=1515180521371 2018-01-05T11:28:41,371 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_2 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:28:41,371 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {src={InputName=src}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:28:41,371 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: src, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:28:41,371 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_2_00 [Map 1] 2018-01-05T11:28:41,371 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_2, dagName=ANALYZE TABLE src COMPUTE STATIS...key,value (Stage-0) 2018-01-05T11:28:41,371 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,371 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,371 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,377 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_2_00 [Map 1] 2018-01-05T11:28:41,377 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_2_00 [Map 1] 2018-01-05T11:28:41,377 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:41,377 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_2_00 [Map 1] 2018-01-05T11:28:41,380 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:28:41,380 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@7fc64911 2018-01-05T11:28:41,381 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-41_020_1421415792491703556-1/djaiswal/_tez_scratch_dir/79b4687e-f6c8-4354-a2f6-9ba8fd058008/map.xml 2018-01-05T11:28:41,381 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-41_020_1421415792491703556-1/djaiswal/_tez_scratch_dir/79b4687e-f6c8-4354-a2f6-9ba8fd058008/map.xml 2018-01-05T11:28:41,381 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-41_020_1421415792491703556-1/djaiswal/_tez_scratch_dir/79b4687e-f6c8-4354-a2f6-9ba8fd058008/map.xml 2018-01-05T11:28:41,381 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:41,381 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:41,381 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:41,393 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:41,393 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.20KB 2018-01-05T11:28:41,393 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: src on vertex vertex_1515180518813_0001_2_00 [Map 1] 2018-01-05T11:28:41,393 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_2_00 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:28:41,394 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:28:41,394 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to ShuffleVertexManager for vertex_1515180518813_0001_2_01 [Reducer 2] 2018-01-05T11:28:41,394 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:41,394 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:28:41,394 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:41,394 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-41_020_1421415792491703556-1/djaiswal/_tez_scratch_dir/79b4687e-f6c8-4354-a2f6-9ba8fd058008/map.xml 2018-01-05T11:28:41,394 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:41,394 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias src 2018-01-05T11:28:41,394 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src 2018-01-05T11:28:41,395 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src 2018-01-05T11:28:41,396 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 0,1 2018-01-05T11:28:41,396 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = key,value 2018-01-05T11:28:41,396 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src 2018-01-05T11:28:41,397 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:41,397 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:41,397 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:41,397 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:41,411 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.05 maxFrac: 0.75 auto: false desiredTaskIput: 104857600 2018-01-05T11:28:41,411 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:28:41,411 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 1 tasks for vertex: vertex_1515180518813_0001_2_01 [Reducer 2] 2018-01-05T11:28:41,411 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_2_01 [Reducer 2] 2018-01-05T11:28:41,411 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_2][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_2_01, reconfigureDoneTime=1515180521411, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:28:41,411 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_2_01 [Reducer 2], numAdditionalOutputs=1 2018-01-05T11:28:41,411 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_2][Event:VERTEX_INITIALIZED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_2_01, initRequestedTime=1515180521394, initedTime=1515180521411, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:41,412 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_2_01 [Reducer 2] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:28:41,413 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Time taken to get FileStatuses: 14 2018-01-05T11:28:41,413 INFO [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:28:41,413 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 15 2018-01-05T11:28:41,413 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 1 2018-01-05T11:28:41,413 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:41,413 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 1. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:28:41,413 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-41_020_1421415792491703556-1/djaiswal/_tez_scratch_dir/79b4687e-f6c8-4354-a2f6-9ba8fd058008/map.xml 2018-01-05T11:28:41,414 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-41_020_1421415792491703556-1/djaiswal/_tez_scratch_dir/79b4687e-f6c8-4354-a2f6-9ba8fd058008/map.xml 2018-01-05T11:28:41,414 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-41_020_1421415792491703556-1/djaiswal/_tez_scratch_dir/79b4687e-f6c8-4354-a2f6-9ba8fd058008/map.xml 2018-01-05T11:28:41,414 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:41,414 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:41,414 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:41,416 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:41,416 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.20KB 2018-01-05T11:28:41,416 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src/kv1.txt to src new group? true 2018-01-05T11:28:41,416 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:28:41,416 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:28:41,416 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:28:41,416 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 107 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 5812 Original splits: 1 2018-01-05T11:28:41,416 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Using original number of splits: 1 desired splits: 1 2018-01-05T11:28:41,416 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 1 grouped split count is 1, for bucket: 1 2018-01-05T11:28:41,416 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:28:41,417 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: src on vertex vertex_1515180518813_0001_2_00 [Map 1] 2018-01-05T11:28:41,418 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_2_00 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:41,418 INFO [App Shared Pool - #0] impl.VertexImpl: Got updated RootInputsSpecs: {src=forAllWorkUnits=true, update=[1]} 2018-01-05T11:28:41,418 INFO [App Shared Pool - #0] impl.VertexImpl: Vertex vertex_1515180518813_0001_2_00 [Map 1] parallelism set to 1 2018-01-05T11:28:41,418 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_2_00 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:41,418 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_2][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_2_00, initRequestedTime=1515180521371, initedTime=1515180521418, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:41,418 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_2_00 [Map 1] with distanceFromRoot: 0 2018-01-05T11:28:41,418 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_2_00 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:28:41,418 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_2][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_2_00, startRequestedTime=1515180521393, startedTime=1515180521418 2018-01-05T11:28:41,418 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_2][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_2_00, reconfigureDoneTime=1515180521418, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:28:41,419 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_2_00 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:41,419 INFO [App Shared Pool - #0] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:28:41,419 INFO [App Shared Pool - #0] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:28:41,419 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_2_00 for vertex: vertex_1515180518813_0001_2_01 [Reducer 2] numStartedSources: 1 numSources: 1 2018-01-05T11:28:41,419 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_2_00 [Map 1] 2018-01-05T11:28:41,419 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_2_01 [Reducer 2] with distanceFromRoot: 1 2018-01-05T11:28:41,419 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_2][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_2_01, startRequestedTime=1515180521419, startedTime=1515180521419 2018-01-05T11:28:41,420 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_2_01 [Reducer 2] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:41,420 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_2][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_2_00_000000, scheduledTime=1515180521420, launchTime=1515180521420 2018-01-05T11:28:41,420 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_2_00 [Map 1] 2018-01-05T11:28:41,421 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_2_00_000000_0, priority=5, capability=, hosts=null 2018-01-05T11:28:41,421 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=5,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:41,421 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=5,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:41,421 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:41,421 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_2_00_000000_0 on hosts=null 2018-01-05T11:28:41,421 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_2_00_000000_0 on hosts=null 2018-01-05T11:28:41,421 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_2_00_000000_0 2018-01-05T11:28:41,421 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:41,422 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #2, task=TaskInfo{task=attempt_1515180518813_0001_2_00_000000_0, priority=5, startTime=0, containerId=null, uniqueId=1, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000002 2018-01-05T11:28:41,422 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_2_00_000000_0; false 2018-01-05T11:28:41,422 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_2_00_000000_0, priority=5, startTime=311541930, containerId=container_111101111_0001_00_000002assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=1, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:41,422 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: OnVertexStarted vertex: Reducer 2 with 0 source tasks and 1 pending tasks 2018-01-05T11:28:41,423 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000002 for node: 10.22.8.182:49818 2018-01-05T11:28:41,423 INFO [App Shared Pool - #1] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Map 1 in vertex: Reducer 2 numBipartiteSourceTasks: 1 2018-01-05T11:28:41,426 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 2, name=ANALYZE TABLE src COMPUTE STATIS...key,value (Stage-0), queryId=djaiswal_20180105112841_c0c75619-7aaf-4c02-a457-8aa38d666b4f 2018-01-05T11:28:41,426 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000002, attempt_1515180518813_0001_2_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:41,426 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000002 for node: 10.22.8.182:49818 2018-01-05T11:28:41,426 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000002 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:41,426 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000002, launchTime=1515180521426 2018-01-05T11:28:41,427 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_2_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000002] on NM: [10.22.8.182:49818] 2018-01-05T11:28:41,427 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:41,427 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:41,427 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_2][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_2_00_000000_0, startTime=1515180521426, containerId=container_111101111_0001_00_000002, nodeId=10.22.8.182:49818 2018-01-05T11:28:41,428 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_2_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112841_c0c75619-7aaf-4c02-a457-8aa38d666b4f, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000002, dagName=ANALYZE TABLE src COMPUTE STATIS...key,value (Stage-0), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{src,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{Reducer 2,org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput,1}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180521371, firstAttemptStartTime=1515180521421, currentAttemptStartTime=1515180521426} 2018-01-05T11:28:41,428 INFO [IPC Server handler 2 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000002, QueryId=djaiswal_20180105112841_c0c75619-7aaf-4c02-a457-8aa38d666b4f-2, DagId=2, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180521428 2018-01-05T11:28:41,428 DEBUG [IPC Server handler 2 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=2} with the ShuffleHandler 2018-01-05T11:28:41,440 INFO [IPC Server handler 2 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:28:41,440 DEBUG [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/2] 2018-01-05T11:28:41,440 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:41,440 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_2_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:41,440 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_2_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180521421, dagStartTime=1515180521371, withinDagPriority=5, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:28:41,440 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_2_00_000000_0 : ACCEPTED 2018-01-05T11:28:41,440 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_2_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:41,441 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_2_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180521421, dagStartTime=1515180521371, withinDagPriority=5, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:28:41,441 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_2_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:41,441 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_2_00_000000_0 to pre-emption queue 2018-01-05T11:28:41,441 INFO [TaskCommunicator # 2] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_2_00_000000_0 2018-01-05T11:28:41,441 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_2_00_000000_0: true 2018-01-05T11:28:41,441 INFO [TaskCommunicator # 2] GuaranteedTasks: Received response for attempt_1515180518813_0001_2_00_000000_0, true 2018-01-05T11:28:41,442 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: taskOwner hashCode:1776250952 2018-01-05T11:28:41,442 INFO [Task-Executor-1] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : ANALYZE TABLE src COMPUTE STATIS...key,value (Stage-0), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_2_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=src, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=Reducer 2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput }}, ] 2018-01-05T11:28:41,442 INFO [Task-Executor-1] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=954728448, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:41,442 INFO [Task-Executor-1] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_2_00_000000_0 vertexName: Map 1 2018-01-05T11:28:41,443 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000002, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_2_00_000000_0, eventCount=1 } 2018-01-05T11:28:41,443 DEBUG [TezTR-518813_1_2_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_2 queryId: djaiswal_20180105112841_c0c75619-7aaf-4c02-a457-8aa38d666b4f instanceType: TaskRunner2Callable 2018-01-05T11:28:41,443 INFO [TezTR-518813_1_2_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_2_00_000000_0 2018-01-05T11:28:41,444 DEBUG [TezTR-518813_1_2_0_0_0] log.PerfLogger: 2018-01-05T11:28:41,445 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:41,446 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:41,450 INFO [IPC Server handler 0 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_2_00_000000_0 sent events: (0-1). 2018-01-05T11:28:41,452 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:28:41,452 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_2_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:28:41,454 DEBUG [TezTR-518813_1_2_0_0_0] log.PerfLogger: 2018-01-05T11:28:41,454 INFO [TezTR-518813_1_2_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:41,454 INFO [TezTR-518813_1_2_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:41,455 INFO [I/O Setup 0 Initialize: {src}] input.MRInput: src using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:28:41,455 INFO [I/O Setup 0 Initialize: {src}] input.MRInputLegacy: src MRInputLegacy deferring initialization 2018-01-05T11:28:41,455 INFO [TezTR-518813_1_2_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:41,456 INFO [I/O Setup 1 Initialize: {Reducer 2}] writers.UnorderedPartitionedKVWriter: Requested BufferSize (tez.runtime.unordered.output.buffer.size-mb) : 100 2018-01-05T11:28:41,456 INFO [TezTR-518813_1_2_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:41,456 INFO [TezTR-518813_1_2_0_0_0] resources.MemoryDistributor: InitialRequests=[src:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [Reducer 2:OUTPUT:104857600:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput] 2018-01-05T11:28:41,456 INFO [TezTR-518813_1_2_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:41,456 INFO [TezTR-518813_1_2_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:41,457 INFO [TezTR-518813_1_2_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 104857600, TotalRequestedScaled: 5.24288E7, TotalJVMHeap: 954728448, TotalAvailable: 639668060, TotalRequested/TotalJVMHeap:0.11 2018-01-05T11:28:41,457 INFO [TezTR-518813_1_2_0_0_0] resources.MemoryDistributor: Allocations=[src:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput:OUTPUT:104857600:104857600] 2018-01-05T11:28:41,457 INFO [TezTR-518813_1_2_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:41,457 INFO [I/O Setup 0 Start: {src}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: src 2018-01-05T11:28:41,457 INFO [TezTR-518813_1_2_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: src being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:28:41,457 INFO [TezTR-518813_1_2_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:28:41,457 INFO [TezTR-518813_1_2_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:28:41,457 INFO [TezTR-518813_1_2_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:41,457 INFO [TezTR-518813_1_2_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_2_00_000000_0 2018-01-05T11:28:41,457 DEBUG [TezTR-518813_1_2_0_0_0] log.PerfLogger: 2018-01-05T11:28:41,458 DEBUG [TezTR-518813_1_2_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_2_00_000000_0_10004 2018-01-05T11:28:41,458 DEBUG [TezTR-518813_1_2_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:41,458 DEBUG [TezTR-518813_1_2_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_2_00_000000_0 2018-01-05T11:28:41,458 INFO [TezTR-518813_1_2_0_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112841_c0c75619-7aaf-4c02-a457-8aa38d666b4f 2018-01-05T11:28:41,458 DEBUG [TezTR-518813_1_2_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:41,458 DEBUG [TezTR-518813_1_2_0_0_0] log.PerfLogger: 2018-01-05T11:28:41,458 DEBUG [TezTR-518813_1_2_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:41,458 DEBUG [TezTR-518813_1_2_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:41,459 DEBUG [TezTR-518813_1_2_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112841_c0c75619-7aaf-4c02-a457-8aa38d666b4f_Map 1__MAP_PLAN__ 2018-01-05T11:28:41,459 INFO [TezTR-518813_1_2_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-41_020_1421415792491703556-1/djaiswal/_tez_scratch_dir/79b4687e-f6c8-4354-a2f6-9ba8fd058008/map.xml 2018-01-05T11:28:41,459 DEBUG [TezTR-518813_1_2_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-41_020_1421415792491703556-1/djaiswal/_tez_scratch_dir/79b4687e-f6c8-4354-a2f6-9ba8fd058008/map.xml 2018-01-05T11:28:41,459 DEBUG [TezTR-518813_1_2_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-41_020_1421415792491703556-1/djaiswal/_tez_scratch_dir/79b4687e-f6c8-4354-a2f6-9ba8fd058008/map.xml 2018-01-05T11:28:41,459 DEBUG [TezTR-518813_1_2_0_0_0] log.PerfLogger: 2018-01-05T11:28:41,459 INFO [TezTR-518813_1_2_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:41,459 DEBUG [TezTR-518813_1_2_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:41,462 DEBUG [TezTR-518813_1_2_0_0_0] log.PerfLogger: 2018-01-05T11:28:41,462 INFO [TezTR-518813_1_2_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.20KB 2018-01-05T11:28:41,463 DEBUG [TezTR-518813_1_2_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:41,463 INFO [TezTR-518813_1_2_0_0_0] tez.MapRecordProcessor: The input names are: [src] 2018-01-05T11:28:41,463 INFO [TezTR-518813_1_2_0_0_0] input.MRInput: Initialized MRInput: src 2018-01-05T11:28:41,464 DEBUG [TezTR-518813_1_2_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 1 2018-01-05T11:28:41,464 INFO [TezTR-518813_1_2_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-41_020_1421415792491703556-1/djaiswal/_tez_scratch_dir/79b4687e-f6c8-4354-a2f6-9ba8fd058008/map.xml 2018-01-05T11:28:41,464 DEBUG [TezTR-518813_1_2_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:41,464 DEBUG [TezTR-518813_1_2_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src/kv1.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct src { string key, string value}, transient_lastDdlTime=1515180520, rawDataSize=5312, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.src}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct src { string key, string value}, transient_lastDdlTime=1515180520, rawDataSize=5312, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.src}, baseFileName=src, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct src { string key, string value}, transient_lastDdlTime=1515180520, rawDataSize=5312, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.src}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct src { string key, string value}, transient_lastDdlTime=1515180520, rawDataSize=5312, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.src}, baseFileName=src, vectorPartitionDesc=null]} 2018-01-05T11:28:41,468 DEBUG [TezTR-518813_1_2_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:41,468 INFO [TezTR-518813_1_2_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:41,468 DEBUG [TezTR-518813_1_2_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:41,469 DEBUG [TezTR-518813_1_2_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:41,469 INFO [TezTR-518813_1_2_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-41_020_1421415792491703556-1/djaiswal/_tez_scratch_dir/79b4687e-f6c8-4354-a2f6-9ba8fd058008/map.xml 2018-01-05T11:28:41,469 DEBUG [TezTR-518813_1_2_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:41,469 INFO [TezTR-518813_1_2_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_2_00_000000_0 2018-01-05T11:28:41,469 INFO [TezTR-518813_1_2_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [string, string] (isAcidRead false) 2018-01-05T11:28:41,470 DEBUG [TezTR-518813_1_2_0_0_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:41,470 DEBUG [TezTR-518813_1_2_0_0_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:41,470 DEBUG [TezTR-518813_1_2_0_0_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:41,470 DEBUG [TezTR-518813_1_2_0_0_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:41,470 WARN [TezTR-518813_1_2_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src/kv1.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:28:41,470 DEBUG [TezTR-518813_1_2_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:28:41,470 WARN [TezTR-518813_1_2_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src/kv1.txt:0+5812 2018-01-05T11:28:41,471 DEBUG [TezTR-518813_1_2_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src/kv1.txt 2018-01-05T11:28:41,471 INFO [TezTR-518813_1_2_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src/kv1.txt:0+5812], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=5812} 2018-01-05T11:28:41,471 INFO [TezTR-518813_1_2_0_0_0] input.MRInput: src initialized RecordReader from event 2018-01-05T11:28:41,471 DEBUG [TezTR-518813_1_2_0_0_0] tez.MapRecordProcessor: Starting Output: Reducer 2 2018-01-05T11:28:41,473 INFO [TezTR-518813_1_2_0_0_0] writers.BaseUnorderedPartitionedKVWriter: Instantiating Partitioner: [org.apache.tez.mapreduce.partition.MRPartitioner] 2018-01-05T11:28:41,514 INFO [TezTR-518813_1_2_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: numBuffers=2, sizePerBuffer=52428800, skipBuffers=true, numPartitions=1, availableMemory=104857600, maxSingleBufferSizeBytes=2147483647, pipelinedShuffle=false, isFinalMergeEnabled=true, numPartitions=1, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:28:41,514 INFO [TezTR-518813_1_2_0_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:28:41,514 INFO [TezTR-518813_1_2_0_0_0] exec.MapOperator: Initializing operator MAP[0] 2018-01-05T11:28:41,514 DEBUG [TezTR-518813_1_2_0_0_0] exec.MapOperator: Initialization Done 0 MAP 2018-01-05T11:28:41,514 DEBUG [TezTR-518813_1_2_0_0_0] exec.MapOperator: Operator 0 MAP initialized 2018-01-05T11:28:41,514 DEBUG [TezTR-518813_1_2_0_0_0] exec.MapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:28:41,515 DEBUG [TezTR-518813_1_2_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:41,515 DEBUG [TezTR-518813_1_2_0_0_0] exec.MapOperator: Adding alias src to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src 2018-01-05T11:28:41,515 DEBUG [TezTR-518813_1_2_0_0_0] exec.MapOperator: dump TS[0] struct 2018-01-05T11:28:41,515 INFO [TezTR-518813_1_2_0_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:28:41,515 INFO [TezTR-518813_1_2_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:41,515 DEBUG [TezTR-518813_1_2_0_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:28:41,516 INFO [TezTR-518813_1_2_0_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:28:41,516 DEBUG [TezTR-518813_1_2_0_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:28:41,516 DEBUG [TezTR-518813_1_2_0_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:28:41,516 DEBUG [TezTR-518813_1_2_0_0_0] exec.TableScanOperator: Initializing children of 0 TS 2018-01-05T11:28:41,516 DEBUG [TezTR-518813_1_2_0_0_0] exec.SelectOperator: Initializing child 1 SEL 2018-01-05T11:28:41,516 INFO [TezTR-518813_1_2_0_0_0] exec.SelectOperator: Initializing operator SEL[1] 2018-01-05T11:28:41,519 INFO [TezTR-518813_1_2_0_0_0] exec.SelectOperator: SELECT struct 2018-01-05T11:28:41,519 DEBUG [TezTR-518813_1_2_0_0_0] exec.SelectOperator: Initialization Done 1 SEL 2018-01-05T11:28:41,519 DEBUG [TezTR-518813_1_2_0_0_0] exec.SelectOperator: Operator 1 SEL initialized 2018-01-05T11:28:41,519 DEBUG [TezTR-518813_1_2_0_0_0] exec.SelectOperator: Initializing children of 1 SEL 2018-01-05T11:28:41,520 DEBUG [TezTR-518813_1_2_0_0_0] exec.GroupByOperator: Initializing child 2 GBY 2018-01-05T11:28:41,520 INFO [TezTR-518813_1_2_0_0_0] exec.GroupByOperator: Initializing operator GBY[2] 2018-01-05T11:28:41,520 DEBUG [TezTR-518813_1_2_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:41,520 DEBUG [TezTR-518813_1_2_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:41,524 INFO [TezTR-518813_1_2_0_0_0] exec.GroupByOperator: Max hash table memory: 93860656 bytes 2018-01-05T11:28:41,525 INFO [TezTR-518813_1_2_0_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:41,525 DEBUG [TezTR-518813_1_2_0_0_0] exec.GroupByOperator: Initialization Done 2 GBY 2018-01-05T11:28:41,525 DEBUG [TezTR-518813_1_2_0_0_0] exec.GroupByOperator: Operator 2 GBY initialized 2018-01-05T11:28:41,525 DEBUG [TezTR-518813_1_2_0_0_0] exec.GroupByOperator: Initializing children of 2 GBY 2018-01-05T11:28:41,525 DEBUG [TezTR-518813_1_2_0_0_0] exec.ReduceSinkOperator: Initializing child 3 RS 2018-01-05T11:28:41,525 INFO [TezTR-518813_1_2_0_0_0] exec.ReduceSinkOperator: Initializing operator RS[3] 2018-01-05T11:28:41,525 DEBUG [TezTR-518813_1_2_0_0_0] exec.ReduceSinkOperator: keys size is 0 2018-01-05T11:28:41,525 INFO [TezTR-518813_1_2_0_0_0] exec.ReduceSinkOperator: Using tag = -1 2018-01-05T11:28:41,525 DEBUG [TezTR-518813_1_2_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:41,528 DEBUG [TezTR-518813_1_2_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:41,541 DEBUG [TezTR-518813_1_2_0_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:28:41,541 DEBUG [TezTR-518813_1_2_0_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS 2018-01-05T11:28:41,541 DEBUG [TezTR-518813_1_2_0_0_0] exec.ReduceSinkOperator: Operator 3 RS initialized 2018-01-05T11:28:41,541 DEBUG [TezTR-518813_1_2_0_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS done is reset. 2018-01-05T11:28:41,541 DEBUG [TezTR-518813_1_2_0_0_0] exec.GroupByOperator: Initialization Done 2 GBY done is reset. 2018-01-05T11:28:41,541 DEBUG [TezTR-518813_1_2_0_0_0] exec.SelectOperator: Initialization Done 1 SEL done is reset. 2018-01-05T11:28:41,541 DEBUG [TezTR-518813_1_2_0_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:28:41,541 INFO [TezTR-518813_1_2_0_0_0] exec.OperatorUtils: Setting output collector: RS[3] --> Reducer 2 2018-01-05T11:28:41,541 DEBUG [TezTR-518813_1_2_0_0_0] log.PerfLogger: 2018-01-05T11:28:41,541 INFO [TezTR-518813_1_2_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-41_020_1421415792491703556-1/djaiswal/_tez_scratch_dir/79b4687e-f6c8-4354-a2f6-9ba8fd058008/map.xml 2018-01-05T11:28:41,541 DEBUG [TezTR-518813_1_2_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:41,542 DEBUG [TezTR-518813_1_2_0_0_0] exec.MapOperator: Processing alias(es) src for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src/kv1.txt 2018-01-05T11:28:41,549 INFO [TezTR-518813_1_2_0_0_0] exec.MapOperator: MAP[0]: records read - 1 2018-01-05T11:28:41,550 INFO [TezTR-518813_1_2_0_0_0] exec.MapOperator: MAP[0]: records read - 10 2018-01-05T11:28:41,555 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000002, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_2_00_000000_0, eventCount=1 } 2018-01-05T11:28:41,556 INFO [TezTR-518813_1_2_0_0_0] exec.MapOperator: MAP[0]: records read - 100 2018-01-05T11:28:41,557 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:41,572 DEBUG [TezTR-518813_1_2_0_0_0] log.PerfLogger: 2018-01-05T11:28:41,572 DEBUG [TezTR-518813_1_2_0_0_0] mr.ObjectCache: djaiswal_20180105112841_c0c75619-7aaf-4c02-a457-8aa38d666b4f_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:28:41,573 DEBUG [TezTR-518813_1_2_0_0_0] exec.MapOperator: close called for operator MAP[0] 2018-01-05T11:28:41,573 INFO [TezTR-518813_1_2_0_0_0] exec.MapOperator: Closing operator MAP[0] 2018-01-05T11:28:41,573 INFO [TezTR-518813_1_2_0_0_0] exec.MapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:500, 2018-01-05T11:28:41,573 DEBUG [TezTR-518813_1_2_0_0_0] exec.MapOperator: Closing child = TS[0] 2018-01-05T11:28:41,573 DEBUG [TezTR-518813_1_2_0_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:28:41,573 DEBUG [TezTR-518813_1_2_0_0_0] exec.TableScanOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:41,573 INFO [TezTR-518813_1_2_0_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:28:41,573 DEBUG [TezTR-518813_1_2_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:41,573 DEBUG [TezTR-518813_1_2_0_0_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-41_029_8026269352618129281-1/-mr-10001 2018-01-05T11:28:41,573 INFO [TezTR-518813_1_2_0_0_0] exec.TableScanOperator: publishing : default.src/ : {numRows=500, rawDataSize=5312} 2018-01-05T11:28:41,597 DEBUG [TezTR-518813_1_2_0_0_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-41_029_8026269352618129281-1/-mr-10001/tmpstats-0_0 2018-01-05T11:28:41,597 DEBUG [TezTR-518813_1_2_0_0_0] fs.FSStatsPublisher: Writing stats in it : {default.src/={numRows=500, rawDataSize=5312}} 2018-01-05T11:28:41,597 DEBUG [TezTR-518813_1_2_0_0_0] exec.TableScanOperator: Closing child = SEL[1] 2018-01-05T11:28:41,597 DEBUG [TezTR-518813_1_2_0_0_0] exec.SelectOperator: close called for operator SEL[1] 2018-01-05T11:28:41,597 DEBUG [TezTR-518813_1_2_0_0_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:41,597 INFO [TezTR-518813_1_2_0_0_0] exec.SelectOperator: Closing operator SEL[1] 2018-01-05T11:28:41,597 DEBUG [TezTR-518813_1_2_0_0_0] exec.SelectOperator: Closing child = GBY[2] 2018-01-05T11:28:41,597 DEBUG [TezTR-518813_1_2_0_0_0] exec.GroupByOperator: close called for operator GBY[2] 2018-01-05T11:28:41,597 DEBUG [TezTR-518813_1_2_0_0_0] exec.GroupByOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:41,597 INFO [TezTR-518813_1_2_0_0_0] exec.GroupByOperator: Closing operator GBY[2] 2018-01-05T11:28:41,597 INFO [TezTR-518813_1_2_0_0_0] exec.GroupByOperator: Begin Hash Table flush: size = 1 2018-01-05T11:28:41,599 INFO [TezTR-518813_1_2_0_0_0] exec.ReduceSinkOperator: keys are [] num distributions: 0 2018-01-05T11:28:41,600 WARN [TezTR-518813_1_2_0_0_0] objectinspector.StandardStructObjectInspector: Trying to access 6 fields inside a list of 7 elements: [String, 3, 1406, 500, 0, 48 4c 4c e0 b5 02 b5 02 c1 a2 58 82 e7 cd 05 bf 93 6a ff c3 51 c0 9b 68 c5 d3 d8 07 fc a3 98 03 bf 92 89 01 81 c8 dc 04 c1 db 96 01 ff 95 e7 02 80 c9 fd 01 c0 bb f2 09 bf c3 37 80 fc 11 c1 e8 c8 07 ff 92 a3 02 c2 8b d5 0a 80 81 ed 03 fe 86 c2 02 80 af 30 c1 88 0e 80 cf 0e bf e8 be 04 80 9f a0 0b 80 be d3 10 c1 ab f3 05 ff eb 19 c0 ba a8 05 c1 c8 1a 80 da 0f c1 8f 86 01 fe e8 b8 04 c4 8c 9c 01 bc 87 ca 02 81 9e b6 04 bf ca ca 08 c0 de 1d c0 c1 ed 03 c0 c0 9d 09 81 ed 35 81 b2 91 05 bf ee ee 04 c1 81 e3 02 fe 9a 6b c0 aa d1 01 82 ba d0 01 81 a5 91 06 fd 8f 50 81 b9 87 04 ff 95 43 80 d7 22 c1 e1 a4 03 81 df f7 04 be c2 ce 01 80 f7 71 80 9a ff 05 c0 9c b5 03 81 da b4 03 ff e2 c4 08 c0 df fd 01 c0 92 56 c0 c5 aa 01 80 fe e0 03 81 ae d7 07 81 c7 86 01 fe b7 a6 04 c1 e3 4e 80 e1 db 01 ff fa c1 07 c0 ca b1 03 c0 ef c8 04 c1 ad 48 ff ed e7 02 80 fd 07 89 f9 93 04 80 bc 16 b8 87 7e ff f9 a4 0a c0 95 2c c0 9c 8a 01 c0 9f db 02 c5 dd 53 bb fe d2 0e c0 ae f4 01 80 e5 fe 02 80 f9 fa 13 c0 ff 8a 07 80 ad eb 01 c0 96 07 80 fe 8b 04 80 fe ca 03 82 ee f5 01 be bc 31 80 ff 81 02 80 87 e1 02 c0 de b1 04 c0 93 62 84 a1 e0 03 bd 9c 44 80 ac 87 06 c1 89 9e 03 fe d3 a4 05 81 9d c4 03 c2 d8 73 be b6 fd 04 81 b6 85 01 be 95 b8 08 81 a9 f6 0c ff f6 47 80 ae d6 08 80 94 97 03 83 b5 14 fd 90 5f 81 c8 4b ff 83 bc 02 c0 ce f2 02 81 a1 d5 06 c0 c4 dc 03 c3 8a 82 03 fc f2 c6 03 c3 db 42 be bf 82 03 bf d6 dd 07 c4 d3 9c 01 fd d5 94 09 c0 9a d9 09 81 b1 8e 07 82 b0 f7 05 bc c2 c2 0b 80 8a 1d c1 eb b8 03 ff dc de 01 85 f4 b7 07 fb d3 8f 07 c0 b2 92 08 80 89 0b c1 b2 ae 01 81 d1 a9 01 be bd a6 01 c0 cf f1 01 80 d3 f1 02 c0 c4 9c 03 81 97 ac 01 81 a6 89 05 80 a8 0a c0 d4 87 04 fe ec 30 80 df 90 06 c1 a7 3c bf a5 d7 0a c0 b9 df 01 82 de 28 fe 91 e7 02 c2 b2 ca 01 bf c2 58 ff b2 2f c0 d1 0c 82 af bb 02 c0 d0 1f fe 95 c0 01 80 f2 75 c2 8a ca 01 be f7 af 02 81 f8 3c c2 8e a8 03 be 96 78 ff ca 39 c2 c3 32 ff ec 80 03 bf ff 23 84 ad bc 06 bc f1 d7 01 82 9a 4e bf d2 dd 02 80 8c 9a 02 ff c2 9f 05 c0 cd 9d 01 c0 96 9d 03 81 82 29 c0 b9 dc 01 bf b1 40 81 86 cf 09 81 99 91 01 be b3 c3 04 81 80 b6 06 c1 eb 88 01 bf ad 6e c1 fa 83 03 c0 9e 01 fe ca c4 01 84 bc 61 fc d8 a1 02 c1 98 b3 04 bf 97 c3 0a 82 d1 57 fe ee a3 03 c5 c1 e8 02 fc de d0 02 ff 94 c3 04 c1 cc be 01 bf d0 d2 02 c0 ee fd 05 84 bc d9 06 bc a0 c4 01 c0 fa 81 05 c1 e1 23 ff cd d9 12 c0 8a db 04 80 99 ee 05 c0 fe a8 05 c0 fe dd 05 c0 9d f6 0a 80 c1 ac 01 c3 95 20 fd 96 d2 03 c0 c8 9e 04 80 b6 e9 02 c1 cd 99 06 81 b7 9c 01 80 af 09 ff 97 83 09 ff e3 4f 89 81 15 f7 c4 d7 01 c0 89 0b c1 e6 90 01 80 ee 82 02 81 da 89 01 c0 f1 12 be db cf 06 c1 f1 a5 02 ff b9 cb 02 c0 87 89 03 c0 a3 06 81 fa 48 ff 94 9f 02 82 c3 c2 06 bf b9 9c 05 81 ed cb 02 82 85 82 02 bc a0 e9 02 c0 f1 e6 01 c2 f8 c2 05 fe c0 de 05 80 a1 d4 02 c0 fa 9a 02 c1 8c a0 08 bf 93 4c 84 ce dc 02 bd c1 e3 04 c1 ff 9f 01 bf d5 e4 02 80 a4 a3 06 83 c1 60 fc 9f 4e c1 f9 cd 0a ff e9 92 02 c2 cf e8 04 be df 97 03 80 aa 96 01 80 94 e4 03 80 86 88 09 81 a6 cc 01 82 89 85 07 bf de 87 06 ff e7 8d 01 c4 cd fb 04 bf d2 ee 07 bc e4 d6 03 c3 e1 18 83 a3 f5 03 bb 9a ab 03 ff c8 21 c0 c3 1e c0 b0 0d 80 e8 cf 02 c4 ab d1 01 bd 9a 75 80 bc 1a c0 96 86 02 ff c9 c0 03 c2 fc 97 03 be ab dc 01 c1 b5 ba 03 83 ce 93 01 bd 97 e6 02 80 d6 07 ff ec d7 02 87 cd a6 02 b9 c5 b6 05 c2 fd b9 04 ff f9 a3 01 ff e5 ad 02 c2 8f 37 bf ac 76 c4 e0 84 01 bb b7 a9 04 c0 dc 3e c0 a2 d4 02 c0 83 20 81 dc 8f 01 80 aa 90 02 80 cd fa 05 c0 bf fc 03 80 b4 b1 04 80 c6 f0 01 ff a6 c1 08 80 a9 f8 07, null] 2018-01-05T11:28:41,600 WARN [TezTR-518813_1_2_0_0_0] objectinspector.StandardStructObjectInspector: ignoring similar errors. 2018-01-05T11:28:41,601 DEBUG [TezTR-518813_1_2_0_0_0] exec.GroupByOperator: Closing child = RS[3] 2018-01-05T11:28:41,601 DEBUG [TezTR-518813_1_2_0_0_0] exec.ReduceSinkOperator: close called for operator RS[3] 2018-01-05T11:28:41,601 DEBUG [TezTR-518813_1_2_0_0_0] exec.ReduceSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:41,601 INFO [TezTR-518813_1_2_0_0_0] exec.ReduceSinkOperator: Closing operator RS[3] 2018-01-05T11:28:41,601 INFO [TezTR-518813_1_2_0_0_0] exec.ReduceSinkOperator: RECORDS_OUT_INTERMEDIATE_Map_1:1, 2018-01-05T11:28:41,601 DEBUG [TezTR-518813_1_2_0_0_0] exec.ReduceSinkOperator: 3 Close done 2018-01-05T11:28:41,601 DEBUG [TezTR-518813_1_2_0_0_0] exec.GroupByOperator: 2 Close done 2018-01-05T11:28:41,601 DEBUG [TezTR-518813_1_2_0_0_0] exec.SelectOperator: 1 Close done 2018-01-05T11:28:41,601 DEBUG [TezTR-518813_1_2_0_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:28:41,601 DEBUG [TezTR-518813_1_2_0_0_0] exec.MapOperator: 0 Close done 2018-01-05T11:28:41,601 INFO [TezTR-518813_1_2_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_2_00_000000_0 2018-01-05T11:28:41,601 INFO [TezTR-518813_1_2_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: Waiting for all spills to complete : Pending : 0 2018-01-05T11:28:41,601 INFO [TezTR-518813_1_2_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: All spills complete 2018-01-05T11:28:41,620 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,620 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: Status: Running (Executing on YARN cluster with App id application_1515180518813_0001) 2018-01-05T11:28:41,624 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,624 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 0(+1)/1 Reducer 2: 0/1 2018-01-05T11:28:41,648 DEBUG [TezTR-518813_1_2_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:41,648 INFO [TezTR-518813_1_2_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:41,649 INFO [TezTR-518813_1_2_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_2_00_000000_0, askedToStop=false 2018-01-05T11:28:41,649 INFO [TezTR-518813_1_2_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_2_00_000000_0, stopRequested=false 2018-01-05T11:28:41,649 INFO [TezTR-518813_1_2_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_2_00_000000_0: Counters: 14 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=500, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=2374, OUTPUT_BYTES_WITH_OVERHEAD=2384, OUTPUT_BYTES_PHYSICAL=2412, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500, RECORDS_OUT_INTERMEDIATE_Map_1=1]] 2018-01-05T11:28:41,649 DEBUG [TezTR-518813_1_2_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_2_0_0_0 thread id: 338 scheme: file bytesRead: 5872 bytesWritten: 2502 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:41,650 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_2_00_000000_0, isTaskDone=true 2018-01-05T11:28:41,650 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000002, requestId=3, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_2_00_000000_0, eventCount=4 } 2018-01-05T11:28:41,655 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_2][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_2_00_000000_0, creationTime=1515180521420, allocationTime=1515180521423, startTime=1515180521426, finishTime=1515180521655, timeTaken=229, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:41,656 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_2_00_000000_0 2018-01-05T11:28:41,656 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_2_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:41,656 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_2_00_000000_0; false 2018-01-05T11:28:41,656 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_2_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:41,656 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:41,656 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:41,656 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_2][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_2_00_000000, startTime=1515180521426, finishTime=1515180521656, timeTaken=230, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_2_00_000000_0, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=5872, FILE_BYTES_WRITTEN=2502, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=500, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=2374, OUTPUT_BYTES_WITH_OVERHEAD=2384, OUTPUT_BYTES_PHYSICAL=2412, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500, RECORDS_OUT_INTERMEDIATE_Map_1=1 2018-01-05T11:28:41,657 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=3, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:41,657 INFO [Task-Executor-1] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_2_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:41,657 INFO [Task-Executor-1] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_2_00_000000_0 2018-01-05T11:28:41,658 INFO [Task-Executor-1] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000002=216 2018-01-05T11:28:41,658 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_2_00_000000_0: true 2018-01-05T11:28:41,658 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_2_00_000000_0 2018-01-05T11:28:41,658 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000002, QueryId=djaiswal_20180105112841_c0c75619-7aaf-4c02-a457-8aa38d666b4f-2, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:28:41,658 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_2_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:41,658 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_2_00_000000_0 2018-01-05T11:28:41,658 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_2_00 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:41,658 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_2_00 [Map 1] 2018-01-05T11:28:41,660 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_2][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_2_00, initRequestedTime=1515180521371, initedTime=1515180521418, startRequestedTime=1515180521393, startedTime=1515180521418, finishTime=1515180521658, timeTaken=240, status=SUCCEEDED, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=5872, FILE_BYTES_WRITTEN=2502, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=500, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=2374, OUTPUT_BYTES_WITH_OVERHEAD=2384, OUTPUT_BYTES_PHYSICAL=2412, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500, RECORDS_OUT_INTERMEDIATE_Map_1=1, vertexStats=firstTaskStartTime=1515180521426, firstTasksToStart=[ task_1515180518813_0001_2_00_000000 ], lastTaskFinishTime=1515180521655, lastTasksToFinish=[ task_1515180518813_0001_2_00_000000 ], minTaskDuration=229, maxTaskDuration=229, avgTaskDuration=229.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_2_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_2_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:41,660 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_2_00 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:41,664 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_2][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000002, stoppedTime=1515180521664, exitStatus=0 2018-01-05T11:28:41,664 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000002 exited with diagnostics set to Container completed. 2018-01-05T11:28:41,664 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_2_01 [Reducer 2] attempt: attempt_1515180518813_0001_2_00_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:28:41,664 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_2_00 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2 2018-01-05T11:28:41,664 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null 2018-01-05T11:28:41,665 INFO [App Shared Pool - #1] vertexmanager.ShuffleVertexManagerBase: All source tasks completed. Ramping up 1 remaining tasks for vertex: Reducer 2 2018-01-05T11:28:41,665 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000002 2018-01-05T11:28:41,665 INFO [App Shared Pool - #1] vertexmanager.ShuffleVertexManagerBase: Scheduling 1 tasks for vertex: Reducer 2 with totalTasks: 1. 1 source tasks completed out of 1. MinSourceTaskCompletedFraction: 1.0 min: 0.05 max: 0.75 2018-01-05T11:28:41,665 INFO [App Shared Pool - #1] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Map 1, destinationVertexName=Reducer 2. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.UnorderedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:28:41,665 INFO [App Shared Pool - #1] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_2_01 [Reducer 2] 2018-01-05T11:28:41,665 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_2][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_2_01_000000, scheduledTime=1515180521665, launchTime=1515180521665 2018-01-05T11:28:41,666 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_2_01 [Reducer 2] 2018-01-05T11:28:41,666 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_2_01_000000_0, priority=14, capability=, hosts=null 2018-01-05T11:28:41,666 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:41,666 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:41,666 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:41,666 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_2_01_000000_0 on hosts=null 2018-01-05T11:28:41,666 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_2_01_000000_0 on hosts=null 2018-01-05T11:28:41,666 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_2_01_000000_0 2018-01-05T11:28:41,666 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:41,666 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #3, task=TaskInfo{task=attempt_1515180518813_0001_2_01_000000_0, priority=14, startTime=0, containerId=null, uniqueId=2, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000003 2018-01-05T11:28:41,666 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_2_01_000000_0; false 2018-01-05T11:28:41,666 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_2_01_000000_0, priority=14, startTime=311542175, containerId=container_111101111_0001_00_000003assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=2, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:41,667 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000003 for node: 10.22.8.182:49818 2018-01-05T11:28:41,667 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000003, attempt_1515180518813_0001_2_01_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:41,668 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000003 for node: 10.22.8.182:49818 2018-01-05T11:28:41,672 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000003 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:41,672 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000003, launchTime=1515180521672 2018-01-05T11:28:41,672 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=2}, sourceName=Map 1, state=S_SUCCEEDED 2018-01-05T11:28:41,673 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_2_01_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000003] on NM: [10.22.8.182:49818] 2018-01-05T11:28:41,673 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:41,673 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:41,673 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_2][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_2_01_000000_0, startTime=1515180521672, containerId=container_111101111_0001_00_000003, nodeId=10.22.8.182:49818 2018-01-05T11:28:41,675 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_2_01_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112841_c0c75619-7aaf-4c02-a457-8aa38d666b4f, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000003, dagName=ANALYZE TABLE src COMPUTE STATIS...key,value (Stage-0), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.UnorderedKVInput,1}}, Outputs={{out_Reducer 2,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=2, completedTaskCount=1, dagStartTime=1515180521371, firstAttemptStartTime=1515180521666, currentAttemptStartTime=1515180521671} 2018-01-05T11:28:41,675 INFO [IPC Server handler 3 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000003, QueryId=djaiswal_20180105112841_c0c75619-7aaf-4c02-a457-8aa38d666b4f-2, DagId=2, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, SubmitTime=1515180521675 2018-01-05T11:28:41,675 DEBUG [IPC Server handler 3 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=2} with the ShuffleHandler 2018-01-05T11:28:41,675 DEBUG [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/2] 2018-01-05T11:28:41,676 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:41,676 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_2_01_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:41,676 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_2_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180521666, dagStartTime=1515180521371, withinDagPriority=14, vertexParallelism= 1, selfAndUpstreamParallelism= 2, selfAndUpstreamComplete= 1}] 2018-01-05T11:28:41,676 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_2_01_000000_0 : ACCEPTED 2018-01-05T11:28:41,676 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_2_01_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:41,676 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_2_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180521666, dagStartTime=1515180521371, withinDagPriority=14, vertexParallelism= 1, selfAndUpstreamParallelism= 2, selfAndUpstreamComplete= 1} 2018-01-05T11:28:41,676 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_2_01_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:41,676 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_2_01_000000_0 to pre-emption queue 2018-01-05T11:28:41,676 INFO [TaskCommunicator # 4] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_2_01_000000_0 2018-01-05T11:28:41,676 INFO [TaskCommunicator # 4] GuaranteedTasks: Received response for attempt_1515180518813_0001_2_01_000000_0, true 2018-01-05T11:28:41,676 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_2_01_000000_0: true 2018-01-05T11:28:41,676 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: taskOwner hashCode:1776250952 2018-01-05T11:28:41,677 INFO [Task-Executor-2] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : ANALYZE TABLE src COMPUTE STATIS...key,value (Stage-0), VertexName: Reducer 2, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_2_01_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:28:41,677 INFO [Task-Executor-2] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=954728448, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:41,677 INFO [Task-Executor-2] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_2_01_000000_0 vertexName: Reducer 2 2018-01-05T11:28:41,677 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000003, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_2_01_000000_0, eventCount=1 } 2018-01-05T11:28:41,677 DEBUG [TezTR-518813_1_2_1_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_2 queryId: djaiswal_20180105112841_c0c75619-7aaf-4c02-a457-8aa38d666b4f instanceType: TaskRunner2Callable 2018-01-05T11:28:41,678 INFO [TezTR-518813_1_2_1_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_2_01_000000_0 2018-01-05T11:28:41,679 DEBUG [TezTR-518813_1_2_1_0_0] log.PerfLogger: 2018-01-05T11:28:41,680 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:41,681 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:41,686 DEBUG [TezTR-518813_1_2_1_0_0] log.PerfLogger: 2018-01-05T11:28:41,686 INFO [TezTR-518813_1_2_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:41,686 INFO [TezTR-518813_1_2_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:41,686 INFO [I/O Setup 1 Initialize: {out_Reducer 2}] output.MROutput: out_Reducer 2: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:28:41,686 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:41,686 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:41,686 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:41,686 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:41,686 INFO [IPC Server handler 1 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_2_01_000000_0 1 events [0,1) total 1 vertex_1515180518813_0001_2_01 [Reducer 2] 2018-01-05T11:28:41,686 INFO [TezTR-518813_1_2_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:41,689 INFO [TezTR-518813_1_2_1_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:41,690 INFO [TezTR-518813_1_2_1_0_0] resources.MemoryDistributor: InitialRequests=[out_Reducer 2:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.UnorderedKVInput] 2018-01-05T11:28:41,690 INFO [TezTR-518813_1_2_1_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:41,690 INFO [TezTR-518813_1_2_1_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:41,690 INFO [TezTR-518813_1_2_1_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 131404920, TotalRequestedScaled: 6.570246E7, TotalJVMHeap: 954728448, TotalAvailable: 639668060, TotalRequested/TotalJVMHeap:0.14 2018-01-05T11:28:41,690 INFO [TezTR-518813_1_2_1_0_0] resources.MemoryDistributor: Allocations=[out_Reducer 2:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [Map 1:org.apache.tez.runtime.library.input.UnorderedKVInput:INPUT:131404920:131404920] 2018-01-05T11:28:41,690 INFO [TezTR-518813_1_2_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:41,690 INFO [TezTR-518813_1_2_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 0 2018-01-05T11:28:41,690 INFO [TezTR-518813_1_2_1_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:41,690 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:28:41,690 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_2_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:28:41,690 INFO [TezTR-518813_1_2_1_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_2_01_000000_0 2018-01-05T11:28:41,691 DEBUG [TezTR-518813_1_2_1_0_0] log.PerfLogger: 2018-01-05T11:28:41,691 DEBUG [TezTR-518813_1_2_1_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_2_01_000000_0_10007 2018-01-05T11:28:41,691 DEBUG [TezTR-518813_1_2_1_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:41,693 DEBUG [TezTR-518813_1_2_1_0_0] mr.ObjectCache: Creating djaiswal_20180105112841_c0c75619-7aaf-4c02-a457-8aa38d666b4f_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:28:41,693 INFO [TezTR-518813_1_2_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-41_020_1421415792491703556-1/djaiswal/_tez_scratch_dir/b1a28a52-de85-4b1c-9d33-1e6fe04a0df6/reduce.xml 2018-01-05T11:28:41,693 DEBUG [TezTR-518813_1_2_1_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-41_020_1421415792491703556-1/djaiswal/_tez_scratch_dir/b1a28a52-de85-4b1c-9d33-1e6fe04a0df6/reduce.xml 2018-01-05T11:28:41,693 DEBUG [TezTR-518813_1_2_1_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-41_020_1421415792491703556-1/djaiswal/_tez_scratch_dir/b1a28a52-de85-4b1c-9d33-1e6fe04a0df6/reduce.xml 2018-01-05T11:28:41,693 DEBUG [TezTR-518813_1_2_1_0_0] log.PerfLogger: 2018-01-05T11:28:41,693 INFO [TezTR-518813_1_2_1_0_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:28:41,693 DEBUG [TezTR-518813_1_2_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:41,700 DEBUG [TezTR-518813_1_2_1_0_0] log.PerfLogger: 2018-01-05T11:28:41,700 INFO [TezTR-518813_1_2_1_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 2.01KB 2018-01-05T11:28:41,700 DEBUG [TezTR-518813_1_2_1_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:41,700 DEBUG [TezTR-518813_1_2_1_0_0] log.PerfLogger: 2018-01-05T11:28:41,700 DEBUG [TezTR-518813_1_2_1_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:41,700 DEBUG [TezTR-518813_1_2_1_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:41,701 DEBUG [TezTR-518813_1_2_1_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:41,701 INFO [TezTR-518813_1_2_1_0_0] impl.SimpleFetchedInputAllocator: Map_1: RequestedMemory=131404920, AssignedMemory=131404920, maxSingleShuffleLimit=32851230 2018-01-05T11:28:41,713 INFO [TezTR-518813_1_2_1_0_0] impl.ShuffleManager: Map_1: numInputs=1, compressionCodec=NoCompressionCodec, numFetchers=1, ifileBufferSize=4096, ifileReadAheadEnabled=true, ifileReadAheadLength=4194304, localDiskFetchEnabled=false, sharedFetchEnabled=false, keepAlive=false, keepAliveMaxConnections=20, connectionTimeout=20000, readTimeout=180000, bufferSize=8192, bufferSize=8192, maxTaskOutputAtOnce=20 2018-01-05T11:28:41,723 INFO [TezTR-518813_1_2_1_0_0] impl.ShuffleInputEventHandlerImpl: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:28:41,723 INFO [TezTR-518813_1_2_1_0_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:28:41,762 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000002, QueryId=djaiswal_20180105112841_c0c75619-7aaf-4c02-a457-8aa38d666b4f-2, DagId=2, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-1, Succeeded=true, StartTime=1515180521441, EndTime=1515180521762 2018-01-05T11:28:41,787 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=2&reduce=0&map=attempt_1515180518813_0001_2_00_000000_0_10006 mapId: [attempt_1515180518813_0001_2_00_000000_0_10006] reduceId: [0] jobId: [job_1515180518813_0001] dagId: [2] keepAlive: false 2018-01-05T11:28:41,789 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=2&reduce=0&map=attempt_1515180518813_0001_2_00_000000_0_10006; hash=...MxkMmIdvPDjCY 2018-01-05T11:28:41,789 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=2&reduce=0&map=attempt_1515180518813_0001_2_00_000000_0_10006;reply=CxKdH9HvTLmiU 2018-01-05T11:28:41,792 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=2, user='djaiswal', attemptId='attempt_1515180518813_0001_2_00_000000_0_10006'} via loader 2018-01-05T11:28:41,793 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=2, user='djaiswal', attemptId='attempt_1515180518813_0001_2_00_000000_0_10006'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:28:41,793 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000003, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_2_01_000000_0, eventCount=1 } 2018-01-05T11:28:41,794 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_2_00_000000_0_10006 not found 2018-01-05T11:28:41,794 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_2_00_000000_0_10006,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/2/output/attempt_1515180518813_0001_2_00_000000_0_10006/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/2/output/attempt_1515180518813_0001_2_00_000000_0_10006/file.out.index 2018-01-05T11:28:41,795 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:28:41,796 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Content Length in shuffle : 2442 2018-01-05T11:28:41,802 INFO [Fetcher_B {Map_1} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=2&reduce=0&map=attempt_1515180518813_0001_2_00_000000_0_10006 sent hash and receievd reply 0 ms 2018-01-05T11:28:41,823 INFO [Fetcher_B {Map_1} #0] ShuffleManager.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_2_00_000000_0_10006} to MEMORY, csize=2388, dsize=2384, EndTime=1515180521823, TimeTaken=20, Rate=0.11 MB/s 2018-01-05T11:28:41,823 INFO [Fetcher_B {Map_1} #0] impl.ShuffleManager: All inputs fetched for input vertex : Map 1 2018-01-05T11:28:41,823 INFO [TezTR-518813_1_2_1_0_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:28:41,823 INFO [Fetcher_B {Map_1} #0] impl.ShuffleManager: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:28:41,823 INFO [ShuffleRunner {Map_1}] impl.ShuffleManager: Map_1: Shutting down FetchScheduler, Was Interrupted: false 2018-01-05T11:28:41,823 INFO [TezTR-518813_1_2_1_0_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:41,823 INFO [ShuffleRunner {Map_1}] impl.ShuffleManager: Map_1: Scheduler thread completed 2018-01-05T11:28:41,824 DEBUG [TezTR-518813_1_2_1_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:28:41,830 DEBUG [TezTR-518813_1_2_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:41,830 DEBUG [TezTR-518813_1_2_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:41,831 DEBUG [TezTR-518813_1_2_1_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:28:41,831 DEBUG [TezTR-518813_1_2_1_0_0] log.PerfLogger: 2018-01-05T11:28:41,831 INFO [TezTR-518813_1_2_1_0_0] exec.GroupByOperator: Initializing operator GBY[4] 2018-01-05T11:28:41,831 DEBUG [TezTR-518813_1_2_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:41,831 DEBUG [TezTR-518813_1_2_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:41,831 INFO [TezTR-518813_1_2_1_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:41,831 DEBUG [TezTR-518813_1_2_1_0_0] exec.GroupByOperator: Initialization Done 4 GBY 2018-01-05T11:28:41,831 DEBUG [TezTR-518813_1_2_1_0_0] exec.GroupByOperator: Operator 4 GBY initialized 2018-01-05T11:28:41,831 DEBUG [TezTR-518813_1_2_1_0_0] exec.GroupByOperator: Initializing children of 4 GBY 2018-01-05T11:28:41,831 DEBUG [TezTR-518813_1_2_1_0_0] exec.FileSinkOperator: Initializing child 6 FS 2018-01-05T11:28:41,831 INFO [TezTR-518813_1_2_1_0_0] exec.FileSinkOperator: Initializing operator FS[6] 2018-01-05T11:28:41,831 DEBUG [TezTR-518813_1_2_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:41,832 INFO [TezTR-518813_1_2_1_0_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@315f0292]:[_col0, _col1]:[struct, struct]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@344b7f85 2018-01-05T11:28:41,833 DEBUG [TezTR-518813_1_2_1_0_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:28:41,834 DEBUG [TezTR-518813_1_2_1_0_0] exec.FileSinkOperator: Initialization Done 6 FS 2018-01-05T11:28:41,834 DEBUG [TezTR-518813_1_2_1_0_0] exec.FileSinkOperator: Operator 6 FS initialized 2018-01-05T11:28:41,834 DEBUG [TezTR-518813_1_2_1_0_0] exec.FileSinkOperator: Initialization Done 6 FS done is reset. 2018-01-05T11:28:41,834 DEBUG [TezTR-518813_1_2_1_0_0] exec.GroupByOperator: Initialization Done 4 GBY done is reset. 2018-01-05T11:28:41,834 INFO [TezTR-518813_1_2_1_0_0] tez.ReduceRecordProcessor: Id =4 Id =6 <\Children> Id = 4 null<\Parent> <\FS> <\Children> <\Parent> <\GBY> 2018-01-05T11:28:41,834 DEBUG [TezTR-518813_1_2_1_0_0] log.PerfLogger: 2018-01-05T11:28:41,834 INFO [TezTR-518813_1_2_1_0_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 2 2018-01-05T11:28:41,836 DEBUG [TezTR-518813_1_2_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:28:41,836 DEBUG [TezTR-518813_1_2_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:28:41,836 DEBUG [TezTR-518813_1_2_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:28:41,836 DEBUG [TezTR-518813_1_2_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:28:41,836 DEBUG [TezTR-518813_1_2_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:28:41,836 DEBUG [TezTR-518813_1_2_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:28:41,839 INFO [TezTR-518813_1_2_1_0_0] readers.UnorderedKVReader: Num Records read: 1 2018-01-05T11:28:41,839 DEBUG [TezTR-518813_1_2_1_0_0] log.PerfLogger: 2018-01-05T11:28:41,839 DEBUG [TezTR-518813_1_2_1_0_0] mr.ObjectCache: djaiswal_20180105112841_c0c75619-7aaf-4c02-a457-8aa38d666b4f_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:28:41,839 DEBUG [TezTR-518813_1_2_1_0_0] exec.GroupByOperator: close called for operator GBY[4] 2018-01-05T11:28:41,839 INFO [TezTR-518813_1_2_1_0_0] exec.GroupByOperator: Closing operator GBY[4] 2018-01-05T11:28:41,840 INFO [TezTR-518813_1_2_1_0_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-41_029_8026269352618129281-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-41_029_8026269352618129281-1/_tmp.-ext-10002/000000_0 2018-01-05T11:28:41,840 INFO [TezTR-518813_1_2_1_0_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-41_029_8026269352618129281-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-41_029_8026269352618129281-1/_task_tmp.-ext-10002/_tmp.000000_0 2018-01-05T11:28:41,840 INFO [TezTR-518813_1_2_1_0_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-41_029_8026269352618129281-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-41_029_8026269352618129281-1/_tmp.-ext-10002/000000_0 2018-01-05T11:28:41,886 INFO [TezTR-518813_1_2_1_0_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:28:41,886 DEBUG [TezTR-518813_1_2_1_0_0] exec.GroupByOperator: Closing child = FS[6] 2018-01-05T11:28:41,886 DEBUG [TezTR-518813_1_2_1_0_0] exec.FileSinkOperator: close called for operator FS[6] 2018-01-05T11:28:41,886 DEBUG [TezTR-518813_1_2_1_0_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:41,886 INFO [TezTR-518813_1_2_1_0_0] exec.FileSinkOperator: Closing operator FS[6] 2018-01-05T11:28:41,886 INFO [TezTR-518813_1_2_1_0_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:28:41,889 INFO [TezTR-518813_1_2_1_0_0] exec.FileSinkOperator: RECORDS_OUT_0:1, 2018-01-05T11:28:41,889 DEBUG [TezTR-518813_1_2_1_0_0] exec.FileSinkOperator: 6 Close done 2018-01-05T11:28:41,889 DEBUG [TezTR-518813_1_2_1_0_0] exec.GroupByOperator: 4 Close done 2018-01-05T11:28:41,889 INFO [TezTR-518813_1_2_1_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_2_01_000000_0 2018-01-05T11:28:41,889 INFO [TezTR-518813_1_2_1_0_0] impl.ShuffleInputEventHandlerImpl: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:28:41,889 INFO [TezTR-518813_1_2_1_0_0] impl.ShuffleManager: Shutting down pending fetchers on sourceMap_1: 0 2018-01-05T11:28:41,889 INFO [TezTR-518813_1_2_1_0_0] output.MROutput: out_Reducer 2 closed 2018-01-05T11:28:41,890 DEBUG [TezTR-518813_1_2_1_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:41,890 INFO [TezTR-518813_1_2_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:41,890 INFO [TezTR-518813_1_2_1_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_2_01_000000_0, askedToStop=false 2018-01-05T11:28:41,890 INFO [TezTR-518813_1_2_1_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_2_01_000000_0, stopRequested=false 2018-01-05T11:28:41,890 INFO [TezTR-518813_1_2_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_2_01_000000_0: Counters: 14 [[org.apache.tez.common.counters.TaskCounter NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=2388, SHUFFLE_BYTES_DECOMPRESSED=2384, SHUFFLE_BYTES_TO_MEM=2388, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=112, FIRST_EVENT_RECEIVED=11, LAST_EVENT_RECEIVED=11][HIVE CREATED_FILES=1, RECORDS_OUT_0=1]] 2018-01-05T11:28:41,890 DEBUG [TezTR-518813_1_2_1_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_2_1_0_0 thread id: 353 scheme: file bytesRead: 0 bytesWritten: 3285 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:41,891 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_2_01_000000_0, isTaskDone=true 2018-01-05T11:28:41,892 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000003, requestId=3, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_2_01_000000_0, eventCount=2 } 2018-01-05T11:28:41,893 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_2][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_2_01_000000_0, creationTime=1515180521665, allocationTime=1515180521667, startTime=1515180521672, finishTime=1515180521893, timeTaken=221, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:41,893 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_2_01_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:41,894 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_2_01_000000_0 2018-01-05T11:28:41,894 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_2_01_000000_0; false 2018-01-05T11:28:41,894 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_2_01_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:41,894 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_2][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_2_01_000000, startTime=1515180521672, finishTime=1515180521894, timeTaken=222, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_2_01_000000_0, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=3285, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=2388, SHUFFLE_BYTES_DECOMPRESSED=2384, SHUFFLE_BYTES_TO_MEM=2388, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=112, FIRST_EVENT_RECEIVED=11, LAST_EVENT_RECEIVED=11, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1 2018-01-05T11:28:41,894 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=3, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:28:41,894 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:41,894 INFO [Task-Executor-2] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_2_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:41,895 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:41,895 INFO [Task-Executor-2] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_2_01_000000_0 2018-01-05T11:28:41,895 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_2_01 [Reducer 2], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:41,895 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_2_01 [Reducer 2] 2018-01-05T11:28:41,895 INFO [Task-Executor-2] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000003=219 2018-01-05T11:28:41,895 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_2_01_000000_0: true 2018-01-05T11:28:41,895 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_2_01_000000_0 2018-01-05T11:28:41,896 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000003, QueryId=djaiswal_20180105112841_c0c75619-7aaf-4c02-a457-8aa38d666b4f-2, Vertex=Reducer 2, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:28:41,896 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_2_01_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:41,896 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_2_01_000000_0 2018-01-05T11:28:41,896 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_2][Event:VERTEX_FINISHED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_2_01, initRequestedTime=1515180521394, initedTime=1515180521411, startRequestedTime=1515180521419, startedTime=1515180521419, finishTime=1515180521895, timeTaken=476, status=SUCCEEDED, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=3285, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=2388, SHUFFLE_BYTES_DECOMPRESSED=2384, SHUFFLE_BYTES_TO_MEM=2388, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=112, FIRST_EVENT_RECEIVED=11, LAST_EVENT_RECEIVED=11, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1, vertexStats=firstTaskStartTime=1515180521672, firstTasksToStart=[ task_1515180518813_0001_2_01_000000 ], lastTaskFinishTime=1515180521893, lastTasksToFinish=[ task_1515180518813_0001_2_01_000000 ], minTaskDuration=221, maxTaskDuration=221, avgTaskDuration=221.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_2_01_000000 ], longestDurationTasks=[ task_1515180518813_0001_2_01_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:41,896 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_2_01 [Reducer 2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:41,897 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_2][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000003, stoppedTime=1515180521897, exitStatus=0 2018-01-05T11:28:41,897 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000003 exited with diagnostics set to Container completed. 2018-01-05T11:28:41,897 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_2_01 [Reducer 2] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2 2018-01-05T11:28:41,897 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null 2018-01-05T11:28:41,897 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_2_00 [Map 1] 2018-01-05T11:28:41,897 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_2_01 [Reducer 2] 2018-01-05T11:28:41,898 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_2, queueSize=0 2018-01-05T11:28:41,898 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_2][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_2, startTime=1515180521371, finishTime=1515180521897, timeTaken=526, status=SUCCEEDED, diagnostics=, counters=Counters: 35, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, AM_CPU_MILLISECONDS=1318, AM_GC_TIME_MILLIS=0, File System Counters, FILE_BYTES_READ=5872, FILE_BYTES_WRITTEN=5787, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=501, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=2374, OUTPUT_BYTES_WITH_OVERHEAD=2384, OUTPUT_BYTES_PHYSICAL=2412, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=2388, SHUFFLE_BYTES_DECOMPRESSED=2384, SHUFFLE_BYTES_TO_MEM=2388, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=112, FIRST_EVENT_RECEIVED=11, LAST_EVENT_RECEIVED=11, HIVE, CREATED_FILES=1, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500, RECORDS_OUT_0=1, RECORDS_OUT_INTERMEDIATE_Map_1=1 2018-01-05T11:28:41,899 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_2 finished with state: SUCCEEDED 2018-01-05T11:28:41,899 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_2 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:28:41,899 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000003 2018-01-05T11:28:41,900 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_2, dagState=SUCCEEDED 2018-01-05T11:28:41,900 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_2_post 2018-01-05T11:28:41,901 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:41,901 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:28:41,901 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:28:41,901 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 2 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=2, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=2,NumRequestsWithoutLocation=2, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=2} 2018-01-05T11:28:41,901 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:28:41,901 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=ANALYZE TABLE src COMPUTE STATIS...key,value (Stage-0), with id=dag_1515180518813_0001_2 2018-01-05T11:28:41,901 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 2, to 10.22.8.182:49818 2018-01-05T11:28:41,901 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=2, Remaining=0 2018-01-05T11:28:41,901 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=ANALYZE TABLE src COMPUTE STATIS...key,value (Stage-0), with id=dag_1515180518813_0001_2 2018-01-05T11:28:41,902 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=2} 2018-01-05T11:28:41,902 INFO [IPC Server handler 0 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=2}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:28:41,902 INFO [IPC Server handler 0 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/2 after 300 seconds 2018-01-05T11:28:41,902 ERROR [IPC Server handler 0 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112841_c0c75619-7aaf-4c02-a457-8aa38d666b4f, dag_1515180518813_0001_2 2018-01-05T11:28:41,903 INFO [IPC Server handler 0 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112841_c0c75619-7aaf-4c02-a457-8aa38d666b4f 2018-01-05T11:28:41,903 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=2} = 0 2018-01-05T11:28:41,903 DEBUG [IPC Server handler 0 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=2} 2018-01-05T11:28:41,903 DEBUG [IPC Server handler 0 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:28:41,903 DEBUG [IPC Server handler 0 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=2} 2018-01-05T11:28:41,904 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,904 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,904 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,904 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 Reducer 2: 1/1 2018-01-05T11:28:41,904 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,919 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,919 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: TaskId for 000000_0 = 000000 2018-01-05T11:28:41,919 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,919 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,919 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:41,927 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-2:STATS] in serial mode 2018-01-05T11:28:41,927 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@116efe65, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:41,927 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:41,927 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:41,927 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:41,927 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:41,927 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:41,927 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:41,927 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:41,927 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:41,927 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:41,927 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-41_029_8026269352618129281-1/-mr-10001 2018-01-05T11:28:41,928 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-41_029_8026269352618129281-1/-mr-10001 2018-01-05T11:28:41,929 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.src/ numRows 2018-01-05T11:28:41,929 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.src/ numRows 500 2018-01-05T11:28:41,929 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.src/ rawDataSize 2018-01-05T11:28:41,929 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.src/ rawDataSize 5312 2018-01-05T11:28:41,930 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:41,930 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:41,931 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:41,931 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:41,931 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:41,931 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:41,931 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:41,931 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:41,933 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:41,933 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:41,934 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:41,934 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:41,934 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:41,934 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:41,934 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:41,934 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=src newtbl=src 2018-01-05T11:28:41,934 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=src newtbl=src 2018-01-05T11:28:41,942 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:41,952 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:41,953 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:41,953 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.src stats: [numFiles=1, numRows=500, totalSize=5812, rawDataSize=5312] 2018-01-05T11:28:41,953 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: About to delete stats tmp dir :file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-41_029_8026269352618129281-1/-mr-10001 2018-01-05T11:28:41,970 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Time taken to get FileStatuses: 14 2018-01-05T11:28:41,970 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:28:41,970 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 15 2018-01-05T11:28:41,970 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FetchOperator: Creating fetchTask with deserializer typeinfo: struct<_col0:struct,_col1:struct> 2018-01-05T11:28:41,970 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FetchOperator: deserializer properties: table properties: {columns=_col0,_col1, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct:struct, escape.delim=\} partition properties: {columns=_col0,_col1, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct:struct, escape.delim=\} 2018-01-05T11:28:41,990 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:41,990 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: write_column_statistics: db=default table=src 2018-01-05T11:28:41,990 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=write_column_statistics: db=default table=src 2018-01-05T11:28:41,999 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000003, QueryId=djaiswal_20180105112841_c0c75619-7aaf-4c02-a457-8aa38d666b4f-2, DagId=2, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-2, Succeeded=true, StartTime=1515180521676, EndTime=1515180521999 2018-01-05T11:28:41,999 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=src tableName=default colName=key 2018-01-05T11:28:42,003 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,007 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=src tableName=default colName=value 2018-01-05T11:28:42,011 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,011 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,011 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,012 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,012 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: ANALYZE TABLE src COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:42,012 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:28:42,012 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@src 2018-01-05T11:28:42,012 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@src 2018-01-05T11:28:42,012 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-41_029_8026269352618129281-1/-mr-10000 2018-01-05T11:28:42,012 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,012 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,012 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:42,012 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=19, setPartitionColumnStatistics_(SetPartitionsStatsRequest, )=21} 2018-01-05T11:28:42,012 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112841_c0c75619-7aaf-4c02-a457-8aa38d666b4f); Time taken: 0.846 seconds 2018-01-05T11:28:42,012 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:42,012 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,012 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:42,012 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,012 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query ANALYZE TABLE src COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:42,012 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-41_020_1421415792491703556-1 2018-01-05T11:28:42,013 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.993 seconds 2018-01-05T11:28:42,013 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:42,013 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:42,013 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:42,013 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:42,013 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,013 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:42,013 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,013 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,013 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,013 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: -- -- Table src1 -- DROP TABLE IF EXISTS src1 2018-01-05T11:28:42,013 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112842_d65a02b3-1029-41cd-92e1-9cdd75732c74): -- -- Table src1 -- DROP TABLE IF EXISTS src1 2018-01-05T11:28:42,013 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,013 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: -- -- Table src1 -- DROP TABLE IF EXISTS src1 2018-01-05T11:28:42,013 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:42,014 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,014 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,014 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,014 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,014 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:42,014 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,014 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src1 2018-01-05T11:28:42,014 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src1 2018-01-05T11:28:42,015 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,015 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,015 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:42,015 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,015 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:42,015 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,015 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:42,015 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:28:42,015 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112842_d65a02b3-1029-41cd-92e1-9cdd75732c74); Time taken: 0.002 seconds 2018-01-05T11:28:42,015 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:42,015 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,015 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112842_d65a02b3-1029-41cd-92e1-9cdd75732c74): -- -- Table src1 -- DROP TABLE IF EXISTS src1 2018-01-05T11:28:42,016 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,016 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: -- -- Table src1 -- DROP TABLE IF EXISTS src1 2018-01-05T11:28:42,016 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:42,016 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,016 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,016 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,016 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,016 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:42,017 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,017 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src1 2018-01-05T11:28:42,017 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src1 2018-01-05T11:28:42,017 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,017 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,017 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table src1 not found: default.src1 table not found 2018-01-05T11:28:42,017 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,017 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src1 2018-01-05T11:28:42,017 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src1 2018-01-05T11:28:42,018 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,018 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,018 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,018 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,018 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: -- -- Table src1 -- DROP TABLE IF EXISTS src1 2018-01-05T11:28:42,018 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:42,018 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,018 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,018 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:42,018 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=1} 2018-01-05T11:28:42,018 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112842_d65a02b3-1029-41cd-92e1-9cdd75732c74); Time taken: 0.003 seconds 2018-01-05T11:28:42,018 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:42,018 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,018 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:42,018 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,018 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query -- -- Table src1 -- DROP TABLE IF EXISTS src1 2018-01-05T11:28:42,018 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.005 seconds 2018-01-05T11:28:42,018 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:42,018 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:42,018 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:42,018 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:42,019 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,019 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:42,019 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,019 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,019 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,019 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: CREATE TABLE src1 (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS TEXTFILE 2018-01-05T11:28:42,019 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112842_aaf8c5d2-d1d6-40eb-8ff1-26374e7ba68d): CREATE TABLE src1 (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS TEXTFILE 2018-01-05T11:28:42,019 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,019 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: CREATE TABLE src1 (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS TEXTFILE 2018-01-05T11:28:42,019 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:42,019 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,019 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,019 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,019 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,019 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:42,019 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:28:42,019 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Creating table default.src1 position=13 2018-01-05T11:28:42,020 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,020 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_database: default 2018-01-05T11:28:42,020 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_database: default 2018-01-05T11:28:42,020 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:42,021 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,021 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,021 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:42,021 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: validation start 2018-01-05T11:28:42,021 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:28:42,021 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Not a partition. 2018-01-05T11:28:42,021 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,021 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:42,021 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,021 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:42,021 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getDatabase_(String, )=1, flushCache_()=0} 2018-01-05T11:28:42,021 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112842_aaf8c5d2-d1d6-40eb-8ff1-26374e7ba68d); Time taken: 0.002 seconds 2018-01-05T11:28:42,021 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:42,021 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,021 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112842_aaf8c5d2-d1d6-40eb-8ff1-26374e7ba68d): CREATE TABLE src1 (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS TEXTFILE 2018-01-05T11:28:42,022 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,022 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: CREATE TABLE src1 (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS TEXTFILE 2018-01-05T11:28:42,022 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: CREATETABLE 2018-01-05T11:28:42,022 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: database:default 2018-01-05T11:28:42,022 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@src1 2018-01-05T11:28:42,022 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,022 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,022 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,022 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,022 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:42,022 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Found class for org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe 2018-01-05T11:28:42,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: creating table default.src1 on file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1 2018-01-05T11:28:42,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,023 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: create_table: Table(tableName:src1, dbName:default, owner:hive_test_user, createTime:1515180522, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:string, comment:default), FieldSchema(name:value, type:string, comment:default)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, numFiles=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:42,023 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=create_table: Table(tableName:src1, dbName:default, owner:hive_test_user, createTime:1515180522, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:string, comment:default), FieldSchema(name:value, type:string, comment:default)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, numFiles=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:42,024 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hive-site.xml 2018-01-05T11:28:42,025 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hivemetastore-site.xml 2018-01-05T11:28:42,025 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Unable to find config file metastore-site.xml 2018-01-05T11:28:42,025 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file null 2018-01-05T11:28:42,036 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:42,036 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: Location: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1 specified for non-external table:src1 2018-01-05T11:28:42,036 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1 2018-01-05T11:28:42,061 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,064 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@src1) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@src1) Type=TABLE WriteType=DDL_NO_LOCK is present 2018-01-05T11:28:42,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,065 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: CREATE TABLE src1 (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS TEXTFILE 2018-01-05T11:28:42,065 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: CREATETABLE 2018-01-05T11:28:42,065 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: database:default 2018-01-05T11:28:42,065 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@src1 2018-01-05T11:28:42,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,065 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:42,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {createTable_(Table, )=42, isCompatibleWith_(Configuration, )=0} 2018-01-05T11:28:42,065 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112842_aaf8c5d2-d1d6-40eb-8ff1-26374e7ba68d); Time taken: 0.044 seconds 2018-01-05T11:28:42,065 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:42,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,065 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:42,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query CREATE TABLE src1 (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS TEXTFILE 2018-01-05T11:28:42,065 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.046 seconds 2018-01-05T11:28:42,065 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:42,065 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:42,065 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:42,065 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:42,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:42,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/kv3.txt" INTO TABLE src1 2018-01-05T11:28:42,066 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112842_33132bf1-98bb-4cd5-8899-b0736dc889ea): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv3.txt" INTO TABLE src1 2018-01-05T11:28:42,066 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,066 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv3.txt" INTO TABLE src1 2018-01-05T11:28:42,066 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:42,066 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,066 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,066 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,066 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,066 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:42,066 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.LoadSemanticAnalyzer: file@null@/Users/djaiswal/hive/hive/data/files/kv3.txt 2018-01-05T11:28:42,066 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,066 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src1 2018-01-05T11:28:42,066 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src1 2018-01-05T11:28:42,072 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,073 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,075 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:42,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,075 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:42,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,075 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:42,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=7, flushCache_()=0} 2018-01-05T11:28:42,075 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112842_33132bf1-98bb-4cd5-8899-b0736dc889ea); Time taken: 0.01 seconds 2018-01-05T11:28:42,075 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:42,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,075 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112842_33132bf1-98bb-4cd5-8899-b0736dc889ea): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv3.txt" INTO TABLE src1 2018-01-05T11:28:42,076 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,076 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv3.txt" INTO TABLE src1 2018-01-05T11:28:42,076 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: LOAD 2018-01-05T11:28:42,076 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/kv3.txt 2018-01-05T11:28:42,076 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@src1 2018-01-05T11:28:42,076 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,076 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,076 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,076 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,077 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MOVE] in serial mode 2018-01-05T11:28:42,077 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@49c37918, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:42,077 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:42,077 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,077 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:42,077 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:42,077 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:42,077 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:42,077 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,077 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:42,077 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Loading data to table default.src1 from file:/Users/djaiswal/hive/hive/data/files/kv3.txt 2018-01-05T11:28:42,078 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,078 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:42,078 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:42,078 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:42,078 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:42,078 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,078 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,079 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,080 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,081 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,081 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:42,081 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:42,081 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,081 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:42,081 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,081 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src1 2018-01-05T11:28:42,081 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src1 2018-01-05T11:28:42,086 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,087 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,089 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,089 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src1 2018-01-05T11:28:42,089 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src1 2018-01-05T11:28:42,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] FileOperations: moving file:/Users/djaiswal/hive/hive/data/files/kv3.txt to file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1 (replace = KEEP_EXISTING) 2018-01-05T11:28:42,096 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:42,096 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:42,125 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Moved src: file:/Users/djaiswal/hive/hive/data/files/kv3.txt, to dest: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1/kv3.txt 2018-01-05T11:28:42,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,126 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=src1 newtbl=src1 2018-01-05T11:28:42,126 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=src1 newtbl=src1 2018-01-05T11:28:42,132 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,139 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,140 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,140 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@src1) Type=TABLE WriteType=INSERT because WriteEntity(default@src1) Type=TABLE WriteType=INSERT is present 2018-01-05T11:28:42,140 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-1:STATS] in serial mode 2018-01-05T11:28:42,140 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@673a5aa7, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:42,140 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:42,140 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,140 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:42,140 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:42,140 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:42,141 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:42,141 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,141 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:42,141 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:42,141 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,141 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:42,142 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:42,142 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:42,142 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:42,142 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,142 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,142 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,143 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,143 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,143 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:42,143 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:42,144 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,144 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:42,144 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,144 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src1 2018-01-05T11:28:42,144 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src1 2018-01-05T11:28:42,149 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,150 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,150 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,150 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=src1 newtbl=src1 2018-01-05T11:28:42,150 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=src1 newtbl=src1 2018-01-05T11:28:42,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,165 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,165 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,165 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.src1 stats: [numFiles=1, numRows=0, totalSize=216, rawDataSize=0] 2018-01-05T11:28:42,165 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,165 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,165 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv3.txt" INTO TABLE src1 2018-01-05T11:28:42,165 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: LOAD 2018-01-05T11:28:42,166 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/kv3.txt 2018-01-05T11:28:42,166 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@src1 2018-01-05T11:28:42,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,166 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:42,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=15, getTable_(String, String, )=6} 2018-01-05T11:28:42,166 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112842_33132bf1-98bb-4cd5-8899-b0736dc889ea); Time taken: 0.091 seconds 2018-01-05T11:28:42,166 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:42,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,166 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:42,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv3.txt" INTO TABLE src1 2018-01-05T11:28:42,166 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.101 seconds 2018-01-05T11:28:42,166 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:42,166 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:42,166 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:42,166 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:42,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:42,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: ANALYZE TABLE src1 COMPUTE STATISTICS 2018-01-05T11:28:42,166 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112842_16059af6-eef3-4eff-a291-f1d4220f6f92): ANALYZE TABLE src1 COMPUTE STATISTICS 2018-01-05T11:28:42,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: ANALYZE TABLE src1 COMPUTE STATISTICS 2018-01-05T11:28:42,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:42,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,167 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:42,167 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Invoking analyze on original query 2018-01-05T11:28:42,167 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Starting Semantic Analysis 2018-01-05T11:28:42,167 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed phase 1 of Semantic Analysis 2018-01-05T11:28:42,167 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for source tables 2018-01-05T11:28:42,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,167 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src1 2018-01-05T11:28:42,167 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src1 2018-01-05T11:28:42,174 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,175 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,175 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,175 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src1 2018-01-05T11:28:42,175 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src1 2018-01-05T11:28:42,180 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,180 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,180 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for subqueries 2018-01-05T11:28:42,180 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for destination tables 2018-01-05T11:28:42,180 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed getting MetaData in Semantic Analysis 2018-01-05T11:28:42,192 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_167_8868162020991472848-1/-mr-10000 2018-01-05T11:28:42,192 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Table Plan for src1 TS[0] 2018-01-05T11:28:42,192 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Body Plan for Query Block null 2018-01-05T11:28:42,192 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Plan for Query Block null 2018-01-05T11:28:42,192 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Before logical optimization TS[0] 2018-01-05T11:28:42,192 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,192 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,192 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,192 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] lineage.Generator: Time taken for lineage transform=0 2018-01-05T11:28:42,192 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,192 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,192 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,192 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,192 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,192 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,192 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,192 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,192 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{src1}key,value: string|{src1}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src1}block__offset__inside__file,INPUT__FILE__NAME: string|{src1}input__file__name,ROW__ID: struct|{src1}row__id,RAW__DATA__SIZE: bigint|{src1}raw__data__size) 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.PredicatePushDown: After PPD: TS[0] 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{src1}key,value: string|{src1}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src1}block__offset__inside__file,INPUT__FILE__NAME: string|{src1}input__file__name,ROW__ID: struct|{src1}row__id,RAW__DATA__SIZE: bigint|{src1}raw__data__size) 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{src1}key,value: string|{src1}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src1}block__offset__inside__file,INPUT__FILE__NAME: string|{src1}input__file__name,ROW__ID: struct|{src1}row__id,RAW__DATA__SIZE: bigint|{src1}raw__data__size) 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: After logical optimization TS[0] 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,194 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,194 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,194 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,194 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=src1 2018-01-05T11:28:42,194 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=src1 2018-01-05T11:28:42,199 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting stats (Num rows: 1 Data size: 2160 Basic stats: COMPLETE Column stats: COMPLETE) on TS[0] 2018-01-05T11:28:42,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (src1): numRows: 1 dataSize: 2160 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:28:42,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on TS[0] 2018-01-05T11:28:42,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:42,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:28:42,205 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Cycle free: true 2018-01-05T11:28:42,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{src1}key,value: string|{src1}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src1}block__offset__inside__file,INPUT__FILE__NAME: string|{src1}input__file__name,ROW__ID: struct|{src1}row__id,RAW__DATA__SIZE: bigint|{src1}raw__data__size) 2018-01-05T11:28:42,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:28:42,206 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1 of table src1 2018-01-05T11:28:42,206 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1 2018-01-05T11:28:42,206 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:28:42,206 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,206 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,206 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:42,206 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:42,206 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:42,206 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:42,206 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:42,206 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:42,206 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:28:42,206 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping vectorization 2018-01-05T11:28:42,206 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:28:42,206 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: llap mode: only 2018-01-05T11:28:42,206 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:28:42,206 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:42,206 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@24f77077 can be done in LLAP 2018-01-05T11:28:42,206 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,206 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed plan generation 2018-01-05T11:28:42,206 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:42,206 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validation start 2018-01-05T11:28:42,206 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Not a partition. 2018-01-05T11:28:42,206 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,207 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:src1.key, type:string, comment:null), FieldSchema(name:src1.value, type:string, comment:null)], properties:null) 2018-01-05T11:28:42,207 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,207 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:42,207 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=13, flushCache_()=0, getTableColumnStatistics_(String, String, List, )=11} 2018-01-05T11:28:42,207 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112842_16059af6-eef3-4eff-a291-f1d4220f6f92); Time taken: 0.041 seconds 2018-01-05T11:28:42,207 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:42,207 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,207 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112842_16059af6-eef3-4eff-a291-f1d4220f6f92): ANALYZE TABLE src1 COMPUTE STATISTICS 2018-01-05T11:28:42,208 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,208 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: ANALYZE TABLE src1 COMPUTE STATISTICS 2018-01-05T11:28:42,208 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:28:42,208 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@src1 2018-01-05T11:28:42,208 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@src1 2018-01-05T11:28:42,208 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,208 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,208 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,208 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Query ID = djaiswal_20180105112842_16059af6-eef3-4eff-a291-f1d4220f6f92 2018-01-05T11:28:42,208 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Total jobs = 1 2018-01-05T11:28:42,208 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,208 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:28:42,208 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MAPRED] in serial mode 2018-01-05T11:28:42,219 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_167_8868162020991472848-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:28:42,239 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:28:42,239 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:28:42,240 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:28:42,240 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112842_16059af6-eef3-4eff-a291-f1d4220f6f92 2018-01-05T11:28:42,240 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Session is already open 2018-01-05T11:28:42,240 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:28:42,240 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,240 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Dag name: ANALYZE TABLE src1 COMPUTE STATISTICS (Stage-0) 2018-01-05T11:28:42,240 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: DagInfo: {"context":"Hive","description":"\n\nANALYZE TABLE src1 COMPUTE STATISTICS"} 2018-01-05T11:28:42,240 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112842_16059af6-eef3-4eff-a291-f1d4220f6f92 with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:28:42,240 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,250 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:28:42,251 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,251 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:28:42,251 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,252 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 1.73KB 2018-01-05T11:28:42,258 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_167_8868162020991472848-1/-mr-10000 2018-01-05T11:28:42,269 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_167_8868162020991472848-1/-mr-10000 2018-01-05T11:28:42,273 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1 2018-01-05T11:28:42,273 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,273 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,273 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,273 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=ANALYZE TABLE src1 COMPUTE STATISTICS (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112842_16059af6-eef3-4eff-a291-f1d4220f6f92 } 2018-01-05T11:28:42,273 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:42,277 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: ANALYZE TABLE src1 COMPUTE STATISTICS (Stage-0) 2018-01-05T11:28:42,277 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:28:42,277 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_3, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_3.dot 2018-01-05T11:28:42,278 INFO [IPC Server handler 1 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_3 2018-01-05T11:28:42,278 WARN [IPC Server handler 1 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:42,278 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Running DAG: ANALYZE TABLE src1 COMPUTE STATISTICS (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112842_16059af6-eef3-4eff-a291-f1d4220f6f92 } 2018-01-05T11:28:42,301 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_3][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_3, submitTime=1515180522277, queueName=null 2018-01-05T11:28:42,302 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_3_00 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_3_00 [Map 1], Default Resources= 2018-01-05T11:28:42,302 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_3_00 [Map 1] 2018-01-05T11:28:42,302 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_3_00 [Map 1] 2018-01-05T11:28:42,302 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:28:42,302 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_3_00 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:42,302 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:28:42,302 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_3][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_3, initTime=1515180522302 2018-01-05T11:28:42,302 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: dag_1515180518813_0001_3 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:28:42,303 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:28:42,303 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_3][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_3, startTime=1515180522303 2018-01-05T11:28:42,303 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_3 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:28:42,303 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {src1={InputName=src1}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:28:42,303 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: src1, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:28:42,303 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_3_00 [Map 1] 2018-01-05T11:28:42,303 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_3, dagName=ANALYZE TABLE src1 COMPUTE STATISTICS (Stage-0) 2018-01-05T11:28:42,303 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,303 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,303 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,309 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_3_00 [Map 1] 2018-01-05T11:28:42,309 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_3_00 [Map 1] 2018-01-05T11:28:42,309 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:42,309 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_3_00 [Map 1] 2018-01-05T11:28:42,312 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:28:42,312 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@736cdca1 2018-01-05T11:28:42,313 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_167_8868162020991472848-1/djaiswal/_tez_scratch_dir/99e0de7c-a314-42c9-8205-aa1581396905/map.xml 2018-01-05T11:28:42,313 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_167_8868162020991472848-1/djaiswal/_tez_scratch_dir/99e0de7c-a314-42c9-8205-aa1581396905/map.xml 2018-01-05T11:28:42,313 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_167_8868162020991472848-1/djaiswal/_tez_scratch_dir/99e0de7c-a314-42c9-8205-aa1581396905/map.xml 2018-01-05T11:28:42,313 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:42,313 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:42,313 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:42,314 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:42,314 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 1.73KB 2018-01-05T11:28:42,314 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_3_00 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:28:42,315 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: src1 on vertex vertex_1515180518813_0001_3_00 [Map 1] 2018-01-05T11:28:42,315 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:28:42,315 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:42,315 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:28:42,315 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:42,315 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_167_8868162020991472848-1/djaiswal/_tez_scratch_dir/99e0de7c-a314-42c9-8205-aa1581396905/map.xml 2018-01-05T11:28:42,315 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:42,315 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias src1 2018-01-05T11:28:42,315 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1 2018-01-05T11:28:42,315 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1 2018-01-05T11:28:42,316 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 2018-01-05T11:28:42,316 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = 2018-01-05T11:28:42,316 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1 2018-01-05T11:28:42,317 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:42,317 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:42,317 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:42,317 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:42,332 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Time taken to get FileStatuses: 15 2018-01-05T11:28:42,332 INFO [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:28:42,333 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 15 2018-01-05T11:28:42,333 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 1 2018-01-05T11:28:42,333 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:42,333 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 1. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:28:42,333 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_167_8868162020991472848-1/djaiswal/_tez_scratch_dir/99e0de7c-a314-42c9-8205-aa1581396905/map.xml 2018-01-05T11:28:42,333 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_167_8868162020991472848-1/djaiswal/_tez_scratch_dir/99e0de7c-a314-42c9-8205-aa1581396905/map.xml 2018-01-05T11:28:42,333 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_167_8868162020991472848-1/djaiswal/_tez_scratch_dir/99e0de7c-a314-42c9-8205-aa1581396905/map.xml 2018-01-05T11:28:42,333 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:42,333 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:42,333 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:42,334 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:42,334 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 1.73KB 2018-01-05T11:28:42,334 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1/kv3.txt to src new group? true 2018-01-05T11:28:42,334 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:28:42,334 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:28:42,334 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:28:42,334 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 4 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 216 Original splits: 1 2018-01-05T11:28:42,334 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Using original number of splits: 1 desired splits: 1 2018-01-05T11:28:42,334 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 1 grouped split count is 1, for bucket: 1 2018-01-05T11:28:42,334 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:28:42,335 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: src1 on vertex vertex_1515180518813_0001_3_00 [Map 1] 2018-01-05T11:28:42,335 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_3_00 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:42,336 INFO [App Shared Pool - #0] impl.VertexImpl: Got updated RootInputsSpecs: {src1=forAllWorkUnits=true, update=[1]} 2018-01-05T11:28:42,336 INFO [App Shared Pool - #0] impl.VertexImpl: Vertex vertex_1515180518813_0001_3_00 [Map 1] parallelism set to 1 2018-01-05T11:28:42,336 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_3_00 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:42,336 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_3_00 [Map 1], numAdditionalOutputs=1 2018-01-05T11:28:42,336 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_3][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_3_00, initRequestedTime=1515180522303, initedTime=1515180522336, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:42,336 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_3_00 [Map 1] with distanceFromRoot: 0 2018-01-05T11:28:42,336 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_3_00 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:28:42,336 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_3][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_3_00, startRequestedTime=1515180522315, startedTime=1515180522336 2018-01-05T11:28:42,336 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_3][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_3_00, reconfigureDoneTime=1515180522336, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:28:42,336 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_3_00 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:42,337 INFO [App Shared Pool - #0] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:28:42,337 INFO [App Shared Pool - #0] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:28:42,337 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_3_00 [Map 1] 2018-01-05T11:28:42,337 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_3][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_3_00_000000, scheduledTime=1515180522337, launchTime=1515180522337 2018-01-05T11:28:42,337 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_3_00 [Map 1] 2018-01-05T11:28:42,338 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_3_00_000000_0, priority=2, capability=, hosts=null 2018-01-05T11:28:42,338 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=2,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:42,338 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=2,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:42,338 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:42,338 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_3_00_000000_0 on hosts=null 2018-01-05T11:28:42,338 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_3_00_000000_0 on hosts=null 2018-01-05T11:28:42,338 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_3_00_000000_0 2018-01-05T11:28:42,338 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:42,338 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #4, task=TaskInfo{task=attempt_1515180518813_0001_3_00_000000_0, priority=2, startTime=0, containerId=null, uniqueId=3, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000004 2018-01-05T11:28:42,338 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_3_00_000000_0; false 2018-01-05T11:28:42,338 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_3_00_000000_0, priority=2, startTime=311542847, containerId=container_111101111_0001_00_000004assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=3, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:42,339 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000004 for node: 10.22.8.182:49818 2018-01-05T11:28:42,341 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 3, name=ANALYZE TABLE src1 COMPUTE STATISTICS (Stage-0), queryId=djaiswal_20180105112842_16059af6-eef3-4eff-a291-f1d4220f6f92 2018-01-05T11:28:42,342 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000004, attempt_1515180518813_0001_3_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:42,342 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000004 for node: 10.22.8.182:49818 2018-01-05T11:28:42,342 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000004 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:42,342 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000004, launchTime=1515180522342 2018-01-05T11:28:42,343 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_3_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000004] on NM: [10.22.8.182:49818] 2018-01-05T11:28:42,343 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:42,343 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:42,343 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_3][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_3_00_000000_0, startTime=1515180522342, containerId=container_111101111_0001_00_000004, nodeId=10.22.8.182:49818 2018-01-05T11:28:42,344 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_3_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112842_16059af6-eef3-4eff-a291-f1d4220f6f92, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000004, dagName=ANALYZE TABLE src1 COMPUTE STATISTICS (Stage-0), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{src1,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{out_Map 1,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180522303, firstAttemptStartTime=1515180522337, currentAttemptStartTime=1515180522342} 2018-01-05T11:28:42,344 INFO [IPC Server handler 2 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000004, QueryId=djaiswal_20180105112842_16059af6-eef3-4eff-a291-f1d4220f6f92-3, DagId=3, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180522344 2018-01-05T11:28:42,344 DEBUG [IPC Server handler 2 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=3} with the ShuffleHandler 2018-01-05T11:28:42,355 INFO [IPC Server handler 2 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:28:42,355 DEBUG [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/3] 2018-01-05T11:28:42,356 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:42,356 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_3_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:42,356 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_3_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180522337, dagStartTime=1515180522303, withinDagPriority=2, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:28:42,356 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_3_00_000000_0 : ACCEPTED 2018-01-05T11:28:42,356 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_3_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:42,356 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_3_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180522337, dagStartTime=1515180522303, withinDagPriority=2, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:28:42,356 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_3_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:42,356 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_3_00_000000_0 to pre-emption queue 2018-01-05T11:28:42,357 INFO [TaskCommunicator # 6] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_3_00_000000_0 2018-01-05T11:28:42,357 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_3_00_000000_0: true 2018-01-05T11:28:42,357 INFO [TaskCommunicator # 6] GuaranteedTasks: Received response for attempt_1515180518813_0001_3_00_000000_0, true 2018-01-05T11:28:42,357 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: taskOwner hashCode:1187020192 2018-01-05T11:28:42,358 INFO [Task-Executor-3] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : ANALYZE TABLE src1 COMPUTE STATISTICS (Stage-0), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_3_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=src1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=out_Map 1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:28:42,358 INFO [Task-Executor-3] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=954728448, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:42,358 INFO [Task-Executor-3] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_3_00_000000_0 vertexName: Map 1 2018-01-05T11:28:42,358 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000004, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_3_00_000000_0, eventCount=1 } 2018-01-05T11:28:42,360 DEBUG [TezTR-518813_1_3_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_3 queryId: djaiswal_20180105112842_16059af6-eef3-4eff-a291-f1d4220f6f92 instanceType: TaskRunner2Callable 2018-01-05T11:28:42,361 INFO [TezTR-518813_1_3_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_3_00_000000_0 2018-01-05T11:28:42,362 DEBUG [TezTR-518813_1_3_0_0_0] log.PerfLogger: 2018-01-05T11:28:42,363 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:42,365 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:42,368 INFO [IPC Server handler 6 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_3_00_000000_0 sent events: (0-1). 2018-01-05T11:28:42,369 DEBUG [TezTR-518813_1_3_0_0_0] log.PerfLogger: 2018-01-05T11:28:42,369 INFO [TezTR-518813_1_3_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:42,369 INFO [TezTR-518813_1_3_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:42,369 INFO [I/O Setup 0 Initialize: {src1}] input.MRInput: src1 using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:28:42,369 INFO [I/O Setup 0 Initialize: {src1}] input.MRInputLegacy: src1 MRInputLegacy deferring initialization 2018-01-05T11:28:42,369 INFO [TezTR-518813_1_3_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:42,369 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:28:42,370 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_3_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:28:42,370 INFO [I/O Setup 1 Initialize: {out_Map 1}] output.MROutput: out_Map 1: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:28:42,370 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:42,370 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:42,370 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:42,370 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:42,370 INFO [TezTR-518813_1_3_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:42,370 INFO [TezTR-518813_1_3_0_0_0] resources.MemoryDistributor: InitialRequests=[src1:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [out_Map 1:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput] 2018-01-05T11:28:42,370 INFO [TezTR-518813_1_3_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:42,370 INFO [TezTR-518813_1_3_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:42,370 INFO [TezTR-518813_1_3_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 954728448, TotalAvailable: 639668060, TotalRequested/TotalJVMHeap:0.00 2018-01-05T11:28:42,370 INFO [TezTR-518813_1_3_0_0_0] resources.MemoryDistributor: Allocations=[src1:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [out_Map 1:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0] 2018-01-05T11:28:42,370 INFO [TezTR-518813_1_3_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:42,370 INFO [TezTR-518813_1_3_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: src1 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:28:42,370 INFO [TezTR-518813_1_3_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:28:42,370 INFO [TezTR-518813_1_3_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:28:42,370 INFO [I/O Setup 0 Start: {src1}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: src1 2018-01-05T11:28:42,370 INFO [TezTR-518813_1_3_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:42,371 INFO [TezTR-518813_1_3_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_3_00_000000_0 2018-01-05T11:28:42,371 DEBUG [TezTR-518813_1_3_0_0_0] log.PerfLogger: 2018-01-05T11:28:42,371 DEBUG [TezTR-518813_1_3_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_3_00_000000_0_10010 2018-01-05T11:28:42,371 DEBUG [TezTR-518813_1_3_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:42,371 DEBUG [TezTR-518813_1_3_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_3_00_000000_0 2018-01-05T11:28:42,371 INFO [TezTR-518813_1_3_0_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112842_16059af6-eef3-4eff-a291-f1d4220f6f92 2018-01-05T11:28:42,371 DEBUG [TezTR-518813_1_3_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:42,371 DEBUG [TezTR-518813_1_3_0_0_0] log.PerfLogger: 2018-01-05T11:28:42,371 DEBUG [TezTR-518813_1_3_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:42,371 DEBUG [TezTR-518813_1_3_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:42,371 DEBUG [TezTR-518813_1_3_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112842_16059af6-eef3-4eff-a291-f1d4220f6f92_Map 1__MAP_PLAN__ 2018-01-05T11:28:42,371 INFO [TezTR-518813_1_3_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_167_8868162020991472848-1/djaiswal/_tez_scratch_dir/99e0de7c-a314-42c9-8205-aa1581396905/map.xml 2018-01-05T11:28:42,371 DEBUG [TezTR-518813_1_3_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_167_8868162020991472848-1/djaiswal/_tez_scratch_dir/99e0de7c-a314-42c9-8205-aa1581396905/map.xml 2018-01-05T11:28:42,371 DEBUG [TezTR-518813_1_3_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_167_8868162020991472848-1/djaiswal/_tez_scratch_dir/99e0de7c-a314-42c9-8205-aa1581396905/map.xml 2018-01-05T11:28:42,372 DEBUG [TezTR-518813_1_3_0_0_0] log.PerfLogger: 2018-01-05T11:28:42,372 INFO [TezTR-518813_1_3_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:42,372 DEBUG [TezTR-518813_1_3_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:42,372 DEBUG [TezTR-518813_1_3_0_0_0] log.PerfLogger: 2018-01-05T11:28:42,372 INFO [TezTR-518813_1_3_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 1.73KB 2018-01-05T11:28:42,373 DEBUG [TezTR-518813_1_3_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:42,373 INFO [TezTR-518813_1_3_0_0_0] tez.MapRecordProcessor: The input names are: [src1] 2018-01-05T11:28:42,373 INFO [TezTR-518813_1_3_0_0_0] input.MRInput: Initialized MRInput: src1 2018-01-05T11:28:42,373 DEBUG [TezTR-518813_1_3_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 1 2018-01-05T11:28:42,373 INFO [TezTR-518813_1_3_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_167_8868162020991472848-1/djaiswal/_tez_scratch_dir/99e0de7c-a314-42c9-8205-aa1581396905/map.xml 2018-01-05T11:28:42,373 DEBUG [TezTR-518813_1_3_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:42,373 DEBUG [TezTR-518813_1_3_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1/kv3.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.src1, numFiles=1, columns.types=string:string, serialization.format=1, serialization.ddl=struct src1 { string key, string value}, columns=key,value, columns.comments=defaultdefault, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, totalSize=216, column.name.delimiter=,, transient_lastDdlTime=1515180522}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.src1, numFiles=1, columns.types=string:string, columns=key,value, serialization.format=1, serialization.ddl=struct src1 { string key, string value}, columns.comments=defaultdefault, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, totalSize=216, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1, column.name.delimiter=,, transient_lastDdlTime=1515180522}, baseFileName=src1, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.src1, numFiles=1, columns.types=string:string, serialization.format=1, serialization.ddl=struct src1 { string key, string value}, columns=key,value, columns.comments=defaultdefault, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, totalSize=216, column.name.delimiter=,, transient_lastDdlTime=1515180522}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.src1, numFiles=1, columns.types=string:string, columns=key,value, serialization.format=1, serialization.ddl=struct src1 { string key, string value}, columns.comments=defaultdefault, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, totalSize=216, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1, column.name.delimiter=,, transient_lastDdlTime=1515180522}, baseFileName=src1, vectorPartitionDesc=null]} 2018-01-05T11:28:42,374 DEBUG [TezTR-518813_1_3_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:42,374 INFO [TezTR-518813_1_3_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:42,374 DEBUG [TezTR-518813_1_3_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:42,375 DEBUG [TezTR-518813_1_3_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:42,375 INFO [TezTR-518813_1_3_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_167_8868162020991472848-1/djaiswal/_tez_scratch_dir/99e0de7c-a314-42c9-8205-aa1581396905/map.xml 2018-01-05T11:28:42,375 DEBUG [TezTR-518813_1_3_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:42,375 INFO [TezTR-518813_1_3_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_3_00_000000_0 2018-01-05T11:28:42,375 INFO [TezTR-518813_1_3_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [string, string] (isAcidRead false) 2018-01-05T11:28:42,376 WARN [TezTR-518813_1_3_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1/kv3.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:28:42,376 DEBUG [TezTR-518813_1_3_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:28:42,376 WARN [TezTR-518813_1_3_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1/kv3.txt:0+216 2018-01-05T11:28:42,376 DEBUG [TezTR-518813_1_3_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1/kv3.txt 2018-01-05T11:28:42,376 INFO [TezTR-518813_1_3_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1/kv3.txt:0+216], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=216} 2018-01-05T11:28:42,376 INFO [TezTR-518813_1_3_0_0_0] input.MRInput: src1 initialized RecordReader from event 2018-01-05T11:28:42,376 DEBUG [TezTR-518813_1_3_0_0_0] tez.MapRecordProcessor: Starting Output: out_Map 1 2018-01-05T11:28:42,377 INFO [TezTR-518813_1_3_0_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:28:42,377 INFO [TezTR-518813_1_3_0_0_0] exec.MapOperator: Initializing operator MAP[0] 2018-01-05T11:28:42,377 DEBUG [TezTR-518813_1_3_0_0_0] exec.MapOperator: Initialization Done 0 MAP 2018-01-05T11:28:42,377 DEBUG [TezTR-518813_1_3_0_0_0] exec.MapOperator: Operator 0 MAP initialized 2018-01-05T11:28:42,377 DEBUG [TezTR-518813_1_3_0_0_0] exec.MapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:28:42,377 DEBUG [TezTR-518813_1_3_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:42,377 DEBUG [TezTR-518813_1_3_0_0_0] exec.MapOperator: Adding alias src1 to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1 2018-01-05T11:28:42,377 DEBUG [TezTR-518813_1_3_0_0_0] exec.MapOperator: dump TS[0] struct 2018-01-05T11:28:42,377 INFO [TezTR-518813_1_3_0_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:28:42,377 INFO [TezTR-518813_1_3_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:42,377 DEBUG [TezTR-518813_1_3_0_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:28:42,377 INFO [TezTR-518813_1_3_0_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:28:42,377 DEBUG [TezTR-518813_1_3_0_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:28:42,377 DEBUG [TezTR-518813_1_3_0_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:28:42,377 DEBUG [TezTR-518813_1_3_0_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:28:42,377 DEBUG [TezTR-518813_1_3_0_0_0] log.PerfLogger: 2018-01-05T11:28:42,377 INFO [TezTR-518813_1_3_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_167_8868162020991472848-1/djaiswal/_tez_scratch_dir/99e0de7c-a314-42c9-8205-aa1581396905/map.xml 2018-01-05T11:28:42,377 DEBUG [TezTR-518813_1_3_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:42,377 DEBUG [TezTR-518813_1_3_0_0_0] exec.MapOperator: Processing alias(es) src1 for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1/kv3.txt 2018-01-05T11:28:42,377 INFO [TezTR-518813_1_3_0_0_0] exec.MapOperator: MAP[0]: records read - 1 2018-01-05T11:28:42,378 INFO [TezTR-518813_1_3_0_0_0] exec.MapOperator: MAP[0]: records read - 10 2018-01-05T11:28:42,378 DEBUG [TezTR-518813_1_3_0_0_0] log.PerfLogger: 2018-01-05T11:28:42,378 DEBUG [TezTR-518813_1_3_0_0_0] mr.ObjectCache: djaiswal_20180105112842_16059af6-eef3-4eff-a291-f1d4220f6f92_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:28:42,378 DEBUG [TezTR-518813_1_3_0_0_0] exec.MapOperator: close called for operator MAP[0] 2018-01-05T11:28:42,378 INFO [TezTR-518813_1_3_0_0_0] exec.MapOperator: Closing operator MAP[0] 2018-01-05T11:28:42,378 INFO [TezTR-518813_1_3_0_0_0] exec.MapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:25, 2018-01-05T11:28:42,378 DEBUG [TezTR-518813_1_3_0_0_0] exec.MapOperator: Closing child = TS[0] 2018-01-05T11:28:42,378 DEBUG [TezTR-518813_1_3_0_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:28:42,378 DEBUG [TezTR-518813_1_3_0_0_0] exec.TableScanOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:42,378 INFO [TezTR-518813_1_3_0_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:28:42,378 DEBUG [TezTR-518813_1_3_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:42,378 DEBUG [TezTR-518813_1_3_0_0_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_167_8868162020991472848-1/-mr-10000 2018-01-05T11:28:42,378 INFO [TezTR-518813_1_3_0_0_0] exec.TableScanOperator: publishing : default.src1/ : {numRows=25, rawDataSize=191} 2018-01-05T11:28:42,401 DEBUG [TezTR-518813_1_3_0_0_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_167_8868162020991472848-1/-mr-10000/tmpstats-0_0 2018-01-05T11:28:42,401 DEBUG [TezTR-518813_1_3_0_0_0] fs.FSStatsPublisher: Writing stats in it : {default.src1/={numRows=25, rawDataSize=191}} 2018-01-05T11:28:42,401 DEBUG [TezTR-518813_1_3_0_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:28:42,401 DEBUG [TezTR-518813_1_3_0_0_0] exec.MapOperator: 0 Close done 2018-01-05T11:28:42,401 INFO [TezTR-518813_1_3_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_3_00_000000_0 2018-01-05T11:28:42,401 INFO [TezTR-518813_1_3_0_0_0] output.MROutput: out_Map 1 closed 2018-01-05T11:28:42,402 DEBUG [TezTR-518813_1_3_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:42,402 INFO [TezTR-518813_1_3_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:42,402 INFO [TezTR-518813_1_3_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_3_00_000000_0, askedToStop=false 2018-01-05T11:28:42,402 INFO [TezTR-518813_1_3_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_3_00_000000_0, stopRequested=false 2018-01-05T11:28:42,403 INFO [TezTR-518813_1_3_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_3_00_000000_0: Counters: 5 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=25, INPUT_SPLIT_LENGTH_BYTES=216, OUTPUT_RECORDS=0][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=25]] 2018-01-05T11:28:42,403 DEBUG [TezTR-518813_1_3_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_3_0_0_0 thread id: 394 scheme: file bytesRead: 232 bytesWritten: 81 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:42,403 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_3_00_000000_0, isTaskDone=true 2018-01-05T11:28:42,403 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000004, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_3_00_000000_0, eventCount=2 } 2018-01-05T11:28:42,405 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_3][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_3_00_000000_0, creationTime=1515180522337, allocationTime=1515180522339, startTime=1515180522342, finishTime=1515180522405, timeTaken=63, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:42,405 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_3_00_000000_0 2018-01-05T11:28:42,406 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:42,405 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_3_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:42,406 INFO [Task-Executor-3] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_3_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:42,406 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_3_00_000000_0; false 2018-01-05T11:28:42,406 INFO [Task-Executor-3] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_3_00_000000_0 2018-01-05T11:28:42,406 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_3_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:42,406 INFO [Task-Executor-3] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000004=49 2018-01-05T11:28:42,406 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_3_00_000000_0: true 2018-01-05T11:28:42,406 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:42,407 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:42,407 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_3_00_000000_0 2018-01-05T11:28:42,407 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000004, QueryId=djaiswal_20180105112842_16059af6-eef3-4eff-a291-f1d4220f6f92-3, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:28:42,407 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_3_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:42,408 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_3_00_000000_0 2018-01-05T11:28:42,408 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_3][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_3_00_000000, startTime=1515180522342, finishTime=1515180522406, timeTaken=64, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_3_00_000000_0, diagnostics=, counters=Counters: 11, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=232, FILE_BYTES_WRITTEN=81, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=25, INPUT_SPLIT_LENGTH_BYTES=216, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=25 2018-01-05T11:28:42,408 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_3_00 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:42,408 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_3_00 [Map 1] 2018-01-05T11:28:42,408 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_3][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_3_00, initRequestedTime=1515180522303, initedTime=1515180522336, startRequestedTime=1515180522315, startedTime=1515180522336, finishTime=1515180522408, timeTaken=72, status=SUCCEEDED, diagnostics=, counters=Counters: 11, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=232, FILE_BYTES_WRITTEN=81, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=25, INPUT_SPLIT_LENGTH_BYTES=216, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=25, vertexStats=firstTaskStartTime=1515180522342, firstTasksToStart=[ task_1515180518813_0001_3_00_000000 ], lastTaskFinishTime=1515180522405, lastTasksToFinish=[ task_1515180518813_0001_3_00_000000 ], minTaskDuration=63, maxTaskDuration=63, avgTaskDuration=63.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_3_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_3_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:42,408 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_3_00 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:42,408 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_3][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000004, stoppedTime=1515180522408, exitStatus=0 2018-01-05T11:28:42,409 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000004 exited with diagnostics set to Container completed. 2018-01-05T11:28:42,409 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_3_00 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1 2018-01-05T11:28:42,409 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null 2018-01-05T11:28:42,409 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_3_00 [Map 1] 2018-01-05T11:28:42,410 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_3, queueSize=0 2018-01-05T11:28:42,410 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_3][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_3, startTime=1515180522303, finishTime=1515180522409, timeTaken=106, status=SUCCEEDED, diagnostics=, counters=Counters: 15, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, OTHER_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=212, AM_GC_TIME_MILLIS=0, File System Counters, FILE_BYTES_READ=232, FILE_BYTES_WRITTEN=81, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=25, INPUT_SPLIT_LENGTH_BYTES=216, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=25 2018-01-05T11:28:42,410 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_3 finished with state: SUCCEEDED 2018-01-05T11:28:42,411 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_3 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:28:42,411 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000004 2018-01-05T11:28:42,411 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_3, dagState=SUCCEEDED 2018-01-05T11:28:42,411 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_3_post 2018-01-05T11:28:42,411 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:42,411 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:28:42,411 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:28:42,411 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 3 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=1, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=1,NumRequestsWithoutLocation=1, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=1} 2018-01-05T11:28:42,411 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:28:42,411 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=ANALYZE TABLE src1 COMPUTE STATISTICS (Stage-0), with id=dag_1515180518813_0001_3 2018-01-05T11:28:42,412 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 3, to 10.22.8.182:49818 2018-01-05T11:28:42,412 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=1, Remaining=0 2018-01-05T11:28:42,412 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=ANALYZE TABLE src1 COMPUTE STATISTICS (Stage-0), with id=dag_1515180518813_0001_3 2018-01-05T11:28:42,414 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=3} 2018-01-05T11:28:42,414 INFO [IPC Server handler 4 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=3}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:28:42,414 INFO [IPC Server handler 4 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/3 after 300 seconds 2018-01-05T11:28:42,414 ERROR [IPC Server handler 4 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112842_16059af6-eef3-4eff-a291-f1d4220f6f92, dag_1515180518813_0001_3 2018-01-05T11:28:42,414 INFO [IPC Server handler 4 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112842_16059af6-eef3-4eff-a291-f1d4220f6f92 2018-01-05T11:28:42,414 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=3} = 0 2018-01-05T11:28:42,414 DEBUG [IPC Server handler 4 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=3} 2018-01-05T11:28:42,414 DEBUG [IPC Server handler 4 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:28:42,414 DEBUG [IPC Server handler 4 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=3} 2018-01-05T11:28:42,414 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,414 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,415 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 2018-01-05T11:28:42,415 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,415 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-2:STATS] in serial mode 2018-01-05T11:28:42,416 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@3cc548f6, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:42,416 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:42,416 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,416 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:42,416 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:42,416 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:42,416 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:42,416 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,416 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:42,416 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:42,416 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_167_8868162020991472848-1/-mr-10000 2018-01-05T11:28:42,416 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_167_8868162020991472848-1/-mr-10000 2018-01-05T11:28:42,417 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.src1/ numRows 2018-01-05T11:28:42,417 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.src1/ numRows 25 2018-01-05T11:28:42,417 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.src1/ rawDataSize 2018-01-05T11:28:42,417 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.src1/ rawDataSize 191 2018-01-05T11:28:42,418 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,418 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:42,418 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:42,418 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:42,418 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:42,418 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,419 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,419 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,420 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,421 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,421 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:42,421 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:42,421 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,421 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:42,421 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,421 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=src1 newtbl=src1 2018-01-05T11:28:42,421 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=src1 newtbl=src1 2018-01-05T11:28:42,427 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,437 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,438 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,438 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.src1 stats: [numFiles=1, numRows=25, totalSize=216, rawDataSize=191] 2018-01-05T11:28:42,438 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: About to delete stats tmp dir :file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_167_8868162020991472848-1/-mr-10000 2018-01-05T11:28:42,438 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,438 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,438 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: ANALYZE TABLE src1 COMPUTE STATISTICS 2018-01-05T11:28:42,438 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:28:42,438 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@src1 2018-01-05T11:28:42,438 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@src1 2018-01-05T11:28:42,438 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,438 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,439 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:42,439 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=17} 2018-01-05T11:28:42,439 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112842_16059af6-eef3-4eff-a291-f1d4220f6f92); Time taken: 0.231 seconds 2018-01-05T11:28:42,439 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:42,439 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,439 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:42,439 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,439 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query ANALYZE TABLE src1 COMPUTE STATISTICS 2018-01-05T11:28:42,439 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_167_8868162020991472848-1 2018-01-05T11:28:42,439 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.273 seconds 2018-01-05T11:28:42,439 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:42,440 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:42,440 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:42,440 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:42,440 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,440 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:42,440 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,440 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,440 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,440 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: ANALYZE TABLE src1 COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:42,440 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112842_6905c06f-f509-4bd9-bd82-f6b7ffb13c54): ANALYZE TABLE src1 COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:42,440 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,440 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: ANALYZE TABLE src1 COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:42,440 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:42,440 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,440 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,441 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,441 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,441 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:42,441 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,441 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src1 2018-01-05T11:28:42,441 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src1 2018-01-05T11:28:42,445 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,446 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,446 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: select compute_stats(`key`, 'hll') , compute_stats(`value`, 'hll') from `default`.`src1` 2018-01-05T11:28:42,446 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: select compute_stats(`key`, 'hll') , compute_stats(`value`, 'hll') from `default`.`src1` 2018-01-05T11:28:42,447 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:42,447 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Invoking analyze on rewritten query 2018-01-05T11:28:42,447 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Starting Semantic Analysis 2018-01-05T11:28:42,447 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:42,447 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:42,447 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed phase 1 of Semantic Analysis 2018-01-05T11:28:42,447 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for source tables 2018-01-05T11:28:42,447 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,447 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src1 2018-01-05T11:28:42,447 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src1 2018-01-05T11:28:42,452 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,453 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,453 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for subqueries 2018-01-05T11:28:42,453 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for destination tables 2018-01-05T11:28:42,465 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed getting MetaData in Semantic Analysis 2018-01-05T11:28:42,465 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_446_7884164969442448570-1/-mr-10001 2018-01-05T11:28:42,465 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Table Plan for src1 TS[0] 2018-01-05T11:28:42,465 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:42,466 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:42,466 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: RR before GB src1{(key,key: string)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)(RAW__DATA__SIZE,RAW__DATA__SIZE: bigint)} after GB {((tok_function compute_stats (tok_table_or_col key) 'hll'),_col0: struct)((tok_function compute_stats (tok_table_or_col value) 'hll'),_col1: struct)} 2018-01-05T11:28:42,466 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: tree: (tok_select (tok_selexpr (tok_function compute_stats (tok_table_or_col key) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col value) 'hll'))) 2018-01-05T11:28:42,466 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: genSelectPlan: input = {((tok_function compute_stats (tok_table_or_col key) 'hll'),_col0: struct)((tok_function compute_stats (tok_table_or_col value) 'hll'),_col1: struct)} starRr = src1{(key,key: string)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)(RAW__DATA__SIZE,RAW__DATA__SIZE: bigint)} 2018-01-05T11:28:42,466 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Select Plan row schema: null{(_c0,_col0: struct)(_c1,_col1: struct)} 2018-01-05T11:28:42,466 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Select Plan for clause: insclause-0 2018-01-05T11:28:42,466 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_446_7884164969442448570-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-42_446_7884164969442448570-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_446_7884164969442448570-1/-mr-10000 2018-01-05T11:28:42,466 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_446_7884164969442448570-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-42_446_7884164969442448570-1 2018-01-05T11:28:42,497 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_446_7884164969442448570-1/-mr-10003 2018-01-05T11:28:42,497 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_446_7884164969442448570-1/-mr-10000 row schema: null{(_c0,_col0: struct)(_c1,_col1: struct)} 2018-01-05T11:28:42,497 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Body Plan for Query Block null 2018-01-05T11:28:42,497 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Plan for Query Block null 2018-01-05T11:28:42,497 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Before logical optimization TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-SEL[5]-FS[6] 2018-01-05T11:28:42,497 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,497 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,497 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] lineage.Generator: Time taken for lineage transform=1 2018-01-05T11:28:42,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{src1}key,value: string|{src1}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src1}block__offset__inside__file,INPUT__FILE__NAME: string|{src1}input__file__name,ROW__ID: struct|{src1}row__id,RAW__DATA__SIZE: bigint|{src1}raw__data__size) 2018-01-05T11:28:42,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: string|{src1}key,value: string|{src1}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src1}block__offset__inside__file,INPUT__FILE__NAME: string|{src1}input__file__name,ROW__ID: struct|{src1}row__id,RAW__DATA__SIZE: bigint|{src1}raw__data__size) 2018-01-05T11:28:42,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:42,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:42,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:28:42,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:42,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:42,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:42,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:42,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:42,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:42,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:42,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:42,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:42,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1) 2018-01-05T11:28:42,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:42,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:42,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0] Column[_col1]) 2018-01-05T11:28:42,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{}) 2018-01-05T11:28:42,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:42,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:42,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,498 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for FS(6) 2018-01-05T11:28:42,498 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(5) 2018-01-05T11:28:42,498 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for GBY(4) 2018-01-05T11:28:42,498 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for RS(3) 2018-01-05T11:28:42,498 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for GBY(2) 2018-01-05T11:28:42,498 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(1) 2018-01-05T11:28:42,498 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:28:42,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.PredicatePushDown: After PPD: TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-SEL[5]-FS[6] 2018-01-05T11:28:42,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{src1}key,value: string|{src1}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src1}block__offset__inside__file,INPUT__FILE__NAME: string|{src1}input__file__name,ROW__ID: struct|{src1}row__id,RAW__DATA__SIZE: bigint|{src1}raw__data__size) 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: string|{src1}key,value: string|{src1}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src1}block__offset__inside__file,INPUT__FILE__NAME: string|{src1}input__file__name,ROW__ID: struct|{src1}row__id,RAW__DATA__SIZE: bigint|{src1}raw__data__size) 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1) 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0] Column[_col1]) 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{}) 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{src1}key,value: string|{src1}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src1}block__offset__inside__file,INPUT__FILE__NAME: string|{src1}input__file__name,ROW__ID: struct|{src1}row__id,RAW__DATA__SIZE: bigint|{src1}raw__data__size) 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: string|{src1}key,value: string|{src1}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src1}block__offset__inside__file,INPUT__FILE__NAME: string|{src1}input__file__name,ROW__ID: struct|{src1}row__id,RAW__DATA__SIZE: bigint|{src1}raw__data__size) 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1) 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0] Column[_col1]) 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{}) 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 3 key:[] 2018-01-05T11:28:42,499 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 3 oldColExprMap: {VALUE._col0=Column[_col0], VALUE._col1=Column[_col1]} 2018-01-05T11:28:42,500 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 3 newColExprMap: {VALUE._col0=Column[_col0], VALUE._col1=Column[_col1]} 2018-01-05T11:28:42,500 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,500 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,500 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,500 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,500 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,500 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,500 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,500 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,500 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,500 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,500 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,500 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,500 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,500 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,500 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,500 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,500 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,500 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,500 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.IdentityProjectRemover: Identity project remover optimization removed : SEL[5] 2018-01-05T11:28:42,500 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,500 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,500 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,500 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,500 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,500 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,500 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,500 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: After logical optimization TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-FS[6] 2018-01-05T11:28:42,500 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,500 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,500 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,500 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,501 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,501 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,501 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=src1 2018-01-05T11:28:42,501 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=src1 2018-01-05T11:28:42,508 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.260679ms + 0.010179ms, the query is [select "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "TAB_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (...)] 2018-01-05T11:28:42,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,510 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000004, QueryId=djaiswal_20180105112842_16059af6-eef3-4eff-a291-f1d4220f6f92-3, DagId=3, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-3, Succeeded=true, StartTime=1515180522357, EndTime=1515180522510 2018-01-05T11:28:42,514 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting stats (Num rows: 25 Data size: 9200 Basic stats: COMPLETE Column stats: NONE) on TS[0] 2018-01-05T11:28:42,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (src1): numRows: 25 dataSize: 9200 basicStatsState: COMPLETE colStatsState: NONE colStats: {value= colName: value colType: string countDistincts: 5 numNulls: 1 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: string countDistincts: 5 numNulls: 1 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true} 2018-01-05T11:28:42,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting stats (Num rows: 25 Data size: 9200 Basic stats: COMPLETE Column stats: NONE) on SEL[1] 2018-01-05T11:28:42,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [1] STATS-SEL[1]: numRows: 25 dataSize: 9200 basicStatsState: COMPLETE colStatsState: NONE colStats: {value= colName: value colType: string countDistincts: 5 numNulls: 1 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: string countDistincts: 5 numNulls: 1 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true} 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[2]: inputSize: 9200 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [Case 1] STATS-GBY[2]: cardinality: 25 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 1248 Basic stats: COMPLETE Column stats: NONE) on GBY[2] 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-GBY[2]: numRows: 1 dataSize: 1248 basicStatsState: COMPLETE colStatsState: NONE colStats: {_col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true} 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 1248 Basic stats: COMPLETE Column stats: NONE) on RS[3] 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-RS[3]: numRows: 1 dataSize: 1248 basicStatsState: COMPLETE colStatsState: NONE colStats: {_col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true} 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[4]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [Case 7] STATS-GBY[4]: cardinality: 0 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[4]: Equals 0 in number of rows. 0 rows will be set to 1 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 1248 Basic stats: COMPLETE Column stats: NONE) on GBY[4] 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-GBY[4]: numRows: 1 dataSize: 1248 basicStatsState: COMPLETE colStatsState: NONE colStats: {_col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true} 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-FS[6]: numRows: 1 dataSize: 1248 basicStatsState: COMPLETE colStatsState: NONE colStats: {_col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true} 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on TS[0] 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on SEL[1] 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[2] 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[3] 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[4] 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FileSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on FS[6] 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,516 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: FS, 6 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: GBY, 4 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: RS, 3 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: GBY, 2 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: SEL, 1 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:28:42,516 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Cycle free: true 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{src1}key,value: string|{src1}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src1}block__offset__inside__file,INPUT__FILE__NAME: string|{src1}input__file__name,ROW__ID: struct|{src1}row__id,RAW__DATA__SIZE: bigint|{src1}raw__data__size) 2018-01-05T11:28:42,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: string|{src1}key,value: string|{src1}value) 2018-01-05T11:28:42,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:42,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:42,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value]) 2018-01-05T11:28:42,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:42,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:42,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:42,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:42,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:42,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:42,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:42,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:42,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:42,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{}) 2018-01-05T11:28:42,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:42,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:42,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: TS[0] 2018-01-05T11:28:42,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: RS[3] 2018-01-05T11:28:42,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:28:42,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1 of table src1 2018-01-05T11:28:42,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1 2018-01-05T11:28:42,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: RS[3] 2018-01-05T11:28:42,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: GBY[4] 2018-01-05T11:28:42,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: FS[6] 2018-01-05T11:28:42,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding reduce work (Reducer 2) for GBY[4] 2018-01-05T11:28:42,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Setting up reduce sink: RS[3] with following reduce work: Reducer 2 2018-01-05T11:28:42,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Removing RS[3] as parent from GBY[4] 2018-01-05T11:28:42,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: FS[6] 2018-01-05T11:28:42,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:28:42,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:42,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:42,518 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:42,518 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:42,518 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:42,518 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:42,518 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:28:42,518 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping vectorization 2018-01-05T11:28:42,518 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:28:42,518 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: llap mode: only 2018-01-05T11:28:42,518 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:28:42,518 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(key, 'hll')' 2018-01-05T11:28:42,518 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'key' 2018-01-05T11:28:42,518 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:42,518 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(value, 'hll')' 2018-01-05T11:28:42,518 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'value' 2018-01-05T11:28:42,518 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:42,518 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'key' 2018-01-05T11:28:42,518 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'value' 2018-01-05T11:28:42,518 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:42,518 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@4088c5e6 can be done in LLAP 2018-01-05T11:28:42,518 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Reducer 2 2018-01-05T11:28:42,518 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col0)' 2018-01-05T11:28:42,518 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:28:42,518 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col1)' 2018-01-05T11:28:42,518 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col1' 2018-01-05T11:28:42,518 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:42,518 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@6e757f39 can be done in LLAP 2018-01-05T11:28:42,518 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,518 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed plan generation 2018-01-05T11:28:42,518 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:42,518 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validation start 2018-01-05T11:28:42,518 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Not a partition. 2018-01-05T11:28:42,518 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:28:42,518 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,518 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0, type:struct, comment:null), FieldSchema(name:_c1, type:struct, comment:null)], properties:null) 2018-01-05T11:28:42,518 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,518 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:42,518 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=1, getTable_(String, String, )=11, flushCache_()=0, getTableColumnStatistics_(String, String, List, )=14} 2018-01-05T11:28:42,518 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112842_6905c06f-f509-4bd9-bd82-f6b7ffb13c54); Time taken: 0.078 seconds 2018-01-05T11:28:42,518 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:42,518 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,518 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112842_6905c06f-f509-4bd9-bd82-f6b7ffb13c54): ANALYZE TABLE src1 COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:42,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,520 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: ANALYZE TABLE src1 COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:42,520 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:28:42,520 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@src1 2018-01-05T11:28:42,520 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@src1 2018-01-05T11:28:42,520 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_446_7884164969442448570-1/-mr-10000 2018-01-05T11:28:42,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,520 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Query ID = djaiswal_20180105112842_6905c06f-f509-4bd9-bd82-f6b7ffb13c54 2018-01-05T11:28:42,520 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Total jobs = 1 2018-01-05T11:28:42,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,520 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:28:42,520 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MAPRED] in serial mode 2018-01-05T11:28:42,543 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_440_447381380713723563-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:28:42,565 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:28:42,565 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:28:42,565 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:28:42,565 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112842_6905c06f-f509-4bd9-bd82-f6b7ffb13c54 2018-01-05T11:28:42,565 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Session is already open 2018-01-05T11:28:42,565 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:28:42,565 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,565 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Dag name: ANALYZE TABLE src1 COMPUTE STATI...key,value (Stage-0) 2018-01-05T11:28:42,565 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: DagInfo: {"context":"Hive","description":"\n\nANALYZE TABLE src1 COMPUTE STATISTICS FOR COLUMNS key,value"} 2018-01-05T11:28:42,565 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112842_6905c06f-f509-4bd9-bd82-f6b7ffb13c54 with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:28:42,565 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,577 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,577 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:28:42,579 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,579 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 2 size: 2.01KB 2018-01-05T11:28:42,598 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,598 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,610 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:28:42,610 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,610 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:28:42,612 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,612 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 3.19KB 2018-01-05T11:28:42,620 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_446_7884164969442448570-1/-mr-10001 2018-01-05T11:28:42,632 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_446_7884164969442448570-1/-mr-10001 2018-01-05T11:28:42,632 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1 2018-01-05T11:28:42,632 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,636 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,636 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,636 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=ANALYZE TABLE src1 COMPUTE STATI...key,value (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112842_6905c06f-f509-4bd9-bd82-f6b7ffb13c54 } 2018-01-05T11:28:42,637 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:42,637 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:42,641 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: ANALYZE TABLE src1 COMPUTE STATI...key,value (Stage-0) 2018-01-05T11:28:42,641 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:28:42,642 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_4, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_4.dot 2018-01-05T11:28:42,642 INFO [IPC Server handler 0 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_4 2018-01-05T11:28:42,642 WARN [IPC Server handler 0 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:42,642 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Running DAG: ANALYZE TABLE src1 COMPUTE STATI...key,value (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112842_6905c06f-f509-4bd9-bd82-f6b7ffb13c54 } 2018-01-05T11:28:42,666 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_4][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_4, submitTime=1515180522641, queueName=null 2018-01-05T11:28:42,667 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_4_00 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_4_00 [Map 1], Default Resources= 2018-01-05T11:28:42,667 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_4_00 [Map 1] 2018-01-05T11:28:42,667 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:28:42,667 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_4_00 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:42,668 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_4_01 [Reducer 2]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_4_01 [Reducer 2], Default Resources= 2018-01-05T11:28:42,668 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_4_01 [Reducer 2] 2018-01-05T11:28:42,668 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 2 2018-01-05T11:28:42,668 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_4_01 [Reducer 2] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:42,668 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:28:42,668 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_4][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_4, initTime=1515180522668 2018-01-05T11:28:42,668 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: dag_1515180518813_0001_4 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:28:42,669 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:28:42,669 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_4][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_4, startTime=1515180522669 2018-01-05T11:28:42,669 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_4 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:28:42,670 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {src1={InputName=src1}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:28:42,670 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: src1, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:28:42,670 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_4_00 [Map 1] 2018-01-05T11:28:42,670 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_4, dagName=ANALYZE TABLE src1 COMPUTE STATI...key,value (Stage-0) 2018-01-05T11:28:42,670 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,670 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,670 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,675 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_4_00 [Map 1] 2018-01-05T11:28:42,675 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_4_00 [Map 1] 2018-01-05T11:28:42,675 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:42,675 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_4_00 [Map 1] 2018-01-05T11:28:42,677 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:28:42,677 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@3b41bf8d 2018-01-05T11:28:42,678 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_440_447381380713723563-1/djaiswal/_tez_scratch_dir/62646ef2-e992-4430-a80b-dc9e035b0238/map.xml 2018-01-05T11:28:42,678 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_440_447381380713723563-1/djaiswal/_tez_scratch_dir/62646ef2-e992-4430-a80b-dc9e035b0238/map.xml 2018-01-05T11:28:42,678 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_440_447381380713723563-1/djaiswal/_tez_scratch_dir/62646ef2-e992-4430-a80b-dc9e035b0238/map.xml 2018-01-05T11:28:42,678 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:42,678 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:42,678 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:42,679 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:42,679 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.19KB 2018-01-05T11:28:42,680 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_4_00 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:28:42,680 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: src1 on vertex vertex_1515180518813_0001_4_00 [Map 1] 2018-01-05T11:28:42,681 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to ShuffleVertexManager for vertex_1515180518813_0001_4_01 [Reducer 2] 2018-01-05T11:28:42,681 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:28:42,681 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:42,681 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:28:42,681 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:42,681 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_440_447381380713723563-1/djaiswal/_tez_scratch_dir/62646ef2-e992-4430-a80b-dc9e035b0238/map.xml 2018-01-05T11:28:42,681 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:42,681 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias src1 2018-01-05T11:28:42,681 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1 2018-01-05T11:28:42,681 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1 2018-01-05T11:28:42,681 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 0,1 2018-01-05T11:28:42,681 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = key,value 2018-01-05T11:28:42,681 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1 2018-01-05T11:28:42,681 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:42,681 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:42,681 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:42,681 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:42,686 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.05 maxFrac: 0.75 auto: false desiredTaskIput: 104857600 2018-01-05T11:28:42,686 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:28:42,686 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 1 tasks for vertex: vertex_1515180518813_0001_4_01 [Reducer 2] 2018-01-05T11:28:42,686 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_4_01 [Reducer 2] 2018-01-05T11:28:42,686 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_4][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_4_01, reconfigureDoneTime=1515180522686, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:28:42,686 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_4_01 [Reducer 2], numAdditionalOutputs=1 2018-01-05T11:28:42,686 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_4][Event:VERTEX_INITIALIZED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_4_01, initRequestedTime=1515180522680, initedTime=1515180522686, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:42,686 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_4_01 [Reducer 2] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:28:42,697 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Time taken to get FileStatuses: 15 2018-01-05T11:28:42,697 INFO [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:28:42,697 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 15 2018-01-05T11:28:42,697 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 1 2018-01-05T11:28:42,697 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:42,697 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 1. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:28:42,697 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_440_447381380713723563-1/djaiswal/_tez_scratch_dir/62646ef2-e992-4430-a80b-dc9e035b0238/map.xml 2018-01-05T11:28:42,697 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_440_447381380713723563-1/djaiswal/_tez_scratch_dir/62646ef2-e992-4430-a80b-dc9e035b0238/map.xml 2018-01-05T11:28:42,697 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_440_447381380713723563-1/djaiswal/_tez_scratch_dir/62646ef2-e992-4430-a80b-dc9e035b0238/map.xml 2018-01-05T11:28:42,697 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:42,697 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:42,697 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:42,699 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:42,699 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.19KB 2018-01-05T11:28:42,699 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1/kv3.txt to src new group? true 2018-01-05T11:28:42,699 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:28:42,699 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:28:42,699 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:28:42,699 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 4 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 216 Original splits: 1 2018-01-05T11:28:42,699 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Using original number of splits: 1 desired splits: 1 2018-01-05T11:28:42,699 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 1 grouped split count is 1, for bucket: 1 2018-01-05T11:28:42,699 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:28:42,699 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: src1 on vertex vertex_1515180518813_0001_4_00 [Map 1] 2018-01-05T11:28:42,700 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_4_00 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:42,700 INFO [App Shared Pool - #0] impl.VertexImpl: Got updated RootInputsSpecs: {src1=forAllWorkUnits=true, update=[1]} 2018-01-05T11:28:42,700 INFO [App Shared Pool - #0] impl.VertexImpl: Vertex vertex_1515180518813_0001_4_00 [Map 1] parallelism set to 1 2018-01-05T11:28:42,700 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_4_00 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:42,700 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_4][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_4_00, initRequestedTime=1515180522669, initedTime=1515180522700, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:42,700 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_4_00 [Map 1] with distanceFromRoot: 0 2018-01-05T11:28:42,700 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_4_00 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:28:42,700 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_4][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_4_00, startRequestedTime=1515180522680, startedTime=1515180522700 2018-01-05T11:28:42,700 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_4][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_4_00, reconfigureDoneTime=1515180522700, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:28:42,700 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_4_00 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:42,700 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_4_00 for vertex: vertex_1515180518813_0001_4_01 [Reducer 2] numStartedSources: 1 numSources: 1 2018-01-05T11:28:42,701 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_4_01 [Reducer 2] with distanceFromRoot: 1 2018-01-05T11:28:42,701 INFO [App Shared Pool - #0] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:28:42,701 INFO [App Shared Pool - #0] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:28:42,701 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_4_00 [Map 1] 2018-01-05T11:28:42,701 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_4][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_4_01, startRequestedTime=1515180522700, startedTime=1515180522701 2018-01-05T11:28:42,701 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_4_01 [Reducer 2] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:42,701 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_4][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_4_00_000000, scheduledTime=1515180522701, launchTime=1515180522701 2018-01-05T11:28:42,701 INFO [App Shared Pool - #1] vertexmanager.ShuffleVertexManagerBase: OnVertexStarted vertex: Reducer 2 with 0 source tasks and 1 pending tasks 2018-01-05T11:28:42,701 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_4_00 [Map 1] 2018-01-05T11:28:42,702 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Map 1 in vertex: Reducer 2 numBipartiteSourceTasks: 1 2018-01-05T11:28:42,702 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_4_00_000000_0, priority=5, capability=, hosts=null 2018-01-05T11:28:42,702 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=5,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:42,702 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=5,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:42,702 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:42,702 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_4_00_000000_0 on hosts=null 2018-01-05T11:28:42,702 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_4_00_000000_0 on hosts=null 2018-01-05T11:28:42,702 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_4_00_000000_0 2018-01-05T11:28:42,702 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:42,702 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #5, task=TaskInfo{task=attempt_1515180518813_0001_4_00_000000_0, priority=5, startTime=0, containerId=null, uniqueId=4, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000005 2018-01-05T11:28:42,702 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_4_00_000000_0; false 2018-01-05T11:28:42,702 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_4_00_000000_0, priority=5, startTime=311543211, containerId=container_111101111_0001_00_000005assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=4, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:42,703 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000005 for node: 10.22.8.182:49818 2018-01-05T11:28:42,705 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 4, name=ANALYZE TABLE src1 COMPUTE STATI...key,value (Stage-0), queryId=djaiswal_20180105112842_6905c06f-f509-4bd9-bd82-f6b7ffb13c54 2018-01-05T11:28:42,705 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000005, attempt_1515180518813_0001_4_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:42,705 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000005 for node: 10.22.8.182:49818 2018-01-05T11:28:42,706 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000005 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:42,706 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000005, launchTime=1515180522706 2018-01-05T11:28:42,707 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_4_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000005] on NM: [10.22.8.182:49818] 2018-01-05T11:28:42,707 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:42,707 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:42,707 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_4][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_4_00_000000_0, startTime=1515180522706, containerId=container_111101111_0001_00_000005, nodeId=10.22.8.182:49818 2018-01-05T11:28:42,707 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_4_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112842_6905c06f-f509-4bd9-bd82-f6b7ffb13c54, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000005, dagName=ANALYZE TABLE src1 COMPUTE STATI...key,value (Stage-0), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{src1,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{Reducer 2,org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput,1}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180522669, firstAttemptStartTime=1515180522702, currentAttemptStartTime=1515180522706} 2018-01-05T11:28:42,707 INFO [IPC Server handler 1 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000005, QueryId=djaiswal_20180105112842_6905c06f-f509-4bd9-bd82-f6b7ffb13c54-4, DagId=4, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180522707 2018-01-05T11:28:42,707 DEBUG [IPC Server handler 1 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=4} with the ShuffleHandler 2018-01-05T11:28:42,719 INFO [IPC Server handler 1 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:28:42,719 DEBUG [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/4] 2018-01-05T11:28:42,719 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:42,719 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_4_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:42,719 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_4_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180522702, dagStartTime=1515180522669, withinDagPriority=5, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:28:42,719 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_4_00_000000_0 : ACCEPTED 2018-01-05T11:28:42,719 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_4_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:42,719 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_4_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180522702, dagStartTime=1515180522669, withinDagPriority=5, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:28:42,719 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_4_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:42,719 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_4_00_000000_0 to pre-emption queue 2018-01-05T11:28:42,720 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_4_00_000000_0: true 2018-01-05T11:28:42,720 INFO [TaskCommunicator # 8] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_4_00_000000_0 2018-01-05T11:28:42,720 INFO [TaskCommunicator # 8] GuaranteedTasks: Received response for attempt_1515180518813_0001_4_00_000000_0, true 2018-01-05T11:28:42,720 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: taskOwner hashCode:1280491721 2018-01-05T11:28:42,721 INFO [Task-Executor-3] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : ANALYZE TABLE src1 COMPUTE STATI...key,value (Stage-0), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_4_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=src1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=Reducer 2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput }}, ] 2018-01-05T11:28:42,721 INFO [Task-Executor-3] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=954728448, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:42,721 INFO [Task-Executor-3] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_4_00_000000_0 vertexName: Map 1 2018-01-05T11:28:42,721 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000005, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_4_00_000000_0, eventCount=1 } 2018-01-05T11:28:42,722 DEBUG [TezTR-518813_1_4_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_4 queryId: djaiswal_20180105112842_6905c06f-f509-4bd9-bd82-f6b7ffb13c54 instanceType: TaskRunner2Callable 2018-01-05T11:28:42,722 INFO [TezTR-518813_1_4_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_4_00_000000_0 2018-01-05T11:28:42,723 DEBUG [TezTR-518813_1_4_0_0_0] log.PerfLogger: 2018-01-05T11:28:42,724 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:42,724 INFO [I/O Setup 1 Initialize: {Reducer 2}] writers.UnorderedPartitionedKVWriter: Requested BufferSize (tez.runtime.unordered.output.buffer.size-mb) : 100 2018-01-05T11:28:42,726 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:42,729 INFO [IPC Server handler 2 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_4_00_000000_0 sent events: (0-1). 2018-01-05T11:28:42,730 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:28:42,730 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_4_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:28:42,730 INFO [I/O Setup 0 Initialize: {src1}] input.MRInput: src1 using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:28:42,730 INFO [I/O Setup 0 Initialize: {src1}] input.MRInputLegacy: src1 MRInputLegacy deferring initialization 2018-01-05T11:28:42,730 DEBUG [TezTR-518813_1_4_0_0_0] log.PerfLogger: 2018-01-05T11:28:42,730 INFO [TezTR-518813_1_4_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:42,730 INFO [TezTR-518813_1_4_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:42,730 INFO [TezTR-518813_1_4_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:42,730 INFO [TezTR-518813_1_4_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:42,730 INFO [TezTR-518813_1_4_0_0_0] resources.MemoryDistributor: InitialRequests=[src1:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [Reducer 2:OUTPUT:104857600:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput] 2018-01-05T11:28:42,731 INFO [TezTR-518813_1_4_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:42,731 INFO [TezTR-518813_1_4_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:42,731 INFO [TezTR-518813_1_4_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 104857600, TotalRequestedScaled: 5.24288E7, TotalJVMHeap: 954728448, TotalAvailable: 639668060, TotalRequested/TotalJVMHeap:0.11 2018-01-05T11:28:42,731 INFO [TezTR-518813_1_4_0_0_0] resources.MemoryDistributor: Allocations=[src1:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput:OUTPUT:104857600:104857600] 2018-01-05T11:28:42,731 INFO [TezTR-518813_1_4_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:42,731 INFO [TezTR-518813_1_4_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: src1 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:28:42,731 INFO [TezTR-518813_1_4_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:28:42,731 INFO [TezTR-518813_1_4_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:28:42,731 INFO [I/O Setup 1 Start: {src1}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: src1 2018-01-05T11:28:42,731 INFO [TezTR-518813_1_4_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:42,731 INFO [TezTR-518813_1_4_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_4_00_000000_0 2018-01-05T11:28:42,731 DEBUG [TezTR-518813_1_4_0_0_0] log.PerfLogger: 2018-01-05T11:28:42,731 DEBUG [TezTR-518813_1_4_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_4_00_000000_0_10013 2018-01-05T11:28:42,732 DEBUG [TezTR-518813_1_4_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:42,732 DEBUG [TezTR-518813_1_4_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_4_00_000000_0 2018-01-05T11:28:42,732 INFO [TezTR-518813_1_4_0_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112842_6905c06f-f509-4bd9-bd82-f6b7ffb13c54 2018-01-05T11:28:42,732 DEBUG [TezTR-518813_1_4_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:42,732 DEBUG [TezTR-518813_1_4_0_0_0] log.PerfLogger: 2018-01-05T11:28:42,732 DEBUG [TezTR-518813_1_4_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:42,732 DEBUG [TezTR-518813_1_4_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:42,732 DEBUG [TezTR-518813_1_4_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112842_6905c06f-f509-4bd9-bd82-f6b7ffb13c54_Map 1__MAP_PLAN__ 2018-01-05T11:28:42,732 INFO [TezTR-518813_1_4_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_440_447381380713723563-1/djaiswal/_tez_scratch_dir/62646ef2-e992-4430-a80b-dc9e035b0238/map.xml 2018-01-05T11:28:42,732 DEBUG [TezTR-518813_1_4_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_440_447381380713723563-1/djaiswal/_tez_scratch_dir/62646ef2-e992-4430-a80b-dc9e035b0238/map.xml 2018-01-05T11:28:42,732 DEBUG [TezTR-518813_1_4_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_440_447381380713723563-1/djaiswal/_tez_scratch_dir/62646ef2-e992-4430-a80b-dc9e035b0238/map.xml 2018-01-05T11:28:42,733 DEBUG [TezTR-518813_1_4_0_0_0] log.PerfLogger: 2018-01-05T11:28:42,733 INFO [TezTR-518813_1_4_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:42,733 DEBUG [TezTR-518813_1_4_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:42,734 DEBUG [TezTR-518813_1_4_0_0_0] log.PerfLogger: 2018-01-05T11:28:42,734 INFO [TezTR-518813_1_4_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.19KB 2018-01-05T11:28:42,734 DEBUG [TezTR-518813_1_4_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:42,735 INFO [TezTR-518813_1_4_0_0_0] tez.MapRecordProcessor: The input names are: [src1] 2018-01-05T11:28:42,735 INFO [TezTR-518813_1_4_0_0_0] input.MRInput: Initialized MRInput: src1 2018-01-05T11:28:42,735 DEBUG [TezTR-518813_1_4_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 1 2018-01-05T11:28:42,735 INFO [TezTR-518813_1_4_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_440_447381380713723563-1/djaiswal/_tez_scratch_dir/62646ef2-e992-4430-a80b-dc9e035b0238/map.xml 2018-01-05T11:28:42,735 DEBUG [TezTR-518813_1_4_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:42,735 DEBUG [TezTR-518813_1_4_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1/kv3.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=25, numFiles=1, serialization.ddl=struct src1 { string key, string value}, transient_lastDdlTime=1515180522, rawDataSize=191, columns.comments=defaultdefault, totalSize=216, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.src1}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=25, numFiles=1, serialization.ddl=struct src1 { string key, string value}, transient_lastDdlTime=1515180522, rawDataSize=191, columns.comments=defaultdefault, totalSize=216, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.src1}, baseFileName=src1, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=25, numFiles=1, serialization.ddl=struct src1 { string key, string value}, transient_lastDdlTime=1515180522, rawDataSize=191, columns.comments=defaultdefault, totalSize=216, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.src1}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=25, numFiles=1, serialization.ddl=struct src1 { string key, string value}, transient_lastDdlTime=1515180522, rawDataSize=191, columns.comments=defaultdefault, totalSize=216, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.src1}, baseFileName=src1, vectorPartitionDesc=null]} 2018-01-05T11:28:42,735 DEBUG [TezTR-518813_1_4_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:42,735 INFO [TezTR-518813_1_4_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:42,735 DEBUG [TezTR-518813_1_4_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:42,737 DEBUG [TezTR-518813_1_4_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:42,737 INFO [TezTR-518813_1_4_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_440_447381380713723563-1/djaiswal/_tez_scratch_dir/62646ef2-e992-4430-a80b-dc9e035b0238/map.xml 2018-01-05T11:28:42,737 DEBUG [TezTR-518813_1_4_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:42,737 INFO [TezTR-518813_1_4_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_4_00_000000_0 2018-01-05T11:28:42,737 INFO [TezTR-518813_1_4_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [string, string] (isAcidRead false) 2018-01-05T11:28:42,737 DEBUG [TezTR-518813_1_4_0_0_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:42,737 DEBUG [TezTR-518813_1_4_0_0_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:42,737 DEBUG [TezTR-518813_1_4_0_0_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:42,737 DEBUG [TezTR-518813_1_4_0_0_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:42,737 WARN [TezTR-518813_1_4_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1/kv3.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:28:42,738 DEBUG [TezTR-518813_1_4_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:28:42,738 WARN [TezTR-518813_1_4_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1/kv3.txt:0+216 2018-01-05T11:28:42,738 DEBUG [TezTR-518813_1_4_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1/kv3.txt 2018-01-05T11:28:42,738 INFO [TezTR-518813_1_4_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1/kv3.txt:0+216], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=216} 2018-01-05T11:28:42,738 INFO [TezTR-518813_1_4_0_0_0] input.MRInput: src1 initialized RecordReader from event 2018-01-05T11:28:42,738 DEBUG [TezTR-518813_1_4_0_0_0] tez.MapRecordProcessor: Starting Output: Reducer 2 2018-01-05T11:28:42,739 INFO [TezTR-518813_1_4_0_0_0] writers.BaseUnorderedPartitionedKVWriter: Instantiating Partitioner: [org.apache.tez.mapreduce.partition.MRPartitioner] 2018-01-05T11:28:42,762 INFO [TezTR-518813_1_4_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: numBuffers=2, sizePerBuffer=52428800, skipBuffers=true, numPartitions=1, availableMemory=104857600, maxSingleBufferSizeBytes=2147483647, pipelinedShuffle=false, isFinalMergeEnabled=true, numPartitions=1, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:28:42,762 INFO [TezTR-518813_1_4_0_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:28:42,762 INFO [TezTR-518813_1_4_0_0_0] exec.MapOperator: Initializing operator MAP[0] 2018-01-05T11:28:42,762 DEBUG [TezTR-518813_1_4_0_0_0] exec.MapOperator: Initialization Done 0 MAP 2018-01-05T11:28:42,762 DEBUG [TezTR-518813_1_4_0_0_0] exec.MapOperator: Operator 0 MAP initialized 2018-01-05T11:28:42,762 DEBUG [TezTR-518813_1_4_0_0_0] exec.MapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:28:42,762 DEBUG [TezTR-518813_1_4_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:42,763 DEBUG [TezTR-518813_1_4_0_0_0] exec.MapOperator: Adding alias src1 to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1 2018-01-05T11:28:42,763 DEBUG [TezTR-518813_1_4_0_0_0] exec.MapOperator: dump TS[0] struct 2018-01-05T11:28:42,763 INFO [TezTR-518813_1_4_0_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:28:42,763 INFO [TezTR-518813_1_4_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:42,763 DEBUG [TezTR-518813_1_4_0_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:28:42,763 INFO [TezTR-518813_1_4_0_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:28:42,763 DEBUG [TezTR-518813_1_4_0_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:28:42,763 DEBUG [TezTR-518813_1_4_0_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:28:42,763 DEBUG [TezTR-518813_1_4_0_0_0] exec.TableScanOperator: Initializing children of 0 TS 2018-01-05T11:28:42,763 DEBUG [TezTR-518813_1_4_0_0_0] exec.SelectOperator: Initializing child 1 SEL 2018-01-05T11:28:42,763 INFO [TezTR-518813_1_4_0_0_0] exec.SelectOperator: Initializing operator SEL[1] 2018-01-05T11:28:42,763 INFO [TezTR-518813_1_4_0_0_0] exec.SelectOperator: SELECT struct 2018-01-05T11:28:42,763 DEBUG [TezTR-518813_1_4_0_0_0] exec.SelectOperator: Initialization Done 1 SEL 2018-01-05T11:28:42,763 DEBUG [TezTR-518813_1_4_0_0_0] exec.SelectOperator: Operator 1 SEL initialized 2018-01-05T11:28:42,763 DEBUG [TezTR-518813_1_4_0_0_0] exec.SelectOperator: Initializing children of 1 SEL 2018-01-05T11:28:42,763 DEBUG [TezTR-518813_1_4_0_0_0] exec.GroupByOperator: Initializing child 2 GBY 2018-01-05T11:28:42,763 INFO [TezTR-518813_1_4_0_0_0] exec.GroupByOperator: Initializing operator GBY[2] 2018-01-05T11:28:42,763 DEBUG [TezTR-518813_1_4_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:42,763 DEBUG [TezTR-518813_1_4_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:42,763 INFO [TezTR-518813_1_4_0_0_0] exec.GroupByOperator: Max hash table memory: 93860656 bytes 2018-01-05T11:28:42,763 INFO [TezTR-518813_1_4_0_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:42,763 DEBUG [TezTR-518813_1_4_0_0_0] exec.GroupByOperator: Initialization Done 2 GBY 2018-01-05T11:28:42,763 DEBUG [TezTR-518813_1_4_0_0_0] exec.GroupByOperator: Operator 2 GBY initialized 2018-01-05T11:28:42,763 DEBUG [TezTR-518813_1_4_0_0_0] exec.GroupByOperator: Initializing children of 2 GBY 2018-01-05T11:28:42,763 DEBUG [TezTR-518813_1_4_0_0_0] exec.ReduceSinkOperator: Initializing child 3 RS 2018-01-05T11:28:42,763 INFO [TezTR-518813_1_4_0_0_0] exec.ReduceSinkOperator: Initializing operator RS[3] 2018-01-05T11:28:42,763 DEBUG [TezTR-518813_1_4_0_0_0] exec.ReduceSinkOperator: keys size is 0 2018-01-05T11:28:42,763 INFO [TezTR-518813_1_4_0_0_0] exec.ReduceSinkOperator: Using tag = -1 2018-01-05T11:28:42,763 DEBUG [TezTR-518813_1_4_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:42,763 DEBUG [TezTR-518813_1_4_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:42,763 DEBUG [TezTR-518813_1_4_0_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:28:42,763 DEBUG [TezTR-518813_1_4_0_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS 2018-01-05T11:28:42,763 DEBUG [TezTR-518813_1_4_0_0_0] exec.ReduceSinkOperator: Operator 3 RS initialized 2018-01-05T11:28:42,763 DEBUG [TezTR-518813_1_4_0_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS done is reset. 2018-01-05T11:28:42,763 DEBUG [TezTR-518813_1_4_0_0_0] exec.GroupByOperator: Initialization Done 2 GBY done is reset. 2018-01-05T11:28:42,763 DEBUG [TezTR-518813_1_4_0_0_0] exec.SelectOperator: Initialization Done 1 SEL done is reset. 2018-01-05T11:28:42,763 DEBUG [TezTR-518813_1_4_0_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:28:42,763 INFO [TezTR-518813_1_4_0_0_0] exec.OperatorUtils: Setting output collector: RS[3] --> Reducer 2 2018-01-05T11:28:42,763 DEBUG [TezTR-518813_1_4_0_0_0] log.PerfLogger: 2018-01-05T11:28:42,763 INFO [TezTR-518813_1_4_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_440_447381380713723563-1/djaiswal/_tez_scratch_dir/62646ef2-e992-4430-a80b-dc9e035b0238/map.xml 2018-01-05T11:28:42,764 DEBUG [TezTR-518813_1_4_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:42,764 DEBUG [TezTR-518813_1_4_0_0_0] exec.MapOperator: Processing alias(es) src1 for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1/kv3.txt 2018-01-05T11:28:42,764 INFO [TezTR-518813_1_4_0_0_0] exec.MapOperator: MAP[0]: records read - 1 2018-01-05T11:28:42,764 INFO [TezTR-518813_1_4_0_0_0] exec.MapOperator: MAP[0]: records read - 10 2018-01-05T11:28:42,765 DEBUG [TezTR-518813_1_4_0_0_0] log.PerfLogger: 2018-01-05T11:28:42,765 DEBUG [TezTR-518813_1_4_0_0_0] mr.ObjectCache: djaiswal_20180105112842_6905c06f-f509-4bd9-bd82-f6b7ffb13c54_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:28:42,765 DEBUG [TezTR-518813_1_4_0_0_0] exec.MapOperator: close called for operator MAP[0] 2018-01-05T11:28:42,765 INFO [TezTR-518813_1_4_0_0_0] exec.MapOperator: Closing operator MAP[0] 2018-01-05T11:28:42,765 INFO [TezTR-518813_1_4_0_0_0] exec.MapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:25, 2018-01-05T11:28:42,765 DEBUG [TezTR-518813_1_4_0_0_0] exec.MapOperator: Closing child = TS[0] 2018-01-05T11:28:42,765 DEBUG [TezTR-518813_1_4_0_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:28:42,765 DEBUG [TezTR-518813_1_4_0_0_0] exec.TableScanOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:42,765 INFO [TezTR-518813_1_4_0_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:28:42,765 DEBUG [TezTR-518813_1_4_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:42,765 DEBUG [TezTR-518813_1_4_0_0_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_446_7884164969442448570-1/-mr-10001 2018-01-05T11:28:42,765 INFO [TezTR-518813_1_4_0_0_0] exec.TableScanOperator: publishing : default.src1/ : {numRows=25, rawDataSize=191} 2018-01-05T11:28:42,788 DEBUG [TezTR-518813_1_4_0_0_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_446_7884164969442448570-1/-mr-10001/tmpstats-0_0 2018-01-05T11:28:42,788 DEBUG [TezTR-518813_1_4_0_0_0] fs.FSStatsPublisher: Writing stats in it : {default.src1/={numRows=25, rawDataSize=191}} 2018-01-05T11:28:42,788 DEBUG [TezTR-518813_1_4_0_0_0] exec.TableScanOperator: Closing child = SEL[1] 2018-01-05T11:28:42,788 DEBUG [TezTR-518813_1_4_0_0_0] exec.SelectOperator: close called for operator SEL[1] 2018-01-05T11:28:42,788 DEBUG [TezTR-518813_1_4_0_0_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:42,788 INFO [TezTR-518813_1_4_0_0_0] exec.SelectOperator: Closing operator SEL[1] 2018-01-05T11:28:42,788 DEBUG [TezTR-518813_1_4_0_0_0] exec.SelectOperator: Closing child = GBY[2] 2018-01-05T11:28:42,788 DEBUG [TezTR-518813_1_4_0_0_0] exec.GroupByOperator: close called for operator GBY[2] 2018-01-05T11:28:42,788 DEBUG [TezTR-518813_1_4_0_0_0] exec.GroupByOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:42,788 INFO [TezTR-518813_1_4_0_0_0] exec.GroupByOperator: Closing operator GBY[2] 2018-01-05T11:28:42,788 INFO [TezTR-518813_1_4_0_0_0] exec.GroupByOperator: Begin Hash Table flush: size = 1 2018-01-05T11:28:42,788 INFO [TezTR-518813_1_4_0_0_0] exec.ReduceSinkOperator: keys are [] num distributions: 0 2018-01-05T11:28:42,788 DEBUG [TezTR-518813_1_4_0_0_0] exec.GroupByOperator: Closing child = RS[3] 2018-01-05T11:28:42,788 DEBUG [TezTR-518813_1_4_0_0_0] exec.ReduceSinkOperator: close called for operator RS[3] 2018-01-05T11:28:42,788 DEBUG [TezTR-518813_1_4_0_0_0] exec.ReduceSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:42,788 INFO [TezTR-518813_1_4_0_0_0] exec.ReduceSinkOperator: Closing operator RS[3] 2018-01-05T11:28:42,788 INFO [TezTR-518813_1_4_0_0_0] exec.ReduceSinkOperator: RECORDS_OUT_INTERMEDIATE_Map_1:1, 2018-01-05T11:28:42,788 DEBUG [TezTR-518813_1_4_0_0_0] exec.ReduceSinkOperator: 3 Close done 2018-01-05T11:28:42,788 DEBUG [TezTR-518813_1_4_0_0_0] exec.GroupByOperator: 2 Close done 2018-01-05T11:28:42,788 DEBUG [TezTR-518813_1_4_0_0_0] exec.SelectOperator: 1 Close done 2018-01-05T11:28:42,788 DEBUG [TezTR-518813_1_4_0_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:28:42,788 DEBUG [TezTR-518813_1_4_0_0_0] exec.MapOperator: 0 Close done 2018-01-05T11:28:42,789 INFO [TezTR-518813_1_4_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_4_00_000000_0 2018-01-05T11:28:42,789 INFO [TezTR-518813_1_4_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: Waiting for all spills to complete : Pending : 0 2018-01-05T11:28:42,789 INFO [TezTR-518813_1_4_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: All spills complete 2018-01-05T11:28:42,800 DEBUG [TezTR-518813_1_4_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:42,800 INFO [TezTR-518813_1_4_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:42,801 INFO [TezTR-518813_1_4_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_4_00_000000_0, askedToStop=false 2018-01-05T11:28:42,801 INFO [TezTR-518813_1_4_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_4_00_000000_0, stopRequested=false 2018-01-05T11:28:42,801 INFO [TezTR-518813_1_4_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_4_00_000000_0: Counters: 14 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=25, INPUT_SPLIT_LENGTH_BYTES=216, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=191, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=228, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=25, RECORDS_OUT_INTERMEDIATE_Map_1=1]] 2018-01-05T11:28:42,801 DEBUG [TezTR-518813_1_4_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_4_0_0_0 thread id: 427 scheme: file bytesRead: 232 bytesWritten: 317 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:42,801 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_4_00_000000_0, isTaskDone=true 2018-01-05T11:28:42,802 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000005, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_4_00_000000_0, eventCount=4 } 2018-01-05T11:28:42,803 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_4][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_4_00_000000_0, creationTime=1515180522701, allocationTime=1515180522703, startTime=1515180522706, finishTime=1515180522803, timeTaken=97, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:42,804 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:42,804 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_4_00_000000_0 2018-01-05T11:28:42,804 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_4_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:42,804 INFO [Task-Executor-3] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_4_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:42,804 INFO [Task-Executor-3] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_4_00_000000_0 2018-01-05T11:28:42,804 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_4_00_000000_0; false 2018-01-05T11:28:42,804 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_4_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:42,804 INFO [Task-Executor-3] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000005=83 2018-01-05T11:28:42,804 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:42,804 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_4_00_000000_0: true 2018-01-05T11:28:42,804 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:42,805 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_4_00_000000_0 2018-01-05T11:28:42,805 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000005, QueryId=djaiswal_20180105112842_6905c06f-f509-4bd9-bd82-f6b7ffb13c54-4, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:28:42,805 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_4][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_4_00_000000, startTime=1515180522706, finishTime=1515180522804, timeTaken=98, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_4_00_000000_0, diagnostics=, counters=Counters: 20, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=232, FILE_BYTES_WRITTEN=317, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=25, INPUT_SPLIT_LENGTH_BYTES=216, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=191, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=228, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=25, RECORDS_OUT_INTERMEDIATE_Map_1=1 2018-01-05T11:28:42,805 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_4_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:42,805 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_4_00_000000_0 2018-01-05T11:28:42,805 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_4_00 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:42,805 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_4_00 [Map 1] 2018-01-05T11:28:42,806 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_4][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_4_00, initRequestedTime=1515180522669, initedTime=1515180522700, startRequestedTime=1515180522680, startedTime=1515180522700, finishTime=1515180522805, timeTaken=105, status=SUCCEEDED, diagnostics=, counters=Counters: 20, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=232, FILE_BYTES_WRITTEN=317, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=25, INPUT_SPLIT_LENGTH_BYTES=216, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=191, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=228, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=25, RECORDS_OUT_INTERMEDIATE_Map_1=1, vertexStats=firstTaskStartTime=1515180522706, firstTasksToStart=[ task_1515180518813_0001_4_00_000000 ], lastTaskFinishTime=1515180522803, lastTasksToFinish=[ task_1515180518813_0001_4_00_000000 ], minTaskDuration=97, maxTaskDuration=97, avgTaskDuration=97.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_4_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_4_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:42,806 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_4_00 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:42,806 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_4][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000005, stoppedTime=1515180522806, exitStatus=0 2018-01-05T11:28:42,806 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000005 exited with diagnostics set to Container completed. 2018-01-05T11:28:42,806 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_4_01 [Reducer 2] attempt: attempt_1515180518813_0001_4_00_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:28:42,806 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_4_00 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2 2018-01-05T11:28:42,806 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null 2018-01-05T11:28:42,806 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000005 2018-01-05T11:28:42,807 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: All source tasks completed. Ramping up 1 remaining tasks for vertex: Reducer 2 2018-01-05T11:28:42,807 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Scheduling 1 tasks for vertex: Reducer 2 with totalTasks: 1. 1 source tasks completed out of 1. MinSourceTaskCompletedFraction: 1.0 min: 0.05 max: 0.75 2018-01-05T11:28:42,807 INFO [App Shared Pool - #0] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Map 1, destinationVertexName=Reducer 2. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.UnorderedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:28:42,807 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_4_01 [Reducer 2] 2018-01-05T11:28:42,808 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_4][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_4_01_000000, scheduledTime=1515180522807, launchTime=1515180522807 2018-01-05T11:28:42,808 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_4_01 [Reducer 2] 2018-01-05T11:28:42,808 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_4_01_000000_0, priority=14, capability=, hosts=null 2018-01-05T11:28:42,808 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:42,808 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:42,808 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:42,808 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_4_01_000000_0 on hosts=null 2018-01-05T11:28:42,808 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_4_01_000000_0 on hosts=null 2018-01-05T11:28:42,808 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_4_01_000000_0 2018-01-05T11:28:42,808 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:42,808 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #6, task=TaskInfo{task=attempt_1515180518813_0001_4_01_000000_0, priority=14, startTime=0, containerId=null, uniqueId=5, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000006 2018-01-05T11:28:42,808 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_4_01_000000_0; false 2018-01-05T11:28:42,808 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_4_01_000000_0, priority=14, startTime=311543317, containerId=container_111101111_0001_00_000006assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=5, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:42,809 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000006 for node: 10.22.8.182:49818 2018-01-05T11:28:42,809 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000006, attempt_1515180518813_0001_4_01_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:42,809 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000006 for node: 10.22.8.182:49818 2018-01-05T11:28:42,810 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000006 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:42,810 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000006, launchTime=1515180522810 2018-01-05T11:28:42,810 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=4}, sourceName=Map 1, state=S_SUCCEEDED 2018-01-05T11:28:42,810 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_4_01_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000006] on NM: [10.22.8.182:49818] 2018-01-05T11:28:42,810 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:42,810 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:42,810 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_4][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_4_01_000000_0, startTime=1515180522810, containerId=container_111101111_0001_00_000006, nodeId=10.22.8.182:49818 2018-01-05T11:28:42,811 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_4_01_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112842_6905c06f-f509-4bd9-bd82-f6b7ffb13c54, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000006, dagName=ANALYZE TABLE src1 COMPUTE STATI...key,value (Stage-0), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.UnorderedKVInput,1}}, Outputs={{out_Reducer 2,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=2, completedTaskCount=1, dagStartTime=1515180522669, firstAttemptStartTime=1515180522808, currentAttemptStartTime=1515180522809} 2018-01-05T11:28:42,811 INFO [IPC Server handler 2 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000006, QueryId=djaiswal_20180105112842_6905c06f-f509-4bd9-bd82-f6b7ffb13c54-4, DagId=4, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, SubmitTime=1515180522811 2018-01-05T11:28:42,811 DEBUG [IPC Server handler 2 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=4} with the ShuffleHandler 2018-01-05T11:28:42,811 DEBUG [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/4] 2018-01-05T11:28:42,811 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:42,811 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_4_01_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:42,811 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_4_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180522808, dagStartTime=1515180522669, withinDagPriority=14, vertexParallelism= 1, selfAndUpstreamParallelism= 2, selfAndUpstreamComplete= 1}] 2018-01-05T11:28:42,811 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_4_01_000000_0 : ACCEPTED 2018-01-05T11:28:42,811 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_4_01_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:42,811 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_4_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180522808, dagStartTime=1515180522669, withinDagPriority=14, vertexParallelism= 1, selfAndUpstreamParallelism= 2, selfAndUpstreamComplete= 1} 2018-01-05T11:28:42,811 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_4_01_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:42,811 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_4_01_000000_0 to pre-emption queue 2018-01-05T11:28:42,812 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_4_01_000000_0: true 2018-01-05T11:28:42,812 INFO [TaskCommunicator # 0] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_4_01_000000_0 2018-01-05T11:28:42,812 INFO [TaskCommunicator # 0] GuaranteedTasks: Received response for attempt_1515180518813_0001_4_01_000000_0, true 2018-01-05T11:28:42,812 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: taskOwner hashCode:1280491721 2018-01-05T11:28:42,812 INFO [Task-Executor-3] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : ANALYZE TABLE src1 COMPUTE STATI...key,value (Stage-0), VertexName: Reducer 2, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_4_01_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:28:42,812 INFO [Task-Executor-3] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=954728448, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:42,812 INFO [Task-Executor-3] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_4_01_000000_0 vertexName: Reducer 2 2018-01-05T11:28:42,813 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000006, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_4_01_000000_0, eventCount=1 } 2018-01-05T11:28:42,813 DEBUG [TezTR-518813_1_4_1_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_4 queryId: djaiswal_20180105112842_6905c06f-f509-4bd9-bd82-f6b7ffb13c54 instanceType: TaskRunner2Callable 2018-01-05T11:28:42,813 INFO [TezTR-518813_1_4_1_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_4_01_000000_0 2018-01-05T11:28:42,813 DEBUG [TezTR-518813_1_4_1_0_0] log.PerfLogger: 2018-01-05T11:28:42,814 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:42,815 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:42,818 INFO [IPC Server handler 6 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_4_01_000000_0 1 events [0,1) total 1 vertex_1515180518813_0001_4_01 [Reducer 2] 2018-01-05T11:28:42,819 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:28:42,819 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_4_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:28:42,820 INFO [I/O Setup 1 Initialize: {out_Reducer 2}] output.MROutput: out_Reducer 2: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:28:42,820 DEBUG [TezTR-518813_1_4_1_0_0] log.PerfLogger: 2018-01-05T11:28:42,820 INFO [TezTR-518813_1_4_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:42,820 INFO [TezTR-518813_1_4_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:42,820 INFO [TezTR-518813_1_4_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:42,820 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:42,820 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:42,820 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:42,820 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:42,820 INFO [TezTR-518813_1_4_1_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:42,820 INFO [TezTR-518813_1_4_1_0_0] resources.MemoryDistributor: InitialRequests=[out_Reducer 2:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.UnorderedKVInput] 2018-01-05T11:28:42,820 INFO [TezTR-518813_1_4_1_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:42,820 INFO [TezTR-518813_1_4_1_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:42,820 INFO [TezTR-518813_1_4_1_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 131404920, TotalRequestedScaled: 6.570246E7, TotalJVMHeap: 954728448, TotalAvailable: 639668060, TotalRequested/TotalJVMHeap:0.14 2018-01-05T11:28:42,820 INFO [TezTR-518813_1_4_1_0_0] resources.MemoryDistributor: Allocations=[out_Reducer 2:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [Map 1:org.apache.tez.runtime.library.input.UnorderedKVInput:INPUT:131404920:131404920] 2018-01-05T11:28:42,820 INFO [TezTR-518813_1_4_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:42,820 INFO [TezTR-518813_1_4_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 0 2018-01-05T11:28:42,820 INFO [TezTR-518813_1_4_1_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:42,821 INFO [TezTR-518813_1_4_1_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_4_01_000000_0 2018-01-05T11:28:42,821 DEBUG [TezTR-518813_1_4_1_0_0] log.PerfLogger: 2018-01-05T11:28:42,821 DEBUG [TezTR-518813_1_4_1_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_4_01_000000_0_10016 2018-01-05T11:28:42,821 DEBUG [TezTR-518813_1_4_1_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:42,821 DEBUG [TezTR-518813_1_4_1_0_0] mr.ObjectCache: Creating djaiswal_20180105112842_6905c06f-f509-4bd9-bd82-f6b7ffb13c54_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:28:42,821 INFO [TezTR-518813_1_4_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_440_447381380713723563-1/djaiswal/_tez_scratch_dir/1810b6da-3c7e-4e8a-8c64-90d03254fa67/reduce.xml 2018-01-05T11:28:42,821 DEBUG [TezTR-518813_1_4_1_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_440_447381380713723563-1/djaiswal/_tez_scratch_dir/1810b6da-3c7e-4e8a-8c64-90d03254fa67/reduce.xml 2018-01-05T11:28:42,821 DEBUG [TezTR-518813_1_4_1_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_440_447381380713723563-1/djaiswal/_tez_scratch_dir/1810b6da-3c7e-4e8a-8c64-90d03254fa67/reduce.xml 2018-01-05T11:28:42,821 DEBUG [TezTR-518813_1_4_1_0_0] log.PerfLogger: 2018-01-05T11:28:42,821 INFO [TezTR-518813_1_4_1_0_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:28:42,821 DEBUG [TezTR-518813_1_4_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:42,822 DEBUG [TezTR-518813_1_4_1_0_0] log.PerfLogger: 2018-01-05T11:28:42,823 INFO [TezTR-518813_1_4_1_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 2.01KB 2018-01-05T11:28:42,823 DEBUG [TezTR-518813_1_4_1_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:42,823 DEBUG [TezTR-518813_1_4_1_0_0] log.PerfLogger: 2018-01-05T11:28:42,823 DEBUG [TezTR-518813_1_4_1_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:42,823 DEBUG [TezTR-518813_1_4_1_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:42,823 DEBUG [TezTR-518813_1_4_1_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:42,823 INFO [TezTR-518813_1_4_1_0_0] impl.SimpleFetchedInputAllocator: Map_1: RequestedMemory=131404920, AssignedMemory=131404920, maxSingleShuffleLimit=32851230 2018-01-05T11:28:42,824 INFO [TezTR-518813_1_4_1_0_0] impl.ShuffleManager: Map_1: numInputs=1, compressionCodec=NoCompressionCodec, numFetchers=1, ifileBufferSize=4096, ifileReadAheadEnabled=true, ifileReadAheadLength=4194304, localDiskFetchEnabled=false, sharedFetchEnabled=false, keepAlive=false, keepAliveMaxConnections=20, connectionTimeout=20000, readTimeout=180000, bufferSize=8192, bufferSize=8192, maxTaskOutputAtOnce=20 2018-01-05T11:28:42,825 INFO [TezTR-518813_1_4_1_0_0] impl.ShuffleInputEventHandlerImpl: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:28:42,825 INFO [TezTR-518813_1_4_1_0_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:28:42,829 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=4&reduce=0&map=attempt_1515180518813_0001_4_00_000000_0_10015 mapId: [attempt_1515180518813_0001_4_00_000000_0_10015] reduceId: [0] jobId: [job_1515180518813_0001] dagId: [4] keepAlive: false 2018-01-05T11:28:42,829 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=4&reduce=0&map=attempt_1515180518813_0001_4_00_000000_0_10015; hash=...8JkhYFYRmfeuM 2018-01-05T11:28:42,829 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=4&reduce=0&map=attempt_1515180518813_0001_4_00_000000_0_10015;reply=AUoQP18a2owQU 2018-01-05T11:28:42,829 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=4, user='djaiswal', attemptId='attempt_1515180518813_0001_4_00_000000_0_10015'} via loader 2018-01-05T11:28:42,829 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=4, user='djaiswal', attemptId='attempt_1515180518813_0001_4_00_000000_0_10015'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:28:42,829 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_4_00_000000_0_10015 not found 2018-01-05T11:28:42,829 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_4_00_000000_0_10015,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/4/output/attempt_1515180518813_0001_4_00_000000_0_10015/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/4/output/attempt_1515180518813_0001_4_00_000000_0_10015/file.out.index 2018-01-05T11:28:42,830 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Content Length in shuffle : 256 2018-01-05T11:28:42,830 INFO [Fetcher_B {Map_1} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=4&reduce=0&map=attempt_1515180518813_0001_4_00_000000_0_10015 sent hash and receievd reply 0 ms 2018-01-05T11:28:42,830 INFO [Fetcher_B {Map_1} #0] ShuffleManager.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_4_00_000000_0_10015} to MEMORY, csize=204, dsize=200, EndTime=1515180522830, TimeTaken=0, Rate=0.00 MB/s 2018-01-05T11:28:42,830 INFO [Fetcher_B {Map_1} #0] impl.ShuffleManager: All inputs fetched for input vertex : Map 1 2018-01-05T11:28:42,830 INFO [TezTR-518813_1_4_1_0_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:28:42,830 INFO [TezTR-518813_1_4_1_0_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:42,830 DEBUG [TezTR-518813_1_4_1_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:28:42,830 DEBUG [TezTR-518813_1_4_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:42,830 INFO [Fetcher_B {Map_1} #0] impl.ShuffleManager: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:28:42,830 DEBUG [TezTR-518813_1_4_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:42,830 INFO [ShuffleRunner {Map_1}] impl.ShuffleManager: Map_1: Shutting down FetchScheduler, Was Interrupted: false 2018-01-05T11:28:42,830 INFO [ShuffleRunner {Map_1}] impl.ShuffleManager: Map_1: Scheduler thread completed 2018-01-05T11:28:42,830 DEBUG [TezTR-518813_1_4_1_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:28:42,831 DEBUG [TezTR-518813_1_4_1_0_0] log.PerfLogger: 2018-01-05T11:28:42,831 INFO [TezTR-518813_1_4_1_0_0] exec.GroupByOperator: Initializing operator GBY[4] 2018-01-05T11:28:42,831 DEBUG [TezTR-518813_1_4_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:42,831 DEBUG [TezTR-518813_1_4_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:42,831 INFO [TezTR-518813_1_4_1_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:42,831 DEBUG [TezTR-518813_1_4_1_0_0] exec.GroupByOperator: Initialization Done 4 GBY 2018-01-05T11:28:42,831 DEBUG [TezTR-518813_1_4_1_0_0] exec.GroupByOperator: Operator 4 GBY initialized 2018-01-05T11:28:42,831 DEBUG [TezTR-518813_1_4_1_0_0] exec.GroupByOperator: Initializing children of 4 GBY 2018-01-05T11:28:42,831 DEBUG [TezTR-518813_1_4_1_0_0] exec.FileSinkOperator: Initializing child 6 FS 2018-01-05T11:28:42,831 INFO [TezTR-518813_1_4_1_0_0] exec.FileSinkOperator: Initializing operator FS[6] 2018-01-05T11:28:42,831 DEBUG [TezTR-518813_1_4_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:42,831 INFO [TezTR-518813_1_4_1_0_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@3a5b0746]:[_col0, _col1]:[struct, struct]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@5ed0d8d8 2018-01-05T11:28:42,831 DEBUG [TezTR-518813_1_4_1_0_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:28:42,831 DEBUG [TezTR-518813_1_4_1_0_0] exec.FileSinkOperator: Initialization Done 6 FS 2018-01-05T11:28:42,831 DEBUG [TezTR-518813_1_4_1_0_0] exec.FileSinkOperator: Operator 6 FS initialized 2018-01-05T11:28:42,831 DEBUG [TezTR-518813_1_4_1_0_0] exec.FileSinkOperator: Initialization Done 6 FS done is reset. 2018-01-05T11:28:42,831 DEBUG [TezTR-518813_1_4_1_0_0] exec.GroupByOperator: Initialization Done 4 GBY done is reset. 2018-01-05T11:28:42,831 INFO [TezTR-518813_1_4_1_0_0] tez.ReduceRecordProcessor: Id =4 Id =6 <\Children> Id = 4 null<\Parent> <\FS> <\Children> <\Parent> <\GBY> 2018-01-05T11:28:42,831 DEBUG [TezTR-518813_1_4_1_0_0] log.PerfLogger: 2018-01-05T11:28:42,831 INFO [TezTR-518813_1_4_1_0_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 2 2018-01-05T11:28:42,831 DEBUG [TezTR-518813_1_4_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:28:42,831 DEBUG [TezTR-518813_1_4_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:28:42,831 DEBUG [TezTR-518813_1_4_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:28:42,831 DEBUG [TezTR-518813_1_4_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:28:42,831 DEBUG [TezTR-518813_1_4_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:28:42,831 DEBUG [TezTR-518813_1_4_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:28:42,832 INFO [TezTR-518813_1_4_1_0_0] readers.UnorderedKVReader: Num Records read: 1 2018-01-05T11:28:42,832 DEBUG [TezTR-518813_1_4_1_0_0] log.PerfLogger: 2018-01-05T11:28:42,832 DEBUG [TezTR-518813_1_4_1_0_0] mr.ObjectCache: djaiswal_20180105112842_6905c06f-f509-4bd9-bd82-f6b7ffb13c54_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:28:42,832 DEBUG [TezTR-518813_1_4_1_0_0] exec.GroupByOperator: close called for operator GBY[4] 2018-01-05T11:28:42,832 INFO [TezTR-518813_1_4_1_0_0] exec.GroupByOperator: Closing operator GBY[4] 2018-01-05T11:28:42,832 INFO [TezTR-518813_1_4_1_0_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_446_7884164969442448570-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-42_446_7884164969442448570-1/_tmp.-ext-10002/000000_0 2018-01-05T11:28:42,832 INFO [TezTR-518813_1_4_1_0_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_446_7884164969442448570-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-42_446_7884164969442448570-1/_task_tmp.-ext-10002/_tmp.000000_0 2018-01-05T11:28:42,832 INFO [TezTR-518813_1_4_1_0_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_446_7884164969442448570-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-42_446_7884164969442448570-1/_tmp.-ext-10002/000000_0 2018-01-05T11:28:42,869 INFO [TezTR-518813_1_4_1_0_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:28:42,869 DEBUG [TezTR-518813_1_4_1_0_0] exec.GroupByOperator: Closing child = FS[6] 2018-01-05T11:28:42,869 DEBUG [TezTR-518813_1_4_1_0_0] exec.FileSinkOperator: close called for operator FS[6] 2018-01-05T11:28:42,869 DEBUG [TezTR-518813_1_4_1_0_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:42,869 INFO [TezTR-518813_1_4_1_0_0] exec.FileSinkOperator: Closing operator FS[6] 2018-01-05T11:28:42,869 INFO [TezTR-518813_1_4_1_0_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:28:42,869 INFO [TezTR-518813_1_4_1_0_0] exec.FileSinkOperator: RECORDS_OUT_0:1, 2018-01-05T11:28:42,869 DEBUG [TezTR-518813_1_4_1_0_0] exec.FileSinkOperator: 6 Close done 2018-01-05T11:28:42,869 DEBUG [TezTR-518813_1_4_1_0_0] exec.GroupByOperator: 4 Close done 2018-01-05T11:28:42,870 INFO [TezTR-518813_1_4_1_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_4_01_000000_0 2018-01-05T11:28:42,870 INFO [TezTR-518813_1_4_1_0_0] impl.ShuffleInputEventHandlerImpl: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:28:42,870 INFO [TezTR-518813_1_4_1_0_0] impl.ShuffleManager: Shutting down pending fetchers on sourceMap_1: 0 2018-01-05T11:28:42,870 INFO [TezTR-518813_1_4_1_0_0] output.MROutput: out_Reducer 2 closed 2018-01-05T11:28:42,870 DEBUG [TezTR-518813_1_4_1_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:42,870 INFO [TezTR-518813_1_4_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:42,870 INFO [TezTR-518813_1_4_1_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_4_01_000000_0, askedToStop=false 2018-01-05T11:28:42,870 INFO [TezTR-518813_1_4_1_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_4_01_000000_0, stopRequested=false 2018-01-05T11:28:42,870 INFO [TezTR-518813_1_4_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_4_01_000000_0: Counters: 14 [[org.apache.tez.common.counters.TaskCounter NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=204, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1][HIVE CREATED_FILES=1, RECORDS_OUT_0=1]] 2018-01-05T11:28:42,871 DEBUG [TezTR-518813_1_4_1_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_4_1_0_0 thread id: 440 scheme: file bytesRead: 0 bytesWritten: 360 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:42,871 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_4_01_000000_0, isTaskDone=true 2018-01-05T11:28:42,871 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000006, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_4_01_000000_0, eventCount=2 } 2018-01-05T11:28:42,872 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_4][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_4_01_000000_0, creationTime=1515180522808, allocationTime=1515180522809, startTime=1515180522810, finishTime=1515180522872, timeTaken=62, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:42,873 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_4_01_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:42,873 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:28:42,873 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_4_01_000000_0 2018-01-05T11:28:42,873 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_4_01_000000_0; false 2018-01-05T11:28:42,873 INFO [Task-Executor-3] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_4_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:42,873 INFO [Task-Executor-3] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_4_01_000000_0 2018-01-05T11:28:42,873 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_4_01_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:42,873 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:42,873 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:42,874 INFO [Task-Executor-3] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000006=61 2018-01-05T11:28:42,874 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_4_01_000000_0: true 2018-01-05T11:28:42,874 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_4][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_4_01_000000, startTime=1515180522810, finishTime=1515180522873, timeTaken=63, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_4_01_000000_0, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=360, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=204, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1 2018-01-05T11:28:42,874 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_4_01 [Reducer 2], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:42,874 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_4_01 [Reducer 2] 2018-01-05T11:28:42,875 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_4][Event:VERTEX_FINISHED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_4_01, initRequestedTime=1515180522680, initedTime=1515180522686, startRequestedTime=1515180522700, startedTime=1515180522701, finishTime=1515180522874, timeTaken=173, status=SUCCEEDED, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=360, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=204, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1, vertexStats=firstTaskStartTime=1515180522810, firstTasksToStart=[ task_1515180518813_0001_4_01_000000 ], lastTaskFinishTime=1515180522872, lastTasksToFinish=[ task_1515180518813_0001_4_01_000000 ], minTaskDuration=62, maxTaskDuration=62, avgTaskDuration=62.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_4_01_000000 ], longestDurationTasks=[ task_1515180518813_0001_4_01_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:42,875 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_4_01 [Reducer 2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:42,875 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_4][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000006, stoppedTime=1515180522875, exitStatus=0 2018-01-05T11:28:42,875 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000006 exited with diagnostics set to Container completed. 2018-01-05T11:28:42,875 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_4_01 [Reducer 2] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2 2018-01-05T11:28:42,875 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null 2018-01-05T11:28:42,875 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_4_00 [Map 1] 2018-01-05T11:28:42,875 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_4_01 [Reducer 2] 2018-01-05T11:28:42,876 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_4, queueSize=0 2018-01-05T11:28:42,877 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_4][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_4, startTime=1515180522669, finishTime=1515180522875, timeTaken=206, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, OTHER_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=331, AM_GC_TIME_MILLIS=0, File System Counters, FILE_BYTES_READ=232, FILE_BYTES_WRITTEN=677, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=26, INPUT_SPLIT_LENGTH_BYTES=216, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=191, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=228, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=204, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1, HIVE, CREATED_FILES=1, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=25, RECORDS_OUT_0=1, RECORDS_OUT_INTERMEDIATE_Map_1=1 2018-01-05T11:28:42,877 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_4 finished with state: SUCCEEDED 2018-01-05T11:28:42,877 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_4 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:28:42,877 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000006 2018-01-05T11:28:42,877 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_4, dagState=SUCCEEDED 2018-01-05T11:28:42,877 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_4_post 2018-01-05T11:28:42,877 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:42,877 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:28:42,877 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:28:42,877 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 4 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=2, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=2,NumRequestsWithoutLocation=2, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=2} 2018-01-05T11:28:42,877 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:28:42,877 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=ANALYZE TABLE src1 COMPUTE STATI...key,value (Stage-0), with id=dag_1515180518813_0001_4 2018-01-05T11:28:42,878 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 4, to 10.22.8.182:49818 2018-01-05T11:28:42,878 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=2, Remaining=0 2018-01-05T11:28:42,878 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=ANALYZE TABLE src1 COMPUTE STATI...key,value (Stage-0), with id=dag_1515180518813_0001_4 2018-01-05T11:28:42,878 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=4} 2018-01-05T11:28:42,878 INFO [IPC Server handler 4 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=4}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:28:42,878 INFO [IPC Server handler 4 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/4 after 300 seconds 2018-01-05T11:28:42,878 ERROR [IPC Server handler 4 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112842_6905c06f-f509-4bd9-bd82-f6b7ffb13c54, dag_1515180518813_0001_4 2018-01-05T11:28:42,878 INFO [IPC Server handler 4 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112842_6905c06f-f509-4bd9-bd82-f6b7ffb13c54 2018-01-05T11:28:42,878 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=4} = 1 2018-01-05T11:28:42,878 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Issuing killFragment for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=4} attempt_1515180518813_0001_4_01_000000_0 2018-01-05T11:28:42,878 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Removing attempt_1515180518813_0001_4_01_000000_0 from preemptionQueue 2018-01-05T11:28:42,878 INFO [IPC Server handler 4 on 49818] impl.TaskRunnerCallable: Ignoring kill request for task attempt_1515180518813_0001_4_01_000000_0 since it's already complete 2018-01-05T11:28:42,879 DEBUG [IPC Server handler 4 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=4} 2018-01-05T11:28:42,879 DEBUG [IPC Server handler 4 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:28:42,879 DEBUG [IPC Server handler 4 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=4} 2018-01-05T11:28:42,879 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,879 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,879 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,879 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,879 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 Reducer 2: 1/1 2018-01-05T11:28:42,879 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,882 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,882 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: TaskId for 000000_0 = 000000 2018-01-05T11:28:42,882 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,882 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,883 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,883 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-2:STATS] in serial mode 2018-01-05T11:28:42,884 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@48614fb3, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:42,884 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:42,884 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,884 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:42,884 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:42,884 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:42,884 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:42,884 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,884 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:42,884 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:42,884 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_446_7884164969442448570-1/-mr-10001 2018-01-05T11:28:42,884 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_446_7884164969442448570-1/-mr-10001 2018-01-05T11:28:42,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.src1/ numRows 2018-01-05T11:28:42,886 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.src1/ numRows 25 2018-01-05T11:28:42,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.src1/ rawDataSize 2018-01-05T11:28:42,886 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.src1/ rawDataSize 191 2018-01-05T11:28:42,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,887 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:42,887 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:42,887 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:42,887 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:42,887 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,887 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,887 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,888 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,889 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,889 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:42,889 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:42,889 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,889 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:42,889 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,889 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=src1 newtbl=src1 2018-01-05T11:28:42,889 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=src1 newtbl=src1 2018-01-05T11:28:42,897 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,904 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,905 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,905 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.src1 stats: [numFiles=1, numRows=25, totalSize=216, rawDataSize=191] 2018-01-05T11:28:42,905 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: About to delete stats tmp dir :file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_446_7884164969442448570-1/-mr-10001 2018-01-05T11:28:42,908 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000005, QueryId=djaiswal_20180105112842_6905c06f-f509-4bd9-bd82-f6b7ffb13c54-4, DagId=4, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-3, Succeeded=true, StartTime=1515180522720, EndTime=1515180522908 2018-01-05T11:28:42,908 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_4_01_000000_0 2018-01-05T11:28:42,908 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_4_01_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:42,908 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_4_01_000000_0 2018-01-05T11:28:42,908 INFO [ExecutionCompletionThread #0] impl.QueryTracker: Ignoring fragmentComplete message for unknown query: QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=4} 2018-01-05T11:28:42,922 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Time taken to get FileStatuses: 15 2018-01-05T11:28:42,922 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:28:42,922 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 15 2018-01-05T11:28:42,922 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FetchOperator: Creating fetchTask with deserializer typeinfo: struct<_col0:struct,_col1:struct> 2018-01-05T11:28:42,922 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FetchOperator: deserializer properties: table properties: {columns=_col0,_col1, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct:struct, escape.delim=\} partition properties: {columns=_col0,_col1, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct:struct, escape.delim=\} 2018-01-05T11:28:42,923 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,923 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: write_column_statistics: db=default table=src1 2018-01-05T11:28:42,923 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=write_column_statistics: db=default table=src1 2018-01-05T11:28:42,928 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=src1 tableName=default colName=key 2018-01-05T11:28:42,929 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=src1 tableName=default colName=value 2018-01-05T11:28:42,932 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,932 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,933 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,933 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,933 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: ANALYZE TABLE src1 COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:42,933 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:28:42,933 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@src1 2018-01-05T11:28:42,933 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@src1 2018-01-05T11:28:42,933 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_446_7884164969442448570-1/-mr-10000 2018-01-05T11:28:42,933 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,933 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,933 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:42,933 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=16, setPartitionColumnStatistics_(SetPartitionsStatsRequest, )=10} 2018-01-05T11:28:42,933 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112842_6905c06f-f509-4bd9-bd82-f6b7ffb13c54); Time taken: 0.415 seconds 2018-01-05T11:28:42,933 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:42,933 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,933 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:42,933 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,933 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query ANALYZE TABLE src1 COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:42,933 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-42_440_447381380713723563-1 2018-01-05T11:28:42,934 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.493 seconds 2018-01-05T11:28:42,934 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:42,934 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:42,934 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:42,934 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:42,934 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,934 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:42,934 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,934 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,934 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,934 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: -- -- Table src_json -- DROP TABLE IF EXISTS src_json 2018-01-05T11:28:42,934 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112842_7d886717-87c4-47a0-b8c7-4c9e9bf6fb4b): -- -- Table src_json -- DROP TABLE IF EXISTS src_json 2018-01-05T11:28:42,934 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,934 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: -- -- Table src_json -- DROP TABLE IF EXISTS src_json 2018-01-05T11:28:42,934 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:42,934 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,934 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,935 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,935 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,935 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:42,935 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,935 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_json 2018-01-05T11:28:42,935 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_json 2018-01-05T11:28:42,935 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,935 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,935 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:42,935 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,935 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:42,935 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,935 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:42,935 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=1, flushCache_()=0} 2018-01-05T11:28:42,936 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112842_7d886717-87c4-47a0-b8c7-4c9e9bf6fb4b); Time taken: 0.001 seconds 2018-01-05T11:28:42,936 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:42,936 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,936 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112842_7d886717-87c4-47a0-b8c7-4c9e9bf6fb4b): -- -- Table src_json -- DROP TABLE IF EXISTS src_json 2018-01-05T11:28:42,937 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,937 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: -- -- Table src_json -- DROP TABLE IF EXISTS src_json 2018-01-05T11:28:42,937 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:42,937 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,937 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,937 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,937 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,937 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:42,937 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,937 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_json 2018-01-05T11:28:42,937 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_json 2018-01-05T11:28:42,937 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,937 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,937 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table src_json not found: default.src_json table not found 2018-01-05T11:28:42,937 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,938 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_json 2018-01-05T11:28:42,938 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_json 2018-01-05T11:28:42,938 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,938 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,938 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,938 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,938 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: -- -- Table src_json -- DROP TABLE IF EXISTS src_json 2018-01-05T11:28:42,938 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:42,938 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,938 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,938 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:42,938 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=1} 2018-01-05T11:28:42,938 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112842_7d886717-87c4-47a0-b8c7-4c9e9bf6fb4b); Time taken: 0.002 seconds 2018-01-05T11:28:42,938 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:42,938 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,938 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:42,938 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,938 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query -- -- Table src_json -- DROP TABLE IF EXISTS src_json 2018-01-05T11:28:42,938 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.004 seconds 2018-01-05T11:28:42,938 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:42,938 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:42,938 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:42,938 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:42,939 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,939 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:42,939 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,939 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,939 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,939 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: CREATE TABLE src_json (json STRING COMMENT 'default') STORED AS TEXTFILE 2018-01-05T11:28:42,939 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112842_6f95f80b-eb70-4ce3-99a3-62faa440a8d1): CREATE TABLE src_json (json STRING COMMENT 'default') STORED AS TEXTFILE 2018-01-05T11:28:42,939 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,939 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: CREATE TABLE src_json (json STRING COMMENT 'default') STORED AS TEXTFILE 2018-01-05T11:28:42,939 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:42,939 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,939 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,939 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,939 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,939 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:42,939 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:28:42,939 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Creating table default.src_json position=13 2018-01-05T11:28:42,939 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,939 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_database: default 2018-01-05T11:28:42,939 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_database: default 2018-01-05T11:28:42,940 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:42,940 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,940 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,940 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:42,940 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: validation start 2018-01-05T11:28:42,940 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:28:42,940 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Not a partition. 2018-01-05T11:28:42,940 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,940 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:42,940 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,940 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:42,940 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getDatabase_(String, )=1, flushCache_()=0} 2018-01-05T11:28:42,940 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112842_6f95f80b-eb70-4ce3-99a3-62faa440a8d1); Time taken: 0.001 seconds 2018-01-05T11:28:42,940 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:42,940 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,940 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112842_6f95f80b-eb70-4ce3-99a3-62faa440a8d1): CREATE TABLE src_json (json STRING COMMENT 'default') STORED AS TEXTFILE 2018-01-05T11:28:42,941 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,941 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: CREATE TABLE src_json (json STRING COMMENT 'default') STORED AS TEXTFILE 2018-01-05T11:28:42,941 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: CREATETABLE 2018-01-05T11:28:42,941 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: database:default 2018-01-05T11:28:42,941 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@src_json 2018-01-05T11:28:42,941 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,941 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,941 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,941 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,941 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:42,942 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Found class for org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe 2018-01-05T11:28:42,942 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: creating table default.src_json on file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json 2018-01-05T11:28:42,942 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,942 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: create_table: Table(tableName:src_json, dbName:default, owner:hive_test_user, createTime:1515180522, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:json, type:string, comment:default)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"json":"true"}}, numFiles=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:42,942 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=create_table: Table(tableName:src_json, dbName:default, owner:hive_test_user, createTime:1515180522, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:json, type:string, comment:default)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"json":"true"}}, numFiles=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:42,943 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hive-site.xml 2018-01-05T11:28:42,944 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hivemetastore-site.xml 2018-01-05T11:28:42,944 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Unable to find config file metastore-site.xml 2018-01-05T11:28:42,944 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file null 2018-01-05T11:28:42,953 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:42,953 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: Location: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json specified for non-external table:src_json 2018-01-05T11:28:42,953 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json 2018-01-05T11:28:42,975 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,979 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,980 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,980 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@src_json) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@src_json) Type=TABLE WriteType=DDL_NO_LOCK is present 2018-01-05T11:28:42,980 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,980 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,980 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: CREATE TABLE src_json (json STRING COMMENT 'default') STORED AS TEXTFILE 2018-01-05T11:28:42,980 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: CREATETABLE 2018-01-05T11:28:42,980 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: database:default 2018-01-05T11:28:42,980 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@src_json 2018-01-05T11:28:42,980 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,980 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,981 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:42,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {createTable_(Table, )=38, isCompatibleWith_(Configuration, )=1} 2018-01-05T11:28:42,981 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112842_6f95f80b-eb70-4ce3-99a3-62faa440a8d1); Time taken: 0.04 seconds 2018-01-05T11:28:42,981 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:42,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,981 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:42,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query CREATE TABLE src_json (json STRING COMMENT 'default') STORED AS TEXTFILE 2018-01-05T11:28:42,981 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.042 seconds 2018-01-05T11:28:42,981 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:42,981 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:42,981 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:42,981 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:42,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:42,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/json.txt" INTO TABLE src_json 2018-01-05T11:28:42,981 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112842_e70c09f1-2c3a-4dad-98c4-f1fccac707a3): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/json.txt" INTO TABLE src_json 2018-01-05T11:28:42,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/json.txt" INTO TABLE src_json 2018-01-05T11:28:42,982 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:42,982 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,982 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,982 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,982 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,982 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:42,982 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.LoadSemanticAnalyzer: file@null@/Users/djaiswal/hive/hive/data/files/json.txt 2018-01-05T11:28:42,982 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,982 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_json 2018-01-05T11:28:42,982 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_json 2018-01-05T11:28:42,987 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,988 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,990 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:42,990 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,990 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:42,990 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,990 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:42,990 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=6, flushCache_()=0} 2018-01-05T11:28:42,990 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112842_e70c09f1-2c3a-4dad-98c4-f1fccac707a3); Time taken: 0.009 seconds 2018-01-05T11:28:42,990 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:42,990 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,990 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112842_e70c09f1-2c3a-4dad-98c4-f1fccac707a3): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/json.txt" INTO TABLE src_json 2018-01-05T11:28:42,991 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,992 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/json.txt" INTO TABLE src_json 2018-01-05T11:28:42,992 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: LOAD 2018-01-05T11:28:42,992 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/json.txt 2018-01-05T11:28:42,992 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@src_json 2018-01-05T11:28:42,992 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,992 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,992 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,992 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:42,992 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MOVE] in serial mode 2018-01-05T11:28:42,992 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@46ae897c, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:42,992 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:42,992 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,992 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:42,992 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:42,992 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:42,992 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:42,992 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,992 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:42,992 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Loading data to table default.src_json from file:/Users/djaiswal/hive/hive/data/files/json.txt 2018-01-05T11:28:42,993 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,993 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:42,993 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:42,993 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:42,993 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:42,993 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,993 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,993 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,995 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,995 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:42,995 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:42,995 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:42,995 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,995 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:42,995 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:42,995 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_json 2018-01-05T11:28:42,995 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_json 2018-01-05T11:28:43,001 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,001 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,003 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,003 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_json 2018-01-05T11:28:43,003 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_json 2018-01-05T11:28:43,008 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,009 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,009 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] FileOperations: moving file:/Users/djaiswal/hive/hive/data/files/json.txt to file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json (replace = KEEP_EXISTING) 2018-01-05T11:28:43,010 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:43,010 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:43,011 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000006, QueryId=djaiswal_20180105112842_6905c06f-f509-4bd9-bd82-f6b7ffb13c54-4, DagId=4, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-3, Succeeded=true, StartTime=1515180522812, EndTime=1515180523011 2018-01-05T11:28:43,035 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Moved src: file:/Users/djaiswal/hive/hive/data/files/json.txt, to dest: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json/json.txt 2018-01-05T11:28:43,036 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,036 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=src_json newtbl=src_json 2018-01-05T11:28:43,036 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=src_json newtbl=src_json 2018-01-05T11:28:43,041 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,047 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,048 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,048 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@src_json) Type=TABLE WriteType=INSERT because WriteEntity(default@src_json) Type=TABLE WriteType=INSERT is present 2018-01-05T11:28:43,048 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-1:STATS] in serial mode 2018-01-05T11:28:43,048 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@c22cf4d, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:43,048 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:43,048 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,048 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:43,048 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:43,048 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:43,048 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:43,048 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,048 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:43,048 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:43,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,049 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:43,049 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:43,049 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:43,049 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:43,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,051 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,051 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,051 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:43,051 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:43,052 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,052 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:43,052 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,052 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_json 2018-01-05T11:28:43,052 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_json 2018-01-05T11:28:43,056 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,057 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,058 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,058 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=src_json newtbl=src_json 2018-01-05T11:28:43,058 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=src_json newtbl=src_json 2018-01-05T11:28:43,063 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,071 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,072 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,072 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.src_json stats: [numFiles=1, numRows=0, totalSize=645, rawDataSize=0] 2018-01-05T11:28:43,072 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,072 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,072 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/json.txt" INTO TABLE src_json 2018-01-05T11:28:43,072 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: LOAD 2018-01-05T11:28:43,072 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/json.txt 2018-01-05T11:28:43,072 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@src_json 2018-01-05T11:28:43,072 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,072 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,072 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:43,072 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=15, getTable_(String, String, )=5} 2018-01-05T11:28:43,072 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112842_e70c09f1-2c3a-4dad-98c4-f1fccac707a3); Time taken: 0.082 seconds 2018-01-05T11:28:43,072 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:43,072 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,072 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:43,072 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,072 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/json.txt" INTO TABLE src_json 2018-01-05T11:28:43,072 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.091 seconds 2018-01-05T11:28:43,072 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:43,072 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:43,072 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:43,072 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:43,073 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,073 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:43,073 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,073 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,073 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,073 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: ANALYZE TABLE src_json COMPUTE STATISTICS 2018-01-05T11:28:43,073 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112843_74d78b4b-f0c4-4916-9a9e-407949a276eb): ANALYZE TABLE src_json COMPUTE STATISTICS 2018-01-05T11:28:43,073 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,073 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: ANALYZE TABLE src_json COMPUTE STATISTICS 2018-01-05T11:28:43,073 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:43,073 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,073 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,073 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,073 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,073 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:43,073 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Invoking analyze on original query 2018-01-05T11:28:43,073 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Starting Semantic Analysis 2018-01-05T11:28:43,073 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed phase 1 of Semantic Analysis 2018-01-05T11:28:43,073 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for source tables 2018-01-05T11:28:43,073 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,073 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_json 2018-01-05T11:28:43,073 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_json 2018-01-05T11:28:43,078 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,078 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,078 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,078 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_json 2018-01-05T11:28:43,078 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_json 2018-01-05T11:28:43,082 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,083 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,083 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for subqueries 2018-01-05T11:28:43,083 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for destination tables 2018-01-05T11:28:43,083 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed getting MetaData in Semantic Analysis 2018-01-05T11:28:43,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_073_743625530122596682-1/-mr-10000 2018-01-05T11:28:43,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Table Plan for src_json TS[0] 2018-01-05T11:28:43,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Body Plan for Query Block null 2018-01-05T11:28:43,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Plan for Query Block null 2018-01-05T11:28:43,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Before logical optimization TS[0] 2018-01-05T11:28:43,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] lineage.Generator: Time taken for lineage transform=1 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(json: string|{src_json}json,BLOCK__OFFSET__INSIDE__FILE: bigint|{src_json}block__offset__inside__file,INPUT__FILE__NAME: string|{src_json}input__file__name,ROW__ID: struct|{src_json}row__id,RAW__DATA__SIZE: bigint|{src_json}raw__data__size) 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.PredicatePushDown: After PPD: TS[0] 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(json: string|{src_json}json,BLOCK__OFFSET__INSIDE__FILE: bigint|{src_json}block__offset__inside__file,INPUT__FILE__NAME: string|{src_json}input__file__name,ROW__ID: struct|{src_json}row__id,RAW__DATA__SIZE: bigint|{src_json}raw__data__size) 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(json: string|{src_json}json,BLOCK__OFFSET__INSIDE__FILE: bigint|{src_json}block__offset__inside__file,INPUT__FILE__NAME: string|{src_json}input__file__name,ROW__ID: struct|{src_json}row__id,RAW__DATA__SIZE: bigint|{src_json}raw__data__size) 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,096 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,096 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,096 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: After logical optimization TS[0] 2018-01-05T11:28:43,096 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,096 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,096 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,096 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,096 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,096 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,096 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=src_json 2018-01-05T11:28:43,096 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=src_json 2018-01-05T11:28:43,100 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting stats (Num rows: 1 Data size: 6450 Basic stats: COMPLETE Column stats: COMPLETE) on TS[0] 2018-01-05T11:28:43,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (src_json): numRows: 1 dataSize: 6450 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:28:43,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on TS[0] 2018-01-05T11:28:43,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:43,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:28:43,108 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Cycle free: true 2018-01-05T11:28:43,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(json: string|{src_json}json,BLOCK__OFFSET__INSIDE__FILE: bigint|{src_json}block__offset__inside__file,INPUT__FILE__NAME: string|{src_json}input__file__name,ROW__ID: struct|{src_json}row__id,RAW__DATA__SIZE: bigint|{src_json}raw__data__size) 2018-01-05T11:28:43,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:28:43,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json of table src_json 2018-01-05T11:28:43,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json 2018-01-05T11:28:43,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:28:43,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:43,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:43,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:43,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:43,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:43,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:43,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:28:43,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping vectorization 2018-01-05T11:28:43,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:28:43,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: llap mode: only 2018-01-05T11:28:43,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:28:43,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:43,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@3390621a can be done in LLAP 2018-01-05T11:28:43,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed plan generation 2018-01-05T11:28:43,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:43,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validation start 2018-01-05T11:28:43,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Not a partition. 2018-01-05T11:28:43,110 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,110 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:src_json.json, type:string, comment:null)], properties:null) 2018-01-05T11:28:43,110 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,110 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:43,110 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=10, flushCache_()=0, getTableColumnStatistics_(String, String, List, )=11} 2018-01-05T11:28:43,110 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112843_74d78b4b-f0c4-4916-9a9e-407949a276eb); Time taken: 0.037 seconds 2018-01-05T11:28:43,110 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:43,110 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,110 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112843_74d78b4b-f0c4-4916-9a9e-407949a276eb): ANALYZE TABLE src_json COMPUTE STATISTICS 2018-01-05T11:28:43,112 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,112 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: ANALYZE TABLE src_json COMPUTE STATISTICS 2018-01-05T11:28:43,112 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:28:43,112 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@src_json 2018-01-05T11:28:43,112 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@src_json 2018-01-05T11:28:43,112 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,112 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,112 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,112 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Query ID = djaiswal_20180105112843_74d78b4b-f0c4-4916-9a9e-407949a276eb 2018-01-05T11:28:43,112 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Total jobs = 1 2018-01-05T11:28:43,112 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,112 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:28:43,112 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MAPRED] in serial mode 2018-01-05T11:28:43,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_073_743625530122596682-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:28:43,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:28:43,147 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:28:43,147 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:28:43,148 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112843_74d78b4b-f0c4-4916-9a9e-407949a276eb 2018-01-05T11:28:43,148 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Session is already open 2018-01-05T11:28:43,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:28:43,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,148 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Dag name: ANALYZE TABLE src_json COMPUTE STATISTICS (Stage-0) 2018-01-05T11:28:43,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: DagInfo: {"context":"Hive","description":"\n\nANALYZE TABLE src_json COMPUTE STATISTICS"} 2018-01-05T11:28:43,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112843_74d78b4b-f0c4-4916-9a9e-407949a276eb with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:28:43,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,162 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:28:43,162 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,162 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:28:43,162 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,163 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 1.69KB 2018-01-05T11:28:43,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_073_743625530122596682-1/-mr-10000 2018-01-05T11:28:43,183 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_073_743625530122596682-1/-mr-10000 2018-01-05T11:28:43,186 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json 2018-01-05T11:28:43,186 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,186 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,186 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,186 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=ANALYZE TABLE src_json COMPUTE STATISTICS (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112843_74d78b4b-f0c4-4916-9a9e-407949a276eb } 2018-01-05T11:28:43,187 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:43,189 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: ANALYZE TABLE src_json COMPUTE STATISTICS (Stage-0) 2018-01-05T11:28:43,189 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:28:43,190 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_5, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_5.dot 2018-01-05T11:28:43,190 INFO [IPC Server handler 0 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_5 2018-01-05T11:28:43,190 WARN [IPC Server handler 0 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:43,190 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Running DAG: ANALYZE TABLE src_json COMPUTE STATISTICS (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112843_74d78b4b-f0c4-4916-9a9e-407949a276eb } 2018-01-05T11:28:43,213 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_5][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_5, submitTime=1515180523189, queueName=null 2018-01-05T11:28:43,213 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_5_00 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_5_00 [Map 1], Default Resources= 2018-01-05T11:28:43,213 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_5_00 [Map 1] 2018-01-05T11:28:43,213 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_5_00 [Map 1] 2018-01-05T11:28:43,214 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:28:43,214 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_5_00 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:43,214 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:28:43,214 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_5][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_5, initTime=1515180523214 2018-01-05T11:28:43,214 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: dag_1515180518813_0001_5 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:28:43,214 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:28:43,214 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_5][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_5, startTime=1515180523214 2018-01-05T11:28:43,214 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_5 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:28:43,214 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {src_json={InputName=src_json}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:28:43,214 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: src_json, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:28:43,214 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_5_00 [Map 1] 2018-01-05T11:28:43,214 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_5, dagName=ANALYZE TABLE src_json COMPUTE STATISTICS (Stage-0) 2018-01-05T11:28:43,214 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,215 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,215 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,219 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_5_00 [Map 1] 2018-01-05T11:28:43,219 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_5_00 [Map 1] 2018-01-05T11:28:43,219 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:43,219 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_5_00 [Map 1] 2018-01-05T11:28:43,222 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:28:43,222 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@58b331c4 2018-01-05T11:28:43,222 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_073_743625530122596682-1/djaiswal/_tez_scratch_dir/1a957e93-5994-4c59-8503-8126e0a8609a/map.xml 2018-01-05T11:28:43,222 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_073_743625530122596682-1/djaiswal/_tez_scratch_dir/1a957e93-5994-4c59-8503-8126e0a8609a/map.xml 2018-01-05T11:28:43,222 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_073_743625530122596682-1/djaiswal/_tez_scratch_dir/1a957e93-5994-4c59-8503-8126e0a8609a/map.xml 2018-01-05T11:28:43,222 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:43,222 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:43,222 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:43,223 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:43,223 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 1.69KB 2018-01-05T11:28:43,223 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_5_00 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:28:43,223 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: src_json on vertex vertex_1515180518813_0001_5_00 [Map 1] 2018-01-05T11:28:43,224 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:28:43,224 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:43,224 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:28:43,224 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:43,224 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_073_743625530122596682-1/djaiswal/_tez_scratch_dir/1a957e93-5994-4c59-8503-8126e0a8609a/map.xml 2018-01-05T11:28:43,224 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:43,224 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias src_json 2018-01-05T11:28:43,224 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json 2018-01-05T11:28:43,224 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json 2018-01-05T11:28:43,225 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 2018-01-05T11:28:43,225 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = 2018-01-05T11:28:43,225 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json 2018-01-05T11:28:43,225 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:43,225 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:43,225 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:43,225 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:43,239 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Time taken to get FileStatuses: 13 2018-01-05T11:28:43,239 INFO [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:28:43,239 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 13 2018-01-05T11:28:43,239 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 1 2018-01-05T11:28:43,239 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:43,239 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 1. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:28:43,239 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_073_743625530122596682-1/djaiswal/_tez_scratch_dir/1a957e93-5994-4c59-8503-8126e0a8609a/map.xml 2018-01-05T11:28:43,239 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_073_743625530122596682-1/djaiswal/_tez_scratch_dir/1a957e93-5994-4c59-8503-8126e0a8609a/map.xml 2018-01-05T11:28:43,239 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_073_743625530122596682-1/djaiswal/_tez_scratch_dir/1a957e93-5994-4c59-8503-8126e0a8609a/map.xml 2018-01-05T11:28:43,239 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:43,239 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:43,239 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:43,240 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:43,240 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 1.69KB 2018-01-05T11:28:43,240 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json/json.txt to src new group? true 2018-01-05T11:28:43,240 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:28:43,240 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:28:43,240 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:28:43,240 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 11 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 645 Original splits: 1 2018-01-05T11:28:43,240 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Using original number of splits: 1 desired splits: 1 2018-01-05T11:28:43,240 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 1 grouped split count is 1, for bucket: 1 2018-01-05T11:28:43,240 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:28:43,241 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: src_json on vertex vertex_1515180518813_0001_5_00 [Map 1] 2018-01-05T11:28:43,241 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_5_00 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:43,241 INFO [App Shared Pool - #0] impl.VertexImpl: Got updated RootInputsSpecs: {src_json=forAllWorkUnits=true, update=[1]} 2018-01-05T11:28:43,242 INFO [App Shared Pool - #0] impl.VertexImpl: Vertex vertex_1515180518813_0001_5_00 [Map 1] parallelism set to 1 2018-01-05T11:28:43,242 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_5_00 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:43,242 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_5_00 [Map 1], numAdditionalOutputs=1 2018-01-05T11:28:43,242 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_5][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_5_00, initRequestedTime=1515180523214, initedTime=1515180523242, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:43,242 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_5_00 [Map 1] with distanceFromRoot: 0 2018-01-05T11:28:43,242 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_5_00 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:28:43,242 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_5][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_5_00, startRequestedTime=1515180523224, startedTime=1515180523242 2018-01-05T11:28:43,242 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_5][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_5_00, reconfigureDoneTime=1515180523242, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:28:43,242 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_5_00 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:43,243 INFO [App Shared Pool - #0] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:28:43,243 INFO [App Shared Pool - #0] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:28:43,243 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_5_00 [Map 1] 2018-01-05T11:28:43,243 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_5][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_5_00_000000, scheduledTime=1515180523243, launchTime=1515180523243 2018-01-05T11:28:43,243 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_5_00 [Map 1] 2018-01-05T11:28:43,243 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_5_00_000000_0, priority=2, capability=, hosts=null 2018-01-05T11:28:43,243 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=2,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:43,243 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=2,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:43,244 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:43,244 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_5_00_000000_0 on hosts=null 2018-01-05T11:28:43,244 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_5_00_000000_0 on hosts=null 2018-01-05T11:28:43,244 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_5_00_000000_0 2018-01-05T11:28:43,244 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:43,244 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #7, task=TaskInfo{task=attempt_1515180518813_0001_5_00_000000_0, priority=2, startTime=0, containerId=null, uniqueId=6, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000007 2018-01-05T11:28:43,244 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_5_00_000000_0; false 2018-01-05T11:28:43,244 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_5_00_000000_0, priority=2, startTime=311543752, containerId=container_111101111_0001_00_000007assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=6, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:43,245 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000007 for node: 10.22.8.182:49818 2018-01-05T11:28:43,247 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 5, name=ANALYZE TABLE src_json COMPUTE STATISTICS (Stage-0), queryId=djaiswal_20180105112843_74d78b4b-f0c4-4916-9a9e-407949a276eb 2018-01-05T11:28:43,247 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000007, attempt_1515180518813_0001_5_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:43,247 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000007 for node: 10.22.8.182:49818 2018-01-05T11:28:43,248 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000007 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:43,248 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000007, launchTime=1515180523248 2018-01-05T11:28:43,249 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_5_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112843_74d78b4b-f0c4-4916-9a9e-407949a276eb, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000007, dagName=ANALYZE TABLE src_json COMPUTE STATISTICS (Stage-0), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{src_json,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{out_Map 1,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180523214, firstAttemptStartTime=1515180523243, currentAttemptStartTime=1515180523247} 2018-01-05T11:28:43,249 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_5_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000007] on NM: [10.22.8.182:49818] 2018-01-05T11:28:43,249 INFO [IPC Server handler 3 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000007, QueryId=djaiswal_20180105112843_74d78b4b-f0c4-4916-9a9e-407949a276eb-5, DagId=5, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180523249 2018-01-05T11:28:43,249 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:43,249 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:43,249 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_5][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_5_00_000000_0, startTime=1515180523248, containerId=container_111101111_0001_00_000007, nodeId=10.22.8.182:49818 2018-01-05T11:28:43,249 DEBUG [IPC Server handler 3 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=5} with the ShuffleHandler 2018-01-05T11:28:43,260 INFO [IPC Server handler 3 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:28:43,260 DEBUG [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/5] 2018-01-05T11:28:43,260 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:43,260 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_5_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:43,260 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_5_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180523243, dagStartTime=1515180523214, withinDagPriority=2, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:28:43,260 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_5_00_000000_0 : ACCEPTED 2018-01-05T11:28:43,260 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_5_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:43,260 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_5_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180523243, dagStartTime=1515180523214, withinDagPriority=2, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:28:43,260 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_5_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:43,261 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_5_00_000000_0 to pre-emption queue 2018-01-05T11:28:43,261 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_5_00_000000_0: true 2018-01-05T11:28:43,261 INFO [TaskCommunicator # 2] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_5_00_000000_0 2018-01-05T11:28:43,261 INFO [TaskCommunicator # 2] GuaranteedTasks: Received response for attempt_1515180518813_0001_5_00_000000_0, true 2018-01-05T11:28:43,261 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: taskOwner hashCode:1834705872 2018-01-05T11:28:43,262 INFO [Task-Executor-3] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : ANALYZE TABLE src_json COMPUTE STATISTICS (Stage-0), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_5_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=src_json, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=out_Map 1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:28:43,262 INFO [Task-Executor-3] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=954728448, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:43,262 INFO [Task-Executor-3] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_5_00_000000_0 vertexName: Map 1 2018-01-05T11:28:43,262 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000007, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_5_00_000000_0, eventCount=1 } 2018-01-05T11:28:43,263 DEBUG [TezTR-518813_1_5_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_5 queryId: djaiswal_20180105112843_74d78b4b-f0c4-4916-9a9e-407949a276eb instanceType: TaskRunner2Callable 2018-01-05T11:28:43,263 INFO [TezTR-518813_1_5_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_5_00_000000_0 2018-01-05T11:28:43,263 DEBUG [TezTR-518813_1_5_0_0_0] log.PerfLogger: 2018-01-05T11:28:43,264 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:43,265 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:43,268 DEBUG [TezTR-518813_1_5_0_0_0] log.PerfLogger: 2018-01-05T11:28:43,268 INFO [TezTR-518813_1_5_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:43,268 INFO [TezTR-518813_1_5_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:43,269 INFO [I/O Setup 0 Initialize: {src_json}] input.MRInput: src_json using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:28:43,269 INFO [I/O Setup 0 Initialize: {src_json}] input.MRInputLegacy: src_json MRInputLegacy deferring initialization 2018-01-05T11:28:43,269 INFO [TezTR-518813_1_5_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:43,269 INFO [I/O Setup 1 Initialize: {out_Map 1}] output.MROutput: out_Map 1: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:28:43,270 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:43,270 INFO [IPC Server handler 11 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_5_00_000000_0 sent events: (0-1). 2018-01-05T11:28:43,270 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:43,270 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:43,270 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:43,270 INFO [TezTR-518813_1_5_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:43,270 INFO [TezTR-518813_1_5_0_0_0] resources.MemoryDistributor: InitialRequests=[src_json:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [out_Map 1:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput] 2018-01-05T11:28:43,270 INFO [TezTR-518813_1_5_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:43,270 INFO [TezTR-518813_1_5_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:43,271 INFO [TezTR-518813_1_5_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 954728448, TotalAvailable: 639668060, TotalRequested/TotalJVMHeap:0.00 2018-01-05T11:28:43,271 INFO [TezTR-518813_1_5_0_0_0] resources.MemoryDistributor: Allocations=[src_json:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [out_Map 1:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0] 2018-01-05T11:28:43,271 INFO [TezTR-518813_1_5_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:43,271 INFO [TezTR-518813_1_5_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: src_json being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:28:43,271 INFO [TezTR-518813_1_5_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:28:43,271 INFO [TezTR-518813_1_5_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:28:43,271 INFO [I/O Setup 0 Start: {src_json}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: src_json 2018-01-05T11:28:43,271 INFO [TezTR-518813_1_5_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:43,271 INFO [TezTR-518813_1_5_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_5_00_000000_0 2018-01-05T11:28:43,271 DEBUG [TezTR-518813_1_5_0_0_0] log.PerfLogger: 2018-01-05T11:28:43,271 DEBUG [TezTR-518813_1_5_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_5_00_000000_0_10019 2018-01-05T11:28:43,271 DEBUG [TezTR-518813_1_5_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:43,271 DEBUG [TezTR-518813_1_5_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_5_00_000000_0 2018-01-05T11:28:43,271 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:28:43,271 INFO [TezTR-518813_1_5_0_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112843_74d78b4b-f0c4-4916-9a9e-407949a276eb 2018-01-05T11:28:43,271 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_5_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:28:43,271 DEBUG [TezTR-518813_1_5_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:43,271 DEBUG [TezTR-518813_1_5_0_0_0] log.PerfLogger: 2018-01-05T11:28:43,271 DEBUG [TezTR-518813_1_5_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:43,271 DEBUG [TezTR-518813_1_5_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:43,271 DEBUG [TezTR-518813_1_5_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112843_74d78b4b-f0c4-4916-9a9e-407949a276eb_Map 1__MAP_PLAN__ 2018-01-05T11:28:43,271 INFO [TezTR-518813_1_5_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_073_743625530122596682-1/djaiswal/_tez_scratch_dir/1a957e93-5994-4c59-8503-8126e0a8609a/map.xml 2018-01-05T11:28:43,271 DEBUG [TezTR-518813_1_5_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_073_743625530122596682-1/djaiswal/_tez_scratch_dir/1a957e93-5994-4c59-8503-8126e0a8609a/map.xml 2018-01-05T11:28:43,271 DEBUG [TezTR-518813_1_5_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_073_743625530122596682-1/djaiswal/_tez_scratch_dir/1a957e93-5994-4c59-8503-8126e0a8609a/map.xml 2018-01-05T11:28:43,272 DEBUG [TezTR-518813_1_5_0_0_0] log.PerfLogger: 2018-01-05T11:28:43,272 INFO [TezTR-518813_1_5_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:43,272 DEBUG [TezTR-518813_1_5_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:43,272 DEBUG [TezTR-518813_1_5_0_0_0] log.PerfLogger: 2018-01-05T11:28:43,272 INFO [TezTR-518813_1_5_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 1.69KB 2018-01-05T11:28:43,273 DEBUG [TezTR-518813_1_5_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:43,273 INFO [TezTR-518813_1_5_0_0_0] tez.MapRecordProcessor: The input names are: [src_json] 2018-01-05T11:28:43,273 INFO [TezTR-518813_1_5_0_0_0] input.MRInput: Initialized MRInput: src_json 2018-01-05T11:28:43,273 DEBUG [TezTR-518813_1_5_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 1 2018-01-05T11:28:43,273 INFO [TezTR-518813_1_5_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_073_743625530122596682-1/djaiswal/_tez_scratch_dir/1a957e93-5994-4c59-8503-8126e0a8609a/map.xml 2018-01-05T11:28:43,273 DEBUG [TezTR-518813_1_5_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:43,273 DEBUG [TezTR-518813_1_5_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json/json.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.src_json, numFiles=1, columns.types=string, serialization.format=1, serialization.ddl=struct src_json { string json}, columns=json, columns.comments=default, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, totalSize=645, column.name.delimiter=,, transient_lastDdlTime=1515180523}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.src_json, numFiles=1, columns.types=string, columns=json, serialization.format=1, serialization.ddl=struct src_json { string json}, columns.comments=default, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, totalSize=645, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json, column.name.delimiter=,, transient_lastDdlTime=1515180523}, baseFileName=src_json, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.src_json, numFiles=1, columns.types=string, serialization.format=1, serialization.ddl=struct src_json { string json}, columns=json, columns.comments=default, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, totalSize=645, column.name.delimiter=,, transient_lastDdlTime=1515180523}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.src_json, numFiles=1, columns.types=string, columns=json, serialization.format=1, serialization.ddl=struct src_json { string json}, columns.comments=default, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, totalSize=645, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json, column.name.delimiter=,, transient_lastDdlTime=1515180523}, baseFileName=src_json, vectorPartitionDesc=null]} 2018-01-05T11:28:43,273 DEBUG [TezTR-518813_1_5_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:43,273 INFO [TezTR-518813_1_5_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:43,273 DEBUG [TezTR-518813_1_5_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:43,274 DEBUG [TezTR-518813_1_5_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:43,275 INFO [TezTR-518813_1_5_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_073_743625530122596682-1/djaiswal/_tez_scratch_dir/1a957e93-5994-4c59-8503-8126e0a8609a/map.xml 2018-01-05T11:28:43,275 DEBUG [TezTR-518813_1_5_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:43,275 INFO [TezTR-518813_1_5_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_5_00_000000_0 2018-01-05T11:28:43,275 INFO [TezTR-518813_1_5_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [json] / schema.evolution.columns.types [string] (isAcidRead false) 2018-01-05T11:28:43,275 WARN [TezTR-518813_1_5_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json/json.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:28:43,275 DEBUG [TezTR-518813_1_5_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:28:43,275 WARN [TezTR-518813_1_5_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json/json.txt:0+645 2018-01-05T11:28:43,276 DEBUG [TezTR-518813_1_5_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json/json.txt 2018-01-05T11:28:43,276 INFO [TezTR-518813_1_5_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json/json.txt:0+645], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=645} 2018-01-05T11:28:43,276 INFO [TezTR-518813_1_5_0_0_0] input.MRInput: src_json initialized RecordReader from event 2018-01-05T11:28:43,276 DEBUG [TezTR-518813_1_5_0_0_0] tez.MapRecordProcessor: Starting Output: out_Map 1 2018-01-05T11:28:43,276 INFO [TezTR-518813_1_5_0_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:28:43,276 INFO [TezTR-518813_1_5_0_0_0] exec.MapOperator: Initializing operator MAP[0] 2018-01-05T11:28:43,276 DEBUG [TezTR-518813_1_5_0_0_0] exec.MapOperator: Initialization Done 0 MAP 2018-01-05T11:28:43,276 DEBUG [TezTR-518813_1_5_0_0_0] exec.MapOperator: Operator 0 MAP initialized 2018-01-05T11:28:43,276 DEBUG [TezTR-518813_1_5_0_0_0] exec.MapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:28:43,276 DEBUG [TezTR-518813_1_5_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:43,276 DEBUG [TezTR-518813_1_5_0_0_0] exec.MapOperator: Adding alias src_json to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json 2018-01-05T11:28:43,276 DEBUG [TezTR-518813_1_5_0_0_0] exec.MapOperator: dump TS[0] struct 2018-01-05T11:28:43,276 INFO [TezTR-518813_1_5_0_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:28:43,276 INFO [TezTR-518813_1_5_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:43,276 DEBUG [TezTR-518813_1_5_0_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:28:43,276 INFO [TezTR-518813_1_5_0_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:28:43,276 DEBUG [TezTR-518813_1_5_0_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:28:43,276 DEBUG [TezTR-518813_1_5_0_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:28:43,276 DEBUG [TezTR-518813_1_5_0_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:28:43,276 DEBUG [TezTR-518813_1_5_0_0_0] log.PerfLogger: 2018-01-05T11:28:43,276 INFO [TezTR-518813_1_5_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_073_743625530122596682-1/djaiswal/_tez_scratch_dir/1a957e93-5994-4c59-8503-8126e0a8609a/map.xml 2018-01-05T11:28:43,276 DEBUG [TezTR-518813_1_5_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:43,276 DEBUG [TezTR-518813_1_5_0_0_0] exec.MapOperator: Processing alias(es) src_json for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json/json.txt 2018-01-05T11:28:43,276 INFO [TezTR-518813_1_5_0_0_0] exec.MapOperator: MAP[0]: records read - 1 2018-01-05T11:28:43,276 DEBUG [TezTR-518813_1_5_0_0_0] log.PerfLogger: 2018-01-05T11:28:43,276 DEBUG [TezTR-518813_1_5_0_0_0] mr.ObjectCache: djaiswal_20180105112843_74d78b4b-f0c4-4916-9a9e-407949a276eb_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:28:43,277 DEBUG [TezTR-518813_1_5_0_0_0] exec.MapOperator: close called for operator MAP[0] 2018-01-05T11:28:43,277 INFO [TezTR-518813_1_5_0_0_0] exec.MapOperator: Closing operator MAP[0] 2018-01-05T11:28:43,277 INFO [TezTR-518813_1_5_0_0_0] exec.MapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:1, 2018-01-05T11:28:43,277 DEBUG [TezTR-518813_1_5_0_0_0] exec.MapOperator: Closing child = TS[0] 2018-01-05T11:28:43,277 DEBUG [TezTR-518813_1_5_0_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:28:43,277 DEBUG [TezTR-518813_1_5_0_0_0] exec.TableScanOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:43,277 INFO [TezTR-518813_1_5_0_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:28:43,277 DEBUG [TezTR-518813_1_5_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:43,277 DEBUG [TezTR-518813_1_5_0_0_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_073_743625530122596682-1/-mr-10000 2018-01-05T11:28:43,277 INFO [TezTR-518813_1_5_0_0_0] exec.TableScanOperator: publishing : default.src_json/ : {numRows=1, rawDataSize=644} 2018-01-05T11:28:43,299 DEBUG [TezTR-518813_1_5_0_0_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_073_743625530122596682-1/-mr-10000/tmpstats-0_0 2018-01-05T11:28:43,299 DEBUG [TezTR-518813_1_5_0_0_0] fs.FSStatsPublisher: Writing stats in it : {default.src_json/={numRows=1, rawDataSize=644}} 2018-01-05T11:28:43,299 DEBUG [TezTR-518813_1_5_0_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:28:43,299 DEBUG [TezTR-518813_1_5_0_0_0] exec.MapOperator: 0 Close done 2018-01-05T11:28:43,299 INFO [TezTR-518813_1_5_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_5_00_000000_0 2018-01-05T11:28:43,299 INFO [TezTR-518813_1_5_0_0_0] output.MROutput: out_Map 1 closed 2018-01-05T11:28:43,300 DEBUG [TezTR-518813_1_5_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:43,300 INFO [TezTR-518813_1_5_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:43,300 INFO [TezTR-518813_1_5_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_5_00_000000_0, askedToStop=false 2018-01-05T11:28:43,300 INFO [TezTR-518813_1_5_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_5_00_000000_0, stopRequested=false 2018-01-05T11:28:43,300 INFO [TezTR-518813_1_5_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_5_00_000000_0: Counters: 5 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=1, INPUT_SPLIT_LENGTH_BYTES=645, OUTPUT_RECORDS=0][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=1]] 2018-01-05T11:28:43,300 DEBUG [TezTR-518813_1_5_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_5_0_0_0 thread id: 477 scheme: file bytesRead: 665 bytesWritten: 85 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:43,301 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_5_00_000000_0, isTaskDone=true 2018-01-05T11:28:43,301 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000007, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_5_00_000000_0, eventCount=2 } 2018-01-05T11:28:43,302 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_5][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_5_00_000000_0, creationTime=1515180523243, allocationTime=1515180523245, startTime=1515180523248, finishTime=1515180523302, timeTaken=54, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:43,303 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:43,303 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_5_00_000000_0 2018-01-05T11:28:43,303 INFO [Task-Executor-3] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_5_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:43,303 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_5_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:43,303 INFO [Task-Executor-3] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_5_00_000000_0 2018-01-05T11:28:43,303 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_5_00_000000_0; false 2018-01-05T11:28:43,303 INFO [Task-Executor-3] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000007=41 2018-01-05T11:28:43,303 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_5_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:43,303 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_5_00_000000_0: true 2018-01-05T11:28:43,303 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:43,303 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:43,304 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_5_00_000000_0 2018-01-05T11:28:43,304 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000007, QueryId=djaiswal_20180105112843_74d78b4b-f0c4-4916-9a9e-407949a276eb-5, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:28:43,304 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_5][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_5_00_000000, startTime=1515180523248, finishTime=1515180523303, timeTaken=55, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_5_00_000000_0, diagnostics=, counters=Counters: 11, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=665, FILE_BYTES_WRITTEN=85, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=1, INPUT_SPLIT_LENGTH_BYTES=645, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=1 2018-01-05T11:28:43,304 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_5_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:43,304 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_5_00_000000_0 2018-01-05T11:28:43,304 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_5_00 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:43,304 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_5_00 [Map 1] 2018-01-05T11:28:43,304 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_5][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_5_00, initRequestedTime=1515180523214, initedTime=1515180523242, startRequestedTime=1515180523224, startedTime=1515180523242, finishTime=1515180523304, timeTaken=62, status=SUCCEEDED, diagnostics=, counters=Counters: 11, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=665, FILE_BYTES_WRITTEN=85, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=1, INPUT_SPLIT_LENGTH_BYTES=645, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=1, vertexStats=firstTaskStartTime=1515180523248, firstTasksToStart=[ task_1515180518813_0001_5_00_000000 ], lastTaskFinishTime=1515180523302, lastTasksToFinish=[ task_1515180518813_0001_5_00_000000 ], minTaskDuration=54, maxTaskDuration=54, avgTaskDuration=54.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_5_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_5_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:43,305 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_5_00 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:43,305 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_5][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000007, stoppedTime=1515180523305, exitStatus=0 2018-01-05T11:28:43,305 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000007 exited with diagnostics set to Container completed. 2018-01-05T11:28:43,305 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_5_00 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1 2018-01-05T11:28:43,305 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null 2018-01-05T11:28:43,305 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_5_00 [Map 1] 2018-01-05T11:28:43,305 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_5, queueSize=0 2018-01-05T11:28:43,306 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_5][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_5, startTime=1515180523214, finishTime=1515180523305, timeTaken=91, status=SUCCEEDED, diagnostics=, counters=Counters: 15, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, OTHER_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=118, AM_GC_TIME_MILLIS=0, File System Counters, FILE_BYTES_READ=665, FILE_BYTES_WRITTEN=85, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=1, INPUT_SPLIT_LENGTH_BYTES=645, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=1 2018-01-05T11:28:43,306 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_5 finished with state: SUCCEEDED 2018-01-05T11:28:43,306 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_5 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:28:43,306 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000007 2018-01-05T11:28:43,307 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_5, dagState=SUCCEEDED 2018-01-05T11:28:43,307 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_5_post 2018-01-05T11:28:43,307 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:43,307 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:28:43,307 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:28:43,307 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 5 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=1, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=1,NumRequestsWithoutLocation=1, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=1} 2018-01-05T11:28:43,307 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:28:43,307 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=ANALYZE TABLE src_json COMPUTE STATISTICS (Stage-0), with id=dag_1515180518813_0001_5 2018-01-05T11:28:43,307 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 5, to 10.22.8.182:49818 2018-01-05T11:28:43,307 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=1, Remaining=0 2018-01-05T11:28:43,307 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=ANALYZE TABLE src_json COMPUTE STATISTICS (Stage-0), with id=dag_1515180518813_0001_5 2018-01-05T11:28:43,307 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=5} 2018-01-05T11:28:43,307 INFO [IPC Server handler 1 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=5}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:28:43,308 INFO [IPC Server handler 1 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/5 after 300 seconds 2018-01-05T11:28:43,308 ERROR [IPC Server handler 1 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112843_74d78b4b-f0c4-4916-9a9e-407949a276eb, dag_1515180518813_0001_5 2018-01-05T11:28:43,308 INFO [IPC Server handler 1 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112843_74d78b4b-f0c4-4916-9a9e-407949a276eb 2018-01-05T11:28:43,308 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=5} = 0 2018-01-05T11:28:43,308 DEBUG [IPC Server handler 1 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=5} 2018-01-05T11:28:43,308 DEBUG [IPC Server handler 1 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:28:43,308 DEBUG [IPC Server handler 1 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=5} 2018-01-05T11:28:43,308 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,308 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,308 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 2018-01-05T11:28:43,308 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,309 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-2:STATS] in serial mode 2018-01-05T11:28:43,309 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@2c989d82, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:43,309 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:43,309 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,309 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:43,309 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:43,309 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:43,309 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:43,309 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,309 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:43,309 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:43,309 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_073_743625530122596682-1/-mr-10000 2018-01-05T11:28:43,310 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_073_743625530122596682-1/-mr-10000 2018-01-05T11:28:43,311 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.src_json/ numRows 2018-01-05T11:28:43,311 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.src_json/ numRows 1 2018-01-05T11:28:43,311 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.src_json/ rawDataSize 2018-01-05T11:28:43,311 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.src_json/ rawDataSize 644 2018-01-05T11:28:43,312 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,312 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:43,312 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:43,312 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:43,312 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:43,313 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,313 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,313 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,316 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,316 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,317 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:43,317 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:43,317 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,317 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:43,317 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,317 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=src_json newtbl=src_json 2018-01-05T11:28:43,317 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=src_json newtbl=src_json 2018-01-05T11:28:43,321 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,328 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,329 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,329 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.src_json stats: [numFiles=1, numRows=1, totalSize=645, rawDataSize=644] 2018-01-05T11:28:43,329 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: About to delete stats tmp dir :file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_073_743625530122596682-1/-mr-10000 2018-01-05T11:28:43,330 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,330 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,330 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: ANALYZE TABLE src_json COMPUTE STATISTICS 2018-01-05T11:28:43,330 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:28:43,330 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@src_json 2018-01-05T11:28:43,330 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@src_json 2018-01-05T11:28:43,330 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,330 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,330 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:43,330 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=12} 2018-01-05T11:28:43,330 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112843_74d78b4b-f0c4-4916-9a9e-407949a276eb); Time taken: 0.22 seconds 2018-01-05T11:28:43,330 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:43,330 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,330 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:43,330 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,330 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query ANALYZE TABLE src_json COMPUTE STATISTICS 2018-01-05T11:28:43,330 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_073_743625530122596682-1 2018-01-05T11:28:43,331 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.257 seconds 2018-01-05T11:28:43,331 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:43,331 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:43,331 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:43,331 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:43,331 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,331 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:43,331 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,331 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,331 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,331 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: ANALYZE TABLE src_json COMPUTE STATISTICS FOR COLUMNS json 2018-01-05T11:28:43,331 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112843_0b779e91-2298-4910-af0f-02be4fa91eae): ANALYZE TABLE src_json COMPUTE STATISTICS FOR COLUMNS json 2018-01-05T11:28:43,331 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,331 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: ANALYZE TABLE src_json COMPUTE STATISTICS FOR COLUMNS json 2018-01-05T11:28:43,331 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:43,331 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,331 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,331 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,331 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,331 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:43,332 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,332 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_json 2018-01-05T11:28:43,332 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_json 2018-01-05T11:28:43,336 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,336 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,336 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: select compute_stats(`json`, 'hll') from `default`.`src_json` 2018-01-05T11:28:43,336 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: select compute_stats(`json`, 'hll') from `default`.`src_json` 2018-01-05T11:28:43,337 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:43,337 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Invoking analyze on rewritten query 2018-01-05T11:28:43,337 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Starting Semantic Analysis 2018-01-05T11:28:43,337 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:43,337 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed phase 1 of Semantic Analysis 2018-01-05T11:28:43,337 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for source tables 2018-01-05T11:28:43,337 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,337 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_json 2018-01-05T11:28:43,337 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_json 2018-01-05T11:28:43,341 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,342 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,342 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for subqueries 2018-01-05T11:28:43,342 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for destination tables 2018-01-05T11:28:43,355 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed getting MetaData in Semantic Analysis 2018-01-05T11:28:43,355 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_336_6588790997440458878-1/-mr-10001 2018-01-05T11:28:43,355 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Table Plan for src_json TS[0] 2018-01-05T11:28:43,356 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:43,356 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: RR before GB src_json{(json,json: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)(RAW__DATA__SIZE,RAW__DATA__SIZE: bigint)} after GB {((tok_function compute_stats (tok_table_or_col json) 'hll'),_col0: struct)} 2018-01-05T11:28:43,356 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: tree: (tok_select (tok_selexpr (tok_function compute_stats (tok_table_or_col json) 'hll'))) 2018-01-05T11:28:43,356 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: genSelectPlan: input = {((tok_function compute_stats (tok_table_or_col json) 'hll'),_col0: struct)} starRr = src_json{(json,json: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)(RAW__DATA__SIZE,RAW__DATA__SIZE: bigint)} 2018-01-05T11:28:43,356 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Select Plan row schema: null{(_c0,_col0: struct)} 2018-01-05T11:28:43,356 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Select Plan for clause: insclause-0 2018-01-05T11:28:43,356 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_336_6588790997440458878-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-43_336_6588790997440458878-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_336_6588790997440458878-1/-mr-10000 2018-01-05T11:28:43,356 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_336_6588790997440458878-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-43_336_6588790997440458878-1 2018-01-05T11:28:43,378 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_336_6588790997440458878-1/-mr-10003 2018-01-05T11:28:43,378 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_336_6588790997440458878-1/-mr-10000 row schema: null{(_c0,_col0: struct)} 2018-01-05T11:28:43,378 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Body Plan for Query Block null 2018-01-05T11:28:43,378 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Plan for Query Block null 2018-01-05T11:28:43,378 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Before logical optimization TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-SEL[5]-FS[6] 2018-01-05T11:28:43,378 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,378 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,378 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,378 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] lineage.Generator: Time taken for lineage transform=0 2018-01-05T11:28:43,378 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,378 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,378 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,378 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,378 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,378 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(json: string|{src_json}json,BLOCK__OFFSET__INSIDE__FILE: bigint|{src_json}block__offset__inside__file,INPUT__FILE__NAME: string|{src_json}input__file__name,ROW__ID: struct|{src_json}row__id,RAW__DATA__SIZE: bigint|{src_json}raw__data__size) 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(json: string|{src_json}json,BLOCK__OFFSET__INSIDE__FILE: bigint|{src_json}block__offset__inside__file,INPUT__FILE__NAME: string|{src_json}input__file__name,ROW__ID: struct|{src_json}row__id,RAW__DATA__SIZE: bigint|{src_json}raw__data__size) 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[json] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col json) 'hll')) 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col json) 'hll')) 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col json) 'hll')) 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0) 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0]) 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{}) 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,379 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for FS(6) 2018-01-05T11:28:43,379 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(5) 2018-01-05T11:28:43,379 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for GBY(4) 2018-01-05T11:28:43,379 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for RS(3) 2018-01-05T11:28:43,379 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for GBY(2) 2018-01-05T11:28:43,379 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(1) 2018-01-05T11:28:43,379 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.PredicatePushDown: After PPD: TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-SEL[5]-FS[6] 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(json: string|{src_json}json,BLOCK__OFFSET__INSIDE__FILE: bigint|{src_json}block__offset__inside__file,INPUT__FILE__NAME: string|{src_json}input__file__name,ROW__ID: struct|{src_json}row__id,RAW__DATA__SIZE: bigint|{src_json}raw__data__size) 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(json: string|{src_json}json,BLOCK__OFFSET__INSIDE__FILE: bigint|{src_json}block__offset__inside__file,INPUT__FILE__NAME: string|{src_json}input__file__name,ROW__ID: struct|{src_json}row__id,RAW__DATA__SIZE: bigint|{src_json}raw__data__size) 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[json] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col json) 'hll')) 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col json) 'hll')) 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col json) 'hll')) 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0) 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0]) 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{}) 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,380 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,380 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,380 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,380 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,380 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(json: string|{src_json}json,BLOCK__OFFSET__INSIDE__FILE: bigint|{src_json}block__offset__inside__file,INPUT__FILE__NAME: string|{src_json}input__file__name,ROW__ID: struct|{src_json}row__id,RAW__DATA__SIZE: bigint|{src_json}raw__data__size) 2018-01-05T11:28:43,380 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(json: string|{src_json}json,BLOCK__OFFSET__INSIDE__FILE: bigint|{src_json}block__offset__inside__file,INPUT__FILE__NAME: string|{src_json}input__file__name,ROW__ID: struct|{src_json}row__id,RAW__DATA__SIZE: bigint|{src_json}raw__data__size) 2018-01-05T11:28:43,380 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:43,380 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:43,380 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[json] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:28:43,380 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col json) 'hll')) 2018-01-05T11:28:43,380 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:43,380 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:43,380 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col json) 'hll')) 2018-01-05T11:28:43,380 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:43,380 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:43,380 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col json) 'hll')) 2018-01-05T11:28:43,380 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:43,380 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:43,380 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0) 2018-01-05T11:28:43,380 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:43,380 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:43,380 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0]) 2018-01-05T11:28:43,380 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{}) 2018-01-05T11:28:43,380 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:43,380 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:43,380 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,380 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,380 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,380 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,380 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 3 key:[] 2018-01-05T11:28:43,380 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 3 oldColExprMap: {VALUE._col0=Column[_col0]} 2018-01-05T11:28:43,380 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 3 newColExprMap: {VALUE._col0=Column[_col0]} 2018-01-05T11:28:43,381 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,381 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,381 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,381 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,381 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,381 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,381 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,381 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,381 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,381 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,381 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,381 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,381 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,381 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,381 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,381 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,381 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,381 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,381 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.IdentityProjectRemover: Identity project remover optimization removed : SEL[5] 2018-01-05T11:28:43,381 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,381 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,381 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,381 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,381 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,381 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,381 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,381 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: After logical optimization TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-FS[6] 2018-01-05T11:28:43,382 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,382 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,382 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,382 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,382 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,382 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,382 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=src_json 2018-01-05T11:28:43,382 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=src_json 2018-01-05T11:28:43,388 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 2.410193ms + 0.006771ms, the query is [select "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "TAB_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (...)] 2018-01-05T11:28:43,388 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,396 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting stats (Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: NONE) on TS[0] 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (src_json): numRows: 1 dataSize: 184 basicStatsState: COMPLETE colStatsState: NONE colStats: {json= colName: json colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: true isEstimated: true} 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting stats (Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: NONE) on SEL[1] 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [1] STATS-SEL[1]: numRows: 1 dataSize: 184 basicStatsState: COMPLETE colStatsState: NONE colStats: {json= colName: json colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: true isEstimated: true} 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[2]: inputSize: 184 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [Case 1] STATS-GBY[2]: cardinality: 1 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 624 Basic stats: COMPLETE Column stats: NONE) on GBY[2] 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-GBY[2]: numRows: 1 dataSize: 624 basicStatsState: COMPLETE colStatsState: NONE colStats: {_col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, json= colName: json colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: true isEstimated: true} 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 624 Basic stats: COMPLETE Column stats: NONE) on RS[3] 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-RS[3]: numRows: 1 dataSize: 624 basicStatsState: COMPLETE colStatsState: NONE colStats: {_col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, json= colName: json colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: true isEstimated: true} 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[4]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [Case 7] STATS-GBY[4]: cardinality: 0 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[4]: Equals 0 in number of rows. 0 rows will be set to 1 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 624 Basic stats: COMPLETE Column stats: NONE) on GBY[4] 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-GBY[4]: numRows: 1 dataSize: 624 basicStatsState: COMPLETE colStatsState: NONE colStats: {_col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, json= colName: json colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: true isEstimated: true} 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-FS[6]: numRows: 1 dataSize: 624 basicStatsState: COMPLETE colStatsState: NONE colStats: {_col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, json= colName: json colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: true isEstimated: true} 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on TS[0] 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on SEL[1] 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[2] 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[3] 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[4] 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FileSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on FS[6] 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,397 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: FS, 6 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:43,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: GBY, 4 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: RS, 3 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: GBY, 2 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: SEL, 1 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:28:43,398 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Cycle free: true 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(json: string|{src_json}json,BLOCK__OFFSET__INSIDE__FILE: bigint|{src_json}block__offset__inside__file,INPUT__FILE__NAME: string|{src_json}input__file__name,ROW__ID: struct|{src_json}row__id,RAW__DATA__SIZE: bigint|{src_json}raw__data__size) 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(json: string|{src_json}json) 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[json]) 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col json) 'hll')) 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col json) 'hll')) 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col json) 'hll')) 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{}) 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: TS[0] 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: RS[3] 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json of table src_json 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: RS[3] 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: GBY[4] 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: FS[6] 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding reduce work (Reducer 2) for GBY[4] 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Setting up reduce sink: RS[3] with following reduce work: Reducer 2 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Removing RS[3] as parent from GBY[4] 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: FS[6] 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:43,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:43,399 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:28:43,399 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping vectorization 2018-01-05T11:28:43,399 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:28:43,399 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: llap mode: only 2018-01-05T11:28:43,399 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:28:43,399 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(json, 'hll')' 2018-01-05T11:28:43,399 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'json' 2018-01-05T11:28:43,399 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:43,399 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'json' 2018-01-05T11:28:43,399 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:43,399 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@121f97fb can be done in LLAP 2018-01-05T11:28:43,399 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Reducer 2 2018-01-05T11:28:43,399 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col0)' 2018-01-05T11:28:43,399 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:28:43,399 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:43,399 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@70a3b can be done in LLAP 2018-01-05T11:28:43,399 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,399 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed plan generation 2018-01-05T11:28:43,399 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:43,399 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validation start 2018-01-05T11:28:43,399 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Not a partition. 2018-01-05T11:28:43,399 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:28:43,399 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,399 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0, type:struct, comment:null)], properties:null) 2018-01-05T11:28:43,399 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,399 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:43,399 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=10, flushCache_()=0, getTableColumnStatistics_(String, String, List, )=15} 2018-01-05T11:28:43,399 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112843_0b779e91-2298-4910-af0f-02be4fa91eae); Time taken: 0.068 seconds 2018-01-05T11:28:43,399 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:43,399 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,399 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112843_0b779e91-2298-4910-af0f-02be4fa91eae): ANALYZE TABLE src_json COMPUTE STATISTICS FOR COLUMNS json 2018-01-05T11:28:43,400 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,400 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: ANALYZE TABLE src_json COMPUTE STATISTICS FOR COLUMNS json 2018-01-05T11:28:43,400 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:28:43,400 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@src_json 2018-01-05T11:28:43,400 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@src_json 2018-01-05T11:28:43,400 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_336_6588790997440458878-1/-mr-10000 2018-01-05T11:28:43,400 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,400 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,400 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,400 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Query ID = djaiswal_20180105112843_0b779e91-2298-4910-af0f-02be4fa91eae 2018-01-05T11:28:43,400 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Total jobs = 1 2018-01-05T11:28:43,400 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,401 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:28:43,401 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MAPRED] in serial mode 2018-01-05T11:28:43,406 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000007, QueryId=djaiswal_20180105112843_74d78b4b-f0c4-4916-9a9e-407949a276eb-5, DagId=5, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-3, Succeeded=true, StartTime=1515180523261, EndTime=1515180523406 2018-01-05T11:28:43,422 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_331_9091869484395642489-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:28:43,443 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:28:43,443 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:28:43,443 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:28:43,443 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112843_0b779e91-2298-4910-af0f-02be4fa91eae 2018-01-05T11:28:43,443 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Session is already open 2018-01-05T11:28:43,443 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:28:43,443 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,443 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Dag name: ANALYZE TABLE src_json COMPUTE STATIS...json (Stage-0) 2018-01-05T11:28:43,443 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: DagInfo: {"context":"Hive","description":"\n\nANALYZE TABLE src_json COMPUTE STATISTICS FOR COLUMNS json"} 2018-01-05T11:28:43,444 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112843_0b779e91-2298-4910-af0f-02be4fa91eae with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:28:43,444 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,455 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,455 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:28:43,456 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,456 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 2 size: 1.84KB 2018-01-05T11:28:43,474 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,474 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,488 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:28:43,488 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,488 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:28:43,490 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,490 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 2.95KB 2018-01-05T11:28:43,497 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_336_6588790997440458878-1/-mr-10001 2018-01-05T11:28:43,508 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_336_6588790997440458878-1/-mr-10001 2018-01-05T11:28:43,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json 2018-01-05T11:28:43,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,512 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,513 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,513 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=ANALYZE TABLE src_json COMPUTE STATIS...json (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112843_0b779e91-2298-4910-af0f-02be4fa91eae } 2018-01-05T11:28:43,513 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:43,513 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:43,517 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: ANALYZE TABLE src_json COMPUTE STATIS...json (Stage-0) 2018-01-05T11:28:43,517 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:28:43,517 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_6, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_6.dot 2018-01-05T11:28:43,518 INFO [IPC Server handler 1 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_6 2018-01-05T11:28:43,518 WARN [IPC Server handler 1 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:43,518 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Running DAG: ANALYZE TABLE src_json COMPUTE STATIS...json (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112843_0b779e91-2298-4910-af0f-02be4fa91eae } 2018-01-05T11:28:43,541 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_6][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_6, submitTime=1515180523517, queueName=null 2018-01-05T11:28:43,545 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_6_00 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_6_00 [Map 1], Default Resources= 2018-01-05T11:28:43,545 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_6_00 [Map 1] 2018-01-05T11:28:43,545 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:28:43,545 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_6_00 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:43,546 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_6_01 [Reducer 2]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_6_01 [Reducer 2], Default Resources= 2018-01-05T11:28:43,546 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_6_01 [Reducer 2] 2018-01-05T11:28:43,546 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 2 2018-01-05T11:28:43,546 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_6_01 [Reducer 2] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:43,546 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:28:43,546 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_6][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_6, initTime=1515180523546 2018-01-05T11:28:43,546 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: dag_1515180518813_0001_6 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:28:43,547 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:28:43,547 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_6][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_6, startTime=1515180523547 2018-01-05T11:28:43,547 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_6 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:28:43,547 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {src_json={InputName=src_json}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:28:43,547 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: src_json, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:28:43,547 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_6_00 [Map 1] 2018-01-05T11:28:43,547 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_6, dagName=ANALYZE TABLE src_json COMPUTE STATIS...json (Stage-0) 2018-01-05T11:28:43,547 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,547 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,547 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,553 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_6_00 [Map 1] 2018-01-05T11:28:43,553 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_6_00 [Map 1] 2018-01-05T11:28:43,553 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:43,553 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_6_00 [Map 1] 2018-01-05T11:28:43,556 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:28:43,556 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@464a55c9 2018-01-05T11:28:43,557 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_331_9091869484395642489-1/djaiswal/_tez_scratch_dir/e102f956-f39a-4a9a-a4e2-8b94f164e121/map.xml 2018-01-05T11:28:43,557 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_331_9091869484395642489-1/djaiswal/_tez_scratch_dir/e102f956-f39a-4a9a-a4e2-8b94f164e121/map.xml 2018-01-05T11:28:43,557 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_331_9091869484395642489-1/djaiswal/_tez_scratch_dir/e102f956-f39a-4a9a-a4e2-8b94f164e121/map.xml 2018-01-05T11:28:43,557 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:43,557 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:43,557 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:43,558 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:43,559 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 2.95KB 2018-01-05T11:28:43,559 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_6_00 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:28:43,559 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: src_json on vertex vertex_1515180518813_0001_6_00 [Map 1] 2018-01-05T11:28:43,559 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:28:43,560 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to ShuffleVertexManager for vertex_1515180518813_0001_6_01 [Reducer 2] 2018-01-05T11:28:43,560 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:43,560 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:28:43,560 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:43,560 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_331_9091869484395642489-1/djaiswal/_tez_scratch_dir/e102f956-f39a-4a9a-a4e2-8b94f164e121/map.xml 2018-01-05T11:28:43,560 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:43,560 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias src_json 2018-01-05T11:28:43,560 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json 2018-01-05T11:28:43,560 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json 2018-01-05T11:28:43,561 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 0 2018-01-05T11:28:43,561 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = json 2018-01-05T11:28:43,561 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json 2018-01-05T11:28:43,562 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:43,562 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:43,562 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:43,562 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:43,567 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.05 maxFrac: 0.75 auto: false desiredTaskIput: 104857600 2018-01-05T11:28:43,567 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:28:43,567 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 1 tasks for vertex: vertex_1515180518813_0001_6_01 [Reducer 2] 2018-01-05T11:28:43,567 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_6_01 [Reducer 2] 2018-01-05T11:28:43,567 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_6][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_6_01, reconfigureDoneTime=1515180523567, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:28:43,567 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_6_01 [Reducer 2], numAdditionalOutputs=1 2018-01-05T11:28:43,567 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_6][Event:VERTEX_INITIALIZED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_6_01, initRequestedTime=1515180523560, initedTime=1515180523567, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:43,567 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_6_01 [Reducer 2] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:28:43,578 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Time taken to get FileStatuses: 15 2018-01-05T11:28:43,578 INFO [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:28:43,578 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 15 2018-01-05T11:28:43,578 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 1 2018-01-05T11:28:43,578 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:43,578 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 1. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:28:43,578 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_331_9091869484395642489-1/djaiswal/_tez_scratch_dir/e102f956-f39a-4a9a-a4e2-8b94f164e121/map.xml 2018-01-05T11:28:43,578 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_331_9091869484395642489-1/djaiswal/_tez_scratch_dir/e102f956-f39a-4a9a-a4e2-8b94f164e121/map.xml 2018-01-05T11:28:43,578 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_331_9091869484395642489-1/djaiswal/_tez_scratch_dir/e102f956-f39a-4a9a-a4e2-8b94f164e121/map.xml 2018-01-05T11:28:43,578 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:43,578 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:43,578 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:43,580 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:43,580 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 2.95KB 2018-01-05T11:28:43,580 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json/json.txt to src new group? true 2018-01-05T11:28:43,580 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:28:43,580 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:28:43,580 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:28:43,580 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 11 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 645 Original splits: 1 2018-01-05T11:28:43,580 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Using original number of splits: 1 desired splits: 1 2018-01-05T11:28:43,580 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 1 grouped split count is 1, for bucket: 1 2018-01-05T11:28:43,580 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:28:43,580 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: src_json on vertex vertex_1515180518813_0001_6_00 [Map 1] 2018-01-05T11:28:43,581 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_6_00 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:43,581 INFO [App Shared Pool - #0] impl.VertexImpl: Got updated RootInputsSpecs: {src_json=forAllWorkUnits=true, update=[1]} 2018-01-05T11:28:43,581 INFO [App Shared Pool - #0] impl.VertexImpl: Vertex vertex_1515180518813_0001_6_00 [Map 1] parallelism set to 1 2018-01-05T11:28:43,581 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_6_00 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:43,581 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_6][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_6_00, initRequestedTime=1515180523547, initedTime=1515180523581, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:43,581 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_6_00 [Map 1] with distanceFromRoot: 0 2018-01-05T11:28:43,581 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_6_00 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:28:43,581 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_6][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_6_00, startRequestedTime=1515180523559, startedTime=1515180523581 2018-01-05T11:28:43,581 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_6][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_6_00, reconfigureDoneTime=1515180523581, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:28:43,582 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_6_00 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:43,582 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_6_00 for vertex: vertex_1515180518813_0001_6_01 [Reducer 2] numStartedSources: 1 numSources: 1 2018-01-05T11:28:43,582 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_6_01 [Reducer 2] with distanceFromRoot: 1 2018-01-05T11:28:43,582 INFO [App Shared Pool - #0] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:28:43,582 INFO [App Shared Pool - #0] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:28:43,582 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_6_00 [Map 1] 2018-01-05T11:28:43,582 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_6][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_6_01, startRequestedTime=1515180523582, startedTime=1515180523582 2018-01-05T11:28:43,582 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_6_01 [Reducer 2] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:43,582 INFO [App Shared Pool - #1] vertexmanager.ShuffleVertexManagerBase: OnVertexStarted vertex: Reducer 2 with 0 source tasks and 1 pending tasks 2018-01-05T11:28:43,582 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_6][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_6_00_000000, scheduledTime=1515180523582, launchTime=1515180523582 2018-01-05T11:28:43,583 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_6_00 [Map 1] 2018-01-05T11:28:43,583 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Map 1 in vertex: Reducer 2 numBipartiteSourceTasks: 1 2018-01-05T11:28:43,583 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_6_00_000000_0, priority=5, capability=, hosts=null 2018-01-05T11:28:43,583 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=5,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:43,583 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=5,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:43,583 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:43,583 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_6_00_000000_0 on hosts=null 2018-01-05T11:28:43,583 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_6_00_000000_0 on hosts=null 2018-01-05T11:28:43,583 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_6_00_000000_0 2018-01-05T11:28:43,583 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:43,583 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #8, task=TaskInfo{task=attempt_1515180518813_0001_6_00_000000_0, priority=5, startTime=0, containerId=null, uniqueId=7, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000008 2018-01-05T11:28:43,583 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_6_00_000000_0; false 2018-01-05T11:28:43,583 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_6_00_000000_0, priority=5, startTime=311544092, containerId=container_111101111_0001_00_000008assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=7, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:43,585 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000008 for node: 10.22.8.182:49818 2018-01-05T11:28:43,587 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 6, name=ANALYZE TABLE src_json COMPUTE STATIS...json (Stage-0), queryId=djaiswal_20180105112843_0b779e91-2298-4910-af0f-02be4fa91eae 2018-01-05T11:28:43,587 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000008, attempt_1515180518813_0001_6_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:43,587 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000008 for node: 10.22.8.182:49818 2018-01-05T11:28:43,587 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000008 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:43,587 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000008, launchTime=1515180523587 2018-01-05T11:28:43,588 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_6_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112843_0b779e91-2298-4910-af0f-02be4fa91eae, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000008, dagName=ANALYZE TABLE src_json COMPUTE STATIS...json (Stage-0), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{src_json,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{Reducer 2,org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput,1}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180523547, firstAttemptStartTime=1515180523583, currentAttemptStartTime=1515180523587} 2018-01-05T11:28:43,588 INFO [IPC Server handler 0 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000008, QueryId=djaiswal_20180105112843_0b779e91-2298-4910-af0f-02be4fa91eae-6, DagId=6, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180523588 2018-01-05T11:28:43,588 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_6_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000008] on NM: [10.22.8.182:49818] 2018-01-05T11:28:43,588 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:43,588 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:43,588 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_6][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_6_00_000000_0, startTime=1515180523587, containerId=container_111101111_0001_00_000008, nodeId=10.22.8.182:49818 2018-01-05T11:28:43,588 DEBUG [IPC Server handler 0 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=6} with the ShuffleHandler 2018-01-05T11:28:43,601 INFO [IPC Server handler 0 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:28:43,601 DEBUG [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/6] 2018-01-05T11:28:43,602 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:43,602 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_6_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:43,602 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_6_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180523583, dagStartTime=1515180523547, withinDagPriority=5, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:28:43,602 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_6_00_000000_0 : ACCEPTED 2018-01-05T11:28:43,602 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_6_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:43,602 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_6_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180523583, dagStartTime=1515180523547, withinDagPriority=5, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:28:43,602 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_6_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:43,602 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_6_00_000000_0 to pre-emption queue 2018-01-05T11:28:43,602 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_6_00_000000_0: true 2018-01-05T11:28:43,602 INFO [TaskCommunicator # 4] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_6_00_000000_0 2018-01-05T11:28:43,602 INFO [TaskCommunicator # 4] GuaranteedTasks: Received response for attempt_1515180518813_0001_6_00_000000_0, true 2018-01-05T11:28:43,603 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: taskOwner hashCode:1076396530 2018-01-05T11:28:43,604 INFO [Task-Executor-3] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : ANALYZE TABLE src_json COMPUTE STATIS...json (Stage-0), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_6_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=src_json, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=Reducer 2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput }}, ] 2018-01-05T11:28:43,604 INFO [Task-Executor-3] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=954728448, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:43,604 INFO [Task-Executor-3] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_6_00_000000_0 vertexName: Map 1 2018-01-05T11:28:43,604 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000008, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_6_00_000000_0, eventCount=1 } 2018-01-05T11:28:43,604 DEBUG [TezTR-518813_1_6_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_6 queryId: djaiswal_20180105112843_0b779e91-2298-4910-af0f-02be4fa91eae instanceType: TaskRunner2Callable 2018-01-05T11:28:43,604 INFO [TezTR-518813_1_6_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_6_00_000000_0 2018-01-05T11:28:43,605 DEBUG [TezTR-518813_1_6_0_0_0] log.PerfLogger: 2018-01-05T11:28:43,605 INFO [I/O Setup 1 Initialize: {Reducer 2}] writers.UnorderedPartitionedKVWriter: Requested BufferSize (tez.runtime.unordered.output.buffer.size-mb) : 100 2018-01-05T11:28:43,605 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:43,607 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:43,610 DEBUG [TezTR-518813_1_6_0_0_0] log.PerfLogger: 2018-01-05T11:28:43,610 INFO [TezTR-518813_1_6_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:43,610 INFO [TezTR-518813_1_6_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:43,610 INFO [TezTR-518813_1_6_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:43,611 INFO [I/O Setup 0 Initialize: {src_json}] input.MRInput: src_json using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:28:43,611 INFO [IPC Server handler 3 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_6_00_000000_0 sent events: (0-1). 2018-01-05T11:28:43,611 INFO [I/O Setup 0 Initialize: {src_json}] input.MRInputLegacy: src_json MRInputLegacy deferring initialization 2018-01-05T11:28:43,611 INFO [TezTR-518813_1_6_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:43,611 INFO [TezTR-518813_1_6_0_0_0] resources.MemoryDistributor: InitialRequests=[src_json:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [Reducer 2:OUTPUT:104857600:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput] 2018-01-05T11:28:43,611 INFO [TezTR-518813_1_6_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:43,611 INFO [TezTR-518813_1_6_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:43,611 INFO [TezTR-518813_1_6_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 104857600, TotalRequestedScaled: 5.24288E7, TotalJVMHeap: 954728448, TotalAvailable: 639668060, TotalRequested/TotalJVMHeap:0.11 2018-01-05T11:28:43,611 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:28:43,611 INFO [TezTR-518813_1_6_0_0_0] resources.MemoryDistributor: Allocations=[src_json:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput:OUTPUT:104857600:104857600] 2018-01-05T11:28:43,611 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_6_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:28:43,611 INFO [TezTR-518813_1_6_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:43,611 INFO [TezTR-518813_1_6_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: src_json being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:28:43,611 INFO [TezTR-518813_1_6_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:28:43,611 INFO [TezTR-518813_1_6_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:28:43,611 INFO [I/O Setup 1 Start: {src_json}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: src_json 2018-01-05T11:28:43,612 INFO [TezTR-518813_1_6_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:43,612 INFO [TezTR-518813_1_6_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_6_00_000000_0 2018-01-05T11:28:43,612 DEBUG [TezTR-518813_1_6_0_0_0] log.PerfLogger: 2018-01-05T11:28:43,612 DEBUG [TezTR-518813_1_6_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_6_00_000000_0_10022 2018-01-05T11:28:43,612 DEBUG [TezTR-518813_1_6_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:43,612 DEBUG [TezTR-518813_1_6_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_6_00_000000_0 2018-01-05T11:28:43,612 INFO [TezTR-518813_1_6_0_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112843_0b779e91-2298-4910-af0f-02be4fa91eae 2018-01-05T11:28:43,612 DEBUG [TezTR-518813_1_6_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:43,612 DEBUG [TezTR-518813_1_6_0_0_0] log.PerfLogger: 2018-01-05T11:28:43,612 DEBUG [TezTR-518813_1_6_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:43,612 DEBUG [TezTR-518813_1_6_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:43,612 DEBUG [TezTR-518813_1_6_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112843_0b779e91-2298-4910-af0f-02be4fa91eae_Map 1__MAP_PLAN__ 2018-01-05T11:28:43,612 INFO [TezTR-518813_1_6_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_331_9091869484395642489-1/djaiswal/_tez_scratch_dir/e102f956-f39a-4a9a-a4e2-8b94f164e121/map.xml 2018-01-05T11:28:43,612 DEBUG [TezTR-518813_1_6_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_331_9091869484395642489-1/djaiswal/_tez_scratch_dir/e102f956-f39a-4a9a-a4e2-8b94f164e121/map.xml 2018-01-05T11:28:43,612 DEBUG [TezTR-518813_1_6_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_331_9091869484395642489-1/djaiswal/_tez_scratch_dir/e102f956-f39a-4a9a-a4e2-8b94f164e121/map.xml 2018-01-05T11:28:43,613 DEBUG [TezTR-518813_1_6_0_0_0] log.PerfLogger: 2018-01-05T11:28:43,613 INFO [TezTR-518813_1_6_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:43,613 DEBUG [TezTR-518813_1_6_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:43,614 DEBUG [TezTR-518813_1_6_0_0_0] log.PerfLogger: 2018-01-05T11:28:43,614 INFO [TezTR-518813_1_6_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 2.95KB 2018-01-05T11:28:43,615 DEBUG [TezTR-518813_1_6_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:43,615 INFO [TezTR-518813_1_6_0_0_0] tez.MapRecordProcessor: The input names are: [src_json] 2018-01-05T11:28:43,615 INFO [TezTR-518813_1_6_0_0_0] input.MRInput: Initialized MRInput: src_json 2018-01-05T11:28:43,616 DEBUG [TezTR-518813_1_6_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 1 2018-01-05T11:28:43,616 INFO [TezTR-518813_1_6_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_331_9091869484395642489-1/djaiswal/_tez_scratch_dir/e102f956-f39a-4a9a-a4e2-8b94f164e121/map.xml 2018-01-05T11:28:43,616 DEBUG [TezTR-518813_1_6_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:43,616 DEBUG [TezTR-518813_1_6_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json/json.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json, columns=json, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=1, numFiles=1, serialization.ddl=struct src_json { string json}, transient_lastDdlTime=1515180523, rawDataSize=644, columns.comments=default, totalSize=645, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.src_json}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json, columns=json, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=1, numFiles=1, serialization.ddl=struct src_json { string json}, transient_lastDdlTime=1515180523, rawDataSize=644, columns.comments=default, totalSize=645, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.src_json}, baseFileName=src_json, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json, columns=json, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=1, numFiles=1, serialization.ddl=struct src_json { string json}, transient_lastDdlTime=1515180523, rawDataSize=644, columns.comments=default, totalSize=645, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.src_json}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json, columns=json, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=1, numFiles=1, serialization.ddl=struct src_json { string json}, transient_lastDdlTime=1515180523, rawDataSize=644, columns.comments=default, totalSize=645, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.src_json}, baseFileName=src_json, vectorPartitionDesc=null]} 2018-01-05T11:28:43,617 DEBUG [TezTR-518813_1_6_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:43,617 INFO [TezTR-518813_1_6_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:43,617 DEBUG [TezTR-518813_1_6_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:43,618 DEBUG [TezTR-518813_1_6_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:43,618 INFO [TezTR-518813_1_6_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_331_9091869484395642489-1/djaiswal/_tez_scratch_dir/e102f956-f39a-4a9a-a4e2-8b94f164e121/map.xml 2018-01-05T11:28:43,618 DEBUG [TezTR-518813_1_6_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:43,618 INFO [TezTR-518813_1_6_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_6_00_000000_0 2018-01-05T11:28:43,618 INFO [TezTR-518813_1_6_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [json] / schema.evolution.columns.types [string] (isAcidRead false) 2018-01-05T11:28:43,619 DEBUG [TezTR-518813_1_6_0_0_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:43,619 DEBUG [TezTR-518813_1_6_0_0_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:43,619 DEBUG [TezTR-518813_1_6_0_0_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:43,619 DEBUG [TezTR-518813_1_6_0_0_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:43,619 WARN [TezTR-518813_1_6_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json/json.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:28:43,619 DEBUG [TezTR-518813_1_6_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:28:43,619 WARN [TezTR-518813_1_6_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json/json.txt:0+645 2018-01-05T11:28:43,620 DEBUG [TezTR-518813_1_6_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json/json.txt 2018-01-05T11:28:43,620 INFO [TezTR-518813_1_6_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json/json.txt:0+645], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=645} 2018-01-05T11:28:43,620 INFO [TezTR-518813_1_6_0_0_0] input.MRInput: src_json initialized RecordReader from event 2018-01-05T11:28:43,620 DEBUG [TezTR-518813_1_6_0_0_0] tez.MapRecordProcessor: Starting Output: Reducer 2 2018-01-05T11:28:43,620 INFO [TezTR-518813_1_6_0_0_0] writers.BaseUnorderedPartitionedKVWriter: Instantiating Partitioner: [org.apache.tez.mapreduce.partition.MRPartitioner] 2018-01-05T11:28:43,643 INFO [TezTR-518813_1_6_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: numBuffers=2, sizePerBuffer=52428800, skipBuffers=true, numPartitions=1, availableMemory=104857600, maxSingleBufferSizeBytes=2147483647, pipelinedShuffle=false, isFinalMergeEnabled=true, numPartitions=1, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:28:43,643 INFO [TezTR-518813_1_6_0_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:28:43,643 INFO [TezTR-518813_1_6_0_0_0] exec.MapOperator: Initializing operator MAP[0] 2018-01-05T11:28:43,643 DEBUG [TezTR-518813_1_6_0_0_0] exec.MapOperator: Initialization Done 0 MAP 2018-01-05T11:28:43,643 DEBUG [TezTR-518813_1_6_0_0_0] exec.MapOperator: Operator 0 MAP initialized 2018-01-05T11:28:43,643 DEBUG [TezTR-518813_1_6_0_0_0] exec.MapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:28:43,644 DEBUG [TezTR-518813_1_6_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:43,644 DEBUG [TezTR-518813_1_6_0_0_0] exec.MapOperator: Adding alias src_json to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json 2018-01-05T11:28:43,644 DEBUG [TezTR-518813_1_6_0_0_0] exec.MapOperator: dump TS[0] struct 2018-01-05T11:28:43,644 INFO [TezTR-518813_1_6_0_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:28:43,644 INFO [TezTR-518813_1_6_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:43,644 DEBUG [TezTR-518813_1_6_0_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:28:43,644 INFO [TezTR-518813_1_6_0_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:28:43,644 DEBUG [TezTR-518813_1_6_0_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:28:43,644 DEBUG [TezTR-518813_1_6_0_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:28:43,644 DEBUG [TezTR-518813_1_6_0_0_0] exec.TableScanOperator: Initializing children of 0 TS 2018-01-05T11:28:43,644 DEBUG [TezTR-518813_1_6_0_0_0] exec.SelectOperator: Initializing child 1 SEL 2018-01-05T11:28:43,644 INFO [TezTR-518813_1_6_0_0_0] exec.SelectOperator: Initializing operator SEL[1] 2018-01-05T11:28:43,644 INFO [TezTR-518813_1_6_0_0_0] exec.SelectOperator: SELECT struct 2018-01-05T11:28:43,644 DEBUG [TezTR-518813_1_6_0_0_0] exec.SelectOperator: Initialization Done 1 SEL 2018-01-05T11:28:43,645 DEBUG [TezTR-518813_1_6_0_0_0] exec.SelectOperator: Operator 1 SEL initialized 2018-01-05T11:28:43,645 DEBUG [TezTR-518813_1_6_0_0_0] exec.SelectOperator: Initializing children of 1 SEL 2018-01-05T11:28:43,645 DEBUG [TezTR-518813_1_6_0_0_0] exec.GroupByOperator: Initializing child 2 GBY 2018-01-05T11:28:43,645 INFO [TezTR-518813_1_6_0_0_0] exec.GroupByOperator: Initializing operator GBY[2] 2018-01-05T11:28:43,645 DEBUG [TezTR-518813_1_6_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:43,645 INFO [TezTR-518813_1_6_0_0_0] exec.GroupByOperator: Max hash table memory: 93860656 bytes 2018-01-05T11:28:43,645 INFO [TezTR-518813_1_6_0_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:43,645 DEBUG [TezTR-518813_1_6_0_0_0] exec.GroupByOperator: Initialization Done 2 GBY 2018-01-05T11:28:43,645 DEBUG [TezTR-518813_1_6_0_0_0] exec.GroupByOperator: Operator 2 GBY initialized 2018-01-05T11:28:43,645 DEBUG [TezTR-518813_1_6_0_0_0] exec.GroupByOperator: Initializing children of 2 GBY 2018-01-05T11:28:43,645 DEBUG [TezTR-518813_1_6_0_0_0] exec.ReduceSinkOperator: Initializing child 3 RS 2018-01-05T11:28:43,645 INFO [TezTR-518813_1_6_0_0_0] exec.ReduceSinkOperator: Initializing operator RS[3] 2018-01-05T11:28:43,645 DEBUG [TezTR-518813_1_6_0_0_0] exec.ReduceSinkOperator: keys size is 0 2018-01-05T11:28:43,645 INFO [TezTR-518813_1_6_0_0_0] exec.ReduceSinkOperator: Using tag = -1 2018-01-05T11:28:43,645 DEBUG [TezTR-518813_1_6_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:43,645 DEBUG [TezTR-518813_1_6_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:43,645 DEBUG [TezTR-518813_1_6_0_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0] columnTypes=[struct] 2018-01-05T11:28:43,645 DEBUG [TezTR-518813_1_6_0_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS 2018-01-05T11:28:43,645 DEBUG [TezTR-518813_1_6_0_0_0] exec.ReduceSinkOperator: Operator 3 RS initialized 2018-01-05T11:28:43,645 DEBUG [TezTR-518813_1_6_0_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS done is reset. 2018-01-05T11:28:43,645 DEBUG [TezTR-518813_1_6_0_0_0] exec.GroupByOperator: Initialization Done 2 GBY done is reset. 2018-01-05T11:28:43,645 DEBUG [TezTR-518813_1_6_0_0_0] exec.SelectOperator: Initialization Done 1 SEL done is reset. 2018-01-05T11:28:43,645 DEBUG [TezTR-518813_1_6_0_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:28:43,645 INFO [TezTR-518813_1_6_0_0_0] exec.OperatorUtils: Setting output collector: RS[3] --> Reducer 2 2018-01-05T11:28:43,645 DEBUG [TezTR-518813_1_6_0_0_0] log.PerfLogger: 2018-01-05T11:28:43,645 INFO [TezTR-518813_1_6_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_331_9091869484395642489-1/djaiswal/_tez_scratch_dir/e102f956-f39a-4a9a-a4e2-8b94f164e121/map.xml 2018-01-05T11:28:43,645 DEBUG [TezTR-518813_1_6_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:43,645 DEBUG [TezTR-518813_1_6_0_0_0] exec.MapOperator: Processing alias(es) src_json for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json/json.txt 2018-01-05T11:28:43,645 INFO [TezTR-518813_1_6_0_0_0] exec.MapOperator: MAP[0]: records read - 1 2018-01-05T11:28:43,645 DEBUG [TezTR-518813_1_6_0_0_0] log.PerfLogger: 2018-01-05T11:28:43,645 DEBUG [TezTR-518813_1_6_0_0_0] mr.ObjectCache: djaiswal_20180105112843_0b779e91-2298-4910-af0f-02be4fa91eae_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:28:43,645 DEBUG [TezTR-518813_1_6_0_0_0] exec.MapOperator: close called for operator MAP[0] 2018-01-05T11:28:43,645 INFO [TezTR-518813_1_6_0_0_0] exec.MapOperator: Closing operator MAP[0] 2018-01-05T11:28:43,645 INFO [TezTR-518813_1_6_0_0_0] exec.MapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:1, 2018-01-05T11:28:43,645 DEBUG [TezTR-518813_1_6_0_0_0] exec.MapOperator: Closing child = TS[0] 2018-01-05T11:28:43,645 DEBUG [TezTR-518813_1_6_0_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:28:43,645 DEBUG [TezTR-518813_1_6_0_0_0] exec.TableScanOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:43,645 INFO [TezTR-518813_1_6_0_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:28:43,645 DEBUG [TezTR-518813_1_6_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:43,645 DEBUG [TezTR-518813_1_6_0_0_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_336_6588790997440458878-1/-mr-10001 2018-01-05T11:28:43,646 INFO [TezTR-518813_1_6_0_0_0] exec.TableScanOperator: publishing : default.src_json/ : {numRows=1, rawDataSize=644} 2018-01-05T11:28:43,669 DEBUG [TezTR-518813_1_6_0_0_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_336_6588790997440458878-1/-mr-10001/tmpstats-0_0 2018-01-05T11:28:43,669 DEBUG [TezTR-518813_1_6_0_0_0] fs.FSStatsPublisher: Writing stats in it : {default.src_json/={numRows=1, rawDataSize=644}} 2018-01-05T11:28:43,670 DEBUG [TezTR-518813_1_6_0_0_0] exec.TableScanOperator: Closing child = SEL[1] 2018-01-05T11:28:43,670 DEBUG [TezTR-518813_1_6_0_0_0] exec.SelectOperator: close called for operator SEL[1] 2018-01-05T11:28:43,670 DEBUG [TezTR-518813_1_6_0_0_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:43,670 INFO [TezTR-518813_1_6_0_0_0] exec.SelectOperator: Closing operator SEL[1] 2018-01-05T11:28:43,670 DEBUG [TezTR-518813_1_6_0_0_0] exec.SelectOperator: Closing child = GBY[2] 2018-01-05T11:28:43,670 DEBUG [TezTR-518813_1_6_0_0_0] exec.GroupByOperator: close called for operator GBY[2] 2018-01-05T11:28:43,670 DEBUG [TezTR-518813_1_6_0_0_0] exec.GroupByOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:43,670 INFO [TezTR-518813_1_6_0_0_0] exec.GroupByOperator: Closing operator GBY[2] 2018-01-05T11:28:43,670 INFO [TezTR-518813_1_6_0_0_0] exec.GroupByOperator: Begin Hash Table flush: size = 1 2018-01-05T11:28:43,670 INFO [TezTR-518813_1_6_0_0_0] exec.ReduceSinkOperator: keys are [] num distributions: 0 2018-01-05T11:28:43,670 DEBUG [TezTR-518813_1_6_0_0_0] exec.GroupByOperator: Closing child = RS[3] 2018-01-05T11:28:43,670 DEBUG [TezTR-518813_1_6_0_0_0] exec.ReduceSinkOperator: close called for operator RS[3] 2018-01-05T11:28:43,670 DEBUG [TezTR-518813_1_6_0_0_0] exec.ReduceSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:43,670 INFO [TezTR-518813_1_6_0_0_0] exec.ReduceSinkOperator: Closing operator RS[3] 2018-01-05T11:28:43,670 INFO [TezTR-518813_1_6_0_0_0] exec.ReduceSinkOperator: RECORDS_OUT_INTERMEDIATE_Map_1:1, 2018-01-05T11:28:43,670 DEBUG [TezTR-518813_1_6_0_0_0] exec.ReduceSinkOperator: 3 Close done 2018-01-05T11:28:43,670 DEBUG [TezTR-518813_1_6_0_0_0] exec.GroupByOperator: 2 Close done 2018-01-05T11:28:43,670 DEBUG [TezTR-518813_1_6_0_0_0] exec.SelectOperator: 1 Close done 2018-01-05T11:28:43,670 DEBUG [TezTR-518813_1_6_0_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:28:43,670 DEBUG [TezTR-518813_1_6_0_0_0] exec.MapOperator: 0 Close done 2018-01-05T11:28:43,670 INFO [TezTR-518813_1_6_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_6_00_000000_0 2018-01-05T11:28:43,670 INFO [TezTR-518813_1_6_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: Waiting for all spills to complete : Pending : 0 2018-01-05T11:28:43,670 INFO [TezTR-518813_1_6_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: All spills complete 2018-01-05T11:28:43,683 DEBUG [TezTR-518813_1_6_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:43,683 INFO [TezTR-518813_1_6_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:43,683 INFO [TezTR-518813_1_6_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_6_00_000000_0, askedToStop=false 2018-01-05T11:28:43,683 INFO [TezTR-518813_1_6_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_6_00_000000_0, stopRequested=false 2018-01-05T11:28:43,683 INFO [TezTR-518813_1_6_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_6_00_000000_0: Counters: 14 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=1, INPUT_SPLIT_LENGTH_BYTES=645, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=34, OUTPUT_BYTES_WITH_OVERHEAD=42, OUTPUT_BYTES_PHYSICAL=70, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=1, RECORDS_OUT_INTERMEDIATE_Map_1=1]] 2018-01-05T11:28:43,683 DEBUG [TezTR-518813_1_6_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_6_0_0_0 thread id: 507 scheme: file bytesRead: 665 bytesWritten: 163 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:43,684 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_6_00_000000_0, isTaskDone=true 2018-01-05T11:28:43,684 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000008, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_6_00_000000_0, eventCount=4 } 2018-01-05T11:28:43,685 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_6][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_6_00_000000_0, creationTime=1515180523582, allocationTime=1515180523585, startTime=1515180523587, finishTime=1515180523685, timeTaken=98, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:43,685 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:43,685 INFO [Task-Executor-3] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_6_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:43,685 INFO [Task-Executor-3] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_6_00_000000_0 2018-01-05T11:28:43,685 INFO [Task-Executor-3] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000008=83 2018-01-05T11:28:43,685 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_6_00_000000_0 2018-01-05T11:28:43,686 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_6_00_000000_0: true 2018-01-05T11:28:43,686 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_6_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:43,686 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_6_00_000000_0; false 2018-01-05T11:28:43,686 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_6_00_000000_0 2018-01-05T11:28:43,686 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_6_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:43,686 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000008, QueryId=djaiswal_20180105112843_0b779e91-2298-4910-af0f-02be4fa91eae-6, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:28:43,686 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_6_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:43,686 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_6_00_000000_0 2018-01-05T11:28:43,686 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:43,686 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:43,686 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_6][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_6_00_000000, startTime=1515180523587, finishTime=1515180523686, timeTaken=99, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_6_00_000000_0, diagnostics=, counters=Counters: 20, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=665, FILE_BYTES_WRITTEN=163, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=1, INPUT_SPLIT_LENGTH_BYTES=645, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=34, OUTPUT_BYTES_WITH_OVERHEAD=42, OUTPUT_BYTES_PHYSICAL=70, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=1, RECORDS_OUT_INTERMEDIATE_Map_1=1 2018-01-05T11:28:43,686 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_6_00 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:43,686 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_6_00 [Map 1] 2018-01-05T11:28:43,687 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_6][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_6_00, initRequestedTime=1515180523547, initedTime=1515180523581, startRequestedTime=1515180523559, startedTime=1515180523581, finishTime=1515180523686, timeTaken=105, status=SUCCEEDED, diagnostics=, counters=Counters: 20, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=665, FILE_BYTES_WRITTEN=163, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=1, INPUT_SPLIT_LENGTH_BYTES=645, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=34, OUTPUT_BYTES_WITH_OVERHEAD=42, OUTPUT_BYTES_PHYSICAL=70, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=1, RECORDS_OUT_INTERMEDIATE_Map_1=1, vertexStats=firstTaskStartTime=1515180523587, firstTasksToStart=[ task_1515180518813_0001_6_00_000000 ], lastTaskFinishTime=1515180523685, lastTasksToFinish=[ task_1515180518813_0001_6_00_000000 ], minTaskDuration=98, maxTaskDuration=98, avgTaskDuration=98.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_6_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_6_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:43,687 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_6_00 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:43,687 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_6][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000008, stoppedTime=1515180523687, exitStatus=0 2018-01-05T11:28:43,687 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000008 exited with diagnostics set to Container completed. 2018-01-05T11:28:43,687 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_6_01 [Reducer 2] attempt: attempt_1515180518813_0001_6_00_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:28:43,687 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_6_00 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2 2018-01-05T11:28:43,687 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null 2018-01-05T11:28:43,687 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: All source tasks completed. Ramping up 1 remaining tasks for vertex: Reducer 2 2018-01-05T11:28:43,687 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Scheduling 1 tasks for vertex: Reducer 2 with totalTasks: 1. 1 source tasks completed out of 1. MinSourceTaskCompletedFraction: 1.0 min: 0.05 max: 0.75 2018-01-05T11:28:43,687 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000008 2018-01-05T11:28:43,687 INFO [App Shared Pool - #0] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Map 1, destinationVertexName=Reducer 2. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.UnorderedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:28:43,687 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_6_01 [Reducer 2] 2018-01-05T11:28:43,688 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_6][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_6_01_000000, scheduledTime=1515180523688, launchTime=1515180523688 2018-01-05T11:28:43,688 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_6_01 [Reducer 2] 2018-01-05T11:28:43,688 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_6_01_000000_0, priority=14, capability=, hosts=null 2018-01-05T11:28:43,688 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:43,688 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:43,688 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:43,688 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_6_01_000000_0 on hosts=null 2018-01-05T11:28:43,688 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_6_01_000000_0 on hosts=null 2018-01-05T11:28:43,688 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_6_01_000000_0 2018-01-05T11:28:43,688 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:43,688 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #9, task=TaskInfo{task=attempt_1515180518813_0001_6_01_000000_0, priority=14, startTime=0, containerId=null, uniqueId=8, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000009 2018-01-05T11:28:43,688 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_6_01_000000_0; false 2018-01-05T11:28:43,688 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_6_01_000000_0, priority=14, startTime=311544197, containerId=container_111101111_0001_00_000009assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=8, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:43,689 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000009 for node: 10.22.8.182:49818 2018-01-05T11:28:43,690 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000009, attempt_1515180518813_0001_6_01_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:43,690 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000009 for node: 10.22.8.182:49818 2018-01-05T11:28:43,690 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000009 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:43,690 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=6}, sourceName=Map 1, state=S_SUCCEEDED 2018-01-05T11:28:43,690 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000009, launchTime=1515180523690 2018-01-05T11:28:43,691 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_6_01_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000009] on NM: [10.22.8.182:49818] 2018-01-05T11:28:43,691 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:43,691 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:43,691 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_6][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_6_01_000000_0, startTime=1515180523690, containerId=container_111101111_0001_00_000009, nodeId=10.22.8.182:49818 2018-01-05T11:28:43,691 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_6_01_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112843_0b779e91-2298-4910-af0f-02be4fa91eae, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000009, dagName=ANALYZE TABLE src_json COMPUTE STATIS...json (Stage-0), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.UnorderedKVInput,1}}, Outputs={{out_Reducer 2,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=2, completedTaskCount=1, dagStartTime=1515180523547, firstAttemptStartTime=1515180523688, currentAttemptStartTime=1515180523690} 2018-01-05T11:28:43,691 INFO [IPC Server handler 4 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000009, QueryId=djaiswal_20180105112843_0b779e91-2298-4910-af0f-02be4fa91eae-6, DagId=6, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, SubmitTime=1515180523691 2018-01-05T11:28:43,691 DEBUG [IPC Server handler 4 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=6} with the ShuffleHandler 2018-01-05T11:28:43,691 DEBUG [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/6] 2018-01-05T11:28:43,691 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:43,692 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_6_01_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:43,692 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_6_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180523688, dagStartTime=1515180523547, withinDagPriority=14, vertexParallelism= 1, selfAndUpstreamParallelism= 2, selfAndUpstreamComplete= 1}] 2018-01-05T11:28:43,692 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_6_01_000000_0 : ACCEPTED 2018-01-05T11:28:43,692 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_6_01_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:43,692 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_6_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180523688, dagStartTime=1515180523547, withinDagPriority=14, vertexParallelism= 1, selfAndUpstreamParallelism= 2, selfAndUpstreamComplete= 1} 2018-01-05T11:28:43,692 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_6_01_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:43,692 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_6_01_000000_0 to pre-emption queue 2018-01-05T11:28:43,692 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_6_01_000000_0: true 2018-01-05T11:28:43,692 INFO [TaskCommunicator # 6] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_6_01_000000_0 2018-01-05T11:28:43,692 INFO [TaskCommunicator # 6] GuaranteedTasks: Received response for attempt_1515180518813_0001_6_01_000000_0, true 2018-01-05T11:28:43,692 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: taskOwner hashCode:1076396530 2018-01-05T11:28:43,692 INFO [Task-Executor-3] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : ANALYZE TABLE src_json COMPUTE STATIS...json (Stage-0), VertexName: Reducer 2, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_6_01_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:28:43,692 INFO [Task-Executor-3] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=954728448, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:43,692 INFO [Task-Executor-3] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_6_01_000000_0 vertexName: Reducer 2 2018-01-05T11:28:43,693 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000009, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_6_01_000000_0, eventCount=1 } 2018-01-05T11:28:43,693 DEBUG [TezTR-518813_1_6_1_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_6 queryId: djaiswal_20180105112843_0b779e91-2298-4910-af0f-02be4fa91eae instanceType: TaskRunner2Callable 2018-01-05T11:28:43,693 INFO [TezTR-518813_1_6_1_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_6_01_000000_0 2018-01-05T11:28:43,693 DEBUG [TezTR-518813_1_6_1_0_0] log.PerfLogger: 2018-01-05T11:28:43,693 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:43,695 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:43,698 INFO [IPC Server handler 2 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_6_01_000000_0 1 events [0,1) total 1 vertex_1515180518813_0001_6_01 [Reducer 2] 2018-01-05T11:28:43,699 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:28:43,699 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_6_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:28:43,699 DEBUG [TezTR-518813_1_6_1_0_0] log.PerfLogger: 2018-01-05T11:28:43,699 INFO [TezTR-518813_1_6_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:43,699 INFO [TezTR-518813_1_6_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:43,699 INFO [TezTR-518813_1_6_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:43,700 INFO [I/O Setup 1 Initialize: {out_Reducer 2}] output.MROutput: out_Reducer 2: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:28:43,700 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:43,700 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:43,700 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:43,700 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:43,700 INFO [TezTR-518813_1_6_1_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:43,700 INFO [TezTR-518813_1_6_1_0_0] resources.MemoryDistributor: InitialRequests=[out_Reducer 2:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.UnorderedKVInput] 2018-01-05T11:28:43,700 INFO [TezTR-518813_1_6_1_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:43,700 INFO [TezTR-518813_1_6_1_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:43,700 INFO [TezTR-518813_1_6_1_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 131404920, TotalRequestedScaled: 6.570246E7, TotalJVMHeap: 954728448, TotalAvailable: 639668060, TotalRequested/TotalJVMHeap:0.14 2018-01-05T11:28:43,700 INFO [TezTR-518813_1_6_1_0_0] resources.MemoryDistributor: Allocations=[out_Reducer 2:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [Map 1:org.apache.tez.runtime.library.input.UnorderedKVInput:INPUT:131404920:131404920] 2018-01-05T11:28:43,700 INFO [TezTR-518813_1_6_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:43,701 INFO [TezTR-518813_1_6_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 0 2018-01-05T11:28:43,701 INFO [TezTR-518813_1_6_1_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:43,701 INFO [TezTR-518813_1_6_1_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_6_01_000000_0 2018-01-05T11:28:43,701 DEBUG [TezTR-518813_1_6_1_0_0] log.PerfLogger: 2018-01-05T11:28:43,701 DEBUG [TezTR-518813_1_6_1_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_6_01_000000_0_10025 2018-01-05T11:28:43,701 DEBUG [TezTR-518813_1_6_1_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:43,701 DEBUG [TezTR-518813_1_6_1_0_0] mr.ObjectCache: Creating djaiswal_20180105112843_0b779e91-2298-4910-af0f-02be4fa91eae_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:28:43,701 INFO [TezTR-518813_1_6_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_331_9091869484395642489-1/djaiswal/_tez_scratch_dir/d123f65a-533d-41ff-b213-fe18a8127cc1/reduce.xml 2018-01-05T11:28:43,701 DEBUG [TezTR-518813_1_6_1_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_331_9091869484395642489-1/djaiswal/_tez_scratch_dir/d123f65a-533d-41ff-b213-fe18a8127cc1/reduce.xml 2018-01-05T11:28:43,701 DEBUG [TezTR-518813_1_6_1_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_331_9091869484395642489-1/djaiswal/_tez_scratch_dir/d123f65a-533d-41ff-b213-fe18a8127cc1/reduce.xml 2018-01-05T11:28:43,701 DEBUG [TezTR-518813_1_6_1_0_0] log.PerfLogger: 2018-01-05T11:28:43,701 INFO [TezTR-518813_1_6_1_0_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:28:43,701 DEBUG [TezTR-518813_1_6_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:43,702 DEBUG [TezTR-518813_1_6_1_0_0] log.PerfLogger: 2018-01-05T11:28:43,703 INFO [TezTR-518813_1_6_1_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 1.84KB 2018-01-05T11:28:43,703 DEBUG [TezTR-518813_1_6_1_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:43,703 DEBUG [TezTR-518813_1_6_1_0_0] log.PerfLogger: 2018-01-05T11:28:43,703 DEBUG [TezTR-518813_1_6_1_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:43,703 DEBUG [TezTR-518813_1_6_1_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:43,706 DEBUG [TezTR-518813_1_6_1_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:43,706 INFO [TezTR-518813_1_6_1_0_0] impl.SimpleFetchedInputAllocator: Map_1: RequestedMemory=131404920, AssignedMemory=131404920, maxSingleShuffleLimit=32851230 2018-01-05T11:28:43,707 INFO [TezTR-518813_1_6_1_0_0] impl.ShuffleManager: Map_1: numInputs=1, compressionCodec=NoCompressionCodec, numFetchers=1, ifileBufferSize=4096, ifileReadAheadEnabled=true, ifileReadAheadLength=4194304, localDiskFetchEnabled=false, sharedFetchEnabled=false, keepAlive=false, keepAliveMaxConnections=20, connectionTimeout=20000, readTimeout=180000, bufferSize=8192, bufferSize=8192, maxTaskOutputAtOnce=20 2018-01-05T11:28:43,707 INFO [TezTR-518813_1_6_1_0_0] impl.ShuffleInputEventHandlerImpl: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:28:43,707 INFO [TezTR-518813_1_6_1_0_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:28:43,708 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=6&reduce=0&map=attempt_1515180518813_0001_6_00_000000_0_10024 mapId: [attempt_1515180518813_0001_6_00_000000_0_10024] reduceId: [0] jobId: [job_1515180518813_0001] dagId: [6] keepAlive: false 2018-01-05T11:28:43,708 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=6&reduce=0&map=attempt_1515180518813_0001_6_00_000000_0_10024; hash=...IeGulyMTngGsA 2018-01-05T11:28:43,709 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=6&reduce=0&map=attempt_1515180518813_0001_6_00_000000_0_10024;reply=PwvrY4fLvTgLY 2018-01-05T11:28:43,709 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=6, user='djaiswal', attemptId='attempt_1515180518813_0001_6_00_000000_0_10024'} via loader 2018-01-05T11:28:43,709 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=6, user='djaiswal', attemptId='attempt_1515180518813_0001_6_00_000000_0_10024'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:28:43,709 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_6_00_000000_0_10024 not found 2018-01-05T11:28:43,709 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_6_00_000000_0_10024,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/6/output/attempt_1515180518813_0001_6_00_000000_0_10024/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/6/output/attempt_1515180518813_0001_6_00_000000_0_10024/file.out.index 2018-01-05T11:28:43,709 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Content Length in shuffle : 96 2018-01-05T11:28:43,710 INFO [Fetcher_B {Map_1} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=6&reduce=0&map=attempt_1515180518813_0001_6_00_000000_0_10024 sent hash and receievd reply 0 ms 2018-01-05T11:28:43,710 INFO [Fetcher_B {Map_1} #0] ShuffleManager.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_6_00_000000_0_10024} to MEMORY, csize=46, dsize=42, EndTime=1515180523710, TimeTaken=0, Rate=0.00 MB/s 2018-01-05T11:28:43,710 INFO [Fetcher_B {Map_1} #0] impl.ShuffleManager: All inputs fetched for input vertex : Map 1 2018-01-05T11:28:43,710 INFO [TezTR-518813_1_6_1_0_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:28:43,710 INFO [TezTR-518813_1_6_1_0_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:43,710 INFO [Fetcher_B {Map_1} #0] impl.ShuffleManager: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:28:43,710 DEBUG [TezTR-518813_1_6_1_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:28:43,710 INFO [ShuffleRunner {Map_1}] impl.ShuffleManager: Map_1: Shutting down FetchScheduler, Was Interrupted: false 2018-01-05T11:28:43,710 DEBUG [TezTR-518813_1_6_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:43,710 INFO [ShuffleRunner {Map_1}] impl.ShuffleManager: Map_1: Scheduler thread completed 2018-01-05T11:28:43,710 DEBUG [TezTR-518813_1_6_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:43,710 DEBUG [TezTR-518813_1_6_1_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0] columnTypes=[struct] 2018-01-05T11:28:43,710 DEBUG [TezTR-518813_1_6_1_0_0] log.PerfLogger: 2018-01-05T11:28:43,710 INFO [TezTR-518813_1_6_1_0_0] exec.GroupByOperator: Initializing operator GBY[4] 2018-01-05T11:28:43,710 DEBUG [TezTR-518813_1_6_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:43,710 INFO [TezTR-518813_1_6_1_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:43,710 DEBUG [TezTR-518813_1_6_1_0_0] exec.GroupByOperator: Initialization Done 4 GBY 2018-01-05T11:28:43,710 DEBUG [TezTR-518813_1_6_1_0_0] exec.GroupByOperator: Operator 4 GBY initialized 2018-01-05T11:28:43,710 DEBUG [TezTR-518813_1_6_1_0_0] exec.GroupByOperator: Initializing children of 4 GBY 2018-01-05T11:28:43,710 DEBUG [TezTR-518813_1_6_1_0_0] exec.FileSinkOperator: Initializing child 6 FS 2018-01-05T11:28:43,710 INFO [TezTR-518813_1_6_1_0_0] exec.FileSinkOperator: Initializing operator FS[6] 2018-01-05T11:28:43,711 DEBUG [TezTR-518813_1_6_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:43,711 INFO [TezTR-518813_1_6_1_0_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@1054e1bb]:[_col0]:[struct]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@1114026a 2018-01-05T11:28:43,711 DEBUG [TezTR-518813_1_6_1_0_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:28:43,711 DEBUG [TezTR-518813_1_6_1_0_0] exec.FileSinkOperator: Initialization Done 6 FS 2018-01-05T11:28:43,711 DEBUG [TezTR-518813_1_6_1_0_0] exec.FileSinkOperator: Operator 6 FS initialized 2018-01-05T11:28:43,711 DEBUG [TezTR-518813_1_6_1_0_0] exec.FileSinkOperator: Initialization Done 6 FS done is reset. 2018-01-05T11:28:43,711 DEBUG [TezTR-518813_1_6_1_0_0] exec.GroupByOperator: Initialization Done 4 GBY done is reset. 2018-01-05T11:28:43,711 INFO [TezTR-518813_1_6_1_0_0] tez.ReduceRecordProcessor: Id =4 Id =6 <\Children> Id = 4 null<\Parent> <\FS> <\Children> <\Parent> <\GBY> 2018-01-05T11:28:43,711 DEBUG [TezTR-518813_1_6_1_0_0] log.PerfLogger: 2018-01-05T11:28:43,711 INFO [TezTR-518813_1_6_1_0_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 2 2018-01-05T11:28:43,711 DEBUG [TezTR-518813_1_6_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:28:43,711 DEBUG [TezTR-518813_1_6_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:28:43,711 DEBUG [TezTR-518813_1_6_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:28:43,711 DEBUG [TezTR-518813_1_6_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:28:43,711 DEBUG [TezTR-518813_1_6_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:28:43,711 DEBUG [TezTR-518813_1_6_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:28:43,711 INFO [TezTR-518813_1_6_1_0_0] readers.UnorderedKVReader: Num Records read: 1 2018-01-05T11:28:43,711 DEBUG [TezTR-518813_1_6_1_0_0] log.PerfLogger: 2018-01-05T11:28:43,711 DEBUG [TezTR-518813_1_6_1_0_0] mr.ObjectCache: djaiswal_20180105112843_0b779e91-2298-4910-af0f-02be4fa91eae_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:28:43,711 DEBUG [TezTR-518813_1_6_1_0_0] exec.GroupByOperator: close called for operator GBY[4] 2018-01-05T11:28:43,711 INFO [TezTR-518813_1_6_1_0_0] exec.GroupByOperator: Closing operator GBY[4] 2018-01-05T11:28:43,712 INFO [TezTR-518813_1_6_1_0_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_336_6588790997440458878-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-43_336_6588790997440458878-1/_tmp.-ext-10002/000000_0 2018-01-05T11:28:43,712 INFO [TezTR-518813_1_6_1_0_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_336_6588790997440458878-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-43_336_6588790997440458878-1/_task_tmp.-ext-10002/_tmp.000000_0 2018-01-05T11:28:43,712 INFO [TezTR-518813_1_6_1_0_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_336_6588790997440458878-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-43_336_6588790997440458878-1/_tmp.-ext-10002/000000_0 2018-01-05T11:28:43,746 INFO [TezTR-518813_1_6_1_0_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:28:43,746 DEBUG [TezTR-518813_1_6_1_0_0] exec.GroupByOperator: Closing child = FS[6] 2018-01-05T11:28:43,746 DEBUG [TezTR-518813_1_6_1_0_0] exec.FileSinkOperator: close called for operator FS[6] 2018-01-05T11:28:43,746 DEBUG [TezTR-518813_1_6_1_0_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:43,746 INFO [TezTR-518813_1_6_1_0_0] exec.FileSinkOperator: Closing operator FS[6] 2018-01-05T11:28:43,746 INFO [TezTR-518813_1_6_1_0_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:28:43,747 INFO [TezTR-518813_1_6_1_0_0] exec.FileSinkOperator: RECORDS_OUT_0:1, 2018-01-05T11:28:43,747 DEBUG [TezTR-518813_1_6_1_0_0] exec.FileSinkOperator: 6 Close done 2018-01-05T11:28:43,747 DEBUG [TezTR-518813_1_6_1_0_0] exec.GroupByOperator: 4 Close done 2018-01-05T11:28:43,747 INFO [TezTR-518813_1_6_1_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_6_01_000000_0 2018-01-05T11:28:43,747 INFO [TezTR-518813_1_6_1_0_0] impl.ShuffleInputEventHandlerImpl: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:28:43,747 INFO [TezTR-518813_1_6_1_0_0] impl.ShuffleManager: Shutting down pending fetchers on sourceMap_1: 0 2018-01-05T11:28:43,747 INFO [TezTR-518813_1_6_1_0_0] output.MROutput: out_Reducer 2 closed 2018-01-05T11:28:43,748 DEBUG [TezTR-518813_1_6_1_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:43,748 INFO [TezTR-518813_1_6_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:43,748 INFO [TezTR-518813_1_6_1_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_6_01_000000_0, askedToStop=false 2018-01-05T11:28:43,748 INFO [TezTR-518813_1_6_1_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_6_01_000000_0, stopRequested=false 2018-01-05T11:28:43,748 INFO [TezTR-518813_1_6_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_6_01_000000_0: Counters: 14 [[org.apache.tez.common.counters.TaskCounter NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=46, SHUFFLE_BYTES_DECOMPRESSED=42, SHUFFLE_BYTES_TO_MEM=46, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=4, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1][HIVE CREATED_FILES=1, RECORDS_OUT_0=1]] 2018-01-05T11:28:43,748 DEBUG [TezTR-518813_1_6_1_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_6_1_0_0 thread id: 519 scheme: file bytesRead: 0 bytesWritten: 149 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:43,749 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_6_01_000000_0, isTaskDone=true 2018-01-05T11:28:43,749 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000009, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_6_01_000000_0, eventCount=2 } 2018-01-05T11:28:43,750 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_6][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_6_01_000000_0, creationTime=1515180523688, allocationTime=1515180523689, startTime=1515180523690, finishTime=1515180523750, timeTaken=60, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:43,750 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_6_01_000000_0 2018-01-05T11:28:43,750 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:28:43,750 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_6_01_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:43,750 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_6_01_000000_0; false 2018-01-05T11:28:43,750 INFO [Task-Executor-3] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_6_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:43,751 INFO [Task-Executor-3] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_6_01_000000_0 2018-01-05T11:28:43,751 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_6_01_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:43,751 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:43,751 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:43,751 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_6][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_6_01_000000, startTime=1515180523690, finishTime=1515180523750, timeTaken=60, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_6_01_000000_0, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=149, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=46, SHUFFLE_BYTES_DECOMPRESSED=42, SHUFFLE_BYTES_TO_MEM=46, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=4, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1 2018-01-05T11:28:43,751 INFO [Task-Executor-3] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000009=59 2018-01-05T11:28:43,751 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_6_01_000000_0: true 2018-01-05T11:28:43,752 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_6_01 [Reducer 2], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:43,752 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_6_01 [Reducer 2] 2018-01-05T11:28:43,752 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_6][Event:VERTEX_FINISHED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_6_01, initRequestedTime=1515180523560, initedTime=1515180523567, startRequestedTime=1515180523582, startedTime=1515180523582, finishTime=1515180523752, timeTaken=170, status=SUCCEEDED, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=149, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=46, SHUFFLE_BYTES_DECOMPRESSED=42, SHUFFLE_BYTES_TO_MEM=46, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=4, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1, vertexStats=firstTaskStartTime=1515180523690, firstTasksToStart=[ task_1515180518813_0001_6_01_000000 ], lastTaskFinishTime=1515180523750, lastTasksToFinish=[ task_1515180518813_0001_6_01_000000 ], minTaskDuration=60, maxTaskDuration=60, avgTaskDuration=60.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_6_01_000000 ], longestDurationTasks=[ task_1515180518813_0001_6_01_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:43,752 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_6_01 [Reducer 2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:43,752 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_6][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000009, stoppedTime=1515180523752, exitStatus=0 2018-01-05T11:28:43,752 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000009 exited with diagnostics set to Container completed. 2018-01-05T11:28:43,752 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_6_01 [Reducer 2] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2 2018-01-05T11:28:43,752 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null 2018-01-05T11:28:43,752 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_6_00 [Map 1] 2018-01-05T11:28:43,752 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_6_01 [Reducer 2] 2018-01-05T11:28:43,754 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_6, queueSize=0 2018-01-05T11:28:43,754 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_6][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_6, startTime=1515180523547, finishTime=1515180523752, timeTaken=205, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, OTHER_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=573, AM_GC_TIME_MILLIS=0, File System Counters, FILE_BYTES_READ=665, FILE_BYTES_WRITTEN=312, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=2, INPUT_SPLIT_LENGTH_BYTES=645, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=34, OUTPUT_BYTES_WITH_OVERHEAD=42, OUTPUT_BYTES_PHYSICAL=70, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=46, SHUFFLE_BYTES_DECOMPRESSED=42, SHUFFLE_BYTES_TO_MEM=46, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=4, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1, HIVE, CREATED_FILES=1, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=1, RECORDS_OUT_0=1, RECORDS_OUT_INTERMEDIATE_Map_1=1 2018-01-05T11:28:43,754 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_6 finished with state: SUCCEEDED 2018-01-05T11:28:43,754 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_6 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:28:43,754 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000009 2018-01-05T11:28:43,754 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_6, dagState=SUCCEEDED 2018-01-05T11:28:43,755 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_6_post 2018-01-05T11:28:43,755 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:43,755 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:28:43,755 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:28:43,755 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 6 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=2, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=2,NumRequestsWithoutLocation=2, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=2} 2018-01-05T11:28:43,755 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:28:43,755 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=ANALYZE TABLE src_json COMPUTE STATIS...json (Stage-0), with id=dag_1515180518813_0001_6 2018-01-05T11:28:43,755 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 6, to 10.22.8.182:49818 2018-01-05T11:28:43,755 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=2, Remaining=0 2018-01-05T11:28:43,755 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=ANALYZE TABLE src_json COMPUTE STATIS...json (Stage-0), with id=dag_1515180518813_0001_6 2018-01-05T11:28:43,755 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=6} 2018-01-05T11:28:43,755 INFO [IPC Server handler 3 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=6}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:28:43,755 INFO [IPC Server handler 3 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/6 after 300 seconds 2018-01-05T11:28:43,756 ERROR [IPC Server handler 3 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112843_0b779e91-2298-4910-af0f-02be4fa91eae, dag_1515180518813_0001_6 2018-01-05T11:28:43,756 INFO [IPC Server handler 3 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112843_0b779e91-2298-4910-af0f-02be4fa91eae 2018-01-05T11:28:43,756 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=6} = 1 2018-01-05T11:28:43,756 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Issuing killFragment for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=6} attempt_1515180518813_0001_6_01_000000_0 2018-01-05T11:28:43,756 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Removing attempt_1515180518813_0001_6_01_000000_0 from preemptionQueue 2018-01-05T11:28:43,756 INFO [IPC Server handler 3 on 49818] impl.TaskRunnerCallable: Ignoring kill request for task attempt_1515180518813_0001_6_01_000000_0 since it's already complete 2018-01-05T11:28:43,756 DEBUG [IPC Server handler 3 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=6} 2018-01-05T11:28:43,756 DEBUG [IPC Server handler 3 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:28:43,756 DEBUG [IPC Server handler 3 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=6} 2018-01-05T11:28:43,756 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,756 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,756 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,756 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,756 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 Reducer 2: 1/1 2018-01-05T11:28:43,756 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,759 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,759 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: TaskId for 000000_0 = 000000 2018-01-05T11:28:43,759 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,759 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,759 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,760 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-2:STATS] in serial mode 2018-01-05T11:28:43,760 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@3e356ed6, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:43,760 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:43,760 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,760 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:43,760 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:43,760 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:43,760 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:43,760 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,760 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:43,760 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:43,760 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_336_6588790997440458878-1/-mr-10001 2018-01-05T11:28:43,760 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_336_6588790997440458878-1/-mr-10001 2018-01-05T11:28:43,761 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.src_json/ numRows 2018-01-05T11:28:43,761 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.src_json/ numRows 1 2018-01-05T11:28:43,761 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.src_json/ rawDataSize 2018-01-05T11:28:43,761 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.src_json/ rawDataSize 644 2018-01-05T11:28:43,762 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,762 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:43,762 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:43,762 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:43,762 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:43,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,765 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,765 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:43,765 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:43,765 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,765 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:43,765 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,765 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=src_json newtbl=src_json 2018-01-05T11:28:43,765 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=src_json newtbl=src_json 2018-01-05T11:28:43,770 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,777 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,778 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,778 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.src_json stats: [numFiles=1, numRows=1, totalSize=645, rawDataSize=644] 2018-01-05T11:28:43,778 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: About to delete stats tmp dir :file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_336_6588790997440458878-1/-mr-10001 2018-01-05T11:28:43,789 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000008, QueryId=djaiswal_20180105112843_0b779e91-2298-4910-af0f-02be4fa91eae-6, DagId=6, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-3, Succeeded=true, StartTime=1515180523602, EndTime=1515180523789 2018-01-05T11:28:43,789 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_6_01_000000_0 2018-01-05T11:28:43,789 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_6_01_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:43,789 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_6_01_000000_0 2018-01-05T11:28:43,789 INFO [ExecutionCompletionThread #0] impl.QueryTracker: Ignoring fragmentComplete message for unknown query: QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=6} 2018-01-05T11:28:43,794 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Time taken to get FileStatuses: 14 2018-01-05T11:28:43,794 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:28:43,794 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 14 2018-01-05T11:28:43,794 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FetchOperator: Creating fetchTask with deserializer typeinfo: struct<_col0:struct> 2018-01-05T11:28:43,794 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FetchOperator: deserializer properties: table properties: {columns=_col0, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct, escape.delim=\} partition properties: {columns=_col0, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct, escape.delim=\} 2018-01-05T11:28:43,795 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,795 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: write_column_statistics: db=default table=src_json 2018-01-05T11:28:43,795 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=write_column_statistics: db=default table=src_json 2018-01-05T11:28:43,800 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=src_json tableName=default colName=json 2018-01-05T11:28:43,804 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,805 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,805 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,805 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,805 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: ANALYZE TABLE src_json COMPUTE STATISTICS FOR COLUMNS json 2018-01-05T11:28:43,805 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:28:43,805 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@src_json 2018-01-05T11:28:43,805 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@src_json 2018-01-05T11:28:43,805 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_336_6588790997440458878-1/-mr-10000 2018-01-05T11:28:43,805 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,805 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,805 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:43,805 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=13, setPartitionColumnStatistics_(SetPartitionsStatsRequest, )=10} 2018-01-05T11:28:43,805 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112843_0b779e91-2298-4910-af0f-02be4fa91eae); Time taken: 0.406 seconds 2018-01-05T11:28:43,805 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:43,805 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,805 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:43,805 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,805 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query ANALYZE TABLE src_json COMPUTE STATISTICS FOR COLUMNS json 2018-01-05T11:28:43,805 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_331_9091869484395642489-1 2018-01-05T11:28:43,806 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.474 seconds 2018-01-05T11:28:43,806 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:43,806 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:43,806 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:43,806 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:43,806 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,806 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:43,806 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,806 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,806 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,806 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: -- -- Table src_sequencefile -- DROP TABLE IF EXISTS src_sequencefile 2018-01-05T11:28:43,806 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112843_5d0e03d8-7518-4584-991f-71584c12b45c): -- -- Table src_sequencefile -- DROP TABLE IF EXISTS src_sequencefile 2018-01-05T11:28:43,806 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,806 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: -- -- Table src_sequencefile -- DROP TABLE IF EXISTS src_sequencefile 2018-01-05T11:28:43,807 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:43,807 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,807 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,807 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,807 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,807 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:43,807 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,807 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_sequencefile 2018-01-05T11:28:43,807 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_sequencefile 2018-01-05T11:28:43,808 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,808 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,808 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:43,808 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,808 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:43,808 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,808 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:43,808 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:28:43,808 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112843_5d0e03d8-7518-4584-991f-71584c12b45c); Time taken: 0.002 seconds 2018-01-05T11:28:43,808 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:43,808 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,808 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112843_5d0e03d8-7518-4584-991f-71584c12b45c): -- -- Table src_sequencefile -- DROP TABLE IF EXISTS src_sequencefile 2018-01-05T11:28:43,809 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,809 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: -- -- Table src_sequencefile -- DROP TABLE IF EXISTS src_sequencefile 2018-01-05T11:28:43,809 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:43,809 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,809 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,809 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,809 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,809 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:43,809 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,809 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_sequencefile 2018-01-05T11:28:43,809 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_sequencefile 2018-01-05T11:28:43,810 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,810 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,810 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table src_sequencefile not found: default.src_sequencefile table not found 2018-01-05T11:28:43,810 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,810 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_sequencefile 2018-01-05T11:28:43,810 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_sequencefile 2018-01-05T11:28:43,810 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,810 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,810 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,810 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,811 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: -- -- Table src_sequencefile -- DROP TABLE IF EXISTS src_sequencefile 2018-01-05T11:28:43,811 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:43,811 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,811 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,811 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:43,811 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=0} 2018-01-05T11:28:43,811 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112843_5d0e03d8-7518-4584-991f-71584c12b45c); Time taken: 0.003 seconds 2018-01-05T11:28:43,811 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:43,811 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,811 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:43,811 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,811 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query -- -- Table src_sequencefile -- DROP TABLE IF EXISTS src_sequencefile 2018-01-05T11:28:43,811 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.005 seconds 2018-01-05T11:28:43,811 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:43,811 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:43,811 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:43,811 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:43,811 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,811 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:43,811 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,811 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,811 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,811 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: CREATE TABLE src_sequencefile (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS SEQUENCEFILE 2018-01-05T11:28:43,811 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112843_f5085275-6168-4449-a935-64e341eebfc6): CREATE TABLE src_sequencefile (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS SEQUENCEFILE 2018-01-05T11:28:43,811 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,811 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: CREATE TABLE src_sequencefile (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS SEQUENCEFILE 2018-01-05T11:28:43,811 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:43,812 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,812 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,812 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,812 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,812 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:43,812 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:28:43,812 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Creating table default.src_sequencefile position=13 2018-01-05T11:28:43,812 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,812 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_database: default 2018-01-05T11:28:43,812 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_database: default 2018-01-05T11:28:43,813 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:43,813 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,813 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,813 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:43,813 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: validation start 2018-01-05T11:28:43,813 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:28:43,813 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Not a partition. 2018-01-05T11:28:43,813 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,813 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:43,813 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,813 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:43,813 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getDatabase_(String, )=1, flushCache_()=0} 2018-01-05T11:28:43,813 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112843_f5085275-6168-4449-a935-64e341eebfc6); Time taken: 0.002 seconds 2018-01-05T11:28:43,813 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:43,813 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,813 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112843_f5085275-6168-4449-a935-64e341eebfc6): CREATE TABLE src_sequencefile (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS SEQUENCEFILE 2018-01-05T11:28:43,814 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,814 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: CREATE TABLE src_sequencefile (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS SEQUENCEFILE 2018-01-05T11:28:43,814 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: CREATETABLE 2018-01-05T11:28:43,814 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: database:default 2018-01-05T11:28:43,814 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@src_sequencefile 2018-01-05T11:28:43,814 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,814 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,814 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,815 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,815 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:43,815 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Found class for org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe 2018-01-05T11:28:43,815 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: creating table default.src_sequencefile on file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile 2018-01-05T11:28:43,816 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,816 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: create_table: Table(tableName:src_sequencefile, dbName:default, owner:hive_test_user, createTime:1515180523, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:string, comment:default), FieldSchema(name:value, type:string, comment:default)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile, inputFormat:org.apache.hadoop.mapred.SequenceFileInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, numFiles=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:43,816 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=create_table: Table(tableName:src_sequencefile, dbName:default, owner:hive_test_user, createTime:1515180523, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:string, comment:default), FieldSchema(name:value, type:string, comment:default)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile, inputFormat:org.apache.hadoop.mapred.SequenceFileInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, numFiles=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:43,817 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hive-site.xml 2018-01-05T11:28:43,817 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hivemetastore-site.xml 2018-01-05T11:28:43,818 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Unable to find config file metastore-site.xml 2018-01-05T11:28:43,818 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file null 2018-01-05T11:28:43,828 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:43,828 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: Location: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile specified for non-external table:src_sequencefile 2018-01-05T11:28:43,829 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile 2018-01-05T11:28:43,849 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,851 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@src_sequencefile) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@src_sequencefile) Type=TABLE WriteType=DDL_NO_LOCK is present 2018-01-05T11:28:43,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,852 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: CREATE TABLE src_sequencefile (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS SEQUENCEFILE 2018-01-05T11:28:43,852 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: CREATETABLE 2018-01-05T11:28:43,852 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: database:default 2018-01-05T11:28:43,852 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@src_sequencefile 2018-01-05T11:28:43,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,852 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:43,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {createTable_(Table, )=36, isCompatibleWith_(Configuration, )=0} 2018-01-05T11:28:43,852 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112843_f5085275-6168-4449-a935-64e341eebfc6); Time taken: 0.039 seconds 2018-01-05T11:28:43,852 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:43,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,852 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:43,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query CREATE TABLE src_sequencefile (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS SEQUENCEFILE 2018-01-05T11:28:43,852 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.041 seconds 2018-01-05T11:28:43,852 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:43,852 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:43,852 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:43,852 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:43,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:43,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/kv1.seq" INTO TABLE src_sequencefile 2018-01-05T11:28:43,853 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112843_02797e76-fb01-4df4-80b8-f99590a4121c): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv1.seq" INTO TABLE src_sequencefile 2018-01-05T11:28:43,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv1.seq" INTO TABLE src_sequencefile 2018-01-05T11:28:43,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:43,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,853 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:43,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.LoadSemanticAnalyzer: file@null@/Users/djaiswal/hive/hive/data/files/kv1.seq 2018-01-05T11:28:43,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,853 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_sequencefile 2018-01-05T11:28:43,853 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_sequencefile 2018-01-05T11:28:43,858 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,858 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,861 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:43,861 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,861 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:43,861 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,861 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:43,861 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=5, flushCache_()=0} 2018-01-05T11:28:43,861 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112843_02797e76-fb01-4df4-80b8-f99590a4121c); Time taken: 0.008 seconds 2018-01-05T11:28:43,861 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:43,861 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,861 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112843_02797e76-fb01-4df4-80b8-f99590a4121c): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv1.seq" INTO TABLE src_sequencefile 2018-01-05T11:28:43,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,862 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv1.seq" INTO TABLE src_sequencefile 2018-01-05T11:28:43,862 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: LOAD 2018-01-05T11:28:43,862 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/kv1.seq 2018-01-05T11:28:43,862 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@src_sequencefile 2018-01-05T11:28:43,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,862 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MOVE] in serial mode 2018-01-05T11:28:43,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@14f03f2, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:43,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:43,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,862 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:43,862 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:43,862 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:43,862 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:43,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:43,862 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Loading data to table default.src_sequencefile from file:/Users/djaiswal/hive/hive/data/files/kv1.seq 2018-01-05T11:28:43,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,864 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:43,864 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:43,864 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:43,864 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:43,865 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,865 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,865 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,867 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,867 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,867 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:43,867 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:43,868 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,868 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:43,868 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,868 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_sequencefile 2018-01-05T11:28:43,868 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_sequencefile 2018-01-05T11:28:43,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,874 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,874 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,874 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_sequencefile 2018-01-05T11:28:43,875 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_sequencefile 2018-01-05T11:28:43,878 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,879 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,879 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] FileOperations: moving file:/Users/djaiswal/hive/hive/data/files/kv1.seq to file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile (replace = KEEP_EXISTING) 2018-01-05T11:28:43,880 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:43,880 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:43,891 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000009, QueryId=djaiswal_20180105112843_0b779e91-2298-4910-af0f-02be4fa91eae-6, DagId=6, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-3, Succeeded=true, StartTime=1515180523692, EndTime=1515180523891 2018-01-05T11:28:43,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Moved src: file:/Users/djaiswal/hive/hive/data/files/kv1.seq, to dest: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile/kv1.seq 2018-01-05T11:28:43,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,908 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=src_sequencefile newtbl=src_sequencefile 2018-01-05T11:28:43,908 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=src_sequencefile newtbl=src_sequencefile 2018-01-05T11:28:43,913 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,932 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,933 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,933 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@src_sequencefile) Type=TABLE WriteType=INSERT because WriteEntity(default@src_sequencefile) Type=TABLE WriteType=INSERT is present 2018-01-05T11:28:43,933 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-1:STATS] in serial mode 2018-01-05T11:28:43,933 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@1deca369, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:43,933 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:43,934 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,934 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:43,934 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:43,934 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:43,934 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:43,934 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,934 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:43,934 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:43,935 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,935 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:43,935 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:43,935 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:43,935 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:43,935 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,935 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,936 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,937 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,938 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,938 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:43,939 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:43,939 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,939 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:43,939 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,939 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_sequencefile 2018-01-05T11:28:43,939 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_sequencefile 2018-01-05T11:28:43,943 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,944 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,945 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,945 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=src_sequencefile newtbl=src_sequencefile 2018-01-05T11:28:43,945 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=src_sequencefile newtbl=src_sequencefile 2018-01-05T11:28:43,949 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,959 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,959 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,959 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.src_sequencefile stats: [numFiles=1, numRows=0, totalSize=10508, rawDataSize=0] 2018-01-05T11:28:43,959 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,959 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,959 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv1.seq" INTO TABLE src_sequencefile 2018-01-05T11:28:43,959 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: LOAD 2018-01-05T11:28:43,959 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/kv1.seq 2018-01-05T11:28:43,959 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@src_sequencefile 2018-01-05T11:28:43,959 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,959 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,960 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:43,960 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=14, getTable_(String, String, )=5} 2018-01-05T11:28:43,960 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112843_02797e76-fb01-4df4-80b8-f99590a4121c); Time taken: 0.098 seconds 2018-01-05T11:28:43,960 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:43,960 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,960 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:43,960 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,960 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv1.seq" INTO TABLE src_sequencefile 2018-01-05T11:28:43,960 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.107 seconds 2018-01-05T11:28:43,960 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:43,960 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:43,960 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:43,960 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:43,960 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,960 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:43,960 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,960 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,960 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,960 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: ANALYZE TABLE src_sequencefile COMPUTE STATISTICS 2018-01-05T11:28:43,960 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112843_5fff5c3e-5bd5-4b02-82c4-dfba8563e391): ANALYZE TABLE src_sequencefile COMPUTE STATISTICS 2018-01-05T11:28:43,960 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,960 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: ANALYZE TABLE src_sequencefile COMPUTE STATISTICS 2018-01-05T11:28:43,960 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:43,960 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,960 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,960 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,960 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,960 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:43,961 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Invoking analyze on original query 2018-01-05T11:28:43,961 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Starting Semantic Analysis 2018-01-05T11:28:43,961 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed phase 1 of Semantic Analysis 2018-01-05T11:28:43,961 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for source tables 2018-01-05T11:28:43,961 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,961 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_sequencefile 2018-01-05T11:28:43,961 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_sequencefile 2018-01-05T11:28:43,964 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,965 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,965 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,965 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_sequencefile 2018-01-05T11:28:43,965 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_sequencefile 2018-01-05T11:28:43,970 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,970 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,970 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for subqueries 2018-01-05T11:28:43,970 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for destination tables 2018-01-05T11:28:43,970 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed getting MetaData in Semantic Analysis 2018-01-05T11:28:43,984 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_960_8916097632880633826-1/-mr-10000 2018-01-05T11:28:43,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Table Plan for src_sequencefile TS[0] 2018-01-05T11:28:43,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Body Plan for Query Block null 2018-01-05T11:28:43,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Plan for Query Block null 2018-01-05T11:28:43,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Before logical optimization TS[0] 2018-01-05T11:28:43,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] lineage.Generator: Time taken for lineage transform=0 2018-01-05T11:28:43,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{src_sequencefile}key,value: string|{src_sequencefile}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src_sequencefile}block__offset__inside__file,INPUT__FILE__NAME: string|{src_sequencefile}input__file__name,ROW__ID: struct|{src_sequencefile}row__id,RAW__DATA__SIZE: bigint|{src_sequencefile}raw__data__size) 2018-01-05T11:28:43,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,985 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:28:43,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.PredicatePushDown: After PPD: TS[0] 2018-01-05T11:28:43,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{src_sequencefile}key,value: string|{src_sequencefile}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src_sequencefile}block__offset__inside__file,INPUT__FILE__NAME: string|{src_sequencefile}input__file__name,ROW__ID: struct|{src_sequencefile}row__id,RAW__DATA__SIZE: bigint|{src_sequencefile}raw__data__size) 2018-01-05T11:28:43,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{src_sequencefile}key,value: string|{src_sequencefile}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src_sequencefile}block__offset__inside__file,INPUT__FILE__NAME: string|{src_sequencefile}input__file__name,ROW__ID: struct|{src_sequencefile}row__id,RAW__DATA__SIZE: bigint|{src_sequencefile}raw__data__size) 2018-01-05T11:28:43,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: After logical optimization TS[0] 2018-01-05T11:28:43,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,986 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=src_sequencefile 2018-01-05T11:28:43,986 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=src_sequencefile 2018-01-05T11:28:43,992 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,998 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:43,999 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:43,999 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting stats (Num rows: 1 Data size: 105080 Basic stats: COMPLETE Column stats: COMPLETE) on TS[0] 2018-01-05T11:28:43,999 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (src_sequencefile): numRows: 1 dataSize: 105080 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:28:43,999 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on TS[0] 2018-01-05T11:28:43,999 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:43,999 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,000 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,000 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,000 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,000 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,000 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,000 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,000 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,000 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,000 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,000 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,000 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,000 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,000 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,000 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,000 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,000 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,000 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,000 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,000 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:44,000 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:28:44,000 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Cycle free: true 2018-01-05T11:28:44,000 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,000 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,000 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,000 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{src_sequencefile}key,value: string|{src_sequencefile}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src_sequencefile}block__offset__inside__file,INPUT__FILE__NAME: string|{src_sequencefile}input__file__name,ROW__ID: struct|{src_sequencefile}row__id,RAW__DATA__SIZE: bigint|{src_sequencefile}raw__data__size) 2018-01-05T11:28:44,000 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,000 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:28:44,000 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile of table src_sequencefile 2018-01-05T11:28:44,001 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile 2018-01-05T11:28:44,001 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:28:44,001 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,001 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,001 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:44,001 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:44,001 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:44,001 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:44,001 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:44,001 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:44,001 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:28:44,001 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping vectorization 2018-01-05T11:28:44,001 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:28:44,001 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: llap mode: only 2018-01-05T11:28:44,001 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:28:44,001 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:44,001 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@2e268c19 can be done in LLAP 2018-01-05T11:28:44,001 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,001 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed plan generation 2018-01-05T11:28:44,001 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:44,001 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validation start 2018-01-05T11:28:44,001 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Not a partition. 2018-01-05T11:28:44,001 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,001 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:src_sequencefile.key, type:string, comment:null), FieldSchema(name:src_sequencefile.value, type:string, comment:null)], properties:null) 2018-01-05T11:28:44,001 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,001 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:44,001 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=9, flushCache_()=0, getTableColumnStatistics_(String, String, List, )=13} 2018-01-05T11:28:44,002 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112843_5fff5c3e-5bd5-4b02-82c4-dfba8563e391); Time taken: 0.041 seconds 2018-01-05T11:28:44,002 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:44,002 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,002 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112843_5fff5c3e-5bd5-4b02-82c4-dfba8563e391): ANALYZE TABLE src_sequencefile COMPUTE STATISTICS 2018-01-05T11:28:44,003 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,003 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: ANALYZE TABLE src_sequencefile COMPUTE STATISTICS 2018-01-05T11:28:44,003 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:28:44,003 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@src_sequencefile 2018-01-05T11:28:44,003 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@src_sequencefile 2018-01-05T11:28:44,003 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,003 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,003 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,003 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Query ID = djaiswal_20180105112843_5fff5c3e-5bd5-4b02-82c4-dfba8563e391 2018-01-05T11:28:44,003 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Total jobs = 1 2018-01-05T11:28:44,003 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,003 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:28:44,003 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MAPRED] in serial mode 2018-01-05T11:28:44,016 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_960_8916097632880633826-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:28:44,042 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:28:44,042 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:28:44,043 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:28:44,043 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112843_5fff5c3e-5bd5-4b02-82c4-dfba8563e391 2018-01-05T11:28:44,043 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Session is already open 2018-01-05T11:28:44,043 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:28:44,043 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,043 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Dag name: ANALYZE TABLE src_sequencefile ...STATISTICS (Stage-0) 2018-01-05T11:28:44,043 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: DagInfo: {"context":"Hive","description":"\n\nANALYZE TABLE src_sequencefile COMPUTE STATISTICS"} 2018-01-05T11:28:44,043 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112843_5fff5c3e-5bd5-4b02-82c4-dfba8563e391 with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:28:44,043 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,055 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:28:44,055 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,055 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:28:44,056 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,056 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 1.73KB 2018-01-05T11:28:44,063 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_960_8916097632880633826-1/-mr-10000 2018-01-05T11:28:44,075 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_960_8916097632880633826-1/-mr-10000 2018-01-05T11:28:44,077 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile 2018-01-05T11:28:44,077 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,077 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,077 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,077 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=ANALYZE TABLE src_sequencefile ...STATISTICS (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112843_5fff5c3e-5bd5-4b02-82c4-dfba8563e391 } 2018-01-05T11:28:44,078 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:44,081 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: ANALYZE TABLE src_sequencefile ...STATISTICS (Stage-0) 2018-01-05T11:28:44,081 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:28:44,081 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_7, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_7.dot 2018-01-05T11:28:44,082 INFO [IPC Server handler 1 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_7 2018-01-05T11:28:44,082 WARN [IPC Server handler 1 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:44,082 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Running DAG: ANALYZE TABLE src_sequencefile ...STATISTICS (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112843_5fff5c3e-5bd5-4b02-82c4-dfba8563e391 } 2018-01-05T11:28:44,105 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_7][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_7, submitTime=1515180524081, queueName=null 2018-01-05T11:28:44,106 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_7_00 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_7_00 [Map 1], Default Resources= 2018-01-05T11:28:44,106 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_7_00 [Map 1] 2018-01-05T11:28:44,106 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_7_00 [Map 1] 2018-01-05T11:28:44,106 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:28:44,106 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_7_00 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:44,106 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:28:44,106 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_7][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_7, initTime=1515180524106 2018-01-05T11:28:44,106 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: dag_1515180518813_0001_7 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:28:44,108 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:28:44,108 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_7][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_7, startTime=1515180524108 2018-01-05T11:28:44,108 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_7 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:28:44,108 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {src_sequencefile={InputName=src_sequencefile}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:28:44,108 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: src_sequencefile, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:28:44,108 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_7_00 [Map 1] 2018-01-05T11:28:44,108 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_7, dagName=ANALYZE TABLE src_sequencefile ...STATISTICS (Stage-0) 2018-01-05T11:28:44,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,112 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_7_00 [Map 1] 2018-01-05T11:28:44,112 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_7_00 [Map 1] 2018-01-05T11:28:44,112 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:44,113 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_7_00 [Map 1] 2018-01-05T11:28:44,115 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:28:44,115 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@10e4e245 2018-01-05T11:28:44,115 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_960_8916097632880633826-1/djaiswal/_tez_scratch_dir/99daa43a-b959-4218-8c9f-51850dd3ba4f/map.xml 2018-01-05T11:28:44,115 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_960_8916097632880633826-1/djaiswal/_tez_scratch_dir/99daa43a-b959-4218-8c9f-51850dd3ba4f/map.xml 2018-01-05T11:28:44,115 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_960_8916097632880633826-1/djaiswal/_tez_scratch_dir/99daa43a-b959-4218-8c9f-51850dd3ba4f/map.xml 2018-01-05T11:28:44,115 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:44,115 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:44,115 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:44,116 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:44,116 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 1.73KB 2018-01-05T11:28:44,116 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_7_00 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:28:44,116 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: src_sequencefile on vertex vertex_1515180518813_0001_7_00 [Map 1] 2018-01-05T11:28:44,116 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:28:44,116 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:44,116 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:28:44,116 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:44,116 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_960_8916097632880633826-1/djaiswal/_tez_scratch_dir/99daa43a-b959-4218-8c9f-51850dd3ba4f/map.xml 2018-01-05T11:28:44,117 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:44,117 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias src_sequencefile 2018-01-05T11:28:44,117 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile 2018-01-05T11:28:44,117 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile 2018-01-05T11:28:44,117 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 2018-01-05T11:28:44,117 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = 2018-01-05T11:28:44,117 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile 2018-01-05T11:28:44,118 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:44,118 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:44,118 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:44,118 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:44,133 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Time taken to get FileStatuses: 14 2018-01-05T11:28:44,133 INFO [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:28:44,133 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 15 2018-01-05T11:28:44,133 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 1 2018-01-05T11:28:44,133 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:44,133 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 1. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:28:44,133 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_960_8916097632880633826-1/djaiswal/_tez_scratch_dir/99daa43a-b959-4218-8c9f-51850dd3ba4f/map.xml 2018-01-05T11:28:44,133 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_960_8916097632880633826-1/djaiswal/_tez_scratch_dir/99daa43a-b959-4218-8c9f-51850dd3ba4f/map.xml 2018-01-05T11:28:44,133 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_960_8916097632880633826-1/djaiswal/_tez_scratch_dir/99daa43a-b959-4218-8c9f-51850dd3ba4f/map.xml 2018-01-05T11:28:44,133 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:44,133 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:44,133 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:44,133 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:44,133 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 1.73KB 2018-01-05T11:28:44,134 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile/kv1.seq to src new group? true 2018-01-05T11:28:44,134 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:28:44,134 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:28:44,134 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:28:44,134 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 194 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 10508 Original splits: 1 2018-01-05T11:28:44,134 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Using original number of splits: 1 desired splits: 1 2018-01-05T11:28:44,134 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 1 grouped split count is 1, for bucket: 1 2018-01-05T11:28:44,134 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:28:44,134 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: src_sequencefile on vertex vertex_1515180518813_0001_7_00 [Map 1] 2018-01-05T11:28:44,135 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_7_00 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:44,135 INFO [App Shared Pool - #0] impl.VertexImpl: Got updated RootInputsSpecs: {src_sequencefile=forAllWorkUnits=true, update=[1]} 2018-01-05T11:28:44,135 INFO [App Shared Pool - #0] impl.VertexImpl: Vertex vertex_1515180518813_0001_7_00 [Map 1] parallelism set to 1 2018-01-05T11:28:44,135 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_7_00 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:44,135 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_7_00 [Map 1], numAdditionalOutputs=1 2018-01-05T11:28:44,135 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_7][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_7_00, initRequestedTime=1515180524108, initedTime=1515180524135, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:44,135 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_7_00 [Map 1] with distanceFromRoot: 0 2018-01-05T11:28:44,136 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_7_00 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:28:44,136 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_7][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_7_00, startRequestedTime=1515180524116, startedTime=1515180524136 2018-01-05T11:28:44,136 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_7][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_7_00, reconfigureDoneTime=1515180524136, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:28:44,136 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_7_00 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:44,136 INFO [App Shared Pool - #0] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:28:44,136 INFO [App Shared Pool - #0] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:28:44,136 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_7_00 [Map 1] 2018-01-05T11:28:44,136 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_7][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_7_00_000000, scheduledTime=1515180524136, launchTime=1515180524136 2018-01-05T11:28:44,137 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_7_00 [Map 1] 2018-01-05T11:28:44,137 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_7_00_000000_0, priority=2, capability=, hosts=null 2018-01-05T11:28:44,137 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=2,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:44,137 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=2,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:44,137 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:44,137 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_7_00_000000_0 on hosts=null 2018-01-05T11:28:44,137 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_7_00_000000_0 on hosts=null 2018-01-05T11:28:44,137 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_7_00_000000_0 2018-01-05T11:28:44,137 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:44,137 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #10, task=TaskInfo{task=attempt_1515180518813_0001_7_00_000000_0, priority=2, startTime=0, containerId=null, uniqueId=9, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000010 2018-01-05T11:28:44,137 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_7_00_000000_0; false 2018-01-05T11:28:44,137 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_7_00_000000_0, priority=2, startTime=311544646, containerId=container_111101111_0001_00_000010assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=9, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:44,138 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000010 for node: 10.22.8.182:49818 2018-01-05T11:28:44,141 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 7, name=ANALYZE TABLE src_sequencefile ...STATISTICS (Stage-0), queryId=djaiswal_20180105112843_5fff5c3e-5bd5-4b02-82c4-dfba8563e391 2018-01-05T11:28:44,141 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000010, attempt_1515180518813_0001_7_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:44,141 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000010 for node: 10.22.8.182:49818 2018-01-05T11:28:44,142 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000010 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:44,142 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000010, launchTime=1515180524142 2018-01-05T11:28:44,143 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_7_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112843_5fff5c3e-5bd5-4b02-82c4-dfba8563e391, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000010, dagName=ANALYZE TABLE src_sequencefile ...STATISTICS (Stage-0), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{src_sequencefile,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{out_Map 1,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180524108, firstAttemptStartTime=1515180524137, currentAttemptStartTime=1515180524142} 2018-01-05T11:28:44,143 INFO [IPC Server handler 1 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000010, QueryId=djaiswal_20180105112843_5fff5c3e-5bd5-4b02-82c4-dfba8563e391-7, DagId=7, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180524143 2018-01-05T11:28:44,143 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_7_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000010] on NM: [10.22.8.182:49818] 2018-01-05T11:28:44,143 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:44,143 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:44,143 DEBUG [IPC Server handler 1 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=7} with the ShuffleHandler 2018-01-05T11:28:44,143 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_7][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_7_00_000000_0, startTime=1515180524143, containerId=container_111101111_0001_00_000010, nodeId=10.22.8.182:49818 2018-01-05T11:28:44,156 INFO [IPC Server handler 1 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:28:44,156 DEBUG [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/7] 2018-01-05T11:28:44,156 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:44,156 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_7_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:44,156 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_7_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180524137, dagStartTime=1515180524108, withinDagPriority=2, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:28:44,156 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_7_00_000000_0 : ACCEPTED 2018-01-05T11:28:44,156 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_7_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:44,156 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_7_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180524137, dagStartTime=1515180524108, withinDagPriority=2, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:28:44,156 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_7_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:44,156 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_7_00_000000_0 to pre-emption queue 2018-01-05T11:28:44,156 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_7_00_000000_0: true 2018-01-05T11:28:44,157 INFO [TaskCommunicator # 8] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_7_00_000000_0 2018-01-05T11:28:44,157 INFO [TaskCommunicator # 8] GuaranteedTasks: Received response for attempt_1515180518813_0001_7_00_000000_0, true 2018-01-05T11:28:44,157 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: taskOwner hashCode:507231068 2018-01-05T11:28:44,158 INFO [Task-Executor-3] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : ANALYZE TABLE src_sequencefile ...STATISTICS (Stage-0), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_7_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=src_sequencefile, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=out_Map 1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:28:44,158 INFO [Task-Executor-3] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=954728448, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:44,158 INFO [Task-Executor-3] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_7_00_000000_0 vertexName: Map 1 2018-01-05T11:28:44,158 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000010, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_7_00_000000_0, eventCount=1 } 2018-01-05T11:28:44,158 DEBUG [TezTR-518813_1_7_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_7 queryId: djaiswal_20180105112843_5fff5c3e-5bd5-4b02-82c4-dfba8563e391 instanceType: TaskRunner2Callable 2018-01-05T11:28:44,158 INFO [TezTR-518813_1_7_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_7_00_000000_0 2018-01-05T11:28:44,159 DEBUG [TezTR-518813_1_7_0_0_0] log.PerfLogger: 2018-01-05T11:28:44,160 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:44,160 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:44,163 INFO [IPC Server handler 11 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_7_00_000000_0 sent events: (0-1). 2018-01-05T11:28:44,164 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:28:44,164 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_7_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:28:44,166 DEBUG [TezTR-518813_1_7_0_0_0] log.PerfLogger: 2018-01-05T11:28:44,166 INFO [TezTR-518813_1_7_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:44,166 INFO [TezTR-518813_1_7_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:44,166 INFO [I/O Setup 0 Initialize: {src_sequencefile}] input.MRInput: src_sequencefile using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:28:44,166 INFO [I/O Setup 0 Initialize: {src_sequencefile}] input.MRInputLegacy: src_sequencefile MRInputLegacy deferring initialization 2018-01-05T11:28:44,166 INFO [TezTR-518813_1_7_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:44,166 INFO [I/O Setup 1 Initialize: {out_Map 1}] output.MROutput: out_Map 1: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:28:44,166 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:44,166 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:44,166 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:44,166 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:44,166 INFO [TezTR-518813_1_7_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:44,166 INFO [TezTR-518813_1_7_0_0_0] resources.MemoryDistributor: InitialRequests=[src_sequencefile:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [out_Map 1:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput] 2018-01-05T11:28:44,166 INFO [TezTR-518813_1_7_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:44,166 INFO [TezTR-518813_1_7_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:44,166 INFO [TezTR-518813_1_7_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 954728448, TotalAvailable: 639668060, TotalRequested/TotalJVMHeap:0.00 2018-01-05T11:28:44,166 INFO [TezTR-518813_1_7_0_0_0] resources.MemoryDistributor: Allocations=[src_sequencefile:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [out_Map 1:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0] 2018-01-05T11:28:44,166 INFO [TezTR-518813_1_7_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:44,166 INFO [TezTR-518813_1_7_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: src_sequencefile being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:28:44,167 INFO [TezTR-518813_1_7_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:28:44,167 INFO [TezTR-518813_1_7_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:28:44,167 INFO [I/O Setup 0 Start: {src_sequencefile}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: src_sequencefile 2018-01-05T11:28:44,167 INFO [TezTR-518813_1_7_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:44,167 INFO [TezTR-518813_1_7_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_7_00_000000_0 2018-01-05T11:28:44,167 DEBUG [TezTR-518813_1_7_0_0_0] log.PerfLogger: 2018-01-05T11:28:44,167 DEBUG [TezTR-518813_1_7_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_7_00_000000_0_10028 2018-01-05T11:28:44,167 DEBUG [TezTR-518813_1_7_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:44,167 DEBUG [TezTR-518813_1_7_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_7_00_000000_0 2018-01-05T11:28:44,167 INFO [TezTR-518813_1_7_0_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112843_5fff5c3e-5bd5-4b02-82c4-dfba8563e391 2018-01-05T11:28:44,167 DEBUG [TezTR-518813_1_7_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:44,167 DEBUG [TezTR-518813_1_7_0_0_0] log.PerfLogger: 2018-01-05T11:28:44,167 DEBUG [TezTR-518813_1_7_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:44,167 DEBUG [TezTR-518813_1_7_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:44,167 DEBUG [TezTR-518813_1_7_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112843_5fff5c3e-5bd5-4b02-82c4-dfba8563e391_Map 1__MAP_PLAN__ 2018-01-05T11:28:44,167 INFO [TezTR-518813_1_7_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_960_8916097632880633826-1/djaiswal/_tez_scratch_dir/99daa43a-b959-4218-8c9f-51850dd3ba4f/map.xml 2018-01-05T11:28:44,168 DEBUG [TezTR-518813_1_7_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_960_8916097632880633826-1/djaiswal/_tez_scratch_dir/99daa43a-b959-4218-8c9f-51850dd3ba4f/map.xml 2018-01-05T11:28:44,168 DEBUG [TezTR-518813_1_7_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_960_8916097632880633826-1/djaiswal/_tez_scratch_dir/99daa43a-b959-4218-8c9f-51850dd3ba4f/map.xml 2018-01-05T11:28:44,168 DEBUG [TezTR-518813_1_7_0_0_0] log.PerfLogger: 2018-01-05T11:28:44,168 INFO [TezTR-518813_1_7_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:44,168 DEBUG [TezTR-518813_1_7_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:44,168 DEBUG [TezTR-518813_1_7_0_0_0] log.PerfLogger: 2018-01-05T11:28:44,168 INFO [TezTR-518813_1_7_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 1.73KB 2018-01-05T11:28:44,169 DEBUG [TezTR-518813_1_7_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:44,169 INFO [TezTR-518813_1_7_0_0_0] tez.MapRecordProcessor: The input names are: [src_sequencefile] 2018-01-05T11:28:44,169 INFO [TezTR-518813_1_7_0_0_0] input.MRInput: Initialized MRInput: src_sequencefile 2018-01-05T11:28:44,169 DEBUG [TezTR-518813_1_7_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 1 2018-01-05T11:28:44,169 INFO [TezTR-518813_1_7_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_960_8916097632880633826-1/djaiswal/_tez_scratch_dir/99daa43a-b959-4218-8c9f-51850dd3ba4f/map.xml 2018-01-05T11:28:44,169 DEBUG [TezTR-518813_1_7_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:44,169 DEBUG [TezTR-518813_1_7_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile/kv1.seq PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.SequenceFileInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat, properties={name=default.src_sequencefile, numFiles=1, columns.types=string:string, serialization.format=1, serialization.ddl=struct src_sequencefile { string key, string value}, columns=key,value, columns.comments=defaultdefault, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.SequenceFileInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile, file.outputformat=org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat, totalSize=10508, column.name.delimiter=,, transient_lastDdlTime=1515180523}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.SequenceFileInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat, properties={name=default.src_sequencefile, numFiles=1, columns.types=string:string, columns=key,value, serialization.format=1, serialization.ddl=struct src_sequencefile { string key, string value}, columns.comments=defaultdefault, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.SequenceFileInputFormat, totalSize=10508, file.outputformat=org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile, column.name.delimiter=,, transient_lastDdlTime=1515180523}, baseFileName=src_sequencefile, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.SequenceFileInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat, properties={name=default.src_sequencefile, numFiles=1, columns.types=string:string, serialization.format=1, serialization.ddl=struct src_sequencefile { string key, string value}, columns=key,value, columns.comments=defaultdefault, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.SequenceFileInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile, file.outputformat=org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat, totalSize=10508, column.name.delimiter=,, transient_lastDdlTime=1515180523}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.SequenceFileInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat, properties={name=default.src_sequencefile, numFiles=1, columns.types=string:string, columns=key,value, serialization.format=1, serialization.ddl=struct src_sequencefile { string key, string value}, columns.comments=defaultdefault, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.SequenceFileInputFormat, totalSize=10508, file.outputformat=org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile, column.name.delimiter=,, transient_lastDdlTime=1515180523}, baseFileName=src_sequencefile, vectorPartitionDesc=null]} 2018-01-05T11:28:44,169 DEBUG [TezTR-518813_1_7_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.SequenceFileInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:44,169 INFO [TezTR-518813_1_7_0_0_0] io.HiveInputFormat: Not using llap for org.apache.hadoop.mapred.SequenceFileInputFormat: supported = false, vectorized = true, cache only = false 2018-01-05T11:28:44,173 DEBUG [TezTR-518813_1_7_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile/kv1.seq 2018-01-05T11:28:44,173 INFO [TezTR-518813_1_7_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.mapred.SequenceFileInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile/kv1.seq:0+10508], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=10508} 2018-01-05T11:28:44,173 INFO [TezTR-518813_1_7_0_0_0] input.MRInput: src_sequencefile initialized RecordReader from event 2018-01-05T11:28:44,173 DEBUG [TezTR-518813_1_7_0_0_0] tez.MapRecordProcessor: Starting Output: out_Map 1 2018-01-05T11:28:44,173 INFO [TezTR-518813_1_7_0_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:28:44,173 INFO [TezTR-518813_1_7_0_0_0] exec.MapOperator: Initializing operator MAP[0] 2018-01-05T11:28:44,173 DEBUG [TezTR-518813_1_7_0_0_0] exec.MapOperator: Initialization Done 0 MAP 2018-01-05T11:28:44,173 DEBUG [TezTR-518813_1_7_0_0_0] exec.MapOperator: Operator 0 MAP initialized 2018-01-05T11:28:44,173 DEBUG [TezTR-518813_1_7_0_0_0] exec.MapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:28:44,174 DEBUG [TezTR-518813_1_7_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:44,174 DEBUG [TezTR-518813_1_7_0_0_0] exec.MapOperator: Adding alias src_sequencefile to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile 2018-01-05T11:28:44,174 DEBUG [TezTR-518813_1_7_0_0_0] exec.MapOperator: dump TS[0] struct 2018-01-05T11:28:44,174 INFO [TezTR-518813_1_7_0_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:28:44,174 INFO [TezTR-518813_1_7_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:44,174 DEBUG [TezTR-518813_1_7_0_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:28:44,174 INFO [TezTR-518813_1_7_0_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:28:44,174 DEBUG [TezTR-518813_1_7_0_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:28:44,174 DEBUG [TezTR-518813_1_7_0_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:28:44,174 DEBUG [TezTR-518813_1_7_0_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:28:44,174 DEBUG [TezTR-518813_1_7_0_0_0] log.PerfLogger: 2018-01-05T11:28:44,175 DEBUG [TezTR-518813_1_7_0_0_0] exec.MapOperator: Processing alias(es) src_sequencefile for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile/kv1.seq 2018-01-05T11:28:44,175 INFO [TezTR-518813_1_7_0_0_0] exec.MapOperator: MAP[0]: records read - 1 2018-01-05T11:28:44,175 INFO [TezTR-518813_1_7_0_0_0] exec.MapOperator: MAP[0]: records read - 10 2018-01-05T11:28:44,176 INFO [TezTR-518813_1_7_0_0_0] exec.MapOperator: MAP[0]: records read - 100 2018-01-05T11:28:44,179 DEBUG [TezTR-518813_1_7_0_0_0] log.PerfLogger: 2018-01-05T11:28:44,179 DEBUG [TezTR-518813_1_7_0_0_0] mr.ObjectCache: djaiswal_20180105112843_5fff5c3e-5bd5-4b02-82c4-dfba8563e391_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:28:44,179 DEBUG [TezTR-518813_1_7_0_0_0] exec.MapOperator: close called for operator MAP[0] 2018-01-05T11:28:44,179 INFO [TezTR-518813_1_7_0_0_0] exec.MapOperator: Closing operator MAP[0] 2018-01-05T11:28:44,179 INFO [TezTR-518813_1_7_0_0_0] exec.MapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:500, 2018-01-05T11:28:44,179 DEBUG [TezTR-518813_1_7_0_0_0] exec.MapOperator: Closing child = TS[0] 2018-01-05T11:28:44,179 DEBUG [TezTR-518813_1_7_0_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:28:44,179 DEBUG [TezTR-518813_1_7_0_0_0] exec.TableScanOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:44,179 INFO [TezTR-518813_1_7_0_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:28:44,179 DEBUG [TezTR-518813_1_7_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:44,179 DEBUG [TezTR-518813_1_7_0_0_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_960_8916097632880633826-1/-mr-10000 2018-01-05T11:28:44,180 INFO [TezTR-518813_1_7_0_0_0] exec.TableScanOperator: publishing : default.src_sequencefile/ : {numRows=500, rawDataSize=5312} 2018-01-05T11:28:44,204 DEBUG [TezTR-518813_1_7_0_0_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_960_8916097632880633826-1/-mr-10000/tmpstats-0_0 2018-01-05T11:28:44,204 DEBUG [TezTR-518813_1_7_0_0_0] fs.FSStatsPublisher: Writing stats in it : {default.src_sequencefile/={numRows=500, rawDataSize=5312}} 2018-01-05T11:28:44,205 DEBUG [TezTR-518813_1_7_0_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:28:44,205 DEBUG [TezTR-518813_1_7_0_0_0] exec.MapOperator: 0 Close done 2018-01-05T11:28:44,205 INFO [TezTR-518813_1_7_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_7_00_000000_0 2018-01-05T11:28:44,205 INFO [TezTR-518813_1_7_0_0_0] output.MROutput: out_Map 1 closed 2018-01-05T11:28:44,205 DEBUG [TezTR-518813_1_7_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:44,205 INFO [TezTR-518813_1_7_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:44,205 INFO [TezTR-518813_1_7_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_7_00_000000_0, askedToStop=false 2018-01-05T11:28:44,205 INFO [TezTR-518813_1_7_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_7_00_000000_0, stopRequested=false 2018-01-05T11:28:44,205 INFO [TezTR-518813_1_7_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_7_00_000000_0: Counters: 5 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=500, INPUT_SPLIT_LENGTH_BYTES=10508, OUTPUT_RECORDS=0][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500]] 2018-01-05T11:28:44,206 DEBUG [TezTR-518813_1_7_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_7_0_0_0 thread id: 556 scheme: file bytesRead: 14788 bytesWritten: 95 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:44,206 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_7_00_000000_0, isTaskDone=true 2018-01-05T11:28:44,206 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000010, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_7_00_000000_0, eventCount=2 } 2018-01-05T11:28:44,208 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_7][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_7_00_000000_0, creationTime=1515180524137, allocationTime=1515180524139, startTime=1515180524143, finishTime=1515180524208, timeTaken=65, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:44,208 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:44,208 INFO [Task-Executor-3] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_7_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:44,208 INFO [Task-Executor-3] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_7_00_000000_0 2018-01-05T11:28:44,208 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_7_00_000000_0 2018-01-05T11:28:44,208 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_7_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:44,208 INFO [Task-Executor-3] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000010=51 2018-01-05T11:28:44,208 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_7_00_000000_0: true 2018-01-05T11:28:44,208 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_7_00_000000_0; false 2018-01-05T11:28:44,208 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_7_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:44,208 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_7_00_000000_0 2018-01-05T11:28:44,208 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:44,208 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000010, QueryId=djaiswal_20180105112843_5fff5c3e-5bd5-4b02-82c4-dfba8563e391-7, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:28:44,209 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_7_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:44,209 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_7][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_7_00_000000, startTime=1515180524143, finishTime=1515180524208, timeTaken=65, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_7_00_000000_0, diagnostics=, counters=Counters: 11, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=14788, FILE_BYTES_WRITTEN=95, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=500, INPUT_SPLIT_LENGTH_BYTES=10508, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500 2018-01-05T11:28:44,209 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_7_00_000000_0 2018-01-05T11:28:44,209 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:44,209 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_7_00 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:44,209 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_7_00 [Map 1] 2018-01-05T11:28:44,209 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_7][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_7_00, initRequestedTime=1515180524108, initedTime=1515180524135, startRequestedTime=1515180524116, startedTime=1515180524136, finishTime=1515180524209, timeTaken=73, status=SUCCEEDED, diagnostics=, counters=Counters: 11, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=14788, FILE_BYTES_WRITTEN=95, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=500, INPUT_SPLIT_LENGTH_BYTES=10508, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500, vertexStats=firstTaskStartTime=1515180524143, firstTasksToStart=[ task_1515180518813_0001_7_00_000000 ], lastTaskFinishTime=1515180524208, lastTasksToFinish=[ task_1515180518813_0001_7_00_000000 ], minTaskDuration=65, maxTaskDuration=65, avgTaskDuration=65.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_7_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_7_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:44,209 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_7_00 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:44,209 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_7][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000010, stoppedTime=1515180524209, exitStatus=0 2018-01-05T11:28:44,210 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000010 exited with diagnostics set to Container completed. 2018-01-05T11:28:44,210 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_7_00 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1 2018-01-05T11:28:44,210 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null 2018-01-05T11:28:44,210 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_7_00 [Map 1] 2018-01-05T11:28:44,210 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_7, queueSize=0 2018-01-05T11:28:44,210 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_7][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_7, startTime=1515180524108, finishTime=1515180524210, timeTaken=102, status=SUCCEEDED, diagnostics=, counters=Counters: 15, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, OTHER_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=205, AM_GC_TIME_MILLIS=0, File System Counters, FILE_BYTES_READ=14788, FILE_BYTES_WRITTEN=95, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=500, INPUT_SPLIT_LENGTH_BYTES=10508, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500 2018-01-05T11:28:44,211 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_7 finished with state: SUCCEEDED 2018-01-05T11:28:44,211 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_7 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:28:44,211 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000010 2018-01-05T11:28:44,211 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_7, dagState=SUCCEEDED 2018-01-05T11:28:44,211 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_7_post 2018-01-05T11:28:44,211 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:44,211 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:28:44,211 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:28:44,211 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 7 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=1, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=1,NumRequestsWithoutLocation=1, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=1} 2018-01-05T11:28:44,211 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:28:44,211 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=ANALYZE TABLE src_sequencefile ...STATISTICS (Stage-0), with id=dag_1515180518813_0001_7 2018-01-05T11:28:44,211 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 7, to 10.22.8.182:49818 2018-01-05T11:28:44,211 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=1, Remaining=0 2018-01-05T11:28:44,211 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=ANALYZE TABLE src_sequencefile ...STATISTICS (Stage-0), with id=dag_1515180518813_0001_7 2018-01-05T11:28:44,212 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=7} 2018-01-05T11:28:44,212 INFO [IPC Server handler 0 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=7}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:28:44,212 INFO [IPC Server handler 0 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/7 after 300 seconds 2018-01-05T11:28:44,212 ERROR [IPC Server handler 0 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112843_5fff5c3e-5bd5-4b02-82c4-dfba8563e391, dag_1515180518813_0001_7 2018-01-05T11:28:44,212 INFO [IPC Server handler 0 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112843_5fff5c3e-5bd5-4b02-82c4-dfba8563e391 2018-01-05T11:28:44,212 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=7} = 0 2018-01-05T11:28:44,212 DEBUG [IPC Server handler 0 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=7} 2018-01-05T11:28:44,212 DEBUG [IPC Server handler 0 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:28:44,212 DEBUG [IPC Server handler 0 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=7} 2018-01-05T11:28:44,212 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,212 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,212 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 2018-01-05T11:28:44,212 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,213 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-2:STATS] in serial mode 2018-01-05T11:28:44,213 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@5276330a, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:44,213 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:44,213 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,213 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:44,213 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:44,213 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:44,213 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:44,213 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,213 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:44,213 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:44,213 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_960_8916097632880633826-1/-mr-10000 2018-01-05T11:28:44,213 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_960_8916097632880633826-1/-mr-10000 2018-01-05T11:28:44,214 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.src_sequencefile/ numRows 2018-01-05T11:28:44,214 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.src_sequencefile/ numRows 500 2018-01-05T11:28:44,215 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.src_sequencefile/ rawDataSize 2018-01-05T11:28:44,215 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.src_sequencefile/ rawDataSize 5312 2018-01-05T11:28:44,216 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,216 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:44,216 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:44,216 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:44,216 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:44,217 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,217 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,217 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,218 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,219 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,219 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:44,219 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:44,219 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,219 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:44,219 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,219 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=src_sequencefile newtbl=src_sequencefile 2018-01-05T11:28:44,219 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=src_sequencefile newtbl=src_sequencefile 2018-01-05T11:28:44,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,231 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,231 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,232 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.src_sequencefile stats: [numFiles=1, numRows=500, totalSize=10508, rawDataSize=5312] 2018-01-05T11:28:44,232 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: About to delete stats tmp dir :file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_960_8916097632880633826-1/-mr-10000 2018-01-05T11:28:44,232 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,232 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,232 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: ANALYZE TABLE src_sequencefile COMPUTE STATISTICS 2018-01-05T11:28:44,232 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:28:44,232 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@src_sequencefile 2018-01-05T11:28:44,232 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@src_sequencefile 2018-01-05T11:28:44,232 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,232 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,232 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:44,232 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=13} 2018-01-05T11:28:44,232 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112843_5fff5c3e-5bd5-4b02-82c4-dfba8563e391); Time taken: 0.23 seconds 2018-01-05T11:28:44,232 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:44,232 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,232 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:44,232 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,232 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query ANALYZE TABLE src_sequencefile COMPUTE STATISTICS 2018-01-05T11:28:44,232 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-43_960_8916097632880633826-1 2018-01-05T11:28:44,233 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.272 seconds 2018-01-05T11:28:44,233 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:44,233 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:44,233 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:44,233 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:44,233 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,233 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:44,233 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,233 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,233 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,233 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: ANALYZE TABLE src_sequencefile COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:44,233 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112844_3409fb09-65bb-44b3-ba39-c8970e1323c4): ANALYZE TABLE src_sequencefile COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:44,233 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,233 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: ANALYZE TABLE src_sequencefile COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:44,234 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:44,234 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,234 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,234 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,234 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,234 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:44,234 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,234 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_sequencefile 2018-01-05T11:28:44,234 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_sequencefile 2018-01-05T11:28:44,238 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,238 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,238 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: select compute_stats(`key`, 'hll') , compute_stats(`value`, 'hll') from `default`.`src_sequencefile` 2018-01-05T11:28:44,239 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: select compute_stats(`key`, 'hll') , compute_stats(`value`, 'hll') from `default`.`src_sequencefile` 2018-01-05T11:28:44,239 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:44,239 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Invoking analyze on rewritten query 2018-01-05T11:28:44,239 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Starting Semantic Analysis 2018-01-05T11:28:44,239 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:44,239 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:44,239 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed phase 1 of Semantic Analysis 2018-01-05T11:28:44,239 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for source tables 2018-01-05T11:28:44,240 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,240 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_sequencefile 2018-01-05T11:28:44,240 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_sequencefile 2018-01-05T11:28:44,243 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,244 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,244 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for subqueries 2018-01-05T11:28:44,244 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for destination tables 2018-01-05T11:28:44,255 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed getting MetaData in Semantic Analysis 2018-01-05T11:28:44,255 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_239_3384404240884212623-1/-mr-10001 2018-01-05T11:28:44,255 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Table Plan for src_sequencefile TS[0] 2018-01-05T11:28:44,256 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:44,256 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:44,256 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: RR before GB src_sequencefile{(key,key: string)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)(RAW__DATA__SIZE,RAW__DATA__SIZE: bigint)} after GB {((tok_function compute_stats (tok_table_or_col key) 'hll'),_col0: struct)((tok_function compute_stats (tok_table_or_col value) 'hll'),_col1: struct)} 2018-01-05T11:28:44,256 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: tree: (tok_select (tok_selexpr (tok_function compute_stats (tok_table_or_col key) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col value) 'hll'))) 2018-01-05T11:28:44,256 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: genSelectPlan: input = {((tok_function compute_stats (tok_table_or_col key) 'hll'),_col0: struct)((tok_function compute_stats (tok_table_or_col value) 'hll'),_col1: struct)} starRr = src_sequencefile{(key,key: string)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)(RAW__DATA__SIZE,RAW__DATA__SIZE: bigint)} 2018-01-05T11:28:44,257 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Select Plan row schema: null{(_c0,_col0: struct)(_c1,_col1: struct)} 2018-01-05T11:28:44,257 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Select Plan for clause: insclause-0 2018-01-05T11:28:44,257 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_239_3384404240884212623-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-44_239_3384404240884212623-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_239_3384404240884212623-1/-mr-10000 2018-01-05T11:28:44,257 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_239_3384404240884212623-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-44_239_3384404240884212623-1 2018-01-05T11:28:44,279 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_239_3384404240884212623-1/-mr-10003 2018-01-05T11:28:44,279 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_239_3384404240884212623-1/-mr-10000 row schema: null{(_c0,_col0: struct)(_c1,_col1: struct)} 2018-01-05T11:28:44,279 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Body Plan for Query Block null 2018-01-05T11:28:44,279 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Plan for Query Block null 2018-01-05T11:28:44,279 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Before logical optimization TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-SEL[5]-FS[6] 2018-01-05T11:28:44,279 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,279 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,279 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,280 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] lineage.Generator: Time taken for lineage transform=1 2018-01-05T11:28:44,280 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,280 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,280 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,280 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,280 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,280 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,280 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,280 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,280 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{src_sequencefile}key,value: string|{src_sequencefile}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src_sequencefile}block__offset__inside__file,INPUT__FILE__NAME: string|{src_sequencefile}input__file__name,ROW__ID: struct|{src_sequencefile}row__id,RAW__DATA__SIZE: bigint|{src_sequencefile}raw__data__size) 2018-01-05T11:28:44,280 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: string|{src_sequencefile}key,value: string|{src_sequencefile}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src_sequencefile}block__offset__inside__file,INPUT__FILE__NAME: string|{src_sequencefile}input__file__name,ROW__ID: struct|{src_sequencefile}row__id,RAW__DATA__SIZE: bigint|{src_sequencefile}raw__data__size) 2018-01-05T11:28:44,280 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:44,280 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:44,280 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:28:44,280 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:44,280 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:44,280 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:44,280 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:44,280 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:44,280 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1) 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0] Column[_col1]) 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{}) 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,281 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for FS(6) 2018-01-05T11:28:44,281 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(5) 2018-01-05T11:28:44,281 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for GBY(4) 2018-01-05T11:28:44,281 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for RS(3) 2018-01-05T11:28:44,281 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for GBY(2) 2018-01-05T11:28:44,281 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(1) 2018-01-05T11:28:44,281 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.PredicatePushDown: After PPD: TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-SEL[5]-FS[6] 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{src_sequencefile}key,value: string|{src_sequencefile}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src_sequencefile}block__offset__inside__file,INPUT__FILE__NAME: string|{src_sequencefile}input__file__name,ROW__ID: struct|{src_sequencefile}row__id,RAW__DATA__SIZE: bigint|{src_sequencefile}raw__data__size) 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: string|{src_sequencefile}key,value: string|{src_sequencefile}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src_sequencefile}block__offset__inside__file,INPUT__FILE__NAME: string|{src_sequencefile}input__file__name,ROW__ID: struct|{src_sequencefile}row__id,RAW__DATA__SIZE: bigint|{src_sequencefile}raw__data__size) 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1) 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0] Column[_col1]) 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{}) 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{src_sequencefile}key,value: string|{src_sequencefile}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src_sequencefile}block__offset__inside__file,INPUT__FILE__NAME: string|{src_sequencefile}input__file__name,ROW__ID: struct|{src_sequencefile}row__id,RAW__DATA__SIZE: bigint|{src_sequencefile}raw__data__size) 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: string|{src_sequencefile}key,value: string|{src_sequencefile}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src_sequencefile}block__offset__inside__file,INPUT__FILE__NAME: string|{src_sequencefile}input__file__name,ROW__ID: struct|{src_sequencefile}row__id,RAW__DATA__SIZE: bigint|{src_sequencefile}raw__data__size) 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1) 2018-01-05T11:28:44,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:44,282 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:44,282 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0] Column[_col1]) 2018-01-05T11:28:44,282 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{}) 2018-01-05T11:28:44,282 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:44,282 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:44,282 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,282 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,282 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,282 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,282 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 3 key:[] 2018-01-05T11:28:44,282 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 3 oldColExprMap: {VALUE._col0=Column[_col0], VALUE._col1=Column[_col1]} 2018-01-05T11:28:44,282 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 3 newColExprMap: {VALUE._col0=Column[_col0], VALUE._col1=Column[_col1]} 2018-01-05T11:28:44,282 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,282 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,282 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,282 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,282 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,282 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,282 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,282 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,283 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,283 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,283 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,283 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,283 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,283 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,283 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,283 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,283 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,283 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,283 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.IdentityProjectRemover: Identity project remover optimization removed : SEL[5] 2018-01-05T11:28:44,283 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,283 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,283 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,283 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,283 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,283 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,283 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,283 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: After logical optimization TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-FS[6] 2018-01-05T11:28:44,283 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,283 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,283 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,283 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,283 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,283 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,283 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=src_sequencefile 2018-01-05T11:28:44,284 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=src_sequencefile 2018-01-05T11:28:44,288 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.270597ms + 0.007618ms, the query is [select "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "TAB_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (...)] 2018-01-05T11:28:44,289 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,295 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,296 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,296 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting stats (Num rows: 500 Data size: 175168 Basic stats: COMPLETE Column stats: NONE) on TS[0] 2018-01-05T11:28:44,296 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (src_sequencefile): numRows: 500 dataSize: 175168 basicStatsState: COMPLETE colStatsState: NONE colStats: {value= colName: value colType: string countDistincts: 100 numNulls: 25 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: string countDistincts: 100 numNulls: 25 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true} 2018-01-05T11:28:44,296 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting stats (Num rows: 500 Data size: 175168 Basic stats: COMPLETE Column stats: NONE) on SEL[1] 2018-01-05T11:28:44,296 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [1] STATS-SEL[1]: numRows: 500 dataSize: 175168 basicStatsState: COMPLETE colStatsState: NONE colStats: {value= colName: value colType: string countDistincts: 100 numNulls: 25 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: string countDistincts: 100 numNulls: 25 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true} 2018-01-05T11:28:44,296 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[2]: inputSize: 175168 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:28:44,296 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [Case 1] STATS-GBY[2]: cardinality: 500 2018-01-05T11:28:44,296 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 1248 Basic stats: COMPLETE Column stats: NONE) on GBY[2] 2018-01-05T11:28:44,296 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-GBY[2]: numRows: 1 dataSize: 1248 basicStatsState: COMPLETE colStatsState: NONE colStats: {_col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true} 2018-01-05T11:28:44,296 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 1248 Basic stats: COMPLETE Column stats: NONE) on RS[3] 2018-01-05T11:28:44,296 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-RS[3]: numRows: 1 dataSize: 1248 basicStatsState: COMPLETE colStatsState: NONE colStats: {_col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true} 2018-01-05T11:28:44,296 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[4]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:28:44,296 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [Case 7] STATS-GBY[4]: cardinality: 0 2018-01-05T11:28:44,296 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[4]: Equals 0 in number of rows. 0 rows will be set to 1 2018-01-05T11:28:44,296 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 1248 Basic stats: COMPLETE Column stats: NONE) on GBY[4] 2018-01-05T11:28:44,296 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-GBY[4]: numRows: 1 dataSize: 1248 basicStatsState: COMPLETE colStatsState: NONE colStats: {_col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true} 2018-01-05T11:28:44,296 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-FS[6]: numRows: 1 dataSize: 1248 basicStatsState: COMPLETE colStatsState: NONE colStats: {_col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true} 2018-01-05T11:28:44,296 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on TS[0] 2018-01-05T11:28:44,296 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on SEL[1] 2018-01-05T11:28:44,296 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[2] 2018-01-05T11:28:44,296 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[3] 2018-01-05T11:28:44,296 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[4] 2018-01-05T11:28:44,296 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FileSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on FS[6] 2018-01-05T11:28:44,296 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,296 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,296 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: FS, 6 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: GBY, 4 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: RS, 3 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: GBY, 2 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: SEL, 1 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:28:44,297 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Cycle free: true 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{src_sequencefile}key,value: string|{src_sequencefile}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src_sequencefile}block__offset__inside__file,INPUT__FILE__NAME: string|{src_sequencefile}input__file__name,ROW__ID: struct|{src_sequencefile}row__id,RAW__DATA__SIZE: bigint|{src_sequencefile}raw__data__size) 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: string|{src_sequencefile}key,value: string|{src_sequencefile}value) 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value]) 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{}) 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: TS[0] 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: RS[3] 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile of table src_sequencefile 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: RS[3] 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: GBY[4] 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: FS[6] 2018-01-05T11:28:44,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding reduce work (Reducer 2) for GBY[4] 2018-01-05T11:28:44,298 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Setting up reduce sink: RS[3] with following reduce work: Reducer 2 2018-01-05T11:28:44,298 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Removing RS[3] as parent from GBY[4] 2018-01-05T11:28:44,298 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: FS[6] 2018-01-05T11:28:44,298 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:28:44,298 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,298 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,298 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:44,298 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:44,298 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:44,298 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:44,298 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:44,298 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:44,298 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:28:44,298 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping vectorization 2018-01-05T11:28:44,298 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:28:44,298 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: llap mode: only 2018-01-05T11:28:44,298 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:28:44,298 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(key, 'hll')' 2018-01-05T11:28:44,298 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'key' 2018-01-05T11:28:44,298 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:44,298 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(value, 'hll')' 2018-01-05T11:28:44,298 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'value' 2018-01-05T11:28:44,298 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:44,298 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'key' 2018-01-05T11:28:44,298 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'value' 2018-01-05T11:28:44,298 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:44,298 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@6c3830ed can be done in LLAP 2018-01-05T11:28:44,298 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Reducer 2 2018-01-05T11:28:44,298 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col0)' 2018-01-05T11:28:44,298 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:28:44,298 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col1)' 2018-01-05T11:28:44,298 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col1' 2018-01-05T11:28:44,298 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:44,298 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@32a541a8 can be done in LLAP 2018-01-05T11:28:44,298 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,298 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed plan generation 2018-01-05T11:28:44,298 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:44,298 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validation start 2018-01-05T11:28:44,298 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Not a partition. 2018-01-05T11:28:44,298 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:28:44,298 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,298 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0, type:struct, comment:null), FieldSchema(name:_c1, type:struct, comment:null)], properties:null) 2018-01-05T11:28:44,298 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,298 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:44,299 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=8, flushCache_()=0, getTableColumnStatistics_(String, String, List, )=13} 2018-01-05T11:28:44,299 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112844_3409fb09-65bb-44b3-ba39-c8970e1323c4); Time taken: 0.065 seconds 2018-01-05T11:28:44,299 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:44,299 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,299 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112844_3409fb09-65bb-44b3-ba39-c8970e1323c4): ANALYZE TABLE src_sequencefile COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:44,300 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,300 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: ANALYZE TABLE src_sequencefile COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:44,300 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:28:44,300 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@src_sequencefile 2018-01-05T11:28:44,300 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@src_sequencefile 2018-01-05T11:28:44,300 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_239_3384404240884212623-1/-mr-10000 2018-01-05T11:28:44,300 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,300 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,300 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,300 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Query ID = djaiswal_20180105112844_3409fb09-65bb-44b3-ba39-c8970e1323c4 2018-01-05T11:28:44,300 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Total jobs = 1 2018-01-05T11:28:44,300 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,300 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:28:44,300 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MAPRED] in serial mode 2018-01-05T11:28:44,310 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000010, QueryId=djaiswal_20180105112843_5fff5c3e-5bd5-4b02-82c4-dfba8563e391-7, DagId=7, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-3, Succeeded=true, StartTime=1515180524156, EndTime=1515180524310 2018-01-05T11:28:44,324 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_233_6189264957254526832-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:28:44,347 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:28:44,347 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:28:44,347 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:28:44,347 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112844_3409fb09-65bb-44b3-ba39-c8970e1323c4 2018-01-05T11:28:44,347 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Session is already open 2018-01-05T11:28:44,347 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:28:44,347 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,347 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Dag name: ANALYZE TABLE src_sequencefile C...key,value (Stage-0) 2018-01-05T11:28:44,347 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: DagInfo: {"context":"Hive","description":"\n\nANALYZE TABLE src_sequencefile COMPUTE STATISTICS FOR COLUMNS key,value"} 2018-01-05T11:28:44,347 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112844_3409fb09-65bb-44b3-ba39-c8970e1323c4 with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:28:44,347 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,359 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,359 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:28:44,360 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,360 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 2 size: 2.01KB 2018-01-05T11:28:44,378 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,378 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,390 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:28:44,390 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,390 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:28:44,391 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,391 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 3.18KB 2018-01-05T11:28:44,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_239_3384404240884212623-1/-mr-10001 2018-01-05T11:28:44,408 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_239_3384404240884212623-1/-mr-10001 2018-01-05T11:28:44,408 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile 2018-01-05T11:28:44,408 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,411 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,411 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,411 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=ANALYZE TABLE src_sequencefile C...key,value (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112844_3409fb09-65bb-44b3-ba39-c8970e1323c4 } 2018-01-05T11:28:44,411 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:44,412 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:44,415 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: ANALYZE TABLE src_sequencefile C...key,value (Stage-0) 2018-01-05T11:28:44,415 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:28:44,415 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_8, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_8.dot 2018-01-05T11:28:44,416 INFO [IPC Server handler 0 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_8 2018-01-05T11:28:44,416 WARN [IPC Server handler 0 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:44,416 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Running DAG: ANALYZE TABLE src_sequencefile C...key,value (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112844_3409fb09-65bb-44b3-ba39-c8970e1323c4 } 2018-01-05T11:28:44,439 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_8][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_8, submitTime=1515180524415, queueName=null 2018-01-05T11:28:44,440 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_8_00 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_8_00 [Map 1], Default Resources= 2018-01-05T11:28:44,440 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_8_00 [Map 1] 2018-01-05T11:28:44,440 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:28:44,440 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_8_00 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:44,440 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_8_01 [Reducer 2]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_8_01 [Reducer 2], Default Resources= 2018-01-05T11:28:44,440 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_8_01 [Reducer 2] 2018-01-05T11:28:44,440 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 2 2018-01-05T11:28:44,441 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_8_01 [Reducer 2] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:44,441 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:28:44,441 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_8][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_8, initTime=1515180524441 2018-01-05T11:28:44,441 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: dag_1515180518813_0001_8 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:28:44,441 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:28:44,441 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_8][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_8, startTime=1515180524441 2018-01-05T11:28:44,442 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_8 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:28:44,442 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {src_sequencefile={InputName=src_sequencefile}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:28:44,442 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: src_sequencefile, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:28:44,442 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_8_00 [Map 1] 2018-01-05T11:28:44,442 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_8, dagName=ANALYZE TABLE src_sequencefile C...key,value (Stage-0) 2018-01-05T11:28:44,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,447 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_8_00 [Map 1] 2018-01-05T11:28:44,447 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_8_00 [Map 1] 2018-01-05T11:28:44,447 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:44,447 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_8_00 [Map 1] 2018-01-05T11:28:44,450 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:28:44,450 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@5b4bf9f1 2018-01-05T11:28:44,450 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_233_6189264957254526832-1/djaiswal/_tez_scratch_dir/54f46451-4013-4ad0-9504-fcabfb597deb/map.xml 2018-01-05T11:28:44,450 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_233_6189264957254526832-1/djaiswal/_tez_scratch_dir/54f46451-4013-4ad0-9504-fcabfb597deb/map.xml 2018-01-05T11:28:44,450 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_233_6189264957254526832-1/djaiswal/_tez_scratch_dir/54f46451-4013-4ad0-9504-fcabfb597deb/map.xml 2018-01-05T11:28:44,450 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:44,450 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:44,450 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:44,452 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:44,452 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.18KB 2018-01-05T11:28:44,452 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_8_00 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:28:44,452 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to ShuffleVertexManager for vertex_1515180518813_0001_8_01 [Reducer 2] 2018-01-05T11:28:44,455 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: src_sequencefile on vertex vertex_1515180518813_0001_8_00 [Map 1] 2018-01-05T11:28:44,455 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:28:44,455 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:44,455 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:28:44,455 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:44,455 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_233_6189264957254526832-1/djaiswal/_tez_scratch_dir/54f46451-4013-4ad0-9504-fcabfb597deb/map.xml 2018-01-05T11:28:44,455 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:44,455 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias src_sequencefile 2018-01-05T11:28:44,455 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile 2018-01-05T11:28:44,455 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile 2018-01-05T11:28:44,456 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 0,1 2018-01-05T11:28:44,456 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = key,value 2018-01-05T11:28:44,456 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile 2018-01-05T11:28:44,457 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:44,457 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:44,457 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:44,457 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:44,462 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.05 maxFrac: 0.75 auto: false desiredTaskIput: 104857600 2018-01-05T11:28:44,462 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:28:44,462 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 1 tasks for vertex: vertex_1515180518813_0001_8_01 [Reducer 2] 2018-01-05T11:28:44,462 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_8_01 [Reducer 2] 2018-01-05T11:28:44,463 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_8][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_8_01, reconfigureDoneTime=1515180524462, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:28:44,463 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_8_01 [Reducer 2], numAdditionalOutputs=1 2018-01-05T11:28:44,463 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_8][Event:VERTEX_INITIALIZED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_8_01, initRequestedTime=1515180524452, initedTime=1515180524463, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:44,463 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_8_01 [Reducer 2] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:28:44,473 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Time taken to get FileStatuses: 16 2018-01-05T11:28:44,473 INFO [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:28:44,473 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 16 2018-01-05T11:28:44,473 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 1 2018-01-05T11:28:44,473 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:44,473 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 1. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:28:44,473 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_233_6189264957254526832-1/djaiswal/_tez_scratch_dir/54f46451-4013-4ad0-9504-fcabfb597deb/map.xml 2018-01-05T11:28:44,473 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_233_6189264957254526832-1/djaiswal/_tez_scratch_dir/54f46451-4013-4ad0-9504-fcabfb597deb/map.xml 2018-01-05T11:28:44,473 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_233_6189264957254526832-1/djaiswal/_tez_scratch_dir/54f46451-4013-4ad0-9504-fcabfb597deb/map.xml 2018-01-05T11:28:44,474 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:44,474 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:44,474 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:44,475 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:44,475 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.18KB 2018-01-05T11:28:44,475 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile/kv1.seq to src new group? true 2018-01-05T11:28:44,475 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:28:44,475 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:28:44,475 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:28:44,475 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 194 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 10508 Original splits: 1 2018-01-05T11:28:44,475 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Using original number of splits: 1 desired splits: 1 2018-01-05T11:28:44,475 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 1 grouped split count is 1, for bucket: 1 2018-01-05T11:28:44,475 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:28:44,475 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: src_sequencefile on vertex vertex_1515180518813_0001_8_00 [Map 1] 2018-01-05T11:28:44,476 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_8_00 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:44,476 INFO [App Shared Pool - #0] impl.VertexImpl: Got updated RootInputsSpecs: {src_sequencefile=forAllWorkUnits=true, update=[1]} 2018-01-05T11:28:44,476 INFO [App Shared Pool - #0] impl.VertexImpl: Vertex vertex_1515180518813_0001_8_00 [Map 1] parallelism set to 1 2018-01-05T11:28:44,476 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_8_00 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:44,477 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_8][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_8_00, initRequestedTime=1515180524442, initedTime=1515180524477, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:44,477 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_8_00 [Map 1] with distanceFromRoot: 0 2018-01-05T11:28:44,477 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_8_00 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:28:44,477 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_8][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_8_00, startRequestedTime=1515180524452, startedTime=1515180524477 2018-01-05T11:28:44,477 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_8][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_8_00, reconfigureDoneTime=1515180524477, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:28:44,477 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_8_00 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:44,477 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_8_00 for vertex: vertex_1515180518813_0001_8_01 [Reducer 2] numStartedSources: 1 numSources: 1 2018-01-05T11:28:44,477 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_8_01 [Reducer 2] with distanceFromRoot: 1 2018-01-05T11:28:44,477 INFO [App Shared Pool - #0] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:28:44,477 INFO [App Shared Pool - #0] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:28:44,477 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_8_00 [Map 1] 2018-01-05T11:28:44,477 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_8][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_8_01, startRequestedTime=1515180524477, startedTime=1515180524477 2018-01-05T11:28:44,477 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_8_01 [Reducer 2] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:44,478 INFO [App Shared Pool - #1] vertexmanager.ShuffleVertexManagerBase: OnVertexStarted vertex: Reducer 2 with 0 source tasks and 1 pending tasks 2018-01-05T11:28:44,478 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_8][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_8_00_000000, scheduledTime=1515180524478, launchTime=1515180524478 2018-01-05T11:28:44,478 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Map 1 in vertex: Reducer 2 numBipartiteSourceTasks: 1 2018-01-05T11:28:44,478 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_8_00 [Map 1] 2018-01-05T11:28:44,478 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_8_00_000000_0, priority=5, capability=, hosts=null 2018-01-05T11:28:44,478 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=5,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:44,478 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=5,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:44,478 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:44,478 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_8_00_000000_0 on hosts=null 2018-01-05T11:28:44,478 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_8_00_000000_0 on hosts=null 2018-01-05T11:28:44,478 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_8_00_000000_0 2018-01-05T11:28:44,478 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:44,479 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #11, task=TaskInfo{task=attempt_1515180518813_0001_8_00_000000_0, priority=5, startTime=0, containerId=null, uniqueId=10, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000011 2018-01-05T11:28:44,479 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_8_00_000000_0; false 2018-01-05T11:28:44,479 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_8_00_000000_0, priority=5, startTime=311544987, containerId=container_111101111_0001_00_000011assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=10, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:44,480 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000011 for node: 10.22.8.182:49818 2018-01-05T11:28:44,482 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 8, name=ANALYZE TABLE src_sequencefile C...key,value (Stage-0), queryId=djaiswal_20180105112844_3409fb09-65bb-44b3-ba39-c8970e1323c4 2018-01-05T11:28:44,482 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000011, attempt_1515180518813_0001_8_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:44,482 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000011 for node: 10.22.8.182:49818 2018-01-05T11:28:44,483 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000011 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:44,483 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000011, launchTime=1515180524483 2018-01-05T11:28:44,483 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_8_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112844_3409fb09-65bb-44b3-ba39-c8970e1323c4, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000011, dagName=ANALYZE TABLE src_sequencefile C...key,value (Stage-0), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{src_sequencefile,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{Reducer 2,org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput,1}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180524441, firstAttemptStartTime=1515180524478, currentAttemptStartTime=1515180524482} 2018-01-05T11:28:44,483 INFO [IPC Server handler 2 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000011, QueryId=djaiswal_20180105112844_3409fb09-65bb-44b3-ba39-c8970e1323c4-8, DagId=8, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180524483 2018-01-05T11:28:44,483 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_8_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000011] on NM: [10.22.8.182:49818] 2018-01-05T11:28:44,483 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:44,483 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:44,484 DEBUG [IPC Server handler 2 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=8} with the ShuffleHandler 2018-01-05T11:28:44,484 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_8][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_8_00_000000_0, startTime=1515180524483, containerId=container_111101111_0001_00_000011, nodeId=10.22.8.182:49818 2018-01-05T11:28:44,496 INFO [IPC Server handler 2 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:28:44,496 DEBUG [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/8] 2018-01-05T11:28:44,496 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:44,496 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_8_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:44,496 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_8_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180524478, dagStartTime=1515180524441, withinDagPriority=5, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:28:44,496 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_8_00_000000_0 : ACCEPTED 2018-01-05T11:28:44,496 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_8_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:44,496 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_8_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180524478, dagStartTime=1515180524441, withinDagPriority=5, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:28:44,496 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_8_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:44,496 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_8_00_000000_0 to pre-emption queue 2018-01-05T11:28:44,496 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_8_00_000000_0: true 2018-01-05T11:28:44,496 INFO [TaskCommunicator # 0] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_8_00_000000_0 2018-01-05T11:28:44,497 INFO [TaskCommunicator # 0] GuaranteedTasks: Received response for attempt_1515180518813_0001_8_00_000000_0, true 2018-01-05T11:28:44,497 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: taskOwner hashCode:606910175 2018-01-05T11:28:44,497 INFO [Task-Executor-3] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : ANALYZE TABLE src_sequencefile C...key,value (Stage-0), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_8_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=src_sequencefile, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=Reducer 2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput }}, ] 2018-01-05T11:28:44,498 INFO [Task-Executor-3] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=954728448, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:44,498 INFO [Task-Executor-3] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_8_00_000000_0 vertexName: Map 1 2018-01-05T11:28:44,498 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000011, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_8_00_000000_0, eventCount=1 } 2018-01-05T11:28:44,498 DEBUG [TezTR-518813_1_8_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_8 queryId: djaiswal_20180105112844_3409fb09-65bb-44b3-ba39-c8970e1323c4 instanceType: TaskRunner2Callable 2018-01-05T11:28:44,498 INFO [TezTR-518813_1_8_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_8_00_000000_0 2018-01-05T11:28:44,498 DEBUG [TezTR-518813_1_8_0_0_0] log.PerfLogger: 2018-01-05T11:28:44,499 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:44,499 INFO [I/O Setup 1 Initialize: {Reducer 2}] writers.UnorderedPartitionedKVWriter: Requested BufferSize (tez.runtime.unordered.output.buffer.size-mb) : 100 2018-01-05T11:28:44,500 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:44,503 INFO [IPC Server handler 5 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_8_00_000000_0 sent events: (0-1). 2018-01-05T11:28:44,503 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:28:44,503 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_8_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:28:44,504 DEBUG [TezTR-518813_1_8_0_0_0] log.PerfLogger: 2018-01-05T11:28:44,504 INFO [TezTR-518813_1_8_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:44,504 INFO [TezTR-518813_1_8_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:44,504 INFO [TezTR-518813_1_8_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:44,505 INFO [I/O Setup 0 Initialize: {src_sequencefile}] input.MRInput: src_sequencefile using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:28:44,505 INFO [I/O Setup 0 Initialize: {src_sequencefile}] input.MRInputLegacy: src_sequencefile MRInputLegacy deferring initialization 2018-01-05T11:28:44,505 INFO [TezTR-518813_1_8_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:44,505 INFO [TezTR-518813_1_8_0_0_0] resources.MemoryDistributor: InitialRequests=[src_sequencefile:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [Reducer 2:OUTPUT:104857600:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput] 2018-01-05T11:28:44,505 INFO [TezTR-518813_1_8_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:44,505 INFO [TezTR-518813_1_8_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:44,505 INFO [TezTR-518813_1_8_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 104857600, TotalRequestedScaled: 5.24288E7, TotalJVMHeap: 954728448, TotalAvailable: 639668060, TotalRequested/TotalJVMHeap:0.11 2018-01-05T11:28:44,505 INFO [TezTR-518813_1_8_0_0_0] resources.MemoryDistributor: Allocations=[src_sequencefile:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput:OUTPUT:104857600:104857600] 2018-01-05T11:28:44,505 INFO [TezTR-518813_1_8_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:44,505 INFO [TezTR-518813_1_8_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: src_sequencefile being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:28:44,505 INFO [TezTR-518813_1_8_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:28:44,505 INFO [TezTR-518813_1_8_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:28:44,505 INFO [I/O Setup 1 Start: {src_sequencefile}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: src_sequencefile 2018-01-05T11:28:44,505 INFO [TezTR-518813_1_8_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:44,505 INFO [TezTR-518813_1_8_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_8_00_000000_0 2018-01-05T11:28:44,505 DEBUG [TezTR-518813_1_8_0_0_0] log.PerfLogger: 2018-01-05T11:28:44,505 DEBUG [TezTR-518813_1_8_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_8_00_000000_0_10031 2018-01-05T11:28:44,505 DEBUG [TezTR-518813_1_8_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:44,505 DEBUG [TezTR-518813_1_8_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_8_00_000000_0 2018-01-05T11:28:44,505 INFO [TezTR-518813_1_8_0_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112844_3409fb09-65bb-44b3-ba39-c8970e1323c4 2018-01-05T11:28:44,505 DEBUG [TezTR-518813_1_8_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:44,505 DEBUG [TezTR-518813_1_8_0_0_0] log.PerfLogger: 2018-01-05T11:28:44,505 DEBUG [TezTR-518813_1_8_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:44,505 DEBUG [TezTR-518813_1_8_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:44,506 DEBUG [TezTR-518813_1_8_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112844_3409fb09-65bb-44b3-ba39-c8970e1323c4_Map 1__MAP_PLAN__ 2018-01-05T11:28:44,506 INFO [TezTR-518813_1_8_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_233_6189264957254526832-1/djaiswal/_tez_scratch_dir/54f46451-4013-4ad0-9504-fcabfb597deb/map.xml 2018-01-05T11:28:44,506 DEBUG [TezTR-518813_1_8_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_233_6189264957254526832-1/djaiswal/_tez_scratch_dir/54f46451-4013-4ad0-9504-fcabfb597deb/map.xml 2018-01-05T11:28:44,506 DEBUG [TezTR-518813_1_8_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_233_6189264957254526832-1/djaiswal/_tez_scratch_dir/54f46451-4013-4ad0-9504-fcabfb597deb/map.xml 2018-01-05T11:28:44,506 DEBUG [TezTR-518813_1_8_0_0_0] log.PerfLogger: 2018-01-05T11:28:44,506 INFO [TezTR-518813_1_8_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:44,506 DEBUG [TezTR-518813_1_8_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:44,507 DEBUG [TezTR-518813_1_8_0_0_0] log.PerfLogger: 2018-01-05T11:28:44,507 INFO [TezTR-518813_1_8_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.18KB 2018-01-05T11:28:44,507 DEBUG [TezTR-518813_1_8_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:44,507 INFO [TezTR-518813_1_8_0_0_0] tez.MapRecordProcessor: The input names are: [src_sequencefile] 2018-01-05T11:28:44,507 INFO [TezTR-518813_1_8_0_0_0] input.MRInput: Initialized MRInput: src_sequencefile 2018-01-05T11:28:44,508 DEBUG [TezTR-518813_1_8_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 1 2018-01-05T11:28:44,508 INFO [TezTR-518813_1_8_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_233_6189264957254526832-1/djaiswal/_tez_scratch_dir/54f46451-4013-4ad0-9504-fcabfb597deb/map.xml 2018-01-05T11:28:44,508 DEBUG [TezTR-518813_1_8_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:44,508 DEBUG [TezTR-518813_1_8_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile/kv1.seq PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.SequenceFileInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct src_sequencefile { string key, string value}, transient_lastDdlTime=1515180524, rawDataSize=5312, columns.comments=defaultdefault, totalSize=10508, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.SequenceFileInputFormat, column.name.delimiter=,, name=default.src_sequencefile}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.SequenceFileInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct src_sequencefile { string key, string value}, transient_lastDdlTime=1515180524, rawDataSize=5312, columns.comments=defaultdefault, totalSize=10508, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.SequenceFileInputFormat, column.name.delimiter=,, name=default.src_sequencefile}, baseFileName=src_sequencefile, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.SequenceFileInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct src_sequencefile { string key, string value}, transient_lastDdlTime=1515180524, rawDataSize=5312, columns.comments=defaultdefault, totalSize=10508, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.SequenceFileInputFormat, column.name.delimiter=,, name=default.src_sequencefile}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.SequenceFileInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct src_sequencefile { string key, string value}, transient_lastDdlTime=1515180524, rawDataSize=5312, columns.comments=defaultdefault, totalSize=10508, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.SequenceFileInputFormat, column.name.delimiter=,, name=default.src_sequencefile}, baseFileName=src_sequencefile, vectorPartitionDesc=null]} 2018-01-05T11:28:44,508 DEBUG [TezTR-518813_1_8_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.SequenceFileInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:44,508 INFO [TezTR-518813_1_8_0_0_0] io.HiveInputFormat: Not using llap for org.apache.hadoop.mapred.SequenceFileInputFormat: supported = false, vectorized = true, cache only = false 2018-01-05T11:28:44,508 DEBUG [TezTR-518813_1_8_0_0_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:44,508 DEBUG [TezTR-518813_1_8_0_0_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:44,508 DEBUG [TezTR-518813_1_8_0_0_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:44,508 DEBUG [TezTR-518813_1_8_0_0_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:44,509 DEBUG [TezTR-518813_1_8_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile/kv1.seq 2018-01-05T11:28:44,509 INFO [TezTR-518813_1_8_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.mapred.SequenceFileInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile/kv1.seq:0+10508], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=10508} 2018-01-05T11:28:44,509 INFO [TezTR-518813_1_8_0_0_0] input.MRInput: src_sequencefile initialized RecordReader from event 2018-01-05T11:28:44,509 DEBUG [TezTR-518813_1_8_0_0_0] tez.MapRecordProcessor: Starting Output: Reducer 2 2018-01-05T11:28:44,510 INFO [TezTR-518813_1_8_0_0_0] writers.BaseUnorderedPartitionedKVWriter: Instantiating Partitioner: [org.apache.tez.mapreduce.partition.MRPartitioner] 2018-01-05T11:28:44,529 INFO [TezTR-518813_1_8_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: numBuffers=2, sizePerBuffer=52428800, skipBuffers=true, numPartitions=1, availableMemory=104857600, maxSingleBufferSizeBytes=2147483647, pipelinedShuffle=false, isFinalMergeEnabled=true, numPartitions=1, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:28:44,529 INFO [TezTR-518813_1_8_0_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:28:44,529 INFO [TezTR-518813_1_8_0_0_0] exec.MapOperator: Initializing operator MAP[0] 2018-01-05T11:28:44,529 DEBUG [TezTR-518813_1_8_0_0_0] exec.MapOperator: Initialization Done 0 MAP 2018-01-05T11:28:44,529 DEBUG [TezTR-518813_1_8_0_0_0] exec.MapOperator: Operator 0 MAP initialized 2018-01-05T11:28:44,529 DEBUG [TezTR-518813_1_8_0_0_0] exec.MapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:28:44,530 DEBUG [TezTR-518813_1_8_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:44,530 DEBUG [TezTR-518813_1_8_0_0_0] exec.MapOperator: Adding alias src_sequencefile to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile 2018-01-05T11:28:44,530 DEBUG [TezTR-518813_1_8_0_0_0] exec.MapOperator: dump TS[0] struct 2018-01-05T11:28:44,530 INFO [TezTR-518813_1_8_0_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:28:44,530 INFO [TezTR-518813_1_8_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:44,530 DEBUG [TezTR-518813_1_8_0_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:28:44,530 INFO [TezTR-518813_1_8_0_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:28:44,530 DEBUG [TezTR-518813_1_8_0_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:28:44,530 DEBUG [TezTR-518813_1_8_0_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:28:44,530 DEBUG [TezTR-518813_1_8_0_0_0] exec.TableScanOperator: Initializing children of 0 TS 2018-01-05T11:28:44,530 DEBUG [TezTR-518813_1_8_0_0_0] exec.SelectOperator: Initializing child 1 SEL 2018-01-05T11:28:44,530 INFO [TezTR-518813_1_8_0_0_0] exec.SelectOperator: Initializing operator SEL[1] 2018-01-05T11:28:44,531 INFO [TezTR-518813_1_8_0_0_0] exec.SelectOperator: SELECT struct 2018-01-05T11:28:44,531 DEBUG [TezTR-518813_1_8_0_0_0] exec.SelectOperator: Initialization Done 1 SEL 2018-01-05T11:28:44,531 DEBUG [TezTR-518813_1_8_0_0_0] exec.SelectOperator: Operator 1 SEL initialized 2018-01-05T11:28:44,531 DEBUG [TezTR-518813_1_8_0_0_0] exec.SelectOperator: Initializing children of 1 SEL 2018-01-05T11:28:44,531 DEBUG [TezTR-518813_1_8_0_0_0] exec.GroupByOperator: Initializing child 2 GBY 2018-01-05T11:28:44,531 INFO [TezTR-518813_1_8_0_0_0] exec.GroupByOperator: Initializing operator GBY[2] 2018-01-05T11:28:44,531 DEBUG [TezTR-518813_1_8_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:44,531 DEBUG [TezTR-518813_1_8_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:44,531 INFO [TezTR-518813_1_8_0_0_0] exec.GroupByOperator: Max hash table memory: 93860656 bytes 2018-01-05T11:28:44,531 INFO [TezTR-518813_1_8_0_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:44,531 DEBUG [TezTR-518813_1_8_0_0_0] exec.GroupByOperator: Initialization Done 2 GBY 2018-01-05T11:28:44,531 DEBUG [TezTR-518813_1_8_0_0_0] exec.GroupByOperator: Operator 2 GBY initialized 2018-01-05T11:28:44,531 DEBUG [TezTR-518813_1_8_0_0_0] exec.GroupByOperator: Initializing children of 2 GBY 2018-01-05T11:28:44,531 DEBUG [TezTR-518813_1_8_0_0_0] exec.ReduceSinkOperator: Initializing child 3 RS 2018-01-05T11:28:44,531 INFO [TezTR-518813_1_8_0_0_0] exec.ReduceSinkOperator: Initializing operator RS[3] 2018-01-05T11:28:44,531 DEBUG [TezTR-518813_1_8_0_0_0] exec.ReduceSinkOperator: keys size is 0 2018-01-05T11:28:44,531 INFO [TezTR-518813_1_8_0_0_0] exec.ReduceSinkOperator: Using tag = -1 2018-01-05T11:28:44,531 DEBUG [TezTR-518813_1_8_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:44,531 DEBUG [TezTR-518813_1_8_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:44,531 DEBUG [TezTR-518813_1_8_0_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:28:44,531 DEBUG [TezTR-518813_1_8_0_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS 2018-01-05T11:28:44,531 DEBUG [TezTR-518813_1_8_0_0_0] exec.ReduceSinkOperator: Operator 3 RS initialized 2018-01-05T11:28:44,531 DEBUG [TezTR-518813_1_8_0_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS done is reset. 2018-01-05T11:28:44,531 DEBUG [TezTR-518813_1_8_0_0_0] exec.GroupByOperator: Initialization Done 2 GBY done is reset. 2018-01-05T11:28:44,531 DEBUG [TezTR-518813_1_8_0_0_0] exec.SelectOperator: Initialization Done 1 SEL done is reset. 2018-01-05T11:28:44,531 DEBUG [TezTR-518813_1_8_0_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:28:44,531 INFO [TezTR-518813_1_8_0_0_0] exec.OperatorUtils: Setting output collector: RS[3] --> Reducer 2 2018-01-05T11:28:44,531 DEBUG [TezTR-518813_1_8_0_0_0] log.PerfLogger: 2018-01-05T11:28:44,531 DEBUG [TezTR-518813_1_8_0_0_0] exec.MapOperator: Processing alias(es) src_sequencefile for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile/kv1.seq 2018-01-05T11:28:44,531 INFO [TezTR-518813_1_8_0_0_0] exec.MapOperator: MAP[0]: records read - 1 2018-01-05T11:28:44,532 INFO [TezTR-518813_1_8_0_0_0] exec.MapOperator: MAP[0]: records read - 10 2018-01-05T11:28:44,533 INFO [TezTR-518813_1_8_0_0_0] exec.MapOperator: MAP[0]: records read - 100 2018-01-05T11:28:44,538 DEBUG [TezTR-518813_1_8_0_0_0] log.PerfLogger: 2018-01-05T11:28:44,538 DEBUG [TezTR-518813_1_8_0_0_0] mr.ObjectCache: djaiswal_20180105112844_3409fb09-65bb-44b3-ba39-c8970e1323c4_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:28:44,538 DEBUG [TezTR-518813_1_8_0_0_0] exec.MapOperator: close called for operator MAP[0] 2018-01-05T11:28:44,538 INFO [TezTR-518813_1_8_0_0_0] exec.MapOperator: Closing operator MAP[0] 2018-01-05T11:28:44,538 INFO [TezTR-518813_1_8_0_0_0] exec.MapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:500, 2018-01-05T11:28:44,538 DEBUG [TezTR-518813_1_8_0_0_0] exec.MapOperator: Closing child = TS[0] 2018-01-05T11:28:44,538 DEBUG [TezTR-518813_1_8_0_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:28:44,538 DEBUG [TezTR-518813_1_8_0_0_0] exec.TableScanOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:44,538 INFO [TezTR-518813_1_8_0_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:28:44,538 DEBUG [TezTR-518813_1_8_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:44,538 DEBUG [TezTR-518813_1_8_0_0_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_239_3384404240884212623-1/-mr-10001 2018-01-05T11:28:44,538 INFO [TezTR-518813_1_8_0_0_0] exec.TableScanOperator: publishing : default.src_sequencefile/ : {numRows=500, rawDataSize=5312} 2018-01-05T11:28:44,560 DEBUG [TezTR-518813_1_8_0_0_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_239_3384404240884212623-1/-mr-10001/tmpstats-0_0 2018-01-05T11:28:44,560 DEBUG [TezTR-518813_1_8_0_0_0] fs.FSStatsPublisher: Writing stats in it : {default.src_sequencefile/={numRows=500, rawDataSize=5312}} 2018-01-05T11:28:44,560 DEBUG [TezTR-518813_1_8_0_0_0] exec.TableScanOperator: Closing child = SEL[1] 2018-01-05T11:28:44,560 DEBUG [TezTR-518813_1_8_0_0_0] exec.SelectOperator: close called for operator SEL[1] 2018-01-05T11:28:44,560 DEBUG [TezTR-518813_1_8_0_0_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:44,560 INFO [TezTR-518813_1_8_0_0_0] exec.SelectOperator: Closing operator SEL[1] 2018-01-05T11:28:44,560 DEBUG [TezTR-518813_1_8_0_0_0] exec.SelectOperator: Closing child = GBY[2] 2018-01-05T11:28:44,560 DEBUG [TezTR-518813_1_8_0_0_0] exec.GroupByOperator: close called for operator GBY[2] 2018-01-05T11:28:44,560 DEBUG [TezTR-518813_1_8_0_0_0] exec.GroupByOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:44,560 INFO [TezTR-518813_1_8_0_0_0] exec.GroupByOperator: Closing operator GBY[2] 2018-01-05T11:28:44,560 INFO [TezTR-518813_1_8_0_0_0] exec.GroupByOperator: Begin Hash Table flush: size = 1 2018-01-05T11:28:44,560 INFO [TezTR-518813_1_8_0_0_0] exec.ReduceSinkOperator: keys are [] num distributions: 0 2018-01-05T11:28:44,560 DEBUG [TezTR-518813_1_8_0_0_0] exec.GroupByOperator: Closing child = RS[3] 2018-01-05T11:28:44,560 DEBUG [TezTR-518813_1_8_0_0_0] exec.ReduceSinkOperator: close called for operator RS[3] 2018-01-05T11:28:44,560 DEBUG [TezTR-518813_1_8_0_0_0] exec.ReduceSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:44,560 INFO [TezTR-518813_1_8_0_0_0] exec.ReduceSinkOperator: Closing operator RS[3] 2018-01-05T11:28:44,560 INFO [TezTR-518813_1_8_0_0_0] exec.ReduceSinkOperator: RECORDS_OUT_INTERMEDIATE_Map_1:1, 2018-01-05T11:28:44,560 DEBUG [TezTR-518813_1_8_0_0_0] exec.ReduceSinkOperator: 3 Close done 2018-01-05T11:28:44,560 DEBUG [TezTR-518813_1_8_0_0_0] exec.GroupByOperator: 2 Close done 2018-01-05T11:28:44,560 DEBUG [TezTR-518813_1_8_0_0_0] exec.SelectOperator: 1 Close done 2018-01-05T11:28:44,560 DEBUG [TezTR-518813_1_8_0_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:28:44,560 DEBUG [TezTR-518813_1_8_0_0_0] exec.MapOperator: 0 Close done 2018-01-05T11:28:44,560 INFO [TezTR-518813_1_8_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_8_00_000000_0 2018-01-05T11:28:44,560 INFO [TezTR-518813_1_8_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: Waiting for all spills to complete : Pending : 0 2018-01-05T11:28:44,560 INFO [TezTR-518813_1_8_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: All spills complete 2018-01-05T11:28:44,572 DEBUG [TezTR-518813_1_8_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:44,572 INFO [TezTR-518813_1_8_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:44,572 INFO [TezTR-518813_1_8_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_8_00_000000_0, askedToStop=false 2018-01-05T11:28:44,572 INFO [TezTR-518813_1_8_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_8_00_000000_0, stopRequested=false 2018-01-05T11:28:44,573 INFO [TezTR-518813_1_8_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_8_00_000000_0: Counters: 14 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=500, INPUT_SPLIT_LENGTH_BYTES=10508, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=2374, OUTPUT_BYTES_WITH_OVERHEAD=2384, OUTPUT_BYTES_PHYSICAL=2412, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500, RECORDS_OUT_INTERMEDIATE_Map_1=1]] 2018-01-05T11:28:44,573 DEBUG [TezTR-518813_1_8_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_8_0_0_0 thread id: 586 scheme: file bytesRead: 14788 bytesWritten: 2515 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:44,573 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_8_00_000000_0, isTaskDone=true 2018-01-05T11:28:44,573 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000011, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_8_00_000000_0, eventCount=4 } 2018-01-05T11:28:44,575 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_8][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_8_00_000000_0, creationTime=1515180524478, allocationTime=1515180524480, startTime=1515180524483, finishTime=1515180524574, timeTaken=91, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:44,575 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:44,575 INFO [Task-Executor-3] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_8_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:44,575 INFO [Task-Executor-3] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_8_00_000000_0 2018-01-05T11:28:44,575 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_8_00_000000_0 2018-01-05T11:28:44,575 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_8_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:44,575 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_8_00_000000_0; false 2018-01-05T11:28:44,575 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_8_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:44,575 INFO [Task-Executor-3] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000011=78 2018-01-05T11:28:44,575 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:44,575 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_8_00_000000_0: true 2018-01-05T11:28:44,576 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:44,576 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_8_00_000000_0 2018-01-05T11:28:44,576 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_8][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_8_00_000000, startTime=1515180524483, finishTime=1515180524575, timeTaken=92, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_8_00_000000_0, diagnostics=, counters=Counters: 20, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=14788, FILE_BYTES_WRITTEN=2515, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=500, INPUT_SPLIT_LENGTH_BYTES=10508, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=2374, OUTPUT_BYTES_WITH_OVERHEAD=2384, OUTPUT_BYTES_PHYSICAL=2412, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500, RECORDS_OUT_INTERMEDIATE_Map_1=1 2018-01-05T11:28:44,576 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000011, QueryId=djaiswal_20180105112844_3409fb09-65bb-44b3-ba39-c8970e1323c4-8, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:28:44,576 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_8_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:44,576 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_8_00_000000_0 2018-01-05T11:28:44,576 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_8_00 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:44,576 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_8_00 [Map 1] 2018-01-05T11:28:44,576 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_8][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_8_00, initRequestedTime=1515180524442, initedTime=1515180524477, startRequestedTime=1515180524452, startedTime=1515180524477, finishTime=1515180524576, timeTaken=99, status=SUCCEEDED, diagnostics=, counters=Counters: 20, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=14788, FILE_BYTES_WRITTEN=2515, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=500, INPUT_SPLIT_LENGTH_BYTES=10508, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=2374, OUTPUT_BYTES_WITH_OVERHEAD=2384, OUTPUT_BYTES_PHYSICAL=2412, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500, RECORDS_OUT_INTERMEDIATE_Map_1=1, vertexStats=firstTaskStartTime=1515180524483, firstTasksToStart=[ task_1515180518813_0001_8_00_000000 ], lastTaskFinishTime=1515180524574, lastTasksToFinish=[ task_1515180518813_0001_8_00_000000 ], minTaskDuration=91, maxTaskDuration=91, avgTaskDuration=91.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_8_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_8_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:44,576 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_8_00 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:44,576 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_8][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000011, stoppedTime=1515180524576, exitStatus=0 2018-01-05T11:28:44,576 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000011 exited with diagnostics set to Container completed. 2018-01-05T11:28:44,576 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_8_01 [Reducer 2] attempt: attempt_1515180518813_0001_8_00_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:28:44,576 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_8_00 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2 2018-01-05T11:28:44,576 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null 2018-01-05T11:28:44,576 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000011 2018-01-05T11:28:44,577 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: All source tasks completed. Ramping up 1 remaining tasks for vertex: Reducer 2 2018-01-05T11:28:44,577 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Scheduling 1 tasks for vertex: Reducer 2 with totalTasks: 1. 1 source tasks completed out of 1. MinSourceTaskCompletedFraction: 1.0 min: 0.05 max: 0.75 2018-01-05T11:28:44,577 INFO [App Shared Pool - #0] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Map 1, destinationVertexName=Reducer 2. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.UnorderedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:28:44,577 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_8_01 [Reducer 2] 2018-01-05T11:28:44,577 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_8][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_8_01_000000, scheduledTime=1515180524577, launchTime=1515180524577 2018-01-05T11:28:44,577 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_8_01 [Reducer 2] 2018-01-05T11:28:44,577 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_8_01_000000_0, priority=14, capability=, hosts=null 2018-01-05T11:28:44,577 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:44,577 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:44,577 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:44,577 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_8_01_000000_0 on hosts=null 2018-01-05T11:28:44,577 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_8_01_000000_0 on hosts=null 2018-01-05T11:28:44,577 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_8_01_000000_0 2018-01-05T11:28:44,577 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:44,577 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #12, task=TaskInfo{task=attempt_1515180518813_0001_8_01_000000_0, priority=14, startTime=0, containerId=null, uniqueId=11, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000012 2018-01-05T11:28:44,577 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_8_01_000000_0; false 2018-01-05T11:28:44,577 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_8_01_000000_0, priority=14, startTime=311545086, containerId=container_111101111_0001_00_000012assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=11, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:44,578 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000012 for node: 10.22.8.182:49818 2018-01-05T11:28:44,578 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000012, attempt_1515180518813_0001_8_01_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:44,578 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000012 for node: 10.22.8.182:49818 2018-01-05T11:28:44,578 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000012 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:44,578 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000012, launchTime=1515180524578 2018-01-05T11:28:44,579 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=8}, sourceName=Map 1, state=S_SUCCEEDED 2018-01-05T11:28:44,579 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_8_01_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000012] on NM: [10.22.8.182:49818] 2018-01-05T11:28:44,579 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:44,579 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:44,579 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_8][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_8_01_000000_0, startTime=1515180524579, containerId=container_111101111_0001_00_000012, nodeId=10.22.8.182:49818 2018-01-05T11:28:44,579 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_8_01_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112844_3409fb09-65bb-44b3-ba39-c8970e1323c4, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000012, dagName=ANALYZE TABLE src_sequencefile C...key,value (Stage-0), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.UnorderedKVInput,1}}, Outputs={{out_Reducer 2,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=2, completedTaskCount=1, dagStartTime=1515180524441, firstAttemptStartTime=1515180524577, currentAttemptStartTime=1515180524578} 2018-01-05T11:28:44,579 INFO [IPC Server handler 3 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000012, QueryId=djaiswal_20180105112844_3409fb09-65bb-44b3-ba39-c8970e1323c4-8, DagId=8, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, SubmitTime=1515180524579 2018-01-05T11:28:44,580 DEBUG [IPC Server handler 3 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=8} with the ShuffleHandler 2018-01-05T11:28:44,580 DEBUG [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/8] 2018-01-05T11:28:44,580 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:44,580 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_8_01_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:44,580 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_8_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180524577, dagStartTime=1515180524441, withinDagPriority=14, vertexParallelism= 1, selfAndUpstreamParallelism= 2, selfAndUpstreamComplete= 1}] 2018-01-05T11:28:44,580 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_8_01_000000_0 : ACCEPTED 2018-01-05T11:28:44,580 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_8_01_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:44,580 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_8_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180524577, dagStartTime=1515180524441, withinDagPriority=14, vertexParallelism= 1, selfAndUpstreamParallelism= 2, selfAndUpstreamComplete= 1} 2018-01-05T11:28:44,580 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_8_01_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:44,580 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_8_01_000000_0 to pre-emption queue 2018-01-05T11:28:44,580 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_8_01_000000_0: true 2018-01-05T11:28:44,580 INFO [TaskCommunicator # 2] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_8_01_000000_0 2018-01-05T11:28:44,580 INFO [TaskCommunicator # 2] GuaranteedTasks: Received response for attempt_1515180518813_0001_8_01_000000_0, true 2018-01-05T11:28:44,580 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: taskOwner hashCode:606910175 2018-01-05T11:28:44,581 INFO [Task-Executor-3] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : ANALYZE TABLE src_sequencefile C...key,value (Stage-0), VertexName: Reducer 2, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_8_01_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:28:44,581 INFO [Task-Executor-3] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=954728448, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:44,581 INFO [Task-Executor-3] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_8_01_000000_0 vertexName: Reducer 2 2018-01-05T11:28:44,581 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000012, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_8_01_000000_0, eventCount=1 } 2018-01-05T11:28:44,581 DEBUG [TezTR-518813_1_8_1_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_8 queryId: djaiswal_20180105112844_3409fb09-65bb-44b3-ba39-c8970e1323c4 instanceType: TaskRunner2Callable 2018-01-05T11:28:44,581 INFO [TezTR-518813_1_8_1_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_8_01_000000_0 2018-01-05T11:28:44,581 DEBUG [TezTR-518813_1_8_1_0_0] log.PerfLogger: 2018-01-05T11:28:44,582 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:44,583 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:44,586 INFO [IPC Server handler 1 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_8_01_000000_0 1 events [0,1) total 1 vertex_1515180518813_0001_8_01 [Reducer 2] 2018-01-05T11:28:44,587 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:28:44,587 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_8_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:28:44,588 DEBUG [TezTR-518813_1_8_1_0_0] log.PerfLogger: 2018-01-05T11:28:44,588 INFO [TezTR-518813_1_8_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:44,588 INFO [TezTR-518813_1_8_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:44,588 INFO [TezTR-518813_1_8_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:44,588 INFO [I/O Setup 1 Initialize: {out_Reducer 2}] output.MROutput: out_Reducer 2: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:28:44,588 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:44,588 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:44,588 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:44,588 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:44,588 INFO [TezTR-518813_1_8_1_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:44,588 INFO [TezTR-518813_1_8_1_0_0] resources.MemoryDistributor: InitialRequests=[out_Reducer 2:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.UnorderedKVInput] 2018-01-05T11:28:44,588 INFO [TezTR-518813_1_8_1_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:44,588 INFO [TezTR-518813_1_8_1_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:44,588 INFO [TezTR-518813_1_8_1_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 131404920, TotalRequestedScaled: 6.570246E7, TotalJVMHeap: 954728448, TotalAvailable: 639668060, TotalRequested/TotalJVMHeap:0.14 2018-01-05T11:28:44,588 INFO [TezTR-518813_1_8_1_0_0] resources.MemoryDistributor: Allocations=[out_Reducer 2:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [Map 1:org.apache.tez.runtime.library.input.UnorderedKVInput:INPUT:131404920:131404920] 2018-01-05T11:28:44,588 INFO [TezTR-518813_1_8_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:44,588 INFO [TezTR-518813_1_8_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 0 2018-01-05T11:28:44,588 INFO [TezTR-518813_1_8_1_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:44,589 INFO [TezTR-518813_1_8_1_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_8_01_000000_0 2018-01-05T11:28:44,589 DEBUG [TezTR-518813_1_8_1_0_0] log.PerfLogger: 2018-01-05T11:28:44,589 DEBUG [TezTR-518813_1_8_1_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_8_01_000000_0_10034 2018-01-05T11:28:44,589 DEBUG [TezTR-518813_1_8_1_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:44,589 DEBUG [TezTR-518813_1_8_1_0_0] mr.ObjectCache: Creating djaiswal_20180105112844_3409fb09-65bb-44b3-ba39-c8970e1323c4_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:28:44,589 INFO [TezTR-518813_1_8_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_233_6189264957254526832-1/djaiswal/_tez_scratch_dir/2e8e678b-b35c-4cce-82d3-92ec6b9b3a99/reduce.xml 2018-01-05T11:28:44,589 DEBUG [TezTR-518813_1_8_1_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_233_6189264957254526832-1/djaiswal/_tez_scratch_dir/2e8e678b-b35c-4cce-82d3-92ec6b9b3a99/reduce.xml 2018-01-05T11:28:44,589 DEBUG [TezTR-518813_1_8_1_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_233_6189264957254526832-1/djaiswal/_tez_scratch_dir/2e8e678b-b35c-4cce-82d3-92ec6b9b3a99/reduce.xml 2018-01-05T11:28:44,589 DEBUG [TezTR-518813_1_8_1_0_0] log.PerfLogger: 2018-01-05T11:28:44,589 INFO [TezTR-518813_1_8_1_0_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:28:44,589 DEBUG [TezTR-518813_1_8_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:44,590 DEBUG [TezTR-518813_1_8_1_0_0] log.PerfLogger: 2018-01-05T11:28:44,590 INFO [TezTR-518813_1_8_1_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 2.01KB 2018-01-05T11:28:44,590 DEBUG [TezTR-518813_1_8_1_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:44,590 DEBUG [TezTR-518813_1_8_1_0_0] log.PerfLogger: 2018-01-05T11:28:44,590 DEBUG [TezTR-518813_1_8_1_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:44,590 DEBUG [TezTR-518813_1_8_1_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:44,591 DEBUG [TezTR-518813_1_8_1_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:44,591 INFO [TezTR-518813_1_8_1_0_0] impl.SimpleFetchedInputAllocator: Map_1: RequestedMemory=131404920, AssignedMemory=131404920, maxSingleShuffleLimit=32851230 2018-01-05T11:28:44,591 INFO [TezTR-518813_1_8_1_0_0] impl.ShuffleManager: Map_1: numInputs=1, compressionCodec=NoCompressionCodec, numFetchers=1, ifileBufferSize=4096, ifileReadAheadEnabled=true, ifileReadAheadLength=4194304, localDiskFetchEnabled=false, sharedFetchEnabled=false, keepAlive=false, keepAliveMaxConnections=20, connectionTimeout=20000, readTimeout=180000, bufferSize=8192, bufferSize=8192, maxTaskOutputAtOnce=20 2018-01-05T11:28:44,592 INFO [TezTR-518813_1_8_1_0_0] impl.ShuffleInputEventHandlerImpl: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:28:44,592 INFO [TezTR-518813_1_8_1_0_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:28:44,593 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=8&reduce=0&map=attempt_1515180518813_0001_8_00_000000_0_10033 mapId: [attempt_1515180518813_0001_8_00_000000_0_10033] reduceId: [0] jobId: [job_1515180518813_0001] dagId: [8] keepAlive: false 2018-01-05T11:28:44,593 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=8&reduce=0&map=attempt_1515180518813_0001_8_00_000000_0_10033; hash=...r74ynRJfGWENI 2018-01-05T11:28:44,593 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=8&reduce=0&map=attempt_1515180518813_0001_8_00_000000_0_10033;reply=+3yVzN5MzNXAs 2018-01-05T11:28:44,593 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=8, user='djaiswal', attemptId='attempt_1515180518813_0001_8_00_000000_0_10033'} via loader 2018-01-05T11:28:44,593 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=8, user='djaiswal', attemptId='attempt_1515180518813_0001_8_00_000000_0_10033'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:28:44,593 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_8_00_000000_0_10033 not found 2018-01-05T11:28:44,593 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_8_00_000000_0_10033,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/8/output/attempt_1515180518813_0001_8_00_000000_0_10033/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/8/output/attempt_1515180518813_0001_8_00_000000_0_10033/file.out.index 2018-01-05T11:28:44,593 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Content Length in shuffle : 2442 2018-01-05T11:28:44,594 INFO [Fetcher_B {Map_1} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=8&reduce=0&map=attempt_1515180518813_0001_8_00_000000_0_10033 sent hash and receievd reply 0 ms 2018-01-05T11:28:44,594 INFO [Fetcher_B {Map_1} #0] ShuffleManager.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_8_00_000000_0_10033} to MEMORY, csize=2388, dsize=2384, EndTime=1515180524594, TimeTaken=0, Rate=0.00 MB/s 2018-01-05T11:28:44,594 INFO [Fetcher_B {Map_1} #0] impl.ShuffleManager: All inputs fetched for input vertex : Map 1 2018-01-05T11:28:44,594 INFO [Fetcher_B {Map_1} #0] impl.ShuffleManager: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:28:44,594 INFO [TezTR-518813_1_8_1_0_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:28:44,594 INFO [ShuffleRunner {Map_1}] impl.ShuffleManager: Map_1: Shutting down FetchScheduler, Was Interrupted: false 2018-01-05T11:28:44,594 INFO [TezTR-518813_1_8_1_0_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:44,594 INFO [ShuffleRunner {Map_1}] impl.ShuffleManager: Map_1: Scheduler thread completed 2018-01-05T11:28:44,594 DEBUG [TezTR-518813_1_8_1_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:28:44,594 DEBUG [TezTR-518813_1_8_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:44,594 DEBUG [TezTR-518813_1_8_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:44,594 DEBUG [TezTR-518813_1_8_1_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:28:44,594 DEBUG [TezTR-518813_1_8_1_0_0] log.PerfLogger: 2018-01-05T11:28:44,594 INFO [TezTR-518813_1_8_1_0_0] exec.GroupByOperator: Initializing operator GBY[4] 2018-01-05T11:28:44,594 DEBUG [TezTR-518813_1_8_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:44,594 DEBUG [TezTR-518813_1_8_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:44,594 INFO [TezTR-518813_1_8_1_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:44,594 DEBUG [TezTR-518813_1_8_1_0_0] exec.GroupByOperator: Initialization Done 4 GBY 2018-01-05T11:28:44,594 DEBUG [TezTR-518813_1_8_1_0_0] exec.GroupByOperator: Operator 4 GBY initialized 2018-01-05T11:28:44,594 DEBUG [TezTR-518813_1_8_1_0_0] exec.GroupByOperator: Initializing children of 4 GBY 2018-01-05T11:28:44,594 DEBUG [TezTR-518813_1_8_1_0_0] exec.FileSinkOperator: Initializing child 6 FS 2018-01-05T11:28:44,594 INFO [TezTR-518813_1_8_1_0_0] exec.FileSinkOperator: Initializing operator FS[6] 2018-01-05T11:28:44,594 DEBUG [TezTR-518813_1_8_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:44,595 INFO [TezTR-518813_1_8_1_0_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@63da7705]:[_col0, _col1]:[struct, struct]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@1ceb6999 2018-01-05T11:28:44,595 DEBUG [TezTR-518813_1_8_1_0_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:28:44,595 DEBUG [TezTR-518813_1_8_1_0_0] exec.FileSinkOperator: Initialization Done 6 FS 2018-01-05T11:28:44,595 DEBUG [TezTR-518813_1_8_1_0_0] exec.FileSinkOperator: Operator 6 FS initialized 2018-01-05T11:28:44,595 DEBUG [TezTR-518813_1_8_1_0_0] exec.FileSinkOperator: Initialization Done 6 FS done is reset. 2018-01-05T11:28:44,595 DEBUG [TezTR-518813_1_8_1_0_0] exec.GroupByOperator: Initialization Done 4 GBY done is reset. 2018-01-05T11:28:44,595 INFO [TezTR-518813_1_8_1_0_0] tez.ReduceRecordProcessor: Id =4 Id =6 <\Children> Id = 4 null<\Parent> <\FS> <\Children> <\Parent> <\GBY> 2018-01-05T11:28:44,595 DEBUG [TezTR-518813_1_8_1_0_0] log.PerfLogger: 2018-01-05T11:28:44,595 INFO [TezTR-518813_1_8_1_0_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 2 2018-01-05T11:28:44,595 DEBUG [TezTR-518813_1_8_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:28:44,595 DEBUG [TezTR-518813_1_8_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:28:44,595 DEBUG [TezTR-518813_1_8_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:28:44,595 DEBUG [TezTR-518813_1_8_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:28:44,595 DEBUG [TezTR-518813_1_8_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:28:44,595 DEBUG [TezTR-518813_1_8_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:28:44,596 INFO [TezTR-518813_1_8_1_0_0] readers.UnorderedKVReader: Num Records read: 1 2018-01-05T11:28:44,596 DEBUG [TezTR-518813_1_8_1_0_0] log.PerfLogger: 2018-01-05T11:28:44,596 DEBUG [TezTR-518813_1_8_1_0_0] mr.ObjectCache: djaiswal_20180105112844_3409fb09-65bb-44b3-ba39-c8970e1323c4_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:28:44,596 DEBUG [TezTR-518813_1_8_1_0_0] exec.GroupByOperator: close called for operator GBY[4] 2018-01-05T11:28:44,596 INFO [TezTR-518813_1_8_1_0_0] exec.GroupByOperator: Closing operator GBY[4] 2018-01-05T11:28:44,596 INFO [TezTR-518813_1_8_1_0_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_239_3384404240884212623-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-44_239_3384404240884212623-1/_tmp.-ext-10002/000000_0 2018-01-05T11:28:44,596 INFO [TezTR-518813_1_8_1_0_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_239_3384404240884212623-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-44_239_3384404240884212623-1/_task_tmp.-ext-10002/_tmp.000000_0 2018-01-05T11:28:44,596 INFO [TezTR-518813_1_8_1_0_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_239_3384404240884212623-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-44_239_3384404240884212623-1/_tmp.-ext-10002/000000_0 2018-01-05T11:28:44,631 INFO [TezTR-518813_1_8_1_0_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:28:44,631 DEBUG [TezTR-518813_1_8_1_0_0] exec.GroupByOperator: Closing child = FS[6] 2018-01-05T11:28:44,631 DEBUG [TezTR-518813_1_8_1_0_0] exec.FileSinkOperator: close called for operator FS[6] 2018-01-05T11:28:44,631 DEBUG [TezTR-518813_1_8_1_0_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:44,631 INFO [TezTR-518813_1_8_1_0_0] exec.FileSinkOperator: Closing operator FS[6] 2018-01-05T11:28:44,631 INFO [TezTR-518813_1_8_1_0_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:28:44,632 INFO [TezTR-518813_1_8_1_0_0] exec.FileSinkOperator: RECORDS_OUT_0:1, 2018-01-05T11:28:44,632 DEBUG [TezTR-518813_1_8_1_0_0] exec.FileSinkOperator: 6 Close done 2018-01-05T11:28:44,632 DEBUG [TezTR-518813_1_8_1_0_0] exec.GroupByOperator: 4 Close done 2018-01-05T11:28:44,632 INFO [TezTR-518813_1_8_1_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_8_01_000000_0 2018-01-05T11:28:44,632 INFO [TezTR-518813_1_8_1_0_0] impl.ShuffleInputEventHandlerImpl: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:28:44,632 INFO [TezTR-518813_1_8_1_0_0] impl.ShuffleManager: Shutting down pending fetchers on sourceMap_1: 0 2018-01-05T11:28:44,632 INFO [TezTR-518813_1_8_1_0_0] output.MROutput: out_Reducer 2 closed 2018-01-05T11:28:44,632 DEBUG [TezTR-518813_1_8_1_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:44,632 INFO [TezTR-518813_1_8_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:44,633 INFO [TezTR-518813_1_8_1_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_8_01_000000_0, askedToStop=false 2018-01-05T11:28:44,633 INFO [TezTR-518813_1_8_1_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_8_01_000000_0, stopRequested=false 2018-01-05T11:28:44,633 INFO [TezTR-518813_1_8_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_8_01_000000_0: Counters: 14 [[org.apache.tez.common.counters.TaskCounter NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=2388, SHUFFLE_BYTES_DECOMPRESSED=2384, SHUFFLE_BYTES_TO_MEM=2388, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=3, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1][HIVE CREATED_FILES=1, RECORDS_OUT_0=1]] 2018-01-05T11:28:44,633 DEBUG [TezTR-518813_1_8_1_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_8_1_0_0 thread id: 598 scheme: file bytesRead: 0 bytesWritten: 3285 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:44,633 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_8_01_000000_0, isTaskDone=true 2018-01-05T11:28:44,634 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000012, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_8_01_000000_0, eventCount=2 } 2018-01-05T11:28:44,635 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_8][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_8_01_000000_0, creationTime=1515180524577, allocationTime=1515180524578, startTime=1515180524579, finishTime=1515180524635, timeTaken=56, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:44,635 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_8_01_000000_0 2018-01-05T11:28:44,635 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_8_01_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:44,635 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:28:44,635 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_8_01_000000_0; false 2018-01-05T11:28:44,635 INFO [Task-Executor-3] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_8_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:44,635 INFO [Task-Executor-3] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_8_01_000000_0 2018-01-05T11:28:44,635 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_8_01_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:44,635 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:44,635 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:44,636 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_8][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_8_01_000000, startTime=1515180524579, finishTime=1515180524635, timeTaken=56, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_8_01_000000_0, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=3285, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=2388, SHUFFLE_BYTES_DECOMPRESSED=2384, SHUFFLE_BYTES_TO_MEM=2388, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=3, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1 2018-01-05T11:28:44,636 INFO [Task-Executor-3] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000012=55 2018-01-05T11:28:44,636 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_8_01_000000_0: true 2018-01-05T11:28:44,636 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_8_01 [Reducer 2], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:44,636 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_8_01 [Reducer 2] 2018-01-05T11:28:44,636 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_8][Event:VERTEX_FINISHED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_8_01, initRequestedTime=1515180524452, initedTime=1515180524463, startRequestedTime=1515180524477, startedTime=1515180524477, finishTime=1515180524636, timeTaken=159, status=SUCCEEDED, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=3285, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=2388, SHUFFLE_BYTES_DECOMPRESSED=2384, SHUFFLE_BYTES_TO_MEM=2388, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=3, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1, vertexStats=firstTaskStartTime=1515180524579, firstTasksToStart=[ task_1515180518813_0001_8_01_000000 ], lastTaskFinishTime=1515180524635, lastTasksToFinish=[ task_1515180518813_0001_8_01_000000 ], minTaskDuration=56, maxTaskDuration=56, avgTaskDuration=56.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_8_01_000000 ], longestDurationTasks=[ task_1515180518813_0001_8_01_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:44,636 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_8_01 [Reducer 2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:44,637 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_8][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000012, stoppedTime=1515180524637, exitStatus=0 2018-01-05T11:28:44,637 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000012 exited with diagnostics set to Container completed. 2018-01-05T11:28:44,637 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_8_01 [Reducer 2] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2 2018-01-05T11:28:44,637 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null 2018-01-05T11:28:44,637 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_8_00 [Map 1] 2018-01-05T11:28:44,637 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_8_01 [Reducer 2] 2018-01-05T11:28:44,637 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_8, queueSize=0 2018-01-05T11:28:44,638 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_8][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_8, startTime=1515180524441, finishTime=1515180524637, timeTaken=196, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, OTHER_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=453, AM_GC_TIME_MILLIS=0, File System Counters, FILE_BYTES_READ=14788, FILE_BYTES_WRITTEN=5800, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=501, INPUT_SPLIT_LENGTH_BYTES=10508, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=2374, OUTPUT_BYTES_WITH_OVERHEAD=2384, OUTPUT_BYTES_PHYSICAL=2412, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=2388, SHUFFLE_BYTES_DECOMPRESSED=2384, SHUFFLE_BYTES_TO_MEM=2388, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=3, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1, HIVE, CREATED_FILES=1, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500, RECORDS_OUT_0=1, RECORDS_OUT_INTERMEDIATE_Map_1=1 2018-01-05T11:28:44,638 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_8 finished with state: SUCCEEDED 2018-01-05T11:28:44,638 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_8 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:28:44,638 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000012 2018-01-05T11:28:44,638 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_8, dagState=SUCCEEDED 2018-01-05T11:28:44,638 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_8_post 2018-01-05T11:28:44,638 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:44,638 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:28:44,638 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:28:44,638 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 8 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=2, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=2,NumRequestsWithoutLocation=2, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=2} 2018-01-05T11:28:44,638 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:28:44,638 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=ANALYZE TABLE src_sequencefile C...key,value (Stage-0), with id=dag_1515180518813_0001_8 2018-01-05T11:28:44,638 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 8, to 10.22.8.182:49818 2018-01-05T11:28:44,638 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=2, Remaining=0 2018-01-05T11:28:44,638 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=ANALYZE TABLE src_sequencefile C...key,value (Stage-0), with id=dag_1515180518813_0001_8 2018-01-05T11:28:44,639 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=8} 2018-01-05T11:28:44,639 INFO [IPC Server handler 1 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=8}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:28:44,639 INFO [IPC Server handler 1 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/8 after 300 seconds 2018-01-05T11:28:44,639 ERROR [IPC Server handler 1 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112844_3409fb09-65bb-44b3-ba39-c8970e1323c4, dag_1515180518813_0001_8 2018-01-05T11:28:44,639 INFO [IPC Server handler 1 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112844_3409fb09-65bb-44b3-ba39-c8970e1323c4 2018-01-05T11:28:44,639 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=8} = 1 2018-01-05T11:28:44,639 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Issuing killFragment for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=8} attempt_1515180518813_0001_8_01_000000_0 2018-01-05T11:28:44,639 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Removing attempt_1515180518813_0001_8_01_000000_0 from preemptionQueue 2018-01-05T11:28:44,639 INFO [IPC Server handler 1 on 49818] impl.TaskRunnerCallable: Ignoring kill request for task attempt_1515180518813_0001_8_01_000000_0 since it's already complete 2018-01-05T11:28:44,639 DEBUG [IPC Server handler 1 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=8} 2018-01-05T11:28:44,639 DEBUG [IPC Server handler 1 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:28:44,639 DEBUG [IPC Server handler 1 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=8} 2018-01-05T11:28:44,640 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,640 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,640 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,640 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,640 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 Reducer 2: 1/1 2018-01-05T11:28:44,640 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,642 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,642 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: TaskId for 000000_0 = 000000 2018-01-05T11:28:44,642 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,642 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,642 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,643 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-2:STATS] in serial mode 2018-01-05T11:28:44,643 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@7ab17cf, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:44,643 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:44,643 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,643 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:44,643 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:44,643 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:44,643 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:44,643 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,643 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:44,643 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:44,643 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_239_3384404240884212623-1/-mr-10001 2018-01-05T11:28:44,644 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_239_3384404240884212623-1/-mr-10001 2018-01-05T11:28:44,644 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.src_sequencefile/ numRows 2018-01-05T11:28:44,644 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.src_sequencefile/ numRows 500 2018-01-05T11:28:44,644 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.src_sequencefile/ rawDataSize 2018-01-05T11:28:44,644 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.src_sequencefile/ rawDataSize 5312 2018-01-05T11:28:44,645 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,645 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:44,645 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:44,645 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:44,646 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:44,646 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,646 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,646 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,647 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,648 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,648 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:44,648 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:44,648 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,648 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:44,648 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,649 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=src_sequencefile newtbl=src_sequencefile 2018-01-05T11:28:44,649 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=src_sequencefile newtbl=src_sequencefile 2018-01-05T11:28:44,653 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,659 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,660 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,660 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.src_sequencefile stats: [numFiles=1, numRows=500, totalSize=10508, rawDataSize=5312] 2018-01-05T11:28:44,660 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: About to delete stats tmp dir :file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_239_3384404240884212623-1/-mr-10001 2018-01-05T11:28:44,676 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000011, QueryId=djaiswal_20180105112844_3409fb09-65bb-44b3-ba39-c8970e1323c4-8, DagId=8, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-3, Succeeded=true, StartTime=1515180524496, EndTime=1515180524676 2018-01-05T11:28:44,676 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Time taken to get FileStatuses: 15 2018-01-05T11:28:44,676 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:28:44,676 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_8_01_000000_0 2018-01-05T11:28:44,676 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_8_01_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:44,676 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_8_01_000000_0 2018-01-05T11:28:44,676 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 15 2018-01-05T11:28:44,676 INFO [ExecutionCompletionThread #0] impl.QueryTracker: Ignoring fragmentComplete message for unknown query: QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=8} 2018-01-05T11:28:44,677 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FetchOperator: Creating fetchTask with deserializer typeinfo: struct<_col0:struct,_col1:struct> 2018-01-05T11:28:44,677 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FetchOperator: deserializer properties: table properties: {columns=_col0,_col1, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct:struct, escape.delim=\} partition properties: {columns=_col0,_col1, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct:struct, escape.delim=\} 2018-01-05T11:28:44,678 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,678 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: write_column_statistics: db=default table=src_sequencefile 2018-01-05T11:28:44,678 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=write_column_statistics: db=default table=src_sequencefile 2018-01-05T11:28:44,683 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=src_sequencefile tableName=default colName=key 2018-01-05T11:28:44,686 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,686 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=src_sequencefile tableName=default colName=value 2018-01-05T11:28:44,689 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,689 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,689 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,690 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,690 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: ANALYZE TABLE src_sequencefile COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:44,690 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:28:44,690 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@src_sequencefile 2018-01-05T11:28:44,690 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@src_sequencefile 2018-01-05T11:28:44,690 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_239_3384404240884212623-1/-mr-10000 2018-01-05T11:28:44,690 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,690 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,690 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:44,690 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=12, setPartitionColumnStatistics_(SetPartitionsStatsRequest, )=11} 2018-01-05T11:28:44,690 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112844_3409fb09-65bb-44b3-ba39-c8970e1323c4); Time taken: 0.391 seconds 2018-01-05T11:28:44,690 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:44,690 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,690 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:44,690 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,690 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query ANALYZE TABLE src_sequencefile COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:44,690 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_233_6189264957254526832-1 2018-01-05T11:28:44,691 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.457 seconds 2018-01-05T11:28:44,691 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:44,691 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:44,691 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:44,691 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:44,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:44,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: -- -- Table src_thrift -- DROP TABLE IF EXISTS src_thrift 2018-01-05T11:28:44,691 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112844_c830d0ee-4dae-4791-8b10-b740bb0bc89c): -- -- Table src_thrift -- DROP TABLE IF EXISTS src_thrift 2018-01-05T11:28:44,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: -- -- Table src_thrift -- DROP TABLE IF EXISTS src_thrift 2018-01-05T11:28:44,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:44,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,692 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:44,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,692 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_thrift 2018-01-05T11:28:44,692 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_thrift 2018-01-05T11:28:44,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,692 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:44,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,692 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:44,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,692 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:44,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=1, flushCache_()=0} 2018-01-05T11:28:44,693 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112844_c830d0ee-4dae-4791-8b10-b740bb0bc89c); Time taken: 0.001 seconds 2018-01-05T11:28:44,693 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:44,693 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,693 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112844_c830d0ee-4dae-4791-8b10-b740bb0bc89c): -- -- Table src_thrift -- DROP TABLE IF EXISTS src_thrift 2018-01-05T11:28:44,694 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,694 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: -- -- Table src_thrift -- DROP TABLE IF EXISTS src_thrift 2018-01-05T11:28:44,694 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:44,694 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,694 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,694 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,694 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,694 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:44,694 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,694 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_thrift 2018-01-05T11:28:44,694 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_thrift 2018-01-05T11:28:44,694 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,694 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,695 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table src_thrift not found: default.src_thrift table not found 2018-01-05T11:28:44,695 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,695 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_thrift 2018-01-05T11:28:44,695 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_thrift 2018-01-05T11:28:44,695 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,695 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,695 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,695 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,695 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: -- -- Table src_thrift -- DROP TABLE IF EXISTS src_thrift 2018-01-05T11:28:44,695 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:44,695 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,695 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,695 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:44,695 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=0} 2018-01-05T11:28:44,695 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112844_c830d0ee-4dae-4791-8b10-b740bb0bc89c); Time taken: 0.002 seconds 2018-01-05T11:28:44,695 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:44,695 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,695 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:44,695 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,695 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query -- -- Table src_thrift -- DROP TABLE IF EXISTS src_thrift 2018-01-05T11:28:44,695 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.004 seconds 2018-01-05T11:28:44,695 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:44,695 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:44,695 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:44,695 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:44,695 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,696 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:44,696 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,696 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,696 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,696 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: CREATE TABLE src_thrift ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.thrift.ThriftDeserializer' WITH SERDEPROPERTIES ( 'serialization.class' = 'org.apache.hadoop.hive.serde2.thrift.test.Complex', 'serialization.format' = 'org.apache.thrift.protocol.TBinaryProtocol') STORED AS SEQUENCEFILE 2018-01-05T11:28:44,696 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112844_9edb10b1-ec61-45f0-8490-bb0ee80de7cd): CREATE TABLE src_thrift ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.thrift.ThriftDeserializer' WITH SERDEPROPERTIES ( 'serialization.class' = 'org.apache.hadoop.hive.serde2.thrift.test.Complex', 'serialization.format' = 'org.apache.thrift.protocol.TBinaryProtocol') STORED AS SEQUENCEFILE 2018-01-05T11:28:44,696 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,696 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: CREATE TABLE src_thrift ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.thrift.ThriftDeserializer' WITH SERDEPROPERTIES ( 'serialization.class' = 'org.apache.hadoop.hive.serde2.thrift.test.Complex', 'serialization.format' = 'org.apache.thrift.protocol.TBinaryProtocol') STORED AS SEQUENCEFILE 2018-01-05T11:28:44,697 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:44,697 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,697 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,697 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,697 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,697 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:44,697 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:28:44,697 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Creating table default.src_thrift position=13 2018-01-05T11:28:44,698 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,698 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_database: default 2018-01-05T11:28:44,698 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_database: default 2018-01-05T11:28:44,698 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:44,698 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,698 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,698 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:44,698 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: validation start 2018-01-05T11:28:44,698 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:28:44,698 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Not a partition. 2018-01-05T11:28:44,698 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,698 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:44,698 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,698 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:44,698 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getDatabase_(String, )=1, flushCache_()=0} 2018-01-05T11:28:44,698 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112844_9edb10b1-ec61-45f0-8490-bb0ee80de7cd); Time taken: 0.002 seconds 2018-01-05T11:28:44,698 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:44,698 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,699 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112844_9edb10b1-ec61-45f0-8490-bb0ee80de7cd): CREATE TABLE src_thrift ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.thrift.ThriftDeserializer' WITH SERDEPROPERTIES ( 'serialization.class' = 'org.apache.hadoop.hive.serde2.thrift.test.Complex', 'serialization.format' = 'org.apache.thrift.protocol.TBinaryProtocol') STORED AS SEQUENCEFILE 2018-01-05T11:28:44,699 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,699 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: CREATE TABLE src_thrift ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.thrift.ThriftDeserializer' WITH SERDEPROPERTIES ( 'serialization.class' = 'org.apache.hadoop.hive.serde2.thrift.test.Complex', 'serialization.format' = 'org.apache.thrift.protocol.TBinaryProtocol') STORED AS SEQUENCEFILE 2018-01-05T11:28:44,699 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: CREATETABLE 2018-01-05T11:28:44,700 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: database:default 2018-01-05T11:28:44,700 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@src_thrift 2018-01-05T11:28:44,700 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,700 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,700 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,700 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,700 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:44,703 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Found class for org.apache.hadoop.hive.serde2.thrift.ThriftDeserializer 2018-01-05T11:28:44,716 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: creating table default.src_thrift on file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_thrift 2018-01-05T11:28:44,717 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,717 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: create_table: Table(tableName:src_thrift, dbName:default, owner:hive_test_user, createTime:1515180524, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:aint, type:int, comment:from deserializer), FieldSchema(name:astring, type:string, comment:from deserializer), FieldSchema(name:lint, type:array, comment:from deserializer), FieldSchema(name:lstring, type:array, comment:from deserializer), FieldSchema(name:lintstring, type:array>, comment:from deserializer), FieldSchema(name:mstringstring, type:map, comment:from deserializer), FieldSchema(name:attributes, type:map,map>>>>, comment:from deserializer), FieldSchema(name:unionfield1, type:uniontype,map>, comment:from deserializer), FieldSchema(name:unionfield2, type:uniontype,map>, comment:from deserializer), FieldSchema(name:unionfield3, type:uniontype,map>, comment:from deserializer)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_thrift, inputFormat:org.apache.hadoop.mapred.SequenceFileInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.thrift.ThriftDeserializer, parameters:{serialization.class=org.apache.hadoop.hive.serde2.thrift.test.Complex, serialization.format=org.apache.thrift.protocol.TBinaryProtocol}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"aint":"true","astring":"true","attributes":"true","lint":"true","lintstring":"true","lstring":"true","mstringstring":"true","unionfield1":"true","unionfield2":"true","unionfield3":"true"}}, numFiles=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:44,717 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=create_table: Table(tableName:src_thrift, dbName:default, owner:hive_test_user, createTime:1515180524, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:aint, type:int, comment:from deserializer), FieldSchema(name:astring, type:string, comment:from deserializer), FieldSchema(name:lint, type:array, comment:from deserializer), FieldSchema(name:lstring, type:array, comment:from deserializer), FieldSchema(name:lintstring, type:array>, comment:from deserializer), FieldSchema(name:mstringstring, type:map, comment:from deserializer), FieldSchema(name:attributes, type:map,map>>>>, comment:from deserializer), FieldSchema(name:unionfield1, type:uniontype,map>, comment:from deserializer), FieldSchema(name:unionfield2, type:uniontype,map>, comment:from deserializer), FieldSchema(name:unionfield3, type:uniontype,map>, comment:from deserializer)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_thrift, inputFormat:org.apache.hadoop.mapred.SequenceFileInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.thrift.ThriftDeserializer, parameters:{serialization.class=org.apache.hadoop.hive.serde2.thrift.test.Complex, serialization.format=org.apache.thrift.protocol.TBinaryProtocol}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"aint":"true","astring":"true","attributes":"true","lint":"true","lintstring":"true","lstring":"true","mstringstring":"true","unionfield1":"true","unionfield2":"true","unionfield3":"true"}}, numFiles=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:44,718 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hive-site.xml 2018-01-05T11:28:44,718 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hivemetastore-site.xml 2018-01-05T11:28:44,718 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Unable to find config file metastore-site.xml 2018-01-05T11:28:44,718 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file null 2018-01-05T11:28:44,727 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:44,727 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: Location: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_thrift specified for non-external table:src_thrift 2018-01-05T11:28:44,728 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_thrift 2018-01-05T11:28:44,749 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,750 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,750 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@src_thrift) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@src_thrift) Type=TABLE WriteType=DDL_NO_LOCK is present 2018-01-05T11:28:44,750 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,750 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,750 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: CREATE TABLE src_thrift ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.thrift.ThriftDeserializer' WITH SERDEPROPERTIES ( 'serialization.class' = 'org.apache.hadoop.hive.serde2.thrift.test.Complex', 'serialization.format' = 'org.apache.thrift.protocol.TBinaryProtocol') STORED AS SEQUENCEFILE 2018-01-05T11:28:44,750 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: CREATETABLE 2018-01-05T11:28:44,750 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: database:default 2018-01-05T11:28:44,750 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@src_thrift 2018-01-05T11:28:44,750 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,750 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,750 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:44,750 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {createTable_(Table, )=33, isCompatibleWith_(Configuration, )=0} 2018-01-05T11:28:44,750 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112844_9edb10b1-ec61-45f0-8490-bb0ee80de7cd); Time taken: 0.052 seconds 2018-01-05T11:28:44,750 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:44,750 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,750 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:44,750 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,750 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query CREATE TABLE src_thrift ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.thrift.ThriftDeserializer' WITH SERDEPROPERTIES ( 'serialization.class' = 'org.apache.hadoop.hive.serde2.thrift.test.Complex', 'serialization.format' = 'org.apache.thrift.protocol.TBinaryProtocol') STORED AS SEQUENCEFILE 2018-01-05T11:28:44,750 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.055 seconds 2018-01-05T11:28:44,750 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:44,750 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:44,750 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:44,750 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:44,751 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,751 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:44,751 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,751 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,751 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,751 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/complex.seq" INTO TABLE src_thrift 2018-01-05T11:28:44,751 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112844_06d0a815-d53d-4453-b60a-ce63c1559647): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/complex.seq" INTO TABLE src_thrift 2018-01-05T11:28:44,751 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,751 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/complex.seq" INTO TABLE src_thrift 2018-01-05T11:28:44,751 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:44,751 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,751 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,751 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,751 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,751 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:44,751 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.LoadSemanticAnalyzer: file@null@/Users/djaiswal/hive/hive/data/files/complex.seq 2018-01-05T11:28:44,752 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,752 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_thrift 2018-01-05T11:28:44,752 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_thrift 2018-01-05T11:28:44,757 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,758 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,760 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:44,760 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,760 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:44,760 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,760 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:44,760 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=6, flushCache_()=0} 2018-01-05T11:28:44,760 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112844_06d0a815-d53d-4453-b60a-ce63c1559647); Time taken: 0.009 seconds 2018-01-05T11:28:44,760 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:44,760 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,760 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112844_06d0a815-d53d-4453-b60a-ce63c1559647): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/complex.seq" INTO TABLE src_thrift 2018-01-05T11:28:44,761 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,761 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/complex.seq" INTO TABLE src_thrift 2018-01-05T11:28:44,761 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: LOAD 2018-01-05T11:28:44,761 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/complex.seq 2018-01-05T11:28:44,761 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@src_thrift 2018-01-05T11:28:44,761 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,761 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,761 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,761 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,761 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MOVE] in serial mode 2018-01-05T11:28:44,761 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@14237e5, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:44,761 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:44,761 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,761 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:44,761 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:44,761 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:44,761 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:44,761 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,761 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:44,761 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Loading data to table default.src_thrift from file:/Users/djaiswal/hive/hive/data/files/complex.seq 2018-01-05T11:28:44,762 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,762 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:44,762 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:44,762 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:44,762 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:44,762 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,764 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:44,764 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:44,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,764 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:44,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,764 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_thrift 2018-01-05T11:28:44,764 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_thrift 2018-01-05T11:28:44,770 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,770 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,771 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,771 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_thrift 2018-01-05T11:28:44,771 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_thrift 2018-01-05T11:28:44,775 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,776 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,776 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] FileOperations: moving file:/Users/djaiswal/hive/hive/data/files/complex.seq to file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_thrift (replace = KEEP_EXISTING) 2018-01-05T11:28:44,776 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:44,776 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:44,779 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000012, QueryId=djaiswal_20180105112844_3409fb09-65bb-44b3-ba39-c8970e1323c4-8, DagId=8, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-3, Succeeded=true, StartTime=1515180524580, EndTime=1515180524779 2018-01-05T11:28:44,800 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Moved src: file:/Users/djaiswal/hive/hive/data/files/complex.seq, to dest: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_thrift/complex.seq 2018-01-05T11:28:44,801 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,801 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=src_thrift newtbl=src_thrift 2018-01-05T11:28:44,801 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=src_thrift newtbl=src_thrift 2018-01-05T11:28:44,807 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,814 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,814 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,814 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@src_thrift) Type=TABLE WriteType=INSERT because WriteEntity(default@src_thrift) Type=TABLE WriteType=INSERT is present 2018-01-05T11:28:44,815 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-1:STATS] in serial mode 2018-01-05T11:28:44,815 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@4c2812aa, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:44,815 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:44,815 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,815 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:44,815 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:44,815 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:44,815 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:44,815 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,815 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:44,815 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:44,815 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,816 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:44,816 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:44,816 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:44,816 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:44,816 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,816 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,816 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,817 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,818 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,818 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:44,818 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:44,818 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,818 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:44,818 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,818 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_thrift 2018-01-05T11:28:44,818 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_thrift 2018-01-05T11:28:44,823 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,824 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,825 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,825 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=src_thrift newtbl=src_thrift 2018-01-05T11:28:44,825 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=src_thrift newtbl=src_thrift 2018-01-05T11:28:44,830 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,838 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,839 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,839 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.src_thrift stats: [numFiles=1, numRows=0, totalSize=3070, rawDataSize=0] 2018-01-05T11:28:44,839 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,839 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,839 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/complex.seq" INTO TABLE src_thrift 2018-01-05T11:28:44,839 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: LOAD 2018-01-05T11:28:44,839 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/complex.seq 2018-01-05T11:28:44,839 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@src_thrift 2018-01-05T11:28:44,839 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,839 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,839 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:44,839 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=14, getTable_(String, String, )=6} 2018-01-05T11:28:44,839 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112844_06d0a815-d53d-4453-b60a-ce63c1559647); Time taken: 0.079 seconds 2018-01-05T11:28:44,839 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:44,839 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,839 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:44,839 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,839 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/complex.seq" INTO TABLE src_thrift 2018-01-05T11:28:44,839 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.088 seconds 2018-01-05T11:28:44,839 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:44,839 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:44,839 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:44,839 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:44,840 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,840 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:44,840 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,840 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,840 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,840 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: ANALYZE TABLE src_thrift COMPUTE STATISTICS 2018-01-05T11:28:44,840 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112844_c0cd953c-25ae-48f6-93a3-d986f3d75bc3): ANALYZE TABLE src_thrift COMPUTE STATISTICS 2018-01-05T11:28:44,840 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,840 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: ANALYZE TABLE src_thrift COMPUTE STATISTICS 2018-01-05T11:28:44,840 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:44,840 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,840 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,840 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,840 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,840 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:44,840 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Invoking analyze on original query 2018-01-05T11:28:44,840 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Starting Semantic Analysis 2018-01-05T11:28:44,840 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed phase 1 of Semantic Analysis 2018-01-05T11:28:44,840 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for source tables 2018-01-05T11:28:44,840 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,840 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_thrift 2018-01-05T11:28:44,840 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_thrift 2018-01-05T11:28:44,847 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,847 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,848 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,848 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_thrift 2018-01-05T11:28:44,848 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_thrift 2018-01-05T11:28:44,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,853 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for subqueries 2018-01-05T11:28:44,853 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for destination tables 2018-01-05T11:28:44,853 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed getting MetaData in Semantic Analysis 2018-01-05T11:28:44,867 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_840_5848572550128803257-1/-mr-10000 2018-01-05T11:28:44,868 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Table Plan for src_thrift TS[0] 2018-01-05T11:28:44,868 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Body Plan for Query Block null 2018-01-05T11:28:44,868 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Plan for Query Block null 2018-01-05T11:28:44,868 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Before logical optimization TS[0] 2018-01-05T11:28:44,868 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,868 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,868 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] lineage.Generator: Time taken for lineage transform=1 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(aint: int|{src_thrift}aint,astring: string|{src_thrift}astring,lint: array|{src_thrift}lint,lstring: array|{src_thrift}lstring,lintstring: array>|{src_thrift}lintstring,mstringstring: map|{src_thrift}mstringstring,attributes: map,map>>>>|{src_thrift}attributes,unionfield1: uniontype,map>|{src_thrift}unionfield1,unionfield2: uniontype,map>|{src_thrift}unionfield2,unionfield3: uniontype,map>|{src_thrift}unionfield3,BLOCK__OFFSET__INSIDE__FILE: bigint|{src_thrift}block__offset__inside__file,INPUT__FILE__NAME: string|{src_thrift}input__file__name,ROW__ID: struct|{src_thrift}row__id,RAW__DATA__SIZE: bigint|{src_thrift}raw__data__size) 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,869 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.PredicatePushDown: After PPD: TS[0] 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(aint: int|{src_thrift}aint,astring: string|{src_thrift}astring,lint: array|{src_thrift}lint,lstring: array|{src_thrift}lstring,lintstring: array>|{src_thrift}lintstring,mstringstring: map|{src_thrift}mstringstring,attributes: map,map>>>>|{src_thrift}attributes,unionfield1: uniontype,map>|{src_thrift}unionfield1,unionfield2: uniontype,map>|{src_thrift}unionfield2,unionfield3: uniontype,map>|{src_thrift}unionfield3,BLOCK__OFFSET__INSIDE__FILE: bigint|{src_thrift}block__offset__inside__file,INPUT__FILE__NAME: string|{src_thrift}input__file__name,ROW__ID: struct|{src_thrift}row__id,RAW__DATA__SIZE: bigint|{src_thrift}raw__data__size) 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(aint: int|{src_thrift}aint,astring: string|{src_thrift}astring,lint: array|{src_thrift}lint,lstring: array|{src_thrift}lstring,lintstring: array>|{src_thrift}lintstring,mstringstring: map|{src_thrift}mstringstring,attributes: map,map>>>>|{src_thrift}attributes,unionfield1: uniontype,map>|{src_thrift}unionfield1,unionfield2: uniontype,map>|{src_thrift}unionfield2,unionfield3: uniontype,map>|{src_thrift}unionfield3,BLOCK__OFFSET__INSIDE__FILE: bigint|{src_thrift}block__offset__inside__file,INPUT__FILE__NAME: string|{src_thrift}input__file__name,ROW__ID: struct|{src_thrift}row__id,RAW__DATA__SIZE: bigint|{src_thrift}raw__data__size) 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: After logical optimization TS[0] 2018-01-05T11:28:44,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,870 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=src_thrift 2018-01-05T11:28:44,870 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=src_thrift 2018-01-05T11:28:44,875 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,879 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:44,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:44,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting stats (Num rows: 1 Data size: 30700 Basic stats: COMPLETE Column stats: COMPLETE) on TS[0] 2018-01-05T11:28:44,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (src_thrift): numRows: 1 dataSize: 30700 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:28:44,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on TS[0] 2018-01-05T11:28:44,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:44,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:28:44,880 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Cycle free: true 2018-01-05T11:28:44,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(aint: int|{src_thrift}aint,astring: string|{src_thrift}astring,lint: array|{src_thrift}lint,lstring: array|{src_thrift}lstring,lintstring: array>|{src_thrift}lintstring,mstringstring: map|{src_thrift}mstringstring,attributes: map,map>>>>|{src_thrift}attributes,unionfield1: uniontype,map>|{src_thrift}unionfield1,unionfield2: uniontype,map>|{src_thrift}unionfield2,unionfield3: uniontype,map>|{src_thrift}unionfield3,BLOCK__OFFSET__INSIDE__FILE: bigint|{src_thrift}block__offset__inside__file,INPUT__FILE__NAME: string|{src_thrift}input__file__name,ROW__ID: struct|{src_thrift}row__id,RAW__DATA__SIZE: bigint|{src_thrift}raw__data__size) 2018-01-05T11:28:44,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:28:44,881 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_thrift of table src_thrift 2018-01-05T11:28:44,881 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_thrift 2018-01-05T11:28:44,881 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:28:44,881 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,881 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,881 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:44,881 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:44,881 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:44,881 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:44,881 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:44,881 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:44,881 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:28:44,881 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping vectorization 2018-01-05T11:28:44,881 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:28:44,881 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: llap mode: only 2018-01-05T11:28:44,881 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:28:44,881 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:44,881 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@2806fe58 can be done in LLAP 2018-01-05T11:28:44,881 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,881 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed plan generation 2018-01-05T11:28:44,881 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:44,881 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validation start 2018-01-05T11:28:44,881 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Not a partition. 2018-01-05T11:28:44,881 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,881 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:src_thrift.aint, type:int, comment:null), FieldSchema(name:src_thrift.astring, type:string, comment:null), FieldSchema(name:src_thrift.lint, type:array, comment:null), FieldSchema(name:src_thrift.lstring, type:array, comment:null), FieldSchema(name:src_thrift.lintstring, type:array>, comment:null), FieldSchema(name:src_thrift.mstringstring, type:map, comment:null), FieldSchema(name:src_thrift.attributes, type:map,map>>>>, comment:null), FieldSchema(name:src_thrift.unionfield1, type:uniontype,map>, comment:null), FieldSchema(name:src_thrift.unionfield2, type:uniontype,map>, comment:null), FieldSchema(name:src_thrift.unionfield3, type:uniontype,map>, comment:null)], properties:null) 2018-01-05T11:28:44,882 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,882 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:44,882 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=12, flushCache_()=0, getTableColumnStatistics_(String, String, List, )=10} 2018-01-05T11:28:44,882 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112844_c0cd953c-25ae-48f6-93a3-d986f3d75bc3); Time taken: 0.042 seconds 2018-01-05T11:28:44,882 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:44,882 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,882 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112844_c0cd953c-25ae-48f6-93a3-d986f3d75bc3): ANALYZE TABLE src_thrift COMPUTE STATISTICS 2018-01-05T11:28:44,883 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,883 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: ANALYZE TABLE src_thrift COMPUTE STATISTICS 2018-01-05T11:28:44,883 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:28:44,883 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@src_thrift 2018-01-05T11:28:44,883 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@src_thrift 2018-01-05T11:28:44,883 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,883 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,883 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,883 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Query ID = djaiswal_20180105112844_c0cd953c-25ae-48f6-93a3-d986f3d75bc3 2018-01-05T11:28:44,883 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Total jobs = 1 2018-01-05T11:28:44,883 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,883 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:28:44,883 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MAPRED] in serial mode 2018-01-05T11:28:44,893 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_840_5848572550128803257-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:28:44,914 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:28:44,915 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:28:44,915 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:28:44,915 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112844_c0cd953c-25ae-48f6-93a3-d986f3d75bc3 2018-01-05T11:28:44,915 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Session is already open 2018-01-05T11:28:44,915 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:28:44,915 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,915 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Dag name: ANALYZE TABLE src_thrift COMPUTE STATISTICS (Stage-0) 2018-01-05T11:28:44,915 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: DagInfo: {"context":"Hive","description":"\n\nANALYZE TABLE src_thrift COMPUTE STATISTICS"} 2018-01-05T11:28:44,915 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112844_c0cd953c-25ae-48f6-93a3-d986f3d75bc3 with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:28:44,915 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,926 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:28:44,926 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,926 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:28:44,927 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,927 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 2.51KB 2018-01-05T11:28:44,932 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_840_5848572550128803257-1/-mr-10000 2018-01-05T11:28:44,943 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_840_5848572550128803257-1/-mr-10000 2018-01-05T11:28:44,945 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_thrift 2018-01-05T11:28:44,945 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,945 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,945 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,945 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=ANALYZE TABLE src_thrift COMPUTE STATISTICS (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112844_c0cd953c-25ae-48f6-93a3-d986f3d75bc3 } 2018-01-05T11:28:44,946 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:44,948 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: ANALYZE TABLE src_thrift COMPUTE STATISTICS (Stage-0) 2018-01-05T11:28:44,948 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:28:44,949 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_9, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_9.dot 2018-01-05T11:28:44,949 INFO [IPC Server handler 0 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_9 2018-01-05T11:28:44,949 WARN [IPC Server handler 0 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:44,949 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Running DAG: ANALYZE TABLE src_thrift COMPUTE STATISTICS (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112844_c0cd953c-25ae-48f6-93a3-d986f3d75bc3 } 2018-01-05T11:28:44,973 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_9][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_9, submitTime=1515180524948, queueName=null 2018-01-05T11:28:44,974 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_9_00 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_9_00 [Map 1], Default Resources= 2018-01-05T11:28:44,974 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_9_00 [Map 1] 2018-01-05T11:28:44,974 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_9_00 [Map 1] 2018-01-05T11:28:44,975 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:28:44,975 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_9_00 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:44,975 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:28:44,975 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_9][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_9, initTime=1515180524975 2018-01-05T11:28:44,975 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: dag_1515180518813_0001_9 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:28:44,975 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:28:44,976 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_9][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_9, startTime=1515180524975 2018-01-05T11:28:44,976 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_9 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:28:44,976 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {src_thrift={InputName=src_thrift}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:28:44,976 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: src_thrift, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:28:44,976 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_9_00 [Map 1] 2018-01-05T11:28:44,976 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_9, dagName=ANALYZE TABLE src_thrift COMPUTE STATISTICS (Stage-0) 2018-01-05T11:28:44,976 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,976 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,976 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:44,981 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_9_00 [Map 1] 2018-01-05T11:28:44,981 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_9_00 [Map 1] 2018-01-05T11:28:44,981 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:44,981 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_9_00 [Map 1] 2018-01-05T11:28:44,984 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:28:44,984 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@5793eb38 2018-01-05T11:28:44,985 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_840_5848572550128803257-1/djaiswal/_tez_scratch_dir/23eb8d6a-96b0-4b92-a081-3eac81911909/map.xml 2018-01-05T11:28:44,985 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_840_5848572550128803257-1/djaiswal/_tez_scratch_dir/23eb8d6a-96b0-4b92-a081-3eac81911909/map.xml 2018-01-05T11:28:44,985 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_840_5848572550128803257-1/djaiswal/_tez_scratch_dir/23eb8d6a-96b0-4b92-a081-3eac81911909/map.xml 2018-01-05T11:28:44,985 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:44,985 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:44,985 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:44,988 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:44,988 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 2.51KB 2018-01-05T11:28:44,989 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_9_00 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:28:44,989 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: src_thrift on vertex vertex_1515180518813_0001_9_00 [Map 1] 2018-01-05T11:28:44,989 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:28:44,989 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:44,989 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:28:44,989 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:44,989 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_840_5848572550128803257-1/djaiswal/_tez_scratch_dir/23eb8d6a-96b0-4b92-a081-3eac81911909/map.xml 2018-01-05T11:28:44,989 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:44,989 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias src_thrift 2018-01-05T11:28:44,989 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_thrift 2018-01-05T11:28:44,989 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_thrift 2018-01-05T11:28:44,989 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 2018-01-05T11:28:44,989 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = 2018-01-05T11:28:44,989 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_thrift 2018-01-05T11:28:44,990 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:44,990 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:44,990 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:44,990 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:45,009 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Time taken to get FileStatuses: 18 2018-01-05T11:28:45,009 INFO [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:28:45,009 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 19 2018-01-05T11:28:45,009 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 1 2018-01-05T11:28:45,010 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:45,010 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 1. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:28:45,010 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_840_5848572550128803257-1/djaiswal/_tez_scratch_dir/23eb8d6a-96b0-4b92-a081-3eac81911909/map.xml 2018-01-05T11:28:45,010 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_840_5848572550128803257-1/djaiswal/_tez_scratch_dir/23eb8d6a-96b0-4b92-a081-3eac81911909/map.xml 2018-01-05T11:28:45,010 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_840_5848572550128803257-1/djaiswal/_tez_scratch_dir/23eb8d6a-96b0-4b92-a081-3eac81911909/map.xml 2018-01-05T11:28:45,010 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:45,010 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:45,010 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:45,010 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:45,010 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 2.51KB 2018-01-05T11:28:45,010 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_thrift/complex.seq to src new group? true 2018-01-05T11:28:45,010 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:28:45,010 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:28:45,010 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:28:45,010 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 56 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 3070 Original splits: 1 2018-01-05T11:28:45,010 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Using original number of splits: 1 desired splits: 1 2018-01-05T11:28:45,010 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 1 grouped split count is 1, for bucket: 1 2018-01-05T11:28:45,010 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:28:45,011 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: src_thrift on vertex vertex_1515180518813_0001_9_00 [Map 1] 2018-01-05T11:28:45,012 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_9_00 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:45,012 INFO [App Shared Pool - #0] impl.VertexImpl: Got updated RootInputsSpecs: {src_thrift=forAllWorkUnits=true, update=[1]} 2018-01-05T11:28:45,012 INFO [App Shared Pool - #0] impl.VertexImpl: Vertex vertex_1515180518813_0001_9_00 [Map 1] parallelism set to 1 2018-01-05T11:28:45,012 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_9_00 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:45,012 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_9_00 [Map 1], numAdditionalOutputs=1 2018-01-05T11:28:45,012 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_9][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_9_00, initRequestedTime=1515180524976, initedTime=1515180525012, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:45,012 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_9_00 [Map 1] with distanceFromRoot: 0 2018-01-05T11:28:45,012 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_9_00 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:28:45,012 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_9][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_9_00, startRequestedTime=1515180524989, startedTime=1515180525012 2018-01-05T11:28:45,013 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_9][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_9_00, reconfigureDoneTime=1515180525012, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:28:45,013 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_9_00 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:45,013 INFO [App Shared Pool - #0] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:28:45,013 INFO [App Shared Pool - #0] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:28:45,013 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_9_00 [Map 1] 2018-01-05T11:28:45,013 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_9][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_9_00_000000, scheduledTime=1515180525013, launchTime=1515180525013 2018-01-05T11:28:45,013 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_9_00 [Map 1] 2018-01-05T11:28:45,013 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_9_00_000000_0, priority=2, capability=, hosts=null 2018-01-05T11:28:45,013 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=2,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:45,013 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=2,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:45,014 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:45,014 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_9_00_000000_0 on hosts=null 2018-01-05T11:28:45,014 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_9_00_000000_0 on hosts=null 2018-01-05T11:28:45,014 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_9_00_000000_0 2018-01-05T11:28:45,014 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:45,014 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #13, task=TaskInfo{task=attempt_1515180518813_0001_9_00_000000_0, priority=2, startTime=0, containerId=null, uniqueId=12, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000013 2018-01-05T11:28:45,014 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_9_00_000000_0; false 2018-01-05T11:28:45,014 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_9_00_000000_0, priority=2, startTime=311545523, containerId=container_111101111_0001_00_000013assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=12, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:45,015 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000013 for node: 10.22.8.182:49818 2018-01-05T11:28:45,018 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 9, name=ANALYZE TABLE src_thrift COMPUTE STATISTICS (Stage-0), queryId=djaiswal_20180105112844_c0cd953c-25ae-48f6-93a3-d986f3d75bc3 2018-01-05T11:28:45,018 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000013, attempt_1515180518813_0001_9_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:45,018 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000013 for node: 10.22.8.182:49818 2018-01-05T11:28:45,019 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000013 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:45,019 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000013, launchTime=1515180525019 2018-01-05T11:28:45,019 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_9_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112844_c0cd953c-25ae-48f6-93a3-d986f3d75bc3, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000013, dagName=ANALYZE TABLE src_thrift COMPUTE STATISTICS (Stage-0), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{src_thrift,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{out_Map 1,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180524975, firstAttemptStartTime=1515180525013, currentAttemptStartTime=1515180525018} 2018-01-05T11:28:45,019 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_9_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000013] on NM: [10.22.8.182:49818] 2018-01-05T11:28:45,020 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:45,020 INFO [IPC Server handler 0 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000013, QueryId=djaiswal_20180105112844_c0cd953c-25ae-48f6-93a3-d986f3d75bc3-9, DagId=9, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180525020 2018-01-05T11:28:45,020 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:45,020 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_9][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_9_00_000000_0, startTime=1515180525019, containerId=container_111101111_0001_00_000013, nodeId=10.22.8.182:49818 2018-01-05T11:28:45,020 DEBUG [IPC Server handler 0 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=9} with the ShuffleHandler 2018-01-05T11:28:45,033 INFO [IPC Server handler 0 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:28:45,033 DEBUG [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/9] 2018-01-05T11:28:45,034 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:45,034 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_9_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:45,034 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_9_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180525013, dagStartTime=1515180524975, withinDagPriority=2, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:28:45,034 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_9_00_000000_0 : ACCEPTED 2018-01-05T11:28:45,034 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_9_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:45,034 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_9_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180525013, dagStartTime=1515180524975, withinDagPriority=2, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:28:45,034 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_9_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:45,034 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_9_00_000000_0 to pre-emption queue 2018-01-05T11:28:45,034 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_9_00_000000_0: true 2018-01-05T11:28:45,034 INFO [TaskCommunicator # 4] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_9_00_000000_0 2018-01-05T11:28:45,034 INFO [TaskCommunicator # 4] GuaranteedTasks: Received response for attempt_1515180518813_0001_9_00_000000_0, true 2018-01-05T11:28:45,034 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: taskOwner hashCode:965956475 2018-01-05T11:28:45,035 INFO [Task-Executor-3] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : ANALYZE TABLE src_thrift COMPUTE STATISTICS (Stage-0), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_9_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=src_thrift, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=out_Map 1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:28:45,035 INFO [Task-Executor-3] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=954728448, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:45,035 INFO [Task-Executor-3] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_9_00_000000_0 vertexName: Map 1 2018-01-05T11:28:45,036 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000013, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_9_00_000000_0, eventCount=1 } 2018-01-05T11:28:45,036 DEBUG [TezTR-518813_1_9_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_9 queryId: djaiswal_20180105112844_c0cd953c-25ae-48f6-93a3-d986f3d75bc3 instanceType: TaskRunner2Callable 2018-01-05T11:28:45,036 INFO [TezTR-518813_1_9_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_9_00_000000_0 2018-01-05T11:28:45,036 DEBUG [TezTR-518813_1_9_0_0_0] log.PerfLogger: 2018-01-05T11:28:45,037 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:45,038 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:45,041 INFO [IPC Server handler 11 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_9_00_000000_0 sent events: (0-1). 2018-01-05T11:28:45,041 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:28:45,041 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_9_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:28:45,042 DEBUG [TezTR-518813_1_9_0_0_0] log.PerfLogger: 2018-01-05T11:28:45,042 INFO [TezTR-518813_1_9_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:45,042 INFO [TezTR-518813_1_9_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:45,042 INFO [I/O Setup 0 Initialize: {src_thrift}] input.MRInput: src_thrift using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:28:45,042 INFO [I/O Setup 0 Initialize: {src_thrift}] input.MRInputLegacy: src_thrift MRInputLegacy deferring initialization 2018-01-05T11:28:45,042 INFO [I/O Setup 1 Initialize: {out_Map 1}] output.MROutput: out_Map 1: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:28:45,042 INFO [TezTR-518813_1_9_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:45,042 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:45,042 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:45,042 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:45,042 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:45,042 INFO [TezTR-518813_1_9_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:45,042 INFO [TezTR-518813_1_9_0_0_0] resources.MemoryDistributor: InitialRequests=[src_thrift:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [out_Map 1:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput] 2018-01-05T11:28:45,042 INFO [TezTR-518813_1_9_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:45,042 INFO [TezTR-518813_1_9_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:45,042 INFO [TezTR-518813_1_9_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 954728448, TotalAvailable: 639668060, TotalRequested/TotalJVMHeap:0.00 2018-01-05T11:28:45,042 INFO [TezTR-518813_1_9_0_0_0] resources.MemoryDistributor: Allocations=[src_thrift:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [out_Map 1:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0] 2018-01-05T11:28:45,042 INFO [TezTR-518813_1_9_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:45,043 INFO [TezTR-518813_1_9_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: src_thrift being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:28:45,043 INFO [TezTR-518813_1_9_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:28:45,043 INFO [TezTR-518813_1_9_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:28:45,043 INFO [I/O Setup 0 Start: {src_thrift}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: src_thrift 2018-01-05T11:28:45,043 INFO [TezTR-518813_1_9_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:45,043 INFO [TezTR-518813_1_9_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_9_00_000000_0 2018-01-05T11:28:45,043 DEBUG [TezTR-518813_1_9_0_0_0] log.PerfLogger: 2018-01-05T11:28:45,043 DEBUG [TezTR-518813_1_9_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_9_00_000000_0_10037 2018-01-05T11:28:45,043 DEBUG [TezTR-518813_1_9_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:45,043 DEBUG [TezTR-518813_1_9_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_9_00_000000_0 2018-01-05T11:28:45,043 INFO [TezTR-518813_1_9_0_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112844_c0cd953c-25ae-48f6-93a3-d986f3d75bc3 2018-01-05T11:28:45,043 DEBUG [TezTR-518813_1_9_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:45,043 DEBUG [TezTR-518813_1_9_0_0_0] log.PerfLogger: 2018-01-05T11:28:45,043 DEBUG [TezTR-518813_1_9_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:45,043 DEBUG [TezTR-518813_1_9_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:45,043 DEBUG [TezTR-518813_1_9_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112844_c0cd953c-25ae-48f6-93a3-d986f3d75bc3_Map 1__MAP_PLAN__ 2018-01-05T11:28:45,043 INFO [TezTR-518813_1_9_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_840_5848572550128803257-1/djaiswal/_tez_scratch_dir/23eb8d6a-96b0-4b92-a081-3eac81911909/map.xml 2018-01-05T11:28:45,043 DEBUG [TezTR-518813_1_9_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_840_5848572550128803257-1/djaiswal/_tez_scratch_dir/23eb8d6a-96b0-4b92-a081-3eac81911909/map.xml 2018-01-05T11:28:45,043 DEBUG [TezTR-518813_1_9_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_840_5848572550128803257-1/djaiswal/_tez_scratch_dir/23eb8d6a-96b0-4b92-a081-3eac81911909/map.xml 2018-01-05T11:28:45,044 DEBUG [TezTR-518813_1_9_0_0_0] log.PerfLogger: 2018-01-05T11:28:45,044 INFO [TezTR-518813_1_9_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:45,044 DEBUG [TezTR-518813_1_9_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:45,044 DEBUG [TezTR-518813_1_9_0_0_0] log.PerfLogger: 2018-01-05T11:28:45,044 INFO [TezTR-518813_1_9_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 2.51KB 2018-01-05T11:28:45,045 DEBUG [TezTR-518813_1_9_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:45,045 INFO [TezTR-518813_1_9_0_0_0] tez.MapRecordProcessor: The input names are: [src_thrift] 2018-01-05T11:28:45,045 INFO [TezTR-518813_1_9_0_0_0] input.MRInput: Initialized MRInput: src_thrift 2018-01-05T11:28:45,045 DEBUG [TezTR-518813_1_9_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 1 2018-01-05T11:28:45,045 INFO [TezTR-518813_1_9_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_840_5848572550128803257-1/djaiswal/_tez_scratch_dir/23eb8d6a-96b0-4b92-a081-3eac81911909/map.xml 2018-01-05T11:28:45,045 DEBUG [TezTR-518813_1_9_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:45,045 DEBUG [TezTR-518813_1_9_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_thrift/complex.seq PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.SequenceFileInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat, properties={columns.types=int:string:array:array:array>:map:map,map>>>>:uniontype,map>:uniontype,map>:uniontype,map>, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_thrift, columns=aint,astring,lint,lstring,lintstring,mstringstring,attributes,unionfield1,unionfield2,unionfield3, serialization.format=org.apache.thrift.protocol.TBinaryProtocol, numRows=0, numFiles=1, serialization.ddl=struct src_thrift { i32 aint, string astring, list lint, list lstring, list> lintstring, map mstringstring, map,map>>>> attributes, uniontype,map> unionfield1, uniontype,map> unionfield2, uniontype,map> unionfield3}, transient_lastDdlTime=1515180524, rawDataSize=0, columns.comments=from deserializerfrom deserializerfrom deserializerfrom deserializerfrom deserializerfrom deserializerfrom deserializerfrom deserializerfrom deserializerfrom deserializer, totalSize=3070, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.thrift.ThriftDeserializer, serialization.class=org.apache.hadoop.hive.serde2.thrift.test.Complex, file.inputformat=org.apache.hadoop.mapred.SequenceFileInputFormat, column.name.delimiter=,, name=default.src_thrift}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.SequenceFileInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat, properties={columns.types=int:string:array:array:array>:map:map,map>>>>:uniontype,map>:uniontype,map>:uniontype,map>, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_thrift, columns=aint,astring,lint,lstring,lintstring,mstringstring,attributes,unionfield1,unionfield2,unionfield3, serialization.format=org.apache.thrift.protocol.TBinaryProtocol, numRows=0, numFiles=1, serialization.ddl=struct src_thrift { i32 aint, string astring, list lint, list lstring, list> lintstring, map mstringstring, map,map>>>> attributes, uniontype,map> unionfield1, uniontype,map> unionfield2, uniontype,map> unionfield3}, transient_lastDdlTime=1515180524, rawDataSize=0, columns.comments=from deserializerfrom deserializerfrom deserializerfrom deserializerfrom deserializerfrom deserializerfrom deserializerfrom deserializerfrom deserializerfrom deserializer, totalSize=3070, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.thrift.ThriftDeserializer, serialization.class=org.apache.hadoop.hive.serde2.thrift.test.Complex, file.inputformat=org.apache.hadoop.mapred.SequenceFileInputFormat, column.name.delimiter=,, name=default.src_thrift}, baseFileName=src_thrift, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_thrift=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.SequenceFileInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat, properties={columns.types=int:string:array:array:array>:map:map,map>>>>:uniontype,map>:uniontype,map>:uniontype,map>, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_thrift, columns=aint,astring,lint,lstring,lintstring,mstringstring,attributes,unionfield1,unionfield2,unionfield3, serialization.format=org.apache.thrift.protocol.TBinaryProtocol, numRows=0, numFiles=1, serialization.ddl=struct src_thrift { i32 aint, string astring, list lint, list lstring, list> lintstring, map mstringstring, map,map>>>> attributes, uniontype,map> unionfield1, uniontype,map> unionfield2, uniontype,map> unionfield3}, transient_lastDdlTime=1515180524, rawDataSize=0, columns.comments=from deserializerfrom deserializerfrom deserializerfrom deserializerfrom deserializerfrom deserializerfrom deserializerfrom deserializerfrom deserializerfrom deserializer, totalSize=3070, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.thrift.ThriftDeserializer, serialization.class=org.apache.hadoop.hive.serde2.thrift.test.Complex, file.inputformat=org.apache.hadoop.mapred.SequenceFileInputFormat, column.name.delimiter=,, name=default.src_thrift}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.SequenceFileInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat, properties={columns.types=int:string:array:array:array>:map:map,map>>>>:uniontype,map>:uniontype,map>:uniontype,map>, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_thrift, columns=aint,astring,lint,lstring,lintstring,mstringstring,attributes,unionfield1,unionfield2,unionfield3, serialization.format=org.apache.thrift.protocol.TBinaryProtocol, numRows=0, numFiles=1, serialization.ddl=struct src_thrift { i32 aint, string astring, list lint, list lstring, list> lintstring, map mstringstring, map,map>>>> attributes, uniontype,map> unionfield1, uniontype,map> unionfield2, uniontype,map> unionfield3}, transient_lastDdlTime=1515180524, rawDataSize=0, columns.comments=from deserializerfrom deserializerfrom deserializerfrom deserializerfrom deserializerfrom deserializerfrom deserializerfrom deserializerfrom deserializerfrom deserializer, totalSize=3070, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.thrift.ThriftDeserializer, serialization.class=org.apache.hadoop.hive.serde2.thrift.test.Complex, file.inputformat=org.apache.hadoop.mapred.SequenceFileInputFormat, column.name.delimiter=,, name=default.src_thrift}, baseFileName=src_thrift, vectorPartitionDesc=null]} 2018-01-05T11:28:45,046 DEBUG [TezTR-518813_1_9_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.SequenceFileInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:45,046 INFO [TezTR-518813_1_9_0_0_0] io.HiveInputFormat: Not using llap for org.apache.hadoop.mapred.SequenceFileInputFormat: supported = false, vectorized = true, cache only = false 2018-01-05T11:28:45,047 DEBUG [TezTR-518813_1_9_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_thrift/complex.seq 2018-01-05T11:28:45,047 INFO [TezTR-518813_1_9_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.mapred.SequenceFileInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_thrift/complex.seq:0+3070], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=3070} 2018-01-05T11:28:45,047 INFO [TezTR-518813_1_9_0_0_0] input.MRInput: src_thrift initialized RecordReader from event 2018-01-05T11:28:45,047 DEBUG [TezTR-518813_1_9_0_0_0] tez.MapRecordProcessor: Starting Output: out_Map 1 2018-01-05T11:28:45,047 INFO [TezTR-518813_1_9_0_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:28:45,047 INFO [TezTR-518813_1_9_0_0_0] exec.MapOperator: Initializing operator MAP[0] 2018-01-05T11:28:45,047 DEBUG [TezTR-518813_1_9_0_0_0] exec.MapOperator: Initialization Done 0 MAP 2018-01-05T11:28:45,047 DEBUG [TezTR-518813_1_9_0_0_0] exec.MapOperator: Operator 0 MAP initialized 2018-01-05T11:28:45,047 DEBUG [TezTR-518813_1_9_0_0_0] exec.MapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:28:45,047 DEBUG [TezTR-518813_1_9_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:45,047 DEBUG [TezTR-518813_1_9_0_0_0] exec.MapOperator: Adding alias src_thrift to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_thrift 2018-01-05T11:28:45,047 DEBUG [TezTR-518813_1_9_0_0_0] exec.MapOperator: dump TS[0] struct,lstring:array,lintstring:array>,mstringstring:map,attributes:map,map>>>>,unionfield1:uniontype,map>,unionfield2:uniontype,map>,unionfield3:uniontype,map>,raw__data__size:bigint> 2018-01-05T11:28:45,047 INFO [TezTR-518813_1_9_0_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:28:45,047 INFO [TezTR-518813_1_9_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:45,047 DEBUG [TezTR-518813_1_9_0_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:28:45,047 INFO [TezTR-518813_1_9_0_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:28:45,047 DEBUG [TezTR-518813_1_9_0_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:28:45,047 DEBUG [TezTR-518813_1_9_0_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:28:45,047 DEBUG [TezTR-518813_1_9_0_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:28:45,047 DEBUG [TezTR-518813_1_9_0_0_0] log.PerfLogger: 2018-01-05T11:28:45,048 DEBUG [TezTR-518813_1_9_0_0_0] exec.MapOperator: Processing alias(es) src_thrift for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_thrift/complex.seq 2018-01-05T11:28:45,052 INFO [TezTR-518813_1_9_0_0_0] exec.MapOperator: MAP[0]: records read - 1 2018-01-05T11:28:45,053 INFO [TezTR-518813_1_9_0_0_0] exec.MapOperator: MAP[0]: records read - 10 2018-01-05T11:28:45,054 DEBUG [TezTR-518813_1_9_0_0_0] log.PerfLogger: 2018-01-05T11:28:45,054 DEBUG [TezTR-518813_1_9_0_0_0] mr.ObjectCache: djaiswal_20180105112844_c0cd953c-25ae-48f6-93a3-d986f3d75bc3_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:28:45,054 DEBUG [TezTR-518813_1_9_0_0_0] exec.MapOperator: close called for operator MAP[0] 2018-01-05T11:28:45,054 INFO [TezTR-518813_1_9_0_0_0] exec.MapOperator: Closing operator MAP[0] 2018-01-05T11:28:45,054 INFO [TezTR-518813_1_9_0_0_0] exec.MapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:11, 2018-01-05T11:28:45,054 DEBUG [TezTR-518813_1_9_0_0_0] exec.MapOperator: Closing child = TS[0] 2018-01-05T11:28:45,054 DEBUG [TezTR-518813_1_9_0_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:28:45,054 DEBUG [TezTR-518813_1_9_0_0_0] exec.TableScanOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:45,054 INFO [TezTR-518813_1_9_0_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:28:45,054 DEBUG [TezTR-518813_1_9_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:45,054 DEBUG [TezTR-518813_1_9_0_0_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_840_5848572550128803257-1/-mr-10000 2018-01-05T11:28:45,054 INFO [TezTR-518813_1_9_0_0_0] exec.TableScanOperator: publishing : default.src_thrift/ : {numRows=11, rawDataSize=0} 2018-01-05T11:28:45,077 DEBUG [TezTR-518813_1_9_0_0_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_840_5848572550128803257-1/-mr-10000/tmpstats-0_0 2018-01-05T11:28:45,077 DEBUG [TezTR-518813_1_9_0_0_0] fs.FSStatsPublisher: Writing stats in it : {default.src_thrift/={numRows=11, rawDataSize=0}} 2018-01-05T11:28:45,077 DEBUG [TezTR-518813_1_9_0_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:28:45,077 DEBUG [TezTR-518813_1_9_0_0_0] exec.MapOperator: 0 Close done 2018-01-05T11:28:45,077 INFO [TezTR-518813_1_9_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_9_00_000000_0 2018-01-05T11:28:45,077 INFO [TezTR-518813_1_9_0_0_0] output.MROutput: out_Map 1 closed 2018-01-05T11:28:45,077 DEBUG [TezTR-518813_1_9_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:45,077 INFO [TezTR-518813_1_9_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:45,078 INFO [TezTR-518813_1_9_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_9_00_000000_0, askedToStop=false 2018-01-05T11:28:45,078 INFO [TezTR-518813_1_9_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_9_00_000000_0, stopRequested=false 2018-01-05T11:28:45,078 INFO [TezTR-518813_1_9_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_9_00_000000_0: Counters: 5 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=11, INPUT_SPLIT_LENGTH_BYTES=3070, OUTPUT_RECORDS=0][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=11]] 2018-01-05T11:28:45,078 DEBUG [TezTR-518813_1_9_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_9_0_0_0 thread id: 636 scheme: file bytesRead: 6204 bytesWritten: 86 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:45,078 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_9_00_000000_0, isTaskDone=true 2018-01-05T11:28:45,078 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000013, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_9_00_000000_0, eventCount=2 } 2018-01-05T11:28:45,079 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_9][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_9_00_000000_0, creationTime=1515180525013, allocationTime=1515180525015, startTime=1515180525019, finishTime=1515180525079, timeTaken=60, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:45,079 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:45,079 INFO [Task-Executor-3] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_9_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:45,079 INFO [Task-Executor-3] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_9_00_000000_0 2018-01-05T11:28:45,079 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_9_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:45,079 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_9_00_000000_0 2018-01-05T11:28:45,079 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_9_00_000000_0; false 2018-01-05T11:28:45,080 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_9_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:45,079 INFO [Task-Executor-3] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000013=45 2018-01-05T11:28:45,080 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_9_00_000000_0: true 2018-01-05T11:28:45,080 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:45,080 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:45,080 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_9_00_000000_0 2018-01-05T11:28:45,080 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000013, QueryId=djaiswal_20180105112844_c0cd953c-25ae-48f6-93a3-d986f3d75bc3-9, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:28:45,080 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_9_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:45,080 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_9_00_000000_0 2018-01-05T11:28:45,080 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_9][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_9_00_000000, startTime=1515180525019, finishTime=1515180525079, timeTaken=60, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_9_00_000000_0, diagnostics=, counters=Counters: 11, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=6204, FILE_BYTES_WRITTEN=86, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=11, INPUT_SPLIT_LENGTH_BYTES=3070, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=11 2018-01-05T11:28:45,081 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_9_00 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:45,081 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_9_00 [Map 1] 2018-01-05T11:28:45,081 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_9][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_9_00, initRequestedTime=1515180524976, initedTime=1515180525012, startRequestedTime=1515180524989, startedTime=1515180525012, finishTime=1515180525081, timeTaken=69, status=SUCCEEDED, diagnostics=, counters=Counters: 11, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=6204, FILE_BYTES_WRITTEN=86, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=11, INPUT_SPLIT_LENGTH_BYTES=3070, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=11, vertexStats=firstTaskStartTime=1515180525019, firstTasksToStart=[ task_1515180518813_0001_9_00_000000 ], lastTaskFinishTime=1515180525079, lastTasksToFinish=[ task_1515180518813_0001_9_00_000000 ], minTaskDuration=60, maxTaskDuration=60, avgTaskDuration=60.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_9_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_9_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:45,081 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_9_00 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:45,081 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_9][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000013, stoppedTime=1515180525081, exitStatus=0 2018-01-05T11:28:45,081 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000013 exited with diagnostics set to Container completed. 2018-01-05T11:28:45,081 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_9_00 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1 2018-01-05T11:28:45,081 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null 2018-01-05T11:28:45,081 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_9_00 [Map 1] 2018-01-05T11:28:45,082 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_9, queueSize=0 2018-01-05T11:28:45,082 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_9][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_9, startTime=1515180524975, finishTime=1515180525081, timeTaken=106, status=SUCCEEDED, diagnostics=, counters=Counters: 15, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, OTHER_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=225, AM_GC_TIME_MILLIS=0, File System Counters, FILE_BYTES_READ=6204, FILE_BYTES_WRITTEN=86, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=11, INPUT_SPLIT_LENGTH_BYTES=3070, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=11 2018-01-05T11:28:45,082 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_9 finished with state: SUCCEEDED 2018-01-05T11:28:45,082 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_9 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:28:45,082 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000013 2018-01-05T11:28:45,082 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_9, dagState=SUCCEEDED 2018-01-05T11:28:45,082 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_9_post 2018-01-05T11:28:45,082 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:45,082 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:28:45,082 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:28:45,082 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 9 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=1, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=1,NumRequestsWithoutLocation=1, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=1} 2018-01-05T11:28:45,082 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:28:45,082 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=ANALYZE TABLE src_thrift COMPUTE STATISTICS (Stage-0), with id=dag_1515180518813_0001_9 2018-01-05T11:28:45,082 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 9, to 10.22.8.182:49818 2018-01-05T11:28:45,083 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=1, Remaining=0 2018-01-05T11:28:45,083 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=ANALYZE TABLE src_thrift COMPUTE STATISTICS (Stage-0), with id=dag_1515180518813_0001_9 2018-01-05T11:28:45,083 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=9} 2018-01-05T11:28:45,083 INFO [IPC Server handler 2 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=9}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:28:45,083 INFO [IPC Server handler 2 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/9 after 300 seconds 2018-01-05T11:28:45,083 ERROR [IPC Server handler 2 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112844_c0cd953c-25ae-48f6-93a3-d986f3d75bc3, dag_1515180518813_0001_9 2018-01-05T11:28:45,083 INFO [IPC Server handler 2 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112844_c0cd953c-25ae-48f6-93a3-d986f3d75bc3 2018-01-05T11:28:45,083 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=9} = 0 2018-01-05T11:28:45,083 DEBUG [IPC Server handler 2 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=9} 2018-01-05T11:28:45,083 DEBUG [IPC Server handler 2 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:28:45,083 DEBUG [IPC Server handler 2 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=9} 2018-01-05T11:28:45,083 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,083 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,083 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 2018-01-05T11:28:45,083 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,084 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-2:STATS] in serial mode 2018-01-05T11:28:45,085 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@3285c141, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:45,085 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:45,085 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,085 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:45,085 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:45,085 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:45,085 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:45,085 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,085 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:45,085 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:45,085 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_840_5848572550128803257-1/-mr-10000 2018-01-05T11:28:45,085 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_840_5848572550128803257-1/-mr-10000 2018-01-05T11:28:45,086 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.src_thrift/ numRows 2018-01-05T11:28:45,086 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.src_thrift/ numRows 11 2018-01-05T11:28:45,086 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.src_thrift/ rawDataSize 2018-01-05T11:28:45,087 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.src_thrift/ rawDataSize 0 2018-01-05T11:28:45,087 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,088 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:45,088 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:45,088 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:45,088 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:45,088 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,088 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,088 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,089 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,090 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:45,090 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:45,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,090 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:45,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,090 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=src_thrift newtbl=src_thrift 2018-01-05T11:28:45,090 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=src_thrift newtbl=src_thrift 2018-01-05T11:28:45,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,103 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,103 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,104 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.src_thrift stats: [numFiles=1, numRows=11, totalSize=3070, rawDataSize=0] 2018-01-05T11:28:45,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: About to delete stats tmp dir :file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_840_5848572550128803257-1/-mr-10000 2018-01-05T11:28:45,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,104 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: ANALYZE TABLE src_thrift COMPUTE STATISTICS 2018-01-05T11:28:45,104 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:28:45,104 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@src_thrift 2018-01-05T11:28:45,104 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@src_thrift 2018-01-05T11:28:45,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,104 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:45,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=13} 2018-01-05T11:28:45,104 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112844_c0cd953c-25ae-48f6-93a3-d986f3d75bc3); Time taken: 0.222 seconds 2018-01-05T11:28:45,104 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:45,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,104 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:45,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query ANALYZE TABLE src_thrift COMPUTE STATISTICS 2018-01-05T11:28:45,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-44_840_5848572550128803257-1 2018-01-05T11:28:45,105 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.264 seconds 2018-01-05T11:28:45,105 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:45,105 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:45,105 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:45,105 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:45,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:45,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: -- -- Table srcbucket -- DROP TABLE IF EXISTS srcbucket_tmp 2018-01-05T11:28:45,105 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112845_2b541a6a-9fcf-4a71-a350-ccaa31add268): -- -- Table srcbucket -- DROP TABLE IF EXISTS srcbucket_tmp 2018-01-05T11:28:45,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: -- -- Table srcbucket -- DROP TABLE IF EXISTS srcbucket_tmp 2018-01-05T11:28:45,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:45,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,106 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:45,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,106 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:45,106 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:45,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,107 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:45,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,107 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:45,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,107 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:45,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:28:45,107 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112845_2b541a6a-9fcf-4a71-a350-ccaa31add268); Time taken: 0.002 seconds 2018-01-05T11:28:45,107 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:45,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,107 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112845_2b541a6a-9fcf-4a71-a350-ccaa31add268): -- -- Table srcbucket -- DROP TABLE IF EXISTS srcbucket_tmp 2018-01-05T11:28:45,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,108 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: -- -- Table srcbucket -- DROP TABLE IF EXISTS srcbucket_tmp 2018-01-05T11:28:45,108 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:45,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,108 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:45,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,108 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:45,108 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:45,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,109 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table srcbucket_tmp not found: default.srcbucket_tmp table not found 2018-01-05T11:28:45,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:45,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:45,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: -- -- Table srcbucket -- DROP TABLE IF EXISTS srcbucket_tmp 2018-01-05T11:28:45,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:45,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:45,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=0} 2018-01-05T11:28:45,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112845_2b541a6a-9fcf-4a71-a350-ccaa31add268); Time taken: 0.002 seconds 2018-01-05T11:28:45,110 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:45,110 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,110 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:45,110 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,110 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query -- -- Table srcbucket -- DROP TABLE IF EXISTS srcbucket_tmp 2018-01-05T11:28:45,110 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.005 seconds 2018-01-05T11:28:45,110 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:45,110 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:45,110 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:45,110 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:45,110 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,110 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:45,110 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,110 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,110 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,110 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS srcbucket 2018-01-05T11:28:45,110 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112845_b6df7ddf-46b1-4664-bf22-5427e50ba1ba): DROP TABLE IF EXISTS srcbucket 2018-01-05T11:28:45,110 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,110 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS srcbucket 2018-01-05T11:28:45,110 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:45,110 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,110 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,111 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,111 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,111 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:45,111 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,111 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket 2018-01-05T11:28:45,111 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket 2018-01-05T11:28:45,112 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,112 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,112 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:45,112 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,112 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:45,112 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,112 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:45,112 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:28:45,112 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112845_b6df7ddf-46b1-4664-bf22-5427e50ba1ba); Time taken: 0.002 seconds 2018-01-05T11:28:45,112 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:45,112 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,112 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112845_b6df7ddf-46b1-4664-bf22-5427e50ba1ba): DROP TABLE IF EXISTS srcbucket 2018-01-05T11:28:45,113 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,113 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS srcbucket 2018-01-05T11:28:45,113 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:45,113 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,113 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,113 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,113 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,113 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:45,113 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,113 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket 2018-01-05T11:28:45,113 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket 2018-01-05T11:28:45,114 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,114 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,114 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table srcbucket not found: default.srcbucket table not found 2018-01-05T11:28:45,114 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,114 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket 2018-01-05T11:28:45,114 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket 2018-01-05T11:28:45,114 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,114 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,114 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,114 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,114 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS srcbucket 2018-01-05T11:28:45,114 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:45,114 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,114 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,114 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:45,114 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=0} 2018-01-05T11:28:45,114 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112845_b6df7ddf-46b1-4664-bf22-5427e50ba1ba); Time taken: 0.002 seconds 2018-01-05T11:28:45,114 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:45,114 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,114 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:45,114 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,115 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query DROP TABLE IF EXISTS srcbucket 2018-01-05T11:28:45,115 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.005 seconds 2018-01-05T11:28:45,115 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:45,115 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:45,115 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:45,115 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:45,115 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,115 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:45,115 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,115 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,115 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,115 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: CREATE TABLE srcbucket_tmp (key INT, value STRING) STORED AS TEXTFILE 2018-01-05T11:28:45,115 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112845_5e42e6b4-d064-405a-aba3-bfce263b8686): CREATE TABLE srcbucket_tmp (key INT, value STRING) STORED AS TEXTFILE 2018-01-05T11:28:45,115 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,115 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: CREATE TABLE srcbucket_tmp (key INT, value STRING) STORED AS TEXTFILE 2018-01-05T11:28:45,115 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:45,115 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,115 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,115 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,115 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,115 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:45,115 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:28:45,116 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Creating table default.srcbucket_tmp position=13 2018-01-05T11:28:45,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,116 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_database: default 2018-01-05T11:28:45,116 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_database: default 2018-01-05T11:28:45,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:45,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,116 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:45,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: validation start 2018-01-05T11:28:45,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:28:45,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Not a partition. 2018-01-05T11:28:45,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,116 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:45,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,117 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:45,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getDatabase_(String, )=0, flushCache_()=0} 2018-01-05T11:28:45,117 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112845_5e42e6b4-d064-405a-aba3-bfce263b8686); Time taken: 0.001 seconds 2018-01-05T11:28:45,117 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:45,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,117 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112845_5e42e6b4-d064-405a-aba3-bfce263b8686): CREATE TABLE srcbucket_tmp (key INT, value STRING) STORED AS TEXTFILE 2018-01-05T11:28:45,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,117 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: CREATE TABLE srcbucket_tmp (key INT, value STRING) STORED AS TEXTFILE 2018-01-05T11:28:45,117 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: CREATETABLE 2018-01-05T11:28:45,117 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: database:default 2018-01-05T11:28:45,118 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@srcbucket_tmp 2018-01-05T11:28:45,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,118 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:45,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Found class for org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe 2018-01-05T11:28:45,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: creating table default.srcbucket_tmp on file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp 2018-01-05T11:28:45,119 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,119 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: create_table: Table(tableName:srcbucket_tmp, dbName:default, owner:hive_test_user, createTime:1515180525, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:int, comment:null), FieldSchema(name:value, type:string, comment:null)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, numFiles=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:45,119 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=create_table: Table(tableName:srcbucket_tmp, dbName:default, owner:hive_test_user, createTime:1515180525, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:int, comment:null), FieldSchema(name:value, type:string, comment:null)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, numFiles=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:45,120 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hive-site.xml 2018-01-05T11:28:45,120 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hivemetastore-site.xml 2018-01-05T11:28:45,120 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Unable to find config file metastore-site.xml 2018-01-05T11:28:45,120 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file null 2018-01-05T11:28:45,129 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:45,130 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: Location: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp specified for non-external table:srcbucket_tmp 2018-01-05T11:28:45,130 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp 2018-01-05T11:28:45,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,149 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,153 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@srcbucket_tmp) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@srcbucket_tmp) Type=TABLE WriteType=DDL_NO_LOCK is present 2018-01-05T11:28:45,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,155 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: CREATE TABLE srcbucket_tmp (key INT, value STRING) STORED AS TEXTFILE 2018-01-05T11:28:45,155 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: CREATETABLE 2018-01-05T11:28:45,155 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: database:default 2018-01-05T11:28:45,155 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@srcbucket_tmp 2018-01-05T11:28:45,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,155 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:45,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {createTable_(Table, )=36, isCompatibleWith_(Configuration, )=0} 2018-01-05T11:28:45,155 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112845_5e42e6b4-d064-405a-aba3-bfce263b8686); Time taken: 0.038 seconds 2018-01-05T11:28:45,155 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:45,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,155 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:45,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query CREATE TABLE srcbucket_tmp (key INT, value STRING) STORED AS TEXTFILE 2018-01-05T11:28:45,156 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.04 seconds 2018-01-05T11:28:45,156 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:45,156 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:45,156 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:45,156 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:45,156 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,156 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:45,156 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,156 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,156 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,156 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: CREATE TABLE srcbucket (key INT, value STRING) CLUSTERED BY (key) INTO 2 BUCKETS STORED AS TEXTFILE 2018-01-05T11:28:45,156 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112845_86125cec-2a2c-4255-9fd5-7f02dc323bda): CREATE TABLE srcbucket (key INT, value STRING) CLUSTERED BY (key) INTO 2 BUCKETS STORED AS TEXTFILE 2018-01-05T11:28:45,156 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,156 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: CREATE TABLE srcbucket (key INT, value STRING) CLUSTERED BY (key) INTO 2 BUCKETS STORED AS TEXTFILE 2018-01-05T11:28:45,157 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:45,157 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,157 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,157 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,157 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,157 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:45,157 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:28:45,157 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Creating table default.srcbucket position=13 2018-01-05T11:28:45,157 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,157 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_database: default 2018-01-05T11:28:45,157 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_database: default 2018-01-05T11:28:45,158 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:45,158 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,158 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,158 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:45,158 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: validation start 2018-01-05T11:28:45,158 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:28:45,158 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Not a partition. 2018-01-05T11:28:45,158 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,158 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:45,158 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,158 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:45,158 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getDatabase_(String, )=1, flushCache_()=0} 2018-01-05T11:28:45,158 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112845_86125cec-2a2c-4255-9fd5-7f02dc323bda); Time taken: 0.002 seconds 2018-01-05T11:28:45,158 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:45,158 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,158 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112845_86125cec-2a2c-4255-9fd5-7f02dc323bda): CREATE TABLE srcbucket (key INT, value STRING) CLUSTERED BY (key) INTO 2 BUCKETS STORED AS TEXTFILE 2018-01-05T11:28:45,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,159 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: CREATE TABLE srcbucket (key INT, value STRING) CLUSTERED BY (key) INTO 2 BUCKETS STORED AS TEXTFILE 2018-01-05T11:28:45,159 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: CREATETABLE 2018-01-05T11:28:45,159 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: database:default 2018-01-05T11:28:45,159 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@srcbucket 2018-01-05T11:28:45,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,160 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:45,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Found class for org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe 2018-01-05T11:28:45,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: creating table default.srcbucket on file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket 2018-01-05T11:28:45,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,161 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: create_table: Table(tableName:srcbucket, dbName:default, owner:hive_test_user, createTime:1515180525, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:int, comment:null), FieldSchema(name:value, type:string, comment:null)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:2, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format=1}), bucketCols:[key], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, numFiles=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:45,161 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=create_table: Table(tableName:srcbucket, dbName:default, owner:hive_test_user, createTime:1515180525, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:int, comment:null), FieldSchema(name:value, type:string, comment:null)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:2, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format=1}), bucketCols:[key], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, numFiles=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:45,161 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hive-site.xml 2018-01-05T11:28:45,162 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hivemetastore-site.xml 2018-01-05T11:28:45,162 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Unable to find config file metastore-site.xml 2018-01-05T11:28:45,162 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file null 2018-01-05T11:28:45,172 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:45,172 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: Location: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket specified for non-external table:srcbucket 2018-01-05T11:28:45,172 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket 2018-01-05T11:28:45,185 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000013, QueryId=djaiswal_20180105112844_c0cd953c-25ae-48f6-93a3-d986f3d75bc3-9, DagId=9, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-3, Succeeded=true, StartTime=1515180525034, EndTime=1515180525185 2018-01-05T11:28:45,189 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,191 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,192 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,192 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@srcbucket) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@srcbucket) Type=TABLE WriteType=DDL_NO_LOCK is present 2018-01-05T11:28:45,192 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,192 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,192 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: CREATE TABLE srcbucket (key INT, value STRING) CLUSTERED BY (key) INTO 2 BUCKETS STORED AS TEXTFILE 2018-01-05T11:28:45,192 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: CREATETABLE 2018-01-05T11:28:45,192 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: database:default 2018-01-05T11:28:45,192 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@srcbucket 2018-01-05T11:28:45,192 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,192 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,192 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:45,192 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {createTable_(Table, )=32, isCompatibleWith_(Configuration, )=0} 2018-01-05T11:28:45,192 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112845_86125cec-2a2c-4255-9fd5-7f02dc323bda); Time taken: 0.034 seconds 2018-01-05T11:28:45,192 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:45,192 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,192 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:45,192 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,192 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query CREATE TABLE srcbucket (key INT, value STRING) CLUSTERED BY (key) INTO 2 BUCKETS STORED AS TEXTFILE 2018-01-05T11:28:45,192 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.036 seconds 2018-01-05T11:28:45,192 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:45,192 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:45,193 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:45,193 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:45,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:45,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/srcbucket0.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:45,193 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112845_38eb3804-88ad-408f-ba42-f5e51a399b54): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/srcbucket0.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:45,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/srcbucket0.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:45,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:45,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,193 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:45,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.LoadSemanticAnalyzer: file@null@/Users/djaiswal/hive/hive/data/files/srcbucket0.txt 2018-01-05T11:28:45,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,193 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:45,193 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:45,197 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,197 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,199 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:45,199 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,199 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:45,199 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,200 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:45,200 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=4, flushCache_()=0} 2018-01-05T11:28:45,200 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112845_38eb3804-88ad-408f-ba42-f5e51a399b54); Time taken: 0.006 seconds 2018-01-05T11:28:45,200 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:45,200 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,200 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112845_38eb3804-88ad-408f-ba42-f5e51a399b54): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/srcbucket0.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:45,200 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,201 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/srcbucket0.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:45,201 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: LOAD 2018-01-05T11:28:45,201 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/srcbucket0.txt 2018-01-05T11:28:45,201 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@srcbucket_tmp 2018-01-05T11:28:45,201 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,201 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,201 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,201 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,201 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MOVE] in serial mode 2018-01-05T11:28:45,201 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@56a8b19f, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:45,201 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:45,201 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,201 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:45,201 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:45,201 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:45,201 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:45,201 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,201 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:45,201 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Loading data to table default.srcbucket_tmp from file:/Users/djaiswal/hive/hive/data/files/srcbucket0.txt 2018-01-05T11:28:45,201 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,202 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:45,202 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:45,202 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:45,202 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:45,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,204 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:45,204 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:45,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,204 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:45,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,204 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:45,204 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:45,208 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,208 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,209 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,209 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:45,209 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:45,234 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,234 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,234 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] FileOperations: moving file:/Users/djaiswal/hive/hive/data/files/srcbucket0.txt to file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp (replace = KEEP_EXISTING) 2018-01-05T11:28:45,234 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:45,234 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:45,257 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Moved src: file:/Users/djaiswal/hive/hive/data/files/srcbucket0.txt, to dest: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket0.txt 2018-01-05T11:28:45,258 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,258 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=srcbucket_tmp newtbl=srcbucket_tmp 2018-01-05T11:28:45,258 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=srcbucket_tmp newtbl=srcbucket_tmp 2018-01-05T11:28:45,262 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,268 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,269 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,269 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@srcbucket_tmp) Type=TABLE WriteType=INSERT because WriteEntity(default@srcbucket_tmp) Type=TABLE WriteType=INSERT is present 2018-01-05T11:28:45,269 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-1:STATS] in serial mode 2018-01-05T11:28:45,269 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@643bef2a, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:45,269 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:45,269 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,269 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:45,269 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:45,269 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:45,269 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:45,269 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,269 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:45,269 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:45,270 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,270 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:45,270 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:45,270 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:45,270 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:45,271 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,271 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,271 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,272 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,273 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,273 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:45,273 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:45,273 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,273 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:45,273 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,273 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:45,273 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:45,278 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,279 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,280 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,280 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=srcbucket_tmp newtbl=srcbucket_tmp 2018-01-05T11:28:45,280 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=srcbucket_tmp newtbl=srcbucket_tmp 2018-01-05T11:28:45,286 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,292 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,293 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,293 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.srcbucket_tmp stats: [numFiles=1, numRows=0, totalSize=5702, rawDataSize=0] 2018-01-05T11:28:45,293 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,293 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,293 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/srcbucket0.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:45,293 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: LOAD 2018-01-05T11:28:45,293 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/srcbucket0.txt 2018-01-05T11:28:45,293 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@srcbucket_tmp 2018-01-05T11:28:45,293 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,293 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,293 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:45,293 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=13, getTable_(String, String, )=6} 2018-01-05T11:28:45,293 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112845_38eb3804-88ad-408f-ba42-f5e51a399b54); Time taken: 0.093 seconds 2018-01-05T11:28:45,293 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:45,293 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,293 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:45,293 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,293 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/srcbucket0.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:45,293 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.1 seconds 2018-01-05T11:28:45,293 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:45,293 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:45,293 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:45,293 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:45,293 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,293 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:45,294 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,294 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,294 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,294 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/srcbucket1.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:45,294 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112845_34bb3d92-116b-4933-92d4-320aa40a32ba): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/srcbucket1.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:45,294 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,294 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/srcbucket1.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:45,294 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:45,294 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,294 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,294 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,294 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,294 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:45,294 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.LoadSemanticAnalyzer: file@null@/Users/djaiswal/hive/hive/data/files/srcbucket1.txt 2018-01-05T11:28:45,294 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,294 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:45,294 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:45,298 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,299 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,301 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:45,301 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,301 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:45,301 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,301 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:45,301 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=5, flushCache_()=0} 2018-01-05T11:28:45,301 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112845_34bb3d92-116b-4933-92d4-320aa40a32ba); Time taken: 0.007 seconds 2018-01-05T11:28:45,301 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:45,301 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,301 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112845_34bb3d92-116b-4933-92d4-320aa40a32ba): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/srcbucket1.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:45,302 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,302 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/srcbucket1.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:45,302 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: LOAD 2018-01-05T11:28:45,302 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/srcbucket1.txt 2018-01-05T11:28:45,302 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@srcbucket_tmp 2018-01-05T11:28:45,302 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,302 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,302 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,303 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,303 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MOVE] in serial mode 2018-01-05T11:28:45,303 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@892f1fa, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:45,303 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:45,303 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,303 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:45,303 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:45,303 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:45,303 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:45,303 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,303 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:45,303 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Loading data to table default.srcbucket_tmp from file:/Users/djaiswal/hive/hive/data/files/srcbucket1.txt 2018-01-05T11:28:45,303 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,304 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:45,304 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:45,304 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:45,304 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:45,304 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,304 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,304 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,305 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,305 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,305 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:45,305 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:45,305 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,306 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:45,306 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,306 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:45,306 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:45,309 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,309 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,311 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,311 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:45,311 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:45,315 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,316 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,316 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] FileOperations: moving file:/Users/djaiswal/hive/hive/data/files/srcbucket1.txt to file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp (replace = KEEP_EXISTING) 2018-01-05T11:28:45,316 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:45,316 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:45,340 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Moved src: file:/Users/djaiswal/hive/hive/data/files/srcbucket1.txt, to dest: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket1.txt 2018-01-05T11:28:45,341 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,341 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=srcbucket_tmp newtbl=srcbucket_tmp 2018-01-05T11:28:45,341 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=srcbucket_tmp newtbl=srcbucket_tmp 2018-01-05T11:28:45,345 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,351 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,351 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,351 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@srcbucket_tmp) Type=TABLE WriteType=INSERT because WriteEntity(default@srcbucket_tmp) Type=TABLE WriteType=INSERT is present 2018-01-05T11:28:45,351 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-1:STATS] in serial mode 2018-01-05T11:28:45,351 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@dd5ea90, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:45,351 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:45,351 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,351 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:45,351 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:45,351 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:45,351 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:45,351 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,351 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:45,351 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:45,352 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,352 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:45,352 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:45,352 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:45,352 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:45,353 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,353 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,353 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,354 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,354 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,354 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:45,354 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:45,354 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,354 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:45,354 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,354 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:45,354 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:45,358 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,358 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,359 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,359 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=srcbucket_tmp newtbl=srcbucket_tmp 2018-01-05T11:28:45,359 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=srcbucket_tmp newtbl=srcbucket_tmp 2018-01-05T11:28:45,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,369 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,369 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,369 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.srcbucket_tmp stats: [numFiles=2, numRows=0, totalSize=11603, rawDataSize=0] 2018-01-05T11:28:45,369 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,369 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,369 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/srcbucket1.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:45,369 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: LOAD 2018-01-05T11:28:45,369 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/srcbucket1.txt 2018-01-05T11:28:45,369 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@srcbucket_tmp 2018-01-05T11:28:45,369 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,369 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,369 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:45,369 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=10, getTable_(String, String, )=4} 2018-01-05T11:28:45,369 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112845_34bb3d92-116b-4933-92d4-320aa40a32ba); Time taken: 0.068 seconds 2018-01-05T11:28:45,369 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:45,369 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,369 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:45,369 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,369 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/srcbucket1.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:45,369 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.076 seconds 2018-01-05T11:28:45,369 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:45,369 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:45,370 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:45,370 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:45,370 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,370 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:45,370 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,370 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,370 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,370 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: INSERT INTO srcbucket SELECT * FROM srcbucket_tmp 2018-01-05T11:28:45,370 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9): INSERT INTO srcbucket SELECT * FROM srcbucket_tmp 2018-01-05T11:28:45,370 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,370 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: INSERT INTO srcbucket SELECT * FROM srcbucket_tmp 2018-01-05T11:28:45,371 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:45,371 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,371 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,371 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,371 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,371 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:45,371 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:28:45,371 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,371 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket 2018-01-05T11:28:45,371 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket 2018-01-05T11:28:45,375 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,376 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,376 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:45,376 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Completed phase 1 of Semantic Analysis 2018-01-05T11:28:45,376 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:28:45,376 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,376 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:45,376 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:45,381 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,382 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,382 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:28:45,382 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:28:45,382 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,382 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket 2018-01-05T11:28:45,382 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket 2018-01-05T11:28:45,386 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:45,386 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,387 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Completed getting MetaData in Semantic Analysis 2018-01-05T11:28:45,991 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:45,998 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:45,998 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_not_null_constraints : db=default tbl=srcbucket_tmp 2018-01-05T11:28:45,998 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_not_null_constraints : db=default tbl=srcbucket_tmp 2018-01-05T11:28:46,008 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:46,009 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:46,010 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:46,010 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=srcbucket_tmp 2018-01-05T11:28:46,010 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=srcbucket_tmp 2018-01-05T11:28:46,018 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:46,019 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:46,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:46,024 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=srcbucket_tmp 2018-01-05T11:28:46,024 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=srcbucket_tmp 2018-01-05T11:28:46,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:46,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:46,025 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:46,025 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_unique_constraints : db=default tbl=srcbucket_tmp 2018-01-05T11:28:46,025 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_unique_constraints : db=default tbl=srcbucket_tmp 2018-01-05T11:28:46,032 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:46,033 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:46,034 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:46,034 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=srcbucket_tmp 2018-01-05T11:28:46,034 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=srcbucket_tmp 2018-01-05T11:28:46,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:46,052 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:46,072 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:28:46,072 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,312 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] calcite.sql2rel: Plan after trimming unused fields HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.srcbucket_tmp]], table:alias=[srcbucket_tmp]) 2018-01-05T11:28:46,326 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Plan before removing subquery: HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.srcbucket_tmp]], table:alias=[srcbucket_tmp]) 2018-01-05T11:28:46,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Plan just after removing subquery: HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.srcbucket_tmp]], table:alias=[srcbucket_tmp]) 2018-01-05T11:28:46,349 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Plan after decorrelation: HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.srcbucket_tmp]], table:alias=[srcbucket_tmp]) 2018-01-05T11:28:46,349 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,354 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,354 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,355 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,355 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,357 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,357 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,358 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,358 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,360 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,373 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,390 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,391 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,391 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,391 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,392 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] calcite.sql2rel: Plan after trimming unused fields HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.srcbucket_tmp]], table:alias=[srcbucket_tmp]) 2018-01-05T11:28:46,392 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,392 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,393 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,393 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,393 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,393 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,394 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,394 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,395 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,395 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,397 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,399 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,399 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,402 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,406 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] translator.PlanModifierForASTConv: Original plan for PlanModifier HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.srcbucket_tmp]], table:alias=[srcbucket_tmp]) 2018-01-05T11:28:46,407 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] translator.PlanModifierForASTConv: Plan after nested convertOpTree HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.srcbucket_tmp]], table:alias=[srcbucket_tmp]) 2018-01-05T11:28:46,409 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] translator.PlanModifierForASTConv: Plan after propagating order HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.srcbucket_tmp]], table:alias=[srcbucket_tmp]) 2018-01-05T11:28:46,410 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] translator.PlanModifierForASTConv: Plan after fixTopOBSchema HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.srcbucket_tmp]], table:alias=[srcbucket_tmp]) 2018-01-05T11:28:46,411 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] translator.PlanModifierForASTConv: Final plan after modifier HiveProject(srcbucket_tmp.key=[$0], srcbucket_tmp.value=[$1]) HiveTableScan(table=[[default.srcbucket_tmp]], table:alias=[srcbucket_tmp]) 2018-01-05T11:28:46,417 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:28:46,417 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:46,417 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:46,417 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:46,422 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:46,423 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:46,423 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:28:46,423 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:28:46,423 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:46,423 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket 2018-01-05T11:28:46,423 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket 2018-01-05T11:28:46,427 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:46,427 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:46,427 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Table Plan for srcbucket_tmp TS[0] 2018-01-05T11:28:46,427 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: RR before GB srcbucket_tmp{(key,key: int)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB srcbucket_tmp{(key,key: int)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:28:46,428 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (. (tok_table_or_col srcbucket_tmp) key) srcbucket_tmp.key) (tok_selexpr (. (tok_table_or_col srcbucket_tmp) value) srcbucket_tmp.value)) 2018-01-05T11:28:46,428 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: genSelectPlan: input = srcbucket_tmp{(key,key: int)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} starRr = null 2018-01-05T11:28:46,428 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Select Plan row schema: null{(srcbucket_tmp.key,_col0: int)(srcbucket_tmp.value,_col1: string)} 2018-01-05T11:28:46,428 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:28:46,428 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/.hive-staging_hive_2018-01-05_11-28-45_370_3662807892329960653-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket 2018-01-05T11:28:46,428 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/.hive-staging_hive_2018-01-05_11-28-45_370_3662807892329960653-1 2018-01-05T11:28:46,454 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-45_370_3662807892329960653-1/-mr-10001 2018-01-05T11:28:46,454 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket row schema: null{(srcbucket_tmp.key,_col0: int)(srcbucket_tmp.value,_col1: string)} 2018-01-05T11:28:46,456 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:46,456 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket 2018-01-05T11:28:46,456 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket 2018-01-05T11:28:46,461 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:46,462 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:46,462 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Generate an operator pipeline to autogather column stats for table default.srcbucket in query INSERT INTO srcbucket SELECT * FROM srcbucket_tmp 2018-01-05T11:28:46,462 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: analyze table `default`.`srcbucket` compute statistics for columns 2018-01-05T11:28:46,462 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:46,463 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:46,463 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket 2018-01-05T11:28:46,463 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket 2018-01-05T11:28:46,467 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:46,468 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:46,468 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: select compute_stats(`key`, 'hll') , compute_stats(`value`, 'hll') from `default`.`srcbucket` 2018-01-05T11:28:46,468 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: select compute_stats(`key`, 'hll') , compute_stats(`value`, 'hll') from `default`.`srcbucket` 2018-01-05T11:28:46,469 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:46,469 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:46,469 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:46,469 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:28:46,469 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:46,469 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket 2018-01-05T11:28:46,469 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket 2018-01-05T11:28:46,474 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:46,474 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:46,474 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:28:46,474 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:28:46,488 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Table Plan for srcbucket TS[0] 2018-01-05T11:28:46,489 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:46,489 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:46,490 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: RR before GB srcbucket{(key,key: int)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB {((tok_function compute_stats (tok_table_or_col key) 'hll'),_col0: struct)((tok_function compute_stats (tok_table_or_col value) 'hll'),_col1: struct)} 2018-01-05T11:28:46,490 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (tok_function compute_stats (tok_table_or_col key) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col value) 'hll'))) 2018-01-05T11:28:46,490 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: genSelectPlan: input = {((tok_function compute_stats (tok_table_or_col key) 'hll'),_col0: struct)((tok_function compute_stats (tok_table_or_col value) 'hll'),_col1: struct)} starRr = srcbucket{(key,key: int)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:28:46,490 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Select Plan row schema: null{(_c0,_col0: struct)(_c1,_col1: struct)} 2018-01-05T11:28:46,490 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:28:46,490 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-46_462_6234719662926073401-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-46_462_6234719662926073401-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-46_462_6234719662926073401-1/-mr-10000 2018-01-05T11:28:46,490 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-46_462_6234719662926073401-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-46_462_6234719662926073401-1 2018-01-05T11:28:46,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-46_462_6234719662926073401-1/-mr-10002 2018-01-05T11:28:46,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-46_462_6234719662926073401-1/-mr-10000 row schema: null{(_c0,_col0: struct)(_c1,_col1: struct)} 2018-01-05T11:28:46,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Body Plan for Query Block null 2018-01-05T11:28:46,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:28:46,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Body Plan for Query Block null 2018-01-05T11:28:46,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:28:46,515 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: CBO Succeeded; optimized logical plan. 2018-01-05T11:28:46,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Before logical optimization TS[0]-SEL[1]-RS[2]-SEL[3]-FS[4] -SEL[1] 2018-01-05T11:28:46,518 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,518 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,518 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,518 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] lineage.Generator: Time taken for lineage transform=0 2018-01-05T11:28:46,519 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,519 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,519 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,519 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,519 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,519 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,519 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for FS(4) 2018-01-05T11:28:46,519 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for FS(6) 2018-01-05T11:28:46,520 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(5) 2018-01-05T11:28:46,520 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for GBY(4) 2018-01-05T11:28:46,520 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for RS(3) 2018-01-05T11:28:46,520 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for GBY(2) 2018-01-05T11:28:46,520 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(1) 2018-01-05T11:28:46,520 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(3) 2018-01-05T11:28:46,520 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for RS(2) 2018-01-05T11:28:46,520 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(1) 2018-01-05T11:28:46,520 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:28:46,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.SimplePredicatePushDown: After PPD: TS[0]-SEL[1]-RS[2]-SEL[3]-FS[4] -SEL[1] 2018-01-05T11:28:46,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 3 key:[] 2018-01-05T11:28:46,520 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 3 oldColExprMap: {VALUE._col0=Column[_col0], VALUE._col1=Column[_col1]} 2018-01-05T11:28:46,520 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 3 newColExprMap: {VALUE._col0=Column[_col0], VALUE._col1=Column[_col1]} 2018-01-05T11:28:46,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 2 key:[] 2018-01-05T11:28:46,520 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 2 oldColExprMap: {VALUE._col0=Column[_col0], VALUE._col1=Column[_col1]} 2018-01-05T11:28:46,520 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 2 newColExprMap: {VALUE._col0=Column[_col0], VALUE._col1=Column[_col1]} 2018-01-05T11:28:46,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,521 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,521 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,521 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,521 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,521 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,521 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,521 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,521 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,521 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,521 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,521 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,524 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,524 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,524 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,524 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,524 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.IdentityProjectRemover: Identity project remover optimization removed : SEL[5] 2018-01-05T11:28:46,525 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,525 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,525 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,525 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,525 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,525 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,525 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,525 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: After logical optimization TS[0]-SEL[1]-RS[2]-SEL[3]-FS[4] -SEL[1] 2018-01-05T11:28:46,525 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,525 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,525 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,525 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,525 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,525 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:46,525 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=srcbucket_tmp 2018-01-05T11:28:46,525 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=srcbucket_tmp 2018-01-05T11:28:46,529 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.250256ms + 0.007657ms, the query is [select "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "TAB_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (...)] 2018-01-05T11:28:46,530 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:46,536 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:46,537 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:46,537 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting stats (Num rows: 1 Data size: 188 Basic stats: COMPLETE Column stats: NONE) on TS[0] 2018-01-05T11:28:46,537 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (srcbucket_tmp): numRows: 1 dataSize: 188 basicStatsState: COMPLETE colStatsState: NONE colStats: {value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: true isEstimated: true, key= colName: key colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: true isEstimated: true} 2018-01-05T11:28:46,537 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting stats (Num rows: 1 Data size: 188 Basic stats: COMPLETE Column stats: NONE) on SEL[1] 2018-01-05T11:28:46,537 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [1] STATS-SEL[1]: numRows: 1 dataSize: 188 basicStatsState: COMPLETE colStatsState: NONE colStats: {value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: true isEstimated: true, key= colName: key colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: true isEstimated: true} 2018-01-05T11:28:46,537 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 188 Basic stats: COMPLETE Column stats: NONE) on RS[2] 2018-01-05T11:28:46,537 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-RS[2]: numRows: 1 dataSize: 188 basicStatsState: COMPLETE colStatsState: NONE colStats: {value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: true isEstimated: true, key= colName: key colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: true isEstimated: true} 2018-01-05T11:28:46,537 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting stats (Num rows: 1 Data size: 188 Basic stats: COMPLETE Column stats: NONE) on SEL[3] 2018-01-05T11:28:46,537 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [1] STATS-SEL[3]: numRows: 1 dataSize: 188 basicStatsState: COMPLETE colStatsState: NONE colStats: {value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: true isEstimated: true, key= colName: key colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: true isEstimated: true} 2018-01-05T11:28:46,537 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-FS[4]: numRows: 1 dataSize: 188 basicStatsState: COMPLETE colStatsState: NONE colStats: {value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: true isEstimated: true, key= colName: key colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: true isEstimated: true} 2018-01-05T11:28:46,537 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting stats (Num rows: 1 Data size: 188 Basic stats: COMPLETE Column stats: NONE) on SEL[1] 2018-01-05T11:28:46,537 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [1] STATS-SEL[1]: numRows: 1 dataSize: 188 basicStatsState: COMPLETE colStatsState: NONE colStats: {value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: true isEstimated: true, key= colName: key colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: true isEstimated: true} 2018-01-05T11:28:46,537 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[2]: inputSize: 188 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:28:46,537 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [Case 1] STATS-GBY[2]: cardinality: 1 2018-01-05T11:28:46,537 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 1052 Basic stats: COMPLETE Column stats: NONE) on GBY[2] 2018-01-05T11:28:46,537 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-GBY[2]: numRows: 1 dataSize: 1052 basicStatsState: COMPLETE colStatsState: NONE colStats: {_col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: true isEstimated: true, key= colName: key colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: true isEstimated: true} 2018-01-05T11:28:46,537 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 1052 Basic stats: COMPLETE Column stats: NONE) on RS[3] 2018-01-05T11:28:46,537 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-RS[3]: numRows: 1 dataSize: 1052 basicStatsState: COMPLETE colStatsState: NONE colStats: {_col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: true isEstimated: true, key= colName: key colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: true isEstimated: true} 2018-01-05T11:28:46,537 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[4]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:28:46,537 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [Case 7] STATS-GBY[4]: cardinality: 0 2018-01-05T11:28:46,537 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[4]: Equals 0 in number of rows. 0 rows will be set to 1 2018-01-05T11:28:46,537 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 1068 Basic stats: COMPLETE Column stats: NONE) on GBY[4] 2018-01-05T11:28:46,537 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-GBY[4]: numRows: 1 dataSize: 1068 basicStatsState: COMPLETE colStatsState: NONE colStats: {_col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: true isEstimated: true, key= colName: key colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: true isEstimated: true} 2018-01-05T11:28:46,537 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-FS[6]: numRows: 1 dataSize: 1068 basicStatsState: COMPLETE colStatsState: NONE colStats: {_col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: true isEstimated: true, key= colName: key colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: true isEstimated: true} 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on TS[0] 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on SEL[1] 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[2] 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on SEL[3] 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FileSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on FS[4] 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on SEL[1] 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[2] 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[3] 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[4] 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FileSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on FS[6] 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,538 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.SetReducerParallelism: Number of reducers determined to be: 2 2018-01-05T11:28:46,538 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: FS, 4 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: FS, 6 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: GBY, 4 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: RS, 3 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: GBY, 2 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: SEL, 1 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: SEL, 3 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: RS, 2 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: SEL, 1 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:28:46,538 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Cycle free: true 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: int|{srcbucket_tmp}key,value: string|{srcbucket_tmp}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{srcbucket_tmp}block__offset__inside__file,INPUT__FILE__NAME: string|{srcbucket_tmp}input__file__name,ROW__ID: struct|{srcbucket_tmp}row__id) 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(_col0: int|{null}srcbucket_tmp.key,_col1: string|{null}srcbucket_tmp.value) 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value]) 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[2] with rs:(VALUE._col0: int|{null}srcbucket_tmp.key,VALUE._col1: string|{null}srcbucket_tmp.value) 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[2] 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[3] with rs:(_col0: int|{null}srcbucket_tmp.key,_col1: string|{null}srcbucket_tmp.value) 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_2 {} 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[3] 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[VALUE._col0] Column[VALUE._col1]) 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[4] with rs:(key: int|{},value: string|{}) 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_3 {} 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[4] 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: int|{srcbucket}key,value: string|{srcbucket}value) 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_3 {} 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0] Column[_col1]) 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{}) 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:46,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,539 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: TS[0] 2018-01-05T11:28:46,539 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: RS[2] 2018-01-05T11:28:46,539 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:28:46,539 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp of table srcbucket_tmp 2018-01-05T11:28:46,539 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp 2018-01-05T11:28:46,539 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: RS[2] 2018-01-05T11:28:46,539 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: SEL[3] 2018-01-05T11:28:46,539 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: FS[4] 2018-01-05T11:28:46,539 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding reduce work (Reducer 2) for SEL[3] 2018-01-05T11:28:46,539 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Setting up reduce sink: RS[2] with following reduce work: Reducer 2 2018-01-05T11:28:46,539 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Removing RS[2] as parent from SEL[3] 2018-01-05T11:28:46,539 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: FS[4] 2018-01-05T11:28:46,539 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: SEL[3] 2018-01-05T11:28:46,539 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: RS[3] 2018-01-05T11:28:46,539 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: RS[3] 2018-01-05T11:28:46,539 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: GBY[4] 2018-01-05T11:28:46,539 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: FS[6] 2018-01-05T11:28:46,539 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding reduce work (Reducer 3) for GBY[4] 2018-01-05T11:28:46,539 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Setting up reduce sink: RS[3] with following reduce work: Reducer 3 2018-01-05T11:28:46,539 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Removing RS[3] as parent from GBY[4] 2018-01-05T11:28:46,539 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: FS[6] 2018-01-05T11:28:46,539 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:46,539 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket 2018-01-05T11:28:46,539 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket 2018-01-05T11:28:46,543 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:46,544 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:46,544 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:28:46,544 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,544 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,544 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:46,544 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:46,544 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:46,544 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:46,544 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:46,544 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:46,544 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:28:46,545 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping vectorization 2018-01-05T11:28:46,545 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:28:46,545 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: llap mode: only 2018-01-05T11:28:46,545 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:28:46,545 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'key' 2018-01-05T11:28:46,545 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'value' 2018-01-05T11:28:46,545 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:46,545 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@1fb320de can be done in LLAP 2018-01-05T11:28:46,545 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Reducer 2 2018-01-05T11:28:46,545 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(key, 'hll')' 2018-01-05T11:28:46,545 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'key' 2018-01-05T11:28:46,545 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:46,545 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(value, 'hll')' 2018-01-05T11:28:46,545 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'value' 2018-01-05T11:28:46,545 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:46,545 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking '_col0' 2018-01-05T11:28:46,545 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking '_col1' 2018-01-05T11:28:46,545 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:28:46,545 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col1' 2018-01-05T11:28:46,545 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:46,545 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@633e4742 can be done in LLAP 2018-01-05T11:28:46,545 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Reducer 3 2018-01-05T11:28:46,545 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col0)' 2018-01-05T11:28:46,545 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:28:46,545 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col1)' 2018-01-05T11:28:46,545 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col1' 2018-01-05T11:28:46,545 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:46,545 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@2bf83b29 can be done in LLAP 2018-01-05T11:28:46,545 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,545 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.HiveStatsUtils: Error requested is 20.0% 2018-01-05T11:28:46,545 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.HiveStatsUtils: Choosing 16 bit vectors.. 2018-01-05T11:28:46,545 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Completed plan generation 2018-01-05T11:28:46,545 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:46,545 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: validation start 2018-01-05T11:28:46,545 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Not a partition. 2018-01-05T11:28:46,546 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,546 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:srcbucket_tmp.key, type:int, comment:null), FieldSchema(name:srcbucket_tmp.value, type:string, comment:null)], properties:null) 2018-01-05T11:28:46,546 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,546 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:46,546 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {getNotNullConstraints_(NotNullConstraintsRequest, )=11, isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=46, flushCache_()=0, getUniqueConstraints_(UniqueConstraintsRequest, )=8, getPrimaryKeys_(PrimaryKeysRequest, )=9, getTableColumnStatistics_(String, String, List, )=12, getForeignKeys_(ForeignKeysRequest, )=18} 2018-01-05T11:28:46,546 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9); Time taken: 1.176 seconds 2018-01-05T11:28:46,546 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:46,546 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,546 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9): INSERT INTO srcbucket SELECT * FROM srcbucket_tmp 2018-01-05T11:28:46,547 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,547 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: INSERT INTO srcbucket SELECT * FROM srcbucket_tmp 2018-01-05T11:28:46,547 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:28:46,547 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@srcbucket_tmp 2018-01-05T11:28:46,547 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@srcbucket 2018-01-05T11:28:46,547 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,547 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,547 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,547 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Query ID = djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9 2018-01-05T11:28:46,547 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Total jobs = 1 2018-01-05T11:28:46,547 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,547 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:28:46,547 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-1:MAPRED] in serial mode 2018-01-05T11:28:46,557 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-45_370_3662807892329960653-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:28:46,580 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:28:46,580 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:28:46,580 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:28:46,580 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9 2018-01-05T11:28:46,580 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Session is already open 2018-01-05T11:28:46,580 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:28:46,580 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,580 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Dag name: INSERT INTO srcbucket SELECT...srcbucket_tmp (Stage-1) 2018-01-05T11:28:46,580 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: DagInfo: {"context":"Hive","description":"\nINSERT INTO srcbucket SELECT * FROM srcbucket_tmp"} 2018-01-05T11:28:46,580 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9 with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:28:46,580 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,591 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:28:46,592 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,593 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 3 size: 2.18KB 2018-01-05T11:28:46,606 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-46_462_6234719662926073401-1/-mr-10002 2018-01-05T11:28:46,616 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-46_462_6234719662926073401-1/-mr-10002 2018-01-05T11:28:46,619 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,619 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,630 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,630 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:28:46,632 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,632 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 2 size: 3.17KB 2018-01-05T11:28:46,647 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-45_370_3662807892329960653-1/-mr-10001 2018-01-05T11:28:46,658 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-45_370_3662807892329960653-1/-mr-10001 2018-01-05T11:28:46,658 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,662 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,674 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:28:46,674 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,674 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:28:46,675 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,675 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 2.25KB 2018-01-05T11:28:46,682 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp 2018-01-05T11:28:46,682 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,684 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,684 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,684 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=INSERT INTO srcbucket SELECT...srcbucket_tmp (Stage-1), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9 } 2018-01-05T11:28:46,685 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:46,685 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:46,685 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:46,688 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: INSERT INTO srcbucket SELECT...srcbucket_tmp (Stage-1) 2018-01-05T11:28:46,688 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:28:46,688 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_10, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_10.dot 2018-01-05T11:28:46,689 INFO [IPC Server handler 1 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_10 2018-01-05T11:28:46,689 WARN [IPC Server handler 1 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:46,689 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Running DAG: INSERT INTO srcbucket SELECT...srcbucket_tmp (Stage-1), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9 } 2018-01-05T11:28:46,713 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_10][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_10, submitTime=1515180526688, queueName=null 2018-01-05T11:28:46,714 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_10_00 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_10_00 [Map 1], Default Resources= 2018-01-05T11:28:46,714 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_10_00 [Map 1] 2018-01-05T11:28:46,714 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:28:46,714 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_10_00 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:46,714 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_10_01 [Reducer 2]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_10_01 [Reducer 2], Default Resources= 2018-01-05T11:28:46,714 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 2 2018-01-05T11:28:46,714 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_10_01 [Reducer 2] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:46,715 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_10_02 [Reducer 3]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_10_02 [Reducer 3], Default Resources= 2018-01-05T11:28:46,715 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_10_02 [Reducer 3] 2018-01-05T11:28:46,715 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 3 2018-01-05T11:28:46,715 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_10_02 [Reducer 3] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:46,715 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:28:46,715 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_10][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_10, initTime=1515180526715 2018-01-05T11:28:46,715 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: dag_1515180518813_0001_10 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:28:46,715 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:28:46,715 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_10][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_10, startTime=1515180526715 2018-01-05T11:28:46,715 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_10 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:28:46,716 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {srcbucket_tmp={InputName=srcbucket_tmp}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:28:46,716 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_10, dagName=INSERT INTO srcbucket SELECT...srcbucket_tmp (Stage-1) 2018-01-05T11:28:46,716 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: srcbucket_tmp, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:28:46,716 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_10_00 [Map 1] 2018-01-05T11:28:46,716 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,716 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,716 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,720 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_10_00 [Map 1] 2018-01-05T11:28:46,720 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_10_00 [Map 1] 2018-01-05T11:28:46,720 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:46,720 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_10_00 [Map 1] 2018-01-05T11:28:46,722 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:28:46,722 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@60e62bf1 2018-01-05T11:28:46,722 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-45_370_3662807892329960653-1/djaiswal/_tez_scratch_dir/64ad8e48-972e-40d6-9b83-c87ea1fa7b37/map.xml 2018-01-05T11:28:46,722 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-45_370_3662807892329960653-1/djaiswal/_tez_scratch_dir/64ad8e48-972e-40d6-9b83-c87ea1fa7b37/map.xml 2018-01-05T11:28:46,722 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-45_370_3662807892329960653-1/djaiswal/_tez_scratch_dir/64ad8e48-972e-40d6-9b83-c87ea1fa7b37/map.xml 2018-01-05T11:28:46,722 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:46,723 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:46,723 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:46,729 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:46,729 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 2.25KB 2018-01-05T11:28:46,729 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_10_00 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:28:46,729 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: srcbucket_tmp on vertex vertex_1515180518813_0001_10_00 [Map 1] 2018-01-05T11:28:46,729 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to ShuffleVertexManager for vertex_1515180518813_0001_10_01 [Reducer 2] 2018-01-05T11:28:46,729 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:28:46,730 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:46,730 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:28:46,730 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:46,730 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-45_370_3662807892329960653-1/djaiswal/_tez_scratch_dir/64ad8e48-972e-40d6-9b83-c87ea1fa7b37/map.xml 2018-01-05T11:28:46,730 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:46,730 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias srcbucket_tmp 2018-01-05T11:28:46,730 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp 2018-01-05T11:28:46,730 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp 2018-01-05T11:28:46,731 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 0,1 2018-01-05T11:28:46,731 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = key,value 2018-01-05T11:28:46,731 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp 2018-01-05T11:28:46,731 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:46,731 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:46,731 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:46,731 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:46,736 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.05 maxFrac: 0.75 auto: false desiredTaskIput: 104857600 2018-01-05T11:28:46,736 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:28:46,736 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 2 tasks for vertex: vertex_1515180518813_0001_10_01 [Reducer 2] 2018-01-05T11:28:46,736 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_10_01 [Reducer 2] 2018-01-05T11:28:46,736 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_10][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_10_01, reconfigureDoneTime=1515180526736, numTasks=2, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:28:46,736 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_10][Event:VERTEX_INITIALIZED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_10_01, initRequestedTime=1515180526729, initedTime=1515180526736, numTasks=2, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:46,736 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_10_01 [Reducer 2] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:28:46,736 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to ShuffleVertexManager for vertex_1515180518813_0001_10_02 [Reducer 3] 2018-01-05T11:28:46,742 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.05 maxFrac: 0.75 auto: false desiredTaskIput: 104857600 2018-01-05T11:28:46,742 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:28:46,742 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 1 tasks for vertex: vertex_1515180518813_0001_10_02 [Reducer 3] 2018-01-05T11:28:46,742 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_10_02 [Reducer 3] 2018-01-05T11:28:46,742 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_10][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_10_02, reconfigureDoneTime=1515180526742, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:28:46,742 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_10_02 [Reducer 3], numAdditionalOutputs=1 2018-01-05T11:28:46,742 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_10][Event:VERTEX_INITIALIZED]: vertexName=Reducer 3, vertexId=vertex_1515180518813_0001_10_02, initRequestedTime=1515180526736, initedTime=1515180526742, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:46,742 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_10_02 [Reducer 3] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:28:46,764 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Time taken to get FileStatuses: 32 2018-01-05T11:28:46,764 INFO [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total input files to process : 2 2018-01-05T11:28:46,764 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total # of splits generated by getSplits: 2, TimeTaken: 32 2018-01-05T11:28:46,764 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 2 2018-01-05T11:28:46,764 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:46,764 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 2. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:28:46,764 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-45_370_3662807892329960653-1/djaiswal/_tez_scratch_dir/64ad8e48-972e-40d6-9b83-c87ea1fa7b37/map.xml 2018-01-05T11:28:46,764 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-45_370_3662807892329960653-1/djaiswal/_tez_scratch_dir/64ad8e48-972e-40d6-9b83-c87ea1fa7b37/map.xml 2018-01-05T11:28:46,764 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-45_370_3662807892329960653-1/djaiswal/_tez_scratch_dir/64ad8e48-972e-40d6-9b83-c87ea1fa7b37/map.xml 2018-01-05T11:28:46,764 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:46,764 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:46,764 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:46,765 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:46,765 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 2.25KB 2018-01-05T11:28:46,765 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket1.txt to src new group? true 2018-01-05T11:28:46,765 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket0.txt to src new group? false 2018-01-05T11:28:46,766 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:28:46,766 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:28:46,766 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:28:46,766 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 214 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 11603 Original splits: 2 2018-01-05T11:28:46,766 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired numSplits: 1 lengthPerGroup: 11603 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 11603 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false 2018-01-05T11:28:46,767 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Number of splits desired: 1 created: 1 splitsProcessed: 2 2018-01-05T11:28:46,767 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 2 grouped split count is 1, for bucket: 1 2018-01-05T11:28:46,767 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:28:46,767 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: srcbucket_tmp on vertex vertex_1515180518813_0001_10_00 [Map 1] 2018-01-05T11:28:46,767 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_10_00 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:46,767 INFO [App Shared Pool - #0] impl.VertexImpl: Got updated RootInputsSpecs: {srcbucket_tmp=forAllWorkUnits=true, update=[1]} 2018-01-05T11:28:46,768 INFO [App Shared Pool - #0] impl.VertexImpl: Vertex vertex_1515180518813_0001_10_00 [Map 1] parallelism set to 1 2018-01-05T11:28:46,768 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_10_00 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:46,768 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_10][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_10_00, initRequestedTime=1515180526716, initedTime=1515180526768, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:46,768 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_10_00 [Map 1] with distanceFromRoot: 0 2018-01-05T11:28:46,768 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_10_00 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:28:46,768 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_10][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_10_00, startRequestedTime=1515180526729, startedTime=1515180526768 2018-01-05T11:28:46,768 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_10][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_10_00, reconfigureDoneTime=1515180526768, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:28:46,768 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_10_00 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:46,768 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_10_00 for vertex: vertex_1515180518813_0001_10_01 [Reducer 2] numStartedSources: 1 numSources: 1 2018-01-05T11:28:46,768 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_10_01 [Reducer 2] with distanceFromRoot: 1 2018-01-05T11:28:46,768 INFO [App Shared Pool - #0] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:28:46,768 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_10][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_10_01, startRequestedTime=1515180526768, startedTime=1515180526768 2018-01-05T11:28:46,768 INFO [App Shared Pool - #0] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:28:46,768 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_10_01 [Reducer 2] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:46,768 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_10_00 [Map 1] 2018-01-05T11:28:46,769 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_10_01 for vertex: vertex_1515180518813_0001_10_02 [Reducer 3] numStartedSources: 1 numSources: 1 2018-01-05T11:28:46,769 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_10_02 [Reducer 3] with distanceFromRoot: 2 2018-01-05T11:28:46,769 INFO [App Shared Pool - #1] vertexmanager.ShuffleVertexManagerBase: OnVertexStarted vertex: Reducer 2 with 0 source tasks and 2 pending tasks 2018-01-05T11:28:46,769 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_10][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_10_02, startRequestedTime=1515180526769, startedTime=1515180526769 2018-01-05T11:28:46,769 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Map 1 in vertex: Reducer 2 numBipartiteSourceTasks: 1 2018-01-05T11:28:46,769 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_10_02 [Reducer 3] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:46,770 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_10][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_10_00_000000, scheduledTime=1515180526770, launchTime=1515180526770 2018-01-05T11:28:46,770 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: OnVertexStarted vertex: Reducer 3 with 0 source tasks and 1 pending tasks 2018-01-05T11:28:46,770 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_10_00 [Map 1] 2018-01-05T11:28:46,770 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_10_00_000000_0, priority=8, capability=, hosts=null 2018-01-05T11:28:46,770 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=8,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:46,770 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Reducer 2 in vertex: Reducer 3 numBipartiteSourceTasks: 2 2018-01-05T11:28:46,770 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=8,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:46,770 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:46,770 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_10_00_000000_0 on hosts=null 2018-01-05T11:28:46,770 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_10_00_000000_0 on hosts=null 2018-01-05T11:28:46,770 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_10_00_000000_0 2018-01-05T11:28:46,770 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:46,770 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #14, task=TaskInfo{task=attempt_1515180518813_0001_10_00_000000_0, priority=8, startTime=0, containerId=null, uniqueId=13, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000014 2018-01-05T11:28:46,770 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_10_00_000000_0; false 2018-01-05T11:28:46,770 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_10_00_000000_0, priority=8, startTime=311547279, containerId=container_111101111_0001_00_000014assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=13, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:46,771 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000014 for node: 10.22.8.182:49818 2018-01-05T11:28:46,773 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 10, name=INSERT INTO srcbucket SELECT...srcbucket_tmp (Stage-1), queryId=djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9 2018-01-05T11:28:46,773 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000014, attempt_1515180518813_0001_10_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:46,773 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000014 for node: 10.22.8.182:49818 2018-01-05T11:28:46,774 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000014 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:46,774 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000014, launchTime=1515180526774 2018-01-05T11:28:46,774 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_10_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000014, dagName=INSERT INTO srcbucket SELECT...srcbucket_tmp (Stage-1), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{srcbucket_tmp,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{Reducer 2,org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput,2}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180526715, firstAttemptStartTime=1515180526770, currentAttemptStartTime=1515180526773} 2018-01-05T11:28:46,774 INFO [IPC Server handler 0 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000014, QueryId=djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9-10, DagId=10, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180526774 2018-01-05T11:28:46,774 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_10_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000014] on NM: [10.22.8.182:49818] 2018-01-05T11:28:46,774 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:46,774 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:46,774 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_10][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_10_00_000000_0, startTime=1515180526774, containerId=container_111101111_0001_00_000014, nodeId=10.22.8.182:49818 2018-01-05T11:28:46,775 DEBUG [IPC Server handler 0 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=10} with the ShuffleHandler 2018-01-05T11:28:46,786 INFO [IPC Server handler 0 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:28:46,786 DEBUG [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/10] 2018-01-05T11:28:46,786 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:46,786 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_10_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:46,786 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_10_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180526770, dagStartTime=1515180526715, withinDagPriority=8, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:28:46,786 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_10_00_000000_0 : ACCEPTED 2018-01-05T11:28:46,786 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_10_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:46,786 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_10_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180526770, dagStartTime=1515180526715, withinDagPriority=8, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:28:46,786 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_10_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:46,786 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_10_00_000000_0 to pre-emption queue 2018-01-05T11:28:46,787 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_10_00_000000_0: true 2018-01-05T11:28:46,787 INFO [TaskCommunicator # 6] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_10_00_000000_0 2018-01-05T11:28:46,787 INFO [TaskCommunicator # 6] GuaranteedTasks: Received response for attempt_1515180518813_0001_10_00_000000_0, true 2018-01-05T11:28:46,787 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: taskOwner hashCode:1706107596 2018-01-05T11:28:46,788 INFO [Task-Executor-3] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : INSERT INTO srcbucket SELECT...srcbucket_tmp (Stage-1), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_10_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=srcbucket_tmp, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=Reducer 2, physicalEdgeCount=2, outputClassName=org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput }}, ] 2018-01-05T11:28:46,788 INFO [Task-Executor-3] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=954728448, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:46,788 INFO [Task-Executor-3] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_10_00_000000_0 vertexName: Map 1 2018-01-05T11:28:46,788 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000014, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_10_00_000000_0, eventCount=1 } 2018-01-05T11:28:46,788 DEBUG [TezTR-518813_1_10_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_10 queryId: djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9 instanceType: TaskRunner2Callable 2018-01-05T11:28:46,788 INFO [TezTR-518813_1_10_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_10_00_000000_0 2018-01-05T11:28:46,789 DEBUG [TezTR-518813_1_10_0_0_0] log.PerfLogger: 2018-01-05T11:28:46,789 INFO [I/O Setup 1 Initialize: {Reducer 2}] writers.UnorderedPartitionedKVWriter: Requested BufferSize (tez.runtime.unordered.output.buffer.size-mb) : 100 2018-01-05T11:28:46,789 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:46,791 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:46,793 INFO [IPC Server handler 6 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_10_00_000000_0 sent events: (0-1). 2018-01-05T11:28:46,794 INFO [I/O Setup 0 Initialize: {srcbucket_tmp}] input.MRInput: srcbucket_tmp using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:28:46,794 INFO [I/O Setup 0 Initialize: {srcbucket_tmp}] input.MRInputLegacy: srcbucket_tmp MRInputLegacy deferring initialization 2018-01-05T11:28:46,794 DEBUG [TezTR-518813_1_10_0_0_0] log.PerfLogger: 2018-01-05T11:28:46,794 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:28:46,794 INFO [TezTR-518813_1_10_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:46,794 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_10_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:28:46,794 INFO [TezTR-518813_1_10_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:46,794 INFO [TezTR-518813_1_10_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:46,794 INFO [TezTR-518813_1_10_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:46,794 INFO [TezTR-518813_1_10_0_0_0] resources.MemoryDistributor: InitialRequests=[srcbucket_tmp:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [Reducer 2:OUTPUT:104857600:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput] 2018-01-05T11:28:46,794 INFO [TezTR-518813_1_10_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:46,794 INFO [TezTR-518813_1_10_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:46,794 INFO [TezTR-518813_1_10_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 104857600, TotalRequestedScaled: 5.24288E7, TotalJVMHeap: 954728448, TotalAvailable: 639668060, TotalRequested/TotalJVMHeap:0.11 2018-01-05T11:28:46,794 INFO [TezTR-518813_1_10_0_0_0] resources.MemoryDistributor: Allocations=[srcbucket_tmp:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput:OUTPUT:104857600:104857600] 2018-01-05T11:28:46,794 INFO [TezTR-518813_1_10_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:46,794 INFO [TezTR-518813_1_10_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: srcbucket_tmp being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:28:46,794 INFO [TezTR-518813_1_10_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:28:46,794 INFO [TezTR-518813_1_10_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:28:46,794 INFO [I/O Setup 1 Start: {srcbucket_tmp}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: srcbucket_tmp 2018-01-05T11:28:46,794 INFO [TezTR-518813_1_10_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:46,795 INFO [TezTR-518813_1_10_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_10_00_000000_0 2018-01-05T11:28:46,795 DEBUG [TezTR-518813_1_10_0_0_0] log.PerfLogger: 2018-01-05T11:28:46,795 DEBUG [TezTR-518813_1_10_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_10_00_000000_0_10040 2018-01-05T11:28:46,795 DEBUG [TezTR-518813_1_10_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:46,795 DEBUG [TezTR-518813_1_10_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_10_00_000000_0 2018-01-05T11:28:46,795 INFO [TezTR-518813_1_10_0_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9 2018-01-05T11:28:46,795 DEBUG [TezTR-518813_1_10_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:46,795 DEBUG [TezTR-518813_1_10_0_0_0] log.PerfLogger: 2018-01-05T11:28:46,795 DEBUG [TezTR-518813_1_10_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:46,795 DEBUG [TezTR-518813_1_10_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:46,795 DEBUG [TezTR-518813_1_10_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9_Map 1__MAP_PLAN__ 2018-01-05T11:28:46,795 INFO [TezTR-518813_1_10_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-45_370_3662807892329960653-1/djaiswal/_tez_scratch_dir/64ad8e48-972e-40d6-9b83-c87ea1fa7b37/map.xml 2018-01-05T11:28:46,795 DEBUG [TezTR-518813_1_10_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-45_370_3662807892329960653-1/djaiswal/_tez_scratch_dir/64ad8e48-972e-40d6-9b83-c87ea1fa7b37/map.xml 2018-01-05T11:28:46,795 DEBUG [TezTR-518813_1_10_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-45_370_3662807892329960653-1/djaiswal/_tez_scratch_dir/64ad8e48-972e-40d6-9b83-c87ea1fa7b37/map.xml 2018-01-05T11:28:46,795 DEBUG [TezTR-518813_1_10_0_0_0] log.PerfLogger: 2018-01-05T11:28:46,795 INFO [TezTR-518813_1_10_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:46,795 DEBUG [TezTR-518813_1_10_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:46,796 DEBUG [TezTR-518813_1_10_0_0_0] log.PerfLogger: 2018-01-05T11:28:46,796 INFO [TezTR-518813_1_10_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 2.25KB 2018-01-05T11:28:46,796 DEBUG [TezTR-518813_1_10_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:46,796 INFO [TezTR-518813_1_10_0_0_0] tez.MapRecordProcessor: The input names are: [srcbucket_tmp] 2018-01-05T11:28:46,796 INFO [TezTR-518813_1_10_0_0_0] input.MRInput: Initialized MRInput: srcbucket_tmp 2018-01-05T11:28:46,797 DEBUG [TezTR-518813_1_10_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 2 2018-01-05T11:28:46,797 INFO [TezTR-518813_1_10_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-45_370_3662807892329960653-1/djaiswal/_tez_scratch_dir/64ad8e48-972e-40d6-9b83-c87ea1fa7b37/map.xml 2018-01-05T11:28:46,797 DEBUG [TezTR-518813_1_10_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:46,798 DEBUG [TezTR-518813_1_10_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket1.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcbucket_tmp, numFiles=2, columns.types=int:string, serialization.format=1, serialization.ddl=struct srcbucket_tmp { i32 key, string value}, columns=key,value, columns.comments=, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, totalSize=11603, column.name.delimiter=,, transient_lastDdlTime=1515180525}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcbucket_tmp, numFiles=2, columns.types=int:string, columns=key,value, serialization.format=1, serialization.ddl=struct srcbucket_tmp { i32 key, string value}, columns.comments=, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, totalSize=11603, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp, column.name.delimiter=,, transient_lastDdlTime=1515180525}, baseFileName=srcbucket_tmp, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcbucket_tmp, numFiles=2, columns.types=int:string, serialization.format=1, serialization.ddl=struct srcbucket_tmp { i32 key, string value}, columns=key,value, columns.comments=, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, totalSize=11603, column.name.delimiter=,, transient_lastDdlTime=1515180525}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcbucket_tmp, numFiles=2, columns.types=int:string, columns=key,value, serialization.format=1, serialization.ddl=struct srcbucket_tmp { i32 key, string value}, columns.comments=, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, totalSize=11603, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp, column.name.delimiter=,, transient_lastDdlTime=1515180525}, baseFileName=srcbucket_tmp, vectorPartitionDesc=null]} 2018-01-05T11:28:46,798 DEBUG [TezTR-518813_1_10_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:46,798 INFO [TezTR-518813_1_10_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:46,798 DEBUG [TezTR-518813_1_10_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:46,799 DEBUG [TezTR-518813_1_10_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:46,799 INFO [TezTR-518813_1_10_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-45_370_3662807892329960653-1/djaiswal/_tez_scratch_dir/64ad8e48-972e-40d6-9b83-c87ea1fa7b37/map.xml 2018-01-05T11:28:46,799 DEBUG [TezTR-518813_1_10_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:46,799 INFO [TezTR-518813_1_10_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_10_00_000000_0 2018-01-05T11:28:46,799 INFO [TezTR-518813_1_10_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:28:46,800 DEBUG [TezTR-518813_1_10_0_0_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:46,800 DEBUG [TezTR-518813_1_10_0_0_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:46,800 DEBUG [TezTR-518813_1_10_0_0_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:46,800 DEBUG [TezTR-518813_1_10_0_0_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:46,800 WARN [TezTR-518813_1_10_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket1.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:28:46,800 DEBUG [TezTR-518813_1_10_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:28:46,800 WARN [TezTR-518813_1_10_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket1.txt:0+5901 2018-01-05T11:28:46,800 DEBUG [TezTR-518813_1_10_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket1.txt 2018-01-05T11:28:46,800 INFO [TezTR-518813_1_10_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket1.txt:0+5901, org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket0.txt:0+5702], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=11603} 2018-01-05T11:28:46,800 INFO [TezTR-518813_1_10_0_0_0] input.MRInput: srcbucket_tmp initialized RecordReader from event 2018-01-05T11:28:46,800 DEBUG [TezTR-518813_1_10_0_0_0] tez.MapRecordProcessor: Starting Output: Reducer 2 2018-01-05T11:28:46,801 INFO [TezTR-518813_1_10_0_0_0] writers.BaseUnorderedPartitionedKVWriter: Instantiating Partitioner: [org.apache.tez.mapreduce.partition.MRPartitioner] 2018-01-05T11:28:46,802 INFO [TezTR-518813_1_10_0_0_0] partition.MRPartitioner: Using oldApi, MRpartitionerClass=org.apache.hadoop.hive.ql.io.DefaultHivePartitioner 2018-01-05T11:28:46,807 INFO [TezTR-518813_1_10_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: numBuffers=2, sizePerBuffer=52428800, skipBuffers=false, numPartitions=2, availableMemory=104857600, maxSingleBufferSizeBytes=2147483647, pipelinedShuffle=false, isFinalMergeEnabled=true, numPartitions=2, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:28:46,808 INFO [TezTR-518813_1_10_0_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:28:46,808 INFO [TezTR-518813_1_10_0_0_0] exec.MapOperator: Initializing operator MAP[0] 2018-01-05T11:28:46,808 DEBUG [TezTR-518813_1_10_0_0_0] exec.MapOperator: Initialization Done 0 MAP 2018-01-05T11:28:46,808 DEBUG [TezTR-518813_1_10_0_0_0] exec.MapOperator: Operator 0 MAP initialized 2018-01-05T11:28:46,808 DEBUG [TezTR-518813_1_10_0_0_0] exec.MapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:28:46,808 DEBUG [TezTR-518813_1_10_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:46,808 DEBUG [TezTR-518813_1_10_0_0_0] exec.MapOperator: Adding alias srcbucket_tmp to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp 2018-01-05T11:28:46,808 DEBUG [TezTR-518813_1_10_0_0_0] exec.MapOperator: dump TS[0] struct 2018-01-05T11:28:46,808 INFO [TezTR-518813_1_10_0_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:28:46,808 INFO [TezTR-518813_1_10_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:46,808 DEBUG [TezTR-518813_1_10_0_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:28:46,808 INFO [TezTR-518813_1_10_0_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:28:46,808 DEBUG [TezTR-518813_1_10_0_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:28:46,808 DEBUG [TezTR-518813_1_10_0_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:28:46,808 DEBUG [TezTR-518813_1_10_0_0_0] exec.TableScanOperator: Initializing children of 0 TS 2018-01-05T11:28:46,808 DEBUG [TezTR-518813_1_10_0_0_0] exec.SelectOperator: Initializing child 1 SEL 2018-01-05T11:28:46,808 INFO [TezTR-518813_1_10_0_0_0] exec.SelectOperator: Initializing operator SEL[1] 2018-01-05T11:28:46,809 INFO [TezTR-518813_1_10_0_0_0] exec.SelectOperator: SELECT struct 2018-01-05T11:28:46,809 DEBUG [TezTR-518813_1_10_0_0_0] exec.SelectOperator: Initialization Done 1 SEL 2018-01-05T11:28:46,809 DEBUG [TezTR-518813_1_10_0_0_0] exec.SelectOperator: Operator 1 SEL initialized 2018-01-05T11:28:46,809 DEBUG [TezTR-518813_1_10_0_0_0] exec.SelectOperator: Initializing children of 1 SEL 2018-01-05T11:28:46,809 DEBUG [TezTR-518813_1_10_0_0_0] exec.ReduceSinkOperator: Initializing child 2 RS 2018-01-05T11:28:46,809 INFO [TezTR-518813_1_10_0_0_0] exec.ReduceSinkOperator: Initializing operator RS[2] 2018-01-05T11:28:46,809 DEBUG [TezTR-518813_1_10_0_0_0] exec.ReduceSinkOperator: keys size is 0 2018-01-05T11:28:46,809 INFO [TezTR-518813_1_10_0_0_0] exec.ReduceSinkOperator: Using tag = -1 2018-01-05T11:28:46,809 DEBUG [TezTR-518813_1_10_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:46,809 DEBUG [TezTR-518813_1_10_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:46,809 DEBUG [TezTR-518813_1_10_0_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[int, string] 2018-01-05T11:28:46,809 DEBUG [TezTR-518813_1_10_0_0_0] exec.ReduceSinkOperator: Initialization Done 2 RS 2018-01-05T11:28:46,809 DEBUG [TezTR-518813_1_10_0_0_0] exec.ReduceSinkOperator: Operator 2 RS initialized 2018-01-05T11:28:46,809 DEBUG [TezTR-518813_1_10_0_0_0] exec.ReduceSinkOperator: Initialization Done 2 RS done is reset. 2018-01-05T11:28:46,809 DEBUG [TezTR-518813_1_10_0_0_0] exec.SelectOperator: Initialization Done 1 SEL done is reset. 2018-01-05T11:28:46,809 DEBUG [TezTR-518813_1_10_0_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:28:46,809 INFO [TezTR-518813_1_10_0_0_0] exec.OperatorUtils: Setting output collector: RS[2] --> Reducer 2 2018-01-05T11:28:46,809 DEBUG [TezTR-518813_1_10_0_0_0] log.PerfLogger: 2018-01-05T11:28:46,809 INFO [TezTR-518813_1_10_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-45_370_3662807892329960653-1/djaiswal/_tez_scratch_dir/64ad8e48-972e-40d6-9b83-c87ea1fa7b37/map.xml 2018-01-05T11:28:46,809 DEBUG [TezTR-518813_1_10_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:46,809 DEBUG [TezTR-518813_1_10_0_0_0] exec.MapOperator: Processing alias(es) srcbucket_tmp for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket1.txt 2018-01-05T11:28:46,809 INFO [TezTR-518813_1_10_0_0_0] exec.ReduceSinkOperator: keys are [] num distributions: 0 2018-01-05T11:28:46,809 INFO [TezTR-518813_1_10_0_0_0] exec.MapOperator: MAP[0]: records read - 1 2018-01-05T11:28:46,810 INFO [TezTR-518813_1_10_0_0_0] exec.MapOperator: MAP[0]: records read - 10 2018-01-05T11:28:46,812 INFO [TezTR-518813_1_10_0_0_0] exec.MapOperator: MAP[0]: records read - 100 2018-01-05T11:28:46,822 DEBUG [TezTR-518813_1_10_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 1 of 2 2018-01-05T11:28:46,822 DEBUG [TezTR-518813_1_10_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket0.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcbucket_tmp, numFiles=2, columns.types=int:string, serialization.format=1, serialization.ddl=struct srcbucket_tmp { i32 key, string value}, columns=key,value, columns.comments=, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, totalSize=11603, column.name.delimiter=,, transient_lastDdlTime=1515180525}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcbucket_tmp, numFiles=2, columns.types=int:string, columns=key,value, serialization.format=1, serialization.ddl=struct srcbucket_tmp { i32 key, string value}, columns.comments=, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, totalSize=11603, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp, column.name.delimiter=,, transient_lastDdlTime=1515180525}, baseFileName=srcbucket_tmp, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcbucket_tmp, numFiles=2, columns.types=int:string, serialization.format=1, serialization.ddl=struct srcbucket_tmp { i32 key, string value}, columns=key,value, columns.comments=, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, totalSize=11603, column.name.delimiter=,, transient_lastDdlTime=1515180525}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcbucket_tmp, numFiles=2, columns.types=int:string, columns=key,value, serialization.format=1, serialization.ddl=struct srcbucket_tmp { i32 key, string value}, columns.comments=, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, totalSize=11603, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp, column.name.delimiter=,, transient_lastDdlTime=1515180525}, baseFileName=srcbucket_tmp, vectorPartitionDesc=null]} 2018-01-05T11:28:46,822 DEBUG [TezTR-518813_1_10_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:46,822 INFO [TezTR-518813_1_10_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:46,822 DEBUG [TezTR-518813_1_10_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:46,823 DEBUG [TezTR-518813_1_10_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:46,823 INFO [TezTR-518813_1_10_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-45_370_3662807892329960653-1/djaiswal/_tez_scratch_dir/64ad8e48-972e-40d6-9b83-c87ea1fa7b37/map.xml 2018-01-05T11:28:46,824 DEBUG [TezTR-518813_1_10_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:46,824 INFO [TezTR-518813_1_10_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_10_00_000000_0 2018-01-05T11:28:46,824 INFO [TezTR-518813_1_10_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:28:46,824 WARN [TezTR-518813_1_10_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket0.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:28:46,824 DEBUG [TezTR-518813_1_10_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:28:46,824 WARN [TezTR-518813_1_10_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket0.txt:0+5702 2018-01-05T11:28:46,825 DEBUG [TezTR-518813_1_10_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket0.txt 2018-01-05T11:28:46,825 INFO [TezTR-518813_1_10_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-45_370_3662807892329960653-1/djaiswal/_tez_scratch_dir/64ad8e48-972e-40d6-9b83-c87ea1fa7b37/map.xml 2018-01-05T11:28:46,825 DEBUG [TezTR-518813_1_10_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:46,825 DEBUG [TezTR-518813_1_10_0_0_0] exec.MapOperator: Processing alias(es) srcbucket_tmp for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket0.txt 2018-01-05T11:28:46,830 INFO [TezTR-518813_1_10_0_0_0] exec.MapOperator: MAP[0]: records read - 1000 2018-01-05T11:28:46,831 DEBUG [TezTR-518813_1_10_0_0_0] log.PerfLogger: 2018-01-05T11:28:46,831 DEBUG [TezTR-518813_1_10_0_0_0] mr.ObjectCache: djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:28:46,831 DEBUG [TezTR-518813_1_10_0_0_0] exec.MapOperator: close called for operator MAP[0] 2018-01-05T11:28:46,831 INFO [TezTR-518813_1_10_0_0_0] exec.MapOperator: Closing operator MAP[0] 2018-01-05T11:28:46,831 INFO [TezTR-518813_1_10_0_0_0] exec.MapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:1000, 2018-01-05T11:28:46,831 DEBUG [TezTR-518813_1_10_0_0_0] exec.MapOperator: Closing child = TS[0] 2018-01-05T11:28:46,831 DEBUG [TezTR-518813_1_10_0_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:28:46,831 DEBUG [TezTR-518813_1_10_0_0_0] exec.TableScanOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:46,831 INFO [TezTR-518813_1_10_0_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:28:46,831 DEBUG [TezTR-518813_1_10_0_0_0] exec.TableScanOperator: Closing child = SEL[1] 2018-01-05T11:28:46,831 DEBUG [TezTR-518813_1_10_0_0_0] exec.SelectOperator: close called for operator SEL[1] 2018-01-05T11:28:46,831 DEBUG [TezTR-518813_1_10_0_0_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:46,831 INFO [TezTR-518813_1_10_0_0_0] exec.SelectOperator: Closing operator SEL[1] 2018-01-05T11:28:46,831 DEBUG [TezTR-518813_1_10_0_0_0] exec.SelectOperator: Closing child = RS[2] 2018-01-05T11:28:46,831 DEBUG [TezTR-518813_1_10_0_0_0] exec.ReduceSinkOperator: close called for operator RS[2] 2018-01-05T11:28:46,831 DEBUG [TezTR-518813_1_10_0_0_0] exec.ReduceSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:46,831 INFO [TezTR-518813_1_10_0_0_0] exec.ReduceSinkOperator: Closing operator RS[2] 2018-01-05T11:28:46,831 INFO [TezTR-518813_1_10_0_0_0] exec.ReduceSinkOperator: RECORDS_OUT_INTERMEDIATE_Map_1:1000, 2018-01-05T11:28:46,831 DEBUG [TezTR-518813_1_10_0_0_0] exec.ReduceSinkOperator: 2 Close done 2018-01-05T11:28:46,831 DEBUG [TezTR-518813_1_10_0_0_0] exec.SelectOperator: 1 Close done 2018-01-05T11:28:46,831 DEBUG [TezTR-518813_1_10_0_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:28:46,831 DEBUG [TezTR-518813_1_10_0_0_0] exec.MapOperator: 0 Close done 2018-01-05T11:28:46,831 INFO [TezTR-518813_1_10_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_10_00_000000_0 2018-01-05T11:28:46,831 INFO [TezTR-518813_1_10_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: Waiting for all spills to complete : Pending : 0 2018-01-05T11:28:46,831 INFO [TezTR-518813_1_10_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: All spills complete 2018-01-05T11:28:46,861 INFO [TezTR-518813_1_10_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: Finished spill 0 2018-01-05T11:28:46,862 DEBUG [TezTR-518813_1_10_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:46,862 INFO [TezTR-518813_1_10_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:46,862 INFO [TezTR-518813_1_10_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_10_00_000000_0, askedToStop=false 2018-01-05T11:28:46,862 INFO [TezTR-518813_1_10_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_10_00_000000_0, stopRequested=false 2018-01-05T11:28:46,862 INFO [TezTR-518813_1_10_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_10_00_000000_0: Counters: 14 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=1000, INPUT_SPLIT_LENGTH_BYTES=11603, OUTPUT_RECORDS=1000, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=12077, OUTPUT_BYTES_WITH_OVERHEAD=26040, OUTPUT_BYTES_PHYSICAL=14145, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=1000, RECORDS_OUT_INTERMEDIATE_Map_1=1000]] 2018-01-05T11:28:46,862 DEBUG [TezTR-518813_1_10_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_10_0_0_0 thread id: 688 scheme: file bytesRead: 11723 bytesWritten: 14153 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:46,863 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_10_00_000000_0, isTaskDone=true 2018-01-05T11:28:46,863 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000014, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_10_00_000000_0, eventCount=4 } 2018-01-05T11:28:46,864 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_10][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_10_00_000000_0, creationTime=1515180526770, allocationTime=1515180526771, startTime=1515180526774, finishTime=1515180526864, timeTaken=90, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:46,864 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:46,864 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_10_00_000000_0 2018-01-05T11:28:46,864 INFO [Task-Executor-3] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_10_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:46,864 INFO [Task-Executor-3] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_10_00_000000_0 2018-01-05T11:28:46,864 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_10_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:46,864 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_10_00_000000_0; false 2018-01-05T11:28:46,864 INFO [Task-Executor-3] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000014=77 2018-01-05T11:28:46,864 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_10_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:46,864 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_10_00_000000_0: true 2018-01-05T11:28:46,864 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_10][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_10_00_000000, startTime=1515180526774, finishTime=1515180526864, timeTaken=90, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_10_00_000000_0, diagnostics=, counters=Counters: 20, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=11723, FILE_BYTES_WRITTEN=14153, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=1000, INPUT_SPLIT_LENGTH_BYTES=11603, OUTPUT_RECORDS=1000, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=12077, OUTPUT_BYTES_WITH_OVERHEAD=26040, OUTPUT_BYTES_PHYSICAL=14145, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=1000, RECORDS_OUT_INTERMEDIATE_Map_1=1000 2018-01-05T11:28:46,864 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:46,864 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_10_00_000000_0 2018-01-05T11:28:46,864 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:46,865 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000014, QueryId=djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9-10, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:28:46,865 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_10_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:46,865 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_10_00_000000_0 2018-01-05T11:28:46,865 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_10_00 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:46,865 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_10_00 [Map 1] 2018-01-05T11:28:46,865 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_10][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_10_00, initRequestedTime=1515180526716, initedTime=1515180526768, startRequestedTime=1515180526729, startedTime=1515180526768, finishTime=1515180526865, timeTaken=97, status=SUCCEEDED, diagnostics=, counters=Counters: 20, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=11723, FILE_BYTES_WRITTEN=14153, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=1000, INPUT_SPLIT_LENGTH_BYTES=11603, OUTPUT_RECORDS=1000, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=12077, OUTPUT_BYTES_WITH_OVERHEAD=26040, OUTPUT_BYTES_PHYSICAL=14145, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=1000, RECORDS_OUT_INTERMEDIATE_Map_1=1000, vertexStats=firstTaskStartTime=1515180526774, firstTasksToStart=[ task_1515180518813_0001_10_00_000000 ], lastTaskFinishTime=1515180526864, lastTasksToFinish=[ task_1515180518813_0001_10_00_000000 ], minTaskDuration=90, maxTaskDuration=90, avgTaskDuration=90.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_10_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_10_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:46,865 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_10_00 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:46,865 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_10][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000014, stoppedTime=1515180526865, exitStatus=0 2018-01-05T11:28:46,865 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000014 exited with diagnostics set to Container completed. 2018-01-05T11:28:46,865 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_10_01 [Reducer 2] attempt: attempt_1515180518813_0001_10_00_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:28:46,865 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_10_00 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3 2018-01-05T11:28:46,865 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null 2018-01-05T11:28:46,865 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000014 2018-01-05T11:28:46,865 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: All source tasks completed. Ramping up 2 remaining tasks for vertex: Reducer 2 2018-01-05T11:28:46,865 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Scheduling 2 tasks for vertex: Reducer 2 with totalTasks: 2. 1 source tasks completed out of 1. MinSourceTaskCompletedFraction: 1.0 min: 0.05 max: 0.75 2018-01-05T11:28:46,866 INFO [App Shared Pool - #0] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Map 1, destinationVertexName=Reducer 2. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.UnorderedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:28:46,866 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_10_01 [Reducer 2] 2018-01-05T11:28:46,866 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_10][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_10_01_000000, scheduledTime=1515180526866, launchTime=1515180526866 2018-01-05T11:28:46,866 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_10][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_10_01_000001, scheduledTime=1515180526866, launchTime=1515180526866 2018-01-05T11:28:46,866 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_10_01 [Reducer 2] 2018-01-05T11:28:46,866 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_10_01_000000_0, priority=20, capability=, hosts=null 2018-01-05T11:28:46,866 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=20,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:46,866 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_10_01_000001_0, priority=20, capability=, hosts=null 2018-01-05T11:28:46,866 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=20,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:46,866 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:46,866 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_10_01_000000_0 on hosts=null 2018-01-05T11:28:46,866 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_10_01_000000_0 on hosts=null 2018-01-05T11:28:46,866 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_10_01_000000_0 2018-01-05T11:28:46,866 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:46,867 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #15, task=TaskInfo{task=attempt_1515180518813_0001_10_01_000000_0, priority=20, startTime=0, containerId=null, uniqueId=14, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000015 2018-01-05T11:28:46,867 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_10_01_000000_0; false 2018-01-05T11:28:46,867 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_10_01_000000_0, priority=20, startTime=311547376, containerId=container_111101111_0001_00_000015assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=14, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:46,867 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=20,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:46,867 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=20,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:46,867 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:46,867 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_10_01_000001_0 on hosts=null 2018-01-05T11:28:46,867 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_10_01_000001_0 on hosts=null 2018-01-05T11:28:46,867 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_10_01_000001_0 2018-01-05T11:28:46,867 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:46,868 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #16, task=TaskInfo{task=attempt_1515180518813_0001_10_01_000001_0, priority=20, startTime=0, containerId=null, uniqueId=15, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, to container=container_111101111_0001_00_000016 2018-01-05T11:28:46,868 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000015 for node: 10.22.8.182:49818 2018-01-05T11:28:46,868 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_10_01_000001_0; false 2018-01-05T11:28:46,868 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_10_01_000001_0, priority=20, startTime=311547376, containerId=container_111101111_0001_00_000016assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=2, ac=6, commF=false, disabled=false}, uniqueId=15, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:46,868 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000015, attempt_1515180518813_0001_10_01_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:46,868 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000015 for node: 10.22.8.182:49818 2018-01-05T11:28:46,868 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000015 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:46,868 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000015, launchTime=1515180526868 2018-01-05T11:28:46,868 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000016 for node: 10.22.8.182:49818 2018-01-05T11:28:46,868 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000016, attempt_1515180518813_0001_10_01_000001_0 for node: 10.22.8.182:49818 2018-01-05T11:28:46,869 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000016 for node: 10.22.8.182:49818 2018-01-05T11:28:46,869 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=10}, sourceName=Map 1, state=S_SUCCEEDED 2018-01-05T11:28:46,870 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_10_01_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000015, dagName=INSERT INTO srcbucket SELECT...srcbucket_tmp (Stage-1), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.UnorderedKVInput,1}}, Outputs={{Reducer 3,org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput,1}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=3, completedTaskCount=1, dagStartTime=1515180526715, firstAttemptStartTime=1515180526866, currentAttemptStartTime=1515180526868} 2018-01-05T11:28:46,870 INFO [IPC Server handler 4 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000015, QueryId=djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9-10, DagId=10, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, SubmitTime=1515180526870 2018-01-05T11:28:46,870 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_10_01_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000015] on NM: [10.22.8.182:49818] 2018-01-05T11:28:46,870 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:46,870 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:46,870 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_10][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_10_01_000000_0, startTime=1515180526869, containerId=container_111101111_0001_00_000015, nodeId=10.22.8.182:49818 2018-01-05T11:28:46,870 DEBUG [IPC Server handler 4 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=10} with the ShuffleHandler 2018-01-05T11:28:46,870 DEBUG [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/10] 2018-01-05T11:28:46,870 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000016 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:46,870 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000016, launchTime=1515180526870 2018-01-05T11:28:46,870 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:46,870 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_10_01_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:46,870 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_10_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180526866, dagStartTime=1515180526715, withinDagPriority=20, vertexParallelism= 2, selfAndUpstreamParallelism= 3, selfAndUpstreamComplete= 1}] 2018-01-05T11:28:46,870 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_10_01_000000_0 : ACCEPTED 2018-01-05T11:28:46,870 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_10_01_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:46,870 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_10_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180526866, dagStartTime=1515180526715, withinDagPriority=20, vertexParallelism= 2, selfAndUpstreamParallelism= 3, selfAndUpstreamComplete= 1} 2018-01-05T11:28:46,870 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_10_01_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:46,870 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_10_01_000000_0 to pre-emption queue 2018-01-05T11:28:46,870 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_10_01_000000_0: true 2018-01-05T11:28:46,870 INFO [TaskCommunicator # 8] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_10_01_000000_0 2018-01-05T11:28:46,870 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: taskOwner hashCode:1706107596 2018-01-05T11:28:46,870 INFO [TaskCommunicator # 8] GuaranteedTasks: Received response for attempt_1515180518813_0001_10_01_000000_0, true 2018-01-05T11:28:46,870 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_10_01_000001_0] submitted. Is using containerId: [container_111101111_0001_00_000016] on NM: [10.22.8.182:49818] 2018-01-05T11:28:46,871 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:46,871 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:46,871 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_10][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_10_01_000001_0, startTime=1515180526870, containerId=container_111101111_0001_00_000016, nodeId=10.22.8.182:49818 2018-01-05T11:28:46,871 INFO [Task-Executor-3] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : INSERT INTO srcbucket SELECT...srcbucket_tmp (Stage-1), VertexName: Reducer 2, VertexParallelism: 2, TaskAttemptID:attempt_1515180518813_0001_10_01_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }}, ], outputSpecList=[{{ destinationVertexName=Reducer 3, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput }}, ] 2018-01-05T11:28:46,871 INFO [Task-Executor-3] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=954728448, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:46,871 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_10_01_000001_0, am_details=10.22.8.182:49824, taskInfo= fragment 1 attempt 0, user=djaiswal, queryId=djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000016, dagName=INSERT INTO srcbucket SELECT...srcbucket_tmp (Stage-1), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.UnorderedKVInput,1}}, Outputs={{Reducer 3,org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput,1}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=3, completedTaskCount=1, dagStartTime=1515180526715, firstAttemptStartTime=1515180526866, currentAttemptStartTime=1515180526869} 2018-01-05T11:28:46,871 INFO [Task-Executor-3] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_10_01_000000_0 vertexName: Reducer 2 2018-01-05T11:28:46,871 INFO [IPC Server handler 3 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000016, QueryId=djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9-10, DagId=10, VertexName=Reducer 2, TaskId=1, TaskAttemptId=0, SubmitTime=1515180526871 2018-01-05T11:28:46,871 DEBUG [IPC Server handler 3 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=10} with the ShuffleHandler 2018-01-05T11:28:46,871 DEBUG [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/10] 2018-01-05T11:28:46,871 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000015, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_10_01_000000_0, eventCount=1 } 2018-01-05T11:28:46,871 DEBUG [TezTR-518813_1_10_1_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_10 queryId: djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9 instanceType: TaskRunner2Callable 2018-01-05T11:28:46,871 INFO [TezTR-518813_1_10_1_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_10_01_000000_0 2018-01-05T11:28:46,871 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=3, runningFragmentCount=1 2018-01-05T11:28:46,871 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_10_01_000001_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:46,871 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_10_01_000001_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180526866, dagStartTime=1515180526715, withinDagPriority=20, vertexParallelism= 2, selfAndUpstreamParallelism= 3, selfAndUpstreamComplete= 1}] 2018-01-05T11:28:46,872 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_10_01_000001_0 : ACCEPTED 2018-01-05T11:28:46,872 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_10_01_000001_0, canFinish=true. Current state: preemptionQueueSize=1, numSlotsAvailable=3, waitQueueSize=1 2018-01-05T11:28:46,872 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_10_01_000001_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180526866, dagStartTime=1515180526715, withinDagPriority=20, vertexParallelism= 2, selfAndUpstreamParallelism= 3, selfAndUpstreamComplete= 1} 2018-01-05T11:28:46,872 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_10_01_000001_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:46,872 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_10_01_000001_0 to pre-emption queue 2018-01-05T11:28:46,872 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_10_01_000001_0: true 2018-01-05T11:28:46,872 DEBUG [TezTR-518813_1_10_1_0_0] log.PerfLogger: 2018-01-05T11:28:46,872 INFO [TaskCommunicator # 9] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_10_01_000001_0 2018-01-05T11:28:46,872 INFO [TaskCommunicator # 9] GuaranteedTasks: Received response for attempt_1515180518813_0001_10_01_000001_0, true 2018-01-05T11:28:46,872 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: taskOwner hashCode:1938286075 2018-01-05T11:28:46,872 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:46,872 INFO [I/O Setup 1 Initialize: {Reducer 3}] writers.UnorderedPartitionedKVWriter: Requested BufferSize (tez.runtime.unordered.output.buffer.size-mb) : 100 2018-01-05T11:28:46,873 INFO [Task-Executor-2] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : INSERT INTO srcbucket SELECT...srcbucket_tmp (Stage-1), VertexName: Reducer 2, VertexParallelism: 2, TaskAttemptID:attempt_1515180518813_0001_10_01_000001_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }}, ], outputSpecList=[{{ destinationVertexName=Reducer 3, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput }}, ] 2018-01-05T11:28:46,873 INFO [Task-Executor-2] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=954728448, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:46,873 INFO [Task-Executor-2] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_10_01_000001_0 vertexName: Reducer 2 2018-01-05T11:28:46,873 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:46,873 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000016, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_10_01_000001_0, eventCount=1 } 2018-01-05T11:28:46,873 DEBUG [TezTR-518813_1_10_1_1_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_10 queryId: djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9 instanceType: TaskRunner2Callable 2018-01-05T11:28:46,873 INFO [TezTR-518813_1_10_1_1_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_10_01_000001_0 2018-01-05T11:28:46,874 DEBUG [TezTR-518813_1_10_1_1_0] log.PerfLogger: 2018-01-05T11:28:46,876 INFO [I/O Setup 1 Initialize: {Reducer 3}] writers.UnorderedPartitionedKVWriter: Requested BufferSize (tez.runtime.unordered.output.buffer.size-mb) : 100 2018-01-05T11:28:46,877 INFO [IPC Server handler 11 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_10_01_000000_0 1 events [0,1) total 1 vertex_1515180518813_0001_10_01 [Reducer 2] 2018-01-05T11:28:46,877 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:46,877 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:28:46,877 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_10_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:28:46,878 DEBUG [TezTR-518813_1_10_1_0_0] log.PerfLogger: 2018-01-05T11:28:46,878 INFO [TezTR-518813_1_10_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:46,878 INFO [TezTR-518813_1_10_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:46,878 INFO [TezTR-518813_1_10_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:46,878 INFO [TezTR-518813_1_10_1_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:46,878 INFO [TezTR-518813_1_10_1_0_0] resources.MemoryDistributor: InitialRequests=[Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.UnorderedKVInput], [Reducer 3:OUTPUT:104857600:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput] 2018-01-05T11:28:46,878 INFO [TezTR-518813_1_10_1_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:46,878 INFO [TezTR-518813_1_10_1_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:46,878 INFO [TezTR-518813_1_10_1_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 236262520, TotalRequestedScaled: 1.1813126E8, TotalJVMHeap: 954728448, TotalAvailable: 639668060, TotalRequested/TotalJVMHeap:0.25 2018-01-05T11:28:46,878 INFO [TezTR-518813_1_10_1_0_0] resources.MemoryDistributor: Allocations=[Map 1:org.apache.tez.runtime.library.input.UnorderedKVInput:INPUT:131404920:131404920], [Reducer 3:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput:OUTPUT:104857600:104857600] 2018-01-05T11:28:46,878 INFO [TezTR-518813_1_10_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:46,878 INFO [TezTR-518813_1_10_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 0 2018-01-05T11:28:46,878 INFO [TezTR-518813_1_10_1_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:46,878 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:46,878 INFO [TezTR-518813_1_10_1_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_10_01_000000_0 2018-01-05T11:28:46,878 DEBUG [TezTR-518813_1_10_1_0_0] log.PerfLogger: 2018-01-05T11:28:46,878 DEBUG [TezTR-518813_1_10_1_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_10_01_000000_0_10043 2018-01-05T11:28:46,878 DEBUG [TezTR-518813_1_10_1_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:46,878 DEBUG [TezTR-518813_1_10_1_0_0] mr.ObjectCache: Creating djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:28:46,879 INFO [TezTR-518813_1_10_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-45_370_3662807892329960653-1/djaiswal/_tez_scratch_dir/2e6c5778-bbd7-443f-9108-b4edbd950582/reduce.xml 2018-01-05T11:28:46,879 DEBUG [TezTR-518813_1_10_1_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-45_370_3662807892329960653-1/djaiswal/_tez_scratch_dir/2e6c5778-bbd7-443f-9108-b4edbd950582/reduce.xml 2018-01-05T11:28:46,879 DEBUG [TezTR-518813_1_10_1_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-45_370_3662807892329960653-1/djaiswal/_tez_scratch_dir/2e6c5778-bbd7-443f-9108-b4edbd950582/reduce.xml 2018-01-05T11:28:46,879 DEBUG [TezTR-518813_1_10_1_0_0] log.PerfLogger: 2018-01-05T11:28:46,879 INFO [TezTR-518813_1_10_1_0_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:28:46,879 DEBUG [TezTR-518813_1_10_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:46,880 INFO [IPC Server handler 10 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_10_01_000001_0 1 events [0,1) total 1 vertex_1515180518813_0001_10_01 [Reducer 2] 2018-01-05T11:28:46,881 DEBUG [TezTR-518813_1_10_1_1_0] log.PerfLogger: 2018-01-05T11:28:46,881 INFO [TezTR-518813_1_10_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:46,881 INFO [TezTR-518813_1_10_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:46,881 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:28:46,881 INFO [TezTR-518813_1_10_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:46,881 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_10_01_000001_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:28:46,881 INFO [TezTR-518813_1_10_1_1_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:46,881 INFO [TezTR-518813_1_10_1_1_0] resources.MemoryDistributor: InitialRequests=[Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.UnorderedKVInput], [Reducer 3:OUTPUT:104857600:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput] 2018-01-05T11:28:46,882 INFO [TezTR-518813_1_10_1_1_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:46,882 INFO [TezTR-518813_1_10_1_1_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:46,882 INFO [TezTR-518813_1_10_1_1_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 236262520, TotalRequestedScaled: 1.1813126E8, TotalJVMHeap: 954728448, TotalAvailable: 639668060, TotalRequested/TotalJVMHeap:0.25 2018-01-05T11:28:46,882 INFO [TezTR-518813_1_10_1_1_0] resources.MemoryDistributor: Allocations=[Map 1:org.apache.tez.runtime.library.input.UnorderedKVInput:INPUT:131404920:131404920], [Reducer 3:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput:OUTPUT:104857600:104857600] 2018-01-05T11:28:46,882 INFO [TezTR-518813_1_10_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:46,882 INFO [TezTR-518813_1_10_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 0 2018-01-05T11:28:46,882 INFO [TezTR-518813_1_10_1_1_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:46,882 INFO [TezTR-518813_1_10_1_1_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_10_01_000001_0 2018-01-05T11:28:46,882 DEBUG [TezTR-518813_1_10_1_1_0] log.PerfLogger: 2018-01-05T11:28:46,882 DEBUG [TezTR-518813_1_10_1_1_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_10_01_000001_0_10046 2018-01-05T11:28:46,882 DEBUG [TezTR-518813_1_10_1_1_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:46,882 DEBUG [TezTR-518813_1_10_1_1_0] mr.ObjectCache: Creating djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:28:46,884 DEBUG [TezTR-518813_1_10_1_0_0] log.PerfLogger: 2018-01-05T11:28:46,884 INFO [TezTR-518813_1_10_1_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 3.17KB 2018-01-05T11:28:46,884 DEBUG [TezTR-518813_1_10_1_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:46,884 DEBUG [TezTR-518813_1_10_1_0_0] log.PerfLogger: 2018-01-05T11:28:46,884 DEBUG [TezTR-518813_1_10_1_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:46,884 DEBUG [TezTR-518813_1_10_1_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:46,884 DEBUG [TezTR-518813_1_10_1_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:46,884 INFO [TezTR-518813_1_10_1_0_0] impl.SimpleFetchedInputAllocator: Map_1: RequestedMemory=131404920, AssignedMemory=131404920, maxSingleShuffleLimit=32851230 2018-01-05T11:28:46,884 INFO [TezTR-518813_1_10_1_1_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-45_370_3662807892329960653-1/djaiswal/_tez_scratch_dir/2e6c5778-bbd7-443f-9108-b4edbd950582/reduce.xml 2018-01-05T11:28:46,884 DEBUG [TezTR-518813_1_10_1_1_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-45_370_3662807892329960653-1/djaiswal/_tez_scratch_dir/2e6c5778-bbd7-443f-9108-b4edbd950582/reduce.xml 2018-01-05T11:28:46,884 DEBUG [TezTR-518813_1_10_1_1_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-45_370_3662807892329960653-1/djaiswal/_tez_scratch_dir/2e6c5778-bbd7-443f-9108-b4edbd950582/reduce.xml 2018-01-05T11:28:46,884 DEBUG [TezTR-518813_1_10_1_1_0] log.PerfLogger: 2018-01-05T11:28:46,884 INFO [TezTR-518813_1_10_1_1_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:28:46,884 DEBUG [TezTR-518813_1_10_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:46,885 DEBUG [TezTR-518813_1_10_1_0_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:46,885 DEBUG [TezTR-518813_1_10_1_0_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:46,885 DEBUG [TezTR-518813_1_10_1_0_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:46,885 DEBUG [TezTR-518813_1_10_1_0_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:46,885 INFO [TezTR-518813_1_10_1_0_0] impl.ShuffleManager: Map_1: numInputs=1, compressionCodec=NoCompressionCodec, numFetchers=1, ifileBufferSize=4096, ifileReadAheadEnabled=true, ifileReadAheadLength=4194304, localDiskFetchEnabled=false, sharedFetchEnabled=false, keepAlive=false, keepAliveMaxConnections=20, connectionTimeout=20000, readTimeout=180000, bufferSize=8192, bufferSize=8192, maxTaskOutputAtOnce=20 2018-01-05T11:28:46,885 INFO [TezTR-518813_1_10_1_0_0] impl.ShuffleInputEventHandlerImpl: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:28:46,885 INFO [TezTR-518813_1_10_1_0_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:28:46,887 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=10&reduce=0&map=attempt_1515180518813_0001_10_00_000000_0_10042 mapId: [attempt_1515180518813_0001_10_00_000000_0_10042] reduceId: [0] jobId: [job_1515180518813_0001] dagId: [10] keepAlive: false 2018-01-05T11:28:46,887 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=10&reduce=0&map=attempt_1515180518813_0001_10_00_000000_0_10042; hash=...hXanyN7S+0i/k 2018-01-05T11:28:46,887 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=10&reduce=0&map=attempt_1515180518813_0001_10_00_000000_0_10042;reply=zDJPBz/NHUOEA 2018-01-05T11:28:46,887 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=10, user='djaiswal', attemptId='attempt_1515180518813_0001_10_00_000000_0_10042'} via loader 2018-01-05T11:28:46,887 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=10, user='djaiswal', attemptId='attempt_1515180518813_0001_10_00_000000_0_10042'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:28:46,887 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_10_00_000000_0_10042 not found 2018-01-05T11:28:46,888 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_10_00_000000_0_10042,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/10/output/attempt_1515180518813_0001_10_00_000000_0_10042/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/10/output/attempt_1515180518813_0001_10_00_000000_0_10042/file.out.index 2018-01-05T11:28:46,888 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Content Length in shuffle : 6981 2018-01-05T11:28:46,888 INFO [Fetcher_B {Map_1} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=10&reduce=0&map=attempt_1515180518813_0001_10_00_000000_0_10042 sent hash and receievd reply 0 ms 2018-01-05T11:28:46,888 INFO [Fetcher_B {Map_1} #0] ShuffleManager.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_10_00_000000_0_10042} to MEMORY, csize=6926, dsize=6922, EndTime=1515180526888, TimeTaken=0, Rate=0.00 MB/s 2018-01-05T11:28:46,888 INFO [Fetcher_B {Map_1} #0] impl.ShuffleManager: All inputs fetched for input vertex : Map 1 2018-01-05T11:28:46,888 INFO [TezTR-518813_1_10_1_0_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:28:46,888 INFO [Fetcher_B {Map_1} #0] impl.ShuffleManager: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.01 MB/s) 2018-01-05T11:28:46,888 INFO [TezTR-518813_1_10_1_0_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:46,888 INFO [ShuffleRunner {Map_1}] impl.ShuffleManager: Map_1: Shutting down FetchScheduler, Was Interrupted: false 2018-01-05T11:28:46,888 DEBUG [TezTR-518813_1_10_1_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:28:46,888 INFO [ShuffleRunner {Map_1}] impl.ShuffleManager: Map_1: Scheduler thread completed 2018-01-05T11:28:46,888 DEBUG [TezTR-518813_1_10_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:46,888 DEBUG [TezTR-518813_1_10_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:46,888 DEBUG [TezTR-518813_1_10_1_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[int, string] 2018-01-05T11:28:46,888 DEBUG [TezTR-518813_1_10_1_0_0] log.PerfLogger: 2018-01-05T11:28:46,888 INFO [TezTR-518813_1_10_1_0_0] exec.SelectOperator: Initializing operator SEL[3] 2018-01-05T11:28:46,889 INFO [TezTR-518813_1_10_1_0_0] exec.SelectOperator: SELECT struct,value:struct<_col0:int,_col1:string>> 2018-01-05T11:28:46,889 DEBUG [TezTR-518813_1_10_1_0_0] exec.SelectOperator: Initialization Done 3 SEL 2018-01-05T11:28:46,889 DEBUG [TezTR-518813_1_10_1_0_0] exec.SelectOperator: Operator 3 SEL initialized 2018-01-05T11:28:46,889 DEBUG [TezTR-518813_1_10_1_0_0] exec.SelectOperator: Initializing children of 3 SEL 2018-01-05T11:28:46,889 DEBUG [TezTR-518813_1_10_1_0_0] exec.FileSinkOperator: Initializing child 4 FS 2018-01-05T11:28:46,889 INFO [TezTR-518813_1_10_1_0_0] exec.FileSinkOperator: Initializing operator FS[4] 2018-01-05T11:28:46,889 DEBUG [TezTR-518813_1_10_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:46,889 INFO [TezTR-518813_1_10_1_0_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@3c9b78e6]:[key, value]:[int, string]] and formatter : org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat@74ad00b3 2018-01-05T11:28:46,889 DEBUG [TezTR-518813_1_10_1_0_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:28:46,889 DEBUG [TezTR-518813_1_10_1_0_0] exec.FileSinkOperator: Initialization Done 4 FS 2018-01-05T11:28:46,889 DEBUG [TezTR-518813_1_10_1_0_0] exec.FileSinkOperator: Operator 4 FS initialized 2018-01-05T11:28:46,889 DEBUG [TezTR-518813_1_10_1_0_0] exec.FileSinkOperator: Initialization Done 4 FS done is reset. 2018-01-05T11:28:46,889 DEBUG [TezTR-518813_1_10_1_0_0] exec.SelectOperator: Initializing child 1 SEL 2018-01-05T11:28:46,889 INFO [TezTR-518813_1_10_1_0_0] exec.SelectOperator: Initializing operator SEL[1] 2018-01-05T11:28:46,889 INFO [TezTR-518813_1_10_1_0_0] exec.SelectOperator: SELECT struct<_col0:int,_col1:string> 2018-01-05T11:28:46,889 DEBUG [TezTR-518813_1_10_1_0_0] exec.SelectOperator: Initialization Done 1 SEL 2018-01-05T11:28:46,889 DEBUG [TezTR-518813_1_10_1_0_0] exec.SelectOperator: Operator 1 SEL initialized 2018-01-05T11:28:46,889 DEBUG [TezTR-518813_1_10_1_0_0] exec.SelectOperator: Initializing children of 1 SEL 2018-01-05T11:28:46,889 DEBUG [TezTR-518813_1_10_1_0_0] exec.GroupByOperator: Initializing child 2 GBY 2018-01-05T11:28:46,889 INFO [TezTR-518813_1_10_1_0_0] exec.GroupByOperator: Initializing operator GBY[2] 2018-01-05T11:28:46,889 DEBUG [TezTR-518813_1_10_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:46,889 DEBUG [TezTR-518813_1_10_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:46,890 INFO [TezTR-518813_1_10_1_0_0] exec.GroupByOperator: Max hash table memory: 93860656 bytes 2018-01-05T11:28:46,891 INFO [TezTR-518813_1_10_1_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:46,891 DEBUG [TezTR-518813_1_10_1_0_0] exec.GroupByOperator: Initialization Done 2 GBY 2018-01-05T11:28:46,891 DEBUG [TezTR-518813_1_10_1_0_0] exec.GroupByOperator: Operator 2 GBY initialized 2018-01-05T11:28:46,891 DEBUG [TezTR-518813_1_10_1_0_0] exec.GroupByOperator: Initializing children of 2 GBY 2018-01-05T11:28:46,891 DEBUG [TezTR-518813_1_10_1_0_0] exec.ReduceSinkOperator: Initializing child 3 RS 2018-01-05T11:28:46,891 INFO [TezTR-518813_1_10_1_0_0] exec.ReduceSinkOperator: Initializing operator RS[3] 2018-01-05T11:28:46,891 DEBUG [TezTR-518813_1_10_1_0_0] exec.ReduceSinkOperator: keys size is 0 2018-01-05T11:28:46,891 INFO [TezTR-518813_1_10_1_0_0] exec.ReduceSinkOperator: Using tag = -1 2018-01-05T11:28:46,891 DEBUG [TezTR-518813_1_10_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:46,891 DEBUG [TezTR-518813_1_10_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:46,891 DEBUG [TezTR-518813_1_10_1_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:28:46,891 DEBUG [TezTR-518813_1_10_1_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS 2018-01-05T11:28:46,891 DEBUG [TezTR-518813_1_10_1_0_0] exec.ReduceSinkOperator: Operator 3 RS initialized 2018-01-05T11:28:46,891 DEBUG [TezTR-518813_1_10_1_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS done is reset. 2018-01-05T11:28:46,891 DEBUG [TezTR-518813_1_10_1_0_0] exec.GroupByOperator: Initialization Done 2 GBY done is reset. 2018-01-05T11:28:46,891 DEBUG [TezTR-518813_1_10_1_0_0] exec.SelectOperator: Initialization Done 1 SEL done is reset. 2018-01-05T11:28:46,891 DEBUG [TezTR-518813_1_10_1_0_0] exec.SelectOperator: Initialization Done 3 SEL done is reset. 2018-01-05T11:28:46,891 INFO [TezTR-518813_1_10_1_0_0] tez.ReduceRecordProcessor: Id =3 Id =4 <\Children> Id = 3 null<\Parent> <\FS> Id =1 Id =2 null <\Children> Id = 1 null<\Parent> <\GBY> <\Children> Id = 3 null<\Parent> <\SEL> <\Children> <\Parent> <\SEL> 2018-01-05T11:28:46,891 INFO [TezTR-518813_1_10_1_0_0] exec.OperatorUtils: Setting output collector: RS[3] --> Reducer 3 2018-01-05T11:28:46,891 DEBUG [TezTR-518813_1_10_1_0_0] log.PerfLogger: 2018-01-05T11:28:46,891 INFO [TezTR-518813_1_10_1_0_0] tez.ReduceRecordProcessor: Starting Output: Reducer 3 2018-01-05T11:28:46,891 INFO [TezTR-518813_1_10_1_0_0] writers.BaseUnorderedPartitionedKVWriter: Instantiating Partitioner: [org.apache.tez.mapreduce.partition.MRPartitioner] 2018-01-05T11:28:46,892 DEBUG [TezTR-518813_1_10_1_1_0] log.PerfLogger: 2018-01-05T11:28:46,892 INFO [TezTR-518813_1_10_1_1_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 3.17KB 2018-01-05T11:28:46,892 DEBUG [TezTR-518813_1_10_1_1_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:46,892 DEBUG [TezTR-518813_1_10_1_1_0] log.PerfLogger: 2018-01-05T11:28:46,892 DEBUG [TezTR-518813_1_10_1_1_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:46,892 DEBUG [TezTR-518813_1_10_1_1_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:46,893 DEBUG [TezTR-518813_1_10_1_1_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:46,893 INFO [TezTR-518813_1_10_1_1_0] impl.SimpleFetchedInputAllocator: Map_1: RequestedMemory=131404920, AssignedMemory=131404920, maxSingleShuffleLimit=32851230 2018-01-05T11:28:46,897 DEBUG [TezTR-518813_1_10_1_1_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:46,897 DEBUG [TezTR-518813_1_10_1_1_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:46,897 DEBUG [TezTR-518813_1_10_1_1_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:46,897 DEBUG [TezTR-518813_1_10_1_1_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:46,897 INFO [TezTR-518813_1_10_1_1_0] impl.ShuffleManager: Map_1: numInputs=1, compressionCodec=NoCompressionCodec, numFetchers=1, ifileBufferSize=4096, ifileReadAheadEnabled=true, ifileReadAheadLength=4194304, localDiskFetchEnabled=false, sharedFetchEnabled=false, keepAlive=false, keepAliveMaxConnections=20, connectionTimeout=20000, readTimeout=180000, bufferSize=8192, bufferSize=8192, maxTaskOutputAtOnce=20 2018-01-05T11:28:46,897 INFO [TezTR-518813_1_10_1_1_0] impl.ShuffleInputEventHandlerImpl: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:28:46,897 INFO [TezTR-518813_1_10_1_1_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:28:46,899 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=10&reduce=1&map=attempt_1515180518813_0001_10_00_000000_0_10042 mapId: [attempt_1515180518813_0001_10_00_000000_0_10042] reduceId: [1] jobId: [job_1515180518813_0001] dagId: [10] keepAlive: false 2018-01-05T11:28:46,899 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=10&reduce=1&map=attempt_1515180518813_0001_10_00_000000_0_10042; hash=.../nT4Eea1vlxr8 2018-01-05T11:28:46,899 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=10&reduce=1&map=attempt_1515180518813_0001_10_00_000000_0_10042;reply=5hwTCXxX4UBwc 2018-01-05T11:28:46,899 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=10, user='djaiswal', attemptId='attempt_1515180518813_0001_10_00_000000_0_10042'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:28:46,899 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache HIT: MapId attempt_1515180518813_0001_10_00_000000_0_10042 found 2018-01-05T11:28:46,899 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_10_00_000000_0_10042,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/10/output/attempt_1515180518813_0001_10_00_000000_0_10042/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/10/output/attempt_1515180518813_0001_10_00_000000_0_10042/file.out.index 2018-01-05T11:28:46,899 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Content Length in shuffle : 7226 2018-01-05T11:28:46,899 INFO [Fetcher_B {Map_1} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=10&reduce=1&map=attempt_1515180518813_0001_10_00_000000_0_10042 sent hash and receievd reply 0 ms 2018-01-05T11:28:46,899 DEBUG [New I/O worker #1] shufflehandler.FadvisedFileRegion: shuffleBufferSize: 131072, path: /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/10/output/attempt_1515180518813_0001_10_00_000000_0_10042/file.out 2018-01-05T11:28:46,899 INFO [Fetcher_B {Map_1} #0] ShuffleManager.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_10_00_000000_0_10042} to MEMORY, csize=7171, dsize=7167, EndTime=1515180526899, TimeTaken=0, Rate=0.00 MB/s 2018-01-05T11:28:46,899 INFO [Fetcher_B {Map_1} #0] impl.ShuffleManager: All inputs fetched for input vertex : Map 1 2018-01-05T11:28:46,899 INFO [TezTR-518813_1_10_1_1_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:28:46,899 INFO [Fetcher_B {Map_1} #0] impl.ShuffleManager: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.01 MB/s) 2018-01-05T11:28:46,899 INFO [TezTR-518813_1_10_1_1_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:46,899 DEBUG [TezTR-518813_1_10_1_1_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:28:46,899 INFO [ShuffleRunner {Map_1}] impl.ShuffleManager: Map_1: Shutting down FetchScheduler, Was Interrupted: false 2018-01-05T11:28:46,899 DEBUG [TezTR-518813_1_10_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:46,900 DEBUG [TezTR-518813_1_10_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:46,899 INFO [ShuffleRunner {Map_1}] impl.ShuffleManager: Map_1: Scheduler thread completed 2018-01-05T11:28:46,900 DEBUG [TezTR-518813_1_10_1_1_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[int, string] 2018-01-05T11:28:46,900 DEBUG [TezTR-518813_1_10_1_1_0] log.PerfLogger: 2018-01-05T11:28:46,900 INFO [TezTR-518813_1_10_1_1_0] exec.SelectOperator: Initializing operator SEL[3] 2018-01-05T11:28:46,900 INFO [TezTR-518813_1_10_1_1_0] exec.SelectOperator: SELECT struct,value:struct<_col0:int,_col1:string>> 2018-01-05T11:28:46,900 DEBUG [TezTR-518813_1_10_1_1_0] exec.SelectOperator: Initialization Done 3 SEL 2018-01-05T11:28:46,900 DEBUG [TezTR-518813_1_10_1_1_0] exec.SelectOperator: Operator 3 SEL initialized 2018-01-05T11:28:46,900 DEBUG [TezTR-518813_1_10_1_1_0] exec.SelectOperator: Initializing children of 3 SEL 2018-01-05T11:28:46,900 DEBUG [TezTR-518813_1_10_1_1_0] exec.FileSinkOperator: Initializing child 4 FS 2018-01-05T11:28:46,900 INFO [TezTR-518813_1_10_1_1_0] exec.FileSinkOperator: Initializing operator FS[4] 2018-01-05T11:28:46,900 DEBUG [TezTR-518813_1_10_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:46,900 INFO [TezTR-518813_1_10_1_1_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@6babbf2c]:[key, value]:[int, string]] and formatter : org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat@621f7628 2018-01-05T11:28:46,900 DEBUG [TezTR-518813_1_10_1_1_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:28:46,900 DEBUG [TezTR-518813_1_10_1_1_0] exec.FileSinkOperator: Initialization Done 4 FS 2018-01-05T11:28:46,900 DEBUG [TezTR-518813_1_10_1_1_0] exec.FileSinkOperator: Operator 4 FS initialized 2018-01-05T11:28:46,900 DEBUG [TezTR-518813_1_10_1_1_0] exec.FileSinkOperator: Initialization Done 4 FS done is reset. 2018-01-05T11:28:46,900 DEBUG [TezTR-518813_1_10_1_1_0] exec.SelectOperator: Initializing child 1 SEL 2018-01-05T11:28:46,900 INFO [TezTR-518813_1_10_1_1_0] exec.SelectOperator: Initializing operator SEL[1] 2018-01-05T11:28:46,900 INFO [TezTR-518813_1_10_1_1_0] exec.SelectOperator: SELECT struct<_col0:int,_col1:string> 2018-01-05T11:28:46,900 DEBUG [TezTR-518813_1_10_1_1_0] exec.SelectOperator: Initialization Done 1 SEL 2018-01-05T11:28:46,900 DEBUG [TezTR-518813_1_10_1_1_0] exec.SelectOperator: Operator 1 SEL initialized 2018-01-05T11:28:46,900 DEBUG [TezTR-518813_1_10_1_1_0] exec.SelectOperator: Initializing children of 1 SEL 2018-01-05T11:28:46,900 DEBUG [TezTR-518813_1_10_1_1_0] exec.GroupByOperator: Initializing child 2 GBY 2018-01-05T11:28:46,900 INFO [TezTR-518813_1_10_1_1_0] exec.GroupByOperator: Initializing operator GBY[2] 2018-01-05T11:28:46,901 DEBUG [TezTR-518813_1_10_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:46,901 DEBUG [TezTR-518813_1_10_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:46,901 INFO [TezTR-518813_1_10_1_1_0] exec.GroupByOperator: Max hash table memory: 93860656 bytes 2018-01-05T11:28:46,901 INFO [TezTR-518813_1_10_1_1_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:46,901 DEBUG [TezTR-518813_1_10_1_1_0] exec.GroupByOperator: Initialization Done 2 GBY 2018-01-05T11:28:46,901 DEBUG [TezTR-518813_1_10_1_1_0] exec.GroupByOperator: Operator 2 GBY initialized 2018-01-05T11:28:46,901 DEBUG [TezTR-518813_1_10_1_1_0] exec.GroupByOperator: Initializing children of 2 GBY 2018-01-05T11:28:46,901 DEBUG [TezTR-518813_1_10_1_1_0] exec.ReduceSinkOperator: Initializing child 3 RS 2018-01-05T11:28:46,901 INFO [TezTR-518813_1_10_1_1_0] exec.ReduceSinkOperator: Initializing operator RS[3] 2018-01-05T11:28:46,901 DEBUG [TezTR-518813_1_10_1_1_0] exec.ReduceSinkOperator: keys size is 0 2018-01-05T11:28:46,901 INFO [TezTR-518813_1_10_1_1_0] exec.ReduceSinkOperator: Using tag = -1 2018-01-05T11:28:46,901 DEBUG [TezTR-518813_1_10_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:46,901 DEBUG [TezTR-518813_1_10_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:46,901 DEBUG [TezTR-518813_1_10_1_1_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:28:46,901 DEBUG [TezTR-518813_1_10_1_1_0] exec.ReduceSinkOperator: Initialization Done 3 RS 2018-01-05T11:28:46,901 DEBUG [TezTR-518813_1_10_1_1_0] exec.ReduceSinkOperator: Operator 3 RS initialized 2018-01-05T11:28:46,901 DEBUG [TezTR-518813_1_10_1_1_0] exec.ReduceSinkOperator: Initialization Done 3 RS done is reset. 2018-01-05T11:28:46,901 DEBUG [TezTR-518813_1_10_1_1_0] exec.GroupByOperator: Initialization Done 2 GBY done is reset. 2018-01-05T11:28:46,901 DEBUG [TezTR-518813_1_10_1_1_0] exec.SelectOperator: Initialization Done 1 SEL done is reset. 2018-01-05T11:28:46,901 DEBUG [TezTR-518813_1_10_1_1_0] exec.SelectOperator: Initialization Done 3 SEL done is reset. 2018-01-05T11:28:46,901 INFO [TezTR-518813_1_10_1_1_0] tez.ReduceRecordProcessor: Id =3 Id =4 <\Children> Id = 3 null<\Parent> <\FS> Id =1 Id =2 null <\Children> Id = 1 null<\Parent> <\GBY> <\Children> Id = 3 null<\Parent> <\SEL> <\Children> <\Parent> <\SEL> 2018-01-05T11:28:46,901 INFO [TezTR-518813_1_10_1_1_0] exec.OperatorUtils: Setting output collector: RS[3] --> Reducer 3 2018-01-05T11:28:46,901 DEBUG [TezTR-518813_1_10_1_1_0] log.PerfLogger: 2018-01-05T11:28:46,901 INFO [TezTR-518813_1_10_1_1_0] tez.ReduceRecordProcessor: Starting Output: Reducer 3 2018-01-05T11:28:46,902 INFO [TezTR-518813_1_10_1_1_0] writers.BaseUnorderedPartitionedKVWriter: Instantiating Partitioner: [org.apache.tez.mapreduce.partition.MRPartitioner] 2018-01-05T11:28:46,955 INFO [TezTR-518813_1_10_1_0_0] writers.UnorderedPartitionedKVWriter: Reducer_3: numBuffers=2, sizePerBuffer=52428800, skipBuffers=true, numPartitions=1, availableMemory=104857600, maxSingleBufferSizeBytes=2147483647, pipelinedShuffle=false, isFinalMergeEnabled=true, numPartitions=1, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:28:46,955 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,955 DEBUG [TezTR-518813_1_10_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:28:46,955 DEBUG [TezTR-518813_1_10_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:28:46,955 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: Status: Running (Executing on YARN cluster with App id application_1515180518813_0001) 2018-01-05T11:28:46,955 DEBUG [TezTR-518813_1_10_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:28:46,955 DEBUG [TezTR-518813_1_10_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:28:46,955 DEBUG [TezTR-518813_1_10_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:28:46,955 DEBUG [TezTR-518813_1_10_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:28:46,955 DEBUG [TezTR-518813_1_10_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:28:46,955 DEBUG [TezTR-518813_1_10_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:28:46,955 DEBUG [TezTR-518813_1_10_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:28:46,955 DEBUG [TezTR-518813_1_10_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:28:46,955 DEBUG [TezTR-518813_1_10_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:28:46,955 DEBUG [TezTR-518813_1_10_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:28:46,955 DEBUG [TezTR-518813_1_10_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:28:46,955 DEBUG [TezTR-518813_1_10_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:28:46,955 DEBUG [TezTR-518813_1_10_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:28:46,956 INFO [TezTR-518813_1_10_1_0_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/.hive-staging_hive_2018-01-05_11-28-45_370_3662807892329960653-1/_tmp.-ext-10000/000000_0 2018-01-05T11:28:46,956 INFO [TezTR-518813_1_10_1_0_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/.hive-staging_hive_2018-01-05_11-28-45_370_3662807892329960653-1/_task_tmp.-ext-10000/_tmp.000000_0 2018-01-05T11:28:46,956 INFO [TezTR-518813_1_10_1_0_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/.hive-staging_hive_2018-01-05_11-28-45_370_3662807892329960653-1/_tmp.-ext-10000/000000_0 2018-01-05T11:28:46,957 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,957 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,957 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:46,957 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 Reducer 2: 0(+2)/2 Reducer 3: 0/1 2018-01-05T11:28:46,968 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000014, QueryId=djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9-10, DagId=10, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-3, Succeeded=true, StartTime=1515180526787, EndTime=1515180526968 2018-01-05T11:28:46,976 INFO [TezTR-518813_1_10_1_1_0] writers.UnorderedPartitionedKVWriter: Reducer_3: numBuffers=2, sizePerBuffer=52428800, skipBuffers=true, numPartitions=1, availableMemory=104857600, maxSingleBufferSizeBytes=2147483647, pipelinedShuffle=false, isFinalMergeEnabled=true, numPartitions=1, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:28:46,977 DEBUG [TezTR-518813_1_10_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:28:46,977 DEBUG [TezTR-518813_1_10_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:28:46,977 DEBUG [TezTR-518813_1_10_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:28:46,977 DEBUG [TezTR-518813_1_10_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:28:46,977 DEBUG [TezTR-518813_1_10_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:28:46,977 DEBUG [TezTR-518813_1_10_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:28:46,977 DEBUG [TezTR-518813_1_10_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:28:46,977 DEBUG [TezTR-518813_1_10_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:28:46,977 DEBUG [TezTR-518813_1_10_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:28:46,977 DEBUG [TezTR-518813_1_10_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:28:46,977 DEBUG [TezTR-518813_1_10_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:28:46,977 DEBUG [TezTR-518813_1_10_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:28:46,977 DEBUG [TezTR-518813_1_10_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:28:46,977 DEBUG [TezTR-518813_1_10_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:28:46,977 DEBUG [TezTR-518813_1_10_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:28:46,977 INFO [TezTR-518813_1_10_1_1_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/.hive-staging_hive_2018-01-05_11-28-45_370_3662807892329960653-1/_tmp.-ext-10000/000001_0 2018-01-05T11:28:46,977 INFO [TezTR-518813_1_10_1_1_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/.hive-staging_hive_2018-01-05_11-28-45_370_3662807892329960653-1/_task_tmp.-ext-10000/_tmp.000001_0 2018-01-05T11:28:46,977 INFO [TezTR-518813_1_10_1_1_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/.hive-staging_hive_2018-01-05_11-28-45_370_3662807892329960653-1/_tmp.-ext-10000/000001_0 2018-01-05T11:28:46,979 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000015, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_10_01_000000_0, eventCount=1 } 2018-01-05T11:28:46,980 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:28:46,983 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000016, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_10_01_000001_0, eventCount=1 } 2018-01-05T11:28:46,984 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:28:46,996 INFO [TezTR-518813_1_10_1_0_0] exec.FileSinkOperator: FS[4]: records written - 1 2018-01-05T11:28:46,997 INFO [TezTR-518813_1_10_1_0_0] exec.FileSinkOperator: FS[4]: records written - 10 2018-01-05T11:28:47,000 INFO [TezTR-518813_1_10_1_1_0] exec.FileSinkOperator: FS[4]: records written - 1 2018-01-05T11:28:47,000 INFO [TezTR-518813_1_10_1_1_0] exec.FileSinkOperator: FS[4]: records written - 10 2018-01-05T11:28:47,000 INFO [TezTR-518813_1_10_1_0_0] exec.FileSinkOperator: FS[4]: records written - 100 2018-01-05T11:28:47,004 INFO [TezTR-518813_1_10_1_1_0] exec.FileSinkOperator: FS[4]: records written - 100 2018-01-05T11:28:47,016 INFO [TezTR-518813_1_10_1_0_0] readers.UnorderedKVReader: Num Records read: 493 2018-01-05T11:28:47,016 DEBUG [TezTR-518813_1_10_1_0_0] log.PerfLogger: 2018-01-05T11:28:47,016 DEBUG [TezTR-518813_1_10_1_0_0] mr.ObjectCache: djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:28:47,016 DEBUG [TezTR-518813_1_10_1_0_0] exec.SelectOperator: close called for operator SEL[3] 2018-01-05T11:28:47,016 INFO [TezTR-518813_1_10_1_0_0] exec.SelectOperator: Closing operator SEL[3] 2018-01-05T11:28:47,016 DEBUG [TezTR-518813_1_10_1_0_0] exec.SelectOperator: Closing child = FS[4] 2018-01-05T11:28:47,016 DEBUG [TezTR-518813_1_10_1_0_0] exec.FileSinkOperator: close called for operator FS[4] 2018-01-05T11:28:47,016 DEBUG [TezTR-518813_1_10_1_0_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:47,016 INFO [TezTR-518813_1_10_1_0_0] exec.FileSinkOperator: Closing operator FS[4] 2018-01-05T11:28:47,016 INFO [TezTR-518813_1_10_1_0_0] exec.FileSinkOperator: FS[4]: records written - 493 2018-01-05T11:28:47,017 DEBUG [TezTR-518813_1_10_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:47,017 DEBUG [TezTR-518813_1_10_1_0_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-45_370_3662807892329960653-1/-mr-10001 2018-01-05T11:28:47,018 INFO [TezTR-518813_1_10_1_1_0] readers.UnorderedKVReader: Num Records read: 507 2018-01-05T11:28:47,018 DEBUG [TezTR-518813_1_10_1_1_0] log.PerfLogger: 2018-01-05T11:28:47,018 DEBUG [TezTR-518813_1_10_1_1_0] mr.ObjectCache: djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:28:47,018 DEBUG [TezTR-518813_1_10_1_1_0] exec.SelectOperator: close called for operator SEL[3] 2018-01-05T11:28:47,018 INFO [TezTR-518813_1_10_1_1_0] exec.SelectOperator: Closing operator SEL[3] 2018-01-05T11:28:47,018 DEBUG [TezTR-518813_1_10_1_1_0] exec.SelectOperator: Closing child = FS[4] 2018-01-05T11:28:47,018 DEBUG [TezTR-518813_1_10_1_1_0] exec.FileSinkOperator: close called for operator FS[4] 2018-01-05T11:28:47,018 DEBUG [TezTR-518813_1_10_1_1_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:47,018 INFO [TezTR-518813_1_10_1_1_0] exec.FileSinkOperator: Closing operator FS[4] 2018-01-05T11:28:47,018 INFO [TezTR-518813_1_10_1_1_0] exec.FileSinkOperator: FS[4]: records written - 507 2018-01-05T11:28:47,019 DEBUG [TezTR-518813_1_10_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:47,019 DEBUG [TezTR-518813_1_10_1_1_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-45_370_3662807892329960653-1/-mr-10001 2018-01-05T11:28:47,039 DEBUG [TezTR-518813_1_10_1_0_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-45_370_3662807892329960653-1/-mr-10001/tmpstats-0 2018-01-05T11:28:47,039 DEBUG [TezTR-518813_1_10_1_0_0] fs.FSStatsPublisher: Writing stats in it : {default.srcbucket/={rawDataSize=5209, numRows=493}} 2018-01-05T11:28:47,039 INFO [TezTR-518813_1_10_1_0_0] exec.FileSinkOperator: RECORDS_OUT_1_default.srcbucket:493, 2018-01-05T11:28:47,039 DEBUG [TezTR-518813_1_10_1_0_0] exec.FileSinkOperator: 4 Close done 2018-01-05T11:28:47,039 DEBUG [TezTR-518813_1_10_1_0_0] exec.SelectOperator: Closing child = SEL[1] 2018-01-05T11:28:47,039 DEBUG [TezTR-518813_1_10_1_0_0] exec.SelectOperator: close called for operator SEL[1] 2018-01-05T11:28:47,039 DEBUG [TezTR-518813_1_10_1_0_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:47,039 INFO [TezTR-518813_1_10_1_0_0] exec.SelectOperator: Closing operator SEL[1] 2018-01-05T11:28:47,039 DEBUG [TezTR-518813_1_10_1_0_0] exec.SelectOperator: Closing child = GBY[2] 2018-01-05T11:28:47,039 DEBUG [TezTR-518813_1_10_1_0_0] exec.GroupByOperator: close called for operator GBY[2] 2018-01-05T11:28:47,039 DEBUG [TezTR-518813_1_10_1_0_0] exec.GroupByOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:47,039 INFO [TezTR-518813_1_10_1_0_0] exec.GroupByOperator: Closing operator GBY[2] 2018-01-05T11:28:47,039 INFO [TezTR-518813_1_10_1_0_0] exec.GroupByOperator: Begin Hash Table flush: size = 1 2018-01-05T11:28:47,040 INFO [TezTR-518813_1_10_1_0_0] exec.ReduceSinkOperator: keys are [] num distributions: 0 2018-01-05T11:28:47,040 WARN [TezTR-518813_1_10_1_0_0] objectinspector.StandardStructObjectInspector: Trying to access 5 fields inside a list of 6 elements: [Long, 0, 498, 0, 48 4c 4c e0 d7 01 d7 01 83 8d 9a 06 c2 ed 9c 02 fd f8 86 07 bf fc 85 05 c4 e3 ca 03 fc c5 eb 17 c1 ab a8 05 83 82 e0 02 bc 87 dd 01 80 d2 03 80 d1 fb 01 81 d0 a7 05 80 97 2d ff 9b 93 16 c4 cd 50 ff a3 9c 04 bd ad 91 0a c1 ac 76 bf e5 3b c0 c0 e1 03 84 8a b9 0c bc ee ce 04 c0 fd 10 c0 92 8a 06 c2 d8 94 07 ff aa 98 01 80 f2 ea 09 c1 a3 9e 05 fe 85 9c 02 81 e7 5e 81 81 8f 09 be f0 8d 03 80 b5 9e 07 80 8e 7f 81 a7 af 01 81 f7 88 03 be bf c0 09 c1 e1 ad 01 82 ba fc 02 bd 95 db 05 80 e9 f0 08 c0 94 8e 0e c0 9c 85 03 83 fc 48 fd 98 a3 07 80 a2 95 03 c2 c2 a0 03 fe c5 d7 05 c1 9b b7 01 ff eb c6 02 c0 d0 96 06 82 bb 19 bf 9f 99 03 bf e8 9f 01 81 bf 3d bf e7 be 05 c1 d9 da 08 bf c2 ce 03 c0 c6 d2 07 c1 ff df 0a 82 fa 6e bd c8 35 80 fc 12 80 ba f7 06 c0 eb 89 04 c4 c1 2f fc 8d 4b 81 85 b0 04 bf e6 50 80 d7 87 07 81 9b 2c c0 8c a6 02 bf 9f a1 0b 80 c3 9f 05 81 be 99 07 80 a7 fc 01 83 b1 8f 07 fd c4 ee 0a c2 c0 9e 01 fd f5 58 c2 ec 98 01 be b0 d3 04 80 87 88 06 c0 c6 ec 08 82 87 45 c0 8d ef 03 be 8a bd 03 81 e3 c4 17 81 a7 93 09 bf 81 f6 02 80 b3 f4 01 c3 b2 6f bc b9 d0 04 80 88 90 05 80 89 83 11 80 f9 ff 05 81 d3 aa 02 c2 d6 83 09 bd af c2 02 80 e1 74 80 b4 a8 01 c3 c1 c9 03 82 b6 cc 03 bb 9c f2 06 c0 f2 f4 03 83 c0 d0 01 fd 8b bb 01 82 d3 42 ff dd e0 02 c1 c3 71 80 df 83 0c c0 d0 90 04 fe 94 b4 12 c0 8d 6a 80 d0 b6 08 c0 a6 e7 0b 80 a0 93 08 c0 88 07 c1 e9 be 01 bf 89 34 82 ab c3 02 bf c8 fd 04 bf 95 cc 06 c1 fa 89 01 c0 9e 23 c2 88 84 01 ff ce ff 05 be fc c4 01 82 89 ca 02 84 c7 fc 01 fb f5 e1 06 ff db a8 02 81 ae f5 03 c4 b2 f0 02 fb a0 cb 09 81 d7 c8 03 bf df 38 c1 fb 97 01 c0 e2 fe 05 80 c5 2b c0 91 ff 01 c1 b9 e2 02 be e5 39 83 f7 b4 01 fd be 65 81 b7 b6 04 80 a4 16 83 dc e9 06 fd fa 99 0d c0 a5 f0 0d ff c9 a1 08 c1 c3 d7 0b bf db bc 04 80 e2 f0 0a c0 ba 21 c0 c0 80 04 80 98 82 01 c1 af 94 0b ff c7 b5 03 82 8e f5 03 ff 8e e1 14 c0 f8 9e 02 c1 ce 77 be e1 9a 05 c0 ca ac 03 c2 9f 9b 03 ff b8 8a 01 81 8a ec 02 be db d0 02 c1 b0 ac 0c 80 d2 9d 08 ff 87 b0 02 80 9a 27 80 b7 cd 01 82 ad e8 04 be fb 1a c0 f5 fd 03 c0 b4 a1 04 83 f3 af 05 fd db d4 01 c0 80 89 03 c0 fd 7b 80 8a e1 0a c1 c7 ce 02 bf a2 dd 03 80 ca d8 02 c0 ad f7 01 c0 b6 8a 03 82 fd d6 03 bf e6 d8 06 bf fd eb 02 c1 85 a5 01 bf b4 d2 03 80 c6 80 15 80 b9 e3 02 85 fd d9 03 bb d5 a2 06 80 be f4 0a c2 9b 5f be 89 c0 17 83 8d c6 02 be da f9 06 bf 96 52 80 97 66 c0 84 a1 03 80 a9 6c c0 8c e4 08 c2 c6 ce 08 ff fd e7 06 c0 8c 94 08 80 e1 82 02 80 c9 f0 03 80 e2 03 bf 97 b5 0a c3 f9 ea 02, null] 2018-01-05T11:28:47,040 WARN [TezTR-518813_1_10_1_0_0] objectinspector.StandardStructObjectInspector: ignoring similar errors. 2018-01-05T11:28:47,040 DEBUG [TezTR-518813_1_10_1_1_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-45_370_3662807892329960653-1/-mr-10001/tmpstats-1 2018-01-05T11:28:47,040 DEBUG [TezTR-518813_1_10_1_1_0] fs.FSStatsPublisher: Writing stats in it : {default.srcbucket/={rawDataSize=5394, numRows=507}} 2018-01-05T11:28:47,040 DEBUG [TezTR-518813_1_10_1_0_0] exec.GroupByOperator: Closing child = RS[3] 2018-01-05T11:28:47,040 DEBUG [TezTR-518813_1_10_1_0_0] exec.ReduceSinkOperator: close called for operator RS[3] 2018-01-05T11:28:47,040 DEBUG [TezTR-518813_1_10_1_0_0] exec.ReduceSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:47,040 INFO [TezTR-518813_1_10_1_0_0] exec.ReduceSinkOperator: Closing operator RS[3] 2018-01-05T11:28:47,040 INFO [TezTR-518813_1_10_1_0_0] exec.ReduceSinkOperator: RECORDS_OUT_INTERMEDIATE_Reducer_2:1, 2018-01-05T11:28:47,040 DEBUG [TezTR-518813_1_10_1_0_0] exec.ReduceSinkOperator: 3 Close done 2018-01-05T11:28:47,040 DEBUG [TezTR-518813_1_10_1_0_0] exec.GroupByOperator: 2 Close done 2018-01-05T11:28:47,040 DEBUG [TezTR-518813_1_10_1_0_0] exec.SelectOperator: 1 Close done 2018-01-05T11:28:47,040 DEBUG [TezTR-518813_1_10_1_0_0] exec.SelectOperator: 3 Close done 2018-01-05T11:28:47,041 INFO [TezTR-518813_1_10_1_1_0] exec.FileSinkOperator: RECORDS_OUT_1_default.srcbucket:507, 2018-01-05T11:28:47,041 DEBUG [TezTR-518813_1_10_1_1_0] exec.FileSinkOperator: 4 Close done 2018-01-05T11:28:47,041 INFO [TezTR-518813_1_10_1_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_10_01_000000_0 2018-01-05T11:28:47,041 INFO [TezTR-518813_1_10_1_0_0] impl.ShuffleInputEventHandlerImpl: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:28:47,041 INFO [TezTR-518813_1_10_1_0_0] impl.ShuffleManager: Shutting down pending fetchers on sourceMap_1: 0 2018-01-05T11:28:47,041 INFO [TezTR-518813_1_10_1_0_0] writers.UnorderedPartitionedKVWriter: Reducer_3: Waiting for all spills to complete : Pending : 0 2018-01-05T11:28:47,041 INFO [TezTR-518813_1_10_1_0_0] writers.UnorderedPartitionedKVWriter: Reducer_3: All spills complete 2018-01-05T11:28:47,041 DEBUG [TezTR-518813_1_10_1_1_0] exec.SelectOperator: Closing child = SEL[1] 2018-01-05T11:28:47,041 DEBUG [TezTR-518813_1_10_1_1_0] exec.SelectOperator: close called for operator SEL[1] 2018-01-05T11:28:47,041 DEBUG [TezTR-518813_1_10_1_1_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:47,041 INFO [TezTR-518813_1_10_1_1_0] exec.SelectOperator: Closing operator SEL[1] 2018-01-05T11:28:47,041 DEBUG [TezTR-518813_1_10_1_1_0] exec.SelectOperator: Closing child = GBY[2] 2018-01-05T11:28:47,041 DEBUG [TezTR-518813_1_10_1_1_0] exec.GroupByOperator: close called for operator GBY[2] 2018-01-05T11:28:47,041 DEBUG [TezTR-518813_1_10_1_1_0] exec.GroupByOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:47,041 INFO [TezTR-518813_1_10_1_1_0] exec.GroupByOperator: Closing operator GBY[2] 2018-01-05T11:28:47,041 INFO [TezTR-518813_1_10_1_1_0] exec.GroupByOperator: Begin Hash Table flush: size = 1 2018-01-05T11:28:47,042 INFO [TezTR-518813_1_10_1_1_0] exec.ReduceSinkOperator: keys are [] num distributions: 0 2018-01-05T11:28:47,042 DEBUG [TezTR-518813_1_10_1_1_0] exec.GroupByOperator: Closing child = RS[3] 2018-01-05T11:28:47,042 DEBUG [TezTR-518813_1_10_1_1_0] exec.ReduceSinkOperator: close called for operator RS[3] 2018-01-05T11:28:47,042 DEBUG [TezTR-518813_1_10_1_1_0] exec.ReduceSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:47,042 INFO [TezTR-518813_1_10_1_1_0] exec.ReduceSinkOperator: Closing operator RS[3] 2018-01-05T11:28:47,042 INFO [TezTR-518813_1_10_1_1_0] exec.ReduceSinkOperator: RECORDS_OUT_INTERMEDIATE_Reducer_2:1, 2018-01-05T11:28:47,042 DEBUG [TezTR-518813_1_10_1_1_0] exec.ReduceSinkOperator: 3 Close done 2018-01-05T11:28:47,042 DEBUG [TezTR-518813_1_10_1_1_0] exec.GroupByOperator: 2 Close done 2018-01-05T11:28:47,042 DEBUG [TezTR-518813_1_10_1_1_0] exec.SelectOperator: 1 Close done 2018-01-05T11:28:47,042 DEBUG [TezTR-518813_1_10_1_1_0] exec.SelectOperator: 3 Close done 2018-01-05T11:28:47,042 INFO [TezTR-518813_1_10_1_1_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_10_01_000001_0 2018-01-05T11:28:47,042 INFO [TezTR-518813_1_10_1_1_0] impl.ShuffleInputEventHandlerImpl: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:28:47,042 INFO [TezTR-518813_1_10_1_1_0] impl.ShuffleManager: Shutting down pending fetchers on sourceMap_1: 0 2018-01-05T11:28:47,042 INFO [TezTR-518813_1_10_1_1_0] writers.UnorderedPartitionedKVWriter: Reducer_3: Waiting for all spills to complete : Pending : 0 2018-01-05T11:28:47,042 INFO [TezTR-518813_1_10_1_1_0] writers.UnorderedPartitionedKVWriter: Reducer_3: All spills complete 2018-01-05T11:28:47,052 DEBUG [TezTR-518813_1_10_1_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:47,052 INFO [TezTR-518813_1_10_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:47,052 INFO [TezTR-518813_1_10_1_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_10_01_000000_0, askedToStop=false 2018-01-05T11:28:47,052 INFO [TezTR-518813_1_10_1_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_10_01_000000_0, stopRequested=false 2018-01-05T11:28:47,052 INFO [TezTR-518813_1_10_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_10_01_000000_0: Counters: 23 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=493, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=2053, OUTPUT_BYTES_WITH_OVERHEAD=2063, OUTPUT_BYTES_PHYSICAL=2091, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=6926, SHUFFLE_BYTES_DECOMPRESSED=6922, SHUFFLE_BYTES_TO_MEM=6926, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=4, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1][HIVE CREATED_FILES=1, RECORDS_OUT_1_default.srcbucket=493, RECORDS_OUT_INTERMEDIATE_Reducer_2=1]] 2018-01-05T11:28:47,053 DEBUG [TezTR-518813_1_10_1_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_10_1_0_0 thread id: 698 scheme: file bytesRead: 0 bytesWritten: 7945 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:47,053 DEBUG [TezTR-518813_1_10_1_1_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:47,053 INFO [TezTR-518813_1_10_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:47,053 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_10_01_000000_0, isTaskDone=true 2018-01-05T11:28:47,054 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000015, requestId=3, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_10_01_000000_0, eventCount=4 } 2018-01-05T11:28:47,054 INFO [TezTR-518813_1_10_1_1_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_10_01_000001_0, askedToStop=false 2018-01-05T11:28:47,054 INFO [TezTR-518813_1_10_1_1_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_10_01_000001_0, stopRequested=false 2018-01-05T11:28:47,054 INFO [TezTR-518813_1_10_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_10_01_000001_0: Counters: 23 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=507, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=2042, OUTPUT_BYTES_WITH_OVERHEAD=2052, OUTPUT_BYTES_PHYSICAL=2080, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=7171, SHUFFLE_BYTES_DECOMPRESSED=7167, SHUFFLE_BYTES_TO_MEM=7171, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][HIVE CREATED_FILES=1, RECORDS_OUT_1_default.srcbucket=507, RECORDS_OUT_INTERMEDIATE_Reducer_2=1]] 2018-01-05T11:28:47,054 DEBUG [TezTR-518813_1_10_1_1_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_10_1_1_0 thread id: 704 scheme: file bytesRead: 0 bytesWritten: 8133 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:47,055 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_10_01_000001_0, isTaskDone=true 2018-01-05T11:28:47,055 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000016, requestId=3, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_10_01_000001_0, eventCount=4 } 2018-01-05T11:28:47,055 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_10][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_10_01_000000_0, creationTime=1515180526866, allocationTime=1515180526868, startTime=1515180526869, finishTime=1515180527055, timeTaken=186, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:47,055 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=3, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:28:47,055 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_10_01_000000_0 2018-01-05T11:28:47,055 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_10_01_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:47,055 INFO [Task-Executor-3] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_10_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:47,056 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_10_01_000000_0; false 2018-01-05T11:28:47,056 INFO [Task-Executor-3] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_10_01_000000_0 2018-01-05T11:28:47,056 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_10_01_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:47,057 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_10][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_10_01_000000, startTime=1515180526869, finishTime=1515180527055, timeTaken=186, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_10_01_000000_0, diagnostics=, counters=Counters: 28, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=7945, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=493, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=2053, OUTPUT_BYTES_WITH_OVERHEAD=2063, OUTPUT_BYTES_PHYSICAL=2091, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=6926, SHUFFLE_BYTES_DECOMPRESSED=6922, SHUFFLE_BYTES_TO_MEM=6926, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=4, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1, HIVE, CREATED_FILES=1, RECORDS_OUT_1_default.srcbucket=493, RECORDS_OUT_INTERMEDIATE_Reducer_2=1 2018-01-05T11:28:47,057 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:47,057 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:47,057 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=3, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:28:47,057 INFO [Task-Executor-2] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_10_01_000001_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:47,057 INFO [Task-Executor-3] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000015=186 2018-01-05T11:28:47,057 INFO [Task-Executor-2] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_10_01_000001_0 2018-01-05T11:28:47,057 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_10_01_000000_0: true 2018-01-05T11:28:47,057 INFO [Task-Executor-2] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000016=185 2018-01-05T11:28:47,057 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_10_01_000000_0 2018-01-05T11:28:47,057 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_10_01_000001_0: true 2018-01-05T11:28:47,057 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_10][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_10_01_000001_0, creationTime=1515180526866, allocationTime=1515180526868, startTime=1515180526870, finishTime=1515180527057, timeTaken=187, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:47,057 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000015, QueryId=djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9-10, Vertex=Reducer 2, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:28:47,057 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_10_01_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=3, preemptionQueueSize=1 2018-01-05T11:28:47,057 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_10_01_000000_0 2018-01-05T11:28:47,058 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000015, QueryId=djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9-10, DagId=10, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-3, Succeeded=true, StartTime=1515180526870, EndTime=1515180527058 2018-01-05T11:28:47,058 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_10_01 [Reducer 2], tasks=2, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:47,058 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_10_01_000001_0 2018-01-05T11:28:47,058 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000016, QueryId=djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9-10, Vertex=Reducer 2, FragmentNum=1, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:28:47,058 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_10_01_000001_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:47,058 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_10_01_000001_0 2018-01-05T11:28:47,058 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_10][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000015, stoppedTime=1515180527058, exitStatus=0 2018-01-05T11:28:47,058 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000015 exited with diagnostics set to Container completed. 2018-01-05T11:28:47,058 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_10_02 [Reducer 3] attempt: attempt_1515180518813_0001_10_01_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:28:47,058 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_10_01_000001_0 2018-01-05T11:28:47,058 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_10_01_000001_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:47,058 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_10_01_000001_0; false 2018-01-05T11:28:47,058 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Scheduling 1 tasks for vertex: Reducer 3 with totalTasks: 1. 1 source tasks completed out of 2. MinSourceTaskCompletedFraction: 0.5 min: 0.05 max: 0.75 2018-01-05T11:28:47,058 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_10_01_000001_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:47,058 INFO [App Shared Pool - #0] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Reducer 2, destinationVertexName=Reducer 3. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.UnorderedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:28:47,058 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_10_02 [Reducer 3] 2018-01-05T11:28:47,058 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:47,058 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:47,058 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_10][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_10_01_000001, startTime=1515180526870, finishTime=1515180527058, timeTaken=188, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_10_01_000001_0, diagnostics=, counters=Counters: 28, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=8133, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=507, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=2042, OUTPUT_BYTES_WITH_OVERHEAD=2052, OUTPUT_BYTES_PHYSICAL=2080, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=7171, SHUFFLE_BYTES_DECOMPRESSED=7167, SHUFFLE_BYTES_TO_MEM=7171, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, HIVE, CREATED_FILES=1, RECORDS_OUT_1_default.srcbucket=507, RECORDS_OUT_INTERMEDIATE_Reducer_2=1 2018-01-05T11:28:47,058 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000015 2018-01-05T11:28:47,058 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_10_01 [Reducer 2], tasks=2, failed=0, killed=0, success=2, completed=2, commits=0, err=null 2018-01-05T11:28:47,058 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_10_01 [Reducer 2] 2018-01-05T11:28:47,059 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_10][Event:VERTEX_FINISHED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_10_01, initRequestedTime=1515180526729, initedTime=1515180526736, startRequestedTime=1515180526768, startedTime=1515180526768, finishTime=1515180527058, timeTaken=290, status=SUCCEEDED, diagnostics=, counters=Counters: 28, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=16078, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=2, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1000, OUTPUT_RECORDS=2, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=4095, OUTPUT_BYTES_WITH_OVERHEAD=4115, OUTPUT_BYTES_PHYSICAL=4171, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=14097, SHUFFLE_BYTES_DECOMPRESSED=14089, SHUFFLE_BYTES_TO_MEM=14097, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, HIVE, CREATED_FILES=2, RECORDS_OUT_1_default.srcbucket=1000, RECORDS_OUT_INTERMEDIATE_Reducer_2=2, vertexStats=firstTaskStartTime=1515180526869, firstTasksToStart=[ task_1515180518813_0001_10_01_000000 ], lastTaskFinishTime=1515180527057, lastTasksToFinish=[ task_1515180518813_0001_10_01_000001 ], minTaskDuration=186, maxTaskDuration=187, avgTaskDuration=186.5, numSuccessfulTasks=2, shortestDurationTasks=[ task_1515180518813_0001_10_01_000000 ], longestDurationTasks=[ task_1515180518813_0001_10_01_000001 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=2, numSucceededTasks=2, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:47,059 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_10_01 [Reducer 2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:47,059 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_10][Event:TASK_STARTED]: vertexName=Reducer 3, taskId=task_1515180518813_0001_10_02_000000, scheduledTime=1515180527059, launchTime=1515180527059 2018-01-05T11:28:47,059 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_10_02 [Reducer 3] attempt: attempt_1515180518813_0001_10_01_000001_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:28:47,059 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_10][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000016, stoppedTime=1515180527059, exitStatus=0 2018-01-05T11:28:47,059 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: All source tasks completed. Ramping up 0 remaining tasks for vertex: Reducer 3 2018-01-05T11:28:47,059 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000016 exited with diagnostics set to Container completed. 2018-01-05T11:28:47,059 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_10_01 [Reducer 2] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3 2018-01-05T11:28:47,059 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null 2018-01-05T11:28:47,059 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_10_02 [Reducer 3] 2018-01-05T11:28:47,059 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000016 2018-01-05T11:28:47,059 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_10_02_000000_0, priority=32, capability=, hosts=null 2018-01-05T11:28:47,059 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=32,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:47,059 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=32,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:47,060 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:47,060 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_10_02_000000_0 on hosts=null 2018-01-05T11:28:47,060 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_10_02_000000_0 on hosts=null 2018-01-05T11:28:47,060 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_10_02_000000_0 2018-01-05T11:28:47,060 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:47,060 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #17, task=TaskInfo{task=attempt_1515180518813_0001_10_02_000000_0, priority=32, startTime=0, containerId=null, uniqueId=16, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000017 2018-01-05T11:28:47,060 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_10_02_000000_0; false 2018-01-05T11:28:47,060 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_10_02_000000_0, priority=32, startTime=311547569, containerId=container_111101111_0001_00_000017assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=16, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:47,060 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000017 for node: 10.22.8.182:49818 2018-01-05T11:28:47,061 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000017, attempt_1515180518813_0001_10_02_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:47,061 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000017 for node: 10.22.8.182:49818 2018-01-05T11:28:47,061 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000017 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:47,061 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000017, launchTime=1515180527061 2018-01-05T11:28:47,061 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=10}, sourceName=Reducer 2, state=S_SUCCEEDED 2018-01-05T11:28:47,062 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_10_02_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000017] on NM: [10.22.8.182:49818] 2018-01-05T11:28:47,062 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:47,062 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:47,062 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_10][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 3, taskAttemptId=attempt_1515180518813_0001_10_02_000000_0, startTime=1515180527061, containerId=container_111101111_0001_00_000017, nodeId=10.22.8.182:49818 2018-01-05T11:28:47,062 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_10_02_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000017, dagName=INSERT INTO srcbucket SELECT...srcbucket_tmp (Stage-1), vertexName=Reducer 3, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Reducer 2,org.apache.tez.runtime.library.input.UnorderedKVInput,2}}, Outputs={{out_Reducer 3,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=4, completedTaskCount=3, dagStartTime=1515180526715, firstAttemptStartTime=1515180527059, currentAttemptStartTime=1515180527061} 2018-01-05T11:28:47,062 INFO [IPC Server handler 0 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000017, QueryId=djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9-10, DagId=10, VertexName=Reducer 3, TaskId=0, TaskAttemptId=0, SubmitTime=1515180527062 2018-01-05T11:28:47,062 DEBUG [IPC Server handler 0 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=10} with the ShuffleHandler 2018-01-05T11:28:47,062 DEBUG [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/10] 2018-01-05T11:28:47,063 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:47,063 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_10_02_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:47,063 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_10_02_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180527059, dagStartTime=1515180526715, withinDagPriority=32, vertexParallelism= 1, selfAndUpstreamParallelism= 4, selfAndUpstreamComplete= 3}] 2018-01-05T11:28:47,063 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_10_02_000000_0 : ACCEPTED 2018-01-05T11:28:47,063 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_10_02_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:47,063 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_10_02_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180527059, dagStartTime=1515180526715, withinDagPriority=32, vertexParallelism= 1, selfAndUpstreamParallelism= 4, selfAndUpstreamComplete= 3} 2018-01-05T11:28:47,063 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_10_02_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:47,063 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_10_02_000000_0 to pre-emption queue 2018-01-05T11:28:47,063 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_10_02_000000_0: true 2018-01-05T11:28:47,063 INFO [TaskCommunicator # 1] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_10_02_000000_0 2018-01-05T11:28:47,063 INFO [TaskCommunicator # 1] GuaranteedTasks: Received response for attempt_1515180518813_0001_10_02_000000_0, true 2018-01-05T11:28:47,063 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: taskOwner hashCode:1706107596 2018-01-05T11:28:47,063 INFO [Task-Executor-2] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : INSERT INTO srcbucket SELECT...srcbucket_tmp (Stage-1), VertexName: Reducer 3, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_10_02_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Reducer 2, physicalEdgeCount=2, inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:28:47,063 INFO [Task-Executor-2] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=954728448, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:47,063 INFO [Task-Executor-2] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_10_02_000000_0 vertexName: Reducer 3 2018-01-05T11:28:47,063 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000017, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_10_02_000000_0, eventCount=1 } 2018-01-05T11:28:47,064 DEBUG [TezTR-518813_1_10_2_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_10 queryId: djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9 instanceType: TaskRunner2Callable 2018-01-05T11:28:47,064 INFO [TezTR-518813_1_10_2_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_10_02_000000_0 2018-01-05T11:28:47,064 DEBUG [TezTR-518813_1_10_2_0_0] log.PerfLogger: 2018-01-05T11:28:47,064 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:47,065 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:47,067 INFO [IPC Server handler 28 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_10_02_000000_0 2 events [0,2) total 2 vertex_1515180518813_0001_10_02 [Reducer 3] 2018-01-05T11:28:47,068 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=2, nextPreRoutedEventId=0, eventCount=2 } 2018-01-05T11:28:47,068 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_10_02_000000_0, eventCount=2 fromEventId=0 nextFromEventId=2 2018-01-05T11:28:47,068 DEBUG [TezTR-518813_1_10_2_0_0] log.PerfLogger: 2018-01-05T11:28:47,068 INFO [TezTR-518813_1_10_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:47,068 INFO [TezTR-518813_1_10_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:47,068 INFO [TezTR-518813_1_10_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:47,069 INFO [I/O Setup 1 Initialize: {out_Reducer 3}] output.MROutput: out_Reducer 3: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:28:47,069 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:47,069 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:47,069 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:47,069 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:47,069 INFO [TezTR-518813_1_10_2_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:47,069 INFO [TezTR-518813_1_10_2_0_0] resources.MemoryDistributor: InitialRequests=[out_Reducer 3:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [Reducer 2:INPUT:131404920:org.apache.tez.runtime.library.input.UnorderedKVInput] 2018-01-05T11:28:47,069 INFO [TezTR-518813_1_10_2_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:47,069 INFO [TezTR-518813_1_10_2_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:47,069 INFO [TezTR-518813_1_10_2_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 131404920, TotalRequestedScaled: 6.570246E7, TotalJVMHeap: 954728448, TotalAvailable: 639668060, TotalRequested/TotalJVMHeap:0.14 2018-01-05T11:28:47,069 INFO [TezTR-518813_1_10_2_0_0] resources.MemoryDistributor: Allocations=[out_Reducer 3:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.input.UnorderedKVInput:INPUT:131404920:131404920] 2018-01-05T11:28:47,069 INFO [TezTR-518813_1_10_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:47,069 INFO [TezTR-518813_1_10_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 0 2018-01-05T11:28:47,069 INFO [TezTR-518813_1_10_2_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:47,070 INFO [TezTR-518813_1_10_2_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_10_02_000000_0 2018-01-05T11:28:47,070 DEBUG [TezTR-518813_1_10_2_0_0] log.PerfLogger: 2018-01-05T11:28:47,070 DEBUG [TezTR-518813_1_10_2_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_10_02_000000_0_10049 2018-01-05T11:28:47,070 DEBUG [TezTR-518813_1_10_2_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:47,070 DEBUG [TezTR-518813_1_10_2_0_0] mr.ObjectCache: Creating djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9_Reducer 3__REDUCE_PLAN__ 2018-01-05T11:28:47,070 INFO [TezTR-518813_1_10_2_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-45_370_3662807892329960653-1/djaiswal/_tez_scratch_dir/81505555-131d-473f-a89a-1c94c886a860/reduce.xml 2018-01-05T11:28:47,070 DEBUG [TezTR-518813_1_10_2_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-45_370_3662807892329960653-1/djaiswal/_tez_scratch_dir/81505555-131d-473f-a89a-1c94c886a860/reduce.xml 2018-01-05T11:28:47,070 DEBUG [TezTR-518813_1_10_2_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-45_370_3662807892329960653-1/djaiswal/_tez_scratch_dir/81505555-131d-473f-a89a-1c94c886a860/reduce.xml 2018-01-05T11:28:47,070 DEBUG [TezTR-518813_1_10_2_0_0] log.PerfLogger: 2018-01-05T11:28:47,070 INFO [TezTR-518813_1_10_2_0_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:28:47,070 DEBUG [TezTR-518813_1_10_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:47,071 DEBUG [TezTR-518813_1_10_2_0_0] log.PerfLogger: 2018-01-05T11:28:47,071 INFO [TezTR-518813_1_10_2_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 3 size: 2.18KB 2018-01-05T11:28:47,071 DEBUG [TezTR-518813_1_10_2_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:47,071 DEBUG [TezTR-518813_1_10_2_0_0] log.PerfLogger: 2018-01-05T11:28:47,071 DEBUG [TezTR-518813_1_10_2_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:47,072 DEBUG [TezTR-518813_1_10_2_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:47,072 DEBUG [TezTR-518813_1_10_2_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:47,072 INFO [TezTR-518813_1_10_2_0_0] impl.SimpleFetchedInputAllocator: Reducer_2: RequestedMemory=131404920, AssignedMemory=131404920, maxSingleShuffleLimit=32851230 2018-01-05T11:28:47,072 INFO [TezTR-518813_1_10_2_0_0] impl.ShuffleManager: Reducer_2: numInputs=2, compressionCodec=NoCompressionCodec, numFetchers=2, ifileBufferSize=4096, ifileReadAheadEnabled=true, ifileReadAheadLength=4194304, localDiskFetchEnabled=false, sharedFetchEnabled=false, keepAlive=false, keepAliveMaxConnections=20, connectionTimeout=20000, readTimeout=180000, bufferSize=8192, bufferSize=8192, maxTaskOutputAtOnce=20 2018-01-05T11:28:47,073 INFO [TezTR-518813_1_10_2_0_0] impl.ShuffleInputEventHandlerImpl: Reducer 2: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:28:47,073 INFO [TezTR-518813_1_10_2_0_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:28:47,074 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=10&reduce=0&map=attempt_1515180518813_0001_10_01_000000_0_10045 mapId: [attempt_1515180518813_0001_10_01_000000_0_10045] reduceId: [0] jobId: [job_1515180518813_0001] dagId: [10] keepAlive: false 2018-01-05T11:28:47,074 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=10&reduce=0&map=attempt_1515180518813_0001_10_01_000000_0_10045; hash=...35C9sDv0WJRTs 2018-01-05T11:28:47,074 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=10&reduce=0&map=attempt_1515180518813_0001_10_01_000000_0_10045;reply=QGy2f+ZF0DTDs 2018-01-05T11:28:47,074 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=10, user='djaiswal', attemptId='attempt_1515180518813_0001_10_01_000000_0_10045'} via loader 2018-01-05T11:28:47,074 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=10&reduce=0&map=attempt_1515180518813_0001_10_01_000001_0_10048 mapId: [attempt_1515180518813_0001_10_01_000001_0_10048] reduceId: [0] jobId: [job_1515180518813_0001] dagId: [10] keepAlive: false 2018-01-05T11:28:47,074 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=10, user='djaiswal', attemptId='attempt_1515180518813_0001_10_01_000000_0_10045'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:28:47,075 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_10_01_000000_0_10045 not found 2018-01-05T11:28:47,075 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=10&reduce=0&map=attempt_1515180518813_0001_10_01_000001_0_10048; hash=...gGjgEI5iC/NRc 2018-01-05T11:28:47,075 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=10&reduce=0&map=attempt_1515180518813_0001_10_01_000001_0_10048;reply=TFWVytFMdMhqQ 2018-01-05T11:28:47,075 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_10_01_000000_0_10045,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/10/output/attempt_1515180518813_0001_10_01_000000_0_10045/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/10/output/attempt_1515180518813_0001_10_01_000000_0_10045/file.out.index 2018-01-05T11:28:47,075 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Content Length in shuffle : 2122 2018-01-05T11:28:47,075 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=10, user='djaiswal', attemptId='attempt_1515180518813_0001_10_01_000001_0_10048'} via loader 2018-01-05T11:28:47,075 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=10, user='djaiswal', attemptId='attempt_1515180518813_0001_10_01_000001_0_10048'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:28:47,075 DEBUG [New I/O worker #2] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_10_01_000001_0_10048 not found 2018-01-05T11:28:47,075 INFO [Fetcher_B {Reducer_2} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=10&reduce=0&map=attempt_1515180518813_0001_10_01_000000_0_10045 sent hash and receievd reply 0 ms 2018-01-05T11:28:47,076 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_10_01_000001_0_10048,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/10/output/attempt_1515180518813_0001_10_01_000001_0_10048/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/10/output/attempt_1515180518813_0001_10_01_000001_0_10048/file.out.index 2018-01-05T11:28:47,076 INFO [Fetcher_B {Reducer_2} #0] ShuffleManager.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_10_01_000000_0_10045} to MEMORY, csize=2067, dsize=2063, EndTime=1515180527076, TimeTaken=0, Rate=0.00 MB/s 2018-01-05T11:28:47,076 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: Content Length in shuffle : 2111 2018-01-05T11:28:47,076 INFO [TezTR-518813_1_10_2_0_0] tez.ReduceRecordProcessor: Main work is Reducer 3 2018-01-05T11:28:47,076 INFO [Fetcher_B {Reducer_2} #0] impl.ShuffleManager: copy(1 (spillsFetched=1) of 2. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:28:47,076 INFO [TezTR-518813_1_10_2_0_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:47,076 DEBUG [TezTR-518813_1_10_2_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:28:47,076 DEBUG [TezTR-518813_1_10_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:47,076 DEBUG [TezTR-518813_1_10_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:47,076 INFO [Fetcher_B {Reducer_2} #1] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=10&reduce=0&map=attempt_1515180518813_0001_10_01_000001_0_10048 sent hash and receievd reply 0 ms 2018-01-05T11:28:47,076 DEBUG [TezTR-518813_1_10_2_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:28:47,076 DEBUG [TezTR-518813_1_10_2_0_0] log.PerfLogger: 2018-01-05T11:28:47,076 INFO [Fetcher_B {Reducer_2} #1] ShuffleManager.fetch: Completed fetch for attempt: {1, 0, attempt_1515180518813_0001_10_01_000001_0_10048} to MEMORY, csize=2056, dsize=2052, EndTime=1515180527076, TimeTaken=0, Rate=0.00 MB/s 2018-01-05T11:28:47,076 INFO [TezTR-518813_1_10_2_0_0] exec.GroupByOperator: Initializing operator GBY[4] 2018-01-05T11:28:47,076 INFO [Fetcher_B {Reducer_2} #1] impl.ShuffleManager: All inputs fetched for input vertex : Reducer 2 2018-01-05T11:28:47,077 INFO [Fetcher_B {Reducer_2} #1] impl.ShuffleManager: copy(2 (spillsFetched=2) of 2. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:28:47,077 INFO [ShuffleRunner {Reducer_2}] impl.ShuffleManager: Reducer_2: Shutting down FetchScheduler, Was Interrupted: false 2018-01-05T11:28:47,077 DEBUG [TezTR-518813_1_10_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:47,077 INFO [ShuffleRunner {Reducer_2}] impl.ShuffleManager: Reducer_2: Scheduler thread completed 2018-01-05T11:28:47,077 DEBUG [TezTR-518813_1_10_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:47,077 INFO [TezTR-518813_1_10_2_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:47,077 DEBUG [TezTR-518813_1_10_2_0_0] exec.GroupByOperator: Initialization Done 4 GBY 2018-01-05T11:28:47,077 DEBUG [TezTR-518813_1_10_2_0_0] exec.GroupByOperator: Operator 4 GBY initialized 2018-01-05T11:28:47,077 DEBUG [TezTR-518813_1_10_2_0_0] exec.GroupByOperator: Initializing children of 4 GBY 2018-01-05T11:28:47,077 DEBUG [TezTR-518813_1_10_2_0_0] exec.FileSinkOperator: Initializing child 6 FS 2018-01-05T11:28:47,077 INFO [TezTR-518813_1_10_2_0_0] exec.FileSinkOperator: Initializing operator FS[6] 2018-01-05T11:28:47,077 DEBUG [TezTR-518813_1_10_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:47,077 INFO [TezTR-518813_1_10_2_0_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@7bd1754f]:[_col0, _col1]:[struct, struct]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@6ef2766 2018-01-05T11:28:47,077 DEBUG [TezTR-518813_1_10_2_0_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:28:47,077 DEBUG [TezTR-518813_1_10_2_0_0] exec.FileSinkOperator: Initialization Done 6 FS 2018-01-05T11:28:47,077 DEBUG [TezTR-518813_1_10_2_0_0] exec.FileSinkOperator: Operator 6 FS initialized 2018-01-05T11:28:47,077 DEBUG [TezTR-518813_1_10_2_0_0] exec.FileSinkOperator: Initialization Done 6 FS done is reset. 2018-01-05T11:28:47,077 DEBUG [TezTR-518813_1_10_2_0_0] exec.GroupByOperator: Initialization Done 4 GBY done is reset. 2018-01-05T11:28:47,077 INFO [TezTR-518813_1_10_2_0_0] tez.ReduceRecordProcessor: Id =4 Id =6 <\Children> Id = 4 null<\Parent> <\FS> <\Children> <\Parent> <\GBY> 2018-01-05T11:28:47,077 DEBUG [TezTR-518813_1_10_2_0_0] log.PerfLogger: 2018-01-05T11:28:47,077 INFO [TezTR-518813_1_10_2_0_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 3 2018-01-05T11:28:47,078 DEBUG [TezTR-518813_1_10_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:28:47,078 DEBUG [TezTR-518813_1_10_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:28:47,078 DEBUG [TezTR-518813_1_10_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:28:47,078 DEBUG [TezTR-518813_1_10_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:28:47,078 DEBUG [TezTR-518813_1_10_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:28:47,078 DEBUG [TezTR-518813_1_10_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:28:47,079 INFO [TezTR-518813_1_10_2_0_0] readers.UnorderedKVReader: Num Records read: 2 2018-01-05T11:28:47,079 DEBUG [TezTR-518813_1_10_2_0_0] log.PerfLogger: 2018-01-05T11:28:47,079 DEBUG [TezTR-518813_1_10_2_0_0] mr.ObjectCache: djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9_Reducer 3__REDUCE_PLAN__ no longer needed 2018-01-05T11:28:47,079 DEBUG [TezTR-518813_1_10_2_0_0] exec.GroupByOperator: close called for operator GBY[4] 2018-01-05T11:28:47,079 INFO [TezTR-518813_1_10_2_0_0] exec.GroupByOperator: Closing operator GBY[4] 2018-01-05T11:28:47,079 INFO [TezTR-518813_1_10_2_0_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-46_462_6234719662926073401-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-46_462_6234719662926073401-1/_tmp.-ext-10001/000000_0 2018-01-05T11:28:47,079 INFO [TezTR-518813_1_10_2_0_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-46_462_6234719662926073401-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-46_462_6234719662926073401-1/_task_tmp.-ext-10001/_tmp.000000_0 2018-01-05T11:28:47,079 INFO [TezTR-518813_1_10_2_0_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-46_462_6234719662926073401-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-46_462_6234719662926073401-1/_tmp.-ext-10001/000000_0 2018-01-05T11:28:47,113 INFO [TezTR-518813_1_10_2_0_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:28:47,113 DEBUG [TezTR-518813_1_10_2_0_0] exec.GroupByOperator: Closing child = FS[6] 2018-01-05T11:28:47,113 DEBUG [TezTR-518813_1_10_2_0_0] exec.FileSinkOperator: close called for operator FS[6] 2018-01-05T11:28:47,113 DEBUG [TezTR-518813_1_10_2_0_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:47,113 INFO [TezTR-518813_1_10_2_0_0] exec.FileSinkOperator: Closing operator FS[6] 2018-01-05T11:28:47,113 INFO [TezTR-518813_1_10_2_0_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:28:47,113 INFO [TezTR-518813_1_10_2_0_0] exec.FileSinkOperator: RECORDS_OUT_0:1, 2018-01-05T11:28:47,113 DEBUG [TezTR-518813_1_10_2_0_0] exec.FileSinkOperator: 6 Close done 2018-01-05T11:28:47,113 DEBUG [TezTR-518813_1_10_2_0_0] exec.GroupByOperator: 4 Close done 2018-01-05T11:28:47,113 INFO [TezTR-518813_1_10_2_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_10_02_000000_0 2018-01-05T11:28:47,113 INFO [TezTR-518813_1_10_2_0_0] impl.ShuffleInputEventHandlerImpl: Reducer 2: numDmeEventsSeen=2, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:28:47,113 INFO [TezTR-518813_1_10_2_0_0] impl.ShuffleManager: Shutting down pending fetchers on sourceReducer_2: 0 2018-01-05T11:28:47,113 INFO [TezTR-518813_1_10_2_0_0] output.MROutput: out_Reducer 3 closed 2018-01-05T11:28:47,113 DEBUG [TezTR-518813_1_10_2_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:47,113 INFO [TezTR-518813_1_10_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:47,114 INFO [TezTR-518813_1_10_2_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_10_02_000000_0, askedToStop=false 2018-01-05T11:28:47,114 INFO [TezTR-518813_1_10_2_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_10_02_000000_0, stopRequested=false 2018-01-05T11:28:47,114 INFO [TezTR-518813_1_10_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_10_02_000000_0: Counters: 14 [[org.apache.tez.common.counters.TaskCounter NUM_SHUFFLED_INPUTS=2, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=2, OUTPUT_RECORDS=0, SHUFFLE_BYTES=4123, SHUFFLE_BYTES_DECOMPRESSED=4115, SHUFFLE_BYTES_TO_MEM=4123, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=5, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1][HIVE CREATED_FILES=1, RECORDS_OUT_0=1]] 2018-01-05T11:28:47,114 DEBUG [TezTR-518813_1_10_2_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_10_2_0_0 thread id: 731 scheme: file bytesRead: 0 bytesWritten: 4381 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:47,115 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_10_02_000000_0, isTaskDone=true 2018-01-05T11:28:47,115 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000017, requestId=2, startIndex=2, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_10_02_000000_0, eventCount=2 } 2018-01-05T11:28:47,116 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_10][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 3, taskAttemptId=attempt_1515180518813_0001_10_02_000000_0, creationTime=1515180527059, allocationTime=1515180527061, startTime=1515180527061, finishTime=1515180527116, timeTaken=55, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:47,117 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_10_02_000000_0 2018-01-05T11:28:47,117 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_10_02_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:47,117 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_10_02_000000_0; false 2018-01-05T11:28:47,117 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_10_02_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:47,117 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:47,117 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=2, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:28:47,117 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:47,117 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_10][Event:TASK_FINISHED]: vertexName=Reducer 3, taskId=task_1515180518813_0001_10_02_000000, startTime=1515180527061, finishTime=1515180527117, timeTaken=56, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_10_02_000000_0, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=4381, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, NUM_SHUFFLED_INPUTS=2, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=2, OUTPUT_RECORDS=0, SHUFFLE_BYTES=4123, SHUFFLE_BYTES_DECOMPRESSED=4115, SHUFFLE_BYTES_TO_MEM=4123, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=5, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1 2018-01-05T11:28:47,117 INFO [Task-Executor-2] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_10_02_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:47,117 INFO [Task-Executor-2] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_10_02_000000_0 2018-01-05T11:28:47,118 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_10_02 [Reducer 3], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:47,118 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_10_02 [Reducer 3] 2018-01-05T11:28:47,118 INFO [Task-Executor-2] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000017=55 2018-01-05T11:28:47,118 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_10_02_000000_0: true 2018-01-05T11:28:47,118 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_10][Event:VERTEX_FINISHED]: vertexName=Reducer 3, vertexId=vertex_1515180518813_0001_10_02, initRequestedTime=1515180526736, initedTime=1515180526742, startRequestedTime=1515180526769, startedTime=1515180526769, finishTime=1515180527118, timeTaken=349, status=SUCCEEDED, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=4381, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, NUM_SHUFFLED_INPUTS=2, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=2, OUTPUT_RECORDS=0, SHUFFLE_BYTES=4123, SHUFFLE_BYTES_DECOMPRESSED=4115, SHUFFLE_BYTES_TO_MEM=4123, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=5, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1, vertexStats=firstTaskStartTime=1515180527061, firstTasksToStart=[ task_1515180518813_0001_10_02_000000 ], lastTaskFinishTime=1515180527116, lastTasksToFinish=[ task_1515180518813_0001_10_02_000000 ], minTaskDuration=55, maxTaskDuration=55, avgTaskDuration=55.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_10_02_000000 ], longestDurationTasks=[ task_1515180518813_0001_10_02_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:47,118 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_10_02 [Reducer 3] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:47,119 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_10][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000017, stoppedTime=1515180527118, exitStatus=0 2018-01-05T11:28:47,119 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000017 exited with diagnostics set to Container completed. 2018-01-05T11:28:47,119 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_10_02 [Reducer 3] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3 2018-01-05T11:28:47,119 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null 2018-01-05T11:28:47,119 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_10_01 [Reducer 2] 2018-01-05T11:28:47,119 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_10_02 [Reducer 3] 2018-01-05T11:28:47,119 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_10_00 [Map 1] 2018-01-05T11:28:47,120 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_10, queueSize=0 2018-01-05T11:28:47,120 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_10][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_10, startTime=1515180526715, finishTime=1515180527119, timeTaken=404, status=SUCCEEDED, diagnostics=, counters=Counters: 38, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=4, TOTAL_LAUNCHED_TASKS=4, OTHER_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=1234, AM_GC_TIME_MILLIS=52, File System Counters, FILE_BYTES_READ=11723, FILE_BYTES_WRITTEN=34612, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=4, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=2002, INPUT_SPLIT_LENGTH_BYTES=11603, OUTPUT_RECORDS=1002, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=16172, OUTPUT_BYTES_WITH_OVERHEAD=30155, OUTPUT_BYTES_PHYSICAL=18316, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=18220, SHUFFLE_BYTES_DECOMPRESSED=18204, SHUFFLE_BYTES_TO_MEM=18220, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, HIVE, CREATED_FILES=3, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=1000, RECORDS_OUT_0=1, RECORDS_OUT_1_default.srcbucket=1000, RECORDS_OUT_INTERMEDIATE_Map_1=1000, RECORDS_OUT_INTERMEDIATE_Reducer_2=2 2018-01-05T11:28:47,120 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_10 finished with state: SUCCEEDED 2018-01-05T11:28:47,120 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_10 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:28:47,120 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000017 2018-01-05T11:28:47,120 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_10, dagState=SUCCEEDED 2018-01-05T11:28:47,120 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_10_post 2018-01-05T11:28:47,120 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:47,120 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:28:47,120 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:28:47,120 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 10 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=4, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=4,NumRequestsWithoutLocation=4, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=4} 2018-01-05T11:28:47,120 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:28:47,120 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=INSERT INTO srcbucket SELECT...srcbucket_tmp (Stage-1), with id=dag_1515180518813_0001_10 2018-01-05T11:28:47,120 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 10, to 10.22.8.182:49818 2018-01-05T11:28:47,120 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=4, Remaining=0 2018-01-05T11:28:47,120 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=INSERT INTO srcbucket SELECT...srcbucket_tmp (Stage-1), with id=dag_1515180518813_0001_10 2018-01-05T11:28:47,121 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=10} 2018-01-05T11:28:47,121 INFO [IPC Server handler 2 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=10}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:28:47,121 INFO [IPC Server handler 2 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/10 after 300 seconds 2018-01-05T11:28:47,121 ERROR [IPC Server handler 2 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9, dag_1515180518813_0001_10 2018-01-05T11:28:47,121 INFO [IPC Server handler 2 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9 2018-01-05T11:28:47,121 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=10} = 1 2018-01-05T11:28:47,121 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Issuing killFragment for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=10} attempt_1515180518813_0001_10_02_000000_0 2018-01-05T11:28:47,121 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Removing attempt_1515180518813_0001_10_02_000000_0 from preemptionQueue 2018-01-05T11:28:47,121 INFO [IPC Server handler 2 on 49818] impl.TaskRunnerCallable: Ignoring kill request for task attempt_1515180518813_0001_10_02_000000_0 since it's already complete 2018-01-05T11:28:47,121 DEBUG [IPC Server handler 2 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=10} 2018-01-05T11:28:47,121 DEBUG [IPC Server handler 2 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:28:47,121 DEBUG [IPC Server handler 2 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=10} 2018-01-05T11:28:47,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,122 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 Reducer 2: 2/2 Reducer 3: 1/1 2018-01-05T11:28:47,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,124 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,124 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: TaskId for 000000_0 = 000000 2018-01-05T11:28:47,124 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: TaskId for 000001_0 = 000001 2018-01-05T11:28:47,124 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,124 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,124 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: TaskId for 000000_0 = 000000 2018-01-05T11:28:47,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,126 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-2:DEPENDENCY_COLLECTION] in serial mode 2018-01-05T11:28:47,126 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MOVE] in serial mode 2018-01-05T11:28:47,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@16af0251, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:47,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:47,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,127 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:47,127 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:47,127 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:47,127 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:47,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:47,127 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Loading data to table default.srcbucket from file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/.hive-staging_hive_2018-01-05_11-28-45_370_3662807892329960653-1/-ext-10000 2018-01-05T11:28:47,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,128 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:47,128 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:47,128 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:47,128 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:47,128 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,128 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,128 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,129 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,129 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,129 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:47,129 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:47,129 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,129 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:47,129 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,130 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket 2018-01-05T11:28:47,130 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket 2018-01-05T11:28:47,134 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,134 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,134 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,134 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket 2018-01-05T11:28:47,134 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket 2018-01-05T11:28:47,138 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,138 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,138 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] FileOperations: moving file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/.hive-staging_hive_2018-01-05_11-28-45_370_3662807892329960653-1/-ext-10000 to file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket (replace = KEEP_EXISTING) 2018-01-05T11:28:47,139 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:47,139 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:47,139 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:47,139 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:47,140 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Moved src: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/.hive-staging_hive_2018-01-05_11-28-45_370_3662807892329960653-1/-ext-10000/000000_0, to dest: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/000000_0 2018-01-05T11:28:47,140 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Moved src: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/.hive-staging_hive_2018-01-05_11-28-45_370_3662807892329960653-1/-ext-10000/000001_0, to dest: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/000001_0 2018-01-05T11:28:47,140 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,140 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=srcbucket newtbl=srcbucket 2018-01-05T11:28:47,140 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=srcbucket newtbl=srcbucket 2018-01-05T11:28:47,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,153 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,153 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,155 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-3:STATS] in serial mode 2018-01-05T11:28:47,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@377d66f, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:47,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:47,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,155 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:47,155 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:47,155 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:47,155 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:47,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:47,155 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:47,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-45_370_3662807892329960653-1/-mr-10001 2018-01-05T11:28:47,155 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-45_370_3662807892329960653-1/-mr-10001 2018-01-05T11:28:47,156 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,157 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:47,157 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:47,157 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:47,157 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:47,157 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,157 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,157 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,158 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,158 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,158 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:47,158 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:47,158 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,158 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:47,158 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,158 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket 2018-01-05T11:28:47,158 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket 2018-01-05T11:28:47,159 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000016, QueryId=djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9-10, DagId=10, VertexName=Reducer 2, TaskId=1, TaskAttemptId=0, ThreadName=Task-Executor-2, Succeeded=true, StartTime=1515180526872, EndTime=1515180527159 2018-01-05T11:28:47,159 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_10_02_000000_0 2018-01-05T11:28:47,159 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_10_02_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:47,159 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_10_02_000000_0 2018-01-05T11:28:47,159 INFO [ExecutionCompletionThread #0] impl.QueryTracker: Ignoring fragmentComplete message for unknown query: QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=10} 2018-01-05T11:28:47,162 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,162 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.srcbucket/ numRows 2018-01-05T11:28:47,163 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.srcbucket/ numRows 1000 2018-01-05T11:28:47,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.srcbucket/ rawDataSize 2018-01-05T11:28:47,163 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.srcbucket/ rawDataSize 10603 2018-01-05T11:28:47,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,163 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=srcbucket newtbl=srcbucket 2018-01-05T11:28:47,163 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=srcbucket newtbl=srcbucket 2018-01-05T11:28:47,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,172 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,173 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,173 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.srcbucket stats: [numFiles=2, numRows=1000, totalSize=11603, rawDataSize=10603] 2018-01-05T11:28:47,173 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: About to delete stats tmp dir :file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-45_370_3662807892329960653-1/-mr-10001 2018-01-05T11:28:47,188 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Time taken to get FileStatuses: 13 2018-01-05T11:28:47,188 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:28:47,188 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 14 2018-01-05T11:28:47,188 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FetchOperator: Creating fetchTask with deserializer typeinfo: struct<_col0:struct,_col1:struct> 2018-01-05T11:28:47,188 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FetchOperator: deserializer properties: table properties: {columns=_col0,_col1, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct:struct, escape.delim=\} partition properties: {columns=_col0,_col1, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct:struct, escape.delim=\} 2018-01-05T11:28:47,191 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,194 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.212952ms + 0.004873ms, the query is [select "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "TAB_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (...)] 2018-01-05T11:28:47,194 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,198 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,202 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: write_column_statistics: db=default table=srcbucket 2018-01-05T11:28:47,202 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=write_column_statistics: db=default table=srcbucket 2018-01-05T11:28:47,206 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=srcbucket tableName=default colName=key 2018-01-05T11:28:47,206 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=srcbucket tableName=default colName=value 2018-01-05T11:28:47,208 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,209 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,209 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,209 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,209 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: INSERT INTO srcbucket SELECT * FROM srcbucket_tmp 2018-01-05T11:28:47,209 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:28:47,209 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@srcbucket_tmp 2018-01-05T11:28:47,209 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@srcbucket 2018-01-05T11:28:47,209 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Lineage: srcbucket.key SIMPLE [(srcbucket_tmp)srcbucket_tmp.FieldSchema(name:key, type:int, comment:null), ] 2018-01-05T11:28:47,209 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Lineage: srcbucket.value SIMPLE [(srcbucket_tmp)srcbucket_tmp.FieldSchema(name:value, type:string, comment:null), ] 2018-01-05T11:28:47,209 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,209 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,209 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:47,209 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=10, getTable_(String, String, )=4, setPartitionColumnStatistics_(SetPartitionsStatsRequest, )=18} 2018-01-05T11:28:47,209 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9); Time taken: 0.663 seconds 2018-01-05T11:28:47,209 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:47,209 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,209 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:47,209 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,209 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query INSERT INTO srcbucket SELECT * FROM srcbucket_tmp 2018-01-05T11:28:47,209 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/.hive-staging_hive_2018-01-05_11-28-45_370_3662807892329960653-1 2018-01-05T11:28:47,209 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-45_370_3662807892329960653-1 2018-01-05T11:28:47,210 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 1.839 seconds 2018-01-05T11:28:47,210 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:47,210 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:47,210 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:47,210 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:47,210 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,210 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:47,210 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,210 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,210 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,210 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: DROP TABLE srcbucket_tmp 2018-01-05T11:28:47,210 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112847_da865a75-0de9-436d-ab4c-16d8c0d1b305): DROP TABLE srcbucket_tmp 2018-01-05T11:28:47,210 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,210 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: DROP TABLE srcbucket_tmp 2018-01-05T11:28:47,211 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:47,211 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,211 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,211 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,211 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,211 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:47,211 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,211 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:47,211 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:47,214 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,215 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,215 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:47,215 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,215 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:47,215 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,215 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:47,215 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=4, flushCache_()=0} 2018-01-05T11:28:47,215 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112847_da865a75-0de9-436d-ab4c-16d8c0d1b305); Time taken: 0.005 seconds 2018-01-05T11:28:47,215 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:47,215 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,215 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112847_da865a75-0de9-436d-ab4c-16d8c0d1b305): DROP TABLE srcbucket_tmp 2018-01-05T11:28:47,217 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,217 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: DROP TABLE srcbucket_tmp 2018-01-05T11:28:47,217 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:47,217 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@srcbucket_tmp 2018-01-05T11:28:47,217 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@srcbucket_tmp 2018-01-05T11:28:47,217 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,217 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,217 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,217 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,217 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:47,217 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,217 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:47,217 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:47,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,222 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:47,222 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:47,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,225 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: drop_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:47,225 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=drop_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:47,262 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000017, QueryId=djaiswal_20180105112845_ee65f7e2-6df7-4d4b-8425-e5374d126fa9-10, DagId=10, VertexName=Reducer 3, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-2, Succeeded=true, StartTime=1515180527063, EndTime=1515180527262 2018-01-05T11:28:47,284 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit, isolation) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,307 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 7.454382ms + 0.006007ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc] 2018-01-05T11:28:47,356 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit, isolation) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,385 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit, isolation) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,424 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit, isolation) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,532 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,533 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.FileUtils: deleting file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp 2018-01-05T11:28:47,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@srcbucket_tmp) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@srcbucket_tmp) Type=TABLE WriteType=DDL_EXCLUSIVE is present 2018-01-05T11:28:47,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,542 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: DROP TABLE srcbucket_tmp 2018-01-05T11:28:47,542 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:47,542 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@srcbucket_tmp 2018-01-05T11:28:47,542 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@srcbucket_tmp 2018-01-05T11:28:47,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,542 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:47,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=320, getTable_(String, String, )=5} 2018-01-05T11:28:47,542 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112847_da865a75-0de9-436d-ab4c-16d8c0d1b305); Time taken: 0.327 seconds 2018-01-05T11:28:47,542 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:47,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,542 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:47,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query DROP TABLE srcbucket_tmp 2018-01-05T11:28:47,542 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.332 seconds 2018-01-05T11:28:47,542 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:47,542 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:47,542 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:47,542 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:47,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:47,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,543 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,543 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: ANALYZE TABLE srcbucket COMPUTE STATISTICS 2018-01-05T11:28:47,543 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112847_e4400b34-36d1-436b-a754-3cde7cdc4901): ANALYZE TABLE srcbucket COMPUTE STATISTICS 2018-01-05T11:28:47,543 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,543 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: ANALYZE TABLE srcbucket COMPUTE STATISTICS 2018-01-05T11:28:47,543 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:47,543 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,543 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,543 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,543 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,543 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:47,543 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Invoking analyze on original query 2018-01-05T11:28:47,543 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Starting Semantic Analysis 2018-01-05T11:28:47,544 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed phase 1 of Semantic Analysis 2018-01-05T11:28:47,544 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for source tables 2018-01-05T11:28:47,544 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,544 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket 2018-01-05T11:28:47,544 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket 2018-01-05T11:28:47,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,556 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket 2018-01-05T11:28:47,556 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket 2018-01-05T11:28:47,563 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,563 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,564 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for subqueries 2018-01-05T11:28:47,564 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for destination tables 2018-01-05T11:28:47,564 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed getting MetaData in Semantic Analysis 2018-01-05T11:28:47,585 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_543_7350708723037104955-1/-mr-10000 2018-01-05T11:28:47,586 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Table Plan for srcbucket TS[0] 2018-01-05T11:28:47,586 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Body Plan for Query Block null 2018-01-05T11:28:47,586 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Plan for Query Block null 2018-01-05T11:28:47,586 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Before logical optimization TS[0] 2018-01-05T11:28:47,586 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,586 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,586 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,586 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] lineage.Generator: Time taken for lineage transform=0 2018-01-05T11:28:47,586 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,586 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,586 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,586 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,586 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,586 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: int|{srcbucket}key,value: string|{srcbucket}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{srcbucket}block__offset__inside__file,INPUT__FILE__NAME: string|{srcbucket}input__file__name,ROW__ID: struct|{srcbucket}row__id,RAW__DATA__SIZE: bigint|{srcbucket}raw__data__size) 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,587 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.PredicatePushDown: After PPD: TS[0] 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: int|{srcbucket}key,value: string|{srcbucket}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{srcbucket}block__offset__inside__file,INPUT__FILE__NAME: string|{srcbucket}input__file__name,ROW__ID: struct|{srcbucket}row__id,RAW__DATA__SIZE: bigint|{srcbucket}raw__data__size) 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: int|{srcbucket}key,value: string|{srcbucket}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{srcbucket}block__offset__inside__file,INPUT__FILE__NAME: string|{srcbucket}input__file__name,ROW__ID: struct|{srcbucket}row__id,RAW__DATA__SIZE: bigint|{srcbucket}raw__data__size) 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,588 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,588 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,588 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,588 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,588 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,588 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,588 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,588 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: After logical optimization TS[0] 2018-01-05T11:28:47,588 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,588 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,588 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,588 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,588 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,588 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,588 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=srcbucket 2018-01-05T11:28:47,588 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=srcbucket 2018-01-05T11:28:47,593 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,599 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting stats (Num rows: 1000 Data size: 10603 Basic stats: COMPLETE Column stats: COMPLETE) on TS[0] 2018-01-05T11:28:47,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (srcbucket): numRows: 1000 dataSize: 10603 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:28:47,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[]]; bucket count: 2 }) on TS[0] 2018-01-05T11:28:47,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:47,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:28:47,600 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Cycle free: true 2018-01-05T11:28:47,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: int|{srcbucket}key,value: string|{srcbucket}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{srcbucket}block__offset__inside__file,INPUT__FILE__NAME: string|{srcbucket}input__file__name,ROW__ID: struct|{srcbucket}row__id,RAW__DATA__SIZE: bigint|{srcbucket}raw__data__size) 2018-01-05T11:28:47,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:28:47,601 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket of table srcbucket 2018-01-05T11:28:47,602 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket 2018-01-05T11:28:47,602 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:28:47,602 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,602 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,602 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:47,602 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:47,602 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:47,602 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:47,602 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:47,603 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:47,603 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:28:47,603 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping vectorization 2018-01-05T11:28:47,603 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:28:47,603 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: llap mode: only 2018-01-05T11:28:47,603 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:28:47,603 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:47,603 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@30ff3726 can be done in LLAP 2018-01-05T11:28:47,603 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,603 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed plan generation 2018-01-05T11:28:47,603 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:47,603 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validation start 2018-01-05T11:28:47,603 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Not a partition. 2018-01-05T11:28:47,603 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,603 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:srcbucket.key, type:int, comment:null), FieldSchema(name:srcbucket.value, type:string, comment:null)], properties:null) 2018-01-05T11:28:47,603 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,603 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:47,603 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=19, flushCache_()=0, getTableColumnStatistics_(String, String, List, )=12} 2018-01-05T11:28:47,603 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112847_e4400b34-36d1-436b-a754-3cde7cdc4901); Time taken: 0.06 seconds 2018-01-05T11:28:47,603 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:47,603 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,603 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112847_e4400b34-36d1-436b-a754-3cde7cdc4901): ANALYZE TABLE srcbucket COMPUTE STATISTICS 2018-01-05T11:28:47,605 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,605 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: ANALYZE TABLE srcbucket COMPUTE STATISTICS 2018-01-05T11:28:47,605 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:28:47,605 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@srcbucket 2018-01-05T11:28:47,605 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@srcbucket 2018-01-05T11:28:47,605 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,605 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,605 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,605 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Query ID = djaiswal_20180105112847_e4400b34-36d1-436b-a754-3cde7cdc4901 2018-01-05T11:28:47,605 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Total jobs = 1 2018-01-05T11:28:47,605 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,605 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:28:47,605 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MAPRED] in serial mode 2018-01-05T11:28:47,615 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_543_7350708723037104955-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:28:47,637 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:28:47,637 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:28:47,637 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:28:47,637 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112847_e4400b34-36d1-436b-a754-3cde7cdc4901 2018-01-05T11:28:47,637 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Session is already open 2018-01-05T11:28:47,637 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:28:47,637 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,637 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Dag name: ANALYZE TABLE srcbucket COMPUTE STATISTICS (Stage-0) 2018-01-05T11:28:47,637 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: DagInfo: {"context":"Hive","description":"\n \nANALYZE TABLE srcbucket COMPUTE STATISTICS"} 2018-01-05T11:28:47,637 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112847_e4400b34-36d1-436b-a754-3cde7cdc4901 with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:28:47,638 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,648 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:28:47,648 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,648 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:28:47,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,650 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 1.87KB 2018-01-05T11:28:47,655 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_543_7350708723037104955-1/-mr-10000 2018-01-05T11:28:47,666 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_543_7350708723037104955-1/-mr-10000 2018-01-05T11:28:47,668 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket 2018-01-05T11:28:47,668 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,668 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,668 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,668 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=ANALYZE TABLE srcbucket COMPUTE STATISTICS (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112847_e4400b34-36d1-436b-a754-3cde7cdc4901 } 2018-01-05T11:28:47,668 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:47,672 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: ANALYZE TABLE srcbucket COMPUTE STATISTICS (Stage-0) 2018-01-05T11:28:47,672 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:28:47,672 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_11, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_11.dot 2018-01-05T11:28:47,673 INFO [IPC Server handler 0 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_11 2018-01-05T11:28:47,673 WARN [IPC Server handler 0 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:47,673 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Running DAG: ANALYZE TABLE srcbucket COMPUTE STATISTICS (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112847_e4400b34-36d1-436b-a754-3cde7cdc4901 } 2018-01-05T11:28:47,694 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_11][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_11, submitTime=1515180527672, queueName=null 2018-01-05T11:28:47,695 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_11_00 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_11_00 [Map 1], Default Resources= 2018-01-05T11:28:47,695 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_11_00 [Map 1] 2018-01-05T11:28:47,695 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_11_00 [Map 1] 2018-01-05T11:28:47,695 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:28:47,695 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_11_00 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:47,695 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:28:47,695 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_11][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_11, initTime=1515180527695 2018-01-05T11:28:47,695 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: dag_1515180518813_0001_11 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:28:47,695 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:28:47,695 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_11][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_11, startTime=1515180527695 2018-01-05T11:28:47,695 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_11 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:28:47,695 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {srcbucket={InputName=srcbucket}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:28:47,695 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: srcbucket, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:28:47,696 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_11_00 [Map 1] 2018-01-05T11:28:47,696 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_11, dagName=ANALYZE TABLE srcbucket COMPUTE STATISTICS (Stage-0) 2018-01-05T11:28:47,696 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,696 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,696 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,700 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_11_00 [Map 1] 2018-01-05T11:28:47,700 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_11_00 [Map 1] 2018-01-05T11:28:47,700 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:47,700 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_11_00 [Map 1] 2018-01-05T11:28:47,702 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:28:47,702 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@402ef3e8 2018-01-05T11:28:47,703 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_543_7350708723037104955-1/djaiswal/_tez_scratch_dir/6a412c7e-e1f8-403d-ba5a-3b838a040103/map.xml 2018-01-05T11:28:47,703 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_543_7350708723037104955-1/djaiswal/_tez_scratch_dir/6a412c7e-e1f8-403d-ba5a-3b838a040103/map.xml 2018-01-05T11:28:47,703 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_543_7350708723037104955-1/djaiswal/_tez_scratch_dir/6a412c7e-e1f8-403d-ba5a-3b838a040103/map.xml 2018-01-05T11:28:47,703 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:47,703 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:47,703 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:47,703 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:47,703 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 1.87KB 2018-01-05T11:28:47,704 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_11_00 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:28:47,704 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: srcbucket on vertex vertex_1515180518813_0001_11_00 [Map 1] 2018-01-05T11:28:47,704 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:28:47,704 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:47,704 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:28:47,704 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:47,704 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_543_7350708723037104955-1/djaiswal/_tez_scratch_dir/6a412c7e-e1f8-403d-ba5a-3b838a040103/map.xml 2018-01-05T11:28:47,704 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:47,704 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias srcbucket 2018-01-05T11:28:47,704 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket 2018-01-05T11:28:47,704 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket 2018-01-05T11:28:47,704 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 2018-01-05T11:28:47,704 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = 2018-01-05T11:28:47,705 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket 2018-01-05T11:28:47,705 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:47,705 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:47,705 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:47,705 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:47,733 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Time taken to get FileStatuses: 28 2018-01-05T11:28:47,733 INFO [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total input files to process : 2 2018-01-05T11:28:47,733 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total # of splits generated by getSplits: 2, TimeTaken: 28 2018-01-05T11:28:47,734 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 2 2018-01-05T11:28:47,734 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:47,734 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 2. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:28:47,734 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_543_7350708723037104955-1/djaiswal/_tez_scratch_dir/6a412c7e-e1f8-403d-ba5a-3b838a040103/map.xml 2018-01-05T11:28:47,734 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_543_7350708723037104955-1/djaiswal/_tez_scratch_dir/6a412c7e-e1f8-403d-ba5a-3b838a040103/map.xml 2018-01-05T11:28:47,734 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_543_7350708723037104955-1/djaiswal/_tez_scratch_dir/6a412c7e-e1f8-403d-ba5a-3b838a040103/map.xml 2018-01-05T11:28:47,734 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:47,734 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:47,734 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:47,734 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:47,734 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 1.87KB 2018-01-05T11:28:47,734 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/000001_0 to src new group? true 2018-01-05T11:28:47,735 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/000000_0 to src new group? false 2018-01-05T11:28:47,735 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:28:47,735 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:28:47,735 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:28:47,735 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 214 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 11603 Original splits: 2 2018-01-05T11:28:47,735 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired numSplits: 1 lengthPerGroup: 11603 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 11603 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false 2018-01-05T11:28:47,735 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Number of splits desired: 1 created: 1 splitsProcessed: 2 2018-01-05T11:28:47,735 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 2 grouped split count is 1, for bucket: 1 2018-01-05T11:28:47,735 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:28:47,735 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: srcbucket on vertex vertex_1515180518813_0001_11_00 [Map 1] 2018-01-05T11:28:47,736 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_11_00 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:47,736 INFO [App Shared Pool - #0] impl.VertexImpl: Got updated RootInputsSpecs: {srcbucket=forAllWorkUnits=true, update=[1]} 2018-01-05T11:28:47,736 INFO [App Shared Pool - #0] impl.VertexImpl: Vertex vertex_1515180518813_0001_11_00 [Map 1] parallelism set to 1 2018-01-05T11:28:47,736 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_11_00 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:47,736 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_11_00 [Map 1], numAdditionalOutputs=1 2018-01-05T11:28:47,736 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_11][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_11_00, initRequestedTime=1515180527695, initedTime=1515180527736, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:47,736 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_11_00 [Map 1] with distanceFromRoot: 0 2018-01-05T11:28:47,736 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_11_00 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:28:47,736 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_11][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_11_00, startRequestedTime=1515180527704, startedTime=1515180527736 2018-01-05T11:28:47,736 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_11][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_11_00, reconfigureDoneTime=1515180527736, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:28:47,736 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_11_00 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:47,737 INFO [App Shared Pool - #0] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:28:47,737 INFO [App Shared Pool - #0] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:28:47,737 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_11_00 [Map 1] 2018-01-05T11:28:47,737 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_11][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_11_00_000000, scheduledTime=1515180527737, launchTime=1515180527737 2018-01-05T11:28:47,737 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_11_00 [Map 1] 2018-01-05T11:28:47,737 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_11_00_000000_0, priority=2, capability=, hosts=null 2018-01-05T11:28:47,737 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=2,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:47,737 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=2,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:47,737 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:47,737 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_11_00_000000_0 on hosts=null 2018-01-05T11:28:47,737 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_11_00_000000_0 on hosts=null 2018-01-05T11:28:47,737 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_11_00_000000_0 2018-01-05T11:28:47,737 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:47,737 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #18, task=TaskInfo{task=attempt_1515180518813_0001_11_00_000000_0, priority=2, startTime=0, containerId=null, uniqueId=17, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000018 2018-01-05T11:28:47,737 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_11_00_000000_0; false 2018-01-05T11:28:47,738 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_11_00_000000_0, priority=2, startTime=311548246, containerId=container_111101111_0001_00_000018assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=17, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:47,738 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000018 for node: 10.22.8.182:49818 2018-01-05T11:28:47,740 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 11, name=ANALYZE TABLE srcbucket COMPUTE STATISTICS (Stage-0), queryId=djaiswal_20180105112847_e4400b34-36d1-436b-a754-3cde7cdc4901 2018-01-05T11:28:47,740 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000018, attempt_1515180518813_0001_11_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:47,740 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000018 for node: 10.22.8.182:49818 2018-01-05T11:28:47,741 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000018 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:47,741 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000018, launchTime=1515180527741 2018-01-05T11:28:47,742 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_11_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000018] on NM: [10.22.8.182:49818] 2018-01-05T11:28:47,742 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:47,742 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:47,742 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_11][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_11_00_000000_0, startTime=1515180527741, containerId=container_111101111_0001_00_000018, nodeId=10.22.8.182:49818 2018-01-05T11:28:47,742 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_11_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112847_e4400b34-36d1-436b-a754-3cde7cdc4901, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000018, dagName=ANALYZE TABLE srcbucket COMPUTE STATISTICS (Stage-0), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{srcbucket,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{out_Map 1,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180527695, firstAttemptStartTime=1515180527737, currentAttemptStartTime=1515180527740} 2018-01-05T11:28:47,742 INFO [IPC Server handler 4 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000018, QueryId=djaiswal_20180105112847_e4400b34-36d1-436b-a754-3cde7cdc4901-11, DagId=11, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180527742 2018-01-05T11:28:47,742 DEBUG [IPC Server handler 4 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=11} with the ShuffleHandler 2018-01-05T11:28:47,753 INFO [IPC Server handler 4 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:28:47,753 DEBUG [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/11] 2018-01-05T11:28:47,753 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:47,753 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_11_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:47,753 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_11_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180527737, dagStartTime=1515180527695, withinDagPriority=2, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:28:47,754 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_11_00_000000_0 : ACCEPTED 2018-01-05T11:28:47,754 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_11_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:47,754 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_11_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180527737, dagStartTime=1515180527695, withinDagPriority=2, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:28:47,754 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_11_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:47,754 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_11_00_000000_0 to pre-emption queue 2018-01-05T11:28:47,754 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_11_00_000000_0: true 2018-01-05T11:28:47,754 INFO [TaskCommunicator # 3] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_11_00_000000_0 2018-01-05T11:28:47,754 INFO [TaskCommunicator # 3] GuaranteedTasks: Received response for attempt_1515180518813_0001_11_00_000000_0, true 2018-01-05T11:28:47,754 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: taskOwner hashCode:400975269 2018-01-05T11:28:47,755 INFO [Task-Executor-2] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : ANALYZE TABLE srcbucket COMPUTE STATISTICS (Stage-0), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_11_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=srcbucket, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=out_Map 1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:28:47,755 INFO [Task-Executor-2] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=954728448, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:47,755 INFO [Task-Executor-2] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_11_00_000000_0 vertexName: Map 1 2018-01-05T11:28:47,755 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000018, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_11_00_000000_0, eventCount=1 } 2018-01-05T11:28:47,755 DEBUG [TezTR-518813_1_11_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_11 queryId: djaiswal_20180105112847_e4400b34-36d1-436b-a754-3cde7cdc4901 instanceType: TaskRunner2Callable 2018-01-05T11:28:47,755 INFO [TezTR-518813_1_11_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_11_00_000000_0 2018-01-05T11:28:47,756 DEBUG [TezTR-518813_1_11_0_0_0] log.PerfLogger: 2018-01-05T11:28:47,757 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:47,758 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:47,760 INFO [IPC Server handler 4 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_11_00_000000_0 sent events: (0-1). 2018-01-05T11:28:47,760 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:28:47,760 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_11_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:28:47,761 DEBUG [TezTR-518813_1_11_0_0_0] log.PerfLogger: 2018-01-05T11:28:47,761 INFO [TezTR-518813_1_11_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:47,761 INFO [TezTR-518813_1_11_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:47,761 INFO [I/O Setup 0 Initialize: {srcbucket}] input.MRInput: srcbucket using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:28:47,761 INFO [I/O Setup 0 Initialize: {srcbucket}] input.MRInputLegacy: srcbucket MRInputLegacy deferring initialization 2018-01-05T11:28:47,761 INFO [TezTR-518813_1_11_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:47,761 INFO [I/O Setup 1 Initialize: {out_Map 1}] output.MROutput: out_Map 1: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:28:47,761 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:47,761 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:47,761 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:47,761 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:47,761 INFO [TezTR-518813_1_11_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:47,761 INFO [TezTR-518813_1_11_0_0_0] resources.MemoryDistributor: InitialRequests=[srcbucket:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [out_Map 1:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput] 2018-01-05T11:28:47,761 INFO [TezTR-518813_1_11_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:47,761 INFO [TezTR-518813_1_11_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:47,761 INFO [TezTR-518813_1_11_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 954728448, TotalAvailable: 639668060, TotalRequested/TotalJVMHeap:0.00 2018-01-05T11:28:47,761 INFO [TezTR-518813_1_11_0_0_0] resources.MemoryDistributor: Allocations=[srcbucket:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [out_Map 1:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0] 2018-01-05T11:28:47,761 INFO [TezTR-518813_1_11_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:47,762 INFO [TezTR-518813_1_11_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: srcbucket being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:28:47,762 INFO [TezTR-518813_1_11_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:28:47,762 INFO [TezTR-518813_1_11_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:28:47,762 INFO [I/O Setup 0 Start: {srcbucket}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: srcbucket 2018-01-05T11:28:47,762 INFO [TezTR-518813_1_11_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:47,762 INFO [TezTR-518813_1_11_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_11_00_000000_0 2018-01-05T11:28:47,762 DEBUG [TezTR-518813_1_11_0_0_0] log.PerfLogger: 2018-01-05T11:28:47,762 DEBUG [TezTR-518813_1_11_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_11_00_000000_0_10052 2018-01-05T11:28:47,762 DEBUG [TezTR-518813_1_11_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:47,762 DEBUG [TezTR-518813_1_11_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_11_00_000000_0 2018-01-05T11:28:47,762 INFO [TezTR-518813_1_11_0_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112847_e4400b34-36d1-436b-a754-3cde7cdc4901 2018-01-05T11:28:47,762 DEBUG [TezTR-518813_1_11_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:47,762 DEBUG [TezTR-518813_1_11_0_0_0] log.PerfLogger: 2018-01-05T11:28:47,762 DEBUG [TezTR-518813_1_11_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:47,762 DEBUG [TezTR-518813_1_11_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:47,762 DEBUG [TezTR-518813_1_11_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112847_e4400b34-36d1-436b-a754-3cde7cdc4901_Map 1__MAP_PLAN__ 2018-01-05T11:28:47,762 INFO [TezTR-518813_1_11_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_543_7350708723037104955-1/djaiswal/_tez_scratch_dir/6a412c7e-e1f8-403d-ba5a-3b838a040103/map.xml 2018-01-05T11:28:47,762 DEBUG [TezTR-518813_1_11_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_543_7350708723037104955-1/djaiswal/_tez_scratch_dir/6a412c7e-e1f8-403d-ba5a-3b838a040103/map.xml 2018-01-05T11:28:47,762 DEBUG [TezTR-518813_1_11_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_543_7350708723037104955-1/djaiswal/_tez_scratch_dir/6a412c7e-e1f8-403d-ba5a-3b838a040103/map.xml 2018-01-05T11:28:47,762 DEBUG [TezTR-518813_1_11_0_0_0] log.PerfLogger: 2018-01-05T11:28:47,762 INFO [TezTR-518813_1_11_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:47,762 DEBUG [TezTR-518813_1_11_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:47,763 DEBUG [TezTR-518813_1_11_0_0_0] log.PerfLogger: 2018-01-05T11:28:47,763 INFO [TezTR-518813_1_11_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 1.87KB 2018-01-05T11:28:47,763 DEBUG [TezTR-518813_1_11_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:47,763 INFO [TezTR-518813_1_11_0_0_0] tez.MapRecordProcessor: The input names are: [srcbucket] 2018-01-05T11:28:47,763 INFO [TezTR-518813_1_11_0_0_0] input.MRInput: Initialized MRInput: srcbucket 2018-01-05T11:28:47,764 DEBUG [TezTR-518813_1_11_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 2 2018-01-05T11:28:47,764 INFO [TezTR-518813_1_11_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_543_7350708723037104955-1/djaiswal/_tez_scratch_dir/6a412c7e-e1f8-403d-ba5a-3b838a040103/map.xml 2018-01-05T11:28:47,764 DEBUG [TezTR-518813_1_11_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:47,764 DEBUG [TezTR-518813_1_11_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/000001_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=1000, numFiles=2, serialization.ddl=struct srcbucket { i32 key, string value}, transient_lastDdlTime=1515180527, rawDataSize=10603, columns.comments=, totalSize=11603, bucket_count=2, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=1000, numFiles=2, serialization.ddl=struct srcbucket { i32 key, string value}, transient_lastDdlTime=1515180527, rawDataSize=10603, columns.comments=, totalSize=11603, bucket_count=2, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket}, baseFileName=srcbucket, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=1000, numFiles=2, serialization.ddl=struct srcbucket { i32 key, string value}, transient_lastDdlTime=1515180527, rawDataSize=10603, columns.comments=, totalSize=11603, bucket_count=2, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=1000, numFiles=2, serialization.ddl=struct srcbucket { i32 key, string value}, transient_lastDdlTime=1515180527, rawDataSize=10603, columns.comments=, totalSize=11603, bucket_count=2, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket}, baseFileName=srcbucket, vectorPartitionDesc=null]} 2018-01-05T11:28:47,764 DEBUG [TezTR-518813_1_11_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:47,764 INFO [TezTR-518813_1_11_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:47,764 DEBUG [TezTR-518813_1_11_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:47,765 DEBUG [TezTR-518813_1_11_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:47,765 INFO [TezTR-518813_1_11_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_543_7350708723037104955-1/djaiswal/_tez_scratch_dir/6a412c7e-e1f8-403d-ba5a-3b838a040103/map.xml 2018-01-05T11:28:47,765 DEBUG [TezTR-518813_1_11_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:47,765 INFO [TezTR-518813_1_11_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_11_00_000000_0 2018-01-05T11:28:47,765 INFO [TezTR-518813_1_11_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:28:47,765 WARN [TezTR-518813_1_11_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/000001_0 (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:28:47,766 DEBUG [TezTR-518813_1_11_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:28:47,766 WARN [TezTR-518813_1_11_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/000001_0:0+5901 2018-01-05T11:28:47,766 DEBUG [TezTR-518813_1_11_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/000001_0 2018-01-05T11:28:47,766 INFO [TezTR-518813_1_11_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/000001_0:0+5901, org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/000000_0:0+5702], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=11603} 2018-01-05T11:28:47,766 INFO [TezTR-518813_1_11_0_0_0] input.MRInput: srcbucket initialized RecordReader from event 2018-01-05T11:28:47,766 DEBUG [TezTR-518813_1_11_0_0_0] tez.MapRecordProcessor: Starting Output: out_Map 1 2018-01-05T11:28:47,766 INFO [TezTR-518813_1_11_0_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:28:47,766 INFO [TezTR-518813_1_11_0_0_0] exec.MapOperator: Initializing operator MAP[0] 2018-01-05T11:28:47,766 DEBUG [TezTR-518813_1_11_0_0_0] exec.MapOperator: Initialization Done 0 MAP 2018-01-05T11:28:47,766 DEBUG [TezTR-518813_1_11_0_0_0] exec.MapOperator: Operator 0 MAP initialized 2018-01-05T11:28:47,766 DEBUG [TezTR-518813_1_11_0_0_0] exec.MapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:28:47,766 DEBUG [TezTR-518813_1_11_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:47,766 DEBUG [TezTR-518813_1_11_0_0_0] exec.MapOperator: Adding alias srcbucket to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket 2018-01-05T11:28:47,766 DEBUG [TezTR-518813_1_11_0_0_0] exec.MapOperator: dump TS[0] struct 2018-01-05T11:28:47,766 INFO [TezTR-518813_1_11_0_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:28:47,766 INFO [TezTR-518813_1_11_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:47,766 DEBUG [TezTR-518813_1_11_0_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:28:47,767 INFO [TezTR-518813_1_11_0_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:28:47,767 DEBUG [TezTR-518813_1_11_0_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:28:47,767 DEBUG [TezTR-518813_1_11_0_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:28:47,767 DEBUG [TezTR-518813_1_11_0_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:28:47,767 DEBUG [TezTR-518813_1_11_0_0_0] log.PerfLogger: 2018-01-05T11:28:47,767 INFO [TezTR-518813_1_11_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_543_7350708723037104955-1/djaiswal/_tez_scratch_dir/6a412c7e-e1f8-403d-ba5a-3b838a040103/map.xml 2018-01-05T11:28:47,767 DEBUG [TezTR-518813_1_11_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:47,767 DEBUG [TezTR-518813_1_11_0_0_0] exec.MapOperator: Processing alias(es) srcbucket for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/000001_0 2018-01-05T11:28:47,767 INFO [TezTR-518813_1_11_0_0_0] exec.MapOperator: MAP[0]: records read - 1 2018-01-05T11:28:47,767 INFO [TezTR-518813_1_11_0_0_0] exec.MapOperator: MAP[0]: records read - 10 2018-01-05T11:28:47,767 INFO [TezTR-518813_1_11_0_0_0] exec.MapOperator: MAP[0]: records read - 100 2018-01-05T11:28:47,767 DEBUG [TezTR-518813_1_11_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 1 of 2 2018-01-05T11:28:47,767 DEBUG [TezTR-518813_1_11_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/000000_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=1000, numFiles=2, serialization.ddl=struct srcbucket { i32 key, string value}, transient_lastDdlTime=1515180527, rawDataSize=10603, columns.comments=, totalSize=11603, bucket_count=2, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=1000, numFiles=2, serialization.ddl=struct srcbucket { i32 key, string value}, transient_lastDdlTime=1515180527, rawDataSize=10603, columns.comments=, totalSize=11603, bucket_count=2, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket}, baseFileName=srcbucket, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=1000, numFiles=2, serialization.ddl=struct srcbucket { i32 key, string value}, transient_lastDdlTime=1515180527, rawDataSize=10603, columns.comments=, totalSize=11603, bucket_count=2, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=1000, numFiles=2, serialization.ddl=struct srcbucket { i32 key, string value}, transient_lastDdlTime=1515180527, rawDataSize=10603, columns.comments=, totalSize=11603, bucket_count=2, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket}, baseFileName=srcbucket, vectorPartitionDesc=null]} 2018-01-05T11:28:47,767 DEBUG [TezTR-518813_1_11_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:47,767 INFO [TezTR-518813_1_11_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:47,767 DEBUG [TezTR-518813_1_11_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:47,768 DEBUG [TezTR-518813_1_11_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:47,768 INFO [TezTR-518813_1_11_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_543_7350708723037104955-1/djaiswal/_tez_scratch_dir/6a412c7e-e1f8-403d-ba5a-3b838a040103/map.xml 2018-01-05T11:28:47,768 DEBUG [TezTR-518813_1_11_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:47,768 INFO [TezTR-518813_1_11_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_11_00_000000_0 2018-01-05T11:28:47,768 INFO [TezTR-518813_1_11_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:28:47,769 WARN [TezTR-518813_1_11_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/000000_0 (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:28:47,769 DEBUG [TezTR-518813_1_11_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:28:47,769 WARN [TezTR-518813_1_11_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/000000_0:0+5702 2018-01-05T11:28:47,769 DEBUG [TezTR-518813_1_11_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/000000_0 2018-01-05T11:28:47,769 INFO [TezTR-518813_1_11_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_543_7350708723037104955-1/djaiswal/_tez_scratch_dir/6a412c7e-e1f8-403d-ba5a-3b838a040103/map.xml 2018-01-05T11:28:47,769 DEBUG [TezTR-518813_1_11_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:47,769 DEBUG [TezTR-518813_1_11_0_0_0] exec.MapOperator: Processing alias(es) srcbucket for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/000000_0 2018-01-05T11:28:47,770 INFO [TezTR-518813_1_11_0_0_0] exec.MapOperator: MAP[0]: records read - 1000 2018-01-05T11:28:47,770 DEBUG [TezTR-518813_1_11_0_0_0] log.PerfLogger: 2018-01-05T11:28:47,770 DEBUG [TezTR-518813_1_11_0_0_0] mr.ObjectCache: djaiswal_20180105112847_e4400b34-36d1-436b-a754-3cde7cdc4901_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:28:47,770 DEBUG [TezTR-518813_1_11_0_0_0] exec.MapOperator: close called for operator MAP[0] 2018-01-05T11:28:47,770 INFO [TezTR-518813_1_11_0_0_0] exec.MapOperator: Closing operator MAP[0] 2018-01-05T11:28:47,770 INFO [TezTR-518813_1_11_0_0_0] exec.MapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:1000, 2018-01-05T11:28:47,770 DEBUG [TezTR-518813_1_11_0_0_0] exec.MapOperator: Closing child = TS[0] 2018-01-05T11:28:47,770 DEBUG [TezTR-518813_1_11_0_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:28:47,770 DEBUG [TezTR-518813_1_11_0_0_0] exec.TableScanOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:47,770 INFO [TezTR-518813_1_11_0_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:28:47,770 DEBUG [TezTR-518813_1_11_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:47,770 DEBUG [TezTR-518813_1_11_0_0_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_543_7350708723037104955-1/-mr-10000 2018-01-05T11:28:47,770 INFO [TezTR-518813_1_11_0_0_0] exec.TableScanOperator: publishing : default.srcbucket/ : {numRows=1000, rawDataSize=10603} 2018-01-05T11:28:47,793 DEBUG [TezTR-518813_1_11_0_0_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_543_7350708723037104955-1/-mr-10000/tmpstats-0_0 2018-01-05T11:28:47,793 DEBUG [TezTR-518813_1_11_0_0_0] fs.FSStatsPublisher: Writing stats in it : {default.srcbucket/={numRows=1000, rawDataSize=10603}} 2018-01-05T11:28:47,793 DEBUG [TezTR-518813_1_11_0_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:28:47,793 DEBUG [TezTR-518813_1_11_0_0_0] exec.MapOperator: 0 Close done 2018-01-05T11:28:47,793 INFO [TezTR-518813_1_11_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_11_00_000000_0 2018-01-05T11:28:47,794 INFO [TezTR-518813_1_11_0_0_0] output.MROutput: out_Map 1 closed 2018-01-05T11:28:47,794 DEBUG [TezTR-518813_1_11_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:47,794 INFO [TezTR-518813_1_11_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:47,794 INFO [TezTR-518813_1_11_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_11_00_000000_0, askedToStop=false 2018-01-05T11:28:47,794 INFO [TezTR-518813_1_11_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_11_00_000000_0, stopRequested=false 2018-01-05T11:28:47,794 INFO [TezTR-518813_1_11_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_11_00_000000_0: Counters: 5 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=1000, INPUT_SPLIT_LENGTH_BYTES=11603, OUTPUT_RECORDS=0][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=1000]] 2018-01-05T11:28:47,794 DEBUG [TezTR-518813_1_11_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_11_0_0_0 thread id: 767 scheme: file bytesRead: 11723 bytesWritten: 90 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:47,794 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_11_00_000000_0, isTaskDone=true 2018-01-05T11:28:47,795 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000018, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_11_00_000000_0, eventCount=2 } 2018-01-05T11:28:47,795 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_11][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_11_00_000000_0, creationTime=1515180527737, allocationTime=1515180527738, startTime=1515180527741, finishTime=1515180527795, timeTaken=54, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:47,795 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:47,795 INFO [Task-Executor-2] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_11_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:47,795 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_11_00_000000_0 2018-01-05T11:28:47,795 INFO [Task-Executor-2] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_11_00_000000_0 2018-01-05T11:28:47,795 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_11_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:47,795 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_11_00_000000_0; false 2018-01-05T11:28:47,795 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_11_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:47,795 INFO [Task-Executor-2] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000018=41 2018-01-05T11:28:47,796 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:47,796 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_11][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_11_00_000000, startTime=1515180527741, finishTime=1515180527795, timeTaken=54, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_11_00_000000_0, diagnostics=, counters=Counters: 11, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=11723, FILE_BYTES_WRITTEN=90, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=1000, INPUT_SPLIT_LENGTH_BYTES=11603, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=1000 2018-01-05T11:28:47,796 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_11_00_000000_0: true 2018-01-05T11:28:47,796 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:47,796 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_11_00_000000_0 2018-01-05T11:28:47,796 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_11_00 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:47,796 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_11_00 [Map 1] 2018-01-05T11:28:47,796 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000018, QueryId=djaiswal_20180105112847_e4400b34-36d1-436b-a754-3cde7cdc4901-11, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:28:47,796 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_11_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:47,796 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_11_00_000000_0 2018-01-05T11:28:47,797 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_11][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_11_00, initRequestedTime=1515180527695, initedTime=1515180527736, startRequestedTime=1515180527704, startedTime=1515180527736, finishTime=1515180527796, timeTaken=60, status=SUCCEEDED, diagnostics=, counters=Counters: 11, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=11723, FILE_BYTES_WRITTEN=90, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=1000, INPUT_SPLIT_LENGTH_BYTES=11603, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=1000, vertexStats=firstTaskStartTime=1515180527741, firstTasksToStart=[ task_1515180518813_0001_11_00_000000 ], lastTaskFinishTime=1515180527795, lastTasksToFinish=[ task_1515180518813_0001_11_00_000000 ], minTaskDuration=54, maxTaskDuration=54, avgTaskDuration=54.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_11_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_11_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:47,797 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_11_00 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:47,797 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_11][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000018, stoppedTime=1515180527797, exitStatus=0 2018-01-05T11:28:47,797 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000018 exited with diagnostics set to Container completed. 2018-01-05T11:28:47,797 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_11_00 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1 2018-01-05T11:28:47,797 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null 2018-01-05T11:28:47,797 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_11_00 [Map 1] 2018-01-05T11:28:47,797 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_11, queueSize=0 2018-01-05T11:28:47,798 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_11][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_11, startTime=1515180527695, finishTime=1515180527797, timeTaken=102, status=SUCCEEDED, diagnostics=, counters=Counters: 15, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, OTHER_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=158, AM_GC_TIME_MILLIS=0, File System Counters, FILE_BYTES_READ=11723, FILE_BYTES_WRITTEN=90, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=1000, INPUT_SPLIT_LENGTH_BYTES=11603, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=1000 2018-01-05T11:28:47,798 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_11 finished with state: SUCCEEDED 2018-01-05T11:28:47,798 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_11 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:28:47,798 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000018 2018-01-05T11:28:47,798 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_11, dagState=SUCCEEDED 2018-01-05T11:28:47,798 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_11_post 2018-01-05T11:28:47,798 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:47,798 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:28:47,798 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:28:47,798 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 11 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=1, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=1,NumRequestsWithoutLocation=1, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=1} 2018-01-05T11:28:47,798 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:28:47,798 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=ANALYZE TABLE srcbucket COMPUTE STATISTICS (Stage-0), with id=dag_1515180518813_0001_11 2018-01-05T11:28:47,798 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 11, to 10.22.8.182:49818 2018-01-05T11:28:47,798 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=1, Remaining=0 2018-01-05T11:28:47,798 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=ANALYZE TABLE srcbucket COMPUTE STATISTICS (Stage-0), with id=dag_1515180518813_0001_11 2018-01-05T11:28:47,798 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=11} 2018-01-05T11:28:47,798 INFO [IPC Server handler 3 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=11}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:28:47,798 INFO [IPC Server handler 3 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/11 after 300 seconds 2018-01-05T11:28:47,799 ERROR [IPC Server handler 3 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112847_e4400b34-36d1-436b-a754-3cde7cdc4901, dag_1515180518813_0001_11 2018-01-05T11:28:47,799 INFO [IPC Server handler 3 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112847_e4400b34-36d1-436b-a754-3cde7cdc4901 2018-01-05T11:28:47,799 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=11} = 0 2018-01-05T11:28:47,799 DEBUG [IPC Server handler 3 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=11} 2018-01-05T11:28:47,799 DEBUG [IPC Server handler 3 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:28:47,799 DEBUG [IPC Server handler 3 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=11} 2018-01-05T11:28:47,799 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,799 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,799 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 2018-01-05T11:28:47,799 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,800 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-2:STATS] in serial mode 2018-01-05T11:28:47,800 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@581cd446, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:47,800 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:47,800 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,800 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:47,800 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:47,800 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:47,800 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:47,800 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,800 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:47,800 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:47,800 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_543_7350708723037104955-1/-mr-10000 2018-01-05T11:28:47,800 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_543_7350708723037104955-1/-mr-10000 2018-01-05T11:28:47,801 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.srcbucket/ numRows 2018-01-05T11:28:47,801 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.srcbucket/ numRows 1000 2018-01-05T11:28:47,801 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.srcbucket/ rawDataSize 2018-01-05T11:28:47,801 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.srcbucket/ rawDataSize 10603 2018-01-05T11:28:47,802 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,802 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:47,802 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:47,802 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:47,802 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:47,803 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,803 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,803 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,822 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,824 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,824 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:47,824 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:47,824 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,824 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:47,824 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,824 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=srcbucket newtbl=srcbucket 2018-01-05T11:28:47,824 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=srcbucket newtbl=srcbucket 2018-01-05T11:28:47,828 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,837 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,837 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,837 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.srcbucket stats: [numFiles=2, numRows=1000, totalSize=11603, rawDataSize=10603] 2018-01-05T11:28:47,837 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: About to delete stats tmp dir :file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_543_7350708723037104955-1/-mr-10000 2018-01-05T11:28:47,838 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,838 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,838 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: ANALYZE TABLE srcbucket COMPUTE STATISTICS 2018-01-05T11:28:47,838 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:28:47,838 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@srcbucket 2018-01-05T11:28:47,838 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@srcbucket 2018-01-05T11:28:47,838 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,838 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,838 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:47,838 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=13} 2018-01-05T11:28:47,838 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112847_e4400b34-36d1-436b-a754-3cde7cdc4901); Time taken: 0.235 seconds 2018-01-05T11:28:47,838 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:47,838 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,838 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:47,838 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,838 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query ANALYZE TABLE srcbucket COMPUTE STATISTICS 2018-01-05T11:28:47,838 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_543_7350708723037104955-1 2018-01-05T11:28:47,839 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.296 seconds 2018-01-05T11:28:47,839 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:47,839 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:47,839 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:47,839 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:47,839 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,839 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:47,839 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,839 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,839 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,839 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: ANALYZE TABLE srcbucket COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:47,839 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112847_e9c71f6a-79ec-4524-a7a9-89f5a111c83e): ANALYZE TABLE srcbucket COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:47,839 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,839 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: ANALYZE TABLE srcbucket COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:47,840 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:47,840 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,840 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,840 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,840 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,840 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:47,840 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,840 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket 2018-01-05T11:28:47,840 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket 2018-01-05T11:28:47,844 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,845 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,845 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: select compute_stats(`key`, 'hll') , compute_stats(`value`, 'hll') from `default`.`srcbucket` 2018-01-05T11:28:47,845 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: select compute_stats(`key`, 'hll') , compute_stats(`value`, 'hll') from `default`.`srcbucket` 2018-01-05T11:28:47,845 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:47,845 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Invoking analyze on rewritten query 2018-01-05T11:28:47,845 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Starting Semantic Analysis 2018-01-05T11:28:47,846 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:47,846 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:47,846 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed phase 1 of Semantic Analysis 2018-01-05T11:28:47,846 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for source tables 2018-01-05T11:28:47,846 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,846 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket 2018-01-05T11:28:47,846 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket 2018-01-05T11:28:47,850 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,850 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,850 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for subqueries 2018-01-05T11:28:47,850 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for destination tables 2018-01-05T11:28:47,862 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed getting MetaData in Semantic Analysis 2018-01-05T11:28:47,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_845_1871307003013194500-1/-mr-10001 2018-01-05T11:28:47,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Table Plan for srcbucket TS[0] 2018-01-05T11:28:47,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:47,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:47,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: RR before GB srcbucket{(key,key: int)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)(RAW__DATA__SIZE,RAW__DATA__SIZE: bigint)} after GB {((tok_function compute_stats (tok_table_or_col key) 'hll'),_col0: struct)((tok_function compute_stats (tok_table_or_col value) 'hll'),_col1: struct)} 2018-01-05T11:28:47,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: tree: (tok_select (tok_selexpr (tok_function compute_stats (tok_table_or_col key) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col value) 'hll'))) 2018-01-05T11:28:47,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: genSelectPlan: input = {((tok_function compute_stats (tok_table_or_col key) 'hll'),_col0: struct)((tok_function compute_stats (tok_table_or_col value) 'hll'),_col1: struct)} starRr = srcbucket{(key,key: int)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)(RAW__DATA__SIZE,RAW__DATA__SIZE: bigint)} 2018-01-05T11:28:47,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Select Plan row schema: null{(_c0,_col0: struct)(_c1,_col1: struct)} 2018-01-05T11:28:47,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Select Plan for clause: insclause-0 2018-01-05T11:28:47,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_845_1871307003013194500-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-47_845_1871307003013194500-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_845_1871307003013194500-1/-mr-10000 2018-01-05T11:28:47,863 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_845_1871307003013194500-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-47_845_1871307003013194500-1 2018-01-05T11:28:47,884 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_845_1871307003013194500-1/-mr-10003 2018-01-05T11:28:47,884 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_845_1871307003013194500-1/-mr-10000 row schema: null{(_c0,_col0: struct)(_c1,_col1: struct)} 2018-01-05T11:28:47,884 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Body Plan for Query Block null 2018-01-05T11:28:47,884 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Plan for Query Block null 2018-01-05T11:28:47,884 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Before logical optimization TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-SEL[5]-FS[6] 2018-01-05T11:28:47,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] lineage.Generator: Time taken for lineage transform=0 2018-01-05T11:28:47,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: int|{srcbucket}key,value: string|{srcbucket}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{srcbucket}block__offset__inside__file,INPUT__FILE__NAME: string|{srcbucket}input__file__name,ROW__ID: struct|{srcbucket}row__id,RAW__DATA__SIZE: bigint|{srcbucket}raw__data__size) 2018-01-05T11:28:47,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: int|{srcbucket}key,value: string|{srcbucket}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{srcbucket}block__offset__inside__file,INPUT__FILE__NAME: string|{srcbucket}input__file__name,ROW__ID: struct|{srcbucket}row__id,RAW__DATA__SIZE: bigint|{srcbucket}raw__data__size) 2018-01-05T11:28:47,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:47,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:47,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:28:47,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:47,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:47,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:47,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:47,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:47,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:47,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:47,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:47,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:47,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1) 2018-01-05T11:28:47,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:47,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:47,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0] Column[_col1]) 2018-01-05T11:28:47,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{}) 2018-01-05T11:28:47,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:47,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:47,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,886 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for FS(6) 2018-01-05T11:28:47,886 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(5) 2018-01-05T11:28:47,886 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for GBY(4) 2018-01-05T11:28:47,886 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for RS(3) 2018-01-05T11:28:47,886 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for GBY(2) 2018-01-05T11:28:47,886 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(1) 2018-01-05T11:28:47,886 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.PredicatePushDown: After PPD: TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-SEL[5]-FS[6] 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: int|{srcbucket}key,value: string|{srcbucket}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{srcbucket}block__offset__inside__file,INPUT__FILE__NAME: string|{srcbucket}input__file__name,ROW__ID: struct|{srcbucket}row__id,RAW__DATA__SIZE: bigint|{srcbucket}raw__data__size) 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: int|{srcbucket}key,value: string|{srcbucket}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{srcbucket}block__offset__inside__file,INPUT__FILE__NAME: string|{srcbucket}input__file__name,ROW__ID: struct|{srcbucket}row__id,RAW__DATA__SIZE: bigint|{srcbucket}raw__data__size) 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1) 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0] Column[_col1]) 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{}) 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: int|{srcbucket}key,value: string|{srcbucket}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{srcbucket}block__offset__inside__file,INPUT__FILE__NAME: string|{srcbucket}input__file__name,ROW__ID: struct|{srcbucket}row__id,RAW__DATA__SIZE: bigint|{srcbucket}raw__data__size) 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: int|{srcbucket}key,value: string|{srcbucket}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{srcbucket}block__offset__inside__file,INPUT__FILE__NAME: string|{srcbucket}input__file__name,ROW__ID: struct|{srcbucket}row__id,RAW__DATA__SIZE: bigint|{srcbucket}raw__data__size) 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1) 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0] Column[_col1]) 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{}) 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 3 key:[] 2018-01-05T11:28:47,886 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 3 oldColExprMap: {VALUE._col0=Column[_col0], VALUE._col1=Column[_col1]} 2018-01-05T11:28:47,886 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 3 newColExprMap: {VALUE._col0=Column[_col0], VALUE._col1=Column[_col1]} 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,887 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,887 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,887 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,887 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,887 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,887 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,887 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,887 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,887 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,887 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,887 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,887 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,887 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,887 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,887 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.IdentityProjectRemover: Identity project remover optimization removed : SEL[5] 2018-01-05T11:28:47,887 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,887 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,887 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,887 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,887 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,887 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,887 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,887 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: After logical optimization TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-FS[6] 2018-01-05T11:28:47,887 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,887 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,887 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,887 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,887 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,887 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,887 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=srcbucket 2018-01-05T11:28:47,887 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=srcbucket 2018-01-05T11:28:47,890 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.331776ms + 0.004605ms, the query is [select "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "TAB_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (...)] 2018-01-05T11:28:47,894 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,897 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000018, QueryId=djaiswal_20180105112847_e4400b34-36d1-436b-a754-3cde7cdc4901-11, DagId=11, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-2, Succeeded=true, StartTime=1515180527754, EndTime=1515180527897 2018-01-05T11:28:47,905 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:47,906 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:47,906 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting stats (Num rows: 1000 Data size: 95000 Basic stats: COMPLETE Column stats: COMPLETE) on TS[0] 2018-01-05T11:28:47,906 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (srcbucket): numRows: 1000 dataSize: 95000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {value= colName: value colType: string countDistincts: 430 numNulls: 0 avgColLen: 6.802 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, key= colName: key colType: int countDistincts: 426 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:47,906 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting stats (Num rows: 1000 Data size: 95000 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[1] 2018-01-05T11:28:47,906 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-SEL[1]: numRows: 1000 dataSize: 95000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {ROW__ID= colName: ROW__ID colType: struct countDistincts: 1000 numNulls: 0 avgColLen: 76.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, RAW__DATA__SIZE= colName: RAW__DATA__SIZE colType: bigint countDistincts: 1000 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, BLOCK__OFFSET__INSIDE__FILE= colName: BLOCK__OFFSET__INSIDE__FILE colType: bigint countDistincts: 1000 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, INPUT__FILE__NAME= colName: INPUT__FILE__NAME colType: string countDistincts: 1000 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, value= colName: value colType: string countDistincts: 430 numNulls: 0 avgColLen: 6.802 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, key= colName: key colType: int countDistincts: 426 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:47,906 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[2]: inputSize: 95000 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[2] hashAgg: true 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [Case 3] STATS-GBY[2]: cardinality: 1 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 864 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[2] 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-GBY[2]: numRows: 1 dataSize: 864 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 864 Basic stats: COMPLETE Column stats: COMPLETE) on RS[3] 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-RS[3]: numRows: 1 dataSize: 864 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {VALUE._col0= colName: VALUE._col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col1= colName: VALUE._col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[4]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[4] hashAgg: false 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [Case 9] STATS-GBY[4]: cardinality: 1 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 880 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[4] 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-GBY[4]: numRows: 1 dataSize: 880 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-FS[6]: numRows: 1 dataSize: 880 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[]]; bucket count: 2 }) on TS[0] 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[]]; bucket count: 2 }) on SEL[1] 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[2] 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[3] 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[4] 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FileSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on FS[6] 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,907 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: FS, 6 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: GBY, 4 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: RS, 3 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: GBY, 2 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: SEL, 1 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:28:47,907 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Cycle free: true 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: int|{srcbucket}key,value: string|{srcbucket}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{srcbucket}block__offset__inside__file,INPUT__FILE__NAME: string|{srcbucket}input__file__name,ROW__ID: struct|{srcbucket}row__id,RAW__DATA__SIZE: bigint|{srcbucket}raw__data__size) 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: int|{srcbucket}key,value: string|{srcbucket}value) 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value]) 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{}) 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: TS[0] 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: RS[3] 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket of table srcbucket 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: RS[3] 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: GBY[4] 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: FS[6] 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding reduce work (Reducer 2) for GBY[4] 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Setting up reduce sink: RS[3] with following reduce work: Reducer 2 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Removing RS[3] as parent from GBY[4] 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: FS[6] 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping vectorization 2018-01-05T11:28:47,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:28:47,909 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: llap mode: only 2018-01-05T11:28:47,909 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:28:47,909 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(key, 'hll')' 2018-01-05T11:28:47,909 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'key' 2018-01-05T11:28:47,909 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:47,909 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(value, 'hll')' 2018-01-05T11:28:47,909 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'value' 2018-01-05T11:28:47,909 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:47,909 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'key' 2018-01-05T11:28:47,909 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'value' 2018-01-05T11:28:47,909 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:47,909 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@66a5a12c can be done in LLAP 2018-01-05T11:28:47,909 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Reducer 2 2018-01-05T11:28:47,909 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col0)' 2018-01-05T11:28:47,909 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:28:47,909 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col1)' 2018-01-05T11:28:47,909 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col1' 2018-01-05T11:28:47,909 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:47,909 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@58507830 can be done in LLAP 2018-01-05T11:28:47,909 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,909 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed plan generation 2018-01-05T11:28:47,909 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:47,909 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validation start 2018-01-05T11:28:47,909 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Not a partition. 2018-01-05T11:28:47,909 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:28:47,909 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,909 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0, type:struct, comment:null), FieldSchema(name:_c1, type:struct, comment:null)], properties:null) 2018-01-05T11:28:47,909 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,909 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:47,909 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=9, flushCache_()=0, getTableColumnStatistics_(String, String, List, )=19} 2018-01-05T11:28:47,909 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112847_e9c71f6a-79ec-4524-a7a9-89f5a111c83e); Time taken: 0.07 seconds 2018-01-05T11:28:47,909 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:47,909 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,909 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112847_e9c71f6a-79ec-4524-a7a9-89f5a111c83e): ANALYZE TABLE srcbucket COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:47,910 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,910 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: ANALYZE TABLE srcbucket COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:47,910 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:28:47,910 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@srcbucket 2018-01-05T11:28:47,910 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@srcbucket 2018-01-05T11:28:47,910 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_845_1871307003013194500-1/-mr-10000 2018-01-05T11:28:47,910 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,910 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,910 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,910 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Query ID = djaiswal_20180105112847_e9c71f6a-79ec-4524-a7a9-89f5a111c83e 2018-01-05T11:28:47,910 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Total jobs = 1 2018-01-05T11:28:47,910 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,910 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:28:47,910 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MAPRED] in serial mode 2018-01-05T11:28:47,933 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_839_5571572109748286427-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:28:47,960 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:28:47,960 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:28:47,960 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:28:47,960 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112847_e9c71f6a-79ec-4524-a7a9-89f5a111c83e 2018-01-05T11:28:47,960 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Session is already open 2018-01-05T11:28:47,960 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:28:47,960 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,960 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Dag name: ANALYZE TABLE srcbucket COMPUTE ...key,value (Stage-0) 2018-01-05T11:28:47,960 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: DagInfo: {"context":"Hive","description":"\n\nANALYZE TABLE srcbucket COMPUTE STATISTICS FOR COLUMNS key,value"} 2018-01-05T11:28:47,960 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112847_e9c71f6a-79ec-4524-a7a9-89f5a111c83e with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:28:47,960 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,973 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,973 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:28:47,974 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,975 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 2 size: 2.05KB 2018-01-05T11:28:47,991 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:47,991 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,004 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:28:48,004 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,004 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:28:48,006 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,006 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 3.55KB 2018-01-05T11:28:48,011 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_845_1871307003013194500-1/-mr-10001 2018-01-05T11:28:48,022 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_845_1871307003013194500-1/-mr-10001 2018-01-05T11:28:48,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket 2018-01-05T11:28:48,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,025 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,025 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,025 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=ANALYZE TABLE srcbucket COMPUTE ...key,value (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112847_e9c71f6a-79ec-4524-a7a9-89f5a111c83e } 2018-01-05T11:28:48,025 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:48,025 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:48,027 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: ANALYZE TABLE srcbucket COMPUTE ...key,value (Stage-0) 2018-01-05T11:28:48,027 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:28:48,028 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_12, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_12.dot 2018-01-05T11:28:48,028 INFO [IPC Server handler 1 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_12 2018-01-05T11:28:48,028 WARN [IPC Server handler 1 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:48,028 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Running DAG: ANALYZE TABLE srcbucket COMPUTE ...key,value (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112847_e9c71f6a-79ec-4524-a7a9-89f5a111c83e } 2018-01-05T11:28:48,051 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_12][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_12, submitTime=1515180528027, queueName=null 2018-01-05T11:28:48,052 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_12_00 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_12_00 [Map 1], Default Resources= 2018-01-05T11:28:48,052 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_12_00 [Map 1] 2018-01-05T11:28:48,052 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:28:48,052 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_12_00 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:48,052 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_12_01 [Reducer 2]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_12_01 [Reducer 2], Default Resources= 2018-01-05T11:28:48,052 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_12_01 [Reducer 2] 2018-01-05T11:28:48,052 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 2 2018-01-05T11:28:48,052 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_12_01 [Reducer 2] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:48,052 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:28:48,052 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_12][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_12, initTime=1515180528052 2018-01-05T11:28:48,052 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: dag_1515180518813_0001_12 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:28:48,053 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:28:48,053 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_12][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_12, startTime=1515180528053 2018-01-05T11:28:48,053 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_12 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:28:48,053 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {srcbucket={InputName=srcbucket}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:28:48,053 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: srcbucket, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:28:48,053 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_12_00 [Map 1] 2018-01-05T11:28:48,053 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_12, dagName=ANALYZE TABLE srcbucket COMPUTE ...key,value (Stage-0) 2018-01-05T11:28:48,053 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,054 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,054 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,057 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_12_00 [Map 1] 2018-01-05T11:28:48,057 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_12_00 [Map 1] 2018-01-05T11:28:48,057 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:48,058 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_12_00 [Map 1] 2018-01-05T11:28:48,060 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:28:48,060 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@2d962715 2018-01-05T11:28:48,060 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_839_5571572109748286427-1/djaiswal/_tez_scratch_dir/5a832d39-78b6-436b-bcba-477620a1b00e/map.xml 2018-01-05T11:28:48,060 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_839_5571572109748286427-1/djaiswal/_tez_scratch_dir/5a832d39-78b6-436b-bcba-477620a1b00e/map.xml 2018-01-05T11:28:48,060 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_839_5571572109748286427-1/djaiswal/_tez_scratch_dir/5a832d39-78b6-436b-bcba-477620a1b00e/map.xml 2018-01-05T11:28:48,060 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:48,060 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:48,060 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:48,061 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:48,061 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.55KB 2018-01-05T11:28:48,061 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_12_00 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:28:48,061 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: srcbucket on vertex vertex_1515180518813_0001_12_00 [Map 1] 2018-01-05T11:28:48,061 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to ShuffleVertexManager for vertex_1515180518813_0001_12_01 [Reducer 2] 2018-01-05T11:28:48,061 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:28:48,061 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:48,061 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:28:48,061 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:48,061 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_839_5571572109748286427-1/djaiswal/_tez_scratch_dir/5a832d39-78b6-436b-bcba-477620a1b00e/map.xml 2018-01-05T11:28:48,061 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:48,061 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias srcbucket 2018-01-05T11:28:48,062 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket 2018-01-05T11:28:48,062 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket 2018-01-05T11:28:48,062 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 0,1 2018-01-05T11:28:48,062 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = key,value 2018-01-05T11:28:48,062 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket 2018-01-05T11:28:48,062 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:48,062 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:48,062 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:48,062 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:48,065 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.05 maxFrac: 0.75 auto: false desiredTaskIput: 104857600 2018-01-05T11:28:48,066 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:28:48,066 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 1 tasks for vertex: vertex_1515180518813_0001_12_01 [Reducer 2] 2018-01-05T11:28:48,066 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_12_01 [Reducer 2] 2018-01-05T11:28:48,066 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_12][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_12_01, reconfigureDoneTime=1515180528066, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:28:48,066 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_12_01 [Reducer 2], numAdditionalOutputs=1 2018-01-05T11:28:48,066 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_12][Event:VERTEX_INITIALIZED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_12_01, initRequestedTime=1515180528061, initedTime=1515180528066, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:48,066 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_12_01 [Reducer 2] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:28:48,090 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Time taken to get FileStatuses: 27 2018-01-05T11:28:48,090 INFO [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total input files to process : 2 2018-01-05T11:28:48,090 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total # of splits generated by getSplits: 2, TimeTaken: 27 2018-01-05T11:28:48,090 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 2 2018-01-05T11:28:48,090 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:48,090 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 2. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:28:48,090 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_839_5571572109748286427-1/djaiswal/_tez_scratch_dir/5a832d39-78b6-436b-bcba-477620a1b00e/map.xml 2018-01-05T11:28:48,090 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_839_5571572109748286427-1/djaiswal/_tez_scratch_dir/5a832d39-78b6-436b-bcba-477620a1b00e/map.xml 2018-01-05T11:28:48,090 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_839_5571572109748286427-1/djaiswal/_tez_scratch_dir/5a832d39-78b6-436b-bcba-477620a1b00e/map.xml 2018-01-05T11:28:48,090 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:48,090 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:48,090 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:48,091 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:48,091 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.55KB 2018-01-05T11:28:48,091 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/000001_0 to src new group? true 2018-01-05T11:28:48,091 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/000000_0 to src new group? false 2018-01-05T11:28:48,091 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:28:48,091 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:28:48,092 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:28:48,092 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 214 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 11603 Original splits: 2 2018-01-05T11:28:48,092 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired numSplits: 1 lengthPerGroup: 11603 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 11603 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false 2018-01-05T11:28:48,092 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Number of splits desired: 1 created: 1 splitsProcessed: 2 2018-01-05T11:28:48,092 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 2 grouped split count is 1, for bucket: 1 2018-01-05T11:28:48,092 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:28:48,092 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: srcbucket on vertex vertex_1515180518813_0001_12_00 [Map 1] 2018-01-05T11:28:48,093 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_12_00 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:48,093 INFO [App Shared Pool - #0] impl.VertexImpl: Got updated RootInputsSpecs: {srcbucket=forAllWorkUnits=true, update=[1]} 2018-01-05T11:28:48,093 INFO [App Shared Pool - #0] impl.VertexImpl: Vertex vertex_1515180518813_0001_12_00 [Map 1] parallelism set to 1 2018-01-05T11:28:48,093 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_12_00 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:48,093 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_12][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_12_00, initRequestedTime=1515180528053, initedTime=1515180528093, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:48,093 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_12_00 [Map 1] with distanceFromRoot: 0 2018-01-05T11:28:48,093 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_12_00 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:28:48,093 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_12][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_12_00, startRequestedTime=1515180528061, startedTime=1515180528093 2018-01-05T11:28:48,093 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_12][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_12_00, reconfigureDoneTime=1515180528093, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:28:48,093 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_12_00 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:48,093 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_12_00 for vertex: vertex_1515180518813_0001_12_01 [Reducer 2] numStartedSources: 1 numSources: 1 2018-01-05T11:28:48,093 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_12_01 [Reducer 2] with distanceFromRoot: 1 2018-01-05T11:28:48,093 INFO [App Shared Pool - #0] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:28:48,093 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_12][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_12_01, startRequestedTime=1515180528093, startedTime=1515180528093 2018-01-05T11:28:48,093 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_12_01 [Reducer 2] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:48,093 INFO [App Shared Pool - #0] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:28:48,093 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_12_00 [Map 1] 2018-01-05T11:28:48,094 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: OnVertexStarted vertex: Reducer 2 with 0 source tasks and 1 pending tasks 2018-01-05T11:28:48,094 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_12][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_12_00_000000, scheduledTime=1515180528094, launchTime=1515180528094 2018-01-05T11:28:48,094 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_12_00 [Map 1] 2018-01-05T11:28:48,094 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Map 1 in vertex: Reducer 2 numBipartiteSourceTasks: 1 2018-01-05T11:28:48,094 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_12_00_000000_0, priority=5, capability=, hosts=null 2018-01-05T11:28:48,094 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=5,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:48,094 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=5,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:48,094 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:48,094 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_12_00_000000_0 on hosts=null 2018-01-05T11:28:48,094 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_12_00_000000_0 on hosts=null 2018-01-05T11:28:48,094 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_12_00_000000_0 2018-01-05T11:28:48,094 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:48,094 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #19, task=TaskInfo{task=attempt_1515180518813_0001_12_00_000000_0, priority=5, startTime=0, containerId=null, uniqueId=18, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000019 2018-01-05T11:28:48,094 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_12_00_000000_0; false 2018-01-05T11:28:48,094 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_12_00_000000_0, priority=5, startTime=311548603, containerId=container_111101111_0001_00_000019assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=18, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:48,095 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000019 for node: 10.22.8.182:49818 2018-01-05T11:28:48,097 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 12, name=ANALYZE TABLE srcbucket COMPUTE ...key,value (Stage-0), queryId=djaiswal_20180105112847_e9c71f6a-79ec-4524-a7a9-89f5a111c83e 2018-01-05T11:28:48,097 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000019, attempt_1515180518813_0001_12_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:48,097 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000019 for node: 10.22.8.182:49818 2018-01-05T11:28:48,098 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000019 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:48,098 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000019, launchTime=1515180528098 2018-01-05T11:28:48,098 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_12_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112847_e9c71f6a-79ec-4524-a7a9-89f5a111c83e, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000019, dagName=ANALYZE TABLE srcbucket COMPUTE ...key,value (Stage-0), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{srcbucket,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{Reducer 2,org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput,1}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180528053, firstAttemptStartTime=1515180528094, currentAttemptStartTime=1515180528097} 2018-01-05T11:28:48,098 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_12_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000019] on NM: [10.22.8.182:49818] 2018-01-05T11:28:48,098 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:48,098 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:48,098 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_12][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_12_00_000000_0, startTime=1515180528098, containerId=container_111101111_0001_00_000019, nodeId=10.22.8.182:49818 2018-01-05T11:28:48,098 INFO [IPC Server handler 2 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000019, QueryId=djaiswal_20180105112847_e9c71f6a-79ec-4524-a7a9-89f5a111c83e-12, DagId=12, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180528098 2018-01-05T11:28:48,099 DEBUG [IPC Server handler 2 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=12} with the ShuffleHandler 2018-01-05T11:28:48,112 INFO [IPC Server handler 2 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:28:48,112 DEBUG [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/12] 2018-01-05T11:28:48,112 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:48,113 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_12_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:48,113 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_12_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180528094, dagStartTime=1515180528053, withinDagPriority=5, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:28:48,113 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_12_00_000000_0 : ACCEPTED 2018-01-05T11:28:48,113 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_12_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:48,113 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_12_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180528094, dagStartTime=1515180528053, withinDagPriority=5, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:28:48,113 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_12_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:48,113 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_12_00_000000_0 to pre-emption queue 2018-01-05T11:28:48,113 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_12_00_000000_0: true 2018-01-05T11:28:48,113 INFO [TaskCommunicator # 5] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_12_00_000000_0 2018-01-05T11:28:48,113 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: taskOwner hashCode:2110340874 2018-01-05T11:28:48,113 INFO [TaskCommunicator # 5] GuaranteedTasks: Received response for attempt_1515180518813_0001_12_00_000000_0, true 2018-01-05T11:28:48,114 INFO [Task-Executor-2] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : ANALYZE TABLE srcbucket COMPUTE ...key,value (Stage-0), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_12_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=srcbucket, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=Reducer 2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput }}, ] 2018-01-05T11:28:48,114 INFO [Task-Executor-2] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=954728448, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:48,114 INFO [Task-Executor-2] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_12_00_000000_0 vertexName: Map 1 2018-01-05T11:28:48,114 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000019, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_12_00_000000_0, eventCount=1 } 2018-01-05T11:28:48,114 DEBUG [TezTR-518813_1_12_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_12 queryId: djaiswal_20180105112847_e9c71f6a-79ec-4524-a7a9-89f5a111c83e instanceType: TaskRunner2Callable 2018-01-05T11:28:48,114 INFO [TezTR-518813_1_12_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_12_00_000000_0 2018-01-05T11:28:48,115 DEBUG [TezTR-518813_1_12_0_0_0] log.PerfLogger: 2018-01-05T11:28:48,116 INFO [I/O Setup 1 Initialize: {Reducer 2}] writers.UnorderedPartitionedKVWriter: Requested BufferSize (tez.runtime.unordered.output.buffer.size-mb) : 100 2018-01-05T11:28:48,116 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:48,117 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:48,119 INFO [IPC Server handler 29 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_12_00_000000_0 sent events: (0-1). 2018-01-05T11:28:48,119 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:28:48,119 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_12_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:28:48,120 DEBUG [TezTR-518813_1_12_0_0_0] log.PerfLogger: 2018-01-05T11:28:48,120 INFO [TezTR-518813_1_12_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:48,120 INFO [TezTR-518813_1_12_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:48,120 INFO [TezTR-518813_1_12_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:48,120 INFO [I/O Setup 0 Initialize: {srcbucket}] input.MRInput: srcbucket using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:28:48,120 INFO [I/O Setup 0 Initialize: {srcbucket}] input.MRInputLegacy: srcbucket MRInputLegacy deferring initialization 2018-01-05T11:28:48,120 INFO [TezTR-518813_1_12_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:48,120 INFO [TezTR-518813_1_12_0_0_0] resources.MemoryDistributor: InitialRequests=[srcbucket:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [Reducer 2:OUTPUT:104857600:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput] 2018-01-05T11:28:48,120 INFO [TezTR-518813_1_12_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:48,120 INFO [TezTR-518813_1_12_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:48,120 INFO [TezTR-518813_1_12_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 104857600, TotalRequestedScaled: 5.24288E7, TotalJVMHeap: 954728448, TotalAvailable: 639668060, TotalRequested/TotalJVMHeap:0.11 2018-01-05T11:28:48,120 INFO [TezTR-518813_1_12_0_0_0] resources.MemoryDistributor: Allocations=[srcbucket:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput:OUTPUT:104857600:104857600] 2018-01-05T11:28:48,120 INFO [TezTR-518813_1_12_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:48,120 INFO [TezTR-518813_1_12_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: srcbucket being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:28:48,120 INFO [I/O Setup 1 Start: {srcbucket}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: srcbucket 2018-01-05T11:28:48,121 INFO [TezTR-518813_1_12_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:28:48,121 INFO [TezTR-518813_1_12_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:28:48,121 INFO [TezTR-518813_1_12_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:48,121 INFO [TezTR-518813_1_12_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_12_00_000000_0 2018-01-05T11:28:48,121 DEBUG [TezTR-518813_1_12_0_0_0] log.PerfLogger: 2018-01-05T11:28:48,121 DEBUG [TezTR-518813_1_12_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_12_00_000000_0_10055 2018-01-05T11:28:48,121 DEBUG [TezTR-518813_1_12_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:48,121 DEBUG [TezTR-518813_1_12_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_12_00_000000_0 2018-01-05T11:28:48,121 INFO [TezTR-518813_1_12_0_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112847_e9c71f6a-79ec-4524-a7a9-89f5a111c83e 2018-01-05T11:28:48,121 DEBUG [TezTR-518813_1_12_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:48,121 DEBUG [TezTR-518813_1_12_0_0_0] log.PerfLogger: 2018-01-05T11:28:48,121 DEBUG [TezTR-518813_1_12_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:48,121 DEBUG [TezTR-518813_1_12_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:48,121 DEBUG [TezTR-518813_1_12_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112847_e9c71f6a-79ec-4524-a7a9-89f5a111c83e_Map 1__MAP_PLAN__ 2018-01-05T11:28:48,121 INFO [TezTR-518813_1_12_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_839_5571572109748286427-1/djaiswal/_tez_scratch_dir/5a832d39-78b6-436b-bcba-477620a1b00e/map.xml 2018-01-05T11:28:48,121 DEBUG [TezTR-518813_1_12_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_839_5571572109748286427-1/djaiswal/_tez_scratch_dir/5a832d39-78b6-436b-bcba-477620a1b00e/map.xml 2018-01-05T11:28:48,122 DEBUG [TezTR-518813_1_12_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_839_5571572109748286427-1/djaiswal/_tez_scratch_dir/5a832d39-78b6-436b-bcba-477620a1b00e/map.xml 2018-01-05T11:28:48,122 DEBUG [TezTR-518813_1_12_0_0_0] log.PerfLogger: 2018-01-05T11:28:48,122 INFO [TezTR-518813_1_12_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:48,122 DEBUG [TezTR-518813_1_12_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:48,123 DEBUG [TezTR-518813_1_12_0_0_0] log.PerfLogger: 2018-01-05T11:28:48,123 INFO [TezTR-518813_1_12_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.55KB 2018-01-05T11:28:48,123 DEBUG [TezTR-518813_1_12_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:48,123 INFO [TezTR-518813_1_12_0_0_0] tez.MapRecordProcessor: The input names are: [srcbucket] 2018-01-05T11:28:48,123 INFO [TezTR-518813_1_12_0_0_0] input.MRInput: Initialized MRInput: srcbucket 2018-01-05T11:28:48,123 DEBUG [TezTR-518813_1_12_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 2 2018-01-05T11:28:48,123 INFO [TezTR-518813_1_12_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_839_5571572109748286427-1/djaiswal/_tez_scratch_dir/5a832d39-78b6-436b-bcba-477620a1b00e/map.xml 2018-01-05T11:28:48,123 DEBUG [TezTR-518813_1_12_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:48,123 DEBUG [TezTR-518813_1_12_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/000001_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=1000, numFiles=2, serialization.ddl=struct srcbucket { i32 key, string value}, transient_lastDdlTime=1515180527, rawDataSize=10603, columns.comments=, totalSize=11603, bucket_count=2, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=1000, numFiles=2, serialization.ddl=struct srcbucket { i32 key, string value}, transient_lastDdlTime=1515180527, rawDataSize=10603, columns.comments=, totalSize=11603, bucket_count=2, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket}, baseFileName=srcbucket, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=1000, numFiles=2, serialization.ddl=struct srcbucket { i32 key, string value}, transient_lastDdlTime=1515180527, rawDataSize=10603, columns.comments=, totalSize=11603, bucket_count=2, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=1000, numFiles=2, serialization.ddl=struct srcbucket { i32 key, string value}, transient_lastDdlTime=1515180527, rawDataSize=10603, columns.comments=, totalSize=11603, bucket_count=2, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket}, baseFileName=srcbucket, vectorPartitionDesc=null]} 2018-01-05T11:28:48,124 DEBUG [TezTR-518813_1_12_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:48,124 INFO [TezTR-518813_1_12_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:48,124 DEBUG [TezTR-518813_1_12_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:48,125 DEBUG [TezTR-518813_1_12_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:48,125 INFO [TezTR-518813_1_12_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_839_5571572109748286427-1/djaiswal/_tez_scratch_dir/5a832d39-78b6-436b-bcba-477620a1b00e/map.xml 2018-01-05T11:28:48,125 DEBUG [TezTR-518813_1_12_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:48,125 INFO [TezTR-518813_1_12_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_12_00_000000_0 2018-01-05T11:28:48,125 INFO [TezTR-518813_1_12_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:28:48,125 DEBUG [TezTR-518813_1_12_0_0_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:48,125 DEBUG [TezTR-518813_1_12_0_0_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:48,125 DEBUG [TezTR-518813_1_12_0_0_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:48,125 DEBUG [TezTR-518813_1_12_0_0_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:48,125 WARN [TezTR-518813_1_12_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/000001_0 (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:28:48,125 DEBUG [TezTR-518813_1_12_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:28:48,125 WARN [TezTR-518813_1_12_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/000001_0:0+5901 2018-01-05T11:28:48,126 DEBUG [TezTR-518813_1_12_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/000001_0 2018-01-05T11:28:48,126 INFO [TezTR-518813_1_12_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/000001_0:0+5901, org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/000000_0:0+5702], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=11603} 2018-01-05T11:28:48,126 INFO [TezTR-518813_1_12_0_0_0] input.MRInput: srcbucket initialized RecordReader from event 2018-01-05T11:28:48,126 DEBUG [TezTR-518813_1_12_0_0_0] tez.MapRecordProcessor: Starting Output: Reducer 2 2018-01-05T11:28:48,126 INFO [TezTR-518813_1_12_0_0_0] writers.BaseUnorderedPartitionedKVWriter: Instantiating Partitioner: [org.apache.tez.mapreduce.partition.MRPartitioner] 2018-01-05T11:28:48,145 INFO [TezTR-518813_1_12_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: numBuffers=2, sizePerBuffer=52428800, skipBuffers=true, numPartitions=1, availableMemory=104857600, maxSingleBufferSizeBytes=2147483647, pipelinedShuffle=false, isFinalMergeEnabled=true, numPartitions=1, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:28:48,145 INFO [TezTR-518813_1_12_0_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:28:48,145 INFO [TezTR-518813_1_12_0_0_0] exec.MapOperator: Initializing operator MAP[0] 2018-01-05T11:28:48,145 DEBUG [TezTR-518813_1_12_0_0_0] exec.MapOperator: Initialization Done 0 MAP 2018-01-05T11:28:48,145 DEBUG [TezTR-518813_1_12_0_0_0] exec.MapOperator: Operator 0 MAP initialized 2018-01-05T11:28:48,145 DEBUG [TezTR-518813_1_12_0_0_0] exec.MapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:28:48,145 DEBUG [TezTR-518813_1_12_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:48,145 DEBUG [TezTR-518813_1_12_0_0_0] exec.MapOperator: Adding alias srcbucket to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket 2018-01-05T11:28:48,146 DEBUG [TezTR-518813_1_12_0_0_0] exec.MapOperator: dump TS[0] struct 2018-01-05T11:28:48,146 INFO [TezTR-518813_1_12_0_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:28:48,146 INFO [TezTR-518813_1_12_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:48,146 DEBUG [TezTR-518813_1_12_0_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:28:48,146 INFO [TezTR-518813_1_12_0_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:28:48,146 DEBUG [TezTR-518813_1_12_0_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:28:48,146 DEBUG [TezTR-518813_1_12_0_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:28:48,146 DEBUG [TezTR-518813_1_12_0_0_0] exec.TableScanOperator: Initializing children of 0 TS 2018-01-05T11:28:48,146 DEBUG [TezTR-518813_1_12_0_0_0] exec.SelectOperator: Initializing child 1 SEL 2018-01-05T11:28:48,146 INFO [TezTR-518813_1_12_0_0_0] exec.SelectOperator: Initializing operator SEL[1] 2018-01-05T11:28:48,146 INFO [TezTR-518813_1_12_0_0_0] exec.SelectOperator: SELECT struct 2018-01-05T11:28:48,146 DEBUG [TezTR-518813_1_12_0_0_0] exec.SelectOperator: Initialization Done 1 SEL 2018-01-05T11:28:48,146 DEBUG [TezTR-518813_1_12_0_0_0] exec.SelectOperator: Operator 1 SEL initialized 2018-01-05T11:28:48,146 DEBUG [TezTR-518813_1_12_0_0_0] exec.SelectOperator: Initializing children of 1 SEL 2018-01-05T11:28:48,146 DEBUG [TezTR-518813_1_12_0_0_0] exec.GroupByOperator: Initializing child 2 GBY 2018-01-05T11:28:48,146 INFO [TezTR-518813_1_12_0_0_0] exec.GroupByOperator: Initializing operator GBY[2] 2018-01-05T11:28:48,146 DEBUG [TezTR-518813_1_12_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:48,146 DEBUG [TezTR-518813_1_12_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:48,146 INFO [TezTR-518813_1_12_0_0_0] exec.GroupByOperator: Max hash table memory: 93860656 bytes 2018-01-05T11:28:48,146 INFO [TezTR-518813_1_12_0_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:48,146 DEBUG [TezTR-518813_1_12_0_0_0] exec.GroupByOperator: Initialization Done 2 GBY 2018-01-05T11:28:48,146 DEBUG [TezTR-518813_1_12_0_0_0] exec.GroupByOperator: Operator 2 GBY initialized 2018-01-05T11:28:48,146 DEBUG [TezTR-518813_1_12_0_0_0] exec.GroupByOperator: Initializing children of 2 GBY 2018-01-05T11:28:48,146 DEBUG [TezTR-518813_1_12_0_0_0] exec.ReduceSinkOperator: Initializing child 3 RS 2018-01-05T11:28:48,146 INFO [TezTR-518813_1_12_0_0_0] exec.ReduceSinkOperator: Initializing operator RS[3] 2018-01-05T11:28:48,146 DEBUG [TezTR-518813_1_12_0_0_0] exec.ReduceSinkOperator: keys size is 0 2018-01-05T11:28:48,146 INFO [TezTR-518813_1_12_0_0_0] exec.ReduceSinkOperator: Using tag = -1 2018-01-05T11:28:48,146 DEBUG [TezTR-518813_1_12_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:48,146 DEBUG [TezTR-518813_1_12_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:48,146 DEBUG [TezTR-518813_1_12_0_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:28:48,146 DEBUG [TezTR-518813_1_12_0_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS 2018-01-05T11:28:48,146 DEBUG [TezTR-518813_1_12_0_0_0] exec.ReduceSinkOperator: Operator 3 RS initialized 2018-01-05T11:28:48,146 DEBUG [TezTR-518813_1_12_0_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS done is reset. 2018-01-05T11:28:48,146 DEBUG [TezTR-518813_1_12_0_0_0] exec.GroupByOperator: Initialization Done 2 GBY done is reset. 2018-01-05T11:28:48,146 DEBUG [TezTR-518813_1_12_0_0_0] exec.SelectOperator: Initialization Done 1 SEL done is reset. 2018-01-05T11:28:48,146 DEBUG [TezTR-518813_1_12_0_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:28:48,146 INFO [TezTR-518813_1_12_0_0_0] exec.OperatorUtils: Setting output collector: RS[3] --> Reducer 2 2018-01-05T11:28:48,146 DEBUG [TezTR-518813_1_12_0_0_0] log.PerfLogger: 2018-01-05T11:28:48,146 INFO [TezTR-518813_1_12_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_839_5571572109748286427-1/djaiswal/_tez_scratch_dir/5a832d39-78b6-436b-bcba-477620a1b00e/map.xml 2018-01-05T11:28:48,146 DEBUG [TezTR-518813_1_12_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:48,146 DEBUG [TezTR-518813_1_12_0_0_0] exec.MapOperator: Processing alias(es) srcbucket for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/000001_0 2018-01-05T11:28:48,147 INFO [TezTR-518813_1_12_0_0_0] exec.MapOperator: MAP[0]: records read - 1 2018-01-05T11:28:48,147 INFO [TezTR-518813_1_12_0_0_0] exec.MapOperator: MAP[0]: records read - 10 2018-01-05T11:28:48,147 INFO [TezTR-518813_1_12_0_0_0] exec.MapOperator: MAP[0]: records read - 100 2018-01-05T11:28:48,150 DEBUG [TezTR-518813_1_12_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 1 of 2 2018-01-05T11:28:48,150 DEBUG [TezTR-518813_1_12_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/000000_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=1000, numFiles=2, serialization.ddl=struct srcbucket { i32 key, string value}, transient_lastDdlTime=1515180527, rawDataSize=10603, columns.comments=, totalSize=11603, bucket_count=2, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=1000, numFiles=2, serialization.ddl=struct srcbucket { i32 key, string value}, transient_lastDdlTime=1515180527, rawDataSize=10603, columns.comments=, totalSize=11603, bucket_count=2, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket}, baseFileName=srcbucket, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=1000, numFiles=2, serialization.ddl=struct srcbucket { i32 key, string value}, transient_lastDdlTime=1515180527, rawDataSize=10603, columns.comments=, totalSize=11603, bucket_count=2, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=1000, numFiles=2, serialization.ddl=struct srcbucket { i32 key, string value}, transient_lastDdlTime=1515180527, rawDataSize=10603, columns.comments=, totalSize=11603, bucket_count=2, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket}, baseFileName=srcbucket, vectorPartitionDesc=null]} 2018-01-05T11:28:48,150 DEBUG [TezTR-518813_1_12_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:48,150 INFO [TezTR-518813_1_12_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:48,150 DEBUG [TezTR-518813_1_12_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:48,151 DEBUG [TezTR-518813_1_12_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:48,151 INFO [TezTR-518813_1_12_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_839_5571572109748286427-1/djaiswal/_tez_scratch_dir/5a832d39-78b6-436b-bcba-477620a1b00e/map.xml 2018-01-05T11:28:48,151 DEBUG [TezTR-518813_1_12_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:48,151 INFO [TezTR-518813_1_12_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_12_00_000000_0 2018-01-05T11:28:48,151 INFO [TezTR-518813_1_12_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:28:48,152 WARN [TezTR-518813_1_12_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/000000_0 (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:28:48,152 DEBUG [TezTR-518813_1_12_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:28:48,152 WARN [TezTR-518813_1_12_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/000000_0:0+5702 2018-01-05T11:28:48,152 DEBUG [TezTR-518813_1_12_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/000000_0 2018-01-05T11:28:48,152 INFO [TezTR-518813_1_12_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_839_5571572109748286427-1/djaiswal/_tez_scratch_dir/5a832d39-78b6-436b-bcba-477620a1b00e/map.xml 2018-01-05T11:28:48,152 DEBUG [TezTR-518813_1_12_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:48,152 DEBUG [TezTR-518813_1_12_0_0_0] exec.MapOperator: Processing alias(es) srcbucket for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket/000000_0 2018-01-05T11:28:48,154 INFO [TezTR-518813_1_12_0_0_0] exec.MapOperator: MAP[0]: records read - 1000 2018-01-05T11:28:48,154 DEBUG [TezTR-518813_1_12_0_0_0] log.PerfLogger: 2018-01-05T11:28:48,154 DEBUG [TezTR-518813_1_12_0_0_0] mr.ObjectCache: djaiswal_20180105112847_e9c71f6a-79ec-4524-a7a9-89f5a111c83e_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:28:48,154 DEBUG [TezTR-518813_1_12_0_0_0] exec.MapOperator: close called for operator MAP[0] 2018-01-05T11:28:48,154 INFO [TezTR-518813_1_12_0_0_0] exec.MapOperator: Closing operator MAP[0] 2018-01-05T11:28:48,154 INFO [TezTR-518813_1_12_0_0_0] exec.MapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:1000, 2018-01-05T11:28:48,154 DEBUG [TezTR-518813_1_12_0_0_0] exec.MapOperator: Closing child = TS[0] 2018-01-05T11:28:48,154 DEBUG [TezTR-518813_1_12_0_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:28:48,154 DEBUG [TezTR-518813_1_12_0_0_0] exec.TableScanOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:48,154 INFO [TezTR-518813_1_12_0_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:28:48,154 DEBUG [TezTR-518813_1_12_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:48,155 DEBUG [TezTR-518813_1_12_0_0_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_845_1871307003013194500-1/-mr-10001 2018-01-05T11:28:48,155 INFO [TezTR-518813_1_12_0_0_0] exec.TableScanOperator: publishing : default.srcbucket/ : {numRows=1000, rawDataSize=10603} 2018-01-05T11:28:48,180 DEBUG [TezTR-518813_1_12_0_0_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_845_1871307003013194500-1/-mr-10001/tmpstats-0_0 2018-01-05T11:28:48,180 DEBUG [TezTR-518813_1_12_0_0_0] fs.FSStatsPublisher: Writing stats in it : {default.srcbucket/={numRows=1000, rawDataSize=10603}} 2018-01-05T11:28:48,180 DEBUG [TezTR-518813_1_12_0_0_0] exec.TableScanOperator: Closing child = SEL[1] 2018-01-05T11:28:48,180 DEBUG [TezTR-518813_1_12_0_0_0] exec.SelectOperator: close called for operator SEL[1] 2018-01-05T11:28:48,180 DEBUG [TezTR-518813_1_12_0_0_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:48,180 INFO [TezTR-518813_1_12_0_0_0] exec.SelectOperator: Closing operator SEL[1] 2018-01-05T11:28:48,180 DEBUG [TezTR-518813_1_12_0_0_0] exec.SelectOperator: Closing child = GBY[2] 2018-01-05T11:28:48,180 DEBUG [TezTR-518813_1_12_0_0_0] exec.GroupByOperator: close called for operator GBY[2] 2018-01-05T11:28:48,180 DEBUG [TezTR-518813_1_12_0_0_0] exec.GroupByOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:48,180 INFO [TezTR-518813_1_12_0_0_0] exec.GroupByOperator: Closing operator GBY[2] 2018-01-05T11:28:48,180 INFO [TezTR-518813_1_12_0_0_0] exec.GroupByOperator: Begin Hash Table flush: size = 1 2018-01-05T11:28:48,181 INFO [TezTR-518813_1_12_0_0_0] exec.ReduceSinkOperator: keys are [] num distributions: 0 2018-01-05T11:28:48,181 DEBUG [TezTR-518813_1_12_0_0_0] exec.GroupByOperator: Closing child = RS[3] 2018-01-05T11:28:48,181 DEBUG [TezTR-518813_1_12_0_0_0] exec.ReduceSinkOperator: close called for operator RS[3] 2018-01-05T11:28:48,181 DEBUG [TezTR-518813_1_12_0_0_0] exec.ReduceSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:48,181 INFO [TezTR-518813_1_12_0_0_0] exec.ReduceSinkOperator: Closing operator RS[3] 2018-01-05T11:28:48,181 INFO [TezTR-518813_1_12_0_0_0] exec.ReduceSinkOperator: RECORDS_OUT_INTERMEDIATE_Map_1:1, 2018-01-05T11:28:48,181 DEBUG [TezTR-518813_1_12_0_0_0] exec.ReduceSinkOperator: 3 Close done 2018-01-05T11:28:48,181 DEBUG [TezTR-518813_1_12_0_0_0] exec.GroupByOperator: 2 Close done 2018-01-05T11:28:48,181 DEBUG [TezTR-518813_1_12_0_0_0] exec.SelectOperator: 1 Close done 2018-01-05T11:28:48,181 DEBUG [TezTR-518813_1_12_0_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:28:48,181 DEBUG [TezTR-518813_1_12_0_0_0] exec.MapOperator: 0 Close done 2018-01-05T11:28:48,181 INFO [TezTR-518813_1_12_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_12_00_000000_0 2018-01-05T11:28:48,181 INFO [TezTR-518813_1_12_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: Waiting for all spills to complete : Pending : 0 2018-01-05T11:28:48,181 INFO [TezTR-518813_1_12_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: All spills complete 2018-01-05T11:28:48,194 DEBUG [TezTR-518813_1_12_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:48,194 INFO [TezTR-518813_1_12_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:48,195 INFO [TezTR-518813_1_12_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_12_00_000000_0, askedToStop=false 2018-01-05T11:28:48,195 INFO [TezTR-518813_1_12_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_12_00_000000_0, stopRequested=false 2018-01-05T11:28:48,195 INFO [TezTR-518813_1_12_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_12_00_000000_0: Counters: 14 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=1000, INPUT_SPLIT_LENGTH_BYTES=11603, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=3188, OUTPUT_BYTES_WITH_OVERHEAD=3198, OUTPUT_BYTES_PHYSICAL=3226, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=1000, RECORDS_OUT_INTERMEDIATE_Map_1=1]] 2018-01-05T11:28:48,195 DEBUG [TezTR-518813_1_12_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_12_0_0_0 thread id: 798 scheme: file bytesRead: 11723 bytesWritten: 3324 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:48,195 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_12_00_000000_0, isTaskDone=true 2018-01-05T11:28:48,195 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000019, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_12_00_000000_0, eventCount=4 } 2018-01-05T11:28:48,196 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_12][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_12_00_000000_0, creationTime=1515180528094, allocationTime=1515180528095, startTime=1515180528098, finishTime=1515180528196, timeTaken=98, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:48,196 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_12_00_000000_0 2018-01-05T11:28:48,196 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_12_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:48,196 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:48,196 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_12_00_000000_0; false 2018-01-05T11:28:48,196 INFO [Task-Executor-2] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_12_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:48,196 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_12_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:48,196 INFO [Task-Executor-2] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_12_00_000000_0 2018-01-05T11:28:48,197 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_12][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_12_00_000000, startTime=1515180528098, finishTime=1515180528196, timeTaken=98, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_12_00_000000_0, diagnostics=, counters=Counters: 20, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=11723, FILE_BYTES_WRITTEN=3324, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=1000, INPUT_SPLIT_LENGTH_BYTES=11603, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=3188, OUTPUT_BYTES_WITH_OVERHEAD=3198, OUTPUT_BYTES_PHYSICAL=3226, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=1000, RECORDS_OUT_INTERMEDIATE_Map_1=1 2018-01-05T11:28:48,197 INFO [Task-Executor-2] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000019=83 2018-01-05T11:28:48,197 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_12_00_000000_0: true 2018-01-05T11:28:48,197 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:48,197 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:48,197 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_12_00_000000_0 2018-01-05T11:28:48,197 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_12_00 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:48,197 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_12_00 [Map 1] 2018-01-05T11:28:48,197 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000019, QueryId=djaiswal_20180105112847_e9c71f6a-79ec-4524-a7a9-89f5a111c83e-12, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:28:48,197 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_12_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:48,197 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_12_00_000000_0 2018-01-05T11:28:48,197 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_12][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_12_00, initRequestedTime=1515180528053, initedTime=1515180528093, startRequestedTime=1515180528061, startedTime=1515180528093, finishTime=1515180528197, timeTaken=104, status=SUCCEEDED, diagnostics=, counters=Counters: 20, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=11723, FILE_BYTES_WRITTEN=3324, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=1000, INPUT_SPLIT_LENGTH_BYTES=11603, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=3188, OUTPUT_BYTES_WITH_OVERHEAD=3198, OUTPUT_BYTES_PHYSICAL=3226, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=1000, RECORDS_OUT_INTERMEDIATE_Map_1=1, vertexStats=firstTaskStartTime=1515180528098, firstTasksToStart=[ task_1515180518813_0001_12_00_000000 ], lastTaskFinishTime=1515180528196, lastTasksToFinish=[ task_1515180518813_0001_12_00_000000 ], minTaskDuration=98, maxTaskDuration=98, avgTaskDuration=98.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_12_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_12_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:48,197 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_12_00 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:48,198 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_12][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000019, stoppedTime=1515180528198, exitStatus=0 2018-01-05T11:28:48,198 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_12_01 [Reducer 2] attempt: attempt_1515180518813_0001_12_00_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:28:48,198 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000019 exited with diagnostics set to Container completed. 2018-01-05T11:28:48,198 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_12_00 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2 2018-01-05T11:28:48,198 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null 2018-01-05T11:28:48,198 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: All source tasks completed. Ramping up 1 remaining tasks for vertex: Reducer 2 2018-01-05T11:28:48,198 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000019 2018-01-05T11:28:48,198 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Scheduling 1 tasks for vertex: Reducer 2 with totalTasks: 1. 1 source tasks completed out of 1. MinSourceTaskCompletedFraction: 1.0 min: 0.05 max: 0.75 2018-01-05T11:28:48,198 INFO [App Shared Pool - #0] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Map 1, destinationVertexName=Reducer 2. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.UnorderedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:28:48,198 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_12_01 [Reducer 2] 2018-01-05T11:28:48,198 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_12][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_12_01_000000, scheduledTime=1515180528198, launchTime=1515180528198 2018-01-05T11:28:48,198 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_12_01 [Reducer 2] 2018-01-05T11:28:48,198 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_12_01_000000_0, priority=14, capability=, hosts=null 2018-01-05T11:28:48,198 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:48,198 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:48,198 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:48,198 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_12_01_000000_0 on hosts=null 2018-01-05T11:28:48,198 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_12_01_000000_0 on hosts=null 2018-01-05T11:28:48,198 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_12_01_000000_0 2018-01-05T11:28:48,198 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:48,198 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #20, task=TaskInfo{task=attempt_1515180518813_0001_12_01_000000_0, priority=14, startTime=0, containerId=null, uniqueId=19, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000020 2018-01-05T11:28:48,198 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_12_01_000000_0; false 2018-01-05T11:28:48,198 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_12_01_000000_0, priority=14, startTime=311548707, containerId=container_111101111_0001_00_000020assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=19, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:48,199 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000020 for node: 10.22.8.182:49818 2018-01-05T11:28:48,199 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000020, attempt_1515180518813_0001_12_01_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:48,199 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000020 for node: 10.22.8.182:49818 2018-01-05T11:28:48,201 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000020 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:48,201 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000020, launchTime=1515180528201 2018-01-05T11:28:48,201 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=12}, sourceName=Map 1, state=S_SUCCEEDED 2018-01-05T11:28:48,201 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_12_01_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000020] on NM: [10.22.8.182:49818] 2018-01-05T11:28:48,201 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:48,201 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:48,201 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_12][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_12_01_000000_0, startTime=1515180528201, containerId=container_111101111_0001_00_000020, nodeId=10.22.8.182:49818 2018-01-05T11:28:48,201 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_12_01_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112847_e9c71f6a-79ec-4524-a7a9-89f5a111c83e, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000020, dagName=ANALYZE TABLE srcbucket COMPUTE ...key,value (Stage-0), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.UnorderedKVInput,1}}, Outputs={{out_Reducer 2,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=2, completedTaskCount=1, dagStartTime=1515180528053, firstAttemptStartTime=1515180528198, currentAttemptStartTime=1515180528200} 2018-01-05T11:28:48,201 INFO [IPC Server handler 3 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000020, QueryId=djaiswal_20180105112847_e9c71f6a-79ec-4524-a7a9-89f5a111c83e-12, DagId=12, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, SubmitTime=1515180528201 2018-01-05T11:28:48,202 DEBUG [IPC Server handler 3 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=12} with the ShuffleHandler 2018-01-05T11:28:48,202 DEBUG [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/12] 2018-01-05T11:28:48,202 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:48,202 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_12_01_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:48,202 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_12_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180528198, dagStartTime=1515180528053, withinDagPriority=14, vertexParallelism= 1, selfAndUpstreamParallelism= 2, selfAndUpstreamComplete= 1}] 2018-01-05T11:28:48,202 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_12_01_000000_0 : ACCEPTED 2018-01-05T11:28:48,202 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_12_01_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:48,202 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_12_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180528198, dagStartTime=1515180528053, withinDagPriority=14, vertexParallelism= 1, selfAndUpstreamParallelism= 2, selfAndUpstreamComplete= 1} 2018-01-05T11:28:48,202 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_12_01_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:48,202 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_12_01_000000_0 to pre-emption queue 2018-01-05T11:28:48,202 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_12_01_000000_0: true 2018-01-05T11:28:48,202 INFO [TaskCommunicator # 7] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_12_01_000000_0 2018-01-05T11:28:48,202 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: taskOwner hashCode:2110340874 2018-01-05T11:28:48,202 INFO [TaskCommunicator # 7] GuaranteedTasks: Received response for attempt_1515180518813_0001_12_01_000000_0, true 2018-01-05T11:28:48,205 INFO [Task-Executor-2] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : ANALYZE TABLE srcbucket COMPUTE ...key,value (Stage-0), VertexName: Reducer 2, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_12_01_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:28:48,205 INFO [Task-Executor-2] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=954728448, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:48,205 INFO [Task-Executor-2] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_12_01_000000_0 vertexName: Reducer 2 2018-01-05T11:28:48,205 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000020, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_12_01_000000_0, eventCount=1 } 2018-01-05T11:28:48,206 DEBUG [TezTR-518813_1_12_1_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_12 queryId: djaiswal_20180105112847_e9c71f6a-79ec-4524-a7a9-89f5a111c83e instanceType: TaskRunner2Callable 2018-01-05T11:28:48,206 INFO [TezTR-518813_1_12_1_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_12_01_000000_0 2018-01-05T11:28:48,206 DEBUG [TezTR-518813_1_12_1_0_0] log.PerfLogger: 2018-01-05T11:28:48,207 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:48,208 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:48,211 INFO [IPC Server handler 19 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_12_01_000000_0 1 events [0,1) total 1 vertex_1515180518813_0001_12_01 [Reducer 2] 2018-01-05T11:28:48,211 DEBUG [TezTR-518813_1_12_1_0_0] log.PerfLogger: 2018-01-05T11:28:48,211 INFO [TezTR-518813_1_12_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:48,211 INFO [TezTR-518813_1_12_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:48,211 INFO [TezTR-518813_1_12_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:48,211 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:28:48,211 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_12_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:28:48,211 INFO [I/O Setup 1 Initialize: {out_Reducer 2}] output.MROutput: out_Reducer 2: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:28:48,212 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:48,212 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:48,212 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:48,212 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:48,212 INFO [TezTR-518813_1_12_1_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:48,212 INFO [TezTR-518813_1_12_1_0_0] resources.MemoryDistributor: InitialRequests=[out_Reducer 2:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.UnorderedKVInput] 2018-01-05T11:28:48,212 INFO [TezTR-518813_1_12_1_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:48,212 INFO [TezTR-518813_1_12_1_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:48,212 INFO [TezTR-518813_1_12_1_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 131404920, TotalRequestedScaled: 6.570246E7, TotalJVMHeap: 954728448, TotalAvailable: 639668060, TotalRequested/TotalJVMHeap:0.14 2018-01-05T11:28:48,212 INFO [TezTR-518813_1_12_1_0_0] resources.MemoryDistributor: Allocations=[out_Reducer 2:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [Map 1:org.apache.tez.runtime.library.input.UnorderedKVInput:INPUT:131404920:131404920] 2018-01-05T11:28:48,212 INFO [TezTR-518813_1_12_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:48,212 INFO [TezTR-518813_1_12_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 0 2018-01-05T11:28:48,212 INFO [TezTR-518813_1_12_1_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:48,212 INFO [TezTR-518813_1_12_1_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_12_01_000000_0 2018-01-05T11:28:48,212 DEBUG [TezTR-518813_1_12_1_0_0] log.PerfLogger: 2018-01-05T11:28:48,212 DEBUG [TezTR-518813_1_12_1_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_12_01_000000_0_10058 2018-01-05T11:28:48,212 DEBUG [TezTR-518813_1_12_1_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:48,212 DEBUG [TezTR-518813_1_12_1_0_0] mr.ObjectCache: Creating djaiswal_20180105112847_e9c71f6a-79ec-4524-a7a9-89f5a111c83e_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:28:48,213 INFO [TezTR-518813_1_12_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_839_5571572109748286427-1/djaiswal/_tez_scratch_dir/fd70eb8b-5dfd-4ebc-b54a-baa89686b172/reduce.xml 2018-01-05T11:28:48,213 DEBUG [TezTR-518813_1_12_1_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_839_5571572109748286427-1/djaiswal/_tez_scratch_dir/fd70eb8b-5dfd-4ebc-b54a-baa89686b172/reduce.xml 2018-01-05T11:28:48,213 DEBUG [TezTR-518813_1_12_1_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_839_5571572109748286427-1/djaiswal/_tez_scratch_dir/fd70eb8b-5dfd-4ebc-b54a-baa89686b172/reduce.xml 2018-01-05T11:28:48,213 DEBUG [TezTR-518813_1_12_1_0_0] log.PerfLogger: 2018-01-05T11:28:48,213 INFO [TezTR-518813_1_12_1_0_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:28:48,213 DEBUG [TezTR-518813_1_12_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:48,214 DEBUG [TezTR-518813_1_12_1_0_0] log.PerfLogger: 2018-01-05T11:28:48,214 INFO [TezTR-518813_1_12_1_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 2.05KB 2018-01-05T11:28:48,214 DEBUG [TezTR-518813_1_12_1_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:48,214 DEBUG [TezTR-518813_1_12_1_0_0] log.PerfLogger: 2018-01-05T11:28:48,214 DEBUG [TezTR-518813_1_12_1_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:48,214 DEBUG [TezTR-518813_1_12_1_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:48,214 DEBUG [TezTR-518813_1_12_1_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:48,214 INFO [TezTR-518813_1_12_1_0_0] impl.SimpleFetchedInputAllocator: Map_1: RequestedMemory=131404920, AssignedMemory=131404920, maxSingleShuffleLimit=32851230 2018-01-05T11:28:48,215 INFO [TezTR-518813_1_12_1_0_0] impl.ShuffleManager: Map_1: numInputs=1, compressionCodec=NoCompressionCodec, numFetchers=1, ifileBufferSize=4096, ifileReadAheadEnabled=true, ifileReadAheadLength=4194304, localDiskFetchEnabled=false, sharedFetchEnabled=false, keepAlive=false, keepAliveMaxConnections=20, connectionTimeout=20000, readTimeout=180000, bufferSize=8192, bufferSize=8192, maxTaskOutputAtOnce=20 2018-01-05T11:28:48,215 INFO [TezTR-518813_1_12_1_0_0] impl.ShuffleInputEventHandlerImpl: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:28:48,215 INFO [TezTR-518813_1_12_1_0_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:28:48,217 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=12&reduce=0&map=attempt_1515180518813_0001_12_00_000000_0_10057 mapId: [attempt_1515180518813_0001_12_00_000000_0_10057] reduceId: [0] jobId: [job_1515180518813_0001] dagId: [12] keepAlive: false 2018-01-05T11:28:48,217 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=12&reduce=0&map=attempt_1515180518813_0001_12_00_000000_0_10057; hash=...0PB6X1BNlSU30 2018-01-05T11:28:48,217 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=12&reduce=0&map=attempt_1515180518813_0001_12_00_000000_0_10057;reply=zTPvp+O8keTJ8 2018-01-05T11:28:48,217 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=12, user='djaiswal', attemptId='attempt_1515180518813_0001_12_00_000000_0_10057'} via loader 2018-01-05T11:28:48,217 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=12, user='djaiswal', attemptId='attempt_1515180518813_0001_12_00_000000_0_10057'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:28:48,217 DEBUG [New I/O worker #2] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_12_00_000000_0_10057 not found 2018-01-05T11:28:48,218 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_12_00_000000_0_10057,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/12/output/attempt_1515180518813_0001_12_00_000000_0_10057/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/12/output/attempt_1515180518813_0001_12_00_000000_0_10057/file.out.index 2018-01-05T11:28:48,218 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: Content Length in shuffle : 3257 2018-01-05T11:28:48,218 INFO [Fetcher_B {Map_1} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=12&reduce=0&map=attempt_1515180518813_0001_12_00_000000_0_10057 sent hash and receievd reply 0 ms 2018-01-05T11:28:48,218 INFO [Fetcher_B {Map_1} #0] ShuffleManager.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_12_00_000000_0_10057} to MEMORY, csize=3202, dsize=3198, EndTime=1515180528218, TimeTaken=0, Rate=0.00 MB/s 2018-01-05T11:28:48,218 INFO [Fetcher_B {Map_1} #0] impl.ShuffleManager: All inputs fetched for input vertex : Map 1 2018-01-05T11:28:48,218 INFO [Fetcher_B {Map_1} #0] impl.ShuffleManager: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:28:48,218 INFO [TezTR-518813_1_12_1_0_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:28:48,218 INFO [ShuffleRunner {Map_1}] impl.ShuffleManager: Map_1: Shutting down FetchScheduler, Was Interrupted: false 2018-01-05T11:28:48,218 INFO [TezTR-518813_1_12_1_0_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:48,218 INFO [ShuffleRunner {Map_1}] impl.ShuffleManager: Map_1: Scheduler thread completed 2018-01-05T11:28:48,218 DEBUG [TezTR-518813_1_12_1_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:28:48,219 DEBUG [TezTR-518813_1_12_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:48,219 DEBUG [TezTR-518813_1_12_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:48,219 DEBUG [TezTR-518813_1_12_1_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:28:48,219 DEBUG [TezTR-518813_1_12_1_0_0] log.PerfLogger: 2018-01-05T11:28:48,219 INFO [TezTR-518813_1_12_1_0_0] exec.GroupByOperator: Initializing operator GBY[4] 2018-01-05T11:28:48,219 DEBUG [TezTR-518813_1_12_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:48,219 DEBUG [TezTR-518813_1_12_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:48,219 INFO [TezTR-518813_1_12_1_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:48,219 DEBUG [TezTR-518813_1_12_1_0_0] exec.GroupByOperator: Initialization Done 4 GBY 2018-01-05T11:28:48,219 DEBUG [TezTR-518813_1_12_1_0_0] exec.GroupByOperator: Operator 4 GBY initialized 2018-01-05T11:28:48,219 DEBUG [TezTR-518813_1_12_1_0_0] exec.GroupByOperator: Initializing children of 4 GBY 2018-01-05T11:28:48,219 DEBUG [TezTR-518813_1_12_1_0_0] exec.FileSinkOperator: Initializing child 6 FS 2018-01-05T11:28:48,219 INFO [TezTR-518813_1_12_1_0_0] exec.FileSinkOperator: Initializing operator FS[6] 2018-01-05T11:28:48,219 DEBUG [TezTR-518813_1_12_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:48,220 INFO [TezTR-518813_1_12_1_0_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@78a1f97e]:[_col0, _col1]:[struct, struct]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@64b45a03 2018-01-05T11:28:48,220 DEBUG [TezTR-518813_1_12_1_0_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:28:48,220 DEBUG [TezTR-518813_1_12_1_0_0] exec.FileSinkOperator: Initialization Done 6 FS 2018-01-05T11:28:48,220 DEBUG [TezTR-518813_1_12_1_0_0] exec.FileSinkOperator: Operator 6 FS initialized 2018-01-05T11:28:48,220 DEBUG [TezTR-518813_1_12_1_0_0] exec.FileSinkOperator: Initialization Done 6 FS done is reset. 2018-01-05T11:28:48,220 DEBUG [TezTR-518813_1_12_1_0_0] exec.GroupByOperator: Initialization Done 4 GBY done is reset. 2018-01-05T11:28:48,220 INFO [TezTR-518813_1_12_1_0_0] tez.ReduceRecordProcessor: Id =4 Id =6 <\Children> Id = 4 null<\Parent> <\FS> <\Children> <\Parent> <\GBY> 2018-01-05T11:28:48,220 DEBUG [TezTR-518813_1_12_1_0_0] log.PerfLogger: 2018-01-05T11:28:48,220 INFO [TezTR-518813_1_12_1_0_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 2 2018-01-05T11:28:48,220 DEBUG [TezTR-518813_1_12_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:28:48,220 DEBUG [TezTR-518813_1_12_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:28:48,220 DEBUG [TezTR-518813_1_12_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:28:48,220 DEBUG [TezTR-518813_1_12_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:28:48,220 DEBUG [TezTR-518813_1_12_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:28:48,220 DEBUG [TezTR-518813_1_12_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:28:48,220 INFO [TezTR-518813_1_12_1_0_0] readers.UnorderedKVReader: Num Records read: 1 2018-01-05T11:28:48,220 DEBUG [TezTR-518813_1_12_1_0_0] log.PerfLogger: 2018-01-05T11:28:48,220 DEBUG [TezTR-518813_1_12_1_0_0] mr.ObjectCache: djaiswal_20180105112847_e9c71f6a-79ec-4524-a7a9-89f5a111c83e_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:28:48,220 DEBUG [TezTR-518813_1_12_1_0_0] exec.GroupByOperator: close called for operator GBY[4] 2018-01-05T11:28:48,220 INFO [TezTR-518813_1_12_1_0_0] exec.GroupByOperator: Closing operator GBY[4] 2018-01-05T11:28:48,221 INFO [TezTR-518813_1_12_1_0_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_845_1871307003013194500-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-47_845_1871307003013194500-1/_tmp.-ext-10002/000000_0 2018-01-05T11:28:48,221 INFO [TezTR-518813_1_12_1_0_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_845_1871307003013194500-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-47_845_1871307003013194500-1/_task_tmp.-ext-10002/_tmp.000000_0 2018-01-05T11:28:48,221 INFO [TezTR-518813_1_12_1_0_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_845_1871307003013194500-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-47_845_1871307003013194500-1/_tmp.-ext-10002/000000_0 2018-01-05T11:28:48,258 INFO [TezTR-518813_1_12_1_0_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:28:48,258 DEBUG [TezTR-518813_1_12_1_0_0] exec.GroupByOperator: Closing child = FS[6] 2018-01-05T11:28:48,258 DEBUG [TezTR-518813_1_12_1_0_0] exec.FileSinkOperator: close called for operator FS[6] 2018-01-05T11:28:48,258 DEBUG [TezTR-518813_1_12_1_0_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:48,258 INFO [TezTR-518813_1_12_1_0_0] exec.FileSinkOperator: Closing operator FS[6] 2018-01-05T11:28:48,258 INFO [TezTR-518813_1_12_1_0_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:28:48,258 INFO [TezTR-518813_1_12_1_0_0] exec.FileSinkOperator: RECORDS_OUT_0:1, 2018-01-05T11:28:48,258 DEBUG [TezTR-518813_1_12_1_0_0] exec.FileSinkOperator: 6 Close done 2018-01-05T11:28:48,258 DEBUG [TezTR-518813_1_12_1_0_0] exec.GroupByOperator: 4 Close done 2018-01-05T11:28:48,258 INFO [TezTR-518813_1_12_1_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_12_01_000000_0 2018-01-05T11:28:48,258 INFO [TezTR-518813_1_12_1_0_0] impl.ShuffleInputEventHandlerImpl: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:28:48,258 INFO [TezTR-518813_1_12_1_0_0] impl.ShuffleManager: Shutting down pending fetchers on sourceMap_1: 0 2018-01-05T11:28:48,258 INFO [TezTR-518813_1_12_1_0_0] output.MROutput: out_Reducer 2 closed 2018-01-05T11:28:48,259 DEBUG [TezTR-518813_1_12_1_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:48,259 INFO [TezTR-518813_1_12_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:48,259 INFO [TezTR-518813_1_12_1_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_12_01_000000_0, askedToStop=false 2018-01-05T11:28:48,259 INFO [TezTR-518813_1_12_1_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_12_01_000000_0, stopRequested=false 2018-01-05T11:28:48,259 INFO [TezTR-518813_1_12_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_12_01_000000_0: Counters: 14 [[org.apache.tez.common.counters.TaskCounter NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=3202, SHUFFLE_BYTES_DECOMPRESSED=3198, SHUFFLE_BYTES_TO_MEM=3202, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=4, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1][HIVE CREATED_FILES=1, RECORDS_OUT_0=1]] 2018-01-05T11:28:48,259 DEBUG [TezTR-518813_1_12_1_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_12_1_0_0 thread id: 810 scheme: file bytesRead: 0 bytesWritten: 4381 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:48,260 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_12_01_000000_0, isTaskDone=true 2018-01-05T11:28:48,261 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000020, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_12_01_000000_0, eventCount=2 } 2018-01-05T11:28:48,262 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_12][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_12_01_000000_0, creationTime=1515180528198, allocationTime=1515180528199, startTime=1515180528201, finishTime=1515180528262, timeTaken=61, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:48,262 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_12_01_000000_0 2018-01-05T11:28:48,262 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_12_01_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:48,262 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:28:48,262 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_12_01_000000_0; false 2018-01-05T11:28:48,262 INFO [Task-Executor-2] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_12_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:48,262 INFO [Task-Executor-2] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_12_01_000000_0 2018-01-05T11:28:48,262 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_12_01_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:48,262 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_12][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_12_01_000000, startTime=1515180528201, finishTime=1515180528262, timeTaken=61, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_12_01_000000_0, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=4381, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=3202, SHUFFLE_BYTES_DECOMPRESSED=3198, SHUFFLE_BYTES_TO_MEM=3202, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=4, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1 2018-01-05T11:28:48,262 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:48,263 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:48,263 INFO [Task-Executor-2] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000020=61 2018-01-05T11:28:48,263 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_12_01_000000_0: true 2018-01-05T11:28:48,263 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_12_01 [Reducer 2], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:48,263 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_12_01 [Reducer 2] 2018-01-05T11:28:48,264 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_12][Event:VERTEX_FINISHED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_12_01, initRequestedTime=1515180528061, initedTime=1515180528066, startRequestedTime=1515180528093, startedTime=1515180528093, finishTime=1515180528263, timeTaken=170, status=SUCCEEDED, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=4381, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=3202, SHUFFLE_BYTES_DECOMPRESSED=3198, SHUFFLE_BYTES_TO_MEM=3202, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=4, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1, vertexStats=firstTaskStartTime=1515180528201, firstTasksToStart=[ task_1515180518813_0001_12_01_000000 ], lastTaskFinishTime=1515180528262, lastTasksToFinish=[ task_1515180518813_0001_12_01_000000 ], minTaskDuration=61, maxTaskDuration=61, avgTaskDuration=61.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_12_01_000000 ], longestDurationTasks=[ task_1515180518813_0001_12_01_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:48,264 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_12_01 [Reducer 2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:48,264 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_12][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000020, stoppedTime=1515180528264, exitStatus=0 2018-01-05T11:28:48,264 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000020 exited with diagnostics set to Container completed. 2018-01-05T11:28:48,264 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_12_01 [Reducer 2] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2 2018-01-05T11:28:48,264 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null 2018-01-05T11:28:48,264 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_12_00 [Map 1] 2018-01-05T11:28:48,264 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_12_01 [Reducer 2] 2018-01-05T11:28:48,264 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_12, queueSize=0 2018-01-05T11:28:48,265 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_12][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_12, startTime=1515180528053, finishTime=1515180528264, timeTaken=211, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, OTHER_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=411, AM_GC_TIME_MILLIS=0, File System Counters, FILE_BYTES_READ=11723, FILE_BYTES_WRITTEN=7705, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1001, INPUT_SPLIT_LENGTH_BYTES=11603, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=3188, OUTPUT_BYTES_WITH_OVERHEAD=3198, OUTPUT_BYTES_PHYSICAL=3226, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=3202, SHUFFLE_BYTES_DECOMPRESSED=3198, SHUFFLE_BYTES_TO_MEM=3202, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=4, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1, HIVE, CREATED_FILES=1, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=1000, RECORDS_OUT_0=1, RECORDS_OUT_INTERMEDIATE_Map_1=1 2018-01-05T11:28:48,265 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_12 finished with state: SUCCEEDED 2018-01-05T11:28:48,265 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_12 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:28:48,265 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000020 2018-01-05T11:28:48,265 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_12, dagState=SUCCEEDED 2018-01-05T11:28:48,265 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_12_post 2018-01-05T11:28:48,265 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:48,265 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:28:48,265 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:28:48,265 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 12 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=2, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=2,NumRequestsWithoutLocation=2, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=2} 2018-01-05T11:28:48,265 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:28:48,265 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=ANALYZE TABLE srcbucket COMPUTE ...key,value (Stage-0), with id=dag_1515180518813_0001_12 2018-01-05T11:28:48,265 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 12, to 10.22.8.182:49818 2018-01-05T11:28:48,265 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=2, Remaining=0 2018-01-05T11:28:48,265 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=ANALYZE TABLE srcbucket COMPUTE ...key,value (Stage-0), with id=dag_1515180518813_0001_12 2018-01-05T11:28:48,265 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=12} 2018-01-05T11:28:48,265 INFO [IPC Server handler 1 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=12}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:28:48,265 INFO [IPC Server handler 1 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/12 after 300 seconds 2018-01-05T11:28:48,266 ERROR [IPC Server handler 1 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112847_e9c71f6a-79ec-4524-a7a9-89f5a111c83e, dag_1515180518813_0001_12 2018-01-05T11:28:48,266 INFO [IPC Server handler 1 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112847_e9c71f6a-79ec-4524-a7a9-89f5a111c83e 2018-01-05T11:28:48,266 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=12} = 1 2018-01-05T11:28:48,266 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Issuing killFragment for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=12} attempt_1515180518813_0001_12_01_000000_0 2018-01-05T11:28:48,266 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Removing attempt_1515180518813_0001_12_01_000000_0 from preemptionQueue 2018-01-05T11:28:48,266 INFO [IPC Server handler 1 on 49818] impl.TaskRunnerCallable: Ignoring kill request for task attempt_1515180518813_0001_12_01_000000_0 since it's already complete 2018-01-05T11:28:48,266 DEBUG [IPC Server handler 1 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=12} 2018-01-05T11:28:48,266 DEBUG [IPC Server handler 1 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:28:48,266 DEBUG [IPC Server handler 1 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=12} 2018-01-05T11:28:48,266 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,266 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,266 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,266 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,266 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 Reducer 2: 1/1 2018-01-05T11:28:48,266 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,268 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,268 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: TaskId for 000000_0 = 000000 2018-01-05T11:28:48,268 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,268 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,268 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,269 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-2:STATS] in serial mode 2018-01-05T11:28:48,269 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@397fd951, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:48,269 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:48,269 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,269 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:48,269 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:48,269 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:48,269 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:48,269 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,269 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:48,269 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:48,270 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_845_1871307003013194500-1/-mr-10001 2018-01-05T11:28:48,270 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_845_1871307003013194500-1/-mr-10001 2018-01-05T11:28:48,271 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.srcbucket/ numRows 2018-01-05T11:28:48,271 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.srcbucket/ numRows 1000 2018-01-05T11:28:48,271 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.srcbucket/ rawDataSize 2018-01-05T11:28:48,271 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.srcbucket/ rawDataSize 10603 2018-01-05T11:28:48,271 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,271 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:48,271 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:48,271 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:48,272 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:48,272 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,272 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,272 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,273 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,273 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,273 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:48,273 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:48,273 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,273 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:48,273 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,273 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=srcbucket newtbl=srcbucket 2018-01-05T11:28:48,273 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=srcbucket newtbl=srcbucket 2018-01-05T11:28:48,277 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,283 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,284 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,284 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.srcbucket stats: [numFiles=2, numRows=1000, totalSize=11603, rawDataSize=10603] 2018-01-05T11:28:48,284 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: About to delete stats tmp dir :file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_845_1871307003013194500-1/-mr-10001 2018-01-05T11:28:48,299 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Time taken to get FileStatuses: 14 2018-01-05T11:28:48,299 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:28:48,299 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 14 2018-01-05T11:28:48,299 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FetchOperator: Creating fetchTask with deserializer typeinfo: struct<_col0:struct,_col1:struct> 2018-01-05T11:28:48,299 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FetchOperator: deserializer properties: table properties: {columns=_col0,_col1, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct:struct, escape.delim=\} partition properties: {columns=_col0,_col1, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct:struct, escape.delim=\} 2018-01-05T11:28:48,301 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,301 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: write_column_statistics: db=default table=srcbucket 2018-01-05T11:28:48,301 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=write_column_statistics: db=default table=srcbucket 2018-01-05T11:28:48,302 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000019, QueryId=djaiswal_20180105112847_e9c71f6a-79ec-4524-a7a9-89f5a111c83e-12, DagId=12, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-2, Succeeded=true, StartTime=1515180528113, EndTime=1515180528302 2018-01-05T11:28:48,302 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_12_01_000000_0 2018-01-05T11:28:48,302 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_12_01_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:48,302 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_12_01_000000_0 2018-01-05T11:28:48,302 INFO [ExecutionCompletionThread #0] impl.QueryTracker: Ignoring fragmentComplete message for unknown query: QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=12} 2018-01-05T11:28:48,305 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=srcbucket tableName=default colName=key 2018-01-05T11:28:48,308 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=srcbucket tableName=default colName=value 2018-01-05T11:28:48,310 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,311 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,311 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,311 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,311 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: ANALYZE TABLE srcbucket COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:48,311 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:28:48,311 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@srcbucket 2018-01-05T11:28:48,311 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@srcbucket 2018-01-05T11:28:48,311 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_845_1871307003013194500-1/-mr-10000 2018-01-05T11:28:48,311 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,311 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,311 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:48,311 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=11, setPartitionColumnStatistics_(SetPartitionsStatsRequest, )=10} 2018-01-05T11:28:48,311 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112847_e9c71f6a-79ec-4524-a7a9-89f5a111c83e); Time taken: 0.402 seconds 2018-01-05T11:28:48,311 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:48,311 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,311 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:48,311 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,311 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query ANALYZE TABLE srcbucket COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:48,311 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-47_839_5571572109748286427-1 2018-01-05T11:28:48,312 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.472 seconds 2018-01-05T11:28:48,312 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:48,312 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:48,312 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:48,312 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:48,312 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,312 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:48,312 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,312 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,312 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,312 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: -- -- Table srcbucket2 -- DROP TABLE IF EXISTS srcbucket_tmp 2018-01-05T11:28:48,312 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112848_7c764529-2e76-4dc1-b20c-14ddc5ee02b7): -- -- Table srcbucket2 -- DROP TABLE IF EXISTS srcbucket_tmp 2018-01-05T11:28:48,312 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,312 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: -- -- Table srcbucket2 -- DROP TABLE IF EXISTS srcbucket_tmp 2018-01-05T11:28:48,312 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:48,312 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,312 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,313 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,313 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,313 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:48,313 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,313 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,313 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,313 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,313 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,313 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:48,313 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,313 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:48,313 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,313 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:48,313 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=1, flushCache_()=0} 2018-01-05T11:28:48,313 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112848_7c764529-2e76-4dc1-b20c-14ddc5ee02b7); Time taken: 0.001 seconds 2018-01-05T11:28:48,313 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:48,313 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,313 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112848_7c764529-2e76-4dc1-b20c-14ddc5ee02b7): -- -- Table srcbucket2 -- DROP TABLE IF EXISTS srcbucket_tmp 2018-01-05T11:28:48,314 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,314 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: -- -- Table srcbucket2 -- DROP TABLE IF EXISTS srcbucket_tmp 2018-01-05T11:28:48,314 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:48,314 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,314 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,314 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,315 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,315 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:48,315 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,315 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,315 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,315 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,315 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,315 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table srcbucket_tmp not found: default.srcbucket_tmp table not found 2018-01-05T11:28:48,315 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,315 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,315 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,316 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,316 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,316 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,316 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,316 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: -- -- Table srcbucket2 -- DROP TABLE IF EXISTS srcbucket_tmp 2018-01-05T11:28:48,316 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:48,316 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,316 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,316 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:48,316 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=1} 2018-01-05T11:28:48,316 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112848_7c764529-2e76-4dc1-b20c-14ddc5ee02b7); Time taken: 0.003 seconds 2018-01-05T11:28:48,316 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:48,316 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,316 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:48,316 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,316 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query -- -- Table srcbucket2 -- DROP TABLE IF EXISTS srcbucket_tmp 2018-01-05T11:28:48,316 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.004 seconds 2018-01-05T11:28:48,316 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:48,316 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:48,316 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:48,316 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:48,316 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,316 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:48,316 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,316 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,316 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,316 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS srcbucket2 2018-01-05T11:28:48,316 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112848_628809ca-3257-490b-9da0-423c23b93c43): DROP TABLE IF EXISTS srcbucket2 2018-01-05T11:28:48,316 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,316 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS srcbucket2 2018-01-05T11:28:48,316 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:48,316 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,316 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,317 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,317 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,317 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:48,317 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,317 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket2 2018-01-05T11:28:48,317 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket2 2018-01-05T11:28:48,317 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,317 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,317 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:48,317 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,317 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:48,317 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,317 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:48,317 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:28:48,317 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112848_628809ca-3257-490b-9da0-423c23b93c43); Time taken: 0.001 seconds 2018-01-05T11:28:48,317 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:48,317 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,317 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112848_628809ca-3257-490b-9da0-423c23b93c43): DROP TABLE IF EXISTS srcbucket2 2018-01-05T11:28:48,318 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,318 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS srcbucket2 2018-01-05T11:28:48,318 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:48,318 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,318 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,318 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,318 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,318 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:48,318 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,318 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket2 2018-01-05T11:28:48,318 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket2 2018-01-05T11:28:48,319 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,319 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,319 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table srcbucket2 not found: default.srcbucket2 table not found 2018-01-05T11:28:48,319 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,319 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket2 2018-01-05T11:28:48,319 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket2 2018-01-05T11:28:48,319 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,319 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,319 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,319 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,319 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS srcbucket2 2018-01-05T11:28:48,319 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:48,319 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,319 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,319 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:48,319 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=0} 2018-01-05T11:28:48,319 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112848_628809ca-3257-490b-9da0-423c23b93c43); Time taken: 0.002 seconds 2018-01-05T11:28:48,319 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:48,319 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,319 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:48,319 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,319 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query DROP TABLE IF EXISTS srcbucket2 2018-01-05T11:28:48,319 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.003 seconds 2018-01-05T11:28:48,320 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:48,320 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:48,320 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:48,320 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:48,320 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,320 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:48,320 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,320 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,320 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,320 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: CREATE TABLE srcbucket_tmp (key INT, value STRING) 2018-01-05T11:28:48,320 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112848_41430d69-5a47-4519-b474-fd27e7f28664): CREATE TABLE srcbucket_tmp (key INT, value STRING) 2018-01-05T11:28:48,320 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,320 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: CREATE TABLE srcbucket_tmp (key INT, value STRING) 2018-01-05T11:28:48,320 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:48,320 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,320 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,320 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,320 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,320 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:48,320 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:28:48,320 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Creating table default.srcbucket_tmp position=13 2018-01-05T11:28:48,320 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,320 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_database: default 2018-01-05T11:28:48,320 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_database: default 2018-01-05T11:28:48,325 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:48,325 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,325 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,325 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:48,325 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: validation start 2018-01-05T11:28:48,325 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:28:48,325 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Not a partition. 2018-01-05T11:28:48,325 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,325 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:48,325 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,325 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:48,325 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getDatabase_(String, )=5, flushCache_()=0} 2018-01-05T11:28:48,325 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112848_41430d69-5a47-4519-b474-fd27e7f28664); Time taken: 0.005 seconds 2018-01-05T11:28:48,325 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:48,325 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,325 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112848_41430d69-5a47-4519-b474-fd27e7f28664): CREATE TABLE srcbucket_tmp (key INT, value STRING) 2018-01-05T11:28:48,326 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,326 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: CREATE TABLE srcbucket_tmp (key INT, value STRING) 2018-01-05T11:28:48,326 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: CREATETABLE 2018-01-05T11:28:48,326 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: database:default 2018-01-05T11:28:48,326 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@srcbucket_tmp 2018-01-05T11:28:48,326 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,326 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,326 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,326 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,326 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:48,326 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Found class for org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe 2018-01-05T11:28:48,327 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: creating table default.srcbucket_tmp on file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp 2018-01-05T11:28:48,327 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,327 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: create_table: Table(tableName:srcbucket_tmp, dbName:default, owner:hive_test_user, createTime:1515180528, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:int, comment:null), FieldSchema(name:value, type:string, comment:null)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, numFiles=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:48,327 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=create_table: Table(tableName:srcbucket_tmp, dbName:default, owner:hive_test_user, createTime:1515180528, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:int, comment:null), FieldSchema(name:value, type:string, comment:null)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, numFiles=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:48,328 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hive-site.xml 2018-01-05T11:28:48,328 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hivemetastore-site.xml 2018-01-05T11:28:48,328 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Unable to find config file metastore-site.xml 2018-01-05T11:28:48,328 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file null 2018-01-05T11:28:48,336 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:48,337 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: Location: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp specified for non-external table:srcbucket_tmp 2018-01-05T11:28:48,337 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp 2018-01-05T11:28:48,366 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,368 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,369 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,369 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@srcbucket_tmp) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@srcbucket_tmp) Type=TABLE WriteType=DDL_NO_LOCK is present 2018-01-05T11:28:48,369 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,369 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,369 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: CREATE TABLE srcbucket_tmp (key INT, value STRING) 2018-01-05T11:28:48,369 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: CREATETABLE 2018-01-05T11:28:48,369 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: database:default 2018-01-05T11:28:48,369 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@srcbucket_tmp 2018-01-05T11:28:48,369 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,369 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,369 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:48,369 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {createTable_(Table, )=42, isCompatibleWith_(Configuration, )=0} 2018-01-05T11:28:48,369 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112848_41430d69-5a47-4519-b474-fd27e7f28664); Time taken: 0.044 seconds 2018-01-05T11:28:48,369 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:48,369 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,369 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:48,369 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,369 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query CREATE TABLE srcbucket_tmp (key INT, value STRING) 2018-01-05T11:28:48,369 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.049 seconds 2018-01-05T11:28:48,369 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:48,369 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:48,369 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:48,369 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:48,369 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,369 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:48,369 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,369 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,369 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,369 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: CREATE TABLE srcbucket2 (key INT, value STRING) CLUSTERED BY (key) INTO 4 BUCKETS STORED AS TEXTFILE 2018-01-05T11:28:48,369 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112848_86056061-4c84-4688-9903-e8e53af0c7bb): CREATE TABLE srcbucket2 (key INT, value STRING) CLUSTERED BY (key) INTO 4 BUCKETS STORED AS TEXTFILE 2018-01-05T11:28:48,369 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,370 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: CREATE TABLE srcbucket2 (key INT, value STRING) CLUSTERED BY (key) INTO 4 BUCKETS STORED AS TEXTFILE 2018-01-05T11:28:48,370 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:48,370 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,370 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,370 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,370 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,370 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:48,370 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:28:48,370 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Creating table default.srcbucket2 position=13 2018-01-05T11:28:48,370 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,370 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_database: default 2018-01-05T11:28:48,370 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_database: default 2018-01-05T11:28:48,371 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:48,371 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,371 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,371 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:48,371 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: validation start 2018-01-05T11:28:48,371 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:28:48,371 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Not a partition. 2018-01-05T11:28:48,371 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,371 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:48,371 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,371 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:48,371 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getDatabase_(String, )=1, flushCache_()=0} 2018-01-05T11:28:48,371 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112848_86056061-4c84-4688-9903-e8e53af0c7bb); Time taken: 0.002 seconds 2018-01-05T11:28:48,371 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:48,371 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,371 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112848_86056061-4c84-4688-9903-e8e53af0c7bb): CREATE TABLE srcbucket2 (key INT, value STRING) CLUSTERED BY (key) INTO 4 BUCKETS STORED AS TEXTFILE 2018-01-05T11:28:48,372 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,372 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: CREATE TABLE srcbucket2 (key INT, value STRING) CLUSTERED BY (key) INTO 4 BUCKETS STORED AS TEXTFILE 2018-01-05T11:28:48,372 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: CREATETABLE 2018-01-05T11:28:48,372 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: database:default 2018-01-05T11:28:48,372 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@srcbucket2 2018-01-05T11:28:48,372 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,372 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,372 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,372 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,372 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:48,373 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Found class for org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe 2018-01-05T11:28:48,373 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: creating table default.srcbucket2 on file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2 2018-01-05T11:28:48,373 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,373 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: create_table: Table(tableName:srcbucket2, dbName:default, owner:hive_test_user, createTime:1515180528, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:int, comment:null), FieldSchema(name:value, type:string, comment:null)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:4, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format=1}), bucketCols:[key], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, numFiles=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:48,373 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=create_table: Table(tableName:srcbucket2, dbName:default, owner:hive_test_user, createTime:1515180528, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:int, comment:null), FieldSchema(name:value, type:string, comment:null)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:4, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format=1}), bucketCols:[key], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, numFiles=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:48,374 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hive-site.xml 2018-01-05T11:28:48,375 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hivemetastore-site.xml 2018-01-05T11:28:48,375 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Unable to find config file metastore-site.xml 2018-01-05T11:28:48,375 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file null 2018-01-05T11:28:48,384 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:48,384 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: Location: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2 specified for non-external table:srcbucket2 2018-01-05T11:28:48,384 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2 2018-01-05T11:28:48,405 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,406 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000020, QueryId=djaiswal_20180105112847_e9c71f6a-79ec-4524-a7a9-89f5a111c83e-12, DagId=12, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-2, Succeeded=true, StartTime=1515180528202, EndTime=1515180528406 2018-01-05T11:28:48,407 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,407 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,407 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@srcbucket2) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@srcbucket2) Type=TABLE WriteType=DDL_NO_LOCK is present 2018-01-05T11:28:48,408 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,408 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,408 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: CREATE TABLE srcbucket2 (key INT, value STRING) CLUSTERED BY (key) INTO 4 BUCKETS STORED AS TEXTFILE 2018-01-05T11:28:48,408 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: CREATETABLE 2018-01-05T11:28:48,408 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: database:default 2018-01-05T11:28:48,408 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@srcbucket2 2018-01-05T11:28:48,408 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,408 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,408 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:48,408 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {createTable_(Table, )=34, isCompatibleWith_(Configuration, )=0} 2018-01-05T11:28:48,408 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112848_86056061-4c84-4688-9903-e8e53af0c7bb); Time taken: 0.037 seconds 2018-01-05T11:28:48,408 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:48,408 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,408 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:48,408 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,408 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query CREATE TABLE srcbucket2 (key INT, value STRING) CLUSTERED BY (key) INTO 4 BUCKETS STORED AS TEXTFILE 2018-01-05T11:28:48,408 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.039 seconds 2018-01-05T11:28:48,408 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:48,408 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:48,408 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:48,408 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:48,408 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,408 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:48,408 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,408 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,408 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,408 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/srcbucket20.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:48,408 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112848_3e5b91e0-3e7e-43d6-b7a1-65dfd9b42f64): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/srcbucket20.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:48,409 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,409 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/srcbucket20.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:48,409 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:48,409 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,409 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,409 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,409 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,409 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:48,409 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.LoadSemanticAnalyzer: file@null@/Users/djaiswal/hive/hive/data/files/srcbucket20.txt 2018-01-05T11:28:48,409 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,409 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,409 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,414 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,414 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,417 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:48,417 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,417 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:48,417 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,417 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:48,417 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=5, flushCache_()=0} 2018-01-05T11:28:48,417 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112848_3e5b91e0-3e7e-43d6-b7a1-65dfd9b42f64); Time taken: 0.009 seconds 2018-01-05T11:28:48,417 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:48,417 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,417 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112848_3e5b91e0-3e7e-43d6-b7a1-65dfd9b42f64): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/srcbucket20.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:48,418 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,418 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/srcbucket20.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:48,418 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: LOAD 2018-01-05T11:28:48,418 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/srcbucket20.txt 2018-01-05T11:28:48,418 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@srcbucket_tmp 2018-01-05T11:28:48,418 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,418 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,418 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,418 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,418 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MOVE] in serial mode 2018-01-05T11:28:48,418 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@207283b4, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:48,418 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:48,418 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,418 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:48,419 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:48,419 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:48,419 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:48,419 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,419 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:48,419 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Loading data to table default.srcbucket_tmp from file:/Users/djaiswal/hive/hive/data/files/srcbucket20.txt 2018-01-05T11:28:48,419 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,420 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:48,420 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:48,420 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:48,420 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:48,420 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,420 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,420 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,421 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,421 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,421 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:48,421 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:48,422 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,422 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:48,422 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,422 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,422 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,427 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,428 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,429 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,429 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,429 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] FileOperations: moving file:/Users/djaiswal/hive/hive/data/files/srcbucket20.txt to file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp (replace = KEEP_EXISTING) 2018-01-05T11:28:48,436 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:48,436 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:48,464 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Moved src: file:/Users/djaiswal/hive/hive/data/files/srcbucket20.txt, to dest: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket20.txt 2018-01-05T11:28:48,465 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,465 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=srcbucket_tmp newtbl=srcbucket_tmp 2018-01-05T11:28:48,465 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=srcbucket_tmp newtbl=srcbucket_tmp 2018-01-05T11:28:48,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,476 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,476 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,476 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@srcbucket_tmp) Type=TABLE WriteType=INSERT because WriteEntity(default@srcbucket_tmp) Type=TABLE WriteType=INSERT is present 2018-01-05T11:28:48,476 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-1:STATS] in serial mode 2018-01-05T11:28:48,476 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@7f56a025, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:48,477 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:48,477 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,477 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:48,477 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:48,477 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:48,477 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:48,477 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,477 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:48,477 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:48,477 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,478 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:48,478 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:48,478 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:48,478 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:48,478 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,478 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,478 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,480 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,480 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,480 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:48,480 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:48,480 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,480 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:48,480 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,480 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,480 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,486 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,487 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,487 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=srcbucket_tmp newtbl=srcbucket_tmp 2018-01-05T11:28:48,487 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=srcbucket_tmp newtbl=srcbucket_tmp 2018-01-05T11:28:48,492 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,501 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,501 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,502 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.srcbucket_tmp stats: [numFiles=1, numRows=0, totalSize=1358, rawDataSize=0] 2018-01-05T11:28:48,502 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,502 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,502 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/srcbucket20.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:48,502 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: LOAD 2018-01-05T11:28:48,502 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/srcbucket20.txt 2018-01-05T11:28:48,502 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@srcbucket_tmp 2018-01-05T11:28:48,502 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,502 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,502 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:48,502 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=15, getTable_(String, String, )=6} 2018-01-05T11:28:48,502 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112848_3e5b91e0-3e7e-43d6-b7a1-65dfd9b42f64); Time taken: 0.085 seconds 2018-01-05T11:28:48,502 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:48,502 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,502 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:48,502 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,502 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/srcbucket20.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:48,502 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.094 seconds 2018-01-05T11:28:48,502 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:48,502 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:48,502 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:48,502 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:48,502 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,502 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:48,502 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,502 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,502 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,502 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/srcbucket21.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:48,502 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112848_496a46e9-a904-41ea-99ce-6363113cc5ff): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/srcbucket21.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:48,503 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,503 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/srcbucket21.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:48,503 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:48,503 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,503 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,503 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,503 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,503 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:48,504 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.LoadSemanticAnalyzer: file@null@/Users/djaiswal/hive/hive/data/files/srcbucket21.txt 2018-01-05T11:28:48,504 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,504 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,504 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,511 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,512 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,515 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:48,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,515 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:48,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,515 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:48,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=8, flushCache_()=0} 2018-01-05T11:28:48,515 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112848_496a46e9-a904-41ea-99ce-6363113cc5ff); Time taken: 0.013 seconds 2018-01-05T11:28:48,515 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:48,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,515 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112848_496a46e9-a904-41ea-99ce-6363113cc5ff): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/srcbucket21.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:48,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,517 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/srcbucket21.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:48,517 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: LOAD 2018-01-05T11:28:48,517 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/srcbucket21.txt 2018-01-05T11:28:48,517 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@srcbucket_tmp 2018-01-05T11:28:48,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,517 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MOVE] in serial mode 2018-01-05T11:28:48,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@295692fd, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:48,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:48,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,517 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:48,517 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:48,517 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:48,517 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:48,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:48,518 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Loading data to table default.srcbucket_tmp from file:/Users/djaiswal/hive/hive/data/files/srcbucket21.txt 2018-01-05T11:28:48,518 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,519 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:48,519 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:48,519 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:48,519 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:48,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,521 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,522 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,522 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:48,522 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:48,522 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,522 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:48,522 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,522 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,522 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,527 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,528 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,530 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,530 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,530 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,535 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,535 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,535 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] FileOperations: moving file:/Users/djaiswal/hive/hive/data/files/srcbucket21.txt to file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp (replace = KEEP_EXISTING) 2018-01-05T11:28:48,536 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:48,536 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:48,568 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Moved src: file:/Users/djaiswal/hive/hive/data/files/srcbucket21.txt, to dest: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket21.txt 2018-01-05T11:28:48,569 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,569 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=srcbucket_tmp newtbl=srcbucket_tmp 2018-01-05T11:28:48,569 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=srcbucket_tmp newtbl=srcbucket_tmp 2018-01-05T11:28:48,574 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,579 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,579 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,579 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@srcbucket_tmp) Type=TABLE WriteType=INSERT because WriteEntity(default@srcbucket_tmp) Type=TABLE WriteType=INSERT is present 2018-01-05T11:28:48,579 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-1:STATS] in serial mode 2018-01-05T11:28:48,580 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@58b69176, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:48,580 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:48,580 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,580 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:48,580 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:48,580 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:48,580 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:48,580 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,580 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:48,580 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:48,580 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,581 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:48,581 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:48,581 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:48,581 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:48,581 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,581 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,581 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,582 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,582 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,582 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:48,582 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:48,582 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,582 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:48,582 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,582 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,583 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,586 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,586 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,587 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=srcbucket_tmp newtbl=srcbucket_tmp 2018-01-05T11:28:48,587 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=srcbucket_tmp newtbl=srcbucket_tmp 2018-01-05T11:28:48,590 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,595 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,596 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,596 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.srcbucket_tmp stats: [numFiles=2, numRows=0, totalSize=2750, rawDataSize=0] 2018-01-05T11:28:48,596 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,596 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,596 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/srcbucket21.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:48,596 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: LOAD 2018-01-05T11:28:48,596 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/srcbucket21.txt 2018-01-05T11:28:48,596 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@srcbucket_tmp 2018-01-05T11:28:48,596 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,596 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,596 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:48,596 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=9, getTable_(String, String, )=4} 2018-01-05T11:28:48,596 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112848_496a46e9-a904-41ea-99ce-6363113cc5ff); Time taken: 0.081 seconds 2018-01-05T11:28:48,596 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:48,596 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,596 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:48,596 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,596 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/srcbucket21.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:48,596 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.094 seconds 2018-01-05T11:28:48,596 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:48,596 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:48,596 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:48,596 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:48,596 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,596 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:48,596 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,596 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,596 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,596 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/srcbucket22.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:48,596 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112848_71509503-68e9-4fe0-8163-7a0db3a47b62): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/srcbucket22.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:48,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/srcbucket22.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:48,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:48,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,597 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:48,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.LoadSemanticAnalyzer: file@null@/Users/djaiswal/hive/hive/data/files/srcbucket22.txt 2018-01-05T11:28:48,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,597 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,597 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,602 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,602 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,604 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:48,604 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,604 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:48,604 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,605 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:48,605 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=5, flushCache_()=0} 2018-01-05T11:28:48,605 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112848_71509503-68e9-4fe0-8163-7a0db3a47b62); Time taken: 0.008 seconds 2018-01-05T11:28:48,605 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:48,605 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,605 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112848_71509503-68e9-4fe0-8163-7a0db3a47b62): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/srcbucket22.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:48,606 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,606 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/srcbucket22.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:48,606 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: LOAD 2018-01-05T11:28:48,606 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/srcbucket22.txt 2018-01-05T11:28:48,606 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@srcbucket_tmp 2018-01-05T11:28:48,606 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,606 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,606 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,606 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,606 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MOVE] in serial mode 2018-01-05T11:28:48,606 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@74cca90, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:48,606 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:48,606 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,606 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:48,606 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:48,606 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:48,606 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:48,606 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,606 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:48,606 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Loading data to table default.srcbucket_tmp from file:/Users/djaiswal/hive/hive/data/files/srcbucket22.txt 2018-01-05T11:28:48,607 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,607 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:48,607 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:48,607 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:48,607 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:48,608 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,608 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,608 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,609 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,609 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,609 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:48,609 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:48,609 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,609 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:48,609 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,609 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,609 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,612 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,613 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,614 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,614 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,614 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,617 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,618 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,618 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] FileOperations: moving file:/Users/djaiswal/hive/hive/data/files/srcbucket22.txt to file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp (replace = KEEP_EXISTING) 2018-01-05T11:28:48,618 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:48,618 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:48,643 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Moved src: file:/Users/djaiswal/hive/hive/data/files/srcbucket22.txt, to dest: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket22.txt 2018-01-05T11:28:48,643 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,643 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=srcbucket_tmp newtbl=srcbucket_tmp 2018-01-05T11:28:48,643 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=srcbucket_tmp newtbl=srcbucket_tmp 2018-01-05T11:28:48,646 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,651 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,652 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,652 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@srcbucket_tmp) Type=TABLE WriteType=INSERT because WriteEntity(default@srcbucket_tmp) Type=TABLE WriteType=INSERT is present 2018-01-05T11:28:48,652 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-1:STATS] in serial mode 2018-01-05T11:28:48,652 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@6b7b046, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:48,652 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:48,652 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,652 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:48,652 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:48,652 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:48,652 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:48,652 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,652 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:48,652 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:48,653 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,653 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:48,653 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:48,653 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:48,653 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:48,653 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,653 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,653 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,654 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,655 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,655 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:48,655 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:48,655 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,655 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:48,655 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,655 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,655 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,660 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,661 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,661 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,661 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=srcbucket_tmp newtbl=srcbucket_tmp 2018-01-05T11:28:48,661 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=srcbucket_tmp newtbl=srcbucket_tmp 2018-01-05T11:28:48,665 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,672 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,673 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,673 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.srcbucket_tmp stats: [numFiles=3, numRows=0, totalSize=4200, rawDataSize=0] 2018-01-05T11:28:48,673 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,673 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,673 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/srcbucket22.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:48,673 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: LOAD 2018-01-05T11:28:48,673 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/srcbucket22.txt 2018-01-05T11:28:48,673 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@srcbucket_tmp 2018-01-05T11:28:48,673 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,673 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,673 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:48,673 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=12, getTable_(String, String, )=6} 2018-01-05T11:28:48,673 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112848_71509503-68e9-4fe0-8163-7a0db3a47b62); Time taken: 0.068 seconds 2018-01-05T11:28:48,673 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:48,673 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,673 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:48,673 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,673 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/srcbucket22.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:48,673 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.077 seconds 2018-01-05T11:28:48,673 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:48,673 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:48,673 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:48,673 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:48,673 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,673 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:48,673 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,673 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,673 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,673 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/srcbucket23.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:48,673 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112848_9dbae9cc-90b3-4d7d-8c7f-126ef35e0806): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/srcbucket23.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:48,674 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,674 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/srcbucket23.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:48,674 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:48,674 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,674 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,674 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,674 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,674 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:48,674 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.LoadSemanticAnalyzer: file@null@/Users/djaiswal/hive/hive/data/files/srcbucket23.txt 2018-01-05T11:28:48,674 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,674 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,674 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,679 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,679 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,681 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:48,681 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,681 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:48,682 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,682 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:48,682 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=5, flushCache_()=0} 2018-01-05T11:28:48,682 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112848_9dbae9cc-90b3-4d7d-8c7f-126ef35e0806); Time taken: 0.009 seconds 2018-01-05T11:28:48,682 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:48,682 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,682 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112848_9dbae9cc-90b3-4d7d-8c7f-126ef35e0806): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/srcbucket23.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:48,683 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,683 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/srcbucket23.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:48,683 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: LOAD 2018-01-05T11:28:48,683 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/srcbucket23.txt 2018-01-05T11:28:48,683 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@srcbucket_tmp 2018-01-05T11:28:48,683 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,683 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,683 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,683 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,683 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MOVE] in serial mode 2018-01-05T11:28:48,683 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@24cc60ef, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:48,683 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:48,683 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,683 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:48,683 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:48,683 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:48,683 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:48,683 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,683 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:48,683 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Loading data to table default.srcbucket_tmp from file:/Users/djaiswal/hive/hive/data/files/srcbucket23.txt 2018-01-05T11:28:48,684 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,684 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:48,685 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:48,685 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:48,685 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:48,685 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,685 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,685 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,686 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,686 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,687 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:48,687 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:48,688 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,688 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:48,688 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,688 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,688 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,693 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,694 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,694 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,694 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,697 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,698 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,698 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] FileOperations: moving file:/Users/djaiswal/hive/hive/data/files/srcbucket23.txt to file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp (replace = KEEP_EXISTING) 2018-01-05T11:28:48,698 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:48,698 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:48,750 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Moved src: file:/Users/djaiswal/hive/hive/data/files/srcbucket23.txt, to dest: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket23.txt 2018-01-05T11:28:48,750 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,750 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=srcbucket_tmp newtbl=srcbucket_tmp 2018-01-05T11:28:48,750 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=srcbucket_tmp newtbl=srcbucket_tmp 2018-01-05T11:28:48,756 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@srcbucket_tmp) Type=TABLE WriteType=INSERT because WriteEntity(default@srcbucket_tmp) Type=TABLE WriteType=INSERT is present 2018-01-05T11:28:48,764 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-1:STATS] in serial mode 2018-01-05T11:28:48,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@53de80b9, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:48,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:48,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,764 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:48,764 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:48,764 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:48,764 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:48,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:48,764 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:48,765 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,765 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:48,765 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:48,765 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:48,765 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:48,766 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,766 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,766 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,767 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,768 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,768 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:48,768 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:48,768 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,768 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:48,768 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,768 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,768 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,777 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,778 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,779 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,779 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=srcbucket_tmp newtbl=srcbucket_tmp 2018-01-05T11:28:48,779 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=srcbucket_tmp newtbl=srcbucket_tmp 2018-01-05T11:28:48,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,797 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,797 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,797 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.srcbucket_tmp stats: [numFiles=4, numRows=0, totalSize=5812, rawDataSize=0] 2018-01-05T11:28:48,797 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,798 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,798 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/srcbucket23.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:48,798 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: LOAD 2018-01-05T11:28:48,798 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/srcbucket23.txt 2018-01-05T11:28:48,798 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@srcbucket_tmp 2018-01-05T11:28:48,798 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,798 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,798 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:48,798 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=18, getTable_(String, String, )=10} 2018-01-05T11:28:48,798 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112848_9dbae9cc-90b3-4d7d-8c7f-126ef35e0806); Time taken: 0.116 seconds 2018-01-05T11:28:48,798 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:48,798 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,798 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:48,798 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,798 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/srcbucket23.txt" INTO TABLE srcbucket_tmp 2018-01-05T11:28:48,798 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.125 seconds 2018-01-05T11:28:48,798 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:48,798 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:48,798 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:48,798 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:48,798 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,798 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:48,798 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,798 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,798 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,798 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: INSERT INTO srcbucket2 SELECT * FROM srcbucket_tmp 2018-01-05T11:28:48,798 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39): INSERT INTO srcbucket2 SELECT * FROM srcbucket_tmp 2018-01-05T11:28:48,798 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,798 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: INSERT INTO srcbucket2 SELECT * FROM srcbucket_tmp 2018-01-05T11:28:48,800 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:48,800 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,800 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,800 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,800 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,800 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:48,800 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:28:48,801 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,801 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket2 2018-01-05T11:28:48,801 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket2 2018-01-05T11:28:48,805 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,806 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,806 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:48,806 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Completed phase 1 of Semantic Analysis 2018-01-05T11:28:48,806 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:28:48,806 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,806 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,806 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,813 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,814 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,814 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:28:48,814 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:28:48,814 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,814 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket2 2018-01-05T11:28:48,814 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket2 2018-01-05T11:28:48,823 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,823 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,823 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Completed getting MetaData in Semantic Analysis 2018-01-05T11:28:48,826 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,827 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,827 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_not_null_constraints : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,827 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_not_null_constraints : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,842 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,842 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,842 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,842 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,842 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,862 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,862 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,863 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_unique_constraints : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,863 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_unique_constraints : db=default tbl=srcbucket_tmp 2018-01-05T11:28:48,874 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,874 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,874 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,874 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=srcbucket_tmp 2018-01-05T11:28:48,875 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=srcbucket_tmp 2018-01-05T11:28:48,911 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:48,911 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:48,912 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:28:48,912 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] calcite.sql2rel: Plan after trimming unused fields HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.srcbucket_tmp]], table:alias=[srcbucket_tmp]) 2018-01-05T11:28:48,982 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Plan before removing subquery: HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.srcbucket_tmp]], table:alias=[srcbucket_tmp]) 2018-01-05T11:28:48,982 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Plan just after removing subquery: HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.srcbucket_tmp]], table:alias=[srcbucket_tmp]) 2018-01-05T11:28:48,983 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Plan after decorrelation: HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.srcbucket_tmp]], table:alias=[srcbucket_tmp]) 2018-01-05T11:28:48,983 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,983 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,983 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,983 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,983 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,984 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,984 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,984 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,984 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,984 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:48,984 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,004 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,004 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,004 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,004 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,005 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] calcite.sql2rel: Plan after trimming unused fields HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.srcbucket_tmp]], table:alias=[srcbucket_tmp]) 2018-01-05T11:28:49,005 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,005 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,005 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,005 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,005 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,005 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,006 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,006 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,006 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,006 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,006 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,006 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,007 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,007 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,007 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,007 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] translator.PlanModifierForASTConv: Original plan for PlanModifier HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.srcbucket_tmp]], table:alias=[srcbucket_tmp]) 2018-01-05T11:28:49,007 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] translator.PlanModifierForASTConv: Plan after nested convertOpTree HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.srcbucket_tmp]], table:alias=[srcbucket_tmp]) 2018-01-05T11:28:49,008 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] translator.PlanModifierForASTConv: Plan after propagating order HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.srcbucket_tmp]], table:alias=[srcbucket_tmp]) 2018-01-05T11:28:49,008 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] translator.PlanModifierForASTConv: Plan after fixTopOBSchema HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.srcbucket_tmp]], table:alias=[srcbucket_tmp]) 2018-01-05T11:28:49,008 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] translator.PlanModifierForASTConv: Final plan after modifier HiveProject(srcbucket_tmp.key=[$0], srcbucket_tmp.value=[$1]) HiveTableScan(table=[[default.srcbucket_tmp]], table:alias=[srcbucket_tmp]) 2018-01-05T11:28:49,008 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:28:49,008 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:49,008 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:49,008 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:49,015 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:49,016 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:49,016 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:28:49,016 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:28:49,016 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:49,016 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket2 2018-01-05T11:28:49,016 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket2 2018-01-05T11:28:49,021 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:49,021 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:49,022 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Table Plan for srcbucket_tmp TS[0] 2018-01-05T11:28:49,022 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: RR before GB srcbucket_tmp{(key,key: int)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB srcbucket_tmp{(key,key: int)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:28:49,022 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (. (tok_table_or_col srcbucket_tmp) key) srcbucket_tmp.key) (tok_selexpr (. (tok_table_or_col srcbucket_tmp) value) srcbucket_tmp.value)) 2018-01-05T11:28:49,022 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: genSelectPlan: input = srcbucket_tmp{(key,key: int)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} starRr = null 2018-01-05T11:28:49,022 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Select Plan row schema: null{(srcbucket_tmp.key,_col0: int)(srcbucket_tmp.value,_col1: string)} 2018-01-05T11:28:49,022 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:28:49,022 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/.hive-staging_hive_2018-01-05_11-28-48_798_103036370913189016-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2 2018-01-05T11:28:49,022 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/.hive-staging_hive_2018-01-05_11-28-48_798_103036370913189016-1 2018-01-05T11:28:49,048 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/-mr-10001 2018-01-05T11:28:49,048 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2 row schema: null{(srcbucket_tmp.key,_col0: int)(srcbucket_tmp.value,_col1: string)} 2018-01-05T11:28:49,048 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:49,048 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket2 2018-01-05T11:28:49,048 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket2 2018-01-05T11:28:49,052 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:49,053 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:49,053 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Generate an operator pipeline to autogather column stats for table default.srcbucket2 in query INSERT INTO srcbucket2 SELECT * FROM srcbucket_tmp 2018-01-05T11:28:49,053 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: analyze table `default`.`srcbucket2` compute statistics for columns 2018-01-05T11:28:49,053 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:49,054 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:49,054 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket2 2018-01-05T11:28:49,054 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket2 2018-01-05T11:28:49,061 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:49,062 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:49,062 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: select compute_stats(`key`, 'hll') , compute_stats(`value`, 'hll') from `default`.`srcbucket2` 2018-01-05T11:28:49,062 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: select compute_stats(`key`, 'hll') , compute_stats(`value`, 'hll') from `default`.`srcbucket2` 2018-01-05T11:28:49,063 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:49,063 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:49,063 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:49,064 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:28:49,064 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:49,064 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket2 2018-01-05T11:28:49,064 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket2 2018-01-05T11:28:49,068 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:49,069 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:49,069 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:28:49,069 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:28:49,080 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Table Plan for srcbucket2 TS[0] 2018-01-05T11:28:49,081 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:49,081 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:49,081 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: RR before GB srcbucket2{(key,key: int)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB {((tok_function compute_stats (tok_table_or_col key) 'hll'),_col0: struct)((tok_function compute_stats (tok_table_or_col value) 'hll'),_col1: struct)} 2018-01-05T11:28:49,081 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (tok_function compute_stats (tok_table_or_col key) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col value) 'hll'))) 2018-01-05T11:28:49,081 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: genSelectPlan: input = {((tok_function compute_stats (tok_table_or_col key) 'hll'),_col0: struct)((tok_function compute_stats (tok_table_or_col value) 'hll'),_col1: struct)} starRr = srcbucket2{(key,key: int)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:28:49,081 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Select Plan row schema: null{(_c0,_col0: struct)(_c1,_col1: struct)} 2018-01-05T11:28:49,081 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:28:49,081 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-49_053_5953809258100630008-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-49_053_5953809258100630008-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-49_053_5953809258100630008-1/-mr-10000 2018-01-05T11:28:49,081 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-49_053_5953809258100630008-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-49_053_5953809258100630008-1 2018-01-05T11:28:49,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-49_053_5953809258100630008-1/-mr-10002 2018-01-05T11:28:49,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-49_053_5953809258100630008-1/-mr-10000 row schema: null{(_c0,_col0: struct)(_c1,_col1: struct)} 2018-01-05T11:28:49,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Body Plan for Query Block null 2018-01-05T11:28:49,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:28:49,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Body Plan for Query Block null 2018-01-05T11:28:49,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:28:49,104 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: CBO Succeeded; optimized logical plan. 2018-01-05T11:28:49,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Before logical optimization TS[0]-SEL[1]-RS[2]-SEL[3]-FS[4] -SEL[1] 2018-01-05T11:28:49,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] lineage.Generator: Time taken for lineage transform=1 2018-01-05T11:28:49,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,105 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for FS(4) 2018-01-05T11:28:49,105 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for FS(6) 2018-01-05T11:28:49,105 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(5) 2018-01-05T11:28:49,105 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for GBY(4) 2018-01-05T11:28:49,105 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for RS(3) 2018-01-05T11:28:49,105 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for GBY(2) 2018-01-05T11:28:49,105 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(1) 2018-01-05T11:28:49,105 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(3) 2018-01-05T11:28:49,105 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for RS(2) 2018-01-05T11:28:49,105 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(1) 2018-01-05T11:28:49,105 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:28:49,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.SimplePredicatePushDown: After PPD: TS[0]-SEL[1]-RS[2]-SEL[3]-FS[4] -SEL[1] 2018-01-05T11:28:49,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 3 key:[] 2018-01-05T11:28:49,106 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 3 oldColExprMap: {VALUE._col0=Column[_col0], VALUE._col1=Column[_col1]} 2018-01-05T11:28:49,106 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 3 newColExprMap: {VALUE._col0=Column[_col0], VALUE._col1=Column[_col1]} 2018-01-05T11:28:49,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 2 key:[] 2018-01-05T11:28:49,106 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 2 oldColExprMap: {VALUE._col0=Column[_col0], VALUE._col1=Column[_col1]} 2018-01-05T11:28:49,106 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 2 newColExprMap: {VALUE._col0=Column[_col0], VALUE._col1=Column[_col1]} 2018-01-05T11:28:49,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.IdentityProjectRemover: Identity project remover optimization removed : SEL[5] 2018-01-05T11:28:49,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: After logical optimization TS[0]-SEL[1]-RS[2]-SEL[3]-FS[4] -SEL[1] 2018-01-05T11:28:49,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:49,107 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=srcbucket_tmp 2018-01-05T11:28:49,107 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=srcbucket_tmp 2018-01-05T11:28:49,110 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.266311ms + 0.005674ms, the query is [select "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "TAB_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (...)] 2018-01-05T11:28:49,111 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:49,115 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:49,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:49,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting stats (Num rows: 1 Data size: 188 Basic stats: COMPLETE Column stats: NONE) on TS[0] 2018-01-05T11:28:49,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (srcbucket_tmp): numRows: 1 dataSize: 188 basicStatsState: COMPLETE colStatsState: NONE colStats: {value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: true isEstimated: true, key= colName: key colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: true isEstimated: true} 2018-01-05T11:28:49,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting stats (Num rows: 1 Data size: 188 Basic stats: COMPLETE Column stats: NONE) on SEL[1] 2018-01-05T11:28:49,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [1] STATS-SEL[1]: numRows: 1 dataSize: 188 basicStatsState: COMPLETE colStatsState: NONE colStats: {value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: true isEstimated: true, key= colName: key colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: true isEstimated: true} 2018-01-05T11:28:49,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 188 Basic stats: COMPLETE Column stats: NONE) on RS[2] 2018-01-05T11:28:49,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-RS[2]: numRows: 1 dataSize: 188 basicStatsState: COMPLETE colStatsState: NONE colStats: {value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: true isEstimated: true, key= colName: key colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: true isEstimated: true} 2018-01-05T11:28:49,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting stats (Num rows: 1 Data size: 188 Basic stats: COMPLETE Column stats: NONE) on SEL[3] 2018-01-05T11:28:49,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [1] STATS-SEL[3]: numRows: 1 dataSize: 188 basicStatsState: COMPLETE colStatsState: NONE colStats: {value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: true isEstimated: true, key= colName: key colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: true isEstimated: true} 2018-01-05T11:28:49,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-FS[4]: numRows: 1 dataSize: 188 basicStatsState: COMPLETE colStatsState: NONE colStats: {value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: true isEstimated: true, key= colName: key colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: true isEstimated: true} 2018-01-05T11:28:49,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting stats (Num rows: 1 Data size: 188 Basic stats: COMPLETE Column stats: NONE) on SEL[1] 2018-01-05T11:28:49,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [1] STATS-SEL[1]: numRows: 1 dataSize: 188 basicStatsState: COMPLETE colStatsState: NONE colStats: {value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: true isEstimated: true, key= colName: key colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: true isEstimated: true} 2018-01-05T11:28:49,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[2]: inputSize: 188 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:28:49,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [Case 1] STATS-GBY[2]: cardinality: 1 2018-01-05T11:28:49,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 1052 Basic stats: COMPLETE Column stats: NONE) on GBY[2] 2018-01-05T11:28:49,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-GBY[2]: numRows: 1 dataSize: 1052 basicStatsState: COMPLETE colStatsState: NONE colStats: {_col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: true isEstimated: true, key= colName: key colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: true isEstimated: true} 2018-01-05T11:28:49,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 1052 Basic stats: COMPLETE Column stats: NONE) on RS[3] 2018-01-05T11:28:49,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-RS[3]: numRows: 1 dataSize: 1052 basicStatsState: COMPLETE colStatsState: NONE colStats: {_col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: true isEstimated: true, key= colName: key colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: true isEstimated: true} 2018-01-05T11:28:49,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[4]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:28:49,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [Case 7] STATS-GBY[4]: cardinality: 0 2018-01-05T11:28:49,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[4]: Equals 0 in number of rows. 0 rows will be set to 1 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 1068 Basic stats: COMPLETE Column stats: NONE) on GBY[4] 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-GBY[4]: numRows: 1 dataSize: 1068 basicStatsState: COMPLETE colStatsState: NONE colStats: {_col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: true isEstimated: true, key= colName: key colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: true isEstimated: true} 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-FS[6]: numRows: 1 dataSize: 1068 basicStatsState: COMPLETE colStatsState: NONE colStats: {_col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: true isEstimated: true, key= colName: key colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: true isEstimated: true} 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on TS[0] 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on SEL[1] 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[2] 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on SEL[3] 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FileSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on FS[4] 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on SEL[1] 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[2] 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[3] 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[4] 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FileSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on FS[6] 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,117 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.SetReducerParallelism: Number of reducers determined to be: 4 2018-01-05T11:28:49,117 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: FS, 4 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: FS, 6 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: GBY, 4 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: RS, 3 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: GBY, 2 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: SEL, 1 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: SEL, 3 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: RS, 2 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: SEL, 1 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:28:49,117 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Cycle free: true 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: int|{srcbucket_tmp}key,value: string|{srcbucket_tmp}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{srcbucket_tmp}block__offset__inside__file,INPUT__FILE__NAME: string|{srcbucket_tmp}input__file__name,ROW__ID: struct|{srcbucket_tmp}row__id) 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(_col0: int|{null}srcbucket_tmp.key,_col1: string|{null}srcbucket_tmp.value) 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value]) 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[2] with rs:(VALUE._col0: int|{null}srcbucket_tmp.key,VALUE._col1: string|{null}srcbucket_tmp.value) 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[2] 2018-01-05T11:28:49,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[3] with rs:(_col0: int|{null}srcbucket_tmp.key,_col1: string|{null}srcbucket_tmp.value) 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_2 {} 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[3] 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[VALUE._col0] Column[VALUE._col1]) 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[4] with rs:(key: int|{},value: string|{}) 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_3 {} 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[4] 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: int|{srcbucket2}key,value: string|{srcbucket2}value) 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_3 {} 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0] Column[_col1]) 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{}) 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: TS[0] 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: RS[2] 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp of table srcbucket_tmp 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: RS[2] 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: SEL[3] 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: FS[4] 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding reduce work (Reducer 2) for SEL[3] 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Setting up reduce sink: RS[2] with following reduce work: Reducer 2 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Removing RS[2] as parent from SEL[3] 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: FS[4] 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: SEL[3] 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: RS[3] 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: RS[3] 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: GBY[4] 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: FS[6] 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding reduce work (Reducer 3) for GBY[4] 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Setting up reduce sink: RS[3] with following reduce work: Reducer 3 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Removing RS[3] as parent from GBY[4] 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: FS[6] 2018-01-05T11:28:49,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:49,119 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket2 2018-01-05T11:28:49,119 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket2 2018-01-05T11:28:49,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:49,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:49,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:28:49,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:49,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:49,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:49,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:49,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:49,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:49,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:28:49,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping vectorization 2018-01-05T11:28:49,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:28:49,123 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: llap mode: only 2018-01-05T11:28:49,123 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:28:49,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'key' 2018-01-05T11:28:49,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'value' 2018-01-05T11:28:49,123 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:49,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@5de4f44e can be done in LLAP 2018-01-05T11:28:49,123 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Reducer 2 2018-01-05T11:28:49,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(key, 'hll')' 2018-01-05T11:28:49,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'key' 2018-01-05T11:28:49,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:49,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(value, 'hll')' 2018-01-05T11:28:49,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'value' 2018-01-05T11:28:49,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:49,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking '_col0' 2018-01-05T11:28:49,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking '_col1' 2018-01-05T11:28:49,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:28:49,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col1' 2018-01-05T11:28:49,123 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:49,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@36e7a455 can be done in LLAP 2018-01-05T11:28:49,123 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Reducer 3 2018-01-05T11:28:49,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col0)' 2018-01-05T11:28:49,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:28:49,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col1)' 2018-01-05T11:28:49,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col1' 2018-01-05T11:28:49,123 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:49,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@627ceaf can be done in LLAP 2018-01-05T11:28:49,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,123 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.HiveStatsUtils: Error requested is 20.0% 2018-01-05T11:28:49,123 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.HiveStatsUtils: Choosing 16 bit vectors.. 2018-01-05T11:28:49,123 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Completed plan generation 2018-01-05T11:28:49,124 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:49,124 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: validation start 2018-01-05T11:28:49,124 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Not a partition. 2018-01-05T11:28:49,124 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,124 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:srcbucket_tmp.key, type:int, comment:null), FieldSchema(name:srcbucket_tmp.value, type:string, comment:null)], properties:null) 2018-01-05T11:28:49,124 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,124 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:49,124 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {getNotNullConstraints_(NotNullConstraintsRequest, )=15, isCompatibleWith_(Configuration, )=1, getTable_(String, String, )=57, flushCache_()=0, getUniqueConstraints_(UniqueConstraintsRequest, )=11, getPrimaryKeys_(PrimaryKeysRequest, )=21, getTableColumnStatistics_(String, String, List, )=9, getForeignKeys_(ForeignKeysRequest, )=37} 2018-01-05T11:28:49,124 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39); Time taken: 0.326 seconds 2018-01-05T11:28:49,124 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:49,124 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,124 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39): INSERT INTO srcbucket2 SELECT * FROM srcbucket_tmp 2018-01-05T11:28:49,125 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,125 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: INSERT INTO srcbucket2 SELECT * FROM srcbucket_tmp 2018-01-05T11:28:49,125 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:28:49,125 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@srcbucket_tmp 2018-01-05T11:28:49,125 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@srcbucket2 2018-01-05T11:28:49,125 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,125 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,125 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,125 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Query ID = djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39 2018-01-05T11:28:49,125 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Total jobs = 1 2018-01-05T11:28:49,125 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,125 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:28:49,125 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-1:MAPRED] in serial mode 2018-01-05T11:28:49,136 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:28:49,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:28:49,160 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:28:49,160 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:28:49,160 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39 2018-01-05T11:28:49,160 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Session is already open 2018-01-05T11:28:49,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:28:49,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,160 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Dag name: INSERT INTO srcbucket2 SELEC...srcbucket_tmp (Stage-1) 2018-01-05T11:28:49,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: DagInfo: {"context":"Hive","description":"\nINSERT INTO srcbucket2 SELECT * FROM srcbucket_tmp"} 2018-01-05T11:28:49,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39 with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:28:49,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,172 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,172 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:28:49,172 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,172 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 3 size: 2.18KB 2018-01-05T11:28:49,186 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-49_053_5953809258100630008-1/-mr-10002 2018-01-05T11:28:49,197 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-49_053_5953809258100630008-1/-mr-10002 2018-01-05T11:28:49,200 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,200 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,213 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,213 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:28:49,214 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,214 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 2 size: 3.17KB 2018-01-05T11:28:49,228 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/-mr-10001 2018-01-05T11:28:49,239 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/-mr-10001 2018-01-05T11:28:49,239 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,241 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,253 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:28:49,253 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,253 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:28:49,254 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,254 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 2.25KB 2018-01-05T11:28:49,259 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp 2018-01-05T11:28:49,259 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,261 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,261 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,261 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=INSERT INTO srcbucket2 SELEC...srcbucket_tmp (Stage-1), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39 } 2018-01-05T11:28:49,261 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:49,262 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:49,262 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:49,264 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: INSERT INTO srcbucket2 SELEC...srcbucket_tmp (Stage-1) 2018-01-05T11:28:49,264 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:28:49,264 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_13, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_13.dot 2018-01-05T11:28:49,265 INFO [IPC Server handler 1 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_13 2018-01-05T11:28:49,265 WARN [IPC Server handler 1 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:49,265 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Running DAG: INSERT INTO srcbucket2 SELEC...srcbucket_tmp (Stage-1), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39 } 2018-01-05T11:28:49,288 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_13, submitTime=1515180529264, queueName=null 2018-01-05T11:28:49,289 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_13_00 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_13_00 [Map 1], Default Resources= 2018-01-05T11:28:49,289 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_13_00 [Map 1] 2018-01-05T11:28:49,289 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:28:49,289 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_13_00 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:49,289 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_13_01 [Reducer 2]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_13_01 [Reducer 2], Default Resources= 2018-01-05T11:28:49,289 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 2 2018-01-05T11:28:49,289 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_13_01 [Reducer 2] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:49,290 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_13_02 [Reducer 3]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_13_02 [Reducer 3], Default Resources= 2018-01-05T11:28:49,290 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_13_02 [Reducer 3] 2018-01-05T11:28:49,290 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 3 2018-01-05T11:28:49,290 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_13_02 [Reducer 3] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:49,290 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:28:49,290 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_13, initTime=1515180529290 2018-01-05T11:28:49,290 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: dag_1515180518813_0001_13 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:28:49,290 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:28:49,290 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_13, startTime=1515180529290 2018-01-05T11:28:49,291 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_13 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:28:49,291 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {srcbucket_tmp={InputName=srcbucket_tmp}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:28:49,291 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: srcbucket_tmp, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:28:49,291 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_13_00 [Map 1] 2018-01-05T11:28:49,291 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_13, dagName=INSERT INTO srcbucket2 SELEC...srcbucket_tmp (Stage-1) 2018-01-05T11:28:49,291 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,291 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,291 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,295 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_13_00 [Map 1] 2018-01-05T11:28:49,295 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_13_00 [Map 1] 2018-01-05T11:28:49,295 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:49,295 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_13_00 [Map 1] 2018-01-05T11:28:49,297 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:28:49,297 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@2e78ec13 2018-01-05T11:28:49,297 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/djaiswal/_tez_scratch_dir/10863ecd-b0a2-4d74-9b26-9f6ef594548e/map.xml 2018-01-05T11:28:49,297 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/djaiswal/_tez_scratch_dir/10863ecd-b0a2-4d74-9b26-9f6ef594548e/map.xml 2018-01-05T11:28:49,297 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/djaiswal/_tez_scratch_dir/10863ecd-b0a2-4d74-9b26-9f6ef594548e/map.xml 2018-01-05T11:28:49,297 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:49,297 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:49,297 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,298 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:49,298 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 2.25KB 2018-01-05T11:28:49,298 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_13_00 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:28:49,298 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: srcbucket_tmp on vertex vertex_1515180518813_0001_13_00 [Map 1] 2018-01-05T11:28:49,299 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to ShuffleVertexManager for vertex_1515180518813_0001_13_01 [Reducer 2] 2018-01-05T11:28:49,299 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:28:49,299 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:49,299 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:28:49,299 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:49,299 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/djaiswal/_tez_scratch_dir/10863ecd-b0a2-4d74-9b26-9f6ef594548e/map.xml 2018-01-05T11:28:49,299 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:49,299 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias srcbucket_tmp 2018-01-05T11:28:49,299 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp 2018-01-05T11:28:49,299 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp 2018-01-05T11:28:49,300 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 0,1 2018-01-05T11:28:49,300 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = key,value 2018-01-05T11:28:49,300 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp 2018-01-05T11:28:49,300 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:49,300 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:49,300 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:49,300 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:49,303 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.05 maxFrac: 0.75 auto: false desiredTaskIput: 104857600 2018-01-05T11:28:49,303 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:28:49,303 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 4 tasks for vertex: vertex_1515180518813_0001_13_01 [Reducer 2] 2018-01-05T11:28:49,303 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_13_01 [Reducer 2] 2018-01-05T11:28:49,303 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_13_01, reconfigureDoneTime=1515180529303, numTasks=4, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:28:49,303 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:VERTEX_INITIALIZED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_13_01, initRequestedTime=1515180529298, initedTime=1515180529303, numTasks=4, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:49,303 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_13_01 [Reducer 2] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:28:49,303 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to ShuffleVertexManager for vertex_1515180518813_0001_13_02 [Reducer 3] 2018-01-05T11:28:49,307 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.05 maxFrac: 0.75 auto: false desiredTaskIput: 104857600 2018-01-05T11:28:49,307 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:28:49,307 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 1 tasks for vertex: vertex_1515180518813_0001_13_02 [Reducer 3] 2018-01-05T11:28:49,307 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_13_02 [Reducer 3] 2018-01-05T11:28:49,307 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_13_02, reconfigureDoneTime=1515180529307, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:28:49,307 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_13_02 [Reducer 3], numAdditionalOutputs=1 2018-01-05T11:28:49,307 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:VERTEX_INITIALIZED]: vertexName=Reducer 3, vertexId=vertex_1515180518813_0001_13_02, initRequestedTime=1515180529303, initedTime=1515180529307, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:49,307 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_13_02 [Reducer 3] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:28:49,354 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Time taken to get FileStatuses: 54 2018-01-05T11:28:49,354 INFO [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total input files to process : 4 2018-01-05T11:28:49,355 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total # of splits generated by getSplits: 4, TimeTaken: 54 2018-01-05T11:28:49,355 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 4 2018-01-05T11:28:49,355 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:49,355 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 4. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:28:49,355 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/djaiswal/_tez_scratch_dir/10863ecd-b0a2-4d74-9b26-9f6ef594548e/map.xml 2018-01-05T11:28:49,355 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/djaiswal/_tez_scratch_dir/10863ecd-b0a2-4d74-9b26-9f6ef594548e/map.xml 2018-01-05T11:28:49,355 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/djaiswal/_tez_scratch_dir/10863ecd-b0a2-4d74-9b26-9f6ef594548e/map.xml 2018-01-05T11:28:49,355 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:49,355 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:49,355 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,356 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:49,356 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 2.25KB 2018-01-05T11:28:49,356 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket23.txt to src new group? true 2018-01-05T11:28:49,356 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket22.txt to src new group? false 2018-01-05T11:28:49,356 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket21.txt to src new group? false 2018-01-05T11:28:49,356 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket20.txt to src new group? false 2018-01-05T11:28:49,356 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:28:49,356 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:28:49,356 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:28:49,356 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 107 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 5812 Original splits: 4 2018-01-05T11:28:49,356 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired numSplits: 1 lengthPerGroup: 5812 numLocations: 1 numSplitsPerLocation: 4 numSplitsInGroup: 4 totalLength: 5812 numOriginalSplits: 4 . Grouping by length: true count: false nodeLocalOnly: false 2018-01-05T11:28:49,356 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Number of splits desired: 1 created: 1 splitsProcessed: 4 2018-01-05T11:28:49,356 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 4 grouped split count is 1, for bucket: 1 2018-01-05T11:28:49,356 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:28:49,356 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: srcbucket_tmp on vertex vertex_1515180518813_0001_13_00 [Map 1] 2018-01-05T11:28:49,358 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_13_00 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:49,358 INFO [App Shared Pool - #0] impl.VertexImpl: Got updated RootInputsSpecs: {srcbucket_tmp=forAllWorkUnits=true, update=[1]} 2018-01-05T11:28:49,358 INFO [App Shared Pool - #0] impl.VertexImpl: Vertex vertex_1515180518813_0001_13_00 [Map 1] parallelism set to 1 2018-01-05T11:28:49,358 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_13_00 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:49,359 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_13_00, initRequestedTime=1515180529291, initedTime=1515180529359, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:49,359 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_13_00 [Map 1] with distanceFromRoot: 0 2018-01-05T11:28:49,359 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_13_00 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:28:49,359 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_13_00, startRequestedTime=1515180529298, startedTime=1515180529359 2018-01-05T11:28:49,359 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_13_00, reconfigureDoneTime=1515180529359, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:28:49,359 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_13_00 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:49,359 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_13_00 for vertex: vertex_1515180518813_0001_13_01 [Reducer 2] numStartedSources: 1 numSources: 1 2018-01-05T11:28:49,359 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_13_01 [Reducer 2] with distanceFromRoot: 1 2018-01-05T11:28:49,359 INFO [App Shared Pool - #0] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:28:49,359 INFO [App Shared Pool - #0] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:28:49,359 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_13_01, startRequestedTime=1515180529359, startedTime=1515180529359 2018-01-05T11:28:49,359 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_13_00 [Map 1] 2018-01-05T11:28:49,359 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_13_01 [Reducer 2] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:49,359 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_13_01 for vertex: vertex_1515180518813_0001_13_02 [Reducer 3] numStartedSources: 1 numSources: 1 2018-01-05T11:28:49,359 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_13_02 [Reducer 3] with distanceFromRoot: 2 2018-01-05T11:28:49,359 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_13_00_000000, scheduledTime=1515180529359, launchTime=1515180529359 2018-01-05T11:28:49,359 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: OnVertexStarted vertex: Reducer 2 with 0 source tasks and 4 pending tasks 2018-01-05T11:28:49,359 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_13_02, startRequestedTime=1515180529359, startedTime=1515180529359 2018-01-05T11:28:49,359 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_13_02 [Reducer 3] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:49,360 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Map 1 in vertex: Reducer 2 numBipartiteSourceTasks: 1 2018-01-05T11:28:49,360 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_13_00 [Map 1] 2018-01-05T11:28:49,360 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: OnVertexStarted vertex: Reducer 3 with 0 source tasks and 1 pending tasks 2018-01-05T11:28:49,360 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Reducer 2 in vertex: Reducer 3 numBipartiteSourceTasks: 4 2018-01-05T11:28:49,360 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_13_00_000000_0, priority=8, capability=, hosts=null 2018-01-05T11:28:49,360 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=8,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:49,360 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=8,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:49,360 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:49,360 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_13_00_000000_0 on hosts=null 2018-01-05T11:28:49,360 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_13_00_000000_0 on hosts=null 2018-01-05T11:28:49,360 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_13_00_000000_0 2018-01-05T11:28:49,360 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:49,360 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #21, task=TaskInfo{task=attempt_1515180518813_0001_13_00_000000_0, priority=8, startTime=0, containerId=null, uniqueId=20, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000021 2018-01-05T11:28:49,360 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_13_00_000000_0; false 2018-01-05T11:28:49,361 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_13_00_000000_0, priority=8, startTime=311549869, containerId=container_111101111_0001_00_000021assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=20, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:49,362 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000021 for node: 10.22.8.182:49818 2018-01-05T11:28:49,364 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 13, name=INSERT INTO srcbucket2 SELEC...srcbucket_tmp (Stage-1), queryId=djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39 2018-01-05T11:28:49,364 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000021, attempt_1515180518813_0001_13_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:49,364 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000021 for node: 10.22.8.182:49818 2018-01-05T11:28:49,364 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000021 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:49,364 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000021, launchTime=1515180529364 2018-01-05T11:28:49,365 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_13_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000021, dagName=INSERT INTO srcbucket2 SELEC...srcbucket_tmp (Stage-1), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{srcbucket_tmp,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{Reducer 2,org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput,4}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180529290, firstAttemptStartTime=1515180529360, currentAttemptStartTime=1515180529364} 2018-01-05T11:28:49,365 INFO [IPC Server handler 0 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000021, QueryId=djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39-13, DagId=13, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180529365 2018-01-05T11:28:49,365 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_13_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000021] on NM: [10.22.8.182:49818] 2018-01-05T11:28:49,365 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:49,365 DEBUG [IPC Server handler 0 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=13} with the ShuffleHandler 2018-01-05T11:28:49,365 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:49,365 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_13_00_000000_0, startTime=1515180529364, containerId=container_111101111_0001_00_000021, nodeId=10.22.8.182:49818 2018-01-05T11:28:49,377 INFO [IPC Server handler 0 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:28:49,377 DEBUG [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/13] 2018-01-05T11:28:49,377 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:49,377 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_13_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:49,377 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_13_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180529360, dagStartTime=1515180529290, withinDagPriority=8, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:28:49,377 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_13_00_000000_0 : ACCEPTED 2018-01-05T11:28:49,377 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_13_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:49,377 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_13_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180529360, dagStartTime=1515180529290, withinDagPriority=8, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:28:49,377 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_13_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:49,377 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_13_00_000000_0 to pre-emption queue 2018-01-05T11:28:49,377 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_13_00_000000_0: true 2018-01-05T11:28:49,378 INFO [TaskCommunicator # 9] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_13_00_000000_0 2018-01-05T11:28:49,378 INFO [TaskCommunicator # 9] GuaranteedTasks: Received response for attempt_1515180518813_0001_13_00_000000_0, true 2018-01-05T11:28:49,378 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: taskOwner hashCode:1387447673 2018-01-05T11:28:49,379 INFO [Task-Executor-2] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : INSERT INTO srcbucket2 SELEC...srcbucket_tmp (Stage-1), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_13_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=srcbucket_tmp, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=Reducer 2, physicalEdgeCount=4, outputClassName=org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput }}, ] 2018-01-05T11:28:49,379 INFO [Task-Executor-2] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=954728448, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:49,379 INFO [Task-Executor-2] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_13_00_000000_0 vertexName: Map 1 2018-01-05T11:28:49,379 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000021, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_13_00_000000_0, eventCount=1 } 2018-01-05T11:28:49,379 DEBUG [TezTR-518813_1_13_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_13 queryId: djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39 instanceType: TaskRunner2Callable 2018-01-05T11:28:49,379 INFO [TezTR-518813_1_13_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_13_00_000000_0 2018-01-05T11:28:49,380 DEBUG [TezTR-518813_1_13_0_0_0] log.PerfLogger: 2018-01-05T11:28:49,380 INFO [I/O Setup 1 Initialize: {Reducer 2}] writers.UnorderedPartitionedKVWriter: Requested BufferSize (tez.runtime.unordered.output.buffer.size-mb) : 100 2018-01-05T11:28:49,381 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:49,381 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:49,385 DEBUG [TezTR-518813_1_13_0_0_0] log.PerfLogger: 2018-01-05T11:28:49,385 INFO [TezTR-518813_1_13_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:49,385 INFO [TezTR-518813_1_13_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:49,385 INFO [TezTR-518813_1_13_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:49,385 INFO [IPC Server handler 18 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_13_00_000000_0 sent events: (0-1). 2018-01-05T11:28:49,385 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:28:49,385 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_13_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:28:49,386 INFO [I/O Setup 0 Initialize: {srcbucket_tmp}] input.MRInput: srcbucket_tmp using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:28:49,386 INFO [I/O Setup 0 Initialize: {srcbucket_tmp}] input.MRInputLegacy: srcbucket_tmp MRInputLegacy deferring initialization 2018-01-05T11:28:49,386 INFO [TezTR-518813_1_13_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:49,386 INFO [TezTR-518813_1_13_0_0_0] resources.MemoryDistributor: InitialRequests=[srcbucket_tmp:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [Reducer 2:OUTPUT:104857600:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput] 2018-01-05T11:28:49,386 INFO [TezTR-518813_1_13_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:49,386 INFO [TezTR-518813_1_13_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:49,386 INFO [TezTR-518813_1_13_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 104857600, TotalRequestedScaled: 5.24288E7, TotalJVMHeap: 954728448, TotalAvailable: 639668060, TotalRequested/TotalJVMHeap:0.11 2018-01-05T11:28:49,386 INFO [TezTR-518813_1_13_0_0_0] resources.MemoryDistributor: Allocations=[srcbucket_tmp:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput:OUTPUT:104857600:104857600] 2018-01-05T11:28:49,386 INFO [TezTR-518813_1_13_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:49,386 INFO [TezTR-518813_1_13_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: srcbucket_tmp being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:28:49,386 INFO [TezTR-518813_1_13_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:28:49,386 INFO [TezTR-518813_1_13_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:28:49,386 INFO [I/O Setup 1 Start: {srcbucket_tmp}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: srcbucket_tmp 2018-01-05T11:28:49,386 INFO [TezTR-518813_1_13_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:49,386 INFO [TezTR-518813_1_13_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_13_00_000000_0 2018-01-05T11:28:49,387 DEBUG [TezTR-518813_1_13_0_0_0] log.PerfLogger: 2018-01-05T11:28:49,387 DEBUG [TezTR-518813_1_13_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_13_00_000000_0_10061 2018-01-05T11:28:49,387 DEBUG [TezTR-518813_1_13_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:49,387 DEBUG [TezTR-518813_1_13_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_13_00_000000_0 2018-01-05T11:28:49,387 INFO [TezTR-518813_1_13_0_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39 2018-01-05T11:28:49,387 DEBUG [TezTR-518813_1_13_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:49,387 DEBUG [TezTR-518813_1_13_0_0_0] log.PerfLogger: 2018-01-05T11:28:49,387 DEBUG [TezTR-518813_1_13_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:49,387 DEBUG [TezTR-518813_1_13_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:49,387 DEBUG [TezTR-518813_1_13_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39_Map 1__MAP_PLAN__ 2018-01-05T11:28:49,387 INFO [TezTR-518813_1_13_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/djaiswal/_tez_scratch_dir/10863ecd-b0a2-4d74-9b26-9f6ef594548e/map.xml 2018-01-05T11:28:49,387 DEBUG [TezTR-518813_1_13_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/djaiswal/_tez_scratch_dir/10863ecd-b0a2-4d74-9b26-9f6ef594548e/map.xml 2018-01-05T11:28:49,387 DEBUG [TezTR-518813_1_13_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/djaiswal/_tez_scratch_dir/10863ecd-b0a2-4d74-9b26-9f6ef594548e/map.xml 2018-01-05T11:28:49,387 DEBUG [TezTR-518813_1_13_0_0_0] log.PerfLogger: 2018-01-05T11:28:49,387 INFO [TezTR-518813_1_13_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:49,387 DEBUG [TezTR-518813_1_13_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,388 DEBUG [TezTR-518813_1_13_0_0_0] log.PerfLogger: 2018-01-05T11:28:49,388 INFO [TezTR-518813_1_13_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 2.25KB 2018-01-05T11:28:49,388 DEBUG [TezTR-518813_1_13_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:49,388 INFO [TezTR-518813_1_13_0_0_0] tez.MapRecordProcessor: The input names are: [srcbucket_tmp] 2018-01-05T11:28:49,388 INFO [TezTR-518813_1_13_0_0_0] input.MRInput: Initialized MRInput: srcbucket_tmp 2018-01-05T11:28:49,389 DEBUG [TezTR-518813_1_13_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 4 2018-01-05T11:28:49,389 INFO [TezTR-518813_1_13_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/djaiswal/_tez_scratch_dir/10863ecd-b0a2-4d74-9b26-9f6ef594548e/map.xml 2018-01-05T11:28:49,389 DEBUG [TezTR-518813_1_13_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:49,389 DEBUG [TezTR-518813_1_13_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket23.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcbucket_tmp, numFiles=4, columns.types=int:string, serialization.format=1, serialization.ddl=struct srcbucket_tmp { i32 key, string value}, columns=key,value, columns.comments=, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, totalSize=5812, column.name.delimiter=,, transient_lastDdlTime=1515180528}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcbucket_tmp, numFiles=4, columns.types=int:string, columns=key,value, serialization.format=1, serialization.ddl=struct srcbucket_tmp { i32 key, string value}, columns.comments=, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, totalSize=5812, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp, column.name.delimiter=,, transient_lastDdlTime=1515180528}, baseFileName=srcbucket_tmp, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcbucket_tmp, numFiles=4, columns.types=int:string, serialization.format=1, serialization.ddl=struct srcbucket_tmp { i32 key, string value}, columns=key,value, columns.comments=, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, totalSize=5812, column.name.delimiter=,, transient_lastDdlTime=1515180528}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcbucket_tmp, numFiles=4, columns.types=int:string, columns=key,value, serialization.format=1, serialization.ddl=struct srcbucket_tmp { i32 key, string value}, columns.comments=, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, totalSize=5812, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp, column.name.delimiter=,, transient_lastDdlTime=1515180528}, baseFileName=srcbucket_tmp, vectorPartitionDesc=null]} 2018-01-05T11:28:49,389 DEBUG [TezTR-518813_1_13_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:49,389 INFO [TezTR-518813_1_13_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:49,389 DEBUG [TezTR-518813_1_13_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:49,390 DEBUG [TezTR-518813_1_13_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:49,390 INFO [TezTR-518813_1_13_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/djaiswal/_tez_scratch_dir/10863ecd-b0a2-4d74-9b26-9f6ef594548e/map.xml 2018-01-05T11:28:49,390 DEBUG [TezTR-518813_1_13_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:49,390 INFO [TezTR-518813_1_13_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_13_00_000000_0 2018-01-05T11:28:49,390 INFO [TezTR-518813_1_13_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:28:49,391 DEBUG [TezTR-518813_1_13_0_0_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:49,391 DEBUG [TezTR-518813_1_13_0_0_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:49,391 DEBUG [TezTR-518813_1_13_0_0_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:49,391 DEBUG [TezTR-518813_1_13_0_0_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:49,391 WARN [TezTR-518813_1_13_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket23.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:28:49,391 DEBUG [TezTR-518813_1_13_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:28:49,391 WARN [TezTR-518813_1_13_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket23.txt:0+1612 2018-01-05T11:28:49,391 DEBUG [TezTR-518813_1_13_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket23.txt 2018-01-05T11:28:49,391 INFO [TezTR-518813_1_13_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket23.txt:0+1612, org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket22.txt:0+1450, org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket21.txt:0+1392, org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket20.txt:0+1358], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=5812} 2018-01-05T11:28:49,391 INFO [TezTR-518813_1_13_0_0_0] input.MRInput: srcbucket_tmp initialized RecordReader from event 2018-01-05T11:28:49,391 DEBUG [TezTR-518813_1_13_0_0_0] tez.MapRecordProcessor: Starting Output: Reducer 2 2018-01-05T11:28:49,392 INFO [TezTR-518813_1_13_0_0_0] writers.BaseUnorderedPartitionedKVWriter: Instantiating Partitioner: [org.apache.tez.mapreduce.partition.MRPartitioner] 2018-01-05T11:28:49,392 INFO [TezTR-518813_1_13_0_0_0] partition.MRPartitioner: Using oldApi, MRpartitionerClass=org.apache.hadoop.hive.ql.io.DefaultHivePartitioner 2018-01-05T11:28:49,397 INFO [TezTR-518813_1_13_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: numBuffers=2, sizePerBuffer=52428800, skipBuffers=false, numPartitions=4, availableMemory=104857600, maxSingleBufferSizeBytes=2147483647, pipelinedShuffle=false, isFinalMergeEnabled=true, numPartitions=4, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:28:49,398 INFO [TezTR-518813_1_13_0_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:28:49,398 INFO [TezTR-518813_1_13_0_0_0] exec.MapOperator: Initializing operator MAP[0] 2018-01-05T11:28:49,398 DEBUG [TezTR-518813_1_13_0_0_0] exec.MapOperator: Initialization Done 0 MAP 2018-01-05T11:28:49,398 DEBUG [TezTR-518813_1_13_0_0_0] exec.MapOperator: Operator 0 MAP initialized 2018-01-05T11:28:49,398 DEBUG [TezTR-518813_1_13_0_0_0] exec.MapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:28:49,398 DEBUG [TezTR-518813_1_13_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,398 DEBUG [TezTR-518813_1_13_0_0_0] exec.MapOperator: Adding alias srcbucket_tmp to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp 2018-01-05T11:28:49,398 DEBUG [TezTR-518813_1_13_0_0_0] exec.MapOperator: dump TS[0] struct 2018-01-05T11:28:49,398 INFO [TezTR-518813_1_13_0_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:28:49,398 INFO [TezTR-518813_1_13_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:49,398 DEBUG [TezTR-518813_1_13_0_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:28:49,398 INFO [TezTR-518813_1_13_0_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:28:49,398 DEBUG [TezTR-518813_1_13_0_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:28:49,398 DEBUG [TezTR-518813_1_13_0_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:28:49,398 DEBUG [TezTR-518813_1_13_0_0_0] exec.TableScanOperator: Initializing children of 0 TS 2018-01-05T11:28:49,398 DEBUG [TezTR-518813_1_13_0_0_0] exec.SelectOperator: Initializing child 1 SEL 2018-01-05T11:28:49,398 INFO [TezTR-518813_1_13_0_0_0] exec.SelectOperator: Initializing operator SEL[1] 2018-01-05T11:28:49,398 INFO [TezTR-518813_1_13_0_0_0] exec.SelectOperator: SELECT struct 2018-01-05T11:28:49,398 DEBUG [TezTR-518813_1_13_0_0_0] exec.SelectOperator: Initialization Done 1 SEL 2018-01-05T11:28:49,398 DEBUG [TezTR-518813_1_13_0_0_0] exec.SelectOperator: Operator 1 SEL initialized 2018-01-05T11:28:49,398 DEBUG [TezTR-518813_1_13_0_0_0] exec.SelectOperator: Initializing children of 1 SEL 2018-01-05T11:28:49,398 DEBUG [TezTR-518813_1_13_0_0_0] exec.ReduceSinkOperator: Initializing child 2 RS 2018-01-05T11:28:49,398 INFO [TezTR-518813_1_13_0_0_0] exec.ReduceSinkOperator: Initializing operator RS[2] 2018-01-05T11:28:49,398 DEBUG [TezTR-518813_1_13_0_0_0] exec.ReduceSinkOperator: keys size is 0 2018-01-05T11:28:49,398 INFO [TezTR-518813_1_13_0_0_0] exec.ReduceSinkOperator: Using tag = -1 2018-01-05T11:28:49,398 DEBUG [TezTR-518813_1_13_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,398 DEBUG [TezTR-518813_1_13_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,399 DEBUG [TezTR-518813_1_13_0_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[int, string] 2018-01-05T11:28:49,399 DEBUG [TezTR-518813_1_13_0_0_0] exec.ReduceSinkOperator: Initialization Done 2 RS 2018-01-05T11:28:49,399 DEBUG [TezTR-518813_1_13_0_0_0] exec.ReduceSinkOperator: Operator 2 RS initialized 2018-01-05T11:28:49,399 DEBUG [TezTR-518813_1_13_0_0_0] exec.ReduceSinkOperator: Initialization Done 2 RS done is reset. 2018-01-05T11:28:49,399 DEBUG [TezTR-518813_1_13_0_0_0] exec.SelectOperator: Initialization Done 1 SEL done is reset. 2018-01-05T11:28:49,399 DEBUG [TezTR-518813_1_13_0_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:28:49,399 INFO [TezTR-518813_1_13_0_0_0] exec.OperatorUtils: Setting output collector: RS[2] --> Reducer 2 2018-01-05T11:28:49,399 DEBUG [TezTR-518813_1_13_0_0_0] log.PerfLogger: 2018-01-05T11:28:49,399 INFO [TezTR-518813_1_13_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/djaiswal/_tez_scratch_dir/10863ecd-b0a2-4d74-9b26-9f6ef594548e/map.xml 2018-01-05T11:28:49,399 DEBUG [TezTR-518813_1_13_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:49,399 DEBUG [TezTR-518813_1_13_0_0_0] exec.MapOperator: Processing alias(es) srcbucket_tmp for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket23.txt 2018-01-05T11:28:49,399 INFO [TezTR-518813_1_13_0_0_0] exec.ReduceSinkOperator: keys are [] num distributions: 0 2018-01-05T11:28:49,399 INFO [TezTR-518813_1_13_0_0_0] exec.MapOperator: MAP[0]: records read - 1 2018-01-05T11:28:49,399 INFO [TezTR-518813_1_13_0_0_0] exec.MapOperator: MAP[0]: records read - 10 2018-01-05T11:28:49,399 INFO [TezTR-518813_1_13_0_0_0] exec.MapOperator: MAP[0]: records read - 100 2018-01-05T11:28:49,400 DEBUG [TezTR-518813_1_13_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 1 of 4 2018-01-05T11:28:49,400 DEBUG [TezTR-518813_1_13_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket22.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcbucket_tmp, numFiles=4, columns.types=int:string, serialization.format=1, serialization.ddl=struct srcbucket_tmp { i32 key, string value}, columns=key,value, columns.comments=, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, totalSize=5812, column.name.delimiter=,, transient_lastDdlTime=1515180528}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcbucket_tmp, numFiles=4, columns.types=int:string, columns=key,value, serialization.format=1, serialization.ddl=struct srcbucket_tmp { i32 key, string value}, columns.comments=, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, totalSize=5812, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp, column.name.delimiter=,, transient_lastDdlTime=1515180528}, baseFileName=srcbucket_tmp, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcbucket_tmp, numFiles=4, columns.types=int:string, serialization.format=1, serialization.ddl=struct srcbucket_tmp { i32 key, string value}, columns=key,value, columns.comments=, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, totalSize=5812, column.name.delimiter=,, transient_lastDdlTime=1515180528}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcbucket_tmp, numFiles=4, columns.types=int:string, columns=key,value, serialization.format=1, serialization.ddl=struct srcbucket_tmp { i32 key, string value}, columns.comments=, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, totalSize=5812, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp, column.name.delimiter=,, transient_lastDdlTime=1515180528}, baseFileName=srcbucket_tmp, vectorPartitionDesc=null]} 2018-01-05T11:28:49,400 DEBUG [TezTR-518813_1_13_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:49,400 INFO [TezTR-518813_1_13_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:49,400 DEBUG [TezTR-518813_1_13_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:49,401 DEBUG [TezTR-518813_1_13_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:49,401 INFO [TezTR-518813_1_13_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/djaiswal/_tez_scratch_dir/10863ecd-b0a2-4d74-9b26-9f6ef594548e/map.xml 2018-01-05T11:28:49,401 DEBUG [TezTR-518813_1_13_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:49,401 INFO [TezTR-518813_1_13_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_13_00_000000_0 2018-01-05T11:28:49,401 INFO [TezTR-518813_1_13_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:28:49,401 WARN [TezTR-518813_1_13_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket22.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:28:49,401 DEBUG [TezTR-518813_1_13_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:28:49,401 WARN [TezTR-518813_1_13_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket22.txt:0+1450 2018-01-05T11:28:49,403 DEBUG [TezTR-518813_1_13_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket22.txt 2018-01-05T11:28:49,403 INFO [TezTR-518813_1_13_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/djaiswal/_tez_scratch_dir/10863ecd-b0a2-4d74-9b26-9f6ef594548e/map.xml 2018-01-05T11:28:49,403 DEBUG [TezTR-518813_1_13_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:49,403 DEBUG [TezTR-518813_1_13_0_0_0] exec.MapOperator: Processing alias(es) srcbucket_tmp for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket22.txt 2018-01-05T11:28:49,404 DEBUG [TezTR-518813_1_13_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 2 of 4 2018-01-05T11:28:49,404 DEBUG [TezTR-518813_1_13_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket21.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcbucket_tmp, numFiles=4, columns.types=int:string, serialization.format=1, serialization.ddl=struct srcbucket_tmp { i32 key, string value}, columns=key,value, columns.comments=, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, totalSize=5812, column.name.delimiter=,, transient_lastDdlTime=1515180528}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcbucket_tmp, numFiles=4, columns.types=int:string, columns=key,value, serialization.format=1, serialization.ddl=struct srcbucket_tmp { i32 key, string value}, columns.comments=, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, totalSize=5812, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp, column.name.delimiter=,, transient_lastDdlTime=1515180528}, baseFileName=srcbucket_tmp, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcbucket_tmp, numFiles=4, columns.types=int:string, serialization.format=1, serialization.ddl=struct srcbucket_tmp { i32 key, string value}, columns=key,value, columns.comments=, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, totalSize=5812, column.name.delimiter=,, transient_lastDdlTime=1515180528}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcbucket_tmp, numFiles=4, columns.types=int:string, columns=key,value, serialization.format=1, serialization.ddl=struct srcbucket_tmp { i32 key, string value}, columns.comments=, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, totalSize=5812, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp, column.name.delimiter=,, transient_lastDdlTime=1515180528}, baseFileName=srcbucket_tmp, vectorPartitionDesc=null]} 2018-01-05T11:28:49,404 DEBUG [TezTR-518813_1_13_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:49,404 INFO [TezTR-518813_1_13_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:49,404 DEBUG [TezTR-518813_1_13_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:49,405 DEBUG [TezTR-518813_1_13_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:49,406 INFO [TezTR-518813_1_13_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/djaiswal/_tez_scratch_dir/10863ecd-b0a2-4d74-9b26-9f6ef594548e/map.xml 2018-01-05T11:28:49,406 DEBUG [TezTR-518813_1_13_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:49,406 INFO [TezTR-518813_1_13_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_13_00_000000_0 2018-01-05T11:28:49,406 INFO [TezTR-518813_1_13_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:28:49,406 WARN [TezTR-518813_1_13_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket21.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:28:49,406 DEBUG [TezTR-518813_1_13_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:28:49,406 WARN [TezTR-518813_1_13_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket21.txt:0+1392 2018-01-05T11:28:49,406 DEBUG [TezTR-518813_1_13_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket21.txt 2018-01-05T11:28:49,407 INFO [TezTR-518813_1_13_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/djaiswal/_tez_scratch_dir/10863ecd-b0a2-4d74-9b26-9f6ef594548e/map.xml 2018-01-05T11:28:49,407 DEBUG [TezTR-518813_1_13_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:49,407 DEBUG [TezTR-518813_1_13_0_0_0] exec.MapOperator: Processing alias(es) srcbucket_tmp for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket21.txt 2018-01-05T11:28:49,407 DEBUG [TezTR-518813_1_13_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 3 of 4 2018-01-05T11:28:49,407 DEBUG [TezTR-518813_1_13_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket20.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcbucket_tmp, numFiles=4, columns.types=int:string, serialization.format=1, serialization.ddl=struct srcbucket_tmp { i32 key, string value}, columns=key,value, columns.comments=, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, totalSize=5812, column.name.delimiter=,, transient_lastDdlTime=1515180528}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcbucket_tmp, numFiles=4, columns.types=int:string, columns=key,value, serialization.format=1, serialization.ddl=struct srcbucket_tmp { i32 key, string value}, columns.comments=, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, totalSize=5812, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp, column.name.delimiter=,, transient_lastDdlTime=1515180528}, baseFileName=srcbucket_tmp, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcbucket_tmp, numFiles=4, columns.types=int:string, serialization.format=1, serialization.ddl=struct srcbucket_tmp { i32 key, string value}, columns=key,value, columns.comments=, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, totalSize=5812, column.name.delimiter=,, transient_lastDdlTime=1515180528}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcbucket_tmp, numFiles=4, columns.types=int:string, columns=key,value, serialization.format=1, serialization.ddl=struct srcbucket_tmp { i32 key, string value}, columns.comments=, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, totalSize=5812, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp, column.name.delimiter=,, transient_lastDdlTime=1515180528}, baseFileName=srcbucket_tmp, vectorPartitionDesc=null]} 2018-01-05T11:28:49,407 DEBUG [TezTR-518813_1_13_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:49,407 INFO [TezTR-518813_1_13_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:49,407 DEBUG [TezTR-518813_1_13_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:49,408 DEBUG [TezTR-518813_1_13_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:49,408 INFO [TezTR-518813_1_13_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/djaiswal/_tez_scratch_dir/10863ecd-b0a2-4d74-9b26-9f6ef594548e/map.xml 2018-01-05T11:28:49,408 DEBUG [TezTR-518813_1_13_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:49,408 INFO [TezTR-518813_1_13_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_13_00_000000_0 2018-01-05T11:28:49,408 INFO [TezTR-518813_1_13_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:28:49,409 WARN [TezTR-518813_1_13_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket20.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:28:49,409 DEBUG [TezTR-518813_1_13_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:28:49,409 WARN [TezTR-518813_1_13_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket20.txt:0+1358 2018-01-05T11:28:49,409 DEBUG [TezTR-518813_1_13_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket20.txt 2018-01-05T11:28:49,409 INFO [TezTR-518813_1_13_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/djaiswal/_tez_scratch_dir/10863ecd-b0a2-4d74-9b26-9f6ef594548e/map.xml 2018-01-05T11:28:49,409 DEBUG [TezTR-518813_1_13_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:49,409 DEBUG [TezTR-518813_1_13_0_0_0] exec.MapOperator: Processing alias(es) srcbucket_tmp for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp/srcbucket20.txt 2018-01-05T11:28:49,409 DEBUG [TezTR-518813_1_13_0_0_0] log.PerfLogger: 2018-01-05T11:28:49,409 DEBUG [TezTR-518813_1_13_0_0_0] mr.ObjectCache: djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:28:49,409 DEBUG [TezTR-518813_1_13_0_0_0] exec.MapOperator: close called for operator MAP[0] 2018-01-05T11:28:49,409 INFO [TezTR-518813_1_13_0_0_0] exec.MapOperator: Closing operator MAP[0] 2018-01-05T11:28:49,409 INFO [TezTR-518813_1_13_0_0_0] exec.MapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:500, 2018-01-05T11:28:49,409 DEBUG [TezTR-518813_1_13_0_0_0] exec.MapOperator: Closing child = TS[0] 2018-01-05T11:28:49,409 DEBUG [TezTR-518813_1_13_0_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:28:49,409 DEBUG [TezTR-518813_1_13_0_0_0] exec.TableScanOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:49,409 INFO [TezTR-518813_1_13_0_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:28:49,409 DEBUG [TezTR-518813_1_13_0_0_0] exec.TableScanOperator: Closing child = SEL[1] 2018-01-05T11:28:49,409 DEBUG [TezTR-518813_1_13_0_0_0] exec.SelectOperator: close called for operator SEL[1] 2018-01-05T11:28:49,409 DEBUG [TezTR-518813_1_13_0_0_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:49,409 INFO [TezTR-518813_1_13_0_0_0] exec.SelectOperator: Closing operator SEL[1] 2018-01-05T11:28:49,410 DEBUG [TezTR-518813_1_13_0_0_0] exec.SelectOperator: Closing child = RS[2] 2018-01-05T11:28:49,410 DEBUG [TezTR-518813_1_13_0_0_0] exec.ReduceSinkOperator: close called for operator RS[2] 2018-01-05T11:28:49,410 DEBUG [TezTR-518813_1_13_0_0_0] exec.ReduceSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:49,410 INFO [TezTR-518813_1_13_0_0_0] exec.ReduceSinkOperator: Closing operator RS[2] 2018-01-05T11:28:49,410 INFO [TezTR-518813_1_13_0_0_0] exec.ReduceSinkOperator: RECORDS_OUT_INTERMEDIATE_Map_1:500, 2018-01-05T11:28:49,410 DEBUG [TezTR-518813_1_13_0_0_0] exec.ReduceSinkOperator: 2 Close done 2018-01-05T11:28:49,410 DEBUG [TezTR-518813_1_13_0_0_0] exec.SelectOperator: 1 Close done 2018-01-05T11:28:49,410 DEBUG [TezTR-518813_1_13_0_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:28:49,410 DEBUG [TezTR-518813_1_13_0_0_0] exec.MapOperator: 0 Close done 2018-01-05T11:28:49,410 INFO [TezTR-518813_1_13_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_13_00_000000_0 2018-01-05T11:28:49,410 INFO [TezTR-518813_1_13_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: Waiting for all spills to complete : Pending : 0 2018-01-05T11:28:49,410 INFO [TezTR-518813_1_13_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: All spills complete 2018-01-05T11:28:49,425 INFO [LlapTaskSchedulerTimedLogThread] tezplugins.LlapTaskSchedulerService: Stats for current dag: NumPreemptedTasks=0, NumRequestedAllocations=1, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=1,NumRequestsWithoutLocation=1, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=1} 2018-01-05T11:28:49,436 INFO [TezTR-518813_1_13_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: Finished spill 0 2018-01-05T11:28:49,437 DEBUG [TezTR-518813_1_13_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:49,437 INFO [TezTR-518813_1_13_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:49,437 INFO [TezTR-518813_1_13_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_13_00_000000_0, askedToStop=false 2018-01-05T11:28:49,437 INFO [TezTR-518813_1_13_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_13_00_000000_0, stopRequested=false 2018-01-05T11:28:49,437 INFO [TezTR-518813_1_13_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_13_00_000000_0: Counters: 14 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=500, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=500, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=6051, OUTPUT_BYTES_WITH_OVERHEAD=13001, OUTPUT_BYTES_PHYSICAL=7187, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500, RECORDS_OUT_INTERMEDIATE_Map_1=500]] 2018-01-05T11:28:49,438 DEBUG [TezTR-518813_1_13_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_13_0_0_0 thread id: 877 scheme: file bytesRead: 5912 bytesWritten: 7195 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:49,438 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_13_00_000000_0, isTaskDone=true 2018-01-05T11:28:49,438 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000021, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_13_00_000000_0, eventCount=4 } 2018-01-05T11:28:49,439 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_13_00_000000_0, creationTime=1515180529359, allocationTime=1515180529362, startTime=1515180529364, finishTime=1515180529439, timeTaken=75, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:49,439 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:49,439 INFO [Task-Executor-2] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_13_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:49,439 INFO [Task-Executor-2] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_13_00_000000_0 2018-01-05T11:28:49,439 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_13_00_000000_0 2018-01-05T11:28:49,439 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_13_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:49,439 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_13_00_000000_0; false 2018-01-05T11:28:49,439 INFO [Task-Executor-2] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000021=61 2018-01-05T11:28:49,440 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_13_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:49,440 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_13_00_000000_0: true 2018-01-05T11:28:49,440 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:49,440 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_13_00_000000_0 2018-01-05T11:28:49,440 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_13_00_000000, startTime=1515180529364, finishTime=1515180529439, timeTaken=75, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_13_00_000000_0, diagnostics=, counters=Counters: 20, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=5912, FILE_BYTES_WRITTEN=7195, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=500, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=500, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=6051, OUTPUT_BYTES_WITH_OVERHEAD=13001, OUTPUT_BYTES_PHYSICAL=7187, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500, RECORDS_OUT_INTERMEDIATE_Map_1=500 2018-01-05T11:28:49,440 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:49,440 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000021, QueryId=djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39-13, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:28:49,440 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_13_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:49,440 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_13_00_000000_0 2018-01-05T11:28:49,440 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_13_00 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:49,440 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_13_00 [Map 1] 2018-01-05T11:28:49,441 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_13_00, initRequestedTime=1515180529291, initedTime=1515180529359, startRequestedTime=1515180529298, startedTime=1515180529359, finishTime=1515180529440, timeTaken=81, status=SUCCEEDED, diagnostics=, counters=Counters: 20, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=5912, FILE_BYTES_WRITTEN=7195, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=500, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=500, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=6051, OUTPUT_BYTES_WITH_OVERHEAD=13001, OUTPUT_BYTES_PHYSICAL=7187, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500, RECORDS_OUT_INTERMEDIATE_Map_1=500, vertexStats=firstTaskStartTime=1515180529364, firstTasksToStart=[ task_1515180518813_0001_13_00_000000 ], lastTaskFinishTime=1515180529439, lastTasksToFinish=[ task_1515180518813_0001_13_00_000000 ], minTaskDuration=75, maxTaskDuration=75, avgTaskDuration=75.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_13_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_13_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:49,441 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_13_00 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:49,441 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000021, stoppedTime=1515180529441, exitStatus=0 2018-01-05T11:28:49,441 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000021 exited with diagnostics set to Container completed. 2018-01-05T11:28:49,441 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_13_01 [Reducer 2] attempt: attempt_1515180518813_0001_13_00_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:28:49,441 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_13_00 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3 2018-01-05T11:28:49,441 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null 2018-01-05T11:28:49,441 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000021 2018-01-05T11:28:49,441 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: All source tasks completed. Ramping up 4 remaining tasks for vertex: Reducer 2 2018-01-05T11:28:49,441 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Scheduling 4 tasks for vertex: Reducer 2 with totalTasks: 4. 1 source tasks completed out of 1. MinSourceTaskCompletedFraction: 1.0 min: 0.05 max: 0.75 2018-01-05T11:28:49,441 INFO [App Shared Pool - #0] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Map 1, destinationVertexName=Reducer 2. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.UnorderedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:28:49,441 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_13_01 [Reducer 2] 2018-01-05T11:28:49,441 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_13_01_000000, scheduledTime=1515180529441, launchTime=1515180529441 2018-01-05T11:28:49,441 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_13_01_000001, scheduledTime=1515180529441, launchTime=1515180529441 2018-01-05T11:28:49,441 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_13_01_000002, scheduledTime=1515180529441, launchTime=1515180529441 2018-01-05T11:28:49,441 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_13_01_000003, scheduledTime=1515180529441, launchTime=1515180529441 2018-01-05T11:28:49,441 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_13_01 [Reducer 2] 2018-01-05T11:28:49,441 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_13_01_000000_0, priority=20, capability=, hosts=null 2018-01-05T11:28:49,441 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=20,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:49,441 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_13_01_000001_0, priority=20, capability=, hosts=null 2018-01-05T11:28:49,441 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=20,c=2]. totalPendingTasks=2. delayedTaskQueueSize=0 2018-01-05T11:28:49,441 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_13_01_000002_0, priority=20, capability=, hosts=null 2018-01-05T11:28:49,441 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=20,c=3]. totalPendingTasks=3. delayedTaskQueueSize=0 2018-01-05T11:28:49,441 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_13_01_000003_0, priority=20, capability=, hosts=null 2018-01-05T11:28:49,441 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=20,c=3]. totalPendingTasks=3. delayedTaskQueueSize=0 2018-01-05T11:28:49,442 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:49,442 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_13_01_000000_0 on hosts=null 2018-01-05T11:28:49,442 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_13_01_000000_0 on hosts=null 2018-01-05T11:28:49,442 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_13_01_000000_0 2018-01-05T11:28:49,442 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:49,442 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #22, task=TaskInfo{task=attempt_1515180518813_0001_13_01_000000_0, priority=20, startTime=0, containerId=null, uniqueId=21, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000022 2018-01-05T11:28:49,442 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_13_01_000000_0; false 2018-01-05T11:28:49,442 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_13_01_000000_0, priority=20, startTime=311549951, containerId=container_111101111_0001_00_000022assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=21, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:49,442 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_13_01_000001_0 on hosts=null 2018-01-05T11:28:49,442 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_13_01_000001_0 on hosts=null 2018-01-05T11:28:49,442 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_13_01_000001_0 2018-01-05T11:28:49,442 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:49,442 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #23, task=TaskInfo{task=attempt_1515180518813_0001_13_01_000001_0, priority=20, startTime=0, containerId=null, uniqueId=22, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, to container=container_111101111_0001_00_000023 2018-01-05T11:28:49,442 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_13_01_000001_0; false 2018-01-05T11:28:49,442 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_13_01_000001_0, priority=20, startTime=311549951, containerId=container_111101111_0001_00_000023assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=2, ac=6, commF=false, disabled=false}, uniqueId=22, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:49,442 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_13_01_000002_0 on hosts=null 2018-01-05T11:28:49,442 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_13_01_000002_0 on hosts=null 2018-01-05T11:28:49,442 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_13_01_000002_0 2018-01-05T11:28:49,442 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=2, ac=6, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:49,442 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #24, task=TaskInfo{task=attempt_1515180518813_0001_13_01_000002_0, priority=20, startTime=0, containerId=null, uniqueId=23, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=2, ac=6, commF=false, disabled=false}, to container=container_111101111_0001_00_000024 2018-01-05T11:28:49,442 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_13_01_000002_0; false 2018-01-05T11:28:49,442 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_13_01_000002_0, priority=20, startTime=311549951, containerId=container_111101111_0001_00_000024assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=3, ac=5, commF=false, disabled=false}, uniqueId=23, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:49,442 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:49,442 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:49,442 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=20,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:49,442 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=20,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:49,442 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:49,442 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_13_01_000003_0 on hosts=null 2018-01-05T11:28:49,442 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_13_01_000003_0 on hosts=null 2018-01-05T11:28:49,442 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_13_01_000003_0 2018-01-05T11:28:49,442 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=3, ac=5, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:49,442 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #25, task=TaskInfo{task=attempt_1515180518813_0001_13_01_000003_0, priority=20, startTime=0, containerId=null, uniqueId=24, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=3, ac=5, commF=false, disabled=false}, to container=container_111101111_0001_00_000025 2018-01-05T11:28:49,442 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_13_01_000003_0; false 2018-01-05T11:28:49,442 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_13_01_000003_0, priority=20, startTime=311549951, containerId=container_111101111_0001_00_000025assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=4, ac=4, commF=false, disabled=false}, uniqueId=24, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:49,442 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000022 for node: 10.22.8.182:49818 2018-01-05T11:28:49,442 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000022, attempt_1515180518813_0001_13_01_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:49,442 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000022 for node: 10.22.8.182:49818 2018-01-05T11:28:49,443 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000023 for node: 10.22.8.182:49818 2018-01-05T11:28:49,443 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000023, attempt_1515180518813_0001_13_01_000001_0 for node: 10.22.8.182:49818 2018-01-05T11:28:49,443 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000023 for node: 10.22.8.182:49818 2018-01-05T11:28:49,443 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=13}, sourceName=Map 1, state=S_SUCCEEDED 2018-01-05T11:28:49,444 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000024 for node: 10.22.8.182:49818 2018-01-05T11:28:49,444 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000024, attempt_1515180518813_0001_13_01_000002_0 for node: 10.22.8.182:49818 2018-01-05T11:28:49,444 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000024 for node: 10.22.8.182:49818 2018-01-05T11:28:49,444 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000025 for node: 10.22.8.182:49818 2018-01-05T11:28:49,444 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000025, attempt_1515180518813_0001_13_01_000003_0 for node: 10.22.8.182:49818 2018-01-05T11:28:49,444 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000025 for node: 10.22.8.182:49818 2018-01-05T11:28:49,444 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_13_01_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000022, dagName=INSERT INTO srcbucket2 SELEC...srcbucket_tmp (Stage-1), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.UnorderedKVInput,1}}, Outputs={{Reducer 3,org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput,1}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=5, completedTaskCount=1, dagStartTime=1515180529290, firstAttemptStartTime=1515180529441, currentAttemptStartTime=1515180529443} 2018-01-05T11:28:49,445 INFO [IPC Server handler 4 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000022, QueryId=djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39-13, DagId=13, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, SubmitTime=1515180529445 2018-01-05T11:28:49,445 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000022 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:49,445 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000022, launchTime=1515180529445 2018-01-05T11:28:49,445 DEBUG [IPC Server handler 4 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=13} with the ShuffleHandler 2018-01-05T11:28:49,445 DEBUG [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/13] 2018-01-05T11:28:49,445 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:49,445 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_13_01_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:49,445 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_13_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180529441, dagStartTime=1515180529290, withinDagPriority=20, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1}] 2018-01-05T11:28:49,445 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_13_01_000000_0 : ACCEPTED 2018-01-05T11:28:49,445 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_13_01_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:49,445 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_13_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180529441, dagStartTime=1515180529290, withinDagPriority=20, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1} 2018-01-05T11:28:49,445 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_13_01_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:49,445 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_13_01_000000_0 to pre-emption queue 2018-01-05T11:28:49,445 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_13_01_000000_0: true 2018-01-05T11:28:49,445 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_13_01_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000022] on NM: [10.22.8.182:49818] 2018-01-05T11:28:49,445 INFO [TaskCommunicator # 1] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_13_01_000000_0 2018-01-05T11:28:49,445 INFO [TaskCommunicator # 1] GuaranteedTasks: Received response for attempt_1515180518813_0001_13_01_000000_0, true 2018-01-05T11:28:49,445 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:49,445 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:49,445 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: taskOwner hashCode:1387447673 2018-01-05T11:28:49,445 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_13_01_000000_0, startTime=1515180529445, containerId=container_111101111_0001_00_000022, nodeId=10.22.8.182:49818 2018-01-05T11:28:49,445 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000023 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:49,445 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000023, launchTime=1515180529445 2018-01-05T11:28:49,446 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_13_01_000001_0, am_details=10.22.8.182:49824, taskInfo= fragment 1 attempt 0, user=djaiswal, queryId=djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000023, dagName=INSERT INTO srcbucket2 SELEC...srcbucket_tmp (Stage-1), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.UnorderedKVInput,1}}, Outputs={{Reducer 3,org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput,1}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=5, completedTaskCount=1, dagStartTime=1515180529290, firstAttemptStartTime=1515180529441, currentAttemptStartTime=1515180529443} 2018-01-05T11:28:49,446 INFO [Task-Executor-2] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : INSERT INTO srcbucket2 SELEC...srcbucket_tmp (Stage-1), VertexName: Reducer 2, VertexParallelism: 4, TaskAttemptID:attempt_1515180518813_0001_13_01_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }}, ], outputSpecList=[{{ destinationVertexName=Reducer 3, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput }}, ] 2018-01-05T11:28:49,446 INFO [IPC Server handler 3 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000023, QueryId=djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39-13, DagId=13, VertexName=Reducer 2, TaskId=1, TaskAttemptId=0, SubmitTime=1515180529446 2018-01-05T11:28:49,446 INFO [Task-Executor-2] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=954728448, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:49,446 INFO [Task-Executor-2] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_13_01_000000_0 vertexName: Reducer 2 2018-01-05T11:28:49,446 DEBUG [IPC Server handler 3 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=13} with the ShuffleHandler 2018-01-05T11:28:49,446 DEBUG [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/13] 2018-01-05T11:28:49,446 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_13_01_000001_0] submitted. Is using containerId: [container_111101111_0001_00_000023] on NM: [10.22.8.182:49818] 2018-01-05T11:28:49,446 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:49,446 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:49,446 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_13_01_000001_0, startTime=1515180529445, containerId=container_111101111_0001_00_000023, nodeId=10.22.8.182:49818 2018-01-05T11:28:49,446 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000022, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_13_01_000000_0, eventCount=1 } 2018-01-05T11:28:49,446 DEBUG [TezTR-518813_1_13_1_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_13 queryId: djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39 instanceType: TaskRunner2Callable 2018-01-05T11:28:49,446 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000024 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:49,446 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000024, launchTime=1515180529446 2018-01-05T11:28:49,446 INFO [TezTR-518813_1_13_1_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_13_01_000000_0 2018-01-05T11:28:49,446 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=3, runningFragmentCount=1 2018-01-05T11:28:49,446 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_13_01_000001_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:49,446 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_13_01_000001_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180529441, dagStartTime=1515180529290, withinDagPriority=20, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1}] 2018-01-05T11:28:49,446 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_13_01_000001_0 : ACCEPTED 2018-01-05T11:28:49,446 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_13_01_000001_0, canFinish=true. Current state: preemptionQueueSize=1, numSlotsAvailable=3, waitQueueSize=1 2018-01-05T11:28:49,446 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_13_01_000001_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180529441, dagStartTime=1515180529290, withinDagPriority=20, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1} 2018-01-05T11:28:49,446 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_13_01_000001_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:49,446 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_13_01_000001_0 to pre-emption queue 2018-01-05T11:28:49,446 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_13_01_000001_0: true 2018-01-05T11:28:49,447 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: taskOwner hashCode:2120498944 2018-01-05T11:28:49,447 INFO [TaskCommunicator # 2] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_13_01_000001_0 2018-01-05T11:28:49,447 INFO [TaskCommunicator # 2] GuaranteedTasks: Received response for attempt_1515180518813_0001_13_01_000001_0, true 2018-01-05T11:28:49,447 DEBUG [TezTR-518813_1_13_1_0_0] log.PerfLogger: 2018-01-05T11:28:49,447 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_13_01_000002_0] submitted. Is using containerId: [container_111101111_0001_00_000024] on NM: [10.22.8.182:49818] 2018-01-05T11:28:49,447 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:49,447 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:49,447 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_13_01_000002_0, startTime=1515180529446, containerId=container_111101111_0001_00_000024, nodeId=10.22.8.182:49818 2018-01-05T11:28:49,447 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000025 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:49,447 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:49,447 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000025, launchTime=1515180529447 2018-01-05T11:28:49,447 INFO [Task-Executor-3] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : INSERT INTO srcbucket2 SELEC...srcbucket_tmp (Stage-1), VertexName: Reducer 2, VertexParallelism: 4, TaskAttemptID:attempt_1515180518813_0001_13_01_000001_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }}, ], outputSpecList=[{{ destinationVertexName=Reducer 3, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput }}, ] 2018-01-05T11:28:49,447 INFO [I/O Setup 1 Initialize: {Reducer 3}] writers.UnorderedPartitionedKVWriter: Requested BufferSize (tez.runtime.unordered.output.buffer.size-mb) : 100 2018-01-05T11:28:49,447 INFO [Task-Executor-3] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=954728448, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:49,447 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_13_01_000003_0] submitted. Is using containerId: [container_111101111_0001_00_000025] on NM: [10.22.8.182:49818] 2018-01-05T11:28:49,448 INFO [Task-Executor-3] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_13_01_000001_0 vertexName: Reducer 2 2018-01-05T11:28:49,448 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:49,448 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:49,448 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_13_01_000003_0, startTime=1515180529447, containerId=container_111101111_0001_00_000025, nodeId=10.22.8.182:49818 2018-01-05T11:28:49,448 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_13_01_000002_0, am_details=10.22.8.182:49824, taskInfo= fragment 2 attempt 0, user=djaiswal, queryId=djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000024, dagName=INSERT INTO srcbucket2 SELEC...srcbucket_tmp (Stage-1), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.UnorderedKVInput,1}}, Outputs={{Reducer 3,org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput,1}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=5, completedTaskCount=1, dagStartTime=1515180529290, firstAttemptStartTime=1515180529441, currentAttemptStartTime=1515180529444} 2018-01-05T11:28:49,448 INFO [IPC Server handler 1 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000024, QueryId=djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39-13, DagId=13, VertexName=Reducer 2, TaskId=2, TaskAttemptId=0, SubmitTime=1515180529448 2018-01-05T11:28:49,448 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000023, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_13_01_000001_0, eventCount=1 } 2018-01-05T11:28:49,448 DEBUG [IPC Server handler 1 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=13} with the ShuffleHandler 2018-01-05T11:28:49,448 DEBUG [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/13] 2018-01-05T11:28:49,448 DEBUG [TezTR-518813_1_13_1_1_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_13 queryId: djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39 instanceType: TaskRunner2Callable 2018-01-05T11:28:49,449 INFO [TezTR-518813_1_13_1_1_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_13_01_000001_0 2018-01-05T11:28:49,449 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=2, runningFragmentCount=2 2018-01-05T11:28:49,449 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:49,449 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_13_01_000002_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:49,449 DEBUG [TezTR-518813_1_13_1_1_0] log.PerfLogger: 2018-01-05T11:28:49,449 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_13_01_000002_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180529441, dagStartTime=1515180529290, withinDagPriority=20, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1}] 2018-01-05T11:28:49,449 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_13_01_000002_0 : ACCEPTED 2018-01-05T11:28:49,449 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_13_01_000002_0, canFinish=true. Current state: preemptionQueueSize=2, numSlotsAvailable=2, waitQueueSize=1 2018-01-05T11:28:49,449 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_13_01_000002_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180529441, dagStartTime=1515180529290, withinDagPriority=20, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1} 2018-01-05T11:28:49,449 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_13_01_000002_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:49,449 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_13_01_000002_0 to pre-emption queue 2018-01-05T11:28:49,449 INFO [TaskCommunicator # 3] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_13_01_000002_0 2018-01-05T11:28:49,449 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:49,449 INFO [TaskCommunicator # 3] GuaranteedTasks: Received response for attempt_1515180518813_0001_13_01_000002_0, true 2018-01-05T11:28:49,449 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_13_01_000002_0: true 2018-01-05T11:28:49,450 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: taskOwner hashCode:629620962 2018-01-05T11:28:49,451 INFO [I/O Setup 1 Initialize: {Reducer 3}] writers.UnorderedPartitionedKVWriter: Requested BufferSize (tez.runtime.unordered.output.buffer.size-mb) : 100 2018-01-05T11:28:49,452 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_13_01_000003_0, am_details=10.22.8.182:49824, taskInfo= fragment 3 attempt 0, user=djaiswal, queryId=djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000025, dagName=INSERT INTO srcbucket2 SELEC...srcbucket_tmp (Stage-1), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.UnorderedKVInput,1}}, Outputs={{Reducer 3,org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput,1}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=5, completedTaskCount=1, dagStartTime=1515180529290, firstAttemptStartTime=1515180529441, currentAttemptStartTime=1515180529444} 2018-01-05T11:28:49,452 INFO [Task-Executor-1] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : INSERT INTO srcbucket2 SELEC...srcbucket_tmp (Stage-1), VertexName: Reducer 2, VertexParallelism: 4, TaskAttemptID:attempt_1515180518813_0001_13_01_000002_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }}, ], outputSpecList=[{{ destinationVertexName=Reducer 3, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput }}, ] 2018-01-05T11:28:49,452 INFO [IPC Server handler 0 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000025, QueryId=djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39-13, DagId=13, VertexName=Reducer 2, TaskId=3, TaskAttemptId=0, SubmitTime=1515180529452 2018-01-05T11:28:49,452 INFO [IPC Server handler 5 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_13_01_000000_0 1 events [0,1) total 1 vertex_1515180518813_0001_13_01 [Reducer 2] 2018-01-05T11:28:49,452 INFO [Task-Executor-1] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=954728448, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:49,452 INFO [Task-Executor-1] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_13_01_000002_0 vertexName: Reducer 2 2018-01-05T11:28:49,452 DEBUG [IPC Server handler 0 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=13} with the ShuffleHandler 2018-01-05T11:28:49,452 DEBUG [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/13] 2018-01-05T11:28:49,452 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:49,453 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000024, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_13_01_000002_0, eventCount=1 } 2018-01-05T11:28:49,453 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:28:49,453 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_13_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:28:49,453 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=1, runningFragmentCount=3 2018-01-05T11:28:49,453 DEBUG [TezTR-518813_1_13_1_2_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_13 queryId: djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39 instanceType: TaskRunner2Callable 2018-01-05T11:28:49,453 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_13_01_000003_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:49,453 INFO [TezTR-518813_1_13_1_2_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_13_01_000002_0 2018-01-05T11:28:49,453 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_13_01_000003_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180529441, dagStartTime=1515180529290, withinDagPriority=20, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1}] 2018-01-05T11:28:49,453 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_13_01_000003_0 : ACCEPTED 2018-01-05T11:28:49,453 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_13_01_000003_0, canFinish=true. Current state: preemptionQueueSize=3, numSlotsAvailable=1, waitQueueSize=1 2018-01-05T11:28:49,454 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_13_01_000003_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180529441, dagStartTime=1515180529290, withinDagPriority=20, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1} 2018-01-05T11:28:49,454 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_13_01_000003_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:49,454 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_13_01_000003_0 to pre-emption queue 2018-01-05T11:28:49,454 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_13_01_000003_0: true 2018-01-05T11:28:49,454 INFO [TaskCommunicator # 4] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_13_01_000003_0 2018-01-05T11:28:49,454 INFO [TaskCommunicator # 4] GuaranteedTasks: Received response for attempt_1515180518813_0001_13_01_000003_0, true 2018-01-05T11:28:49,454 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:1014366899 2018-01-05T11:28:49,454 DEBUG [TezTR-518813_1_13_1_2_0] log.PerfLogger: 2018-01-05T11:28:49,454 INFO [IPC Server handler 3 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_13_01_000001_0 1 events [0,1) total 1 vertex_1515180518813_0001_13_01 [Reducer 2] 2018-01-05T11:28:49,454 DEBUG [TezTR-518813_1_13_1_0_0] log.PerfLogger: 2018-01-05T11:28:49,454 INFO [TezTR-518813_1_13_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:49,454 INFO [TezTR-518813_1_13_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:49,454 INFO [TezTR-518813_1_13_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:49,454 INFO [TezTR-518813_1_13_1_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:49,454 INFO [TezTR-518813_1_13_1_0_0] resources.MemoryDistributor: InitialRequests=[Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.UnorderedKVInput], [Reducer 3:OUTPUT:104857600:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput] 2018-01-05T11:28:49,454 INFO [TezTR-518813_1_13_1_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:49,455 INFO [TezTR-518813_1_13_1_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:49,455 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : INSERT INTO srcbucket2 SELEC...srcbucket_tmp (Stage-1), VertexName: Reducer 2, VertexParallelism: 4, TaskAttemptID:attempt_1515180518813_0001_13_01_000003_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }}, ], outputSpecList=[{{ destinationVertexName=Reducer 3, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput }}, ] 2018-01-05T11:28:49,455 INFO [TezTR-518813_1_13_1_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 236262520, TotalRequestedScaled: 1.1813126E8, TotalJVMHeap: 954728448, TotalAvailable: 639668060, TotalRequested/TotalJVMHeap:0.25 2018-01-05T11:28:49,455 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:49,455 INFO [TezTR-518813_1_13_1_0_0] resources.MemoryDistributor: Allocations=[Map 1:org.apache.tez.runtime.library.input.UnorderedKVInput:INPUT:131404920:131404920], [Reducer 3:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput:OUTPUT:104857600:104857600] 2018-01-05T11:28:49,455 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=954728448, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:49,455 INFO [TezTR-518813_1_13_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:49,455 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_13_01_000003_0 vertexName: Reducer 2 2018-01-05T11:28:49,455 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:28:49,455 INFO [TezTR-518813_1_13_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 0 2018-01-05T11:28:49,455 INFO [TezTR-518813_1_13_1_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:49,455 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_13_01_000001_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:28:49,455 INFO [I/O Setup 1 Initialize: {Reducer 3}] writers.UnorderedPartitionedKVWriter: Requested BufferSize (tez.runtime.unordered.output.buffer.size-mb) : 100 2018-01-05T11:28:49,455 INFO [TezTR-518813_1_13_1_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_13_01_000000_0 2018-01-05T11:28:49,455 DEBUG [TezTR-518813_1_13_1_0_0] log.PerfLogger: 2018-01-05T11:28:49,455 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000025, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_13_01_000003_0, eventCount=1 } 2018-01-05T11:28:49,455 DEBUG [TezTR-518813_1_13_1_3_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_13 queryId: djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39 instanceType: TaskRunner2Callable 2018-01-05T11:28:49,455 DEBUG [TezTR-518813_1_13_1_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_13_01_000000_0_10064 2018-01-05T11:28:49,456 DEBUG [TezTR-518813_1_13_1_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:49,456 INFO [TezTR-518813_1_13_1_3_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_13_01_000003_0 2018-01-05T11:28:49,456 DEBUG [TezTR-518813_1_13_1_0_0] mr.ObjectCache: Creating djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:28:49,456 INFO [TezTR-518813_1_13_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/djaiswal/_tez_scratch_dir/3fd70205-c245-4b8e-9c69-f4cf50e400e9/reduce.xml 2018-01-05T11:28:49,456 DEBUG [TezTR-518813_1_13_1_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/djaiswal/_tez_scratch_dir/3fd70205-c245-4b8e-9c69-f4cf50e400e9/reduce.xml 2018-01-05T11:28:49,456 DEBUG [TezTR-518813_1_13_1_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/djaiswal/_tez_scratch_dir/3fd70205-c245-4b8e-9c69-f4cf50e400e9/reduce.xml 2018-01-05T11:28:49,456 DEBUG [TezTR-518813_1_13_1_0_0] log.PerfLogger: 2018-01-05T11:28:49,456 INFO [TezTR-518813_1_13_1_0_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:28:49,456 DEBUG [TezTR-518813_1_13_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,456 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:49,456 DEBUG [TezTR-518813_1_13_1_3_0] log.PerfLogger: 2018-01-05T11:28:49,457 DEBUG [TezTR-518813_1_13_1_1_0] log.PerfLogger: 2018-01-05T11:28:49,457 INFO [TezTR-518813_1_13_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:49,457 INFO [TezTR-518813_1_13_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:49,457 INFO [TezTR-518813_1_13_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:49,457 INFO [TezTR-518813_1_13_1_1_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:49,457 INFO [TezTR-518813_1_13_1_1_0] resources.MemoryDistributor: InitialRequests=[Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.UnorderedKVInput], [Reducer 3:OUTPUT:104857600:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput] 2018-01-05T11:28:49,457 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:49,457 INFO [TezTR-518813_1_13_1_1_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:49,457 INFO [TezTR-518813_1_13_1_1_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:49,457 INFO [I/O Setup 1 Initialize: {Reducer 3}] writers.UnorderedPartitionedKVWriter: Requested BufferSize (tez.runtime.unordered.output.buffer.size-mb) : 100 2018-01-05T11:28:49,457 INFO [TezTR-518813_1_13_1_1_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 236262520, TotalRequestedScaled: 1.1813126E8, TotalJVMHeap: 954728448, TotalAvailable: 639668060, TotalRequested/TotalJVMHeap:0.25 2018-01-05T11:28:49,458 INFO [TezTR-518813_1_13_1_1_0] resources.MemoryDistributor: Allocations=[Map 1:org.apache.tez.runtime.library.input.UnorderedKVInput:INPUT:131404920:131404920], [Reducer 3:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput:OUTPUT:104857600:104857600] 2018-01-05T11:28:49,458 INFO [TezTR-518813_1_13_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:49,458 INFO [TezTR-518813_1_13_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 0 2018-01-05T11:28:49,458 INFO [TezTR-518813_1_13_1_1_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:49,458 INFO [IPC Server handler 1 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_13_01_000002_0 1 events [0,1) total 1 vertex_1515180518813_0001_13_01 [Reducer 2] 2018-01-05T11:28:49,458 INFO [TezTR-518813_1_13_1_1_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_13_01_000001_0 2018-01-05T11:28:49,458 DEBUG [TezTR-518813_1_13_1_1_0] log.PerfLogger: 2018-01-05T11:28:49,458 DEBUG [TezTR-518813_1_13_1_1_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_13_01_000001_0_10067 2018-01-05T11:28:49,458 DEBUG [TezTR-518813_1_13_1_1_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:49,458 DEBUG [TezTR-518813_1_13_1_1_0] mr.ObjectCache: Creating djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:28:49,458 INFO [TezTR-518813_1_13_1_1_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/djaiswal/_tez_scratch_dir/3fd70205-c245-4b8e-9c69-f4cf50e400e9/reduce.xml 2018-01-05T11:28:49,458 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:28:49,459 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_13_01_000002_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:28:49,459 DEBUG [TezTR-518813_1_13_1_1_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/djaiswal/_tez_scratch_dir/3fd70205-c245-4b8e-9c69-f4cf50e400e9/reduce.xml 2018-01-05T11:28:49,459 DEBUG [TezTR-518813_1_13_1_1_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/djaiswal/_tez_scratch_dir/3fd70205-c245-4b8e-9c69-f4cf50e400e9/reduce.xml 2018-01-05T11:28:49,459 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:49,459 DEBUG [TezTR-518813_1_13_1_1_0] log.PerfLogger: 2018-01-05T11:28:49,459 INFO [TezTR-518813_1_13_1_1_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:28:49,459 DEBUG [TezTR-518813_1_13_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,459 DEBUG [TezTR-518813_1_13_1_0_0] log.PerfLogger: 2018-01-05T11:28:49,459 INFO [TezTR-518813_1_13_1_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 3.17KB 2018-01-05T11:28:49,460 DEBUG [TezTR-518813_1_13_1_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:49,460 DEBUG [TezTR-518813_1_13_1_0_0] log.PerfLogger: 2018-01-05T11:28:49,460 DEBUG [TezTR-518813_1_13_1_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:49,460 DEBUG [TezTR-518813_1_13_1_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:49,460 DEBUG [TezTR-518813_1_13_1_2_0] log.PerfLogger: 2018-01-05T11:28:49,460 INFO [TezTR-518813_1_13_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:49,460 INFO [TezTR-518813_1_13_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:49,460 INFO [TezTR-518813_1_13_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:49,460 INFO [TezTR-518813_1_13_1_2_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:49,460 INFO [IPC Server handler 2 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_13_01_000003_0 1 events [0,1) total 1 vertex_1515180518813_0001_13_01 [Reducer 2] 2018-01-05T11:28:49,460 INFO [TezTR-518813_1_13_1_2_0] resources.MemoryDistributor: InitialRequests=[Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.UnorderedKVInput], [Reducer 3:OUTPUT:104857600:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput] 2018-01-05T11:28:49,460 INFO [TezTR-518813_1_13_1_2_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:49,460 INFO [TezTR-518813_1_13_1_2_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:49,460 INFO [TezTR-518813_1_13_1_2_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 236262520, TotalRequestedScaled: 1.1813126E8, TotalJVMHeap: 954728448, TotalAvailable: 639668060, TotalRequested/TotalJVMHeap:0.25 2018-01-05T11:28:49,460 INFO [TezTR-518813_1_13_1_2_0] resources.MemoryDistributor: Allocations=[Map 1:org.apache.tez.runtime.library.input.UnorderedKVInput:INPUT:131404920:131404920], [Reducer 3:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput:OUTPUT:104857600:104857600] 2018-01-05T11:28:49,460 INFO [TezTR-518813_1_13_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:49,460 INFO [TezTR-518813_1_13_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 0 2018-01-05T11:28:49,460 INFO [TezTR-518813_1_13_1_2_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:49,460 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:28:49,460 INFO [TezTR-518813_1_13_1_2_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_13_01_000002_0 2018-01-05T11:28:49,460 DEBUG [TezTR-518813_1_13_1_2_0] log.PerfLogger: 2018-01-05T11:28:49,460 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_13_01_000003_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:28:49,460 DEBUG [TezTR-518813_1_13_1_2_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_13_01_000002_0_10070 2018-01-05T11:28:49,460 DEBUG [TezTR-518813_1_13_1_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:49,461 DEBUG [TezTR-518813_1_13_1_2_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:49,461 DEBUG [TezTR-518813_1_13_1_2_0] mr.ObjectCache: Creating djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:28:49,461 INFO [TezTR-518813_1_13_1_0_0] impl.SimpleFetchedInputAllocator: Map_1: RequestedMemory=131404920, AssignedMemory=131404920, maxSingleShuffleLimit=32851230 2018-01-05T11:28:49,461 INFO [TezTR-518813_1_13_1_2_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/djaiswal/_tez_scratch_dir/3fd70205-c245-4b8e-9c69-f4cf50e400e9/reduce.xml 2018-01-05T11:28:49,461 DEBUG [TezTR-518813_1_13_1_2_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/djaiswal/_tez_scratch_dir/3fd70205-c245-4b8e-9c69-f4cf50e400e9/reduce.xml 2018-01-05T11:28:49,461 DEBUG [TezTR-518813_1_13_1_2_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/djaiswal/_tez_scratch_dir/3fd70205-c245-4b8e-9c69-f4cf50e400e9/reduce.xml 2018-01-05T11:28:49,461 DEBUG [TezTR-518813_1_13_1_2_0] log.PerfLogger: 2018-01-05T11:28:49,461 INFO [TezTR-518813_1_13_1_2_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:28:49,461 DEBUG [TezTR-518813_1_13_1_1_0] log.PerfLogger: 2018-01-05T11:28:49,461 DEBUG [TezTR-518813_1_13_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,461 INFO [TezTR-518813_1_13_1_1_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 3.17KB 2018-01-05T11:28:49,461 DEBUG [TezTR-518813_1_13_1_0_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:49,461 DEBUG [TezTR-518813_1_13_1_0_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:49,461 DEBUG [TezTR-518813_1_13_1_0_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:49,461 DEBUG [TezTR-518813_1_13_1_0_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:49,461 DEBUG [TezTR-518813_1_13_1_1_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:49,461 DEBUG [TezTR-518813_1_13_1_1_0] log.PerfLogger: 2018-01-05T11:28:49,461 DEBUG [TezTR-518813_1_13_1_1_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:49,461 DEBUG [TezTR-518813_1_13_1_1_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:49,461 INFO [TezTR-518813_1_13_1_0_0] impl.ShuffleManager: Map_1: numInputs=1, compressionCodec=NoCompressionCodec, numFetchers=1, ifileBufferSize=4096, ifileReadAheadEnabled=true, ifileReadAheadLength=4194304, localDiskFetchEnabled=false, sharedFetchEnabled=false, keepAlive=false, keepAliveMaxConnections=20, connectionTimeout=20000, readTimeout=180000, bufferSize=8192, bufferSize=8192, maxTaskOutputAtOnce=20 2018-01-05T11:28:49,461 DEBUG [TezTR-518813_1_13_1_3_0] log.PerfLogger: 2018-01-05T11:28:49,461 INFO [TezTR-518813_1_13_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:49,461 INFO [TezTR-518813_1_13_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:49,461 INFO [TezTR-518813_1_13_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:49,461 INFO [TezTR-518813_1_13_1_3_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:49,461 INFO [TezTR-518813_1_13_1_3_0] resources.MemoryDistributor: InitialRequests=[Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.UnorderedKVInput], [Reducer 3:OUTPUT:104857600:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput] 2018-01-05T11:28:49,461 DEBUG [TezTR-518813_1_13_1_1_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:49,461 INFO [TezTR-518813_1_13_1_0_0] impl.ShuffleInputEventHandlerImpl: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:28:49,461 INFO [TezTR-518813_1_13_1_0_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:28:49,461 INFO [TezTR-518813_1_13_1_3_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:49,461 INFO [TezTR-518813_1_13_1_1_0] impl.SimpleFetchedInputAllocator: Map_1: RequestedMemory=131404920, AssignedMemory=131404920, maxSingleShuffleLimit=32851230 2018-01-05T11:28:49,462 INFO [TezTR-518813_1_13_1_3_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:49,462 INFO [TezTR-518813_1_13_1_3_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 236262520, TotalRequestedScaled: 1.1813126E8, TotalJVMHeap: 954728448, TotalAvailable: 639668060, TotalRequested/TotalJVMHeap:0.25 2018-01-05T11:28:49,462 INFO [TezTR-518813_1_13_1_3_0] resources.MemoryDistributor: Allocations=[Map 1:org.apache.tez.runtime.library.input.UnorderedKVInput:INPUT:131404920:131404920], [Reducer 3:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput:OUTPUT:104857600:104857600] 2018-01-05T11:28:49,462 INFO [TezTR-518813_1_13_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:49,462 INFO [TezTR-518813_1_13_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 0 2018-01-05T11:28:49,462 INFO [TezTR-518813_1_13_1_3_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:49,462 INFO [TezTR-518813_1_13_1_3_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_13_01_000003_0 2018-01-05T11:28:49,462 DEBUG [TezTR-518813_1_13_1_3_0] log.PerfLogger: 2018-01-05T11:28:49,462 DEBUG [TezTR-518813_1_13_1_3_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_13_01_000003_0_10073 2018-01-05T11:28:49,462 DEBUG [TezTR-518813_1_13_1_3_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:49,462 DEBUG [TezTR-518813_1_13_1_3_0] mr.ObjectCache: Creating djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:28:49,463 INFO [TezTR-518813_1_13_1_3_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/djaiswal/_tez_scratch_dir/3fd70205-c245-4b8e-9c69-f4cf50e400e9/reduce.xml 2018-01-05T11:28:49,463 DEBUG [TezTR-518813_1_13_1_3_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/djaiswal/_tez_scratch_dir/3fd70205-c245-4b8e-9c69-f4cf50e400e9/reduce.xml 2018-01-05T11:28:49,463 DEBUG [TezTR-518813_1_13_1_3_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/djaiswal/_tez_scratch_dir/3fd70205-c245-4b8e-9c69-f4cf50e400e9/reduce.xml 2018-01-05T11:28:49,463 DEBUG [TezTR-518813_1_13_1_3_0] log.PerfLogger: 2018-01-05T11:28:49,463 INFO [TezTR-518813_1_13_1_3_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:28:49,463 DEBUG [TezTR-518813_1_13_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,463 DEBUG [TezTR-518813_1_13_1_1_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:49,463 DEBUG [TezTR-518813_1_13_1_1_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:49,463 DEBUG [TezTR-518813_1_13_1_1_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:49,463 DEBUG [TezTR-518813_1_13_1_1_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:49,463 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=13&reduce=0&map=attempt_1515180518813_0001_13_00_000000_0_10063 mapId: [attempt_1515180518813_0001_13_00_000000_0_10063] reduceId: [0] jobId: [job_1515180518813_0001] dagId: [13] keepAlive: false 2018-01-05T11:28:49,463 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=13&reduce=0&map=attempt_1515180518813_0001_13_00_000000_0_10063; hash=...HZjX3i6HC+bio 2018-01-05T11:28:49,463 INFO [TezTR-518813_1_13_1_1_0] impl.ShuffleManager: Map_1: numInputs=1, compressionCodec=NoCompressionCodec, numFetchers=1, ifileBufferSize=4096, ifileReadAheadEnabled=true, ifileReadAheadLength=4194304, localDiskFetchEnabled=false, sharedFetchEnabled=false, keepAlive=false, keepAliveMaxConnections=20, connectionTimeout=20000, readTimeout=180000, bufferSize=8192, bufferSize=8192, maxTaskOutputAtOnce=20 2018-01-05T11:28:49,463 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=13&reduce=0&map=attempt_1515180518813_0001_13_00_000000_0_10063;reply=sTpgDTVnoTGfE 2018-01-05T11:28:49,463 DEBUG [TezTR-518813_1_13_1_2_0] log.PerfLogger: 2018-01-05T11:28:49,463 INFO [TezTR-518813_1_13_1_2_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 3.17KB 2018-01-05T11:28:49,463 INFO [TezTR-518813_1_13_1_1_0] impl.ShuffleInputEventHandlerImpl: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:28:49,463 INFO [TezTR-518813_1_13_1_1_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:28:49,464 DEBUG [TezTR-518813_1_13_1_2_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:49,464 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=13, user='djaiswal', attemptId='attempt_1515180518813_0001_13_00_000000_0_10063'} via loader 2018-01-05T11:28:49,464 DEBUG [TezTR-518813_1_13_1_2_0] log.PerfLogger: 2018-01-05T11:28:49,464 DEBUG [TezTR-518813_1_13_1_2_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:49,464 DEBUG [TezTR-518813_1_13_1_2_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:49,464 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=13, user='djaiswal', attemptId='attempt_1515180518813_0001_13_00_000000_0_10063'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:28:49,464 DEBUG [New I/O worker #2] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_13_00_000000_0_10063 not found 2018-01-05T11:28:49,464 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_13_00_000000_0_10063,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/13/output/attempt_1515180518813_0001_13_00_000000_0_10063/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/13/output/attempt_1515180518813_0001_13_00_000000_0_10063/file.out.index 2018-01-05T11:28:49,464 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: Content Length in shuffle : 1734 2018-01-05T11:28:49,464 DEBUG [TezTR-518813_1_13_1_2_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:49,464 INFO [TezTR-518813_1_13_1_2_0] impl.SimpleFetchedInputAllocator: Map_1: RequestedMemory=131404920, AssignedMemory=131404920, maxSingleShuffleLimit=32851230 2018-01-05T11:28:49,464 DEBUG [TezTR-518813_1_13_1_3_0] log.PerfLogger: 2018-01-05T11:28:49,465 INFO [TezTR-518813_1_13_1_3_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 3.17KB 2018-01-05T11:28:49,465 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=13&reduce=1&map=attempt_1515180518813_0001_13_00_000000_0_10063 mapId: [attempt_1515180518813_0001_13_00_000000_0_10063] reduceId: [1] jobId: [job_1515180518813_0001] dagId: [13] keepAlive: false 2018-01-05T11:28:49,465 INFO [Fetcher_B {Map_1} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=13&reduce=0&map=attempt_1515180518813_0001_13_00_000000_0_10063 sent hash and receievd reply 0 ms 2018-01-05T11:28:49,465 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=13&reduce=1&map=attempt_1515180518813_0001_13_00_000000_0_10063; hash=...b32Hwaa46eG4c 2018-01-05T11:28:49,465 DEBUG [TezTR-518813_1_13_1_3_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:49,465 DEBUG [TezTR-518813_1_13_1_3_0] log.PerfLogger: 2018-01-05T11:28:49,465 DEBUG [TezTR-518813_1_13_1_3_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:49,465 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=13&reduce=1&map=attempt_1515180518813_0001_13_00_000000_0_10063;reply=tMqMAGn+hfjQE 2018-01-05T11:28:49,465 DEBUG [TezTR-518813_1_13_1_3_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:49,465 INFO [Fetcher_B {Map_1} #0] ShuffleManager.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_13_00_000000_0_10063} to MEMORY, csize=1679, dsize=1675, EndTime=1515180529465, TimeTaken=0, Rate=0.00 MB/s 2018-01-05T11:28:49,465 INFO [Fetcher_B {Map_1} #0] impl.ShuffleManager: All inputs fetched for input vertex : Map 1 2018-01-05T11:28:49,465 INFO [TezTR-518813_1_13_1_0_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:28:49,465 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=13, user='djaiswal', attemptId='attempt_1515180518813_0001_13_00_000000_0_10063'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:28:49,465 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache HIT: MapId attempt_1515180518813_0001_13_00_000000_0_10063 found 2018-01-05T11:28:49,465 DEBUG [TezTR-518813_1_13_1_2_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:49,465 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_13_00_000000_0_10063,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/13/output/attempt_1515180518813_0001_13_00_000000_0_10063/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/13/output/attempt_1515180518813_0001_13_00_000000_0_10063/file.out.index 2018-01-05T11:28:49,465 INFO [TezTR-518813_1_13_1_0_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:49,465 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Content Length in shuffle : 2027 2018-01-05T11:28:49,465 DEBUG [TezTR-518813_1_13_1_2_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:49,465 DEBUG [TezTR-518813_1_13_1_2_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:49,465 INFO [Fetcher_B {Map_1} #0] impl.ShuffleManager: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:28:49,465 DEBUG [TezTR-518813_1_13_1_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:28:49,465 DEBUG [TezTR-518813_1_13_1_2_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:49,465 DEBUG [TezTR-518813_1_13_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,465 INFO [ShuffleRunner {Map_1}] impl.ShuffleManager: Map_1: Shutting down FetchScheduler, Was Interrupted: false 2018-01-05T11:28:49,465 DEBUG [TezTR-518813_1_13_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,465 INFO [ShuffleRunner {Map_1}] impl.ShuffleManager: Map_1: Scheduler thread completed 2018-01-05T11:28:49,465 DEBUG [TezTR-518813_1_13_1_3_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:49,465 DEBUG [TezTR-518813_1_13_1_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[int, string] 2018-01-05T11:28:49,465 INFO [Fetcher_B {Map_1} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=13&reduce=1&map=attempt_1515180518813_0001_13_00_000000_0_10063 sent hash and receievd reply 0 ms 2018-01-05T11:28:49,465 INFO [TezTR-518813_1_13_1_2_0] impl.ShuffleManager: Map_1: numInputs=1, compressionCodec=NoCompressionCodec, numFetchers=1, ifileBufferSize=4096, ifileReadAheadEnabled=true, ifileReadAheadLength=4194304, localDiskFetchEnabled=false, sharedFetchEnabled=false, keepAlive=false, keepAliveMaxConnections=20, connectionTimeout=20000, readTimeout=180000, bufferSize=8192, bufferSize=8192, maxTaskOutputAtOnce=20 2018-01-05T11:28:49,465 INFO [TezTR-518813_1_13_1_3_0] impl.SimpleFetchedInputAllocator: Map_1: RequestedMemory=131404920, AssignedMemory=131404920, maxSingleShuffleLimit=32851230 2018-01-05T11:28:49,465 DEBUG [TezTR-518813_1_13_1_0_0] log.PerfLogger: 2018-01-05T11:28:49,465 DEBUG [New I/O worker #1] shufflehandler.FadvisedFileRegion: shuffleBufferSize: 131072, path: /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/13/output/attempt_1515180518813_0001_13_00_000000_0_10063/file.out 2018-01-05T11:28:49,465 INFO [TezTR-518813_1_13_1_0_0] exec.SelectOperator: Initializing operator SEL[3] 2018-01-05T11:28:49,465 INFO [TezTR-518813_1_13_1_0_0] exec.SelectOperator: SELECT struct,value:struct<_col0:int,_col1:string>> 2018-01-05T11:28:49,465 INFO [Fetcher_B {Map_1} #0] ShuffleManager.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_13_00_000000_0_10063} to MEMORY, csize=1972, dsize=1968, EndTime=1515180529465, TimeTaken=0, Rate=0.00 MB/s 2018-01-05T11:28:49,465 DEBUG [TezTR-518813_1_13_1_0_0] exec.SelectOperator: Initialization Done 3 SEL 2018-01-05T11:28:49,465 DEBUG [TezTR-518813_1_13_1_0_0] exec.SelectOperator: Operator 3 SEL initialized 2018-01-05T11:28:49,465 DEBUG [TezTR-518813_1_13_1_0_0] exec.SelectOperator: Initializing children of 3 SEL 2018-01-05T11:28:49,465 DEBUG [TezTR-518813_1_13_1_0_0] exec.FileSinkOperator: Initializing child 4 FS 2018-01-05T11:28:49,465 INFO [Fetcher_B {Map_1} #0] impl.ShuffleManager: All inputs fetched for input vertex : Map 1 2018-01-05T11:28:49,465 INFO [TezTR-518813_1_13_1_1_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:28:49,465 INFO [TezTR-518813_1_13_1_0_0] exec.FileSinkOperator: Initializing operator FS[4] 2018-01-05T11:28:49,465 DEBUG [TezTR-518813_1_13_1_3_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:49,465 DEBUG [TezTR-518813_1_13_1_3_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:49,465 DEBUG [TezTR-518813_1_13_1_3_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:49,465 INFO [TezTR-518813_1_13_1_1_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:49,465 INFO [Fetcher_B {Map_1} #0] impl.ShuffleManager: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:28:49,465 INFO [TezTR-518813_1_13_1_2_0] impl.ShuffleInputEventHandlerImpl: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:28:49,466 INFO [TezTR-518813_1_13_1_2_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:28:49,466 INFO [ShuffleRunner {Map_1}] impl.ShuffleManager: Map_1: Shutting down FetchScheduler, Was Interrupted: false 2018-01-05T11:28:49,466 DEBUG [TezTR-518813_1_13_1_1_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:28:49,465 DEBUG [TezTR-518813_1_13_1_3_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:49,466 INFO [ShuffleRunner {Map_1}] impl.ShuffleManager: Map_1: Scheduler thread completed 2018-01-05T11:28:49,466 DEBUG [TezTR-518813_1_13_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,466 DEBUG [TezTR-518813_1_13_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,466 DEBUG [TezTR-518813_1_13_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,466 DEBUG [TezTR-518813_1_13_1_1_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[int, string] 2018-01-05T11:28:49,466 DEBUG [TezTR-518813_1_13_1_1_0] log.PerfLogger: 2018-01-05T11:28:49,466 INFO [TezTR-518813_1_13_1_1_0] exec.SelectOperator: Initializing operator SEL[3] 2018-01-05T11:28:49,466 INFO [TezTR-518813_1_13_1_3_0] impl.ShuffleManager: Map_1: numInputs=1, compressionCodec=NoCompressionCodec, numFetchers=1, ifileBufferSize=4096, ifileReadAheadEnabled=true, ifileReadAheadLength=4194304, localDiskFetchEnabled=false, sharedFetchEnabled=false, keepAlive=false, keepAliveMaxConnections=20, connectionTimeout=20000, readTimeout=180000, bufferSize=8192, bufferSize=8192, maxTaskOutputAtOnce=20 2018-01-05T11:28:49,466 INFO [TezTR-518813_1_13_1_1_0] exec.SelectOperator: SELECT struct,value:struct<_col0:int,_col1:string>> 2018-01-05T11:28:49,466 INFO [TezTR-518813_1_13_1_0_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@545d4fcd]:[key, value]:[int, string]] and formatter : org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat@6ff49a02 2018-01-05T11:28:49,466 DEBUG [TezTR-518813_1_13_1_1_0] exec.SelectOperator: Initialization Done 3 SEL 2018-01-05T11:28:49,466 DEBUG [TezTR-518813_1_13_1_1_0] exec.SelectOperator: Operator 3 SEL initialized 2018-01-05T11:28:49,466 DEBUG [TezTR-518813_1_13_1_1_0] exec.SelectOperator: Initializing children of 3 SEL 2018-01-05T11:28:49,466 DEBUG [TezTR-518813_1_13_1_1_0] exec.FileSinkOperator: Initializing child 4 FS 2018-01-05T11:28:49,466 INFO [TezTR-518813_1_13_1_1_0] exec.FileSinkOperator: Initializing operator FS[4] 2018-01-05T11:28:49,466 DEBUG [TezTR-518813_1_13_1_0_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:28:49,467 DEBUG [TezTR-518813_1_13_1_0_0] exec.FileSinkOperator: Initialization Done 4 FS 2018-01-05T11:28:49,467 DEBUG [TezTR-518813_1_13_1_0_0] exec.FileSinkOperator: Operator 4 FS initialized 2018-01-05T11:28:49,467 DEBUG [TezTR-518813_1_13_1_0_0] exec.FileSinkOperator: Initialization Done 4 FS done is reset. 2018-01-05T11:28:49,467 DEBUG [TezTR-518813_1_13_1_0_0] exec.SelectOperator: Initializing child 1 SEL 2018-01-05T11:28:49,467 INFO [TezTR-518813_1_13_1_0_0] exec.SelectOperator: Initializing operator SEL[1] 2018-01-05T11:28:49,467 INFO [TezTR-518813_1_13_1_0_0] exec.SelectOperator: SELECT struct<_col0:int,_col1:string> 2018-01-05T11:28:49,467 DEBUG [TezTR-518813_1_13_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,467 INFO [TezTR-518813_1_13_1_3_0] impl.ShuffleInputEventHandlerImpl: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:28:49,467 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=13&reduce=2&map=attempt_1515180518813_0001_13_00_000000_0_10063 mapId: [attempt_1515180518813_0001_13_00_000000_0_10063] reduceId: [2] jobId: [job_1515180518813_0001] dagId: [13] keepAlive: false 2018-01-05T11:28:49,467 DEBUG [TezTR-518813_1_13_1_0_0] exec.SelectOperator: Initialization Done 1 SEL 2018-01-05T11:28:49,467 INFO [TezTR-518813_1_13_1_1_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@6fe64080]:[key, value]:[int, string]] and formatter : org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat@6871acb9 2018-01-05T11:28:49,467 INFO [TezTR-518813_1_13_1_3_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:28:49,467 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=13&reduce=2&map=attempt_1515180518813_0001_13_00_000000_0_10063; hash=...9ztZzAppL0wOc 2018-01-05T11:28:49,467 DEBUG [TezTR-518813_1_13_1_1_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:28:49,467 DEBUG [TezTR-518813_1_13_1_0_0] exec.SelectOperator: Operator 1 SEL initialized 2018-01-05T11:28:49,467 DEBUG [TezTR-518813_1_13_1_0_0] exec.SelectOperator: Initializing children of 1 SEL 2018-01-05T11:28:49,467 DEBUG [TezTR-518813_1_13_1_0_0] exec.GroupByOperator: Initializing child 2 GBY 2018-01-05T11:28:49,467 DEBUG [TezTR-518813_1_13_1_1_0] exec.FileSinkOperator: Initialization Done 4 FS 2018-01-05T11:28:49,467 INFO [TezTR-518813_1_13_1_0_0] exec.GroupByOperator: Initializing operator GBY[2] 2018-01-05T11:28:49,467 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=13&reduce=2&map=attempt_1515180518813_0001_13_00_000000_0_10063;reply=Q/CGUbTOKhp/U 2018-01-05T11:28:49,467 DEBUG [TezTR-518813_1_13_1_1_0] exec.FileSinkOperator: Operator 4 FS initialized 2018-01-05T11:28:49,468 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=13, user='djaiswal', attemptId='attempt_1515180518813_0001_13_00_000000_0_10063'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:28:49,468 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache HIT: MapId attempt_1515180518813_0001_13_00_000000_0_10063 found 2018-01-05T11:28:49,468 DEBUG [TezTR-518813_1_13_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,468 DEBUG [TezTR-518813_1_13_1_1_0] exec.FileSinkOperator: Initialization Done 4 FS done is reset. 2018-01-05T11:28:49,468 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_13_00_000000_0_10063,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/13/output/attempt_1515180518813_0001_13_00_000000_0_10063/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/13/output/attempt_1515180518813_0001_13_00_000000_0_10063/file.out.index 2018-01-05T11:28:49,468 DEBUG [TezTR-518813_1_13_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,468 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Content Length in shuffle : 1865 2018-01-05T11:28:49,468 DEBUG [TezTR-518813_1_13_1_1_0] exec.SelectOperator: Initializing child 1 SEL 2018-01-05T11:28:49,468 INFO [TezTR-518813_1_13_1_1_0] exec.SelectOperator: Initializing operator SEL[1] 2018-01-05T11:28:49,468 INFO [TezTR-518813_1_13_1_1_0] exec.SelectOperator: SELECT struct<_col0:int,_col1:string> 2018-01-05T11:28:49,468 DEBUG [TezTR-518813_1_13_1_1_0] exec.SelectOperator: Initialization Done 1 SEL 2018-01-05T11:28:49,468 DEBUG [TezTR-518813_1_13_1_1_0] exec.SelectOperator: Operator 1 SEL initialized 2018-01-05T11:28:49,468 DEBUG [TezTR-518813_1_13_1_1_0] exec.SelectOperator: Initializing children of 1 SEL 2018-01-05T11:28:49,468 INFO [TezTR-518813_1_13_1_0_0] exec.GroupByOperator: Max hash table memory: 93860656 bytes 2018-01-05T11:28:49,468 DEBUG [TezTR-518813_1_13_1_1_0] exec.GroupByOperator: Initializing child 2 GBY 2018-01-05T11:28:49,468 INFO [TezTR-518813_1_13_1_1_0] exec.GroupByOperator: Initializing operator GBY[2] 2018-01-05T11:28:49,468 INFO [TezTR-518813_1_13_1_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:49,468 DEBUG [TezTR-518813_1_13_1_0_0] exec.GroupByOperator: Initialization Done 2 GBY 2018-01-05T11:28:49,468 DEBUG [TezTR-518813_1_13_1_0_0] exec.GroupByOperator: Operator 2 GBY initialized 2018-01-05T11:28:49,468 DEBUG [TezTR-518813_1_13_1_0_0] exec.GroupByOperator: Initializing children of 2 GBY 2018-01-05T11:28:49,468 DEBUG [TezTR-518813_1_13_1_0_0] exec.ReduceSinkOperator: Initializing child 3 RS 2018-01-05T11:28:49,468 INFO [TezTR-518813_1_13_1_0_0] exec.ReduceSinkOperator: Initializing operator RS[3] 2018-01-05T11:28:49,468 DEBUG [TezTR-518813_1_13_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,468 DEBUG [TezTR-518813_1_13_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,468 DEBUG [TezTR-518813_1_13_1_0_0] exec.ReduceSinkOperator: keys size is 0 2018-01-05T11:28:49,468 INFO [Fetcher_B {Map_1} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=13&reduce=2&map=attempt_1515180518813_0001_13_00_000000_0_10063 sent hash and receievd reply 0 ms 2018-01-05T11:28:49,468 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=13&reduce=3&map=attempt_1515180518813_0001_13_00_000000_0_10063 mapId: [attempt_1515180518813_0001_13_00_000000_0_10063] reduceId: [3] jobId: [job_1515180518813_0001] dagId: [13] keepAlive: false 2018-01-05T11:28:49,468 INFO [TezTR-518813_1_13_1_0_0] exec.ReduceSinkOperator: Using tag = -1 2018-01-05T11:28:49,468 DEBUG [New I/O worker #1] shufflehandler.FadvisedFileRegion: shuffleBufferSize: 131072, path: /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/13/output/attempt_1515180518813_0001_13_00_000000_0_10063/file.out 2018-01-05T11:28:49,468 INFO [TezTR-518813_1_13_1_1_0] exec.GroupByOperator: Max hash table memory: 93860656 bytes 2018-01-05T11:28:49,468 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=13&reduce=3&map=attempt_1515180518813_0001_13_00_000000_0_10063; hash=...LmIZ/HkqQN7E4 2018-01-05T11:28:49,468 DEBUG [TezTR-518813_1_13_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,468 DEBUG [TezTR-518813_1_13_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,468 INFO [TezTR-518813_1_13_1_1_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:49,468 DEBUG [TezTR-518813_1_13_1_1_0] exec.GroupByOperator: Initialization Done 2 GBY 2018-01-05T11:28:49,468 DEBUG [TezTR-518813_1_13_1_1_0] exec.GroupByOperator: Operator 2 GBY initialized 2018-01-05T11:28:49,468 DEBUG [TezTR-518813_1_13_1_1_0] exec.GroupByOperator: Initializing children of 2 GBY 2018-01-05T11:28:49,468 DEBUG [TezTR-518813_1_13_1_1_0] exec.ReduceSinkOperator: Initializing child 3 RS 2018-01-05T11:28:49,468 INFO [TezTR-518813_1_13_1_1_0] exec.ReduceSinkOperator: Initializing operator RS[3] 2018-01-05T11:28:49,468 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=13&reduce=3&map=attempt_1515180518813_0001_13_00_000000_0_10063;reply=G4tJj0swHz0ck 2018-01-05T11:28:49,468 INFO [Fetcher_B {Map_1} #0] ShuffleManager.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_13_00_000000_0_10063} to MEMORY, csize=1810, dsize=1806, EndTime=1515180529468, TimeTaken=0, Rate=0.00 MB/s 2018-01-05T11:28:49,468 DEBUG [TezTR-518813_1_13_1_1_0] exec.ReduceSinkOperator: keys size is 0 2018-01-05T11:28:49,468 INFO [Fetcher_B {Map_1} #0] impl.ShuffleManager: All inputs fetched for input vertex : Map 1 2018-01-05T11:28:49,468 DEBUG [TezTR-518813_1_13_1_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:28:49,468 DEBUG [TezTR-518813_1_13_1_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS 2018-01-05T11:28:49,468 DEBUG [TezTR-518813_1_13_1_0_0] exec.ReduceSinkOperator: Operator 3 RS initialized 2018-01-05T11:28:49,468 DEBUG [TezTR-518813_1_13_1_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS done is reset. 2018-01-05T11:28:49,468 DEBUG [TezTR-518813_1_13_1_0_0] exec.GroupByOperator: Initialization Done 2 GBY done is reset. 2018-01-05T11:28:49,468 DEBUG [TezTR-518813_1_13_1_0_0] exec.SelectOperator: Initialization Done 1 SEL done is reset. 2018-01-05T11:28:49,468 DEBUG [TezTR-518813_1_13_1_0_0] exec.SelectOperator: Initialization Done 3 SEL done is reset. 2018-01-05T11:28:49,468 INFO [Fetcher_B {Map_1} #0] impl.ShuffleManager: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:28:49,468 INFO [TezTR-518813_1_13_1_2_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:28:49,468 INFO [TezTR-518813_1_13_1_1_0] exec.ReduceSinkOperator: Using tag = -1 2018-01-05T11:28:49,468 DEBUG [TezTR-518813_1_13_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,468 INFO [TezTR-518813_1_13_1_2_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:49,468 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=13, user='djaiswal', attemptId='attempt_1515180518813_0001_13_00_000000_0_10063'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:28:49,468 INFO [ShuffleRunner {Map_1}] impl.ShuffleManager: Map_1: Shutting down FetchScheduler, Was Interrupted: false 2018-01-05T11:28:49,468 DEBUG [New I/O worker #2] shufflehandler.IndexCache: IndexCache HIT: MapId attempt_1515180518813_0001_13_00_000000_0_10063 found 2018-01-05T11:28:49,468 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_13_00_000000_0_10063,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/13/output/attempt_1515180518813_0001_13_00_000000_0_10063/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/13/output/attempt_1515180518813_0001_13_00_000000_0_10063/file.out.index 2018-01-05T11:28:49,468 DEBUG [TezTR-518813_1_13_1_2_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:28:49,468 DEBUG [TezTR-518813_1_13_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,468 INFO [TezTR-518813_1_13_1_0_0] tez.ReduceRecordProcessor: Id =3 Id =4 <\Children> Id = 3 null<\Parent> <\FS> Id =1 Id =2 null <\Children> Id = 1 null<\Parent> <\GBY> <\Children> Id = 3 null<\Parent> <\SEL> <\Children> <\Parent> <\SEL> 2018-01-05T11:28:49,468 INFO [ShuffleRunner {Map_1}] impl.ShuffleManager: Map_1: Scheduler thread completed 2018-01-05T11:28:49,468 INFO [TezTR-518813_1_13_1_0_0] exec.OperatorUtils: Setting output collector: RS[3] --> Reducer 3 2018-01-05T11:28:49,468 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: Content Length in shuffle : 1685 2018-01-05T11:28:49,468 DEBUG [TezTR-518813_1_13_1_1_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:28:49,468 DEBUG [TezTR-518813_1_13_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,468 DEBUG [TezTR-518813_1_13_1_1_0] exec.ReduceSinkOperator: Initialization Done 3 RS 2018-01-05T11:28:49,468 DEBUG [TezTR-518813_1_13_1_1_0] exec.ReduceSinkOperator: Operator 3 RS initialized 2018-01-05T11:28:49,468 DEBUG [TezTR-518813_1_13_1_0_0] log.PerfLogger: 2018-01-05T11:28:49,468 INFO [TezTR-518813_1_13_1_0_0] tez.ReduceRecordProcessor: Starting Output: Reducer 3 2018-01-05T11:28:49,468 DEBUG [TezTR-518813_1_13_1_1_0] exec.ReduceSinkOperator: Initialization Done 3 RS done is reset. 2018-01-05T11:28:49,469 DEBUG [TezTR-518813_1_13_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,469 DEBUG [TezTR-518813_1_13_1_1_0] exec.GroupByOperator: Initialization Done 2 GBY done is reset. 2018-01-05T11:28:49,469 INFO [TezTR-518813_1_13_1_0_0] writers.BaseUnorderedPartitionedKVWriter: Instantiating Partitioner: [org.apache.tez.mapreduce.partition.MRPartitioner] 2018-01-05T11:28:49,469 DEBUG [TezTR-518813_1_13_1_1_0] exec.SelectOperator: Initialization Done 1 SEL done is reset. 2018-01-05T11:28:49,469 DEBUG [TezTR-518813_1_13_1_2_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[int, string] 2018-01-05T11:28:49,469 DEBUG [TezTR-518813_1_13_1_1_0] exec.SelectOperator: Initialization Done 3 SEL done is reset. 2018-01-05T11:28:49,469 DEBUG [TezTR-518813_1_13_1_2_0] log.PerfLogger: 2018-01-05T11:28:49,469 INFO [TezTR-518813_1_13_1_1_0] tez.ReduceRecordProcessor: Id =3 Id =4 <\Children> Id = 3 null<\Parent> <\FS> Id =1 Id =2 null <\Children> Id = 1 null<\Parent> <\GBY> <\Children> Id = 3 null<\Parent> <\SEL> <\Children> <\Parent> <\SEL> 2018-01-05T11:28:49,469 INFO [TezTR-518813_1_13_1_2_0] exec.SelectOperator: Initializing operator SEL[3] 2018-01-05T11:28:49,469 INFO [Fetcher_B {Map_1} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=13&reduce=3&map=attempt_1515180518813_0001_13_00_000000_0_10063 sent hash and receievd reply 0 ms 2018-01-05T11:28:49,469 INFO [TezTR-518813_1_13_1_2_0] exec.SelectOperator: SELECT struct,value:struct<_col0:int,_col1:string>> 2018-01-05T11:28:49,469 INFO [TezTR-518813_1_13_1_1_0] exec.OperatorUtils: Setting output collector: RS[3] --> Reducer 3 2018-01-05T11:28:49,469 DEBUG [New I/O worker #2] shufflehandler.FadvisedFileRegion: shuffleBufferSize: 131072, path: /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/13/output/attempt_1515180518813_0001_13_00_000000_0_10063/file.out 2018-01-05T11:28:49,469 DEBUG [TezTR-518813_1_13_1_2_0] exec.SelectOperator: Initialization Done 3 SEL 2018-01-05T11:28:49,469 DEBUG [TezTR-518813_1_13_1_2_0] exec.SelectOperator: Operator 3 SEL initialized 2018-01-05T11:28:49,469 DEBUG [TezTR-518813_1_13_1_1_0] log.PerfLogger: 2018-01-05T11:28:49,469 DEBUG [TezTR-518813_1_13_1_2_0] exec.SelectOperator: Initializing children of 3 SEL 2018-01-05T11:28:49,469 DEBUG [TezTR-518813_1_13_1_2_0] exec.FileSinkOperator: Initializing child 4 FS 2018-01-05T11:28:49,469 INFO [TezTR-518813_1_13_1_1_0] tez.ReduceRecordProcessor: Starting Output: Reducer 3 2018-01-05T11:28:49,469 INFO [Fetcher_B {Map_1} #0] ShuffleManager.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_13_00_000000_0_10063} to MEMORY, csize=1630, dsize=1626, EndTime=1515180529469, TimeTaken=0, Rate=0.00 MB/s 2018-01-05T11:28:49,475 INFO [TezTR-518813_1_13_1_3_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:28:49,469 INFO [TezTR-518813_1_13_1_2_0] exec.FileSinkOperator: Initializing operator FS[4] 2018-01-05T11:28:49,475 INFO [TezTR-518813_1_13_1_1_0] writers.BaseUnorderedPartitionedKVWriter: Instantiating Partitioner: [org.apache.tez.mapreduce.partition.MRPartitioner] 2018-01-05T11:28:49,474 INFO [Fetcher_B {Map_1} #0] impl.ShuffleManager: All inputs fetched for input vertex : Map 1 2018-01-05T11:28:49,475 INFO [TezTR-518813_1_13_1_3_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:49,475 DEBUG [TezTR-518813_1_13_1_3_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:28:49,475 DEBUG [TezTR-518813_1_13_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,475 INFO [Fetcher_B {Map_1} #0] impl.ShuffleManager: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:28:49,475 DEBUG [TezTR-518813_1_13_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,475 DEBUG [TezTR-518813_1_13_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,475 DEBUG [TezTR-518813_1_13_1_3_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[int, string] 2018-01-05T11:28:49,475 INFO [ShuffleRunner {Map_1}] impl.ShuffleManager: Map_1: Shutting down FetchScheduler, Was Interrupted: false 2018-01-05T11:28:49,475 DEBUG [TezTR-518813_1_13_1_3_0] log.PerfLogger: 2018-01-05T11:28:49,475 INFO [TezTR-518813_1_13_1_3_0] exec.SelectOperator: Initializing operator SEL[3] 2018-01-05T11:28:49,475 INFO [ShuffleRunner {Map_1}] impl.ShuffleManager: Map_1: Scheduler thread completed 2018-01-05T11:28:49,475 INFO [TezTR-518813_1_13_1_3_0] exec.SelectOperator: SELECT struct,value:struct<_col0:int,_col1:string>> 2018-01-05T11:28:49,475 DEBUG [TezTR-518813_1_13_1_3_0] exec.SelectOperator: Initialization Done 3 SEL 2018-01-05T11:28:49,475 DEBUG [TezTR-518813_1_13_1_3_0] exec.SelectOperator: Operator 3 SEL initialized 2018-01-05T11:28:49,475 DEBUG [TezTR-518813_1_13_1_3_0] exec.SelectOperator: Initializing children of 3 SEL 2018-01-05T11:28:49,475 INFO [TezTR-518813_1_13_1_2_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@1fc4f994]:[key, value]:[int, string]] and formatter : org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat@69e5892a 2018-01-05T11:28:49,475 DEBUG [TezTR-518813_1_13_1_3_0] exec.FileSinkOperator: Initializing child 4 FS 2018-01-05T11:28:49,475 INFO [TezTR-518813_1_13_1_3_0] exec.FileSinkOperator: Initializing operator FS[4] 2018-01-05T11:28:49,475 DEBUG [TezTR-518813_1_13_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,475 DEBUG [TezTR-518813_1_13_1_2_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:28:49,475 INFO [TezTR-518813_1_13_1_3_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@7b956089]:[key, value]:[int, string]] and formatter : org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat@a418ffa 2018-01-05T11:28:49,475 DEBUG [TezTR-518813_1_13_1_2_0] exec.FileSinkOperator: Initialization Done 4 FS 2018-01-05T11:28:49,475 DEBUG [TezTR-518813_1_13_1_2_0] exec.FileSinkOperator: Operator 4 FS initialized 2018-01-05T11:28:49,475 DEBUG [TezTR-518813_1_13_1_2_0] exec.FileSinkOperator: Initialization Done 4 FS done is reset. 2018-01-05T11:28:49,475 DEBUG [TezTR-518813_1_13_1_2_0] exec.SelectOperator: Initializing child 1 SEL 2018-01-05T11:28:49,475 DEBUG [TezTR-518813_1_13_1_3_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:28:49,475 DEBUG [TezTR-518813_1_13_1_3_0] exec.FileSinkOperator: Initialization Done 4 FS 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_3_0] exec.FileSinkOperator: Operator 4 FS initialized 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_3_0] exec.FileSinkOperator: Initialization Done 4 FS done is reset. 2018-01-05T11:28:49,475 INFO [TezTR-518813_1_13_1_2_0] exec.SelectOperator: Initializing operator SEL[1] 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_3_0] exec.SelectOperator: Initializing child 1 SEL 2018-01-05T11:28:49,476 INFO [TezTR-518813_1_13_1_2_0] exec.SelectOperator: SELECT struct<_col0:int,_col1:string> 2018-01-05T11:28:49,476 INFO [TezTR-518813_1_13_1_3_0] exec.SelectOperator: Initializing operator SEL[1] 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_2_0] exec.SelectOperator: Initialization Done 1 SEL 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_2_0] exec.SelectOperator: Operator 1 SEL initialized 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_2_0] exec.SelectOperator: Initializing children of 1 SEL 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_2_0] exec.GroupByOperator: Initializing child 2 GBY 2018-01-05T11:28:49,476 INFO [TezTR-518813_1_13_1_3_0] exec.SelectOperator: SELECT struct<_col0:int,_col1:string> 2018-01-05T11:28:49,476 INFO [TezTR-518813_1_13_1_2_0] exec.GroupByOperator: Initializing operator GBY[2] 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_3_0] exec.SelectOperator: Initialization Done 1 SEL 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_3_0] exec.SelectOperator: Operator 1 SEL initialized 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_3_0] exec.SelectOperator: Initializing children of 1 SEL 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_3_0] exec.GroupByOperator: Initializing child 2 GBY 2018-01-05T11:28:49,476 INFO [TezTR-518813_1_13_1_3_0] exec.GroupByOperator: Initializing operator GBY[2] 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,476 INFO [TezTR-518813_1_13_1_2_0] exec.GroupByOperator: Max hash table memory: 93860656 bytes 2018-01-05T11:28:49,476 INFO [TezTR-518813_1_13_1_3_0] exec.GroupByOperator: Max hash table memory: 93860656 bytes 2018-01-05T11:28:49,476 INFO [TezTR-518813_1_13_1_2_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:49,476 INFO [TezTR-518813_1_13_1_3_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_3_0] exec.GroupByOperator: Initialization Done 2 GBY 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_3_0] exec.GroupByOperator: Operator 2 GBY initialized 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_3_0] exec.GroupByOperator: Initializing children of 2 GBY 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_3_0] exec.ReduceSinkOperator: Initializing child 3 RS 2018-01-05T11:28:49,476 INFO [TezTR-518813_1_13_1_3_0] exec.ReduceSinkOperator: Initializing operator RS[3] 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_2_0] exec.GroupByOperator: Initialization Done 2 GBY 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_3_0] exec.ReduceSinkOperator: keys size is 0 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_2_0] exec.GroupByOperator: Operator 2 GBY initialized 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_2_0] exec.GroupByOperator: Initializing children of 2 GBY 2018-01-05T11:28:49,476 INFO [TezTR-518813_1_13_1_3_0] exec.ReduceSinkOperator: Using tag = -1 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_2_0] exec.ReduceSinkOperator: Initializing child 3 RS 2018-01-05T11:28:49,476 INFO [TezTR-518813_1_13_1_2_0] exec.ReduceSinkOperator: Initializing operator RS[3] 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_2_0] exec.ReduceSinkOperator: keys size is 0 2018-01-05T11:28:49,476 INFO [TezTR-518813_1_13_1_2_0] exec.ReduceSinkOperator: Using tag = -1 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_3_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_2_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_2_0] exec.ReduceSinkOperator: Initialization Done 3 RS 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_2_0] exec.ReduceSinkOperator: Operator 3 RS initialized 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_3_0] exec.ReduceSinkOperator: Initialization Done 3 RS 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_3_0] exec.ReduceSinkOperator: Operator 3 RS initialized 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_2_0] exec.ReduceSinkOperator: Initialization Done 3 RS done is reset. 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_3_0] exec.ReduceSinkOperator: Initialization Done 3 RS done is reset. 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_3_0] exec.GroupByOperator: Initialization Done 2 GBY done is reset. 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_3_0] exec.SelectOperator: Initialization Done 1 SEL done is reset. 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_3_0] exec.SelectOperator: Initialization Done 3 SEL done is reset. 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_2_0] exec.GroupByOperator: Initialization Done 2 GBY done is reset. 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_2_0] exec.SelectOperator: Initialization Done 1 SEL done is reset. 2018-01-05T11:28:49,476 DEBUG [TezTR-518813_1_13_1_2_0] exec.SelectOperator: Initialization Done 3 SEL done is reset. 2018-01-05T11:28:49,476 INFO [TezTR-518813_1_13_1_3_0] tez.ReduceRecordProcessor: Id =3 Id =4 <\Children> Id = 3 null<\Parent> <\FS> Id =1 Id =2 null <\Children> Id = 1 null<\Parent> <\GBY> <\Children> Id = 3 null<\Parent> <\SEL> <\Children> <\Parent> <\SEL> 2018-01-05T11:28:49,477 INFO [TezTR-518813_1_13_1_2_0] tez.ReduceRecordProcessor: Id =3 Id =4 <\Children> Id = 3 null<\Parent> <\FS> Id =1 Id =2 null <\Children> Id = 1 null<\Parent> <\GBY> <\Children> Id = 3 null<\Parent> <\SEL> <\Children> <\Parent> <\SEL> 2018-01-05T11:28:49,477 INFO [TezTR-518813_1_13_1_3_0] exec.OperatorUtils: Setting output collector: RS[3] --> Reducer 3 2018-01-05T11:28:49,477 INFO [TezTR-518813_1_13_1_2_0] exec.OperatorUtils: Setting output collector: RS[3] --> Reducer 3 2018-01-05T11:28:49,477 DEBUG [TezTR-518813_1_13_1_3_0] log.PerfLogger: 2018-01-05T11:28:49,477 DEBUG [TezTR-518813_1_13_1_2_0] log.PerfLogger: 2018-01-05T11:28:49,477 INFO [TezTR-518813_1_13_1_2_0] tez.ReduceRecordProcessor: Starting Output: Reducer 3 2018-01-05T11:28:49,477 INFO [TezTR-518813_1_13_1_3_0] tez.ReduceRecordProcessor: Starting Output: Reducer 3 2018-01-05T11:28:49,481 INFO [TezTR-518813_1_13_1_3_0] writers.BaseUnorderedPartitionedKVWriter: Instantiating Partitioner: [org.apache.tez.mapreduce.partition.MRPartitioner] 2018-01-05T11:28:49,481 INFO [TezTR-518813_1_13_1_2_0] writers.BaseUnorderedPartitionedKVWriter: Instantiating Partitioner: [org.apache.tez.mapreduce.partition.MRPartitioner] 2018-01-05T11:28:49,794 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000023, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_13_01_000001_0, eventCount=1 } 2018-01-05T11:28:49,794 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000021, QueryId=djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39-13, DagId=13, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-2, Succeeded=true, StartTime=1515180529377, EndTime=1515180529794 2018-01-05T11:28:49,794 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000025, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_13_01_000003_0, eventCount=1 } 2018-01-05T11:28:49,794 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000022, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_13_01_000000_0, eventCount=1 } 2018-01-05T11:28:49,794 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000024, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_13_01_000002_0, eventCount=1 } 2018-01-05T11:28:49,795 INFO [TezTR-518813_1_13_1_0_0] writers.UnorderedPartitionedKVWriter: Reducer_3: numBuffers=2, sizePerBuffer=52428800, skipBuffers=true, numPartitions=1, availableMemory=104857600, maxSingleBufferSizeBytes=2147483647, pipelinedShuffle=false, isFinalMergeEnabled=true, numPartitions=1, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:28:49,795 DEBUG [TezTR-518813_1_13_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:28:49,795 DEBUG [TezTR-518813_1_13_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:28:49,796 DEBUG [TezTR-518813_1_13_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:28:49,796 DEBUG [TezTR-518813_1_13_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:28:49,796 DEBUG [TezTR-518813_1_13_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:28:49,796 DEBUG [TezTR-518813_1_13_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:28:49,795 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:28:49,795 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:28:49,795 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:28:49,796 DEBUG [TezTR-518813_1_13_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:28:49,795 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:28:49,796 DEBUG [TezTR-518813_1_13_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:28:49,795 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,796 DEBUG [TezTR-518813_1_13_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:28:49,796 DEBUG [TezTR-518813_1_13_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:28:49,796 DEBUG [TezTR-518813_1_13_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:28:49,796 DEBUG [TezTR-518813_1_13_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:28:49,796 DEBUG [TezTR-518813_1_13_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:28:49,796 DEBUG [TezTR-518813_1_13_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:28:49,796 DEBUG [TezTR-518813_1_13_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:28:49,796 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: Status: Running (Executing on YARN cluster with App id application_1515180518813_0001) 2018-01-05T11:28:49,796 INFO [TezTR-518813_1_13_1_0_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/.hive-staging_hive_2018-01-05_11-28-48_798_103036370913189016-1/_tmp.-ext-10000/000000_0 2018-01-05T11:28:49,796 INFO [TezTR-518813_1_13_1_0_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/.hive-staging_hive_2018-01-05_11-28-48_798_103036370913189016-1/_task_tmp.-ext-10000/_tmp.000000_0 2018-01-05T11:28:49,796 INFO [TezTR-518813_1_13_1_0_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/.hive-staging_hive_2018-01-05_11-28-48_798_103036370913189016-1/_tmp.-ext-10000/000000_0 2018-01-05T11:28:49,798 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,798 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,798 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,799 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 Reducer 2: 0(+4)/4 Reducer 3: 0/1 2018-01-05T11:28:49,812 INFO [TezTR-518813_1_13_1_3_0] writers.UnorderedPartitionedKVWriter: Reducer_3: numBuffers=2, sizePerBuffer=52428800, skipBuffers=true, numPartitions=1, availableMemory=104857600, maxSingleBufferSizeBytes=2147483647, pipelinedShuffle=false, isFinalMergeEnabled=true, numPartitions=1, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:28:49,812 INFO [TezTR-518813_1_13_1_1_0] writers.UnorderedPartitionedKVWriter: Reducer_3: numBuffers=2, sizePerBuffer=52428800, skipBuffers=true, numPartitions=1, availableMemory=104857600, maxSingleBufferSizeBytes=2147483647, pipelinedShuffle=false, isFinalMergeEnabled=true, numPartitions=1, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:28:49,812 DEBUG [TezTR-518813_1_13_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:28:49,812 DEBUG [TezTR-518813_1_13_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:28:49,812 DEBUG [TezTR-518813_1_13_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:28:49,812 DEBUG [TezTR-518813_1_13_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:28:49,812 DEBUG [TezTR-518813_1_13_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:28:49,812 DEBUG [TezTR-518813_1_13_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:28:49,812 DEBUG [TezTR-518813_1_13_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:28:49,812 DEBUG [TezTR-518813_1_13_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:28:49,812 DEBUG [TezTR-518813_1_13_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:28:49,812 DEBUG [TezTR-518813_1_13_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:28:49,813 DEBUG [TezTR-518813_1_13_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:28:49,813 DEBUG [TezTR-518813_1_13_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:28:49,813 DEBUG [TezTR-518813_1_13_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:28:49,813 DEBUG [TezTR-518813_1_13_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:28:49,813 DEBUG [TezTR-518813_1_13_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:28:49,813 DEBUG [TezTR-518813_1_13_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:28:49,813 DEBUG [TezTR-518813_1_13_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:28:49,813 DEBUG [TezTR-518813_1_13_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:28:49,813 DEBUG [TezTR-518813_1_13_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:28:49,813 DEBUG [TezTR-518813_1_13_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:28:49,813 DEBUG [TezTR-518813_1_13_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:28:49,813 DEBUG [TezTR-518813_1_13_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:28:49,813 DEBUG [TezTR-518813_1_13_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:28:49,813 DEBUG [TezTR-518813_1_13_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:28:49,813 DEBUG [TezTR-518813_1_13_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:28:49,813 DEBUG [TezTR-518813_1_13_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:28:49,813 DEBUG [TezTR-518813_1_13_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:28:49,813 DEBUG [TezTR-518813_1_13_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:28:49,813 DEBUG [TezTR-518813_1_13_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:28:49,813 DEBUG [TezTR-518813_1_13_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:28:49,813 INFO [TezTR-518813_1_13_1_1_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/.hive-staging_hive_2018-01-05_11-28-48_798_103036370913189016-1/_tmp.-ext-10000/000001_0 2018-01-05T11:28:49,813 INFO [TezTR-518813_1_13_1_1_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/.hive-staging_hive_2018-01-05_11-28-48_798_103036370913189016-1/_task_tmp.-ext-10000/_tmp.000001_0 2018-01-05T11:28:49,813 INFO [TezTR-518813_1_13_1_1_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/.hive-staging_hive_2018-01-05_11-28-48_798_103036370913189016-1/_tmp.-ext-10000/000001_0 2018-01-05T11:28:49,813 INFO [TezTR-518813_1_13_1_3_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/.hive-staging_hive_2018-01-05_11-28-48_798_103036370913189016-1/_tmp.-ext-10000/000003_0 2018-01-05T11:28:49,813 INFO [TezTR-518813_1_13_1_3_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/.hive-staging_hive_2018-01-05_11-28-48_798_103036370913189016-1/_task_tmp.-ext-10000/_tmp.000003_0 2018-01-05T11:28:49,813 INFO [TezTR-518813_1_13_1_3_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/.hive-staging_hive_2018-01-05_11-28-48_798_103036370913189016-1/_tmp.-ext-10000/000003_0 2018-01-05T11:28:49,813 INFO [TezTR-518813_1_13_1_2_0] writers.UnorderedPartitionedKVWriter: Reducer_3: numBuffers=2, sizePerBuffer=52428800, skipBuffers=true, numPartitions=1, availableMemory=104857600, maxSingleBufferSizeBytes=2147483647, pipelinedShuffle=false, isFinalMergeEnabled=true, numPartitions=1, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:28:49,813 DEBUG [TezTR-518813_1_13_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:28:49,813 DEBUG [TezTR-518813_1_13_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:28:49,813 DEBUG [TezTR-518813_1_13_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:28:49,813 DEBUG [TezTR-518813_1_13_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:28:49,813 DEBUG [TezTR-518813_1_13_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:28:49,813 DEBUG [TezTR-518813_1_13_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:28:49,813 DEBUG [TezTR-518813_1_13_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:28:49,813 DEBUG [TezTR-518813_1_13_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:28:49,813 DEBUG [TezTR-518813_1_13_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:28:49,813 DEBUG [TezTR-518813_1_13_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:28:49,813 DEBUG [TezTR-518813_1_13_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:28:49,813 DEBUG [TezTR-518813_1_13_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:28:49,813 DEBUG [TezTR-518813_1_13_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:28:49,813 DEBUG [TezTR-518813_1_13_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:28:49,813 DEBUG [TezTR-518813_1_13_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:28:49,813 INFO [TezTR-518813_1_13_1_2_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/.hive-staging_hive_2018-01-05_11-28-48_798_103036370913189016-1/_tmp.-ext-10000/000002_0 2018-01-05T11:28:49,814 INFO [TezTR-518813_1_13_1_2_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/.hive-staging_hive_2018-01-05_11-28-48_798_103036370913189016-1/_task_tmp.-ext-10000/_tmp.000002_0 2018-01-05T11:28:49,814 INFO [TezTR-518813_1_13_1_2_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/.hive-staging_hive_2018-01-05_11-28-48_798_103036370913189016-1/_tmp.-ext-10000/000002_0 2018-01-05T11:28:49,836 INFO [TezTR-518813_1_13_1_0_0] exec.FileSinkOperator: FS[4]: records written - 1 2018-01-05T11:28:49,837 INFO [TezTR-518813_1_13_1_0_0] exec.FileSinkOperator: FS[4]: records written - 10 2018-01-05T11:28:49,837 INFO [TezTR-518813_1_13_1_3_0] exec.FileSinkOperator: FS[4]: records written - 1 2018-01-05T11:28:49,837 INFO [TezTR-518813_1_13_1_1_0] exec.FileSinkOperator: FS[4]: records written - 1 2018-01-05T11:28:49,837 INFO [TezTR-518813_1_13_1_2_0] exec.FileSinkOperator: FS[4]: records written - 1 2018-01-05T11:28:49,838 INFO [TezTR-518813_1_13_1_3_0] exec.FileSinkOperator: FS[4]: records written - 10 2018-01-05T11:28:49,838 INFO [TezTR-518813_1_13_1_1_0] exec.FileSinkOperator: FS[4]: records written - 10 2018-01-05T11:28:49,838 INFO [TezTR-518813_1_13_1_2_0] exec.FileSinkOperator: FS[4]: records written - 10 2018-01-05T11:28:49,838 INFO [TezTR-518813_1_13_1_0_0] exec.FileSinkOperator: FS[4]: records written - 100 2018-01-05T11:28:49,839 INFO [TezTR-518813_1_13_1_0_0] readers.UnorderedKVReader: Num Records read: 119 2018-01-05T11:28:49,839 DEBUG [TezTR-518813_1_13_1_0_0] log.PerfLogger: 2018-01-05T11:28:49,839 DEBUG [TezTR-518813_1_13_1_0_0] mr.ObjectCache: djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:28:49,839 DEBUG [TezTR-518813_1_13_1_0_0] exec.SelectOperator: close called for operator SEL[3] 2018-01-05T11:28:49,839 INFO [TezTR-518813_1_13_1_0_0] exec.SelectOperator: Closing operator SEL[3] 2018-01-05T11:28:49,839 DEBUG [TezTR-518813_1_13_1_0_0] exec.SelectOperator: Closing child = FS[4] 2018-01-05T11:28:49,839 DEBUG [TezTR-518813_1_13_1_0_0] exec.FileSinkOperator: close called for operator FS[4] 2018-01-05T11:28:49,839 DEBUG [TezTR-518813_1_13_1_0_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:49,839 INFO [TezTR-518813_1_13_1_0_0] exec.FileSinkOperator: Closing operator FS[4] 2018-01-05T11:28:49,839 INFO [TezTR-518813_1_13_1_0_0] exec.FileSinkOperator: FS[4]: records written - 119 2018-01-05T11:28:49,840 INFO [TezTR-518813_1_13_1_3_0] exec.FileSinkOperator: FS[4]: records written - 100 2018-01-05T11:28:49,840 INFO [TezTR-518813_1_13_1_1_0] exec.FileSinkOperator: FS[4]: records written - 100 2018-01-05T11:28:49,840 DEBUG [TezTR-518813_1_13_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,840 DEBUG [TezTR-518813_1_13_1_0_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/-mr-10001 2018-01-05T11:28:49,841 INFO [TezTR-518813_1_13_1_3_0] readers.UnorderedKVReader: Num Records read: 115 2018-01-05T11:28:49,841 DEBUG [TezTR-518813_1_13_1_3_0] log.PerfLogger: 2018-01-05T11:28:49,841 DEBUG [TezTR-518813_1_13_1_3_0] mr.ObjectCache: djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:28:49,841 DEBUG [TezTR-518813_1_13_1_3_0] exec.SelectOperator: close called for operator SEL[3] 2018-01-05T11:28:49,841 INFO [TezTR-518813_1_13_1_3_0] exec.SelectOperator: Closing operator SEL[3] 2018-01-05T11:28:49,841 DEBUG [TezTR-518813_1_13_1_3_0] exec.SelectOperator: Closing child = FS[4] 2018-01-05T11:28:49,841 DEBUG [TezTR-518813_1_13_1_3_0] exec.FileSinkOperator: close called for operator FS[4] 2018-01-05T11:28:49,841 DEBUG [TezTR-518813_1_13_1_3_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:49,841 INFO [TezTR-518813_1_13_1_3_0] exec.FileSinkOperator: Closing operator FS[4] 2018-01-05T11:28:49,841 INFO [TezTR-518813_1_13_1_3_0] exec.FileSinkOperator: FS[4]: records written - 115 2018-01-05T11:28:49,841 INFO [TezTR-518813_1_13_1_2_0] exec.FileSinkOperator: FS[4]: records written - 100 2018-01-05T11:28:49,842 INFO [TezTR-518813_1_13_1_1_0] readers.UnorderedKVReader: Num Records read: 138 2018-01-05T11:28:49,842 DEBUG [TezTR-518813_1_13_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,842 DEBUG [TezTR-518813_1_13_1_1_0] log.PerfLogger: 2018-01-05T11:28:49,842 DEBUG [TezTR-518813_1_13_1_1_0] mr.ObjectCache: djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:28:49,842 DEBUG [TezTR-518813_1_13_1_1_0] exec.SelectOperator: close called for operator SEL[3] 2018-01-05T11:28:49,842 INFO [TezTR-518813_1_13_1_1_0] exec.SelectOperator: Closing operator SEL[3] 2018-01-05T11:28:49,842 DEBUG [TezTR-518813_1_13_1_1_0] exec.SelectOperator: Closing child = FS[4] 2018-01-05T11:28:49,842 DEBUG [TezTR-518813_1_13_1_3_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/-mr-10001 2018-01-05T11:28:49,842 DEBUG [TezTR-518813_1_13_1_1_0] exec.FileSinkOperator: close called for operator FS[4] 2018-01-05T11:28:49,842 DEBUG [TezTR-518813_1_13_1_1_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:49,842 INFO [TezTR-518813_1_13_1_1_0] exec.FileSinkOperator: Closing operator FS[4] 2018-01-05T11:28:49,842 INFO [TezTR-518813_1_13_1_1_0] exec.FileSinkOperator: FS[4]: records written - 138 2018-01-05T11:28:49,842 INFO [TezTR-518813_1_13_1_2_0] readers.UnorderedKVReader: Num Records read: 128 2018-01-05T11:28:49,842 DEBUG [TezTR-518813_1_13_1_2_0] log.PerfLogger: 2018-01-05T11:28:49,842 DEBUG [TezTR-518813_1_13_1_2_0] mr.ObjectCache: djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:28:49,842 DEBUG [TezTR-518813_1_13_1_2_0] exec.SelectOperator: close called for operator SEL[3] 2018-01-05T11:28:49,842 INFO [TezTR-518813_1_13_1_2_0] exec.SelectOperator: Closing operator SEL[3] 2018-01-05T11:28:49,842 DEBUG [TezTR-518813_1_13_1_2_0] exec.SelectOperator: Closing child = FS[4] 2018-01-05T11:28:49,842 DEBUG [TezTR-518813_1_13_1_2_0] exec.FileSinkOperator: close called for operator FS[4] 2018-01-05T11:28:49,842 DEBUG [TezTR-518813_1_13_1_2_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:49,842 INFO [TezTR-518813_1_13_1_2_0] exec.FileSinkOperator: Closing operator FS[4] 2018-01-05T11:28:49,842 INFO [TezTR-518813_1_13_1_2_0] exec.FileSinkOperator: FS[4]: records written - 128 2018-01-05T11:28:49,843 DEBUG [TezTR-518813_1_13_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,843 DEBUG [TezTR-518813_1_13_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,843 DEBUG [TezTR-518813_1_13_1_1_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/-mr-10001 2018-01-05T11:28:49,843 DEBUG [TezTR-518813_1_13_1_2_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/-mr-10001 2018-01-05T11:28:49,872 DEBUG [TezTR-518813_1_13_1_0_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/-mr-10001/tmpstats-0 2018-01-05T11:28:49,872 DEBUG [TezTR-518813_1_13_1_0_0] fs.FSStatsPublisher: Writing stats in it : {default.srcbucket2/={rawDataSize=1255, numRows=119}} 2018-01-05T11:28:49,873 INFO [TezTR-518813_1_13_1_0_0] exec.FileSinkOperator: RECORDS_OUT_1_default.srcbucket2:119, 2018-01-05T11:28:49,873 DEBUG [TezTR-518813_1_13_1_0_0] exec.FileSinkOperator: 4 Close done 2018-01-05T11:28:49,873 DEBUG [TezTR-518813_1_13_1_0_0] exec.SelectOperator: Closing child = SEL[1] 2018-01-05T11:28:49,873 DEBUG [TezTR-518813_1_13_1_0_0] exec.SelectOperator: close called for operator SEL[1] 2018-01-05T11:28:49,873 DEBUG [TezTR-518813_1_13_1_0_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:49,873 INFO [TezTR-518813_1_13_1_0_0] exec.SelectOperator: Closing operator SEL[1] 2018-01-05T11:28:49,873 DEBUG [TezTR-518813_1_13_1_0_0] exec.SelectOperator: Closing child = GBY[2] 2018-01-05T11:28:49,873 DEBUG [TezTR-518813_1_13_1_0_0] exec.GroupByOperator: close called for operator GBY[2] 2018-01-05T11:28:49,873 DEBUG [TezTR-518813_1_13_1_0_0] exec.GroupByOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:49,873 INFO [TezTR-518813_1_13_1_0_0] exec.GroupByOperator: Closing operator GBY[2] 2018-01-05T11:28:49,873 INFO [TezTR-518813_1_13_1_0_0] exec.GroupByOperator: Begin Hash Table flush: size = 1 2018-01-05T11:28:49,873 INFO [TezTR-518813_1_13_1_0_0] exec.ReduceSinkOperator: keys are [] num distributions: 0 2018-01-05T11:28:49,873 DEBUG [TezTR-518813_1_13_1_0_0] exec.GroupByOperator: Closing child = RS[3] 2018-01-05T11:28:49,873 DEBUG [TezTR-518813_1_13_1_0_0] exec.ReduceSinkOperator: close called for operator RS[3] 2018-01-05T11:28:49,873 DEBUG [TezTR-518813_1_13_1_0_0] exec.ReduceSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:49,873 INFO [TezTR-518813_1_13_1_0_0] exec.ReduceSinkOperator: Closing operator RS[3] 2018-01-05T11:28:49,873 INFO [TezTR-518813_1_13_1_0_0] exec.ReduceSinkOperator: RECORDS_OUT_INTERMEDIATE_Reducer_2:1, 2018-01-05T11:28:49,873 DEBUG [TezTR-518813_1_13_1_0_0] exec.ReduceSinkOperator: 3 Close done 2018-01-05T11:28:49,873 DEBUG [TezTR-518813_1_13_1_2_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/-mr-10001/tmpstats-2 2018-01-05T11:28:49,873 DEBUG [TezTR-518813_1_13_1_0_0] exec.GroupByOperator: 2 Close done 2018-01-05T11:28:49,873 DEBUG [TezTR-518813_1_13_1_0_0] exec.SelectOperator: 1 Close done 2018-01-05T11:28:49,873 DEBUG [TezTR-518813_1_13_1_0_0] exec.SelectOperator: 3 Close done 2018-01-05T11:28:49,873 DEBUG [TezTR-518813_1_13_1_2_0] fs.FSStatsPublisher: Writing stats in it : {default.srcbucket2/={rawDataSize=1354, numRows=128}} 2018-01-05T11:28:49,873 INFO [TezTR-518813_1_13_1_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_13_01_000000_0 2018-01-05T11:28:49,874 INFO [TezTR-518813_1_13_1_0_0] impl.ShuffleInputEventHandlerImpl: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:28:49,874 INFO [TezTR-518813_1_13_1_0_0] impl.ShuffleManager: Shutting down pending fetchers on sourceMap_1: 0 2018-01-05T11:28:49,874 INFO [TezTR-518813_1_13_1_2_0] exec.FileSinkOperator: RECORDS_OUT_1_default.srcbucket2:128, 2018-01-05T11:28:49,874 DEBUG [TezTR-518813_1_13_1_2_0] exec.FileSinkOperator: 4 Close done 2018-01-05T11:28:49,874 INFO [TezTR-518813_1_13_1_0_0] writers.UnorderedPartitionedKVWriter: Reducer_3: Waiting for all spills to complete : Pending : 0 2018-01-05T11:28:49,874 INFO [TezTR-518813_1_13_1_0_0] writers.UnorderedPartitionedKVWriter: Reducer_3: All spills complete 2018-01-05T11:28:49,874 DEBUG [TezTR-518813_1_13_1_2_0] exec.SelectOperator: Closing child = SEL[1] 2018-01-05T11:28:49,874 DEBUG [TezTR-518813_1_13_1_2_0] exec.SelectOperator: close called for operator SEL[1] 2018-01-05T11:28:49,874 DEBUG [TezTR-518813_1_13_1_2_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:49,874 INFO [TezTR-518813_1_13_1_2_0] exec.SelectOperator: Closing operator SEL[1] 2018-01-05T11:28:49,874 DEBUG [TezTR-518813_1_13_1_2_0] exec.SelectOperator: Closing child = GBY[2] 2018-01-05T11:28:49,874 DEBUG [TezTR-518813_1_13_1_2_0] exec.GroupByOperator: close called for operator GBY[2] 2018-01-05T11:28:49,874 DEBUG [TezTR-518813_1_13_1_2_0] exec.GroupByOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:49,874 INFO [TezTR-518813_1_13_1_2_0] exec.GroupByOperator: Closing operator GBY[2] 2018-01-05T11:28:49,874 INFO [TezTR-518813_1_13_1_2_0] exec.GroupByOperator: Begin Hash Table flush: size = 1 2018-01-05T11:28:49,874 INFO [TezTR-518813_1_13_1_2_0] exec.ReduceSinkOperator: keys are [] num distributions: 0 2018-01-05T11:28:49,874 DEBUG [TezTR-518813_1_13_1_2_0] exec.GroupByOperator: Closing child = RS[3] 2018-01-05T11:28:49,874 DEBUG [TezTR-518813_1_13_1_2_0] exec.ReduceSinkOperator: close called for operator RS[3] 2018-01-05T11:28:49,874 DEBUG [TezTR-518813_1_13_1_2_0] exec.ReduceSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:49,874 INFO [TezTR-518813_1_13_1_2_0] exec.ReduceSinkOperator: Closing operator RS[3] 2018-01-05T11:28:49,874 INFO [TezTR-518813_1_13_1_2_0] exec.ReduceSinkOperator: RECORDS_OUT_INTERMEDIATE_Reducer_2:1, 2018-01-05T11:28:49,874 DEBUG [TezTR-518813_1_13_1_2_0] exec.ReduceSinkOperator: 3 Close done 2018-01-05T11:28:49,874 DEBUG [TezTR-518813_1_13_1_2_0] exec.GroupByOperator: 2 Close done 2018-01-05T11:28:49,874 DEBUG [TezTR-518813_1_13_1_2_0] exec.SelectOperator: 1 Close done 2018-01-05T11:28:49,874 DEBUG [TezTR-518813_1_13_1_2_0] exec.SelectOperator: 3 Close done 2018-01-05T11:28:49,874 INFO [TezTR-518813_1_13_1_2_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_13_01_000002_0 2018-01-05T11:28:49,874 INFO [TezTR-518813_1_13_1_2_0] impl.ShuffleInputEventHandlerImpl: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:28:49,874 INFO [TezTR-518813_1_13_1_2_0] impl.ShuffleManager: Shutting down pending fetchers on sourceMap_1: 0 2018-01-05T11:28:49,874 INFO [TezTR-518813_1_13_1_2_0] writers.UnorderedPartitionedKVWriter: Reducer_3: Waiting for all spills to complete : Pending : 0 2018-01-05T11:28:49,874 INFO [TezTR-518813_1_13_1_2_0] writers.UnorderedPartitionedKVWriter: Reducer_3: All spills complete 2018-01-05T11:28:49,876 DEBUG [TezTR-518813_1_13_1_3_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/-mr-10001/tmpstats-3 2018-01-05T11:28:49,876 DEBUG [TezTR-518813_1_13_1_3_0] fs.FSStatsPublisher: Writing stats in it : {default.srcbucket2/={rawDataSize=1227, numRows=115}} 2018-01-05T11:28:49,876 INFO [TezTR-518813_1_13_1_3_0] exec.FileSinkOperator: RECORDS_OUT_1_default.srcbucket2:115, 2018-01-05T11:28:49,876 DEBUG [TezTR-518813_1_13_1_3_0] exec.FileSinkOperator: 4 Close done 2018-01-05T11:28:49,876 DEBUG [TezTR-518813_1_13_1_3_0] exec.SelectOperator: Closing child = SEL[1] 2018-01-05T11:28:49,876 DEBUG [TezTR-518813_1_13_1_3_0] exec.SelectOperator: close called for operator SEL[1] 2018-01-05T11:28:49,876 DEBUG [TezTR-518813_1_13_1_3_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:49,876 INFO [TezTR-518813_1_13_1_3_0] exec.SelectOperator: Closing operator SEL[1] 2018-01-05T11:28:49,876 DEBUG [TezTR-518813_1_13_1_3_0] exec.SelectOperator: Closing child = GBY[2] 2018-01-05T11:28:49,876 DEBUG [TezTR-518813_1_13_1_3_0] exec.GroupByOperator: close called for operator GBY[2] 2018-01-05T11:28:49,876 DEBUG [TezTR-518813_1_13_1_3_0] exec.GroupByOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:49,876 INFO [TezTR-518813_1_13_1_3_0] exec.GroupByOperator: Closing operator GBY[2] 2018-01-05T11:28:49,876 INFO [TezTR-518813_1_13_1_3_0] exec.GroupByOperator: Begin Hash Table flush: size = 1 2018-01-05T11:28:49,876 INFO [TezTR-518813_1_13_1_3_0] exec.ReduceSinkOperator: keys are [] num distributions: 0 2018-01-05T11:28:49,876 DEBUG [TezTR-518813_1_13_1_3_0] exec.GroupByOperator: Closing child = RS[3] 2018-01-05T11:28:49,876 DEBUG [TezTR-518813_1_13_1_3_0] exec.ReduceSinkOperator: close called for operator RS[3] 2018-01-05T11:28:49,876 DEBUG [TezTR-518813_1_13_1_3_0] exec.ReduceSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:49,876 INFO [TezTR-518813_1_13_1_3_0] exec.ReduceSinkOperator: Closing operator RS[3] 2018-01-05T11:28:49,876 INFO [TezTR-518813_1_13_1_3_0] exec.ReduceSinkOperator: RECORDS_OUT_INTERMEDIATE_Reducer_2:1, 2018-01-05T11:28:49,876 DEBUG [TezTR-518813_1_13_1_3_0] exec.ReduceSinkOperator: 3 Close done 2018-01-05T11:28:49,876 DEBUG [TezTR-518813_1_13_1_3_0] exec.GroupByOperator: 2 Close done 2018-01-05T11:28:49,876 DEBUG [TezTR-518813_1_13_1_3_0] exec.SelectOperator: 1 Close done 2018-01-05T11:28:49,876 DEBUG [TezTR-518813_1_13_1_3_0] exec.SelectOperator: 3 Close done 2018-01-05T11:28:49,876 INFO [TezTR-518813_1_13_1_3_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_13_01_000003_0 2018-01-05T11:28:49,876 INFO [TezTR-518813_1_13_1_3_0] impl.ShuffleInputEventHandlerImpl: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:28:49,876 INFO [TezTR-518813_1_13_1_3_0] impl.ShuffleManager: Shutting down pending fetchers on sourceMap_1: 0 2018-01-05T11:28:49,876 INFO [TezTR-518813_1_13_1_3_0] writers.UnorderedPartitionedKVWriter: Reducer_3: Waiting for all spills to complete : Pending : 0 2018-01-05T11:28:49,876 INFO [TezTR-518813_1_13_1_3_0] writers.UnorderedPartitionedKVWriter: Reducer_3: All spills complete 2018-01-05T11:28:49,877 DEBUG [TezTR-518813_1_13_1_1_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/-mr-10001/tmpstats-1 2018-01-05T11:28:49,877 DEBUG [TezTR-518813_1_13_1_1_0] fs.FSStatsPublisher: Writing stats in it : {default.srcbucket2/={rawDataSize=1476, numRows=138}} 2018-01-05T11:28:49,878 INFO [TezTR-518813_1_13_1_1_0] exec.FileSinkOperator: RECORDS_OUT_1_default.srcbucket2:138, 2018-01-05T11:28:49,878 DEBUG [TezTR-518813_1_13_1_1_0] exec.FileSinkOperator: 4 Close done 2018-01-05T11:28:49,878 DEBUG [TezTR-518813_1_13_1_1_0] exec.SelectOperator: Closing child = SEL[1] 2018-01-05T11:28:49,878 DEBUG [TezTR-518813_1_13_1_1_0] exec.SelectOperator: close called for operator SEL[1] 2018-01-05T11:28:49,878 DEBUG [TezTR-518813_1_13_1_1_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:49,878 INFO [TezTR-518813_1_13_1_1_0] exec.SelectOperator: Closing operator SEL[1] 2018-01-05T11:28:49,878 DEBUG [TezTR-518813_1_13_1_1_0] exec.SelectOperator: Closing child = GBY[2] 2018-01-05T11:28:49,878 DEBUG [TezTR-518813_1_13_1_1_0] exec.GroupByOperator: close called for operator GBY[2] 2018-01-05T11:28:49,878 DEBUG [TezTR-518813_1_13_1_1_0] exec.GroupByOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:49,878 INFO [TezTR-518813_1_13_1_1_0] exec.GroupByOperator: Closing operator GBY[2] 2018-01-05T11:28:49,878 INFO [TezTR-518813_1_13_1_1_0] exec.GroupByOperator: Begin Hash Table flush: size = 1 2018-01-05T11:28:49,878 INFO [TezTR-518813_1_13_1_1_0] exec.ReduceSinkOperator: keys are [] num distributions: 0 2018-01-05T11:28:49,878 DEBUG [TezTR-518813_1_13_1_1_0] exec.GroupByOperator: Closing child = RS[3] 2018-01-05T11:28:49,878 DEBUG [TezTR-518813_1_13_1_1_0] exec.ReduceSinkOperator: close called for operator RS[3] 2018-01-05T11:28:49,878 DEBUG [TezTR-518813_1_13_1_1_0] exec.ReduceSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:49,878 INFO [TezTR-518813_1_13_1_1_0] exec.ReduceSinkOperator: Closing operator RS[3] 2018-01-05T11:28:49,878 INFO [TezTR-518813_1_13_1_1_0] exec.ReduceSinkOperator: RECORDS_OUT_INTERMEDIATE_Reducer_2:1, 2018-01-05T11:28:49,878 DEBUG [TezTR-518813_1_13_1_1_0] exec.ReduceSinkOperator: 3 Close done 2018-01-05T11:28:49,878 DEBUG [TezTR-518813_1_13_1_1_0] exec.GroupByOperator: 2 Close done 2018-01-05T11:28:49,878 DEBUG [TezTR-518813_1_13_1_1_0] exec.SelectOperator: 1 Close done 2018-01-05T11:28:49,878 DEBUG [TezTR-518813_1_13_1_1_0] exec.SelectOperator: 3 Close done 2018-01-05T11:28:49,878 INFO [TezTR-518813_1_13_1_1_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_13_01_000001_0 2018-01-05T11:28:49,878 INFO [TezTR-518813_1_13_1_1_0] impl.ShuffleInputEventHandlerImpl: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:28:49,878 INFO [TezTR-518813_1_13_1_1_0] impl.ShuffleManager: Shutting down pending fetchers on sourceMap_1: 0 2018-01-05T11:28:49,878 INFO [TezTR-518813_1_13_1_1_0] writers.UnorderedPartitionedKVWriter: Reducer_3: Waiting for all spills to complete : Pending : 0 2018-01-05T11:28:49,878 INFO [TezTR-518813_1_13_1_1_0] writers.UnorderedPartitionedKVWriter: Reducer_3: All spills complete 2018-01-05T11:28:49,888 DEBUG [TezTR-518813_1_13_1_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:49,888 DEBUG [TezTR-518813_1_13_1_2_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:49,888 INFO [TezTR-518813_1_13_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:49,888 INFO [TezTR-518813_1_13_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:49,888 INFO [TezTR-518813_1_13_1_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_13_01_000000_0, askedToStop=false 2018-01-05T11:28:49,888 INFO [TezTR-518813_1_13_1_2_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_13_01_000002_0, askedToStop=false 2018-01-05T11:28:49,888 INFO [TezTR-518813_1_13_1_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_13_01_000000_0, stopRequested=false 2018-01-05T11:28:49,888 INFO [TezTR-518813_1_13_1_2_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_13_01_000002_0, stopRequested=false 2018-01-05T11:28:49,888 INFO [TezTR-518813_1_13_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_13_01_000002_0: Counters: 23 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=128, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=722, OUTPUT_BYTES_WITH_OVERHEAD=732, OUTPUT_BYTES_PHYSICAL=760, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=1810, SHUFFLE_BYTES_DECOMPRESSED=1806, SHUFFLE_BYTES_TO_MEM=1810, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=4, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1][HIVE CREATED_FILES=1, RECORDS_OUT_1_default.srcbucket2=128, RECORDS_OUT_INTERMEDIATE_Reducer_2=1]] 2018-01-05T11:28:49,888 INFO [TezTR-518813_1_13_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_13_01_000000_0: Counters: 23 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=119, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=674, OUTPUT_BYTES_WITH_OVERHEAD=684, OUTPUT_BYTES_PHYSICAL=712, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=1679, SHUFFLE_BYTES_DECOMPRESSED=1675, SHUFFLE_BYTES_TO_MEM=1679, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=4, FIRST_EVENT_RECEIVED=0, LAST_EVENT_RECEIVED=0][HIVE CREATED_FILES=1, RECORDS_OUT_1_default.srcbucket2=119, RECORDS_OUT_INTERMEDIATE_Reducer_2=1]] 2018-01-05T11:28:49,889 DEBUG [TezTR-518813_1_13_1_2_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_13_1_2_0 thread id: 899 scheme: file bytesRead: 0 bytesWritten: 2359 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:49,889 DEBUG [TezTR-518813_1_13_1_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_13_1_0_0 thread id: 887 scheme: file bytesRead: 0 bytesWritten: 2203 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:49,889 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_13_01_000002_0, isTaskDone=true 2018-01-05T11:28:49,889 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_13_01_000000_0, isTaskDone=true 2018-01-05T11:28:49,890 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000024, requestId=3, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_13_01_000002_0, eventCount=4 } 2018-01-05T11:28:49,890 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000022, requestId=3, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_13_01_000000_0, eventCount=4 } 2018-01-05T11:28:49,891 DEBUG [TezTR-518813_1_13_1_3_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:49,892 DEBUG [TezTR-518813_1_13_1_1_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:49,892 INFO [TezTR-518813_1_13_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:49,892 INFO [TezTR-518813_1_13_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:49,892 INFO [TezTR-518813_1_13_1_1_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_13_01_000001_0, askedToStop=false 2018-01-05T11:28:49,892 INFO [TezTR-518813_1_13_1_1_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_13_01_000001_0, stopRequested=false 2018-01-05T11:28:49,892 INFO [TezTR-518813_1_13_1_3_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_13_01_000003_0, askedToStop=false 2018-01-05T11:28:49,892 INFO [TezTR-518813_1_13_1_3_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_13_01_000003_0, stopRequested=false 2018-01-05T11:28:49,892 INFO [TezTR-518813_1_13_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_13_01_000003_0: Counters: 23 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=115, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=658, OUTPUT_BYTES_WITH_OVERHEAD=668, OUTPUT_BYTES_PHYSICAL=696, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=1630, SHUFFLE_BYTES_DECOMPRESSED=1626, SHUFFLE_BYTES_TO_MEM=1630, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2][HIVE CREATED_FILES=1, RECORDS_OUT_1_default.srcbucket2=115, RECORDS_OUT_INTERMEDIATE_Reducer_2=1]] 2018-01-05T11:28:49,892 INFO [TezTR-518813_1_13_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_13_01_000001_0: Counters: 23 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=138, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=709, OUTPUT_BYTES_WITH_OVERHEAD=719, OUTPUT_BYTES_PHYSICAL=747, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=1972, SHUFFLE_BYTES_DECOMPRESSED=1968, SHUFFLE_BYTES_TO_MEM=1972, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=4, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1][HIVE CREATED_FILES=1, RECORDS_OUT_1_default.srcbucket2=138, RECORDS_OUT_INTERMEDIATE_Reducer_2=1]] 2018-01-05T11:28:49,892 DEBUG [TezTR-518813_1_13_1_3_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_13_1_3_0 thread id: 906 scheme: file bytesRead: 0 bytesWritten: 2155 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:49,892 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_13_01_000002_0, creationTime=1515180529441, allocationTime=1515180529444, startTime=1515180529446, finishTime=1515180529892, timeTaken=446, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:49,892 DEBUG [TezTR-518813_1_13_1_1_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_13_1_1_0 thread id: 893 scheme: file bytesRead: 0 bytesWritten: 2482 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:49,893 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_13_01_000003_0, isTaskDone=true 2018-01-05T11:28:49,893 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=3, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:28:49,893 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000025, requestId=3, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_13_01_000003_0, eventCount=4 } 2018-01-05T11:28:49,893 INFO [Task-Executor-1] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_13_01_000002_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:49,893 INFO [Task-Executor-1] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_13_01_000002_0 2018-01-05T11:28:49,893 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_13_01_000002_0 2018-01-05T11:28:49,893 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_13_01_000001_0, isTaskDone=true 2018-01-05T11:28:49,894 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000023, requestId=3, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_13_01_000001_0, eventCount=4 } 2018-01-05T11:28:49,894 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=3, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:28:49,893 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_13_01_000002_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:49,894 INFO [Task-Executor-2] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_13_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:49,894 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_13_01_000002_0; false 2018-01-05T11:28:49,894 INFO [Task-Executor-2] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_13_01_000000_0 2018-01-05T11:28:49,894 INFO [Task-Executor-1] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000024=444 2018-01-05T11:28:49,894 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_13_01_000002, startTime=1515180529446, finishTime=1515180529893, timeTaken=447, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_13_01_000002_0, diagnostics=, counters=Counters: 28, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=2359, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=128, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=722, OUTPUT_BYTES_WITH_OVERHEAD=732, OUTPUT_BYTES_PHYSICAL=760, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=1810, SHUFFLE_BYTES_DECOMPRESSED=1806, SHUFFLE_BYTES_TO_MEM=1810, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=4, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1, HIVE, CREATED_FILES=1, RECORDS_OUT_1_default.srcbucket2=128, RECORDS_OUT_INTERMEDIATE_Reducer_2=1 2018-01-05T11:28:49,894 INFO [Task-Executor-2] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000022=448 2018-01-05T11:28:49,894 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_13_01_000002_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:49,894 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_13_01_000000_0: true 2018-01-05T11:28:49,894 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_13_01_000002_0: true 2018-01-05T11:28:49,894 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_13_01_000000_0 2018-01-05T11:28:49,895 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_13_01_000000_0, creationTime=1515180529441, allocationTime=1515180529442, startTime=1515180529445, finishTime=1515180529894, timeTaken=449, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:49,895 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000022, QueryId=djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39-13, Vertex=Reducer 2, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:28:49,894 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:49,895 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_13_01 [Reducer 2], tasks=4, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:49,895 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:49,895 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_13_01_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=1, preemptionQueueSize=3 2018-01-05T11:28:49,895 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_13_01_000000_0 2018-01-05T11:28:49,895 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000022, QueryId=djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39-13, DagId=13, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-2, Succeeded=true, StartTime=1515180529445, EndTime=1515180529895 2018-01-05T11:28:49,895 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000024, stoppedTime=1515180529895, exitStatus=0 2018-01-05T11:28:49,895 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_13_01_000000_0 2018-01-05T11:28:49,895 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_13_01_000002_0 2018-01-05T11:28:49,895 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000024, QueryId=djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39-13, Vertex=Reducer 2, FragmentNum=2, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:28:49,896 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_13_01_000002_0 complete. WaitQueueSize=0, numSlotsAvailable=2, preemptionQueueSize=2 2018-01-05T11:28:49,896 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_13_01_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:49,896 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_13_01_000000_0; false 2018-01-05T11:28:49,896 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_13_01_000000, startTime=1515180529445, finishTime=1515180529895, timeTaken=450, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_13_01_000000_0, diagnostics=, counters=Counters: 28, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=2203, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=119, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=674, OUTPUT_BYTES_WITH_OVERHEAD=684, OUTPUT_BYTES_PHYSICAL=712, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=1679, SHUFFLE_BYTES_DECOMPRESSED=1675, SHUFFLE_BYTES_TO_MEM=1679, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=4, FIRST_EVENT_RECEIVED=0, LAST_EVENT_RECEIVED=0, HIVE, CREATED_FILES=1, RECORDS_OUT_1_default.srcbucket2=119, RECORDS_OUT_INTERMEDIATE_Reducer_2=1 2018-01-05T11:28:49,896 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_13_01_000002_0 2018-01-05T11:28:49,896 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000024 exited with diagnostics set to Container completed. 2018-01-05T11:28:49,896 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000024, QueryId=djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39-13, DagId=13, VertexName=Reducer 2, TaskId=2, TaskAttemptId=0, ThreadName=Task-Executor-1, Succeeded=true, StartTime=1515180529449, EndTime=1515180529896 2018-01-05T11:28:49,896 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_13_02 [Reducer 3] attempt: attempt_1515180518813_0001_13_01_000002_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:28:49,896 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=3, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:28:49,896 INFO [Task-Executor-3] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_13_01_000001_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:49,896 INFO [Task-Executor-3] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_13_01_000001_0 2018-01-05T11:28:49,896 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_13_01_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:49,896 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=3, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:28:49,896 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_13_01_000003_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:49,896 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_13_01_000003_0 2018-01-05T11:28:49,896 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_13_01_000003_0, creationTime=1515180529441, allocationTime=1515180529444, startTime=1515180529447, finishTime=1515180529896, timeTaken=449, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:49,896 INFO [Task-Executor-3] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000023=449 2018-01-05T11:28:49,897 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_13_01_000001_0: true 2018-01-05T11:28:49,897 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: Scheduling 1 tasks for vertex: Reducer 3 with totalTasks: 1. 1 source tasks completed out of 4. MinSourceTaskCompletedFraction: 0.25 min: 0.05 max: 0.75 2018-01-05T11:28:49,897 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:49,897 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:49,897 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_13_01_000001_0 2018-01-05T11:28:49,897 INFO [App Shared Pool - #2] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Reducer 2, destinationVertexName=Reducer 3. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.UnorderedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:28:49,897 INFO [App Shared Pool - #2] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_13_02 [Reducer 3] 2018-01-05T11:28:49,897 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000025=443 2018-01-05T11:28:49,897 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000023, QueryId=djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39-13, Vertex=Reducer 2, FragmentNum=1, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:28:49,897 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_13_01_000003_0: true 2018-01-05T11:28:49,897 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_13_01_000001_0, creationTime=1515180529441, allocationTime=1515180529443, startTime=1515180529445, finishTime=1515180529897, timeTaken=452, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:49,897 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_13_01_000001_0 complete. WaitQueueSize=0, numSlotsAvailable=3, preemptionQueueSize=1 2018-01-05T11:28:49,897 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_13_01_000001_0 2018-01-05T11:28:49,898 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000024 2018-01-05T11:28:49,898 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000023, QueryId=djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39-13, DagId=13, VertexName=Reducer 2, TaskId=1, TaskAttemptId=0, ThreadName=Task-Executor-3, Succeeded=true, StartTime=1515180529446, EndTime=1515180529898 2018-01-05T11:28:49,898 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_13_01 [Reducer 2], tasks=4, failed=0, killed=0, success=2, completed=2, commits=0, err=null 2018-01-05T11:28:49,898 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_13_01_000003_0 2018-01-05T11:28:49,898 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000022, stoppedTime=1515180529898, exitStatus=0 2018-01-05T11:28:49,898 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000025, QueryId=djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39-13, Vertex=Reducer 2, FragmentNum=3, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:28:49,898 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_13_01_000003_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:49,898 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_13_01_000003_0 2018-01-05T11:28:49,898 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_13_01_000003_0 2018-01-05T11:28:49,898 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_13_01_000003_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:49,898 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_13_01_000003_0; false 2018-01-05T11:28:49,898 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_13_01_000003_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:49,898 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_13_01_000003, startTime=1515180529447, finishTime=1515180529898, timeTaken=451, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_13_01_000003_0, diagnostics=, counters=Counters: 28, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=2155, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=115, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=658, OUTPUT_BYTES_WITH_OVERHEAD=668, OUTPUT_BYTES_PHYSICAL=696, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=1630, SHUFFLE_BYTES_DECOMPRESSED=1626, SHUFFLE_BYTES_TO_MEM=1630, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, HIVE, CREATED_FILES=1, RECORDS_OUT_1_default.srcbucket2=115, RECORDS_OUT_INTERMEDIATE_Reducer_2=1 2018-01-05T11:28:49,898 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:49,898 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:49,898 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000022 exited with diagnostics set to Container completed. 2018-01-05T11:28:49,898 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:TASK_STARTED]: vertexName=Reducer 3, taskId=task_1515180518813_0001_13_02_000000, scheduledTime=1515180529898, launchTime=1515180529898 2018-01-05T11:28:49,899 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_13_01_000001_0 2018-01-05T11:28:49,899 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_13_01_000001_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:49,899 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_13_01_000001_0; false 2018-01-05T11:28:49,899 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_13_01_000001_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:49,899 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:49,899 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:49,899 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_13_01_000001, startTime=1515180529445, finishTime=1515180529899, timeTaken=454, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_13_01_000001_0, diagnostics=, counters=Counters: 28, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=2482, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=138, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=709, OUTPUT_BYTES_WITH_OVERHEAD=719, OUTPUT_BYTES_PHYSICAL=747, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=1972, SHUFFLE_BYTES_DECOMPRESSED=1968, SHUFFLE_BYTES_TO_MEM=1972, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=4, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1, HIVE, CREATED_FILES=1, RECORDS_OUT_1_default.srcbucket2=138, RECORDS_OUT_INTERMEDIATE_Reducer_2=1 2018-01-05T11:28:49,899 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_13_02 [Reducer 3] attempt: attempt_1515180518813_0001_13_01_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:28:49,899 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000022 2018-01-05T11:28:49,899 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_13_01 [Reducer 2], tasks=4, failed=0, killed=0, success=3, completed=3, commits=0, err=null 2018-01-05T11:28:49,899 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000025, stoppedTime=1515180529899, exitStatus=0 2018-01-05T11:28:49,899 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000025 exited with diagnostics set to Container completed. 2018-01-05T11:28:49,900 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_13_02 [Reducer 3] 2018-01-05T11:28:49,900 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_13_01 [Reducer 2], tasks=4, failed=0, killed=0, success=4, completed=4, commits=0, err=null 2018-01-05T11:28:49,900 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_13_01 [Reducer 2] 2018-01-05T11:28:49,900 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:VERTEX_FINISHED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_13_01, initRequestedTime=1515180529298, initedTime=1515180529303, startRequestedTime=1515180529359, startedTime=1515180529359, finishTime=1515180529900, timeTaken=541, status=SUCCEEDED, diagnostics=, counters=Counters: 28, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=9199, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=4, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=500, OUTPUT_RECORDS=4, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=2763, OUTPUT_BYTES_WITH_OVERHEAD=2803, OUTPUT_BYTES_PHYSICAL=2915, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=7091, SHUFFLE_BYTES_DECOMPRESSED=7075, SHUFFLE_BYTES_TO_MEM=7091, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=22, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, HIVE, CREATED_FILES=4, RECORDS_OUT_1_default.srcbucket2=500, RECORDS_OUT_INTERMEDIATE_Reducer_2=4, vertexStats=firstTaskStartTime=1515180529445, firstTasksToStart=[ task_1515180518813_0001_13_01_000000,task_1515180518813_0001_13_01_000001 ], lastTaskFinishTime=1515180529897, lastTasksToFinish=[ task_1515180518813_0001_13_01_000001 ], minTaskDuration=446, maxTaskDuration=452, avgTaskDuration=449.0, numSuccessfulTasks=4, shortestDurationTasks=[ task_1515180518813_0001_13_01_000002 ], longestDurationTasks=[ task_1515180518813_0001_13_01_000001 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=4, numSucceededTasks=4, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:49,900 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_13_01 [Reducer 2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:49,900 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000023, stoppedTime=1515180529900, exitStatus=0 2018-01-05T11:28:49,900 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000023 exited with diagnostics set to Container completed. 2018-01-05T11:28:49,900 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_13_02 [Reducer 3] attempt: attempt_1515180518813_0001_13_01_000003_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:28:49,900 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000025 2018-01-05T11:28:49,900 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_13_02 [Reducer 3] attempt: attempt_1515180518813_0001_13_01_000001_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:28:49,900 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_13_01 [Reducer 2] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3 2018-01-05T11:28:49,900 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null 2018-01-05T11:28:49,900 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: All source tasks completed. Ramping up 0 remaining tasks for vertex: Reducer 3 2018-01-05T11:28:49,900 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000023 2018-01-05T11:28:49,900 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_13_02_000000_0, priority=32, capability=, hosts=null 2018-01-05T11:28:49,900 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=32,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:49,900 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=32,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:49,900 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:49,900 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_13_02_000000_0 on hosts=null 2018-01-05T11:28:49,900 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_13_02_000000_0 on hosts=null 2018-01-05T11:28:49,900 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_13_02_000000_0 2018-01-05T11:28:49,900 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:49,900 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #26, task=TaskInfo{task=attempt_1515180518813_0001_13_02_000000_0, priority=32, startTime=0, containerId=null, uniqueId=25, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000026 2018-01-05T11:28:49,900 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_13_02_000000_0; false 2018-01-05T11:28:49,900 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_13_02_000000_0, priority=32, startTime=311550409, containerId=container_111101111_0001_00_000026assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=25, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:49,901 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000026 for node: 10.22.8.182:49818 2018-01-05T11:28:49,901 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000026, attempt_1515180518813_0001_13_02_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:49,901 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000026 for node: 10.22.8.182:49818 2018-01-05T11:28:49,901 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000026 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:49,901 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000026, launchTime=1515180529901 2018-01-05T11:28:49,901 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=13}, sourceName=Reducer 2, state=S_SUCCEEDED 2018-01-05T11:28:49,902 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_13_02_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000026] on NM: [10.22.8.182:49818] 2018-01-05T11:28:49,902 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:49,902 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:49,902 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 3, taskAttemptId=attempt_1515180518813_0001_13_02_000000_0, startTime=1515180529901, containerId=container_111101111_0001_00_000026, nodeId=10.22.8.182:49818 2018-01-05T11:28:49,902 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_13_02_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000026, dagName=INSERT INTO srcbucket2 SELEC...srcbucket_tmp (Stage-1), vertexName=Reducer 3, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Reducer 2,org.apache.tez.runtime.library.input.UnorderedKVInput,4}}, Outputs={{out_Reducer 3,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=6, completedTaskCount=5, dagStartTime=1515180529290, firstAttemptStartTime=1515180529900, currentAttemptStartTime=1515180529901} 2018-01-05T11:28:49,902 INFO [IPC Server handler 4 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000026, QueryId=djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39-13, DagId=13, VertexName=Reducer 3, TaskId=0, TaskAttemptId=0, SubmitTime=1515180529902 2018-01-05T11:28:49,902 DEBUG [IPC Server handler 4 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=13} with the ShuffleHandler 2018-01-05T11:28:49,902 DEBUG [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/13] 2018-01-05T11:28:49,903 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:49,903 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_13_02_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:49,903 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_13_02_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180529900, dagStartTime=1515180529290, withinDagPriority=32, vertexParallelism= 1, selfAndUpstreamParallelism= 6, selfAndUpstreamComplete= 5}] 2018-01-05T11:28:49,903 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_13_02_000000_0 : ACCEPTED 2018-01-05T11:28:49,903 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_13_02_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:49,903 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_13_02_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180529900, dagStartTime=1515180529290, withinDagPriority=32, vertexParallelism= 1, selfAndUpstreamParallelism= 6, selfAndUpstreamComplete= 5} 2018-01-05T11:28:49,903 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_13_02_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:49,903 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_13_02_000000_0 to pre-emption queue 2018-01-05T11:28:49,903 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_13_02_000000_0: true 2018-01-05T11:28:49,903 INFO [TaskCommunicator # 6] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_13_02_000000_0 2018-01-05T11:28:49,903 INFO [TaskCommunicator # 6] GuaranteedTasks: Received response for attempt_1515180518813_0001_13_02_000000_0, true 2018-01-05T11:28:49,903 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:629620962 2018-01-05T11:28:49,904 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : INSERT INTO srcbucket2 SELEC...srcbucket_tmp (Stage-1), VertexName: Reducer 3, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_13_02_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Reducer 2, physicalEdgeCount=4, inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:28:49,904 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1039138816, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:49,904 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_13_02_000000_0 vertexName: Reducer 3 2018-01-05T11:28:49,904 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000026, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_13_02_000000_0, eventCount=1 } 2018-01-05T11:28:49,904 DEBUG [TezTR-518813_1_13_2_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_13 queryId: djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39 instanceType: TaskRunner2Callable 2018-01-05T11:28:49,904 INFO [TezTR-518813_1_13_2_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_13_02_000000_0 2018-01-05T11:28:49,904 DEBUG [TezTR-518813_1_13_2_0_0] log.PerfLogger: 2018-01-05T11:28:49,905 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:49,906 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:49,908 INFO [IPC Server handler 28 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_13_02_000000_0 4 events [0,4) total 4 vertex_1515180518813_0001_13_02 [Reducer 3] 2018-01-05T11:28:49,908 DEBUG [TezTR-518813_1_13_2_0_0] log.PerfLogger: 2018-01-05T11:28:49,908 INFO [TezTR-518813_1_13_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:49,908 INFO [TezTR-518813_1_13_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:49,908 INFO [TezTR-518813_1_13_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:49,908 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=4, nextPreRoutedEventId=0, eventCount=4 } 2018-01-05T11:28:49,908 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_13_02_000000_0, eventCount=4 fromEventId=0 nextFromEventId=4 2018-01-05T11:28:49,908 INFO [I/O Setup 1 Initialize: {out_Reducer 3}] output.MROutput: out_Reducer 3: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:28:49,909 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:49,909 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:49,909 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:49,909 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:49,909 INFO [TezTR-518813_1_13_2_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:49,909 INFO [TezTR-518813_1_13_2_0_0] resources.MemoryDistributor: InitialRequests=[out_Reducer 3:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [Reducer 2:INPUT:131404920:org.apache.tez.runtime.library.input.UnorderedKVInput] 2018-01-05T11:28:49,909 INFO [TezTR-518813_1_13_2_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:49,909 INFO [TezTR-518813_1_13_2_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:49,909 INFO [TezTR-518813_1_13_2_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 131404920, TotalRequestedScaled: 6.570246E7, TotalJVMHeap: 1039138816, TotalAvailable: 696223006, TotalRequested/TotalJVMHeap:0.13 2018-01-05T11:28:49,909 INFO [TezTR-518813_1_13_2_0_0] resources.MemoryDistributor: Allocations=[out_Reducer 3:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.input.UnorderedKVInput:INPUT:131404920:131404920] 2018-01-05T11:28:49,909 INFO [TezTR-518813_1_13_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:49,909 INFO [TezTR-518813_1_13_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 0 2018-01-05T11:28:49,909 INFO [TezTR-518813_1_13_2_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:49,909 INFO [TezTR-518813_1_13_2_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_13_02_000000_0 2018-01-05T11:28:49,909 DEBUG [TezTR-518813_1_13_2_0_0] log.PerfLogger: 2018-01-05T11:28:49,909 DEBUG [TezTR-518813_1_13_2_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_13_02_000000_0_10076 2018-01-05T11:28:49,909 DEBUG [TezTR-518813_1_13_2_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:49,909 DEBUG [TezTR-518813_1_13_2_0_0] mr.ObjectCache: Creating djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39_Reducer 3__REDUCE_PLAN__ 2018-01-05T11:28:49,909 INFO [TezTR-518813_1_13_2_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/djaiswal/_tez_scratch_dir/d04abd3a-bec5-4086-b3e0-84f0b907fdfa/reduce.xml 2018-01-05T11:28:49,910 DEBUG [TezTR-518813_1_13_2_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/djaiswal/_tez_scratch_dir/d04abd3a-bec5-4086-b3e0-84f0b907fdfa/reduce.xml 2018-01-05T11:28:49,910 DEBUG [TezTR-518813_1_13_2_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/djaiswal/_tez_scratch_dir/d04abd3a-bec5-4086-b3e0-84f0b907fdfa/reduce.xml 2018-01-05T11:28:49,910 DEBUG [TezTR-518813_1_13_2_0_0] log.PerfLogger: 2018-01-05T11:28:49,910 INFO [TezTR-518813_1_13_2_0_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:28:49,910 DEBUG [TezTR-518813_1_13_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,911 DEBUG [TezTR-518813_1_13_2_0_0] log.PerfLogger: 2018-01-05T11:28:49,911 INFO [TezTR-518813_1_13_2_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 3 size: 2.18KB 2018-01-05T11:28:49,911 DEBUG [TezTR-518813_1_13_2_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:49,911 DEBUG [TezTR-518813_1_13_2_0_0] log.PerfLogger: 2018-01-05T11:28:49,911 DEBUG [TezTR-518813_1_13_2_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:49,911 DEBUG [TezTR-518813_1_13_2_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:49,912 DEBUG [TezTR-518813_1_13_2_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:49,912 INFO [TezTR-518813_1_13_2_0_0] impl.SimpleFetchedInputAllocator: Reducer_2: RequestedMemory=131404920, AssignedMemory=131404920, maxSingleShuffleLimit=32851230 2018-01-05T11:28:49,912 INFO [TezTR-518813_1_13_2_0_0] impl.ShuffleManager: Reducer_2: numInputs=4, compressionCodec=NoCompressionCodec, numFetchers=4, ifileBufferSize=4096, ifileReadAheadEnabled=true, ifileReadAheadLength=4194304, localDiskFetchEnabled=false, sharedFetchEnabled=false, keepAlive=false, keepAliveMaxConnections=20, connectionTimeout=20000, readTimeout=180000, bufferSize=8192, bufferSize=8192, maxTaskOutputAtOnce=20 2018-01-05T11:28:49,912 INFO [TezTR-518813_1_13_2_0_0] impl.ShuffleInputEventHandlerImpl: Reducer 2: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:28:49,913 INFO [TezTR-518813_1_13_2_0_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:28:49,914 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=13&reduce=0&map=attempt_1515180518813_0001_13_01_000003_0_10075,attempt_1515180518813_0001_13_01_000001_0_10069 mapId: [attempt_1515180518813_0001_13_01_000003_0_10075, attempt_1515180518813_0001_13_01_000001_0_10069] reduceId: [0] jobId: [job_1515180518813_0001] dagId: [13] keepAlive: false 2018-01-05T11:28:49,914 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=13&reduce=0&map=attempt_1515180518813_0001_13_01_000002_0_10072,attempt_1515180518813_0001_13_01_000000_0_10066 mapId: [attempt_1515180518813_0001_13_01_000002_0_10072, attempt_1515180518813_0001_13_01_000000_0_10066] reduceId: [0] jobId: [job_1515180518813_0001] dagId: [13] keepAlive: false 2018-01-05T11:28:49,914 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=13&reduce=0&map=attempt_1515180518813_0001_13_01_000003_0_10075,attempt_1515180518813_0001_13_01_000001_0_10069; hash=...Jg/fHJFObrPfg 2018-01-05T11:28:49,914 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=13&reduce=0&map=attempt_1515180518813_0001_13_01_000002_0_10072,attempt_1515180518813_0001_13_01_000000_0_10066; hash=...OOQLibMnDCpi0 2018-01-05T11:28:49,914 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=13&reduce=0&map=attempt_1515180518813_0001_13_01_000003_0_10075,attempt_1515180518813_0001_13_01_000001_0_10069;reply=+og22a1eJM3Ow 2018-01-05T11:28:49,914 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=13&reduce=0&map=attempt_1515180518813_0001_13_01_000002_0_10072,attempt_1515180518813_0001_13_01_000000_0_10066;reply=+4kt9kqwKwEPU 2018-01-05T11:28:49,914 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=13, user='djaiswal', attemptId='attempt_1515180518813_0001_13_01_000002_0_10072'} via loader 2018-01-05T11:28:49,914 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=13, user='djaiswal', attemptId='attempt_1515180518813_0001_13_01_000003_0_10075'} via loader 2018-01-05T11:28:49,914 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=13, user='djaiswal', attemptId='attempt_1515180518813_0001_13_01_000002_0_10072'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:28:49,914 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=13, user='djaiswal', attemptId='attempt_1515180518813_0001_13_01_000003_0_10075'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:28:49,914 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_13_01_000003_0_10075 not found 2018-01-05T11:28:49,914 DEBUG [New I/O worker #2] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_13_01_000002_0_10072 not found 2018-01-05T11:28:49,915 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_13_01_000003_0_10075,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/13/output/attempt_1515180518813_0001_13_01_000003_0_10075/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/13/output/attempt_1515180518813_0001_13_01_000003_0_10075/file.out.index 2018-01-05T11:28:49,915 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_13_01_000002_0_10072,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/13/output/attempt_1515180518813_0001_13_01_000002_0_10072/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/13/output/attempt_1515180518813_0001_13_01_000002_0_10072/file.out.index 2018-01-05T11:28:49,915 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=13, user='djaiswal', attemptId='attempt_1515180518813_0001_13_01_000001_0_10069'} via loader 2018-01-05T11:28:49,915 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=13, user='djaiswal', attemptId='attempt_1515180518813_0001_13_01_000001_0_10069'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:28:49,915 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_13_01_000001_0_10069 not found 2018-01-05T11:28:49,915 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=13, user='djaiswal', attemptId='attempt_1515180518813_0001_13_01_000000_0_10066'} via loader 2018-01-05T11:28:49,915 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=13, user='djaiswal', attemptId='attempt_1515180518813_0001_13_01_000000_0_10066'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:28:49,915 DEBUG [New I/O worker #2] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_13_01_000000_0_10066 not found 2018-01-05T11:28:49,915 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_13_01_000001_0_10069,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/13/output/attempt_1515180518813_0001_13_01_000001_0_10069/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/13/output/attempt_1515180518813_0001_13_01_000001_0_10069/file.out.index 2018-01-05T11:28:49,915 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Content Length in shuffle : 1505 2018-01-05T11:28:49,915 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_13_01_000000_0_10066,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/13/output/attempt_1515180518813_0001_13_01_000000_0_10066/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/13/output/attempt_1515180518813_0001_13_01_000000_0_10066/file.out.index 2018-01-05T11:28:49,915 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: Content Length in shuffle : 1534 2018-01-05T11:28:49,915 INFO [Fetcher_B {Reducer_2} #1] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=13&reduce=0&map=attempt_1515180518813_0001_13_01_000003_0_10075,attempt_1515180518813_0001_13_01_000001_0_10069 sent hash and receievd reply 0 ms 2018-01-05T11:28:49,916 INFO [Fetcher_B {Reducer_2} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=13&reduce=0&map=attempt_1515180518813_0001_13_01_000002_0_10072,attempt_1515180518813_0001_13_01_000000_0_10066 sent hash and receievd reply 0 ms 2018-01-05T11:28:49,916 INFO [Fetcher_B {Reducer_2} #1] ShuffleManager.fetch: Completed fetch for attempt: {3, 0, attempt_1515180518813_0001_13_01_000003_0_10075} to MEMORY, csize=672, dsize=668, EndTime=1515180529916, TimeTaken=1, Rate=0.64 MB/s 2018-01-05T11:28:49,916 INFO [TezTR-518813_1_13_2_0_0] tez.ReduceRecordProcessor: Main work is Reducer 3 2018-01-05T11:28:49,916 INFO [Fetcher_B {Reducer_2} #1] impl.ShuffleManager: copy(1 (spillsFetched=1) of 4. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:28:49,916 INFO [TezTR-518813_1_13_2_0_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:49,916 INFO [Fetcher_B {Reducer_2} #0] ShuffleManager.fetch: Completed fetch for attempt: {2, 0, attempt_1515180518813_0001_13_01_000002_0_10072} to MEMORY, csize=736, dsize=732, EndTime=1515180529916, TimeTaken=0, Rate=0.00 MB/s 2018-01-05T11:28:49,916 DEBUG [TezTR-518813_1_13_2_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:28:49,916 DEBUG [TezTR-518813_1_13_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,916 INFO [Fetcher_B {Reducer_2} #1] ShuffleManager.fetch: Completed fetch for attempt: {1, 0, attempt_1515180518813_0001_13_01_000001_0_10069} to MEMORY, csize=723, dsize=719, EndTime=1515180529916, TimeTaken=0, Rate=0.00 MB/s 2018-01-05T11:28:49,916 DEBUG [TezTR-518813_1_13_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,916 INFO [Fetcher_B {Reducer_2} #0] ShuffleManager.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_13_01_000000_0_10066} to MEMORY, csize=688, dsize=684, EndTime=1515180529916, TimeTaken=0, Rate=0.00 MB/s 2018-01-05T11:28:49,916 INFO [Fetcher_B {Reducer_2} #0] impl.ShuffleManager: All inputs fetched for input vertex : Reducer 2 2018-01-05T11:28:49,917 INFO [Fetcher_B {Reducer_2} #0] impl.ShuffleManager: copy(4 (spillsFetched=4) of 4. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:28:49,917 INFO [ShuffleRunner {Reducer_2}] impl.ShuffleManager: Reducer_2: Shutting down FetchScheduler, Was Interrupted: false 2018-01-05T11:28:49,917 DEBUG [TezTR-518813_1_13_2_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:28:49,917 DEBUG [TezTR-518813_1_13_2_0_0] log.PerfLogger: 2018-01-05T11:28:49,917 INFO [ShuffleRunner {Reducer_2}] impl.ShuffleManager: Reducer_2: Scheduler thread completed 2018-01-05T11:28:49,917 INFO [TezTR-518813_1_13_2_0_0] exec.GroupByOperator: Initializing operator GBY[4] 2018-01-05T11:28:49,917 DEBUG [TezTR-518813_1_13_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,917 DEBUG [TezTR-518813_1_13_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,917 INFO [TezTR-518813_1_13_2_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:49,917 DEBUG [TezTR-518813_1_13_2_0_0] exec.GroupByOperator: Initialization Done 4 GBY 2018-01-05T11:28:49,917 DEBUG [TezTR-518813_1_13_2_0_0] exec.GroupByOperator: Operator 4 GBY initialized 2018-01-05T11:28:49,917 DEBUG [TezTR-518813_1_13_2_0_0] exec.GroupByOperator: Initializing children of 4 GBY 2018-01-05T11:28:49,917 DEBUG [TezTR-518813_1_13_2_0_0] exec.FileSinkOperator: Initializing child 6 FS 2018-01-05T11:28:49,917 INFO [TezTR-518813_1_13_2_0_0] exec.FileSinkOperator: Initializing operator FS[6] 2018-01-05T11:28:49,917 DEBUG [TezTR-518813_1_13_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:49,917 INFO [TezTR-518813_1_13_2_0_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@3610e02f]:[_col0, _col1]:[struct, struct]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@537114b0 2018-01-05T11:28:49,918 DEBUG [TezTR-518813_1_13_2_0_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:28:49,918 DEBUG [TezTR-518813_1_13_2_0_0] exec.FileSinkOperator: Initialization Done 6 FS 2018-01-05T11:28:49,918 DEBUG [TezTR-518813_1_13_2_0_0] exec.FileSinkOperator: Operator 6 FS initialized 2018-01-05T11:28:49,918 DEBUG [TezTR-518813_1_13_2_0_0] exec.FileSinkOperator: Initialization Done 6 FS done is reset. 2018-01-05T11:28:49,918 DEBUG [TezTR-518813_1_13_2_0_0] exec.GroupByOperator: Initialization Done 4 GBY done is reset. 2018-01-05T11:28:49,918 INFO [TezTR-518813_1_13_2_0_0] tez.ReduceRecordProcessor: Id =4 Id =6 <\Children> Id = 4 null<\Parent> <\FS> <\Children> <\Parent> <\GBY> 2018-01-05T11:28:49,918 DEBUG [TezTR-518813_1_13_2_0_0] log.PerfLogger: 2018-01-05T11:28:49,918 INFO [TezTR-518813_1_13_2_0_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 3 2018-01-05T11:28:49,918 DEBUG [TezTR-518813_1_13_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:28:49,918 DEBUG [TezTR-518813_1_13_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:28:49,918 DEBUG [TezTR-518813_1_13_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:28:49,918 DEBUG [TezTR-518813_1_13_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:28:49,918 DEBUG [TezTR-518813_1_13_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:28:49,918 DEBUG [TezTR-518813_1_13_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:28:49,919 INFO [TezTR-518813_1_13_2_0_0] readers.UnorderedKVReader: Num Records read: 4 2018-01-05T11:28:49,919 DEBUG [TezTR-518813_1_13_2_0_0] log.PerfLogger: 2018-01-05T11:28:49,919 DEBUG [TezTR-518813_1_13_2_0_0] mr.ObjectCache: djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39_Reducer 3__REDUCE_PLAN__ no longer needed 2018-01-05T11:28:49,919 DEBUG [TezTR-518813_1_13_2_0_0] exec.GroupByOperator: close called for operator GBY[4] 2018-01-05T11:28:49,919 INFO [TezTR-518813_1_13_2_0_0] exec.GroupByOperator: Closing operator GBY[4] 2018-01-05T11:28:49,920 INFO [TezTR-518813_1_13_2_0_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-49_053_5953809258100630008-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-49_053_5953809258100630008-1/_tmp.-ext-10001/000000_0 2018-01-05T11:28:49,920 INFO [TezTR-518813_1_13_2_0_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-49_053_5953809258100630008-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-49_053_5953809258100630008-1/_task_tmp.-ext-10001/_tmp.000000_0 2018-01-05T11:28:49,920 INFO [TezTR-518813_1_13_2_0_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-49_053_5953809258100630008-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-49_053_5953809258100630008-1/_tmp.-ext-10001/000000_0 2018-01-05T11:28:49,955 INFO [TezTR-518813_1_13_2_0_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:28:49,955 DEBUG [TezTR-518813_1_13_2_0_0] exec.GroupByOperator: Closing child = FS[6] 2018-01-05T11:28:49,955 DEBUG [TezTR-518813_1_13_2_0_0] exec.FileSinkOperator: close called for operator FS[6] 2018-01-05T11:28:49,955 DEBUG [TezTR-518813_1_13_2_0_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:49,955 INFO [TezTR-518813_1_13_2_0_0] exec.FileSinkOperator: Closing operator FS[6] 2018-01-05T11:28:49,955 INFO [TezTR-518813_1_13_2_0_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:28:49,956 INFO [TezTR-518813_1_13_2_0_0] exec.FileSinkOperator: RECORDS_OUT_0:1, 2018-01-05T11:28:49,956 DEBUG [TezTR-518813_1_13_2_0_0] exec.FileSinkOperator: 6 Close done 2018-01-05T11:28:49,956 DEBUG [TezTR-518813_1_13_2_0_0] exec.GroupByOperator: 4 Close done 2018-01-05T11:28:49,956 INFO [TezTR-518813_1_13_2_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_13_02_000000_0 2018-01-05T11:28:49,956 INFO [TezTR-518813_1_13_2_0_0] impl.ShuffleInputEventHandlerImpl: Reducer 2: numDmeEventsSeen=4, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:28:49,956 INFO [TezTR-518813_1_13_2_0_0] impl.ShuffleManager: Shutting down pending fetchers on sourceReducer_2: 0 2018-01-05T11:28:49,956 INFO [TezTR-518813_1_13_2_0_0] output.MROutput: out_Reducer 3 closed 2018-01-05T11:28:49,956 DEBUG [TezTR-518813_1_13_2_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:49,956 INFO [TezTR-518813_1_13_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:49,956 INFO [TezTR-518813_1_13_2_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_13_02_000000_0, askedToStop=false 2018-01-05T11:28:49,956 INFO [TezTR-518813_1_13_2_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_13_02_000000_0, stopRequested=false 2018-01-05T11:28:49,956 INFO [TezTR-518813_1_13_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_13_02_000000_0: Counters: 14 [[org.apache.tez.common.counters.TaskCounter NUM_SHUFFLED_INPUTS=4, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=4, OUTPUT_RECORDS=0, SHUFFLE_BYTES=2819, SHUFFLE_BYTES_DECOMPRESSED=2803, SHUFFLE_BYTES_TO_MEM=2819, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=5, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1][HIVE CREATED_FILES=1, RECORDS_OUT_0=1]] 2018-01-05T11:28:49,956 DEBUG [TezTR-518813_1_13_2_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_13_2_0_0 thread id: 955 scheme: file bytesRead: 0 bytesWritten: 3185 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:49,957 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_13_02_000000_0, isTaskDone=true 2018-01-05T11:28:49,957 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000026, requestId=2, startIndex=4, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_13_02_000000_0, eventCount=2 } 2018-01-05T11:28:49,958 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 3, taskAttemptId=attempt_1515180518813_0001_13_02_000000_0, creationTime=1515180529899, allocationTime=1515180529901, startTime=1515180529901, finishTime=1515180529958, timeTaken=57, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:49,958 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=4, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:28:49,958 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_13_02_000000_0 2018-01-05T11:28:49,958 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_13_02_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:49,958 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_13_02_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:49,958 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_13_02_000000_0 2018-01-05T11:28:49,958 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_13_02_000000_0; false 2018-01-05T11:28:49,958 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_13_02_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:49,959 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:49,959 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000026=55 2018-01-05T11:28:49,959 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_13_02_000000_0: true 2018-01-05T11:28:49,959 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:49,959 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:TASK_FINISHED]: vertexName=Reducer 3, taskId=task_1515180518813_0001_13_02_000000, startTime=1515180529901, finishTime=1515180529958, timeTaken=57, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_13_02_000000_0, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=3185, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, NUM_SHUFFLED_INPUTS=4, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=4, OUTPUT_RECORDS=0, SHUFFLE_BYTES=2819, SHUFFLE_BYTES_DECOMPRESSED=2803, SHUFFLE_BYTES_TO_MEM=2819, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=5, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1 2018-01-05T11:28:49,959 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_13_02 [Reducer 3], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:49,959 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_13_02 [Reducer 3] 2018-01-05T11:28:49,959 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:VERTEX_FINISHED]: vertexName=Reducer 3, vertexId=vertex_1515180518813_0001_13_02, initRequestedTime=1515180529303, initedTime=1515180529307, startRequestedTime=1515180529359, startedTime=1515180529359, finishTime=1515180529959, timeTaken=600, status=SUCCEEDED, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=3185, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, NUM_SHUFFLED_INPUTS=4, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=4, OUTPUT_RECORDS=0, SHUFFLE_BYTES=2819, SHUFFLE_BYTES_DECOMPRESSED=2803, SHUFFLE_BYTES_TO_MEM=2819, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=5, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1, vertexStats=firstTaskStartTime=1515180529901, firstTasksToStart=[ task_1515180518813_0001_13_02_000000 ], lastTaskFinishTime=1515180529958, lastTasksToFinish=[ task_1515180518813_0001_13_02_000000 ], minTaskDuration=57, maxTaskDuration=57, avgTaskDuration=57.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_13_02_000000 ], longestDurationTasks=[ task_1515180518813_0001_13_02_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:49,959 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_13_02 [Reducer 3] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:49,959 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000026, stoppedTime=1515180529959, exitStatus=0 2018-01-05T11:28:49,959 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000026 exited with diagnostics set to Container completed. 2018-01-05T11:28:49,960 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_13_02 [Reducer 3] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3 2018-01-05T11:28:49,960 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null 2018-01-05T11:28:49,960 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_13_01 [Reducer 2] 2018-01-05T11:28:49,960 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_13_00 [Map 1] 2018-01-05T11:28:49,960 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_13_02 [Reducer 3] 2018-01-05T11:28:49,960 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_13, queueSize=0 2018-01-05T11:28:49,961 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_13][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_13, startTime=1515180529290, finishTime=1515180529960, timeTaken=670, status=SUCCEEDED, diagnostics=, counters=Counters: 38, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=6, TOTAL_LAUNCHED_TASKS=6, OTHER_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=1804, AM_GC_TIME_MILLIS=306, File System Counters, FILE_BYTES_READ=5912, FILE_BYTES_WRITTEN=19579, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=8, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1004, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=504, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=8814, OUTPUT_BYTES_WITH_OVERHEAD=15804, OUTPUT_BYTES_PHYSICAL=10102, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=9910, SHUFFLE_BYTES_DECOMPRESSED=9878, SHUFFLE_BYTES_TO_MEM=9910, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=27, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, HIVE, CREATED_FILES=5, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500, RECORDS_OUT_0=1, RECORDS_OUT_1_default.srcbucket2=500, RECORDS_OUT_INTERMEDIATE_Map_1=500, RECORDS_OUT_INTERMEDIATE_Reducer_2=4 2018-01-05T11:28:49,961 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_13 finished with state: SUCCEEDED 2018-01-05T11:28:49,961 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_13 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:28:49,961 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000026 2018-01-05T11:28:49,961 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_13, dagState=SUCCEEDED 2018-01-05T11:28:49,961 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_13_post 2018-01-05T11:28:49,961 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:49,961 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:28:49,961 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:28:49,961 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 13 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=6, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=6,NumRequestsWithoutLocation=6, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=6} 2018-01-05T11:28:49,961 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:28:49,961 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=INSERT INTO srcbucket2 SELEC...srcbucket_tmp (Stage-1), with id=dag_1515180518813_0001_13 2018-01-05T11:28:49,961 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 13, to 10.22.8.182:49818 2018-01-05T11:28:49,961 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=6, Remaining=0 2018-01-05T11:28:49,961 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=INSERT INTO srcbucket2 SELEC...srcbucket_tmp (Stage-1), with id=dag_1515180518813_0001_13 2018-01-05T11:28:49,961 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=13} 2018-01-05T11:28:49,961 INFO [IPC Server handler 3 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=13}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:28:49,962 INFO [IPC Server handler 3 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/13 after 300 seconds 2018-01-05T11:28:49,962 ERROR [IPC Server handler 3 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39, dag_1515180518813_0001_13 2018-01-05T11:28:49,962 INFO [IPC Server handler 3 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39 2018-01-05T11:28:49,962 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=13} = 1 2018-01-05T11:28:49,962 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Issuing killFragment for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=13} attempt_1515180518813_0001_13_02_000000_0 2018-01-05T11:28:49,962 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Removing attempt_1515180518813_0001_13_02_000000_0 from preemptionQueue 2018-01-05T11:28:49,962 INFO [IPC Server handler 3 on 49818] impl.TaskRunnerCallable: Ignoring kill request for task attempt_1515180518813_0001_13_02_000000_0 since it's already complete 2018-01-05T11:28:49,962 DEBUG [IPC Server handler 3 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=13} 2018-01-05T11:28:49,962 DEBUG [IPC Server handler 3 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:28:49,962 DEBUG [IPC Server handler 3 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=13} 2018-01-05T11:28:49,963 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,963 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,963 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,963 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 Reducer 2: 4/4 Reducer 3: 1/1 2018-01-05T11:28:49,963 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,965 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,965 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: TaskId for 000000_0 = 000000 2018-01-05T11:28:49,965 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: TaskId for 000001_0 = 000001 2018-01-05T11:28:49,965 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: TaskId for 000002_0 = 000002 2018-01-05T11:28:49,965 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: TaskId for 000003_0 = 000003 2018-01-05T11:28:49,965 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,965 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,965 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,966 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,966 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: TaskId for 000000_0 = 000000 2018-01-05T11:28:49,966 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,966 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,967 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:49,967 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-2:DEPENDENCY_COLLECTION] in serial mode 2018-01-05T11:28:49,967 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MOVE] in serial mode 2018-01-05T11:28:49,967 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@29c15c32, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:49,967 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:49,967 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:49,967 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:49,967 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:49,967 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:49,967 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:49,967 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:49,968 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:49,968 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Loading data to table default.srcbucket2 from file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/.hive-staging_hive_2018-01-05_11-28-48_798_103036370913189016-1/-ext-10000 2018-01-05T11:28:49,968 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:49,969 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:49,969 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:49,969 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:49,969 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:49,969 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:49,969 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:49,969 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:49,970 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:49,970 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:49,970 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:49,970 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:49,971 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:49,971 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:49,971 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:49,971 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket2 2018-01-05T11:28:49,971 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket2 2018-01-05T11:28:49,975 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:49,975 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:49,975 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:49,975 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket2 2018-01-05T11:28:49,975 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket2 2018-01-05T11:28:49,979 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:49,979 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:49,979 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] FileOperations: moving file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/.hive-staging_hive_2018-01-05_11-28-48_798_103036370913189016-1/-ext-10000 to file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2 (replace = KEEP_EXISTING) 2018-01-05T11:28:49,980 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:49,980 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:49,980 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:49,980 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:49,980 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:49,980 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:49,981 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:49,981 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:49,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Moved src: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/.hive-staging_hive_2018-01-05_11-28-48_798_103036370913189016-1/-ext-10000/000000_0, to dest: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000000_0 2018-01-05T11:28:49,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Moved src: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/.hive-staging_hive_2018-01-05_11-28-48_798_103036370913189016-1/-ext-10000/000001_0, to dest: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000001_0 2018-01-05T11:28:49,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Moved src: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/.hive-staging_hive_2018-01-05_11-28-48_798_103036370913189016-1/-ext-10000/000002_0, to dest: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000002_0 2018-01-05T11:28:49,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Moved src: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/.hive-staging_hive_2018-01-05_11-28-48_798_103036370913189016-1/-ext-10000/000003_0, to dest: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000003_0 2018-01-05T11:28:49,982 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:49,982 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=srcbucket2 newtbl=srcbucket2 2018-01-05T11:28:49,982 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=srcbucket2 newtbl=srcbucket2 2018-01-05T11:28:49,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:49,992 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:49,992 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:49,993 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-3:STATS] in serial mode 2018-01-05T11:28:49,993 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@6f66fa18, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:49,993 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:49,993 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:49,993 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:49,993 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:49,993 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:49,993 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:49,993 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:49,993 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:49,993 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:49,993 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/-mr-10001 2018-01-05T11:28:49,993 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/-mr-10001 2018-01-05T11:28:49,995 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:49,995 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:49,995 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:49,995 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:49,995 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:49,995 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:49,995 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:49,995 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:49,996 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:49,996 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:49,996 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:49,996 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:49,996 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:49,997 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:49,997 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:49,997 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket2 2018-01-05T11:28:49,997 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket2 2018-01-05T11:28:50,000 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,000 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,001 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.srcbucket2/ numRows 2018-01-05T11:28:50,001 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.srcbucket2/ numRows 500 2018-01-05T11:28:50,001 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.srcbucket2/ rawDataSize 2018-01-05T11:28:50,001 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.srcbucket2/ rawDataSize 5312 2018-01-05T11:28:50,001 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,001 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=srcbucket2 newtbl=srcbucket2 2018-01-05T11:28:50,001 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=srcbucket2 newtbl=srcbucket2 2018-01-05T11:28:50,002 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000025, QueryId=djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39-13, DagId=13, VertexName=Reducer 2, TaskId=3, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180529454, EndTime=1515180530002 2018-01-05T11:28:50,002 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_13_02_000000_0 2018-01-05T11:28:50,002 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_13_02_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:50,002 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_13_02_000000_0 2018-01-05T11:28:50,002 INFO [ExecutionCompletionThread #0] impl.QueryTracker: Ignoring fragmentComplete message for unknown query: QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=13} 2018-01-05T11:28:50,005 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,011 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,011 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,012 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.srcbucket2 stats: [numFiles=4, numRows=500, totalSize=5812, rawDataSize=5312] 2018-01-05T11:28:50,012 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: About to delete stats tmp dir :file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1/-mr-10001 2018-01-05T11:28:50,028 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Time taken to get FileStatuses: 14 2018-01-05T11:28:50,028 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:28:50,028 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 14 2018-01-05T11:28:50,028 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FetchOperator: Creating fetchTask with deserializer typeinfo: struct<_col0:struct,_col1:struct> 2018-01-05T11:28:50,028 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FetchOperator: deserializer properties: table properties: {columns=_col0,_col1, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct:struct, escape.delim=\} partition properties: {columns=_col0,_col1, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct:struct, escape.delim=\} 2018-01-05T11:28:50,029 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,033 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.268448ms + 0.00634ms, the query is [select "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "TAB_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (...)] 2018-01-05T11:28:50,034 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,039 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,045 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,046 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: write_column_statistics: db=default table=srcbucket2 2018-01-05T11:28:50,046 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=write_column_statistics: db=default table=srcbucket2 2018-01-05T11:28:50,050 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=srcbucket2 tableName=default colName=key 2018-01-05T11:28:50,055 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=srcbucket2 tableName=default colName=value 2018-01-05T11:28:50,056 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,058 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,058 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,058 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,058 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,058 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: INSERT INTO srcbucket2 SELECT * FROM srcbucket_tmp 2018-01-05T11:28:50,058 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:28:50,058 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@srcbucket_tmp 2018-01-05T11:28:50,058 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@srcbucket2 2018-01-05T11:28:50,059 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Lineage: srcbucket2.key SIMPLE [(srcbucket_tmp)srcbucket_tmp.FieldSchema(name:key, type:int, comment:null), ] 2018-01-05T11:28:50,059 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Lineage: srcbucket2.value SIMPLE [(srcbucket_tmp)srcbucket_tmp.FieldSchema(name:value, type:string, comment:null), ] 2018-01-05T11:28:50,059 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,059 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,059 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:50,059 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=10, getTable_(String, String, )=3, setPartitionColumnStatistics_(SetPartitionsStatsRequest, )=29} 2018-01-05T11:28:50,059 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39); Time taken: 0.935 seconds 2018-01-05T11:28:50,059 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:50,059 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,059 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:50,059 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,059 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query INSERT INTO srcbucket2 SELECT * FROM srcbucket_tmp 2018-01-05T11:28:50,059 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/.hive-staging_hive_2018-01-05_11-28-48_798_103036370913189016-1 2018-01-05T11:28:50,059 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-48_798_103036370913189016-1 2018-01-05T11:28:50,060 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 1.261 seconds 2018-01-05T11:28:50,060 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:50,060 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:50,060 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:50,060 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:50,060 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,060 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:50,060 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,060 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,060 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,060 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: DROP TABLE srcbucket_tmp 2018-01-05T11:28:50,060 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112850_b2c2a699-4f26-4aeb-b393-c7900401f808): DROP TABLE srcbucket_tmp 2018-01-05T11:28:50,060 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,060 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: DROP TABLE srcbucket_tmp 2018-01-05T11:28:50,061 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:50,061 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,061 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,061 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,061 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,061 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:50,061 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,061 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:50,061 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:50,067 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,068 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,068 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:50,068 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,068 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:50,068 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,068 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:50,068 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=7, flushCache_()=0} 2018-01-05T11:28:50,068 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112850_b2c2a699-4f26-4aeb-b393-c7900401f808); Time taken: 0.008 seconds 2018-01-05T11:28:50,068 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:50,068 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,068 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112850_b2c2a699-4f26-4aeb-b393-c7900401f808): DROP TABLE srcbucket_tmp 2018-01-05T11:28:50,069 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,069 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: DROP TABLE srcbucket_tmp 2018-01-05T11:28:50,069 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:50,069 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@srcbucket_tmp 2018-01-05T11:28:50,069 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@srcbucket_tmp 2018-01-05T11:28:50,069 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,069 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,069 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,069 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,070 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:50,070 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,070 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:50,070 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:50,073 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,074 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,074 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,074 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:50,074 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:50,078 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,078 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,078 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,079 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: drop_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:50,079 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=drop_table : db=default tbl=srcbucket_tmp 2018-01-05T11:28:50,097 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 6.74945ms + 0.006948ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc] 2018-01-05T11:28:50,107 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000026, QueryId=djaiswal_20180105112848_693c52a5-9ea5-4c7a-8dd4-ea2d230c0b39-13, DagId=13, VertexName=Reducer 3, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180529903, EndTime=1515180530107 2018-01-05T11:28:50,133 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,133 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.FileUtils: deleting file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_tmp 2018-01-05T11:28:50,135 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,135 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@srcbucket_tmp) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@srcbucket_tmp) Type=TABLE WriteType=DDL_EXCLUSIVE is present 2018-01-05T11:28:50,135 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,135 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,135 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: DROP TABLE srcbucket_tmp 2018-01-05T11:28:50,135 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:50,135 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@srcbucket_tmp 2018-01-05T11:28:50,135 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@srcbucket_tmp 2018-01-05T11:28:50,135 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,135 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,135 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:50,135 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=61, getTable_(String, String, )=4} 2018-01-05T11:28:50,135 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112850_b2c2a699-4f26-4aeb-b393-c7900401f808); Time taken: 0.067 seconds 2018-01-05T11:28:50,135 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:50,135 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,135 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:50,135 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,135 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query DROP TABLE srcbucket_tmp 2018-01-05T11:28:50,136 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.075 seconds 2018-01-05T11:28:50,136 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:50,136 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:50,136 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:50,136 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:50,136 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,136 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:50,136 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,136 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,136 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,136 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: ANALYZE TABLE srcbucket2 COMPUTE STATISTICS 2018-01-05T11:28:50,136 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112850_e39d2de5-06f7-4fe9-ad4b-81b9a060d703): ANALYZE TABLE srcbucket2 COMPUTE STATISTICS 2018-01-05T11:28:50,136 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,136 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: ANALYZE TABLE srcbucket2 COMPUTE STATISTICS 2018-01-05T11:28:50,136 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:50,136 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,136 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,137 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,137 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,137 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:50,137 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Invoking analyze on original query 2018-01-05T11:28:50,137 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Starting Semantic Analysis 2018-01-05T11:28:50,137 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed phase 1 of Semantic Analysis 2018-01-05T11:28:50,137 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for source tables 2018-01-05T11:28:50,137 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,137 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket2 2018-01-05T11:28:50,137 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket2 2018-01-05T11:28:50,142 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,142 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,142 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,142 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket2 2018-01-05T11:28:50,142 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket2 2018-01-05T11:28:50,146 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,147 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for subqueries 2018-01-05T11:28:50,147 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for destination tables 2018-01-05T11:28:50,147 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed getting MetaData in Semantic Analysis 2018-01-05T11:28:50,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_136_7492829147931145598-1/-mr-10000 2018-01-05T11:28:50,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Table Plan for srcbucket2 TS[0] 2018-01-05T11:28:50,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Body Plan for Query Block null 2018-01-05T11:28:50,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Plan for Query Block null 2018-01-05T11:28:50,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Before logical optimization TS[0] 2018-01-05T11:28:50,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] lineage.Generator: Time taken for lineage transform=0 2018-01-05T11:28:50,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: int|{srcbucket2}key,value: string|{srcbucket2}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{srcbucket2}block__offset__inside__file,INPUT__FILE__NAME: string|{srcbucket2}input__file__name,ROW__ID: struct|{srcbucket2}row__id,RAW__DATA__SIZE: bigint|{srcbucket2}raw__data__size) 2018-01-05T11:28:50,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,160 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:28:50,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.PredicatePushDown: After PPD: TS[0] 2018-01-05T11:28:50,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: int|{srcbucket2}key,value: string|{srcbucket2}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{srcbucket2}block__offset__inside__file,INPUT__FILE__NAME: string|{srcbucket2}input__file__name,ROW__ID: struct|{srcbucket2}row__id,RAW__DATA__SIZE: bigint|{srcbucket2}raw__data__size) 2018-01-05T11:28:50,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: int|{srcbucket2}key,value: string|{srcbucket2}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{srcbucket2}block__offset__inside__file,INPUT__FILE__NAME: string|{srcbucket2}input__file__name,ROW__ID: struct|{srcbucket2}row__id,RAW__DATA__SIZE: bigint|{srcbucket2}raw__data__size) 2018-01-05T11:28:50,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: After logical optimization TS[0] 2018-01-05T11:28:50,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,161 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,161 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=srcbucket2 2018-01-05T11:28:50,161 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=srcbucket2 2018-01-05T11:28:50,165 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting stats (Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: COMPLETE) on TS[0] 2018-01-05T11:28:50,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (srcbucket2): numRows: 500 dataSize: 5312 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:28:50,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[]]; bucket count: 4 }) on TS[0] 2018-01-05T11:28:50,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:50,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:28:50,170 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Cycle free: true 2018-01-05T11:28:50,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: int|{srcbucket2}key,value: string|{srcbucket2}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{srcbucket2}block__offset__inside__file,INPUT__FILE__NAME: string|{srcbucket2}input__file__name,ROW__ID: struct|{srcbucket2}row__id,RAW__DATA__SIZE: bigint|{srcbucket2}raw__data__size) 2018-01-05T11:28:50,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:28:50,171 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2 of table srcbucket2 2018-01-05T11:28:50,171 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2 2018-01-05T11:28:50,171 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:28:50,171 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,171 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,171 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:50,171 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:50,171 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:50,171 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:50,171 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:50,171 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:50,171 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:28:50,171 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping vectorization 2018-01-05T11:28:50,171 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:28:50,171 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: llap mode: only 2018-01-05T11:28:50,171 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:28:50,172 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:50,172 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@433ae644 can be done in LLAP 2018-01-05T11:28:50,172 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,172 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed plan generation 2018-01-05T11:28:50,172 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:50,172 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validation start 2018-01-05T11:28:50,172 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Not a partition. 2018-01-05T11:28:50,172 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,172 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:srcbucket2.key, type:int, comment:null), FieldSchema(name:srcbucket2.value, type:string, comment:null)], properties:null) 2018-01-05T11:28:50,172 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,172 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:50,172 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=1, getTable_(String, String, )=10, flushCache_()=0, getTableColumnStatistics_(String, String, List, )=9} 2018-01-05T11:28:50,172 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112850_e39d2de5-06f7-4fe9-ad4b-81b9a060d703); Time taken: 0.036 seconds 2018-01-05T11:28:50,172 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:50,172 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,172 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112850_e39d2de5-06f7-4fe9-ad4b-81b9a060d703): ANALYZE TABLE srcbucket2 COMPUTE STATISTICS 2018-01-05T11:28:50,173 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,173 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: ANALYZE TABLE srcbucket2 COMPUTE STATISTICS 2018-01-05T11:28:50,173 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:28:50,173 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@srcbucket2 2018-01-05T11:28:50,173 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@srcbucket2 2018-01-05T11:28:50,173 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,173 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,173 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,173 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Query ID = djaiswal_20180105112850_e39d2de5-06f7-4fe9-ad4b-81b9a060d703 2018-01-05T11:28:50,173 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Total jobs = 1 2018-01-05T11:28:50,173 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,174 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:28:50,174 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MAPRED] in serial mode 2018-01-05T11:28:50,186 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_136_7492829147931145598-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:28:50,214 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:28:50,214 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:28:50,214 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:28:50,214 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112850_e39d2de5-06f7-4fe9-ad4b-81b9a060d703 2018-01-05T11:28:50,214 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Session is already open 2018-01-05T11:28:50,214 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:28:50,214 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,214 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Dag name: ANALYZE TABLE srcbucket2 COMPUTE STATISTICS (Stage-0) 2018-01-05T11:28:50,214 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: DagInfo: {"context":"Hive","description":"\n\nANALYZE TABLE srcbucket2 COMPUTE STATISTICS"} 2018-01-05T11:28:50,214 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112850_e39d2de5-06f7-4fe9-ad4b-81b9a060d703 with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:28:50,214 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,228 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:28:50,228 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,229 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:28:50,229 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,229 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 1.86KB 2018-01-05T11:28:50,235 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_136_7492829147931145598-1/-mr-10000 2018-01-05T11:28:50,247 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_136_7492829147931145598-1/-mr-10000 2018-01-05T11:28:50,249 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2 2018-01-05T11:28:50,249 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,249 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,249 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,249 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=ANALYZE TABLE srcbucket2 COMPUTE STATISTICS (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112850_e39d2de5-06f7-4fe9-ad4b-81b9a060d703 } 2018-01-05T11:28:50,249 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:50,251 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: ANALYZE TABLE srcbucket2 COMPUTE STATISTICS (Stage-0) 2018-01-05T11:28:50,251 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:28:50,251 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_14, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_14.dot 2018-01-05T11:28:50,252 INFO [IPC Server handler 0 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_14 2018-01-05T11:28:50,252 WARN [IPC Server handler 0 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:50,252 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Running DAG: ANALYZE TABLE srcbucket2 COMPUTE STATISTICS (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112850_e39d2de5-06f7-4fe9-ad4b-81b9a060d703 } 2018-01-05T11:28:50,273 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_14][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_14, submitTime=1515180530251, queueName=null 2018-01-05T11:28:50,274 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_14_00 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_14_00 [Map 1], Default Resources= 2018-01-05T11:28:50,274 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_14_00 [Map 1] 2018-01-05T11:28:50,274 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_14_00 [Map 1] 2018-01-05T11:28:50,274 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:28:50,274 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_14_00 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:50,274 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:28:50,274 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_14][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_14, initTime=1515180530274 2018-01-05T11:28:50,274 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: dag_1515180518813_0001_14 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:28:50,275 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:28:50,275 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_14][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_14, startTime=1515180530275 2018-01-05T11:28:50,275 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_14 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:28:50,275 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {srcbucket2={InputName=srcbucket2}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:28:50,275 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: srcbucket2, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:28:50,275 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_14_00 [Map 1] 2018-01-05T11:28:50,275 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_14, dagName=ANALYZE TABLE srcbucket2 COMPUTE STATISTICS (Stage-0) 2018-01-05T11:28:50,275 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,275 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,275 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,279 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_14_00 [Map 1] 2018-01-05T11:28:50,279 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_14_00 [Map 1] 2018-01-05T11:28:50,279 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:50,279 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_14_00 [Map 1] 2018-01-05T11:28:50,281 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:28:50,281 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@23be7f4f 2018-01-05T11:28:50,281 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_136_7492829147931145598-1/djaiswal/_tez_scratch_dir/139e8cb9-4200-476d-ac85-88abc0cef61a/map.xml 2018-01-05T11:28:50,281 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_136_7492829147931145598-1/djaiswal/_tez_scratch_dir/139e8cb9-4200-476d-ac85-88abc0cef61a/map.xml 2018-01-05T11:28:50,281 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_136_7492829147931145598-1/djaiswal/_tez_scratch_dir/139e8cb9-4200-476d-ac85-88abc0cef61a/map.xml 2018-01-05T11:28:50,281 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:50,281 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:50,281 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:50,281 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:50,281 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 1.86KB 2018-01-05T11:28:50,282 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_14_00 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:28:50,282 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: srcbucket2 on vertex vertex_1515180518813_0001_14_00 [Map 1] 2018-01-05T11:28:50,282 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:28:50,282 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:50,282 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:28:50,282 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:50,282 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_136_7492829147931145598-1/djaiswal/_tez_scratch_dir/139e8cb9-4200-476d-ac85-88abc0cef61a/map.xml 2018-01-05T11:28:50,282 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:50,282 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias srcbucket2 2018-01-05T11:28:50,282 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2 2018-01-05T11:28:50,282 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2 2018-01-05T11:28:50,283 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 2018-01-05T11:28:50,283 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = 2018-01-05T11:28:50,283 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2 2018-01-05T11:28:50,283 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:50,283 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:50,283 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:50,283 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:50,343 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Time taken to get FileStatuses: 60 2018-01-05T11:28:50,343 INFO [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total input files to process : 4 2018-01-05T11:28:50,343 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total # of splits generated by getSplits: 4, TimeTaken: 60 2018-01-05T11:28:50,343 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 4 2018-01-05T11:28:50,343 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:50,343 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 4. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:28:50,343 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_136_7492829147931145598-1/djaiswal/_tez_scratch_dir/139e8cb9-4200-476d-ac85-88abc0cef61a/map.xml 2018-01-05T11:28:50,343 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_136_7492829147931145598-1/djaiswal/_tez_scratch_dir/139e8cb9-4200-476d-ac85-88abc0cef61a/map.xml 2018-01-05T11:28:50,343 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_136_7492829147931145598-1/djaiswal/_tez_scratch_dir/139e8cb9-4200-476d-ac85-88abc0cef61a/map.xml 2018-01-05T11:28:50,343 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:50,343 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:50,343 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:50,344 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:50,344 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 1.86KB 2018-01-05T11:28:50,344 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000001_0 to src new group? true 2018-01-05T11:28:50,344 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000002_0 to src new group? false 2018-01-05T11:28:50,344 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000000_0 to src new group? false 2018-01-05T11:28:50,344 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000003_0 to src new group? false 2018-01-05T11:28:50,344 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:28:50,344 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:28:50,344 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:28:50,344 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 107 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 5812 Original splits: 4 2018-01-05T11:28:50,344 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired numSplits: 1 lengthPerGroup: 5812 numLocations: 1 numSplitsPerLocation: 4 numSplitsInGroup: 4 totalLength: 5812 numOriginalSplits: 4 . Grouping by length: true count: false nodeLocalOnly: false 2018-01-05T11:28:50,344 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Number of splits desired: 1 created: 1 splitsProcessed: 4 2018-01-05T11:28:50,344 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 4 grouped split count is 1, for bucket: 1 2018-01-05T11:28:50,344 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:28:50,345 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: srcbucket2 on vertex vertex_1515180518813_0001_14_00 [Map 1] 2018-01-05T11:28:50,345 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_14_00 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:50,346 INFO [App Shared Pool - #0] impl.VertexImpl: Got updated RootInputsSpecs: {srcbucket2=forAllWorkUnits=true, update=[1]} 2018-01-05T11:28:50,346 INFO [App Shared Pool - #0] impl.VertexImpl: Vertex vertex_1515180518813_0001_14_00 [Map 1] parallelism set to 1 2018-01-05T11:28:50,346 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_14_00 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:50,346 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_14_00 [Map 1], numAdditionalOutputs=1 2018-01-05T11:28:50,346 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_14][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_14_00, initRequestedTime=1515180530275, initedTime=1515180530346, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:50,346 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_14_00 [Map 1] with distanceFromRoot: 0 2018-01-05T11:28:50,346 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_14_00 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:28:50,346 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_14][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_14_00, startRequestedTime=1515180530282, startedTime=1515180530346 2018-01-05T11:28:50,346 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_14][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_14_00, reconfigureDoneTime=1515180530346, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:28:50,346 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_14_00 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:50,346 INFO [App Shared Pool - #0] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:28:50,346 INFO [App Shared Pool - #0] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:28:50,346 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_14_00 [Map 1] 2018-01-05T11:28:50,347 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_14][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_14_00_000000, scheduledTime=1515180530347, launchTime=1515180530347 2018-01-05T11:28:50,347 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_14_00 [Map 1] 2018-01-05T11:28:50,347 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_14_00_000000_0, priority=2, capability=, hosts=null 2018-01-05T11:28:50,347 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=2,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:50,347 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=2,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:50,347 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:50,347 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_14_00_000000_0 on hosts=null 2018-01-05T11:28:50,347 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_14_00_000000_0 on hosts=null 2018-01-05T11:28:50,347 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_14_00_000000_0 2018-01-05T11:28:50,347 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:50,347 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #27, task=TaskInfo{task=attempt_1515180518813_0001_14_00_000000_0, priority=2, startTime=0, containerId=null, uniqueId=26, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000027 2018-01-05T11:28:50,347 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_14_00_000000_0; false 2018-01-05T11:28:50,347 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_14_00_000000_0, priority=2, startTime=311550856, containerId=container_111101111_0001_00_000027assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=26, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:50,347 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000027 for node: 10.22.8.182:49818 2018-01-05T11:28:50,350 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 14, name=ANALYZE TABLE srcbucket2 COMPUTE STATISTICS (Stage-0), queryId=djaiswal_20180105112850_e39d2de5-06f7-4fe9-ad4b-81b9a060d703 2018-01-05T11:28:50,350 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000027, attempt_1515180518813_0001_14_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:50,350 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000027 for node: 10.22.8.182:49818 2018-01-05T11:28:50,351 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000027 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:50,351 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000027, launchTime=1515180530351 2018-01-05T11:28:50,352 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_14_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000027] on NM: [10.22.8.182:49818] 2018-01-05T11:28:50,352 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:50,352 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:50,352 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_14_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112850_e39d2de5-06f7-4fe9-ad4b-81b9a060d703, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000027, dagName=ANALYZE TABLE srcbucket2 COMPUTE STATISTICS (Stage-0), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{srcbucket2,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{out_Map 1,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180530275, firstAttemptStartTime=1515180530347, currentAttemptStartTime=1515180530350} 2018-01-05T11:28:50,352 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_14][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_14_00_000000_0, startTime=1515180530351, containerId=container_111101111_0001_00_000027, nodeId=10.22.8.182:49818 2018-01-05T11:28:50,352 INFO [IPC Server handler 1 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000027, QueryId=djaiswal_20180105112850_e39d2de5-06f7-4fe9-ad4b-81b9a060d703-14, DagId=14, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180530352 2018-01-05T11:28:50,352 DEBUG [IPC Server handler 1 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=14} with the ShuffleHandler 2018-01-05T11:28:50,369 INFO [IPC Server handler 1 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:28:50,369 DEBUG [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/14] 2018-01-05T11:28:50,369 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:50,369 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_14_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:50,369 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_14_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180530347, dagStartTime=1515180530275, withinDagPriority=2, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:28:50,369 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_14_00_000000_0 : ACCEPTED 2018-01-05T11:28:50,370 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_14_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:50,370 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_14_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180530347, dagStartTime=1515180530275, withinDagPriority=2, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:28:50,370 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_14_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:50,370 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_14_00_000000_0 to pre-emption queue 2018-01-05T11:28:50,370 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_14_00_000000_0: true 2018-01-05T11:28:50,370 INFO [TaskCommunicator # 8] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_14_00_000000_0 2018-01-05T11:28:50,370 INFO [TaskCommunicator # 8] GuaranteedTasks: Received response for attempt_1515180518813_0001_14_00_000000_0, true 2018-01-05T11:28:50,370 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:1000809636 2018-01-05T11:28:50,371 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : ANALYZE TABLE srcbucket2 COMPUTE STATISTICS (Stage-0), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_14_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=srcbucket2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=out_Map 1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:28:50,371 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1039138816, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:50,371 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_14_00_000000_0 vertexName: Map 1 2018-01-05T11:28:50,371 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000027, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_14_00_000000_0, eventCount=1 } 2018-01-05T11:28:50,372 DEBUG [TezTR-518813_1_14_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_14 queryId: djaiswal_20180105112850_e39d2de5-06f7-4fe9-ad4b-81b9a060d703 instanceType: TaskRunner2Callable 2018-01-05T11:28:50,372 INFO [TezTR-518813_1_14_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_14_00_000000_0 2018-01-05T11:28:50,372 DEBUG [TezTR-518813_1_14_0_0_0] log.PerfLogger: 2018-01-05T11:28:50,373 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:50,374 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:50,376 INFO [IPC Server handler 18 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_14_00_000000_0 sent events: (0-1). 2018-01-05T11:28:50,377 DEBUG [TezTR-518813_1_14_0_0_0] log.PerfLogger: 2018-01-05T11:28:50,377 INFO [TezTR-518813_1_14_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:50,377 INFO [TezTR-518813_1_14_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:50,377 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:28:50,377 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_14_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:28:50,377 INFO [I/O Setup 0 Initialize: {srcbucket2}] input.MRInput: srcbucket2 using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:28:50,377 INFO [I/O Setup 0 Initialize: {srcbucket2}] input.MRInputLegacy: srcbucket2 MRInputLegacy deferring initialization 2018-01-05T11:28:50,377 INFO [TezTR-518813_1_14_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:50,378 INFO [I/O Setup 1 Initialize: {out_Map 1}] output.MROutput: out_Map 1: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:28:50,378 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:50,378 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:50,378 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:50,378 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:50,378 INFO [TezTR-518813_1_14_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:50,378 INFO [TezTR-518813_1_14_0_0_0] resources.MemoryDistributor: InitialRequests=[srcbucket2:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [out_Map 1:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput] 2018-01-05T11:28:50,378 INFO [TezTR-518813_1_14_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:50,378 INFO [TezTR-518813_1_14_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:50,378 INFO [TezTR-518813_1_14_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 1039138816, TotalAvailable: 696223006, TotalRequested/TotalJVMHeap:0.00 2018-01-05T11:28:50,378 INFO [TezTR-518813_1_14_0_0_0] resources.MemoryDistributor: Allocations=[srcbucket2:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [out_Map 1:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0] 2018-01-05T11:28:50,378 INFO [TezTR-518813_1_14_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:50,378 INFO [TezTR-518813_1_14_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: srcbucket2 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:28:50,378 INFO [TezTR-518813_1_14_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:28:50,378 INFO [TezTR-518813_1_14_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:28:50,378 INFO [I/O Setup 0 Start: {srcbucket2}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: srcbucket2 2018-01-05T11:28:50,378 INFO [TezTR-518813_1_14_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:50,378 INFO [TezTR-518813_1_14_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_14_00_000000_0 2018-01-05T11:28:50,378 DEBUG [TezTR-518813_1_14_0_0_0] log.PerfLogger: 2018-01-05T11:28:50,379 DEBUG [TezTR-518813_1_14_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_14_00_000000_0_10079 2018-01-05T11:28:50,379 DEBUG [TezTR-518813_1_14_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:50,379 DEBUG [TezTR-518813_1_14_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_14_00_000000_0 2018-01-05T11:28:50,379 INFO [TezTR-518813_1_14_0_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112850_e39d2de5-06f7-4fe9-ad4b-81b9a060d703 2018-01-05T11:28:50,379 DEBUG [TezTR-518813_1_14_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:50,379 DEBUG [TezTR-518813_1_14_0_0_0] log.PerfLogger: 2018-01-05T11:28:50,379 DEBUG [TezTR-518813_1_14_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:50,379 DEBUG [TezTR-518813_1_14_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:50,379 DEBUG [TezTR-518813_1_14_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112850_e39d2de5-06f7-4fe9-ad4b-81b9a060d703_Map 1__MAP_PLAN__ 2018-01-05T11:28:50,379 INFO [TezTR-518813_1_14_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_136_7492829147931145598-1/djaiswal/_tez_scratch_dir/139e8cb9-4200-476d-ac85-88abc0cef61a/map.xml 2018-01-05T11:28:50,379 DEBUG [TezTR-518813_1_14_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_136_7492829147931145598-1/djaiswal/_tez_scratch_dir/139e8cb9-4200-476d-ac85-88abc0cef61a/map.xml 2018-01-05T11:28:50,379 DEBUG [TezTR-518813_1_14_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_136_7492829147931145598-1/djaiswal/_tez_scratch_dir/139e8cb9-4200-476d-ac85-88abc0cef61a/map.xml 2018-01-05T11:28:50,379 DEBUG [TezTR-518813_1_14_0_0_0] log.PerfLogger: 2018-01-05T11:28:50,379 INFO [TezTR-518813_1_14_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:50,379 DEBUG [TezTR-518813_1_14_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:50,380 DEBUG [TezTR-518813_1_14_0_0_0] log.PerfLogger: 2018-01-05T11:28:50,380 INFO [TezTR-518813_1_14_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 1.86KB 2018-01-05T11:28:50,380 DEBUG [TezTR-518813_1_14_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:50,380 INFO [TezTR-518813_1_14_0_0_0] tez.MapRecordProcessor: The input names are: [srcbucket2] 2018-01-05T11:28:50,380 INFO [TezTR-518813_1_14_0_0_0] input.MRInput: Initialized MRInput: srcbucket2 2018-01-05T11:28:50,381 DEBUG [TezTR-518813_1_14_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 4 2018-01-05T11:28:50,381 INFO [TezTR-518813_1_14_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_136_7492829147931145598-1/djaiswal/_tez_scratch_dir/139e8cb9-4200-476d-ac85-88abc0cef61a/map.xml 2018-01-05T11:28:50,381 DEBUG [TezTR-518813_1_14_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:50,381 DEBUG [TezTR-518813_1_14_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000001_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=4, serialization.ddl=struct srcbucket2 { i32 key, string value}, transient_lastDdlTime=1515180530, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket2}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=4, serialization.ddl=struct srcbucket2 { i32 key, string value}, transient_lastDdlTime=1515180530, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket2}, baseFileName=srcbucket2, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=4, serialization.ddl=struct srcbucket2 { i32 key, string value}, transient_lastDdlTime=1515180530, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket2}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=4, serialization.ddl=struct srcbucket2 { i32 key, string value}, transient_lastDdlTime=1515180530, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket2}, baseFileName=srcbucket2, vectorPartitionDesc=null]} 2018-01-05T11:28:50,381 DEBUG [TezTR-518813_1_14_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:50,381 INFO [TezTR-518813_1_14_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:50,381 DEBUG [TezTR-518813_1_14_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:50,383 DEBUG [TezTR-518813_1_14_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:50,383 INFO [TezTR-518813_1_14_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_136_7492829147931145598-1/djaiswal/_tez_scratch_dir/139e8cb9-4200-476d-ac85-88abc0cef61a/map.xml 2018-01-05T11:28:50,383 DEBUG [TezTR-518813_1_14_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:50,383 INFO [TezTR-518813_1_14_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_14_00_000000_0 2018-01-05T11:28:50,383 INFO [TezTR-518813_1_14_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:28:50,383 WARN [TezTR-518813_1_14_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000001_0 (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:28:50,383 DEBUG [TezTR-518813_1_14_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:28:50,383 WARN [TezTR-518813_1_14_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000001_0:0+1614 2018-01-05T11:28:50,384 DEBUG [TezTR-518813_1_14_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000001_0 2018-01-05T11:28:50,384 INFO [TezTR-518813_1_14_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000001_0:0+1614, org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000002_0:0+1482, org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000000_0:0+1374, org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000003_0:0+1342], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=5812} 2018-01-05T11:28:50,384 INFO [TezTR-518813_1_14_0_0_0] input.MRInput: srcbucket2 initialized RecordReader from event 2018-01-05T11:28:50,384 DEBUG [TezTR-518813_1_14_0_0_0] tez.MapRecordProcessor: Starting Output: out_Map 1 2018-01-05T11:28:50,384 INFO [TezTR-518813_1_14_0_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:28:50,384 INFO [TezTR-518813_1_14_0_0_0] exec.MapOperator: Initializing operator MAP[0] 2018-01-05T11:28:50,384 DEBUG [TezTR-518813_1_14_0_0_0] exec.MapOperator: Initialization Done 0 MAP 2018-01-05T11:28:50,384 DEBUG [TezTR-518813_1_14_0_0_0] exec.MapOperator: Operator 0 MAP initialized 2018-01-05T11:28:50,384 DEBUG [TezTR-518813_1_14_0_0_0] exec.MapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:28:50,384 DEBUG [TezTR-518813_1_14_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:50,384 DEBUG [TezTR-518813_1_14_0_0_0] exec.MapOperator: Adding alias srcbucket2 to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2 2018-01-05T11:28:50,384 DEBUG [TezTR-518813_1_14_0_0_0] exec.MapOperator: dump TS[0] struct 2018-01-05T11:28:50,384 INFO [TezTR-518813_1_14_0_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:28:50,384 INFO [TezTR-518813_1_14_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:50,384 DEBUG [TezTR-518813_1_14_0_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:28:50,384 INFO [TezTR-518813_1_14_0_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:28:50,384 DEBUG [TezTR-518813_1_14_0_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:28:50,384 DEBUG [TezTR-518813_1_14_0_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:28:50,384 DEBUG [TezTR-518813_1_14_0_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:28:50,384 DEBUG [TezTR-518813_1_14_0_0_0] log.PerfLogger: 2018-01-05T11:28:50,384 INFO [TezTR-518813_1_14_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_136_7492829147931145598-1/djaiswal/_tez_scratch_dir/139e8cb9-4200-476d-ac85-88abc0cef61a/map.xml 2018-01-05T11:28:50,384 DEBUG [TezTR-518813_1_14_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:50,384 DEBUG [TezTR-518813_1_14_0_0_0] exec.MapOperator: Processing alias(es) srcbucket2 for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000001_0 2018-01-05T11:28:50,385 INFO [TezTR-518813_1_14_0_0_0] exec.MapOperator: MAP[0]: records read - 1 2018-01-05T11:28:50,385 INFO [TezTR-518813_1_14_0_0_0] exec.MapOperator: MAP[0]: records read - 10 2018-01-05T11:28:50,385 INFO [TezTR-518813_1_14_0_0_0] exec.MapOperator: MAP[0]: records read - 100 2018-01-05T11:28:50,385 DEBUG [TezTR-518813_1_14_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 1 of 4 2018-01-05T11:28:50,385 DEBUG [TezTR-518813_1_14_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000002_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=4, serialization.ddl=struct srcbucket2 { i32 key, string value}, transient_lastDdlTime=1515180530, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket2}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=4, serialization.ddl=struct srcbucket2 { i32 key, string value}, transient_lastDdlTime=1515180530, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket2}, baseFileName=srcbucket2, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=4, serialization.ddl=struct srcbucket2 { i32 key, string value}, transient_lastDdlTime=1515180530, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket2}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=4, serialization.ddl=struct srcbucket2 { i32 key, string value}, transient_lastDdlTime=1515180530, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket2}, baseFileName=srcbucket2, vectorPartitionDesc=null]} 2018-01-05T11:28:50,385 DEBUG [TezTR-518813_1_14_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:50,385 INFO [TezTR-518813_1_14_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:50,385 DEBUG [TezTR-518813_1_14_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:50,386 DEBUG [TezTR-518813_1_14_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:50,386 INFO [TezTR-518813_1_14_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_136_7492829147931145598-1/djaiswal/_tez_scratch_dir/139e8cb9-4200-476d-ac85-88abc0cef61a/map.xml 2018-01-05T11:28:50,386 DEBUG [TezTR-518813_1_14_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:50,386 INFO [TezTR-518813_1_14_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_14_00_000000_0 2018-01-05T11:28:50,386 INFO [TezTR-518813_1_14_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:28:50,386 WARN [TezTR-518813_1_14_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000002_0 (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:28:50,387 DEBUG [TezTR-518813_1_14_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:28:50,387 WARN [TezTR-518813_1_14_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000002_0:0+1482 2018-01-05T11:28:50,387 DEBUG [TezTR-518813_1_14_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000002_0 2018-01-05T11:28:50,387 INFO [TezTR-518813_1_14_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_136_7492829147931145598-1/djaiswal/_tez_scratch_dir/139e8cb9-4200-476d-ac85-88abc0cef61a/map.xml 2018-01-05T11:28:50,387 DEBUG [TezTR-518813_1_14_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:50,387 DEBUG [TezTR-518813_1_14_0_0_0] exec.MapOperator: Processing alias(es) srcbucket2 for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000002_0 2018-01-05T11:28:50,387 DEBUG [TezTR-518813_1_14_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 2 of 4 2018-01-05T11:28:50,387 DEBUG [TezTR-518813_1_14_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000000_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=4, serialization.ddl=struct srcbucket2 { i32 key, string value}, transient_lastDdlTime=1515180530, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket2}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=4, serialization.ddl=struct srcbucket2 { i32 key, string value}, transient_lastDdlTime=1515180530, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket2}, baseFileName=srcbucket2, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=4, serialization.ddl=struct srcbucket2 { i32 key, string value}, transient_lastDdlTime=1515180530, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket2}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=4, serialization.ddl=struct srcbucket2 { i32 key, string value}, transient_lastDdlTime=1515180530, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket2}, baseFileName=srcbucket2, vectorPartitionDesc=null]} 2018-01-05T11:28:50,387 DEBUG [TezTR-518813_1_14_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:50,387 INFO [TezTR-518813_1_14_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:50,387 DEBUG [TezTR-518813_1_14_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:50,388 DEBUG [TezTR-518813_1_14_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:50,388 INFO [TezTR-518813_1_14_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_136_7492829147931145598-1/djaiswal/_tez_scratch_dir/139e8cb9-4200-476d-ac85-88abc0cef61a/map.xml 2018-01-05T11:28:50,388 DEBUG [TezTR-518813_1_14_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:50,388 INFO [TezTR-518813_1_14_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_14_00_000000_0 2018-01-05T11:28:50,389 INFO [TezTR-518813_1_14_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:28:50,389 WARN [TezTR-518813_1_14_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000000_0 (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:28:50,389 DEBUG [TezTR-518813_1_14_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:28:50,389 WARN [TezTR-518813_1_14_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000000_0:0+1374 2018-01-05T11:28:50,389 DEBUG [TezTR-518813_1_14_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000000_0 2018-01-05T11:28:50,389 INFO [TezTR-518813_1_14_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_136_7492829147931145598-1/djaiswal/_tez_scratch_dir/139e8cb9-4200-476d-ac85-88abc0cef61a/map.xml 2018-01-05T11:28:50,389 DEBUG [TezTR-518813_1_14_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:50,389 DEBUG [TezTR-518813_1_14_0_0_0] exec.MapOperator: Processing alias(es) srcbucket2 for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000000_0 2018-01-05T11:28:50,390 DEBUG [TezTR-518813_1_14_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 3 of 4 2018-01-05T11:28:50,390 DEBUG [TezTR-518813_1_14_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000003_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=4, serialization.ddl=struct srcbucket2 { i32 key, string value}, transient_lastDdlTime=1515180530, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket2}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=4, serialization.ddl=struct srcbucket2 { i32 key, string value}, transient_lastDdlTime=1515180530, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket2}, baseFileName=srcbucket2, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=4, serialization.ddl=struct srcbucket2 { i32 key, string value}, transient_lastDdlTime=1515180530, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket2}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=4, serialization.ddl=struct srcbucket2 { i32 key, string value}, transient_lastDdlTime=1515180530, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket2}, baseFileName=srcbucket2, vectorPartitionDesc=null]} 2018-01-05T11:28:50,390 DEBUG [TezTR-518813_1_14_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:50,390 INFO [TezTR-518813_1_14_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:50,390 DEBUG [TezTR-518813_1_14_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:50,391 DEBUG [TezTR-518813_1_14_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:50,391 INFO [TezTR-518813_1_14_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_136_7492829147931145598-1/djaiswal/_tez_scratch_dir/139e8cb9-4200-476d-ac85-88abc0cef61a/map.xml 2018-01-05T11:28:50,391 DEBUG [TezTR-518813_1_14_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:50,391 INFO [TezTR-518813_1_14_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_14_00_000000_0 2018-01-05T11:28:50,391 INFO [TezTR-518813_1_14_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:28:50,391 WARN [TezTR-518813_1_14_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000003_0 (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:28:50,391 DEBUG [TezTR-518813_1_14_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:28:50,391 WARN [TezTR-518813_1_14_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000003_0:0+1342 2018-01-05T11:28:50,392 DEBUG [TezTR-518813_1_14_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000003_0 2018-01-05T11:28:50,392 INFO [TezTR-518813_1_14_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_136_7492829147931145598-1/djaiswal/_tez_scratch_dir/139e8cb9-4200-476d-ac85-88abc0cef61a/map.xml 2018-01-05T11:28:50,392 DEBUG [TezTR-518813_1_14_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:50,392 DEBUG [TezTR-518813_1_14_0_0_0] exec.MapOperator: Processing alias(es) srcbucket2 for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000003_0 2018-01-05T11:28:50,392 DEBUG [TezTR-518813_1_14_0_0_0] log.PerfLogger: 2018-01-05T11:28:50,392 DEBUG [TezTR-518813_1_14_0_0_0] mr.ObjectCache: djaiswal_20180105112850_e39d2de5-06f7-4fe9-ad4b-81b9a060d703_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:28:50,392 DEBUG [TezTR-518813_1_14_0_0_0] exec.MapOperator: close called for operator MAP[0] 2018-01-05T11:28:50,392 INFO [TezTR-518813_1_14_0_0_0] exec.MapOperator: Closing operator MAP[0] 2018-01-05T11:28:50,392 INFO [TezTR-518813_1_14_0_0_0] exec.MapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:500, 2018-01-05T11:28:50,392 DEBUG [TezTR-518813_1_14_0_0_0] exec.MapOperator: Closing child = TS[0] 2018-01-05T11:28:50,392 DEBUG [TezTR-518813_1_14_0_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:28:50,392 DEBUG [TezTR-518813_1_14_0_0_0] exec.TableScanOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:50,392 INFO [TezTR-518813_1_14_0_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:28:50,392 DEBUG [TezTR-518813_1_14_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:50,392 DEBUG [TezTR-518813_1_14_0_0_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_136_7492829147931145598-1/-mr-10000 2018-01-05T11:28:50,392 INFO [TezTR-518813_1_14_0_0_0] exec.TableScanOperator: publishing : default.srcbucket2/ : {numRows=500, rawDataSize=5312} 2018-01-05T11:28:50,416 DEBUG [TezTR-518813_1_14_0_0_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_136_7492829147931145598-1/-mr-10000/tmpstats-0_0 2018-01-05T11:28:50,416 DEBUG [TezTR-518813_1_14_0_0_0] fs.FSStatsPublisher: Writing stats in it : {default.srcbucket2/={numRows=500, rawDataSize=5312}} 2018-01-05T11:28:50,416 DEBUG [TezTR-518813_1_14_0_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:28:50,416 DEBUG [TezTR-518813_1_14_0_0_0] exec.MapOperator: 0 Close done 2018-01-05T11:28:50,416 INFO [TezTR-518813_1_14_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_14_00_000000_0 2018-01-05T11:28:50,416 INFO [TezTR-518813_1_14_0_0_0] output.MROutput: out_Map 1 closed 2018-01-05T11:28:50,416 DEBUG [TezTR-518813_1_14_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:50,416 INFO [TezTR-518813_1_14_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:50,416 INFO [TezTR-518813_1_14_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_14_00_000000_0, askedToStop=false 2018-01-05T11:28:50,416 INFO [TezTR-518813_1_14_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_14_00_000000_0, stopRequested=false 2018-01-05T11:28:50,416 INFO [TezTR-518813_1_14_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_14_00_000000_0: Counters: 5 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=500, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=0][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500]] 2018-01-05T11:28:50,416 DEBUG [TezTR-518813_1_14_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_14_0_0_0 thread id: 995 scheme: file bytesRead: 5912 bytesWritten: 89 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:50,416 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_14_00_000000_0, isTaskDone=true 2018-01-05T11:28:50,417 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000027, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_14_00_000000_0, eventCount=2 } 2018-01-05T11:28:50,417 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_14][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_14_00_000000_0, creationTime=1515180530347, allocationTime=1515180530348, startTime=1515180530351, finishTime=1515180530417, timeTaken=66, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:50,418 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_14_00_000000_0 2018-01-05T11:28:50,418 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_14_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:50,418 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_14_00_000000_0; false 2018-01-05T11:28:50,418 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_14_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:50,418 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:50,418 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:50,418 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:50,418 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_14][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_14_00_000000, startTime=1515180530351, finishTime=1515180530418, timeTaken=67, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_14_00_000000_0, diagnostics=, counters=Counters: 11, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=5912, FILE_BYTES_WRITTEN=89, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=500, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500 2018-01-05T11:28:50,418 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_14_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:50,418 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_14_00_000000_0 2018-01-05T11:28:50,418 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000027=48 2018-01-05T11:28:50,418 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_14_00_000000_0: true 2018-01-05T11:28:50,419 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_14_00 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:50,419 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_14_00 [Map 1] 2018-01-05T11:28:50,419 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_14_00_000000_0 2018-01-05T11:28:50,419 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000027, QueryId=djaiswal_20180105112850_e39d2de5-06f7-4fe9-ad4b-81b9a060d703-14, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:28:50,419 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_14_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:50,419 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_14_00_000000_0 2018-01-05T11:28:50,419 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_14][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_14_00, initRequestedTime=1515180530275, initedTime=1515180530346, startRequestedTime=1515180530282, startedTime=1515180530346, finishTime=1515180530419, timeTaken=73, status=SUCCEEDED, diagnostics=, counters=Counters: 11, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=5912, FILE_BYTES_WRITTEN=89, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=500, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500, vertexStats=firstTaskStartTime=1515180530351, firstTasksToStart=[ task_1515180518813_0001_14_00_000000 ], lastTaskFinishTime=1515180530417, lastTasksToFinish=[ task_1515180518813_0001_14_00_000000 ], minTaskDuration=66, maxTaskDuration=66, avgTaskDuration=66.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_14_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_14_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:50,419 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_14_00 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:50,419 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_14][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000027, stoppedTime=1515180530419, exitStatus=0 2018-01-05T11:28:50,419 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000027 exited with diagnostics set to Container completed. 2018-01-05T11:28:50,419 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_14_00 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1 2018-01-05T11:28:50,419 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null 2018-01-05T11:28:50,419 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_14_00 [Map 1] 2018-01-05T11:28:50,420 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_14, queueSize=0 2018-01-05T11:28:50,420 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_14][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_14, startTime=1515180530275, finishTime=1515180530419, timeTaken=144, status=SUCCEEDED, diagnostics=, counters=Counters: 15, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, OTHER_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=496, AM_GC_TIME_MILLIS=0, File System Counters, FILE_BYTES_READ=5912, FILE_BYTES_WRITTEN=89, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=500, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500 2018-01-05T11:28:50,420 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_14 finished with state: SUCCEEDED 2018-01-05T11:28:50,420 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_14 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:28:50,420 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000027 2018-01-05T11:28:50,420 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_14, dagState=SUCCEEDED 2018-01-05T11:28:50,420 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_14_post 2018-01-05T11:28:50,420 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:50,420 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:28:50,420 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:28:50,420 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 14 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=1, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=1,NumRequestsWithoutLocation=1, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=1} 2018-01-05T11:28:50,420 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:28:50,420 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=ANALYZE TABLE srcbucket2 COMPUTE STATISTICS (Stage-0), with id=dag_1515180518813_0001_14 2018-01-05T11:28:50,420 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 14, to 10.22.8.182:49818 2018-01-05T11:28:50,420 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=1, Remaining=0 2018-01-05T11:28:50,420 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=ANALYZE TABLE srcbucket2 COMPUTE STATISTICS (Stage-0), with id=dag_1515180518813_0001_14 2018-01-05T11:28:50,421 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=14} 2018-01-05T11:28:50,421 INFO [IPC Server handler 0 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=14}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:28:50,421 INFO [IPC Server handler 0 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/14 after 300 seconds 2018-01-05T11:28:50,421 ERROR [IPC Server handler 0 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112850_e39d2de5-06f7-4fe9-ad4b-81b9a060d703, dag_1515180518813_0001_14 2018-01-05T11:28:50,421 INFO [IPC Server handler 0 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112850_e39d2de5-06f7-4fe9-ad4b-81b9a060d703 2018-01-05T11:28:50,421 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=14} = 0 2018-01-05T11:28:50,421 DEBUG [IPC Server handler 0 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=14} 2018-01-05T11:28:50,421 DEBUG [IPC Server handler 0 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:28:50,421 DEBUG [IPC Server handler 0 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=14} 2018-01-05T11:28:50,421 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,421 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,421 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 2018-01-05T11:28:50,421 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,422 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-2:STATS] in serial mode 2018-01-05T11:28:50,422 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@1a445b5, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:50,422 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:50,422 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,422 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:50,422 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:50,422 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:50,422 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:50,422 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,422 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:50,422 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:50,423 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_136_7492829147931145598-1/-mr-10000 2018-01-05T11:28:50,423 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_136_7492829147931145598-1/-mr-10000 2018-01-05T11:28:50,424 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.srcbucket2/ numRows 2018-01-05T11:28:50,424 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.srcbucket2/ numRows 500 2018-01-05T11:28:50,424 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.srcbucket2/ rawDataSize 2018-01-05T11:28:50,424 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.srcbucket2/ rawDataSize 5312 2018-01-05T11:28:50,425 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,425 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:50,425 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:50,425 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:50,425 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:50,426 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,426 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,426 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,427 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,428 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,428 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:50,428 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:50,428 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,428 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:50,428 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,428 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=srcbucket2 newtbl=srcbucket2 2018-01-05T11:28:50,428 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=srcbucket2 newtbl=srcbucket2 2018-01-05T11:28:50,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,443 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,443 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.srcbucket2 stats: [numFiles=4, numRows=500, totalSize=5812, rawDataSize=5312] 2018-01-05T11:28:50,443 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: About to delete stats tmp dir :file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_136_7492829147931145598-1/-mr-10000 2018-01-05T11:28:50,443 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,443 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,443 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: ANALYZE TABLE srcbucket2 COMPUTE STATISTICS 2018-01-05T11:28:50,443 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:28:50,443 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@srcbucket2 2018-01-05T11:28:50,443 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@srcbucket2 2018-01-05T11:28:50,443 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,443 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,443 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:50,444 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=15} 2018-01-05T11:28:50,444 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112850_e39d2de5-06f7-4fe9-ad4b-81b9a060d703); Time taken: 0.271 seconds 2018-01-05T11:28:50,444 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:50,444 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,444 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:50,444 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,444 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query ANALYZE TABLE srcbucket2 COMPUTE STATISTICS 2018-01-05T11:28:50,444 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_136_7492829147931145598-1 2018-01-05T11:28:50,445 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.308 seconds 2018-01-05T11:28:50,445 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:50,445 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:50,445 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:50,445 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:50,445 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,445 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:50,445 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,445 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,445 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,445 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: ANALYZE TABLE srcbucket2 COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:50,445 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112850_4efbe87f-26c9-4872-b247-f9c909a06058): ANALYZE TABLE srcbucket2 COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:50,446 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,446 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: ANALYZE TABLE srcbucket2 COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:50,446 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:50,446 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,446 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,446 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,446 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,446 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:50,446 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,446 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket2 2018-01-05T11:28:50,446 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket2 2018-01-05T11:28:50,451 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,452 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,452 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: select compute_stats(`key`, 'hll') , compute_stats(`value`, 'hll') from `default`.`srcbucket2` 2018-01-05T11:28:50,452 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: select compute_stats(`key`, 'hll') , compute_stats(`value`, 'hll') from `default`.`srcbucket2` 2018-01-05T11:28:50,453 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:50,453 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Invoking analyze on rewritten query 2018-01-05T11:28:50,453 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Starting Semantic Analysis 2018-01-05T11:28:50,453 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:50,454 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:50,454 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed phase 1 of Semantic Analysis 2018-01-05T11:28:50,454 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for source tables 2018-01-05T11:28:50,454 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,454 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket2 2018-01-05T11:28:50,454 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket2 2018-01-05T11:28:50,459 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,460 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,460 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for subqueries 2018-01-05T11:28:50,460 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for destination tables 2018-01-05T11:28:50,474 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed getting MetaData in Semantic Analysis 2018-01-05T11:28:50,474 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_452_2329691818928553056-1/-mr-10001 2018-01-05T11:28:50,474 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Table Plan for srcbucket2 TS[0] 2018-01-05T11:28:50,474 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:50,475 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:50,476 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: RR before GB srcbucket2{(key,key: int)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)(RAW__DATA__SIZE,RAW__DATA__SIZE: bigint)} after GB {((tok_function compute_stats (tok_table_or_col key) 'hll'),_col0: struct)((tok_function compute_stats (tok_table_or_col value) 'hll'),_col1: struct)} 2018-01-05T11:28:50,476 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: tree: (tok_select (tok_selexpr (tok_function compute_stats (tok_table_or_col key) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col value) 'hll'))) 2018-01-05T11:28:50,476 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: genSelectPlan: input = {((tok_function compute_stats (tok_table_or_col key) 'hll'),_col0: struct)((tok_function compute_stats (tok_table_or_col value) 'hll'),_col1: struct)} starRr = srcbucket2{(key,key: int)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)(RAW__DATA__SIZE,RAW__DATA__SIZE: bigint)} 2018-01-05T11:28:50,476 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Select Plan row schema: null{(_c0,_col0: struct)(_c1,_col1: struct)} 2018-01-05T11:28:50,476 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Select Plan for clause: insclause-0 2018-01-05T11:28:50,476 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_452_2329691818928553056-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-50_452_2329691818928553056-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_452_2329691818928553056-1/-mr-10000 2018-01-05T11:28:50,476 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_452_2329691818928553056-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-50_452_2329691818928553056-1 2018-01-05T11:28:50,507 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_452_2329691818928553056-1/-mr-10003 2018-01-05T11:28:50,507 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_452_2329691818928553056-1/-mr-10000 row schema: null{(_c0,_col0: struct)(_c1,_col1: struct)} 2018-01-05T11:28:50,507 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Body Plan for Query Block null 2018-01-05T11:28:50,507 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Plan for Query Block null 2018-01-05T11:28:50,507 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Before logical optimization TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-SEL[5]-FS[6] 2018-01-05T11:28:50,507 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,507 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,507 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,508 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] lineage.Generator: Time taken for lineage transform=1 2018-01-05T11:28:50,508 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,508 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,508 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,508 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,508 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,508 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,508 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,508 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: int|{srcbucket2}key,value: string|{srcbucket2}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{srcbucket2}block__offset__inside__file,INPUT__FILE__NAME: string|{srcbucket2}input__file__name,ROW__ID: struct|{srcbucket2}row__id,RAW__DATA__SIZE: bigint|{srcbucket2}raw__data__size) 2018-01-05T11:28:50,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: int|{srcbucket2}key,value: string|{srcbucket2}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{srcbucket2}block__offset__inside__file,INPUT__FILE__NAME: string|{srcbucket2}input__file__name,ROW__ID: struct|{srcbucket2}row__id,RAW__DATA__SIZE: bigint|{srcbucket2}raw__data__size) 2018-01-05T11:28:50,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:50,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:50,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:28:50,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:50,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:50,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:50,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:50,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:50,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:50,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:50,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:50,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:50,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1) 2018-01-05T11:28:50,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:50,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:50,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0] Column[_col1]) 2018-01-05T11:28:50,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{}) 2018-01-05T11:28:50,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:50,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:50,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,509 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for FS(6) 2018-01-05T11:28:50,509 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(5) 2018-01-05T11:28:50,509 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for GBY(4) 2018-01-05T11:28:50,509 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for RS(3) 2018-01-05T11:28:50,509 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for GBY(2) 2018-01-05T11:28:50,509 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(1) 2018-01-05T11:28:50,509 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:28:50,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.PredicatePushDown: After PPD: TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-SEL[5]-FS[6] 2018-01-05T11:28:50,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: int|{srcbucket2}key,value: string|{srcbucket2}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{srcbucket2}block__offset__inside__file,INPUT__FILE__NAME: string|{srcbucket2}input__file__name,ROW__ID: struct|{srcbucket2}row__id,RAW__DATA__SIZE: bigint|{srcbucket2}raw__data__size) 2018-01-05T11:28:50,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: int|{srcbucket2}key,value: string|{srcbucket2}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{srcbucket2}block__offset__inside__file,INPUT__FILE__NAME: string|{srcbucket2}input__file__name,ROW__ID: struct|{srcbucket2}row__id,RAW__DATA__SIZE: bigint|{srcbucket2}raw__data__size) 2018-01-05T11:28:50,510 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:50,510 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:50,512 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:28:50,513 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:50,513 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:50,513 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:50,513 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:50,513 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:50,513 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:50,513 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:50,513 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:50,513 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:50,513 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1) 2018-01-05T11:28:50,513 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:50,513 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:50,513 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0] Column[_col1]) 2018-01-05T11:28:50,513 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{}) 2018-01-05T11:28:50,513 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:50,513 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:50,513 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,513 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,513 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,513 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,514 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,514 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,514 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,514 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,514 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: int|{srcbucket2}key,value: string|{srcbucket2}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{srcbucket2}block__offset__inside__file,INPUT__FILE__NAME: string|{srcbucket2}input__file__name,ROW__ID: struct|{srcbucket2}row__id,RAW__DATA__SIZE: bigint|{srcbucket2}raw__data__size) 2018-01-05T11:28:50,514 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: int|{srcbucket2}key,value: string|{srcbucket2}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{srcbucket2}block__offset__inside__file,INPUT__FILE__NAME: string|{srcbucket2}input__file__name,ROW__ID: struct|{srcbucket2}row__id,RAW__DATA__SIZE: bigint|{srcbucket2}raw__data__size) 2018-01-05T11:28:50,514 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:50,514 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:50,514 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:28:50,514 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:50,514 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:50,514 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:50,514 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:50,514 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:50,514 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:50,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:50,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:50,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:50,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1) 2018-01-05T11:28:50,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:50,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:50,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0] Column[_col1]) 2018-01-05T11:28:50,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{}) 2018-01-05T11:28:50,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:50,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:50,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 3 key:[] 2018-01-05T11:28:50,515 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 3 oldColExprMap: {VALUE._col0=Column[_col0], VALUE._col1=Column[_col1]} 2018-01-05T11:28:50,515 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 3 newColExprMap: {VALUE._col0=Column[_col0], VALUE._col1=Column[_col1]} 2018-01-05T11:28:50,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.IdentityProjectRemover: Identity project remover optimization removed : SEL[5] 2018-01-05T11:28:50,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: After logical optimization TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-FS[6] 2018-01-05T11:28:50,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,516 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=srcbucket2 2018-01-05T11:28:50,516 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=srcbucket2 2018-01-05T11:28:50,520 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000027, QueryId=djaiswal_20180105112850_e39d2de5-06f7-4fe9-ad4b-81b9a060d703-14, DagId=14, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180530370, EndTime=1515180530520 2018-01-05T11:28:50,521 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.279893ms + 0.008811ms, the query is [select "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "TAB_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (...)] 2018-01-05T11:28:50,522 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,529 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,530 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,530 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting stats (Num rows: 500 Data size: 47500 Basic stats: COMPLETE Column stats: COMPLETE) on TS[0] 2018-01-05T11:28:50,530 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (srcbucket2): numRows: 500 dataSize: 47500 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {value= colName: value colType: string countDistincts: 309 numNulls: 0 avgColLen: 6.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, key= colName: key colType: int countDistincts: 289 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:50,530 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting stats (Num rows: 500 Data size: 47500 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[1] 2018-01-05T11:28:50,530 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-SEL[1]: numRows: 500 dataSize: 47500 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {ROW__ID= colName: ROW__ID colType: struct countDistincts: 500 numNulls: 0 avgColLen: 76.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, RAW__DATA__SIZE= colName: RAW__DATA__SIZE colType: bigint countDistincts: 500 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, BLOCK__OFFSET__INSIDE__FILE= colName: BLOCK__OFFSET__INSIDE__FILE colType: bigint countDistincts: 500 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, INPUT__FILE__NAME= colName: INPUT__FILE__NAME colType: string countDistincts: 500 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, value= colName: value colType: string countDistincts: 309 numNulls: 0 avgColLen: 6.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, key= colName: key colType: int countDistincts: 289 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:50,530 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[2]: inputSize: 47500 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:28:50,530 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[2] hashAgg: true 2018-01-05T11:28:50,530 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [Case 3] STATS-GBY[2]: cardinality: 1 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 864 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[2] 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-GBY[2]: numRows: 1 dataSize: 864 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 864 Basic stats: COMPLETE Column stats: COMPLETE) on RS[3] 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-RS[3]: numRows: 1 dataSize: 864 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {VALUE._col0= colName: VALUE._col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col1= colName: VALUE._col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[4]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[4] hashAgg: false 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [Case 9] STATS-GBY[4]: cardinality: 1 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 880 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[4] 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-GBY[4]: numRows: 1 dataSize: 880 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-FS[6]: numRows: 1 dataSize: 880 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[]]; bucket count: 4 }) on TS[0] 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[]]; bucket count: 4 }) on SEL[1] 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[2] 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[3] 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[4] 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FileSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on FS[6] 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,531 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: FS, 6 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: GBY, 4 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: RS, 3 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: GBY, 2 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: SEL, 1 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:28:50,531 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Cycle free: true 2018-01-05T11:28:50,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,532 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,532 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,532 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: int|{srcbucket2}key,value: string|{srcbucket2}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{srcbucket2}block__offset__inside__file,INPUT__FILE__NAME: string|{srcbucket2}input__file__name,ROW__ID: struct|{srcbucket2}row__id,RAW__DATA__SIZE: bigint|{srcbucket2}raw__data__size) 2018-01-05T11:28:50,532 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: int|{srcbucket2}key,value: string|{srcbucket2}value) 2018-01-05T11:28:50,532 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:50,532 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:50,532 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value]) 2018-01-05T11:28:50,532 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:50,532 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:50,532 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:50,532 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:50,532 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:50,532 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:50,532 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:50,532 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:50,532 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:50,532 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{}) 2018-01-05T11:28:50,532 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:50,532 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:50,532 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,532 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: TS[0] 2018-01-05T11:28:50,532 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: RS[3] 2018-01-05T11:28:50,532 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:28:50,532 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2 of table srcbucket2 2018-01-05T11:28:50,532 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2 2018-01-05T11:28:50,532 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: RS[3] 2018-01-05T11:28:50,532 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: GBY[4] 2018-01-05T11:28:50,532 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: FS[6] 2018-01-05T11:28:50,532 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding reduce work (Reducer 2) for GBY[4] 2018-01-05T11:28:50,532 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Setting up reduce sink: RS[3] with following reduce work: Reducer 2 2018-01-05T11:28:50,532 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Removing RS[3] as parent from GBY[4] 2018-01-05T11:28:50,533 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: FS[6] 2018-01-05T11:28:50,533 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:28:50,533 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,533 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,533 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:50,533 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:50,533 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:50,533 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:50,533 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:50,533 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:50,533 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:28:50,533 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping vectorization 2018-01-05T11:28:50,533 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:28:50,533 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: llap mode: only 2018-01-05T11:28:50,533 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:28:50,533 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(key, 'hll')' 2018-01-05T11:28:50,533 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'key' 2018-01-05T11:28:50,533 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:50,533 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(value, 'hll')' 2018-01-05T11:28:50,533 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'value' 2018-01-05T11:28:50,533 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:50,533 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'key' 2018-01-05T11:28:50,533 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'value' 2018-01-05T11:28:50,533 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:50,533 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@2d05291f can be done in LLAP 2018-01-05T11:28:50,533 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Reducer 2 2018-01-05T11:28:50,533 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col0)' 2018-01-05T11:28:50,533 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:28:50,533 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col1)' 2018-01-05T11:28:50,533 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col1' 2018-01-05T11:28:50,533 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:50,533 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@2f9739e0 can be done in LLAP 2018-01-05T11:28:50,533 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,534 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed plan generation 2018-01-05T11:28:50,534 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:50,534 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validation start 2018-01-05T11:28:50,534 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Not a partition. 2018-01-05T11:28:50,534 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:28:50,534 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,534 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0, type:struct, comment:null), FieldSchema(name:_c1, type:struct, comment:null)], properties:null) 2018-01-05T11:28:50,534 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,534 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:50,534 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=12, flushCache_()=0, getTableColumnStatistics_(String, String, List, )=14} 2018-01-05T11:28:50,534 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112850_4efbe87f-26c9-4872-b247-f9c909a06058); Time taken: 0.089 seconds 2018-01-05T11:28:50,534 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:50,534 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,534 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112850_4efbe87f-26c9-4872-b247-f9c909a06058): ANALYZE TABLE srcbucket2 COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:50,535 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,535 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: ANALYZE TABLE srcbucket2 COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:50,535 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:28:50,535 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@srcbucket2 2018-01-05T11:28:50,535 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@srcbucket2 2018-01-05T11:28:50,535 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_452_2329691818928553056-1/-mr-10000 2018-01-05T11:28:50,535 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,535 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,535 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,535 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Query ID = djaiswal_20180105112850_4efbe87f-26c9-4872-b247-f9c909a06058 2018-01-05T11:28:50,535 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Total jobs = 1 2018-01-05T11:28:50,535 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,536 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:28:50,536 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MAPRED] in serial mode 2018-01-05T11:28:50,546 DEBUG [AMReporterQueueDrainer] impl.AMReporter: Removing am 10.22.8.182:49824 with last associated dag QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=1} from heartbeat with taskCount=0, amFailed=false, isDone=true 2018-01-05T11:28:50,566 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_445_7459849956253588393-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:28:50,588 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:28:50,588 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:28:50,588 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:28:50,588 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112850_4efbe87f-26c9-4872-b247-f9c909a06058 2018-01-05T11:28:50,588 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Session is already open 2018-01-05T11:28:50,588 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:28:50,588 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,588 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Dag name: ANALYZE TABLE srcbucket2 COMPUTE...key,value (Stage-0) 2018-01-05T11:28:50,588 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: DagInfo: {"context":"Hive","description":"\n\nANALYZE TABLE srcbucket2 COMPUTE STATISTICS FOR COLUMNS key,value"} 2018-01-05T11:28:50,588 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112850_4efbe87f-26c9-4872-b247-f9c909a06058 with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:28:50,588 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,599 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,599 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:28:50,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,600 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 2 size: 2.05KB 2018-01-05T11:28:50,618 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,618 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,630 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:28:50,630 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,630 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:28:50,631 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,631 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 3.55KB 2018-01-05T11:28:50,638 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_452_2329691818928553056-1/-mr-10001 2018-01-05T11:28:50,652 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_452_2329691818928553056-1/-mr-10001 2018-01-05T11:28:50,653 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2 2018-01-05T11:28:50,653 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,657 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,657 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,657 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=ANALYZE TABLE srcbucket2 COMPUTE...key,value (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112850_4efbe87f-26c9-4872-b247-f9c909a06058 } 2018-01-05T11:28:50,658 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:50,658 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:50,660 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: ANALYZE TABLE srcbucket2 COMPUTE...key,value (Stage-0) 2018-01-05T11:28:50,660 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:28:50,660 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_15, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_15.dot 2018-01-05T11:28:50,661 INFO [IPC Server handler 1 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_15 2018-01-05T11:28:50,661 WARN [IPC Server handler 1 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:50,661 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Running DAG: ANALYZE TABLE srcbucket2 COMPUTE...key,value (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112850_4efbe87f-26c9-4872-b247-f9c909a06058 } 2018-01-05T11:28:50,689 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_15][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_15, submitTime=1515180530660, queueName=null 2018-01-05T11:28:50,690 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_15_00 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_15_00 [Map 1], Default Resources= 2018-01-05T11:28:50,690 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_15_00 [Map 1] 2018-01-05T11:28:50,690 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:28:50,690 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_15_00 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:50,690 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_15_01 [Reducer 2]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_15_01 [Reducer 2], Default Resources= 2018-01-05T11:28:50,690 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_15_01 [Reducer 2] 2018-01-05T11:28:50,690 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 2 2018-01-05T11:28:50,690 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_15_01 [Reducer 2] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:50,690 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:28:50,691 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_15][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_15, initTime=1515180530691 2018-01-05T11:28:50,691 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: dag_1515180518813_0001_15 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:28:50,691 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:28:50,691 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_15][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_15, startTime=1515180530691 2018-01-05T11:28:50,691 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_15 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:28:50,691 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {srcbucket2={InputName=srcbucket2}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:28:50,691 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: srcbucket2, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:28:50,691 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_15_00 [Map 1] 2018-01-05T11:28:50,691 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_15, dagName=ANALYZE TABLE srcbucket2 COMPUTE...key,value (Stage-0) 2018-01-05T11:28:50,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,696 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_15_00 [Map 1] 2018-01-05T11:28:50,696 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_15_00 [Map 1] 2018-01-05T11:28:50,696 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:50,696 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_15_00 [Map 1] 2018-01-05T11:28:50,698 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:28:50,698 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@3a486e8a 2018-01-05T11:28:50,698 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_445_7459849956253588393-1/djaiswal/_tez_scratch_dir/05e6516a-f89b-4869-a585-686186680d76/map.xml 2018-01-05T11:28:50,698 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_445_7459849956253588393-1/djaiswal/_tez_scratch_dir/05e6516a-f89b-4869-a585-686186680d76/map.xml 2018-01-05T11:28:50,698 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_445_7459849956253588393-1/djaiswal/_tez_scratch_dir/05e6516a-f89b-4869-a585-686186680d76/map.xml 2018-01-05T11:28:50,698 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:50,698 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:50,698 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:50,699 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:50,699 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.55KB 2018-01-05T11:28:50,699 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_15_00 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:28:50,700 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: srcbucket2 on vertex vertex_1515180518813_0001_15_00 [Map 1] 2018-01-05T11:28:50,700 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to ShuffleVertexManager for vertex_1515180518813_0001_15_01 [Reducer 2] 2018-01-05T11:28:50,700 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:28:50,700 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:50,700 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:28:50,700 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:50,700 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_445_7459849956253588393-1/djaiswal/_tez_scratch_dir/05e6516a-f89b-4869-a585-686186680d76/map.xml 2018-01-05T11:28:50,700 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:50,700 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias srcbucket2 2018-01-05T11:28:50,700 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2 2018-01-05T11:28:50,700 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2 2018-01-05T11:28:50,700 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 0,1 2018-01-05T11:28:50,700 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = key,value 2018-01-05T11:28:50,700 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2 2018-01-05T11:28:50,701 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:50,701 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:50,701 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:50,701 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:50,704 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.05 maxFrac: 0.75 auto: false desiredTaskIput: 104857600 2018-01-05T11:28:50,704 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:28:50,704 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 1 tasks for vertex: vertex_1515180518813_0001_15_01 [Reducer 2] 2018-01-05T11:28:50,704 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_15_01 [Reducer 2] 2018-01-05T11:28:50,704 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_15][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_15_01, reconfigureDoneTime=1515180530704, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:28:50,704 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_15_01 [Reducer 2], numAdditionalOutputs=1 2018-01-05T11:28:50,705 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_15][Event:VERTEX_INITIALIZED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_15_01, initRequestedTime=1515180530699, initedTime=1515180530704, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:50,705 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_15_01 [Reducer 2] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:28:50,752 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Time taken to get FileStatuses: 51 2018-01-05T11:28:50,752 INFO [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total input files to process : 4 2018-01-05T11:28:50,752 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total # of splits generated by getSplits: 4, TimeTaken: 51 2018-01-05T11:28:50,752 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 4 2018-01-05T11:28:50,752 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:50,752 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 4. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:28:50,752 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_445_7459849956253588393-1/djaiswal/_tez_scratch_dir/05e6516a-f89b-4869-a585-686186680d76/map.xml 2018-01-05T11:28:50,752 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_445_7459849956253588393-1/djaiswal/_tez_scratch_dir/05e6516a-f89b-4869-a585-686186680d76/map.xml 2018-01-05T11:28:50,752 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_445_7459849956253588393-1/djaiswal/_tez_scratch_dir/05e6516a-f89b-4869-a585-686186680d76/map.xml 2018-01-05T11:28:50,752 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:50,752 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:50,752 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:50,753 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:50,753 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.55KB 2018-01-05T11:28:50,753 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000001_0 to src new group? true 2018-01-05T11:28:50,753 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000002_0 to src new group? false 2018-01-05T11:28:50,753 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000000_0 to src new group? false 2018-01-05T11:28:50,753 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000003_0 to src new group? false 2018-01-05T11:28:50,753 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:28:50,754 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:28:50,754 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:28:50,754 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 107 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 5812 Original splits: 4 2018-01-05T11:28:50,754 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired numSplits: 1 lengthPerGroup: 5812 numLocations: 1 numSplitsPerLocation: 4 numSplitsInGroup: 4 totalLength: 5812 numOriginalSplits: 4 . Grouping by length: true count: false nodeLocalOnly: false 2018-01-05T11:28:50,754 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Number of splits desired: 1 created: 1 splitsProcessed: 4 2018-01-05T11:28:50,754 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 4 grouped split count is 1, for bucket: 1 2018-01-05T11:28:50,754 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:28:50,754 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: srcbucket2 on vertex vertex_1515180518813_0001_15_00 [Map 1] 2018-01-05T11:28:50,755 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_15_00 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:50,755 INFO [App Shared Pool - #0] impl.VertexImpl: Got updated RootInputsSpecs: {srcbucket2=forAllWorkUnits=true, update=[1]} 2018-01-05T11:28:50,755 INFO [App Shared Pool - #0] impl.VertexImpl: Vertex vertex_1515180518813_0001_15_00 [Map 1] parallelism set to 1 2018-01-05T11:28:50,755 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_15_00 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:50,755 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_15][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_15_00, initRequestedTime=1515180530691, initedTime=1515180530755, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:50,755 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_15_00 [Map 1] with distanceFromRoot: 0 2018-01-05T11:28:50,755 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_15_00 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:28:50,755 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_15][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_15_00, startRequestedTime=1515180530699, startedTime=1515180530755 2018-01-05T11:28:50,755 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_15][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_15_00, reconfigureDoneTime=1515180530755, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:28:50,755 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_15_00 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:50,755 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_15_00 for vertex: vertex_1515180518813_0001_15_01 [Reducer 2] numStartedSources: 1 numSources: 1 2018-01-05T11:28:50,755 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_15_01 [Reducer 2] with distanceFromRoot: 1 2018-01-05T11:28:50,755 INFO [App Shared Pool - #0] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:28:50,755 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_15][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_15_01, startRequestedTime=1515180530755, startedTime=1515180530755 2018-01-05T11:28:50,755 INFO [App Shared Pool - #0] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:28:50,755 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_15_01 [Reducer 2] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:50,755 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_15_00 [Map 1] 2018-01-05T11:28:50,756 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: OnVertexStarted vertex: Reducer 2 with 0 source tasks and 1 pending tasks 2018-01-05T11:28:50,756 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_15][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_15_00_000000, scheduledTime=1515180530756, launchTime=1515180530756 2018-01-05T11:28:50,756 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Map 1 in vertex: Reducer 2 numBipartiteSourceTasks: 1 2018-01-05T11:28:50,756 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_15_00 [Map 1] 2018-01-05T11:28:50,757 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_15_00_000000_0, priority=5, capability=, hosts=null 2018-01-05T11:28:50,757 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=5,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:50,757 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=5,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:50,757 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:50,757 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_15_00_000000_0 on hosts=null 2018-01-05T11:28:50,757 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_15_00_000000_0 on hosts=null 2018-01-05T11:28:50,757 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_15_00_000000_0 2018-01-05T11:28:50,757 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:50,757 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #28, task=TaskInfo{task=attempt_1515180518813_0001_15_00_000000_0, priority=5, startTime=0, containerId=null, uniqueId=27, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000028 2018-01-05T11:28:50,757 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_15_00_000000_0; false 2018-01-05T11:28:50,757 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_15_00_000000_0, priority=5, startTime=311551266, containerId=container_111101111_0001_00_000028assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=27, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:50,757 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000028 for node: 10.22.8.182:49818 2018-01-05T11:28:50,759 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 15, name=ANALYZE TABLE srcbucket2 COMPUTE...key,value (Stage-0), queryId=djaiswal_20180105112850_4efbe87f-26c9-4872-b247-f9c909a06058 2018-01-05T11:28:50,759 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000028, attempt_1515180518813_0001_15_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:50,759 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000028 for node: 10.22.8.182:49818 2018-01-05T11:28:50,759 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000028 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:50,759 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000028, launchTime=1515180530759 2018-01-05T11:28:50,760 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_15_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000028] on NM: [10.22.8.182:49818] 2018-01-05T11:28:50,760 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:50,760 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_15_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112850_4efbe87f-26c9-4872-b247-f9c909a06058, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000028, dagName=ANALYZE TABLE srcbucket2 COMPUTE...key,value (Stage-0), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{srcbucket2,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{Reducer 2,org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput,1}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180530691, firstAttemptStartTime=1515180530756, currentAttemptStartTime=1515180530759} 2018-01-05T11:28:50,760 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:50,760 INFO [IPC Server handler 2 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000028, QueryId=djaiswal_20180105112850_4efbe87f-26c9-4872-b247-f9c909a06058-15, DagId=15, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180530760 2018-01-05T11:28:50,760 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_15][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_15_00_000000_0, startTime=1515180530759, containerId=container_111101111_0001_00_000028, nodeId=10.22.8.182:49818 2018-01-05T11:28:50,760 DEBUG [IPC Server handler 2 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=15} with the ShuffleHandler 2018-01-05T11:28:50,771 INFO [IPC Server handler 2 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:28:50,771 DEBUG [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/15] 2018-01-05T11:28:50,771 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:50,771 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_15_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:50,771 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_15_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180530756, dagStartTime=1515180530691, withinDagPriority=5, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:28:50,771 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_15_00_000000_0 : ACCEPTED 2018-01-05T11:28:50,771 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_15_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:50,771 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_15_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180530756, dagStartTime=1515180530691, withinDagPriority=5, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:28:50,772 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_15_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:50,772 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_15_00_000000_0 to pre-emption queue 2018-01-05T11:28:50,772 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_15_00_000000_0: true 2018-01-05T11:28:50,772 INFO [TaskCommunicator # 0] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_15_00_000000_0 2018-01-05T11:28:50,773 INFO [TaskCommunicator # 0] GuaranteedTasks: Received response for attempt_1515180518813_0001_15_00_000000_0, true 2018-01-05T11:28:50,773 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:653102606 2018-01-05T11:28:50,773 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : ANALYZE TABLE srcbucket2 COMPUTE...key,value (Stage-0), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_15_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=srcbucket2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=Reducer 2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput }}, ] 2018-01-05T11:28:50,773 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1039138816, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:50,773 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_15_00_000000_0 vertexName: Map 1 2018-01-05T11:28:50,774 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000028, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_15_00_000000_0, eventCount=1 } 2018-01-05T11:28:50,774 DEBUG [TezTR-518813_1_15_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_15 queryId: djaiswal_20180105112850_4efbe87f-26c9-4872-b247-f9c909a06058 instanceType: TaskRunner2Callable 2018-01-05T11:28:50,774 INFO [TezTR-518813_1_15_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_15_00_000000_0 2018-01-05T11:28:50,774 DEBUG [TezTR-518813_1_15_0_0_0] log.PerfLogger: 2018-01-05T11:28:50,775 INFO [I/O Setup 1 Initialize: {Reducer 2}] writers.UnorderedPartitionedKVWriter: Requested BufferSize (tez.runtime.unordered.output.buffer.size-mb) : 100 2018-01-05T11:28:50,775 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:50,776 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:50,778 INFO [IPC Server handler 4 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_15_00_000000_0 sent events: (0-1). 2018-01-05T11:28:50,778 DEBUG [TezTR-518813_1_15_0_0_0] log.PerfLogger: 2018-01-05T11:28:50,778 INFO [TezTR-518813_1_15_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:50,778 INFO [TezTR-518813_1_15_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:50,778 INFO [TezTR-518813_1_15_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:50,779 INFO [I/O Setup 0 Initialize: {srcbucket2}] input.MRInput: srcbucket2 using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:28:50,779 INFO [I/O Setup 0 Initialize: {srcbucket2}] input.MRInputLegacy: srcbucket2 MRInputLegacy deferring initialization 2018-01-05T11:28:50,779 INFO [TezTR-518813_1_15_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:50,779 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:28:50,779 INFO [TezTR-518813_1_15_0_0_0] resources.MemoryDistributor: InitialRequests=[srcbucket2:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [Reducer 2:OUTPUT:104857600:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput] 2018-01-05T11:28:50,779 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_15_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:28:50,779 INFO [TezTR-518813_1_15_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:50,779 INFO [TezTR-518813_1_15_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:50,779 INFO [TezTR-518813_1_15_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 104857600, TotalRequestedScaled: 5.24288E7, TotalJVMHeap: 1039138816, TotalAvailable: 696223006, TotalRequested/TotalJVMHeap:0.10 2018-01-05T11:28:50,779 INFO [TezTR-518813_1_15_0_0_0] resources.MemoryDistributor: Allocations=[srcbucket2:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput:OUTPUT:104857600:104857600] 2018-01-05T11:28:50,779 INFO [TezTR-518813_1_15_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:50,779 INFO [TezTR-518813_1_15_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: srcbucket2 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:28:50,779 INFO [TezTR-518813_1_15_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:28:50,779 INFO [TezTR-518813_1_15_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:28:50,779 INFO [I/O Setup 1 Start: {srcbucket2}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: srcbucket2 2018-01-05T11:28:50,779 INFO [TezTR-518813_1_15_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:50,779 INFO [TezTR-518813_1_15_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_15_00_000000_0 2018-01-05T11:28:50,779 DEBUG [TezTR-518813_1_15_0_0_0] log.PerfLogger: 2018-01-05T11:28:50,779 DEBUG [TezTR-518813_1_15_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_15_00_000000_0_10082 2018-01-05T11:28:50,779 DEBUG [TezTR-518813_1_15_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:50,779 DEBUG [TezTR-518813_1_15_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_15_00_000000_0 2018-01-05T11:28:50,779 INFO [TezTR-518813_1_15_0_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112850_4efbe87f-26c9-4872-b247-f9c909a06058 2018-01-05T11:28:50,779 DEBUG [TezTR-518813_1_15_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:50,779 DEBUG [TezTR-518813_1_15_0_0_0] log.PerfLogger: 2018-01-05T11:28:50,779 DEBUG [TezTR-518813_1_15_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:50,779 DEBUG [TezTR-518813_1_15_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:50,779 DEBUG [TezTR-518813_1_15_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112850_4efbe87f-26c9-4872-b247-f9c909a06058_Map 1__MAP_PLAN__ 2018-01-05T11:28:50,779 INFO [TezTR-518813_1_15_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_445_7459849956253588393-1/djaiswal/_tez_scratch_dir/05e6516a-f89b-4869-a585-686186680d76/map.xml 2018-01-05T11:28:50,780 DEBUG [TezTR-518813_1_15_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_445_7459849956253588393-1/djaiswal/_tez_scratch_dir/05e6516a-f89b-4869-a585-686186680d76/map.xml 2018-01-05T11:28:50,780 DEBUG [TezTR-518813_1_15_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_445_7459849956253588393-1/djaiswal/_tez_scratch_dir/05e6516a-f89b-4869-a585-686186680d76/map.xml 2018-01-05T11:28:50,780 DEBUG [TezTR-518813_1_15_0_0_0] log.PerfLogger: 2018-01-05T11:28:50,780 INFO [TezTR-518813_1_15_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:50,780 DEBUG [TezTR-518813_1_15_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:50,781 DEBUG [TezTR-518813_1_15_0_0_0] log.PerfLogger: 2018-01-05T11:28:50,781 INFO [TezTR-518813_1_15_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.55KB 2018-01-05T11:28:50,781 DEBUG [TezTR-518813_1_15_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:50,781 INFO [TezTR-518813_1_15_0_0_0] tez.MapRecordProcessor: The input names are: [srcbucket2] 2018-01-05T11:28:50,782 INFO [TezTR-518813_1_15_0_0_0] input.MRInput: Initialized MRInput: srcbucket2 2018-01-05T11:28:50,782 DEBUG [TezTR-518813_1_15_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 4 2018-01-05T11:28:50,782 INFO [TezTR-518813_1_15_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_445_7459849956253588393-1/djaiswal/_tez_scratch_dir/05e6516a-f89b-4869-a585-686186680d76/map.xml 2018-01-05T11:28:50,782 DEBUG [TezTR-518813_1_15_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:50,782 DEBUG [TezTR-518813_1_15_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000001_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=4, serialization.ddl=struct srcbucket2 { i32 key, string value}, transient_lastDdlTime=1515180530, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket2}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=4, serialization.ddl=struct srcbucket2 { i32 key, string value}, transient_lastDdlTime=1515180530, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket2}, baseFileName=srcbucket2, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=4, serialization.ddl=struct srcbucket2 { i32 key, string value}, transient_lastDdlTime=1515180530, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket2}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=4, serialization.ddl=struct srcbucket2 { i32 key, string value}, transient_lastDdlTime=1515180530, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket2}, baseFileName=srcbucket2, vectorPartitionDesc=null]} 2018-01-05T11:28:50,782 DEBUG [TezTR-518813_1_15_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:50,782 INFO [TezTR-518813_1_15_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:50,782 DEBUG [TezTR-518813_1_15_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:50,783 DEBUG [TezTR-518813_1_15_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:50,783 INFO [TezTR-518813_1_15_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_445_7459849956253588393-1/djaiswal/_tez_scratch_dir/05e6516a-f89b-4869-a585-686186680d76/map.xml 2018-01-05T11:28:50,783 DEBUG [TezTR-518813_1_15_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:50,783 INFO [TezTR-518813_1_15_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_15_00_000000_0 2018-01-05T11:28:50,783 INFO [TezTR-518813_1_15_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:28:50,784 DEBUG [TezTR-518813_1_15_0_0_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:50,784 DEBUG [TezTR-518813_1_15_0_0_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:50,784 DEBUG [TezTR-518813_1_15_0_0_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:50,784 DEBUG [TezTR-518813_1_15_0_0_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:50,784 WARN [TezTR-518813_1_15_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000001_0 (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:28:50,784 DEBUG [TezTR-518813_1_15_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:28:50,784 WARN [TezTR-518813_1_15_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000001_0:0+1614 2018-01-05T11:28:50,784 DEBUG [TezTR-518813_1_15_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000001_0 2018-01-05T11:28:50,784 INFO [TezTR-518813_1_15_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000001_0:0+1614, org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000002_0:0+1482, org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000000_0:0+1374, org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000003_0:0+1342], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=5812} 2018-01-05T11:28:50,784 INFO [TezTR-518813_1_15_0_0_0] input.MRInput: srcbucket2 initialized RecordReader from event 2018-01-05T11:28:50,784 DEBUG [TezTR-518813_1_15_0_0_0] tez.MapRecordProcessor: Starting Output: Reducer 2 2018-01-05T11:28:50,785 INFO [TezTR-518813_1_15_0_0_0] writers.BaseUnorderedPartitionedKVWriter: Instantiating Partitioner: [org.apache.tez.mapreduce.partition.MRPartitioner] 2018-01-05T11:28:50,809 INFO [TezTR-518813_1_15_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: numBuffers=2, sizePerBuffer=52428800, skipBuffers=true, numPartitions=1, availableMemory=104857600, maxSingleBufferSizeBytes=2147483647, pipelinedShuffle=false, isFinalMergeEnabled=true, numPartitions=1, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:28:50,809 INFO [TezTR-518813_1_15_0_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:28:50,809 INFO [TezTR-518813_1_15_0_0_0] exec.MapOperator: Initializing operator MAP[0] 2018-01-05T11:28:50,809 DEBUG [TezTR-518813_1_15_0_0_0] exec.MapOperator: Initialization Done 0 MAP 2018-01-05T11:28:50,809 DEBUG [TezTR-518813_1_15_0_0_0] exec.MapOperator: Operator 0 MAP initialized 2018-01-05T11:28:50,809 DEBUG [TezTR-518813_1_15_0_0_0] exec.MapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:28:50,809 DEBUG [TezTR-518813_1_15_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:50,810 DEBUG [TezTR-518813_1_15_0_0_0] exec.MapOperator: Adding alias srcbucket2 to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2 2018-01-05T11:28:50,810 DEBUG [TezTR-518813_1_15_0_0_0] exec.MapOperator: dump TS[0] struct 2018-01-05T11:28:50,810 INFO [TezTR-518813_1_15_0_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:28:50,810 INFO [TezTR-518813_1_15_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:50,810 DEBUG [TezTR-518813_1_15_0_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:28:50,810 INFO [TezTR-518813_1_15_0_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:28:50,810 DEBUG [TezTR-518813_1_15_0_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:28:50,810 DEBUG [TezTR-518813_1_15_0_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:28:50,810 DEBUG [TezTR-518813_1_15_0_0_0] exec.TableScanOperator: Initializing children of 0 TS 2018-01-05T11:28:50,810 DEBUG [TezTR-518813_1_15_0_0_0] exec.SelectOperator: Initializing child 1 SEL 2018-01-05T11:28:50,810 INFO [TezTR-518813_1_15_0_0_0] exec.SelectOperator: Initializing operator SEL[1] 2018-01-05T11:28:50,810 INFO [TezTR-518813_1_15_0_0_0] exec.SelectOperator: SELECT struct 2018-01-05T11:28:50,810 DEBUG [TezTR-518813_1_15_0_0_0] exec.SelectOperator: Initialization Done 1 SEL 2018-01-05T11:28:50,810 DEBUG [TezTR-518813_1_15_0_0_0] exec.SelectOperator: Operator 1 SEL initialized 2018-01-05T11:28:50,810 DEBUG [TezTR-518813_1_15_0_0_0] exec.SelectOperator: Initializing children of 1 SEL 2018-01-05T11:28:50,810 DEBUG [TezTR-518813_1_15_0_0_0] exec.GroupByOperator: Initializing child 2 GBY 2018-01-05T11:28:50,810 INFO [TezTR-518813_1_15_0_0_0] exec.GroupByOperator: Initializing operator GBY[2] 2018-01-05T11:28:50,810 DEBUG [TezTR-518813_1_15_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:50,810 DEBUG [TezTR-518813_1_15_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:50,810 INFO [TezTR-518813_1_15_0_0_0] exec.GroupByOperator: Max hash table memory: 93860656 bytes 2018-01-05T11:28:50,810 INFO [TezTR-518813_1_15_0_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:50,810 DEBUG [TezTR-518813_1_15_0_0_0] exec.GroupByOperator: Initialization Done 2 GBY 2018-01-05T11:28:50,810 DEBUG [TezTR-518813_1_15_0_0_0] exec.GroupByOperator: Operator 2 GBY initialized 2018-01-05T11:28:50,810 DEBUG [TezTR-518813_1_15_0_0_0] exec.GroupByOperator: Initializing children of 2 GBY 2018-01-05T11:28:50,810 DEBUG [TezTR-518813_1_15_0_0_0] exec.ReduceSinkOperator: Initializing child 3 RS 2018-01-05T11:28:50,810 INFO [TezTR-518813_1_15_0_0_0] exec.ReduceSinkOperator: Initializing operator RS[3] 2018-01-05T11:28:50,810 DEBUG [TezTR-518813_1_15_0_0_0] exec.ReduceSinkOperator: keys size is 0 2018-01-05T11:28:50,810 INFO [TezTR-518813_1_15_0_0_0] exec.ReduceSinkOperator: Using tag = -1 2018-01-05T11:28:50,810 DEBUG [TezTR-518813_1_15_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:50,810 DEBUG [TezTR-518813_1_15_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:50,810 DEBUG [TezTR-518813_1_15_0_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:28:50,810 DEBUG [TezTR-518813_1_15_0_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS 2018-01-05T11:28:50,810 DEBUG [TezTR-518813_1_15_0_0_0] exec.ReduceSinkOperator: Operator 3 RS initialized 2018-01-05T11:28:50,810 DEBUG [TezTR-518813_1_15_0_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS done is reset. 2018-01-05T11:28:50,810 DEBUG [TezTR-518813_1_15_0_0_0] exec.GroupByOperator: Initialization Done 2 GBY done is reset. 2018-01-05T11:28:50,810 DEBUG [TezTR-518813_1_15_0_0_0] exec.SelectOperator: Initialization Done 1 SEL done is reset. 2018-01-05T11:28:50,810 DEBUG [TezTR-518813_1_15_0_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:28:50,810 INFO [TezTR-518813_1_15_0_0_0] exec.OperatorUtils: Setting output collector: RS[3] --> Reducer 2 2018-01-05T11:28:50,810 DEBUG [TezTR-518813_1_15_0_0_0] log.PerfLogger: 2018-01-05T11:28:50,810 INFO [TezTR-518813_1_15_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_445_7459849956253588393-1/djaiswal/_tez_scratch_dir/05e6516a-f89b-4869-a585-686186680d76/map.xml 2018-01-05T11:28:50,810 DEBUG [TezTR-518813_1_15_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:50,810 DEBUG [TezTR-518813_1_15_0_0_0] exec.MapOperator: Processing alias(es) srcbucket2 for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000001_0 2018-01-05T11:28:50,811 INFO [TezTR-518813_1_15_0_0_0] exec.MapOperator: MAP[0]: records read - 1 2018-01-05T11:28:50,811 INFO [TezTR-518813_1_15_0_0_0] exec.MapOperator: MAP[0]: records read - 10 2018-01-05T11:28:50,811 INFO [TezTR-518813_1_15_0_0_0] exec.MapOperator: MAP[0]: records read - 100 2018-01-05T11:28:50,811 DEBUG [TezTR-518813_1_15_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 1 of 4 2018-01-05T11:28:50,811 DEBUG [TezTR-518813_1_15_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000002_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=4, serialization.ddl=struct srcbucket2 { i32 key, string value}, transient_lastDdlTime=1515180530, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket2}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=4, serialization.ddl=struct srcbucket2 { i32 key, string value}, transient_lastDdlTime=1515180530, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket2}, baseFileName=srcbucket2, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=4, serialization.ddl=struct srcbucket2 { i32 key, string value}, transient_lastDdlTime=1515180530, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket2}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=4, serialization.ddl=struct srcbucket2 { i32 key, string value}, transient_lastDdlTime=1515180530, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket2}, baseFileName=srcbucket2, vectorPartitionDesc=null]} 2018-01-05T11:28:50,811 DEBUG [TezTR-518813_1_15_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:50,811 INFO [TezTR-518813_1_15_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:50,811 DEBUG [TezTR-518813_1_15_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:50,812 DEBUG [TezTR-518813_1_15_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:50,812 INFO [TezTR-518813_1_15_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_445_7459849956253588393-1/djaiswal/_tez_scratch_dir/05e6516a-f89b-4869-a585-686186680d76/map.xml 2018-01-05T11:28:50,812 DEBUG [TezTR-518813_1_15_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:50,812 INFO [TezTR-518813_1_15_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_15_00_000000_0 2018-01-05T11:28:50,812 INFO [TezTR-518813_1_15_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:28:50,813 WARN [TezTR-518813_1_15_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000002_0 (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:28:50,813 DEBUG [TezTR-518813_1_15_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:28:50,813 WARN [TezTR-518813_1_15_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000002_0:0+1482 2018-01-05T11:28:50,813 DEBUG [TezTR-518813_1_15_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000002_0 2018-01-05T11:28:50,813 INFO [TezTR-518813_1_15_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_445_7459849956253588393-1/djaiswal/_tez_scratch_dir/05e6516a-f89b-4869-a585-686186680d76/map.xml 2018-01-05T11:28:50,813 DEBUG [TezTR-518813_1_15_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:50,813 DEBUG [TezTR-518813_1_15_0_0_0] exec.MapOperator: Processing alias(es) srcbucket2 for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000002_0 2018-01-05T11:28:50,814 DEBUG [TezTR-518813_1_15_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 2 of 4 2018-01-05T11:28:50,814 DEBUG [TezTR-518813_1_15_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000000_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=4, serialization.ddl=struct srcbucket2 { i32 key, string value}, transient_lastDdlTime=1515180530, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket2}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=4, serialization.ddl=struct srcbucket2 { i32 key, string value}, transient_lastDdlTime=1515180530, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket2}, baseFileName=srcbucket2, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=4, serialization.ddl=struct srcbucket2 { i32 key, string value}, transient_lastDdlTime=1515180530, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket2}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=4, serialization.ddl=struct srcbucket2 { i32 key, string value}, transient_lastDdlTime=1515180530, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket2}, baseFileName=srcbucket2, vectorPartitionDesc=null]} 2018-01-05T11:28:50,814 DEBUG [TezTR-518813_1_15_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:50,814 INFO [TezTR-518813_1_15_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:50,814 DEBUG [TezTR-518813_1_15_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:50,815 DEBUG [TezTR-518813_1_15_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:50,815 INFO [TezTR-518813_1_15_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_445_7459849956253588393-1/djaiswal/_tez_scratch_dir/05e6516a-f89b-4869-a585-686186680d76/map.xml 2018-01-05T11:28:50,815 DEBUG [TezTR-518813_1_15_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:50,815 INFO [TezTR-518813_1_15_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_15_00_000000_0 2018-01-05T11:28:50,815 INFO [TezTR-518813_1_15_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:28:50,816 WARN [TezTR-518813_1_15_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000000_0 (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:28:50,816 DEBUG [TezTR-518813_1_15_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:28:50,816 WARN [TezTR-518813_1_15_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000000_0:0+1374 2018-01-05T11:28:50,817 DEBUG [TezTR-518813_1_15_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000000_0 2018-01-05T11:28:50,817 INFO [TezTR-518813_1_15_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_445_7459849956253588393-1/djaiswal/_tez_scratch_dir/05e6516a-f89b-4869-a585-686186680d76/map.xml 2018-01-05T11:28:50,817 DEBUG [TezTR-518813_1_15_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:50,817 DEBUG [TezTR-518813_1_15_0_0_0] exec.MapOperator: Processing alias(es) srcbucket2 for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000000_0 2018-01-05T11:28:50,817 DEBUG [TezTR-518813_1_15_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 3 of 4 2018-01-05T11:28:50,817 DEBUG [TezTR-518813_1_15_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000003_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=4, serialization.ddl=struct srcbucket2 { i32 key, string value}, transient_lastDdlTime=1515180530, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket2}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=4, serialization.ddl=struct srcbucket2 { i32 key, string value}, transient_lastDdlTime=1515180530, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket2}, baseFileName=srcbucket2, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=4, serialization.ddl=struct srcbucket2 { i32 key, string value}, transient_lastDdlTime=1515180530, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket2}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=4, serialization.ddl=struct srcbucket2 { i32 key, string value}, transient_lastDdlTime=1515180530, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket2}, baseFileName=srcbucket2, vectorPartitionDesc=null]} 2018-01-05T11:28:50,818 DEBUG [TezTR-518813_1_15_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:50,818 INFO [TezTR-518813_1_15_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:50,818 DEBUG [TezTR-518813_1_15_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:50,819 DEBUG [TezTR-518813_1_15_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:50,819 INFO [TezTR-518813_1_15_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_445_7459849956253588393-1/djaiswal/_tez_scratch_dir/05e6516a-f89b-4869-a585-686186680d76/map.xml 2018-01-05T11:28:50,819 DEBUG [TezTR-518813_1_15_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:50,819 INFO [TezTR-518813_1_15_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_15_00_000000_0 2018-01-05T11:28:50,819 INFO [TezTR-518813_1_15_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:28:50,819 WARN [TezTR-518813_1_15_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000003_0 (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:28:50,819 DEBUG [TezTR-518813_1_15_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:28:50,819 WARN [TezTR-518813_1_15_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000003_0:0+1342 2018-01-05T11:28:50,820 DEBUG [TezTR-518813_1_15_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000003_0 2018-01-05T11:28:50,820 INFO [TezTR-518813_1_15_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_445_7459849956253588393-1/djaiswal/_tez_scratch_dir/05e6516a-f89b-4869-a585-686186680d76/map.xml 2018-01-05T11:28:50,820 DEBUG [TezTR-518813_1_15_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:50,820 DEBUG [TezTR-518813_1_15_0_0_0] exec.MapOperator: Processing alias(es) srcbucket2 for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2/000003_0 2018-01-05T11:28:50,821 DEBUG [TezTR-518813_1_15_0_0_0] log.PerfLogger: 2018-01-05T11:28:50,821 DEBUG [TezTR-518813_1_15_0_0_0] mr.ObjectCache: djaiswal_20180105112850_4efbe87f-26c9-4872-b247-f9c909a06058_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:28:50,821 DEBUG [TezTR-518813_1_15_0_0_0] exec.MapOperator: close called for operator MAP[0] 2018-01-05T11:28:50,821 INFO [TezTR-518813_1_15_0_0_0] exec.MapOperator: Closing operator MAP[0] 2018-01-05T11:28:50,821 INFO [TezTR-518813_1_15_0_0_0] exec.MapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:500, 2018-01-05T11:28:50,821 DEBUG [TezTR-518813_1_15_0_0_0] exec.MapOperator: Closing child = TS[0] 2018-01-05T11:28:50,821 DEBUG [TezTR-518813_1_15_0_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:28:50,821 DEBUG [TezTR-518813_1_15_0_0_0] exec.TableScanOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:50,821 INFO [TezTR-518813_1_15_0_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:28:50,821 DEBUG [TezTR-518813_1_15_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:50,821 DEBUG [TezTR-518813_1_15_0_0_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_452_2329691818928553056-1/-mr-10001 2018-01-05T11:28:50,821 INFO [TezTR-518813_1_15_0_0_0] exec.TableScanOperator: publishing : default.srcbucket2/ : {numRows=500, rawDataSize=5312} 2018-01-05T11:28:50,844 DEBUG [TezTR-518813_1_15_0_0_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_452_2329691818928553056-1/-mr-10001/tmpstats-0_0 2018-01-05T11:28:50,844 DEBUG [TezTR-518813_1_15_0_0_0] fs.FSStatsPublisher: Writing stats in it : {default.srcbucket2/={numRows=500, rawDataSize=5312}} 2018-01-05T11:28:50,844 DEBUG [TezTR-518813_1_15_0_0_0] exec.TableScanOperator: Closing child = SEL[1] 2018-01-05T11:28:50,844 DEBUG [TezTR-518813_1_15_0_0_0] exec.SelectOperator: close called for operator SEL[1] 2018-01-05T11:28:50,844 DEBUG [TezTR-518813_1_15_0_0_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:50,844 INFO [TezTR-518813_1_15_0_0_0] exec.SelectOperator: Closing operator SEL[1] 2018-01-05T11:28:50,844 DEBUG [TezTR-518813_1_15_0_0_0] exec.SelectOperator: Closing child = GBY[2] 2018-01-05T11:28:50,844 DEBUG [TezTR-518813_1_15_0_0_0] exec.GroupByOperator: close called for operator GBY[2] 2018-01-05T11:28:50,844 DEBUG [TezTR-518813_1_15_0_0_0] exec.GroupByOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:50,844 INFO [TezTR-518813_1_15_0_0_0] exec.GroupByOperator: Closing operator GBY[2] 2018-01-05T11:28:50,844 INFO [TezTR-518813_1_15_0_0_0] exec.GroupByOperator: Begin Hash Table flush: size = 1 2018-01-05T11:28:50,844 INFO [TezTR-518813_1_15_0_0_0] exec.ReduceSinkOperator: keys are [] num distributions: 0 2018-01-05T11:28:50,844 DEBUG [TezTR-518813_1_15_0_0_0] exec.GroupByOperator: Closing child = RS[3] 2018-01-05T11:28:50,844 DEBUG [TezTR-518813_1_15_0_0_0] exec.ReduceSinkOperator: close called for operator RS[3] 2018-01-05T11:28:50,844 DEBUG [TezTR-518813_1_15_0_0_0] exec.ReduceSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:50,844 INFO [TezTR-518813_1_15_0_0_0] exec.ReduceSinkOperator: Closing operator RS[3] 2018-01-05T11:28:50,844 INFO [TezTR-518813_1_15_0_0_0] exec.ReduceSinkOperator: RECORDS_OUT_INTERMEDIATE_Map_1:1, 2018-01-05T11:28:50,844 DEBUG [TezTR-518813_1_15_0_0_0] exec.ReduceSinkOperator: 3 Close done 2018-01-05T11:28:50,844 DEBUG [TezTR-518813_1_15_0_0_0] exec.GroupByOperator: 2 Close done 2018-01-05T11:28:50,844 DEBUG [TezTR-518813_1_15_0_0_0] exec.SelectOperator: 1 Close done 2018-01-05T11:28:50,844 DEBUG [TezTR-518813_1_15_0_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:28:50,844 DEBUG [TezTR-518813_1_15_0_0_0] exec.MapOperator: 0 Close done 2018-01-05T11:28:50,844 INFO [TezTR-518813_1_15_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_15_00_000000_0 2018-01-05T11:28:50,844 INFO [TezTR-518813_1_15_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: Waiting for all spills to complete : Pending : 0 2018-01-05T11:28:50,844 INFO [TezTR-518813_1_15_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: All spills complete 2018-01-05T11:28:50,856 DEBUG [TezTR-518813_1_15_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:50,856 INFO [TezTR-518813_1_15_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:50,856 INFO [TezTR-518813_1_15_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_15_00_000000_0, askedToStop=false 2018-01-05T11:28:50,856 INFO [TezTR-518813_1_15_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_15_00_000000_0, stopRequested=false 2018-01-05T11:28:50,856 INFO [TezTR-518813_1_15_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_15_00_000000_0: Counters: 14 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=500, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=2365, OUTPUT_BYTES_WITH_OVERHEAD=2375, OUTPUT_BYTES_PHYSICAL=2403, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500, RECORDS_OUT_INTERMEDIATE_Map_1=1]] 2018-01-05T11:28:50,856 DEBUG [TezTR-518813_1_15_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_15_0_0_0 thread id: 1028 scheme: file bytesRead: 5912 bytesWritten: 2500 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:50,857 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_15_00_000000_0, isTaskDone=true 2018-01-05T11:28:50,857 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000028, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_15_00_000000_0, eventCount=4 } 2018-01-05T11:28:50,858 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_15][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_15_00_000000_0, creationTime=1515180530756, allocationTime=1515180530757, startTime=1515180530759, finishTime=1515180530858, timeTaken=99, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:50,858 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:50,858 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_15_00_000000_0 2018-01-05T11:28:50,858 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_15_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:50,858 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_15_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:50,858 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_15_00_000000_0 2018-01-05T11:28:50,858 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_15_00_000000_0; false 2018-01-05T11:28:50,858 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_15_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:50,858 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000028=85 2018-01-05T11:28:50,858 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:50,858 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_15_00_000000_0: true 2018-01-05T11:28:50,859 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:50,859 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_15_00_000000_0 2018-01-05T11:28:50,859 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_15][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_15_00_000000, startTime=1515180530759, finishTime=1515180530858, timeTaken=99, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_15_00_000000_0, diagnostics=, counters=Counters: 20, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=5912, FILE_BYTES_WRITTEN=2500, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=500, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=2365, OUTPUT_BYTES_WITH_OVERHEAD=2375, OUTPUT_BYTES_PHYSICAL=2403, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500, RECORDS_OUT_INTERMEDIATE_Map_1=1 2018-01-05T11:28:50,859 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000028, QueryId=djaiswal_20180105112850_4efbe87f-26c9-4872-b247-f9c909a06058-15, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:28:50,859 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_15_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:50,859 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_15_00_000000_0 2018-01-05T11:28:50,859 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_15_00 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:50,859 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_15_00 [Map 1] 2018-01-05T11:28:50,859 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_15][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_15_00, initRequestedTime=1515180530691, initedTime=1515180530755, startRequestedTime=1515180530699, startedTime=1515180530755, finishTime=1515180530859, timeTaken=104, status=SUCCEEDED, diagnostics=, counters=Counters: 20, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=5912, FILE_BYTES_WRITTEN=2500, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=500, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=2365, OUTPUT_BYTES_WITH_OVERHEAD=2375, OUTPUT_BYTES_PHYSICAL=2403, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500, RECORDS_OUT_INTERMEDIATE_Map_1=1, vertexStats=firstTaskStartTime=1515180530759, firstTasksToStart=[ task_1515180518813_0001_15_00_000000 ], lastTaskFinishTime=1515180530858, lastTasksToFinish=[ task_1515180518813_0001_15_00_000000 ], minTaskDuration=99, maxTaskDuration=99, avgTaskDuration=99.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_15_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_15_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:50,859 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_15_00 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:50,859 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_15][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000028, stoppedTime=1515180530859, exitStatus=0 2018-01-05T11:28:50,859 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000028 exited with diagnostics set to Container completed. 2018-01-05T11:28:50,859 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_15_01 [Reducer 2] attempt: attempt_1515180518813_0001_15_00_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:28:50,860 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_15_00 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2 2018-01-05T11:28:50,860 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null 2018-01-05T11:28:50,860 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000028 2018-01-05T11:28:50,860 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: All source tasks completed. Ramping up 1 remaining tasks for vertex: Reducer 2 2018-01-05T11:28:50,860 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Scheduling 1 tasks for vertex: Reducer 2 with totalTasks: 1. 1 source tasks completed out of 1. MinSourceTaskCompletedFraction: 1.0 min: 0.05 max: 0.75 2018-01-05T11:28:50,860 INFO [App Shared Pool - #0] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Map 1, destinationVertexName=Reducer 2. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.UnorderedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:28:50,860 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_15_01 [Reducer 2] 2018-01-05T11:28:50,860 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_15][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_15_01_000000, scheduledTime=1515180530860, launchTime=1515180530860 2018-01-05T11:28:50,860 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_15_01 [Reducer 2] 2018-01-05T11:28:50,860 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_15_01_000000_0, priority=14, capability=, hosts=null 2018-01-05T11:28:50,860 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:50,860 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:50,860 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:50,860 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_15_01_000000_0 on hosts=null 2018-01-05T11:28:50,860 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_15_01_000000_0 on hosts=null 2018-01-05T11:28:50,860 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_15_01_000000_0 2018-01-05T11:28:50,860 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:50,860 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #29, task=TaskInfo{task=attempt_1515180518813_0001_15_01_000000_0, priority=14, startTime=0, containerId=null, uniqueId=28, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000029 2018-01-05T11:28:50,860 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_15_01_000000_0; false 2018-01-05T11:28:50,860 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_15_01_000000_0, priority=14, startTime=311551369, containerId=container_111101111_0001_00_000029assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=28, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:50,860 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000029 for node: 10.22.8.182:49818 2018-01-05T11:28:50,860 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000029, attempt_1515180518813_0001_15_01_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:50,860 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000029 for node: 10.22.8.182:49818 2018-01-05T11:28:50,861 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000029 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:50,861 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000029, launchTime=1515180530861 2018-01-05T11:28:50,861 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=15}, sourceName=Map 1, state=S_SUCCEEDED 2018-01-05T11:28:50,862 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_15_01_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000029] on NM: [10.22.8.182:49818] 2018-01-05T11:28:50,862 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:50,862 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:50,862 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_15][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_15_01_000000_0, startTime=1515180530861, containerId=container_111101111_0001_00_000029, nodeId=10.22.8.182:49818 2018-01-05T11:28:50,862 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_15_01_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112850_4efbe87f-26c9-4872-b247-f9c909a06058, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000029, dagName=ANALYZE TABLE srcbucket2 COMPUTE...key,value (Stage-0), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.UnorderedKVInput,1}}, Outputs={{out_Reducer 2,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=2, completedTaskCount=1, dagStartTime=1515180530691, firstAttemptStartTime=1515180530860, currentAttemptStartTime=1515180530861} 2018-01-05T11:28:50,862 INFO [IPC Server handler 3 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000029, QueryId=djaiswal_20180105112850_4efbe87f-26c9-4872-b247-f9c909a06058-15, DagId=15, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, SubmitTime=1515180530862 2018-01-05T11:28:50,862 DEBUG [IPC Server handler 3 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=15} with the ShuffleHandler 2018-01-05T11:28:50,862 DEBUG [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/15] 2018-01-05T11:28:50,863 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:50,863 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_15_01_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:50,863 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_15_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180530860, dagStartTime=1515180530691, withinDagPriority=14, vertexParallelism= 1, selfAndUpstreamParallelism= 2, selfAndUpstreamComplete= 1}] 2018-01-05T11:28:50,863 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_15_01_000000_0 : ACCEPTED 2018-01-05T11:28:50,863 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_15_01_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:50,863 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_15_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180530860, dagStartTime=1515180530691, withinDagPriority=14, vertexParallelism= 1, selfAndUpstreamParallelism= 2, selfAndUpstreamComplete= 1} 2018-01-05T11:28:50,863 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_15_01_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:50,863 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_15_01_000000_0 to pre-emption queue 2018-01-05T11:28:50,863 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_15_01_000000_0: true 2018-01-05T11:28:50,863 INFO [TaskCommunicator # 2] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_15_01_000000_0 2018-01-05T11:28:50,863 INFO [TaskCommunicator # 2] GuaranteedTasks: Received response for attempt_1515180518813_0001_15_01_000000_0, true 2018-01-05T11:28:50,863 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:653102606 2018-01-05T11:28:50,864 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : ANALYZE TABLE srcbucket2 COMPUTE...key,value (Stage-0), VertexName: Reducer 2, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_15_01_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:28:50,864 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1039138816, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:50,864 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_15_01_000000_0 vertexName: Reducer 2 2018-01-05T11:28:50,864 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000029, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_15_01_000000_0, eventCount=1 } 2018-01-05T11:28:50,864 DEBUG [TezTR-518813_1_15_1_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_15 queryId: djaiswal_20180105112850_4efbe87f-26c9-4872-b247-f9c909a06058 instanceType: TaskRunner2Callable 2018-01-05T11:28:50,864 INFO [TezTR-518813_1_15_1_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_15_01_000000_0 2018-01-05T11:28:50,864 DEBUG [TezTR-518813_1_15_1_0_0] log.PerfLogger: 2018-01-05T11:28:50,865 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:50,866 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:50,867 INFO [IPC Server handler 26 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_15_01_000000_0 1 events [0,1) total 1 vertex_1515180518813_0001_15_01 [Reducer 2] 2018-01-05T11:28:50,868 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:28:50,868 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_15_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:28:50,868 DEBUG [TezTR-518813_1_15_1_0_0] log.PerfLogger: 2018-01-05T11:28:50,868 INFO [TezTR-518813_1_15_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:50,868 INFO [TezTR-518813_1_15_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:50,868 INFO [TezTR-518813_1_15_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:50,869 INFO [I/O Setup 1 Initialize: {out_Reducer 2}] output.MROutput: out_Reducer 2: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:28:50,869 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:50,869 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:50,869 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:50,869 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:50,869 INFO [TezTR-518813_1_15_1_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:50,869 INFO [TezTR-518813_1_15_1_0_0] resources.MemoryDistributor: InitialRequests=[out_Reducer 2:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.UnorderedKVInput] 2018-01-05T11:28:50,869 INFO [TezTR-518813_1_15_1_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:50,869 INFO [TezTR-518813_1_15_1_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:50,869 INFO [TezTR-518813_1_15_1_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 131404920, TotalRequestedScaled: 6.570246E7, TotalJVMHeap: 1039138816, TotalAvailable: 696223006, TotalRequested/TotalJVMHeap:0.13 2018-01-05T11:28:50,869 INFO [TezTR-518813_1_15_1_0_0] resources.MemoryDistributor: Allocations=[out_Reducer 2:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [Map 1:org.apache.tez.runtime.library.input.UnorderedKVInput:INPUT:131404920:131404920] 2018-01-05T11:28:50,869 INFO [TezTR-518813_1_15_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:50,869 INFO [TezTR-518813_1_15_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 0 2018-01-05T11:28:50,869 INFO [TezTR-518813_1_15_1_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:50,869 INFO [TezTR-518813_1_15_1_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_15_01_000000_0 2018-01-05T11:28:50,869 DEBUG [TezTR-518813_1_15_1_0_0] log.PerfLogger: 2018-01-05T11:28:50,869 DEBUG [TezTR-518813_1_15_1_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_15_01_000000_0_10085 2018-01-05T11:28:50,869 DEBUG [TezTR-518813_1_15_1_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:50,869 DEBUG [TezTR-518813_1_15_1_0_0] mr.ObjectCache: Creating djaiswal_20180105112850_4efbe87f-26c9-4872-b247-f9c909a06058_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:28:50,869 INFO [TezTR-518813_1_15_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_445_7459849956253588393-1/djaiswal/_tez_scratch_dir/8bcf0979-e4ff-4715-a6da-c6be1e9ba60c/reduce.xml 2018-01-05T11:28:50,869 DEBUG [TezTR-518813_1_15_1_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_445_7459849956253588393-1/djaiswal/_tez_scratch_dir/8bcf0979-e4ff-4715-a6da-c6be1e9ba60c/reduce.xml 2018-01-05T11:28:50,869 DEBUG [TezTR-518813_1_15_1_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_445_7459849956253588393-1/djaiswal/_tez_scratch_dir/8bcf0979-e4ff-4715-a6da-c6be1e9ba60c/reduce.xml 2018-01-05T11:28:50,869 DEBUG [TezTR-518813_1_15_1_0_0] log.PerfLogger: 2018-01-05T11:28:50,869 INFO [TezTR-518813_1_15_1_0_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:28:50,869 DEBUG [TezTR-518813_1_15_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:50,870 DEBUG [TezTR-518813_1_15_1_0_0] log.PerfLogger: 2018-01-05T11:28:50,870 INFO [TezTR-518813_1_15_1_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 2.05KB 2018-01-05T11:28:50,870 DEBUG [TezTR-518813_1_15_1_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:50,870 DEBUG [TezTR-518813_1_15_1_0_0] log.PerfLogger: 2018-01-05T11:28:50,870 DEBUG [TezTR-518813_1_15_1_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:50,870 DEBUG [TezTR-518813_1_15_1_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:50,871 DEBUG [TezTR-518813_1_15_1_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:50,871 INFO [TezTR-518813_1_15_1_0_0] impl.SimpleFetchedInputAllocator: Map_1: RequestedMemory=131404920, AssignedMemory=131404920, maxSingleShuffleLimit=32851230 2018-01-05T11:28:50,871 INFO [TezTR-518813_1_15_1_0_0] impl.ShuffleManager: Map_1: numInputs=1, compressionCodec=NoCompressionCodec, numFetchers=1, ifileBufferSize=4096, ifileReadAheadEnabled=true, ifileReadAheadLength=4194304, localDiskFetchEnabled=false, sharedFetchEnabled=false, keepAlive=false, keepAliveMaxConnections=20, connectionTimeout=20000, readTimeout=180000, bufferSize=8192, bufferSize=8192, maxTaskOutputAtOnce=20 2018-01-05T11:28:50,871 INFO [TezTR-518813_1_15_1_0_0] impl.ShuffleInputEventHandlerImpl: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:28:50,871 INFO [TezTR-518813_1_15_1_0_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:28:50,872 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=15&reduce=0&map=attempt_1515180518813_0001_15_00_000000_0_10084 mapId: [attempt_1515180518813_0001_15_00_000000_0_10084] reduceId: [0] jobId: [job_1515180518813_0001] dagId: [15] keepAlive: false 2018-01-05T11:28:50,872 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=15&reduce=0&map=attempt_1515180518813_0001_15_00_000000_0_10084; hash=...g//Yaca/qAufU 2018-01-05T11:28:50,872 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=15&reduce=0&map=attempt_1515180518813_0001_15_00_000000_0_10084;reply=jfWBruana8lTc 2018-01-05T11:28:50,873 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=15, user='djaiswal', attemptId='attempt_1515180518813_0001_15_00_000000_0_10084'} via loader 2018-01-05T11:28:50,873 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=15, user='djaiswal', attemptId='attempt_1515180518813_0001_15_00_000000_0_10084'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:28:50,873 DEBUG [New I/O worker #2] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_15_00_000000_0_10084 not found 2018-01-05T11:28:50,873 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_15_00_000000_0_10084,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/15/output/attempt_1515180518813_0001_15_00_000000_0_10084/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/15/output/attempt_1515180518813_0001_15_00_000000_0_10084/file.out.index 2018-01-05T11:28:50,873 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: Content Length in shuffle : 2434 2018-01-05T11:28:50,873 INFO [Fetcher_B {Map_1} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=15&reduce=0&map=attempt_1515180518813_0001_15_00_000000_0_10084 sent hash and receievd reply 0 ms 2018-01-05T11:28:50,873 INFO [Fetcher_B {Map_1} #0] ShuffleManager.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_15_00_000000_0_10084} to MEMORY, csize=2379, dsize=2375, EndTime=1515180530873, TimeTaken=0, Rate=0.00 MB/s 2018-01-05T11:28:50,873 INFO [Fetcher_B {Map_1} #0] impl.ShuffleManager: All inputs fetched for input vertex : Map 1 2018-01-05T11:28:50,873 INFO [TezTR-518813_1_15_1_0_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:28:50,873 INFO [Fetcher_B {Map_1} #0] impl.ShuffleManager: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:28:50,874 INFO [TezTR-518813_1_15_1_0_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:50,874 INFO [ShuffleRunner {Map_1}] impl.ShuffleManager: Map_1: Shutting down FetchScheduler, Was Interrupted: false 2018-01-05T11:28:50,874 DEBUG [TezTR-518813_1_15_1_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:28:50,874 INFO [ShuffleRunner {Map_1}] impl.ShuffleManager: Map_1: Scheduler thread completed 2018-01-05T11:28:50,874 DEBUG [TezTR-518813_1_15_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:50,874 DEBUG [TezTR-518813_1_15_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:50,874 DEBUG [TezTR-518813_1_15_1_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:28:50,874 DEBUG [TezTR-518813_1_15_1_0_0] log.PerfLogger: 2018-01-05T11:28:50,874 INFO [TezTR-518813_1_15_1_0_0] exec.GroupByOperator: Initializing operator GBY[4] 2018-01-05T11:28:50,874 DEBUG [TezTR-518813_1_15_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:50,874 DEBUG [TezTR-518813_1_15_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:50,874 INFO [TezTR-518813_1_15_1_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:50,874 DEBUG [TezTR-518813_1_15_1_0_0] exec.GroupByOperator: Initialization Done 4 GBY 2018-01-05T11:28:50,874 DEBUG [TezTR-518813_1_15_1_0_0] exec.GroupByOperator: Operator 4 GBY initialized 2018-01-05T11:28:50,874 DEBUG [TezTR-518813_1_15_1_0_0] exec.GroupByOperator: Initializing children of 4 GBY 2018-01-05T11:28:50,874 DEBUG [TezTR-518813_1_15_1_0_0] exec.FileSinkOperator: Initializing child 6 FS 2018-01-05T11:28:50,874 INFO [TezTR-518813_1_15_1_0_0] exec.FileSinkOperator: Initializing operator FS[6] 2018-01-05T11:28:50,874 DEBUG [TezTR-518813_1_15_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:50,874 INFO [TezTR-518813_1_15_1_0_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@62031fe5]:[_col0, _col1]:[struct, struct]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@431b8ec3 2018-01-05T11:28:50,874 DEBUG [TezTR-518813_1_15_1_0_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:28:50,874 DEBUG [TezTR-518813_1_15_1_0_0] exec.FileSinkOperator: Initialization Done 6 FS 2018-01-05T11:28:50,874 DEBUG [TezTR-518813_1_15_1_0_0] exec.FileSinkOperator: Operator 6 FS initialized 2018-01-05T11:28:50,874 DEBUG [TezTR-518813_1_15_1_0_0] exec.FileSinkOperator: Initialization Done 6 FS done is reset. 2018-01-05T11:28:50,874 DEBUG [TezTR-518813_1_15_1_0_0] exec.GroupByOperator: Initialization Done 4 GBY done is reset. 2018-01-05T11:28:50,874 INFO [TezTR-518813_1_15_1_0_0] tez.ReduceRecordProcessor: Id =4 Id =6 <\Children> Id = 4 null<\Parent> <\FS> <\Children> <\Parent> <\GBY> 2018-01-05T11:28:50,874 DEBUG [TezTR-518813_1_15_1_0_0] log.PerfLogger: 2018-01-05T11:28:50,874 INFO [TezTR-518813_1_15_1_0_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 2 2018-01-05T11:28:50,874 DEBUG [TezTR-518813_1_15_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:28:50,874 DEBUG [TezTR-518813_1_15_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:28:50,874 DEBUG [TezTR-518813_1_15_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:28:50,874 DEBUG [TezTR-518813_1_15_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:28:50,874 DEBUG [TezTR-518813_1_15_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:28:50,874 DEBUG [TezTR-518813_1_15_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:28:50,875 INFO [TezTR-518813_1_15_1_0_0] readers.UnorderedKVReader: Num Records read: 1 2018-01-05T11:28:50,875 DEBUG [TezTR-518813_1_15_1_0_0] log.PerfLogger: 2018-01-05T11:28:50,875 DEBUG [TezTR-518813_1_15_1_0_0] mr.ObjectCache: djaiswal_20180105112850_4efbe87f-26c9-4872-b247-f9c909a06058_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:28:50,875 DEBUG [TezTR-518813_1_15_1_0_0] exec.GroupByOperator: close called for operator GBY[4] 2018-01-05T11:28:50,875 INFO [TezTR-518813_1_15_1_0_0] exec.GroupByOperator: Closing operator GBY[4] 2018-01-05T11:28:50,875 INFO [TezTR-518813_1_15_1_0_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_452_2329691818928553056-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-50_452_2329691818928553056-1/_tmp.-ext-10002/000000_0 2018-01-05T11:28:50,875 INFO [TezTR-518813_1_15_1_0_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_452_2329691818928553056-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-50_452_2329691818928553056-1/_task_tmp.-ext-10002/_tmp.000000_0 2018-01-05T11:28:50,875 INFO [TezTR-518813_1_15_1_0_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_452_2329691818928553056-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-50_452_2329691818928553056-1/_tmp.-ext-10002/000000_0 2018-01-05T11:28:50,909 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,909 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: Status: Running (Executing on YARN cluster with App id application_1515180518813_0001) 2018-01-05T11:28:50,910 INFO [TezTR-518813_1_15_1_0_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:28:50,910 DEBUG [TezTR-518813_1_15_1_0_0] exec.GroupByOperator: Closing child = FS[6] 2018-01-05T11:28:50,910 DEBUG [TezTR-518813_1_15_1_0_0] exec.FileSinkOperator: close called for operator FS[6] 2018-01-05T11:28:50,910 DEBUG [TezTR-518813_1_15_1_0_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:50,910 INFO [TezTR-518813_1_15_1_0_0] exec.FileSinkOperator: Closing operator FS[6] 2018-01-05T11:28:50,910 INFO [TezTR-518813_1_15_1_0_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:28:50,910 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,910 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,910 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,910 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 Reducer 2: 0(+1)/1 2018-01-05T11:28:50,911 INFO [TezTR-518813_1_15_1_0_0] exec.FileSinkOperator: RECORDS_OUT_0:1, 2018-01-05T11:28:50,911 DEBUG [TezTR-518813_1_15_1_0_0] exec.FileSinkOperator: 6 Close done 2018-01-05T11:28:50,911 DEBUG [TezTR-518813_1_15_1_0_0] exec.GroupByOperator: 4 Close done 2018-01-05T11:28:50,911 INFO [TezTR-518813_1_15_1_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_15_01_000000_0 2018-01-05T11:28:50,911 INFO [TezTR-518813_1_15_1_0_0] impl.ShuffleInputEventHandlerImpl: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:28:50,911 INFO [TezTR-518813_1_15_1_0_0] impl.ShuffleManager: Shutting down pending fetchers on sourceMap_1: 0 2018-01-05T11:28:50,911 INFO [TezTR-518813_1_15_1_0_0] output.MROutput: out_Reducer 2 closed 2018-01-05T11:28:50,911 DEBUG [TezTR-518813_1_15_1_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:50,911 INFO [TezTR-518813_1_15_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:50,911 INFO [TezTR-518813_1_15_1_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_15_01_000000_0, askedToStop=false 2018-01-05T11:28:50,911 INFO [TezTR-518813_1_15_1_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_15_01_000000_0, stopRequested=false 2018-01-05T11:28:50,911 INFO [TezTR-518813_1_15_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_15_01_000000_0: Counters: 14 [[org.apache.tez.common.counters.TaskCounter NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=2379, SHUFFLE_BYTES_DECOMPRESSED=2375, SHUFFLE_BYTES_TO_MEM=2379, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=3, FIRST_EVENT_RECEIVED=0, LAST_EVENT_RECEIVED=0][HIVE CREATED_FILES=1, RECORDS_OUT_0=1]] 2018-01-05T11:28:50,911 DEBUG [TezTR-518813_1_15_1_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_15_1_0_0 thread id: 1040 scheme: file bytesRead: 0 bytesWritten: 3277 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:50,912 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_15_01_000000_0, isTaskDone=true 2018-01-05T11:28:50,912 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000029, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_15_01_000000_0, eventCount=2 } 2018-01-05T11:28:50,913 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_15][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_15_01_000000_0, creationTime=1515180530860, allocationTime=1515180530860, startTime=1515180530861, finishTime=1515180530913, timeTaken=52, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:50,913 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_15_01_000000_0 2018-01-05T11:28:50,913 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_15_01_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:50,913 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_15_01_000000_0; false 2018-01-05T11:28:50,913 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_15_01_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:50,913 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_15][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_15_01_000000, startTime=1515180530861, finishTime=1515180530913, timeTaken=52, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_15_01_000000_0, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=3277, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=2379, SHUFFLE_BYTES_DECOMPRESSED=2375, SHUFFLE_BYTES_TO_MEM=2379, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=3, FIRST_EVENT_RECEIVED=0, LAST_EVENT_RECEIVED=0, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1 2018-01-05T11:28:50,914 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:28:50,914 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_15_01 [Reducer 2], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:50,914 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_15_01 [Reducer 2] 2018-01-05T11:28:50,914 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_15_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:50,914 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_15_01_000000_0 2018-01-05T11:28:50,914 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:50,914 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:50,915 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_15][Event:VERTEX_FINISHED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_15_01, initRequestedTime=1515180530699, initedTime=1515180530704, startRequestedTime=1515180530755, startedTime=1515180530755, finishTime=1515180530914, timeTaken=159, status=SUCCEEDED, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=3277, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=2379, SHUFFLE_BYTES_DECOMPRESSED=2375, SHUFFLE_BYTES_TO_MEM=2379, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=3, FIRST_EVENT_RECEIVED=0, LAST_EVENT_RECEIVED=0, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1, vertexStats=firstTaskStartTime=1515180530861, firstTasksToStart=[ task_1515180518813_0001_15_01_000000 ], lastTaskFinishTime=1515180530913, lastTasksToFinish=[ task_1515180518813_0001_15_01_000000 ], minTaskDuration=52, maxTaskDuration=52, avgTaskDuration=52.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_15_01_000000 ], longestDurationTasks=[ task_1515180518813_0001_15_01_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:50,915 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000029=51 2018-01-05T11:28:50,915 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_15_01_000000_0: true 2018-01-05T11:28:50,915 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_15_01 [Reducer 2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:50,915 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_15][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000029, stoppedTime=1515180530915, exitStatus=0 2018-01-05T11:28:50,915 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000029 exited with diagnostics set to Container completed. 2018-01-05T11:28:50,915 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_15_01 [Reducer 2] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2 2018-01-05T11:28:50,915 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null 2018-01-05T11:28:50,915 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_15_01 [Reducer 2] 2018-01-05T11:28:50,915 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_15_00 [Map 1] 2018-01-05T11:28:50,916 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_15, queueSize=0 2018-01-05T11:28:50,916 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_15][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_15, startTime=1515180530691, finishTime=1515180530915, timeTaken=224, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, OTHER_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=413, AM_GC_TIME_MILLIS=0, File System Counters, FILE_BYTES_READ=5912, FILE_BYTES_WRITTEN=5777, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=501, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=2365, OUTPUT_BYTES_WITH_OVERHEAD=2375, OUTPUT_BYTES_PHYSICAL=2403, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=2379, SHUFFLE_BYTES_DECOMPRESSED=2375, SHUFFLE_BYTES_TO_MEM=2379, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=3, FIRST_EVENT_RECEIVED=0, LAST_EVENT_RECEIVED=0, HIVE, CREATED_FILES=1, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500, RECORDS_OUT_0=1, RECORDS_OUT_INTERMEDIATE_Map_1=1 2018-01-05T11:28:50,916 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_15 finished with state: SUCCEEDED 2018-01-05T11:28:50,916 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_15 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:28:50,916 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000029 2018-01-05T11:28:50,916 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_15, dagState=SUCCEEDED 2018-01-05T11:28:50,916 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_15_post 2018-01-05T11:28:50,916 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:50,916 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:28:50,916 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:28:50,916 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 15 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=2, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=2,NumRequestsWithoutLocation=2, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=2} 2018-01-05T11:28:50,916 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:28:50,916 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=ANALYZE TABLE srcbucket2 COMPUTE...key,value (Stage-0), with id=dag_1515180518813_0001_15 2018-01-05T11:28:50,916 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 15, to 10.22.8.182:49818 2018-01-05T11:28:50,916 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=2, Remaining=0 2018-01-05T11:28:50,917 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=ANALYZE TABLE srcbucket2 COMPUTE...key,value (Stage-0), with id=dag_1515180518813_0001_15 2018-01-05T11:28:50,917 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=15} 2018-01-05T11:28:50,917 INFO [IPC Server handler 1 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=15}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:28:50,917 INFO [IPC Server handler 1 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/15 after 300 seconds 2018-01-05T11:28:50,917 ERROR [IPC Server handler 1 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112850_4efbe87f-26c9-4872-b247-f9c909a06058, dag_1515180518813_0001_15 2018-01-05T11:28:50,917 INFO [IPC Server handler 1 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112850_4efbe87f-26c9-4872-b247-f9c909a06058 2018-01-05T11:28:50,917 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=15} = 1 2018-01-05T11:28:50,917 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Issuing killFragment for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=15} attempt_1515180518813_0001_15_01_000000_0 2018-01-05T11:28:50,917 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Removing attempt_1515180518813_0001_15_01_000000_0 from preemptionQueue 2018-01-05T11:28:50,917 INFO [IPC Server handler 1 on 49818] impl.TaskRunnerCallable: Ignoring kill request for task attempt_1515180518813_0001_15_01_000000_0 since it's already complete 2018-01-05T11:28:50,917 DEBUG [IPC Server handler 1 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=15} 2018-01-05T11:28:50,917 DEBUG [IPC Server handler 1 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:28:50,917 DEBUG [IPC Server handler 1 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=15} 2018-01-05T11:28:50,918 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,918 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 Reducer 2: 1/1 2018-01-05T11:28:50,918 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,920 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,920 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: TaskId for 000000_0 = 000000 2018-01-05T11:28:50,920 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,920 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,921 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,922 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-2:STATS] in serial mode 2018-01-05T11:28:50,923 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@30df67b8, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:50,923 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:50,923 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,923 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:50,923 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:50,923 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:50,923 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:50,923 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,923 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:50,923 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:50,923 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_452_2329691818928553056-1/-mr-10001 2018-01-05T11:28:50,923 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_452_2329691818928553056-1/-mr-10001 2018-01-05T11:28:50,925 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.srcbucket2/ numRows 2018-01-05T11:28:50,925 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.srcbucket2/ numRows 500 2018-01-05T11:28:50,925 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.srcbucket2/ rawDataSize 2018-01-05T11:28:50,925 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.srcbucket2/ rawDataSize 5312 2018-01-05T11:28:50,925 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,926 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:50,926 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:50,926 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:50,926 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:50,926 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,926 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,926 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,927 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,928 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,928 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:50,928 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:50,928 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,928 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:50,928 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,928 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=srcbucket2 newtbl=srcbucket2 2018-01-05T11:28:50,928 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=srcbucket2 newtbl=srcbucket2 2018-01-05T11:28:50,932 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,938 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,938 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,939 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.srcbucket2 stats: [numFiles=4, numRows=500, totalSize=5812, rawDataSize=5312] 2018-01-05T11:28:50,939 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: About to delete stats tmp dir :file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_452_2329691818928553056-1/-mr-10001 2018-01-05T11:28:50,961 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Time taken to get FileStatuses: 21 2018-01-05T11:28:50,961 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:28:50,961 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 21 2018-01-05T11:28:50,961 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FetchOperator: Creating fetchTask with deserializer typeinfo: struct<_col0:struct,_col1:struct> 2018-01-05T11:28:50,961 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FetchOperator: deserializer properties: table properties: {columns=_col0,_col1, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct:struct, escape.delim=\} partition properties: {columns=_col0,_col1, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct:struct, escape.delim=\} 2018-01-05T11:28:50,961 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000028, QueryId=djaiswal_20180105112850_4efbe87f-26c9-4872-b247-f9c909a06058-15, DagId=15, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180530772, EndTime=1515180530961 2018-01-05T11:28:50,961 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_15_01_000000_0 2018-01-05T11:28:50,961 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_15_01_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:50,961 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_15_01_000000_0 2018-01-05T11:28:50,961 INFO [ExecutionCompletionThread #0] impl.QueryTracker: Ignoring fragmentComplete message for unknown query: QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=15} 2018-01-05T11:28:50,962 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,962 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: write_column_statistics: db=default table=srcbucket2 2018-01-05T11:28:50,962 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=write_column_statistics: db=default table=srcbucket2 2018-01-05T11:28:50,968 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=srcbucket2 tableName=default colName=key 2018-01-05T11:28:50,973 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=srcbucket2 tableName=default colName=value 2018-01-05T11:28:50,978 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,978 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,978 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,978 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,978 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: ANALYZE TABLE srcbucket2 COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:50,978 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:28:50,978 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@srcbucket2 2018-01-05T11:28:50,978 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@srcbucket2 2018-01-05T11:28:50,978 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_452_2329691818928553056-1/-mr-10000 2018-01-05T11:28:50,978 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,978 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,979 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:50,979 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=11, setPartitionColumnStatistics_(SetPartitionsStatsRequest, )=16} 2018-01-05T11:28:50,979 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112850_4efbe87f-26c9-4872-b247-f9c909a06058); Time taken: 0.444 seconds 2018-01-05T11:28:50,979 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:50,979 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,979 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:50,979 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,979 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query ANALYZE TABLE srcbucket2 COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:50,979 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-50_445_7459849956253588393-1 2018-01-05T11:28:50,979 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.534 seconds 2018-01-05T11:28:50,979 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:50,980 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:50,980 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:50,980 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:50,980 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,980 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:50,980 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,980 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,980 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,980 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: -- -- Table srcpart -- DROP TABLE IF EXISTS srcpart 2018-01-05T11:28:50,980 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112850_5704c5ca-630a-4315-ba80-4f9fc6dddffa): -- -- Table srcpart -- DROP TABLE IF EXISTS srcpart 2018-01-05T11:28:50,980 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,980 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: -- -- Table srcpart -- DROP TABLE IF EXISTS srcpart 2018-01-05T11:28:50,980 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:50,980 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,980 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,980 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,980 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,980 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:50,980 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,980 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcpart 2018-01-05T11:28:50,980 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcpart 2018-01-05T11:28:50,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,981 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:50,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,981 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:50,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,981 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:50,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:28:50,981 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112850_5704c5ca-630a-4315-ba80-4f9fc6dddffa); Time taken: 0.001 seconds 2018-01-05T11:28:50,981 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:50,981 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,981 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112850_5704c5ca-630a-4315-ba80-4f9fc6dddffa): -- -- Table srcpart -- DROP TABLE IF EXISTS srcpart 2018-01-05T11:28:50,983 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,983 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: -- -- Table srcpart -- DROP TABLE IF EXISTS srcpart 2018-01-05T11:28:50,983 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:50,983 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,983 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,983 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,983 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,983 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:50,983 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,983 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcpart 2018-01-05T11:28:50,983 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcpart 2018-01-05T11:28:50,984 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,984 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,984 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table srcpart not found: default.srcpart table not found 2018-01-05T11:28:50,984 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,984 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcpart 2018-01-05T11:28:50,984 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcpart 2018-01-05T11:28:50,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,985 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: -- -- Table srcpart -- DROP TABLE IF EXISTS srcpart 2018-01-05T11:28:50,985 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:50,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,985 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:50,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=1} 2018-01-05T11:28:50,985 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112850_5704c5ca-630a-4315-ba80-4f9fc6dddffa); Time taken: 0.004 seconds 2018-01-05T11:28:50,985 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:50,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,985 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:50,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query -- -- Table srcpart -- DROP TABLE IF EXISTS srcpart 2018-01-05T11:28:50,985 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.005 seconds 2018-01-05T11:28:50,985 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:50,985 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:50,985 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:50,985 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:50,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:50,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: CREATE TABLE srcpart (key STRING COMMENT 'default', value STRING COMMENT 'default') PARTITIONED BY (ds STRING, hr STRING) STORED AS TEXTFILE 2018-01-05T11:28:50,985 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112850_2c457fae-d6c8-43a6-9aa0-d1f42c498a03): CREATE TABLE srcpart (key STRING COMMENT 'default', value STRING COMMENT 'default') PARTITIONED BY (ds STRING, hr STRING) STORED AS TEXTFILE 2018-01-05T11:28:50,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: CREATE TABLE srcpart (key STRING COMMENT 'default', value STRING COMMENT 'default') PARTITIONED BY (ds STRING, hr STRING) STORED AS TEXTFILE 2018-01-05T11:28:50,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:50,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,987 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,987 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,987 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:50,987 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:28:50,987 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Creating table default.srcpart position=13 2018-01-05T11:28:50,987 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,987 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_database: default 2018-01-05T11:28:50,987 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_database: default 2018-01-05T11:28:50,988 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:50,988 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:50,988 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,988 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:50,988 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: validation start 2018-01-05T11:28:50,988 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:28:50,988 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Not a partition. 2018-01-05T11:28:50,988 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,988 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:50,988 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,988 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:50,988 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=1, getDatabase_(String, )=1, flushCache_()=0} 2018-01-05T11:28:50,988 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112850_2c457fae-d6c8-43a6-9aa0-d1f42c498a03); Time taken: 0.003 seconds 2018-01-05T11:28:50,988 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:50,988 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,988 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112850_2c457fae-d6c8-43a6-9aa0-d1f42c498a03): CREATE TABLE srcpart (key STRING COMMENT 'default', value STRING COMMENT 'default') PARTITIONED BY (ds STRING, hr STRING) STORED AS TEXTFILE 2018-01-05T11:28:50,989 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,989 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: CREATE TABLE srcpart (key STRING COMMENT 'default', value STRING COMMENT 'default') PARTITIONED BY (ds STRING, hr STRING) STORED AS TEXTFILE 2018-01-05T11:28:50,989 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: CREATETABLE 2018-01-05T11:28:50,989 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: database:default 2018-01-05T11:28:50,989 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@srcpart 2018-01-05T11:28:50,989 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,989 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,989 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,989 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:50,990 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:50,990 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Found class for org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe 2018-01-05T11:28:50,990 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: creating table default.srcpart on file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart 2018-01-05T11:28:50,990 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:50,990 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: create_table: Table(tableName:srcpart, dbName:default, owner:hive_test_user, createTime:1515180530, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:string, comment:default), FieldSchema(name:value, type:string, comment:default)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[FieldSchema(name:ds, type:string, comment:null), FieldSchema(name:hr, type:string, comment:null)], parameters:{}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:50,990 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=create_table: Table(tableName:srcpart, dbName:default, owner:hive_test_user, createTime:1515180530, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:string, comment:default), FieldSchema(name:value, type:string, comment:default)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[FieldSchema(name:ds, type:string, comment:null), FieldSchema(name:hr, type:string, comment:null)], parameters:{}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:50,991 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hive-site.xml 2018-01-05T11:28:50,992 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hivemetastore-site.xml 2018-01-05T11:28:50,992 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Unable to find config file metastore-site.xml 2018-01-05T11:28:50,992 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file null 2018-01-05T11:28:51,003 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:51,003 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: Location: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart specified for non-external table:srcpart 2018-01-05T11:28:51,003 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart 2018-01-05T11:28:51,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@srcpart) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@srcpart) Type=TABLE WriteType=DDL_NO_LOCK is present 2018-01-05T11:28:51,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,024 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: CREATE TABLE srcpart (key STRING COMMENT 'default', value STRING COMMENT 'default') PARTITIONED BY (ds STRING, hr STRING) STORED AS TEXTFILE 2018-01-05T11:28:51,024 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: CREATETABLE 2018-01-05T11:28:51,024 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: database:default 2018-01-05T11:28:51,024 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@srcpart 2018-01-05T11:28:51,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,025 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,025 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:51,025 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {createTable_(Table, )=34, isCompatibleWith_(Configuration, )=0} 2018-01-05T11:28:51,025 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112850_2c457fae-d6c8-43a6-9aa0-d1f42c498a03); Time taken: 0.037 seconds 2018-01-05T11:28:51,025 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:51,025 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,025 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:51,025 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,025 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query CREATE TABLE srcpart (key STRING COMMENT 'default', value STRING COMMENT 'default') PARTITIONED BY (ds STRING, hr STRING) STORED AS TEXTFILE 2018-01-05T11:28:51,025 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.04 seconds 2018-01-05T11:28:51,025 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:51,025 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:51,025 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:51,025 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:51,025 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,025 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:51,025 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,025 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,025 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,025 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/kv1.txt" OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-08", hr="11") 2018-01-05T11:28:51,025 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112851_dfa05213-0aeb-4970-bcf2-3e1cf31bbdcd): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv1.txt" OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-08", hr="11") 2018-01-05T11:28:51,025 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,025 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv1.txt" OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-08", hr="11") 2018-01-05T11:28:51,026 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:51,026 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,026 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,027 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,027 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,027 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:51,027 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.LoadSemanticAnalyzer: file@null@/Users/djaiswal/hive/hive/data/files/kv1.txt 2018-01-05T11:28:51,027 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,027 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcpart 2018-01-05T11:28:51,027 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcpart 2018-01-05T11:28:51,031 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,031 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,032 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,033 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcpart[2008-04-08,11] 2018-01-05T11:28:51,033 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcpart[2008-04-08,11] 2018-01-05T11:28:51,036 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,043 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,044 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,046 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,046 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcpart[2008-04-08,11] 2018-01-05T11:28:51,046 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcpart[2008-04-08,11] 2018-01-05T11:28:51,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,051 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,052 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,052 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:51,052 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,052 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:51,052 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,052 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:51,052 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=1, getTable_(String, String, )=4, flushCache_()=0} 2018-01-05T11:28:51,052 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112851_dfa05213-0aeb-4970-bcf2-3e1cf31bbdcd); Time taken: 0.027 seconds 2018-01-05T11:28:51,052 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:51,052 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,052 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112851_dfa05213-0aeb-4970-bcf2-3e1cf31bbdcd): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv1.txt" OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-08", hr="11") 2018-01-05T11:28:51,053 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,053 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv1.txt" OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-08", hr="11") 2018-01-05T11:28:51,053 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: LOAD 2018-01-05T11:28:51,053 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/kv1.txt 2018-01-05T11:28:51,053 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@srcpart 2018-01-05T11:28:51,053 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,053 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,053 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,053 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,053 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MOVE] in serial mode 2018-01-05T11:28:51,053 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@20e06d7b, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:51,054 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:51,054 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,054 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:51,054 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:51,054 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:51,054 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:51,054 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,054 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,054 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:51,054 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:51,054 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,054 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:51,054 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,054 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:51,054 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:51,054 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,054 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Loading data to table default.srcpart partition (ds=2008-04-08, hr=11) from file:/Users/djaiswal/hive/hive/data/files/kv1.txt 2018-01-05T11:28:51,054 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,055 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:51,055 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:51,055 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:51,055 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:51,055 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,055 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,055 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,056 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,056 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,056 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:51,056 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:51,056 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,057 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:51,057 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,057 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcpart 2018-01-05T11:28:51,057 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcpart 2018-01-05T11:28:51,060 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,061 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,061 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,061 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcpart[2008-04-08,11] 2018-01-05T11:28:51,061 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcpart[2008-04-08,11] 2018-01-05T11:28:51,065 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000029, QueryId=djaiswal_20180105112850_4efbe87f-26c9-4872-b247-f9c909a06058-15, DagId=15, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180530863, EndTime=1515180531065 2018-01-05T11:28:51,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,067 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,068 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,068 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.MoveTask: Partition is: {ds=2008-04-08, hr=11} 2018-01-05T11:28:51,069 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,069 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: partition_name_has_valid_characters 2018-01-05T11:28:51,069 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=partition_name_has_valid_characters 2018-01-05T11:28:51,069 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,069 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,069 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcpart 2018-01-05T11:28:51,069 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcpart 2018-01-05T11:28:51,074 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,075 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcpart[2008-04-08,11] 2018-01-05T11:28:51,075 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcpart[2008-04-08,11] 2018-01-05T11:28:51,079 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,081 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,081 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,081 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,082 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11 2018-01-05T11:28:51,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/data/files/kv1.txt/ and the destination path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11/kv1.txt/ 2018-01-05T11:28:51,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11/kv1.txt/ and the destination path is /Users/djaiswal/hive/hive/data/files/kv1.txt/ 2018-01-05T11:28:51,130 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,131 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: altering partition for table srcpart with partition spec : {ds=2008-04-08, hr=11} 2018-01-05T11:28:51,131 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: No new files were created, and is not a replace, or we're inserting into a partition that does not exist yet. Skipping generating INSERT event. 2018-01-05T11:28:51,131 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Adding new partition {ds=2008-04-08, hr=11} 2018-01-05T11:28:51,131 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,131 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: add_partition : db=default tbl=srcpart 2018-01-05T11:28:51,131 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=add_partition : db=default tbl=srcpart 2018-01-05T11:28:51,144 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,146 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,158 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,160 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcpart[2008-04-08,11] 2018-01-05T11:28:51,160 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcpart[2008-04-08,11] 2018-01-05T11:28:51,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,177 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,178 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,178 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-1:STATS] in serial mode 2018-01-05T11:28:51,178 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@79e7f5b1, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:51,178 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:51,178 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,178 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:51,178 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:51,178 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:51,178 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:51,178 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,178 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,178 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:51,178 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:51,178 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,178 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:51,178 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,178 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:51,178 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:51,178 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,178 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:51,179 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,179 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:51,179 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:51,179 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:51,179 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:51,179 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,179 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,180 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,182 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,183 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,183 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:51,183 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:51,183 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,183 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:51,183 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,183 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcpart 2018-01-05T11:28:51,183 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcpart 2018-01-05T11:28:51,187 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,187 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,187 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,187 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcpart[2008-04-08,11] 2018-01-05T11:28:51,187 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcpart[2008-04-08,11] 2018-01-05T11:28:51,191 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,196 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,196 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,197 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Getting file stats of all partitions. threadpool size:15 2018-01-05T11:28:51,198 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Finished getting file stats of all partitions! 2018-01-05T11:28:51,198 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Partition {ds=2008-04-08, hr=11} stats: [numFiles=1, numRows=0, totalSize=5812, rawDataSize=0] 2018-01-05T11:28:51,198 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,198 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_partitions : db=default tbl=srcpart 2018-01-05T11:28:51,198 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_partitions : db=default tbl=srcpart 2018-01-05T11:28:51,198 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: New partition values:[2008-04-08, 11] 2018-01-05T11:28:51,201 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,212 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,216 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,217 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,217 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,217 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,217 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv1.txt" OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-08", hr="11") 2018-01-05T11:28:51,217 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: LOAD 2018-01-05T11:28:51,217 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/kv1.txt 2018-01-05T11:28:51,217 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@srcpart 2018-01-05T11:28:51,217 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@srcpart@ds=2008-04-08/hr=11 2018-01-05T11:28:51,217 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,217 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,217 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:51,217 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_partitions_(String, String, List, EnvironmentContext, )=19, getTable_(String, String, )=4, getPartitionWithAuthInfo_(String, String, List, String, List, )=9} 2018-01-05T11:28:51,217 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112851_dfa05213-0aeb-4970-bcf2-3e1cf31bbdcd); Time taken: 0.165 seconds 2018-01-05T11:28:51,217 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:51,217 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,217 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:51,217 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,217 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv1.txt" OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-08", hr="11") 2018-01-05T11:28:51,217 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.192 seconds 2018-01-05T11:28:51,217 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:51,217 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:51,217 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:51,217 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:51,218 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,218 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:51,218 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,218 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,218 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,218 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/kv1.txt" OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-08", hr="12") 2018-01-05T11:28:51,218 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112851_1019e609-d36c-4fb6-b693-fd3a2f69aa6b): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv1.txt" OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-08", hr="12") 2018-01-05T11:28:51,218 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,218 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv1.txt" OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-08", hr="12") 2018-01-05T11:28:51,218 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:51,218 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,218 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,218 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,218 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,218 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:51,218 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.LoadSemanticAnalyzer: file@null@/Users/djaiswal/hive/hive/data/files/kv1.txt 2018-01-05T11:28:51,218 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,218 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcpart 2018-01-05T11:28:51,218 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcpart 2018-01-05T11:28:51,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,222 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcpart[2008-04-08,12] 2018-01-05T11:28:51,222 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcpart[2008-04-08,12] 2018-01-05T11:28:51,226 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,228 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,228 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,230 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcpart[2008-04-08,12] 2018-01-05T11:28:51,230 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcpart[2008-04-08,12] 2018-01-05T11:28:51,234 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,235 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,235 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,236 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:51,236 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,236 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:51,236 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,236 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:51,236 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=4, flushCache_()=0} 2018-01-05T11:28:51,236 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112851_1019e609-d36c-4fb6-b693-fd3a2f69aa6b); Time taken: 0.018 seconds 2018-01-05T11:28:51,236 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:51,236 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,236 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112851_1019e609-d36c-4fb6-b693-fd3a2f69aa6b): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv1.txt" OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-08", hr="12") 2018-01-05T11:28:51,237 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,237 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv1.txt" OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-08", hr="12") 2018-01-05T11:28:51,237 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: LOAD 2018-01-05T11:28:51,237 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/kv1.txt 2018-01-05T11:28:51,237 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@srcpart 2018-01-05T11:28:51,237 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,237 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,237 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,237 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,237 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MOVE] in serial mode 2018-01-05T11:28:51,237 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@96c760b, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:51,237 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:51,237 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,237 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:51,237 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:51,237 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:51,237 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:51,237 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,237 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,237 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:51,237 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:51,237 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,237 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:51,237 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,237 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:51,237 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:51,237 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,237 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Loading data to table default.srcpart partition (ds=2008-04-08, hr=12) from file:/Users/djaiswal/hive/hive/data/files/kv1.txt 2018-01-05T11:28:51,238 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,238 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:51,238 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:51,238 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:51,238 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:51,238 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,238 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,238 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,239 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,240 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,240 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:51,240 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:51,240 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,240 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:51,240 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,240 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcpart 2018-01-05T11:28:51,240 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcpart 2018-01-05T11:28:51,243 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,244 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,244 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,244 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcpart[2008-04-08,12] 2018-01-05T11:28:51,244 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcpart[2008-04-08,12] 2018-01-05T11:28:51,248 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,249 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,250 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,251 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.MoveTask: Partition is: {ds=2008-04-08, hr=12} 2018-01-05T11:28:51,251 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,251 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: partition_name_has_valid_characters 2018-01-05T11:28:51,251 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=partition_name_has_valid_characters 2018-01-05T11:28:51,251 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,251 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,251 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcpart 2018-01-05T11:28:51,251 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcpart 2018-01-05T11:28:51,255 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,256 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,256 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,256 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcpart[2008-04-08,12] 2018-01-05T11:28:51,256 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcpart[2008-04-08,12] 2018-01-05T11:28:51,259 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,261 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,263 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,263 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,263 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12 2018-01-05T11:28:51,276 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/data/files/kv1.txt/ and the destination path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12/kv1.txt/ 2018-01-05T11:28:51,276 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12/kv1.txt/ and the destination path is /Users/djaiswal/hive/hive/data/files/kv1.txt/ 2018-01-05T11:28:51,301 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,301 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: altering partition for table srcpart with partition spec : {ds=2008-04-08, hr=12} 2018-01-05T11:28:51,301 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: No new files were created, and is not a replace, or we're inserting into a partition that does not exist yet. Skipping generating INSERT event. 2018-01-05T11:28:51,302 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Adding new partition {ds=2008-04-08, hr=12} 2018-01-05T11:28:51,302 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,302 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: add_partition : db=default tbl=srcpart 2018-01-05T11:28:51,302 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=add_partition : db=default tbl=srcpart 2018-01-05T11:28:51,311 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,312 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,312 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,312 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcpart[2008-04-08,12] 2018-01-05T11:28:51,312 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcpart[2008-04-08,12] 2018-01-05T11:28:51,315 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,320 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,320 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,321 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-1:STATS] in serial mode 2018-01-05T11:28:51,321 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@7c095332, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:51,321 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:51,321 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,321 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:51,321 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:51,321 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:51,321 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:51,321 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,321 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,321 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:51,321 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:51,321 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,321 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:51,321 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,321 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:51,321 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:51,321 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,321 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:51,321 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,322 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:51,322 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:51,322 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:51,322 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:51,322 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,322 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,322 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,323 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,323 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,323 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:51,323 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:51,323 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,324 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:51,324 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,324 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcpart 2018-01-05T11:28:51,324 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcpart 2018-01-05T11:28:51,327 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,327 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,327 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,327 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcpart[2008-04-08,12] 2018-01-05T11:28:51,327 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcpart[2008-04-08,12] 2018-01-05T11:28:51,330 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,336 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,337 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,337 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Getting file stats of all partitions. threadpool size:15 2018-01-05T11:28:51,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Finished getting file stats of all partitions! 2018-01-05T11:28:51,338 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Partition {ds=2008-04-08, hr=12} stats: [numFiles=1, numRows=0, totalSize=5812, rawDataSize=0] 2018-01-05T11:28:51,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,338 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_partitions : db=default tbl=srcpart 2018-01-05T11:28:51,338 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_partitions : db=default tbl=srcpart 2018-01-05T11:28:51,338 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: New partition values:[2008-04-08, 12] 2018-01-05T11:28:51,342 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,348 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,352 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,353 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,353 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,353 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,353 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv1.txt" OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-08", hr="12") 2018-01-05T11:28:51,353 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: LOAD 2018-01-05T11:28:51,353 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/kv1.txt 2018-01-05T11:28:51,353 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@srcpart 2018-01-05T11:28:51,353 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@srcpart@ds=2008-04-08/hr=12 2018-01-05T11:28:51,353 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,353 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,353 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:51,353 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_partitions_(String, String, List, EnvironmentContext, )=15, getTable_(String, String, )=3, getPartitionWithAuthInfo_(String, String, List, String, List, )=10} 2018-01-05T11:28:51,353 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112851_1019e609-d36c-4fb6-b693-fd3a2f69aa6b); Time taken: 0.117 seconds 2018-01-05T11:28:51,353 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:51,353 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,353 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:51,353 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,353 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv1.txt" OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-08", hr="12") 2018-01-05T11:28:51,353 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.135 seconds 2018-01-05T11:28:51,353 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:51,353 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:51,353 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:51,353 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:51,353 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,353 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:51,353 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,353 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,353 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,353 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/kv1.txt" OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-09", hr="11") 2018-01-05T11:28:51,354 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112851_49ec6444-a366-4035-8e9b-4c8bdbfc59d1): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv1.txt" OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-09", hr="11") 2018-01-05T11:28:51,354 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,354 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv1.txt" OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-09", hr="11") 2018-01-05T11:28:51,354 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:51,354 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,354 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,354 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,354 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,354 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:51,354 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.LoadSemanticAnalyzer: file@null@/Users/djaiswal/hive/hive/data/files/kv1.txt 2018-01-05T11:28:51,354 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,354 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcpart 2018-01-05T11:28:51,354 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcpart 2018-01-05T11:28:51,357 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,358 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,358 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,358 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcpart[2008-04-09,11] 2018-01-05T11:28:51,358 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcpart[2008-04-09,11] 2018-01-05T11:28:51,360 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,363 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcpart[2008-04-09,11] 2018-01-05T11:28:51,363 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcpart[2008-04-09,11] 2018-01-05T11:28:51,367 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,368 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,369 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,369 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:51,369 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,369 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:51,369 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,369 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:51,369 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=4, flushCache_()=0} 2018-01-05T11:28:51,369 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112851_49ec6444-a366-4035-8e9b-4c8bdbfc59d1); Time taken: 0.016 seconds 2018-01-05T11:28:51,369 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:51,369 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,370 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112851_49ec6444-a366-4035-8e9b-4c8bdbfc59d1): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv1.txt" OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-09", hr="11") 2018-01-05T11:28:51,371 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,371 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv1.txt" OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-09", hr="11") 2018-01-05T11:28:51,371 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: LOAD 2018-01-05T11:28:51,371 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/kv1.txt 2018-01-05T11:28:51,371 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@srcpart 2018-01-05T11:28:51,371 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,371 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,371 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,371 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,371 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MOVE] in serial mode 2018-01-05T11:28:51,371 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@79f6e671, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:51,371 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:51,371 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,371 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:51,371 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:51,371 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:51,371 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:51,371 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,371 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,371 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:51,371 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:51,371 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,371 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:51,371 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,371 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:51,371 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:51,371 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,371 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Loading data to table default.srcpart partition (ds=2008-04-09, hr=11) from file:/Users/djaiswal/hive/hive/data/files/kv1.txt 2018-01-05T11:28:51,371 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,372 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:51,372 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:51,372 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:51,372 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:51,372 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,372 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,372 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,373 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,373 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,373 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:51,373 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:51,373 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,373 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:51,373 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,374 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcpart 2018-01-05T11:28:51,374 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcpart 2018-01-05T11:28:51,377 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,378 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,378 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,378 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcpart[2008-04-09,11] 2018-01-05T11:28:51,378 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcpart[2008-04-09,11] 2018-01-05T11:28:51,381 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,383 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,383 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,384 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.MoveTask: Partition is: {ds=2008-04-09, hr=11} 2018-01-05T11:28:51,384 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,384 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: partition_name_has_valid_characters 2018-01-05T11:28:51,384 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=partition_name_has_valid_characters 2018-01-05T11:28:51,384 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,384 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,384 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcpart 2018-01-05T11:28:51,384 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcpart 2018-01-05T11:28:51,387 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,387 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,388 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,388 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcpart[2008-04-09,11] 2018-01-05T11:28:51,388 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcpart[2008-04-09,11] 2018-01-05T11:28:51,391 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,392 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,392 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,392 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,393 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11 2018-01-05T11:28:51,418 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/data/files/kv1.txt/ and the destination path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11/kv1.txt/ 2018-01-05T11:28:51,418 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11/kv1.txt/ and the destination path is /Users/djaiswal/hive/hive/data/files/kv1.txt/ 2018-01-05T11:28:51,440 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,440 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: altering partition for table srcpart with partition spec : {ds=2008-04-09, hr=11} 2018-01-05T11:28:51,440 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: No new files were created, and is not a replace, or we're inserting into a partition that does not exist yet. Skipping generating INSERT event. 2018-01-05T11:28:51,440 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Adding new partition {ds=2008-04-09, hr=11} 2018-01-05T11:28:51,440 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,440 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: add_partition : db=default tbl=srcpart 2018-01-05T11:28:51,440 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=add_partition : db=default tbl=srcpart 2018-01-05T11:28:51,441 DEBUG [AMReporterQueueDrainer] impl.AMReporter: Removing am 10.22.8.182:49824 with last associated dag QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=2} from heartbeat with taskCount=0, amFailed=false, isDone=true 2018-01-05T11:28:51,449 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,450 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,450 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,450 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcpart[2008-04-09,11] 2018-01-05T11:28:51,450 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcpart[2008-04-09,11] 2018-01-05T11:28:51,453 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,458 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,458 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,459 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-1:STATS] in serial mode 2018-01-05T11:28:51,459 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@393fe097, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:51,459 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:51,459 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,459 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:51,459 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:51,459 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:51,459 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:51,459 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,459 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,459 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:51,459 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:51,459 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,459 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:51,459 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,459 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:51,459 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:51,459 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,459 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:51,459 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,460 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:51,460 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:51,460 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:51,460 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:51,460 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,460 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,460 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,461 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,461 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,461 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:51,461 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:51,461 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,461 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:51,461 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,461 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcpart 2018-01-05T11:28:51,461 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcpart 2018-01-05T11:28:51,464 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,464 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,464 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,464 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcpart[2008-04-09,11] 2018-01-05T11:28:51,464 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcpart[2008-04-09,11] 2018-01-05T11:28:51,467 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Getting file stats of all partitions. threadpool size:15 2018-01-05T11:28:51,472 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Finished getting file stats of all partitions! 2018-01-05T11:28:51,472 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Partition {ds=2008-04-09, hr=11} stats: [numFiles=1, numRows=0, totalSize=5812, rawDataSize=0] 2018-01-05T11:28:51,472 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,472 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_partitions : db=default tbl=srcpart 2018-01-05T11:28:51,472 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_partitions : db=default tbl=srcpart 2018-01-05T11:28:51,472 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: New partition values:[2008-04-09, 11] 2018-01-05T11:28:51,475 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,481 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,484 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,485 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv1.txt" OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-09", hr="11") 2018-01-05T11:28:51,485 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: LOAD 2018-01-05T11:28:51,485 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/kv1.txt 2018-01-05T11:28:51,485 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@srcpart 2018-01-05T11:28:51,485 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@srcpart@ds=2008-04-09/hr=11 2018-01-05T11:28:51,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,485 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:51,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_partitions_(String, String, List, EnvironmentContext, )=13, getTable_(String, String, )=3, getPartitionWithAuthInfo_(String, String, List, String, List, )=7} 2018-01-05T11:28:51,485 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112851_49ec6444-a366-4035-8e9b-4c8bdbfc59d1); Time taken: 0.116 seconds 2018-01-05T11:28:51,485 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:51,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,485 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:51,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv1.txt" OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-09", hr="11") 2018-01-05T11:28:51,485 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.132 seconds 2018-01-05T11:28:51,485 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:51,485 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:51,485 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:51,485 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:51,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:51,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/kv1.txt" OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-09", hr="12") 2018-01-05T11:28:51,485 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112851_c1cc8d40-205d-4a44-a946-7ef55e279bb0): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv1.txt" OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-09", hr="12") 2018-01-05T11:28:51,486 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,486 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv1.txt" OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-09", hr="12") 2018-01-05T11:28:51,486 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:51,486 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,486 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,486 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,486 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,486 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:51,486 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.LoadSemanticAnalyzer: file@null@/Users/djaiswal/hive/hive/data/files/kv1.txt 2018-01-05T11:28:51,486 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,486 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcpart 2018-01-05T11:28:51,486 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcpart 2018-01-05T11:28:51,489 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,490 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,490 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,490 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcpart[2008-04-09,12] 2018-01-05T11:28:51,490 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcpart[2008-04-09,12] 2018-01-05T11:28:51,493 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,494 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,494 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,496 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,496 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcpart[2008-04-09,12] 2018-01-05T11:28:51,496 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcpart[2008-04-09,12] 2018-01-05T11:28:51,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,500 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,500 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,500 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:51,500 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,500 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:51,500 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,500 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:51,500 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=4, flushCache_()=0} 2018-01-05T11:28:51,500 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112851_c1cc8d40-205d-4a44-a946-7ef55e279bb0); Time taken: 0.015 seconds 2018-01-05T11:28:51,500 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:51,500 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,500 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112851_c1cc8d40-205d-4a44-a946-7ef55e279bb0): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv1.txt" OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-09", hr="12") 2018-01-05T11:28:51,501 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,501 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv1.txt" OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-09", hr="12") 2018-01-05T11:28:51,502 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: LOAD 2018-01-05T11:28:51,502 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/kv1.txt 2018-01-05T11:28:51,502 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@srcpart 2018-01-05T11:28:51,502 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,502 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,502 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,502 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,502 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MOVE] in serial mode 2018-01-05T11:28:51,502 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@66da0ab7, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:51,502 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:51,502 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,502 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:51,502 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:51,502 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:51,502 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:51,502 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,502 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,502 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:51,502 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:51,502 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,502 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:51,502 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,502 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:51,502 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:51,502 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,502 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Loading data to table default.srcpart partition (ds=2008-04-09, hr=12) from file:/Users/djaiswal/hive/hive/data/files/kv1.txt 2018-01-05T11:28:51,503 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,503 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:51,503 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:51,503 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:51,503 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:51,503 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,503 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,503 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,504 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,504 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,504 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:51,504 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:51,504 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,504 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:51,504 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,504 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcpart 2018-01-05T11:28:51,504 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcpart 2018-01-05T11:28:51,507 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,507 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,508 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,508 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcpart[2008-04-09,12] 2018-01-05T11:28:51,508 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcpart[2008-04-09,12] 2018-01-05T11:28:51,510 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,512 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,512 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,513 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.MoveTask: Partition is: {ds=2008-04-09, hr=12} 2018-01-05T11:28:51,513 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,513 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: partition_name_has_valid_characters 2018-01-05T11:28:51,513 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=partition_name_has_valid_characters 2018-01-05T11:28:51,513 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,513 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,513 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcpart 2018-01-05T11:28:51,513 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcpart 2018-01-05T11:28:51,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,516 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcpart[2008-04-09,12] 2018-01-05T11:28:51,516 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcpart[2008-04-09,12] 2018-01-05T11:28:51,519 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,521 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,521 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,521 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12 2018-01-05T11:28:51,533 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/data/files/kv1.txt/ and the destination path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12/kv1.txt/ 2018-01-05T11:28:51,533 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12/kv1.txt/ and the destination path is /Users/djaiswal/hive/hive/data/files/kv1.txt/ 2018-01-05T11:28:51,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: altering partition for table srcpart with partition spec : {ds=2008-04-09, hr=12} 2018-01-05T11:28:51,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: No new files were created, and is not a replace, or we're inserting into a partition that does not exist yet. Skipping generating INSERT event. 2018-01-05T11:28:51,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Adding new partition {ds=2008-04-09, hr=12} 2018-01-05T11:28:51,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,556 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: add_partition : db=default tbl=srcpart 2018-01-05T11:28:51,556 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=add_partition : db=default tbl=srcpart 2018-01-05T11:28:51,563 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,564 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,564 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,564 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcpart[2008-04-09,12] 2018-01-05T11:28:51,564 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcpart[2008-04-09,12] 2018-01-05T11:28:51,567 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,570 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,571 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,571 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-1:STATS] in serial mode 2018-01-05T11:28:51,571 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@754e3050, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:51,571 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:51,571 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,571 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:51,571 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:51,571 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:51,571 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:51,571 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,571 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,571 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:51,571 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:51,571 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,571 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:51,571 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,571 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:51,571 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:51,571 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,571 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:51,572 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,572 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:51,572 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:51,572 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:51,572 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:51,572 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,572 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,572 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,573 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,573 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,573 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:51,573 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:51,573 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,573 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:51,573 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,573 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcpart 2018-01-05T11:28:51,573 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcpart 2018-01-05T11:28:51,576 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,576 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,577 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,577 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcpart[2008-04-09,12] 2018-01-05T11:28:51,577 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcpart[2008-04-09,12] 2018-01-05T11:28:51,579 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,583 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,584 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,584 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Getting file stats of all partitions. threadpool size:15 2018-01-05T11:28:51,584 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Finished getting file stats of all partitions! 2018-01-05T11:28:51,584 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Partition {ds=2008-04-09, hr=12} stats: [numFiles=1, numRows=0, totalSize=5812, rawDataSize=0] 2018-01-05T11:28:51,585 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,585 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_partitions : db=default tbl=srcpart 2018-01-05T11:28:51,585 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_partitions : db=default tbl=srcpart 2018-01-05T11:28:51,585 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: New partition values:[2008-04-09, 12] 2018-01-05T11:28:51,587 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,592 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,595 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,596 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,596 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,596 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,596 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv1.txt" OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-09", hr="12") 2018-01-05T11:28:51,596 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: LOAD 2018-01-05T11:28:51,596 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/kv1.txt 2018-01-05T11:28:51,596 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@srcpart 2018-01-05T11:28:51,596 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@srcpart@ds=2008-04-09/hr=12 2018-01-05T11:28:51,596 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,596 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,596 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:51,596 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_partitions_(String, String, List, EnvironmentContext, )=12, getTable_(String, String, )=4, getPartitionWithAuthInfo_(String, String, List, String, List, )=7} 2018-01-05T11:28:51,596 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112851_c1cc8d40-205d-4a44-a946-7ef55e279bb0); Time taken: 0.096 seconds 2018-01-05T11:28:51,596 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:51,596 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,596 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:51,596 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,596 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/kv1.txt" OVERWRITE INTO TABLE srcpart PARTITION (ds="2008-04-09", hr="12") 2018-01-05T11:28:51,596 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.111 seconds 2018-01-05T11:28:51,596 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:51,596 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:51,596 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:51,596 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:51,596 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,596 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:51,596 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,596 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,596 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,596 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: ANALYZE TABLE srcpart PARTITION(ds, hr) COMPUTE STATISTICS 2018-01-05T11:28:51,596 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112851_c7191e3d-978b-4094-ad5d-df22da119ed8): ANALYZE TABLE srcpart PARTITION(ds, hr) COMPUTE STATISTICS 2018-01-05T11:28:51,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: ANALYZE TABLE srcpart PARTITION(ds, hr) COMPUTE STATISTICS 2018-01-05T11:28:51,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:51,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,597 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:51,597 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Invoking analyze on original query 2018-01-05T11:28:51,597 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Starting Semantic Analysis 2018-01-05T11:28:51,597 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed phase 1 of Semantic Analysis 2018-01-05T11:28:51,597 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for source tables 2018-01-05T11:28:51,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,597 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcpart 2018-01-05T11:28:51,597 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcpart 2018-01-05T11:28:51,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,600 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcpart 2018-01-05T11:28:51,600 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcpart 2018-01-05T11:28:51,603 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,604 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,604 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.BaseSemanticAnalyzer: Dynamic partitioning is used; only validating 0 columns 2018-01-05T11:28:51,604 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,604 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcpart 2018-01-05T11:28:51,604 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcpart 2018-01-05T11:28:51,606 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,607 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,607 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,607 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partitions_names_ps : db=default tbl=srcpart[,] 2018-01-05T11:28:51,607 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions_names_ps : db=default tbl=srcpart[,] 2018-01-05T11:28:51,610 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,620 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,620 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,621 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,621 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partitions_by_names : db=default tbl=srcpart 2018-01-05T11:28:51,621 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions_by_names : db=default tbl=srcpart 2018-01-05T11:28:51,624 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,630 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 4.279909ms + 0.006128ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? where "PARTITIONS"."PART_NAME" in (?,?,?,?)] 2018-01-05T11:28:51,635 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 4.187075ms + 1.050286ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (1,2,3,4) order by "PART_NAME" asc] 2018-01-05T11:28:51,640 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 4.451617ms + 0.446938ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (1,2,3,4) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:28:51,642 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 1.752772ms + 0.091393ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (1,2,3,4) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:51,644 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 2.09283ms + 0.015654ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (11,12,13,14) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:28:51,646 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 2.014251ms + 0.017338ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (11,12,13,14) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:51,649 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 2.603414ms + 0.022821ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (11,12,13,14) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:51,652 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 2.372533ms + 0.017636ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (11,12,13,14) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:51,654 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 2.000447ms + 0.094443ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (10) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:51,656 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 2.157857ms + 0.234663ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (11,12,13,14) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:28:51,656 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,676 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,678 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,678 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for subqueries 2018-01-05T11:28:51,678 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for destination tables 2018-01-05T11:28:51,678 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed getting MetaData in Semantic Analysis 2018-01-05T11:28:51,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-51_597_8151257429790130914-1/-mr-10000 2018-01-05T11:28:51,691 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: XXX: adding part: srcpart(ds=2008-04-08/hr=11) 2018-01-05T11:28:51,691 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: XXX: adding part: srcpart(ds=2008-04-08/hr=12) 2018-01-05T11:28:51,691 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: XXX: adding part: srcpart(ds=2008-04-09/hr=11) 2018-01-05T11:28:51,691 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: XXX: adding part: srcpart(ds=2008-04-09/hr=12) 2018-01-05T11:28:51,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Table Plan for srcpart TS[0] 2018-01-05T11:28:51,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Body Plan for Query Block null 2018-01-05T11:28:51,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Plan for Query Block null 2018-01-05T11:28:51,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Before logical optimization TS[0] 2018-01-05T11:28:51,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] lineage.Generator: Time taken for lineage transform=0 2018-01-05T11:28:51,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{srcpart}key,value: string|{srcpart}value,ds: string|{srcpart}ds,hr: string|{srcpart}hr,BLOCK__OFFSET__INSIDE__FILE: bigint|{srcpart}block__offset__inside__file,INPUT__FILE__NAME: string|{srcpart}input__file__name,ROW__ID: struct|{srcpart}row__id,RAW__DATA__SIZE: bigint|{srcpart}raw__data__size) 2018-01-05T11:28:51,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,691 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:28:51,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.PredicatePushDown: After PPD: TS[0] 2018-01-05T11:28:51,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{srcpart}key,value: string|{srcpart}value,ds: string|{srcpart}ds,hr: string|{srcpart}hr,BLOCK__OFFSET__INSIDE__FILE: bigint|{srcpart}block__offset__inside__file,INPUT__FILE__NAME: string|{srcpart}input__file__name,ROW__ID: struct|{srcpart}row__id,RAW__DATA__SIZE: bigint|{srcpart}raw__data__size) 2018-01-05T11:28:51,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,691 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{srcpart}key,value: string|{srcpart}value,ds: string|{srcpart}ds,hr: string|{srcpart}hr,BLOCK__OFFSET__INSIDE__FILE: bigint|{srcpart}block__offset__inside__file,INPUT__FILE__NAME: string|{srcpart}input__file__name,ROW__ID: struct|{srcpart}row__id,RAW__DATA__SIZE: bigint|{srcpart}raw__data__size) 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: After logical optimization TS[0] 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,692 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,693 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,693 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,693 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partitions : db=default tbl=srcpart 2018-01-05T11:28:51,693 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions : db=default tbl=srcpart 2018-01-05T11:28:51,696 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,700 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 4.342665ms + 0.005151ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? ] 2018-01-05T11:28:51,701 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.36179ms + 0.165236ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (1,2,3,4) order by "PART_NAME" asc] 2018-01-05T11:28:51,702 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.156709ms + 0.2955ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (1,2,3,4) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:28:51,702 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.191504ms + 0.042837ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (1,2,3,4) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:51,702 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.146179ms + 0.010332ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (11,12,13,14) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:28:51,702 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.101303ms + 0.00896ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (11,12,13,14) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:51,702 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.0937ms + 0.008199ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (11,12,13,14) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:51,702 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.097359ms + 0.007976ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (11,12,13,14) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:51,703 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.139607ms + 0.028461ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (10) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:51,703 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.088653ms + 0.093371ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (11,12,13,14) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:28:51,703 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,728 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,729 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,730 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,730 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting stats (Num rows: 1 Data size: 232488 Basic stats: PARTIAL Column stats: NONE) on TS[0] 2018-01-05T11:28:51,730 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (srcpart): numRows: 1 dataSize: 232488 basicStatsState: PARTIAL colStatsState: NONE colStats: {} 2018-01-05T11:28:51,730 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on TS[0] 2018-01-05T11:28:51,730 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,730 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,730 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,730 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,730 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,730 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,730 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,730 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,730 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,730 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,730 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,730 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,730 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,730 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,730 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,730 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,730 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,730 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,730 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,730 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,730 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:51,730 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:28:51,730 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Cycle free: true 2018-01-05T11:28:51,730 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,730 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,730 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,739 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{srcpart}key,value: string|{srcpart}value,ds: string|{srcpart}ds,hr: string|{srcpart}hr,BLOCK__OFFSET__INSIDE__FILE: bigint|{srcpart}block__offset__inside__file,INPUT__FILE__NAME: string|{srcpart}input__file__name,ROW__ID: struct|{srcpart}row__id,RAW__DATA__SIZE: bigint|{srcpart}raw__data__size) 2018-01-05T11:28:51,739 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,739 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:28:51,740 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12 of table srcpart 2018-01-05T11:28:51,740 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11 of table srcpart 2018-01-05T11:28:51,740 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11 of table srcpart 2018-01-05T11:28:51,740 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12 of table srcpart 2018-01-05T11:28:51,740 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12 2018-01-05T11:28:51,740 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11 2018-01-05T11:28:51,740 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11 2018-01-05T11:28:51,740 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12 2018-01-05T11:28:51,740 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:28:51,740 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,741 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,741 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:51,741 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:51,741 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:51,741 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:51,741 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:51,741 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:51,741 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:28:51,741 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping vectorization 2018-01-05T11:28:51,741 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:28:51,741 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: llap mode: only 2018-01-05T11:28:51,741 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:28:51,741 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:51,741 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@2b15d066 can be done in LLAP 2018-01-05T11:28:51,741 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,741 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed plan generation 2018-01-05T11:28:51,741 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:51,741 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validation start 2018-01-05T11:28:51,741 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Not a partition. 2018-01-05T11:28:51,741 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validated ds=2008-04-08/hr=11 2018-01-05T11:28:51,741 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: srcpart 2018-01-05T11:28:51,742 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,742 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partitions_ps_with_auth : db=default tbl=srcpart[2008-04-08,11] 2018-01-05T11:28:51,742 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions_ps_with_auth : db=default tbl=srcpart[2008-04-08,11] 2018-01-05T11:28:51,745 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,764 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcpart[2008-04-08,11] 2018-01-05T11:28:51,764 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcpart[2008-04-08,11] 2018-01-05T11:28:51,768 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,773 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,774 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,774 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validated ds=2008-04-08/hr=12 2018-01-05T11:28:51,774 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: srcpart 2018-01-05T11:28:51,774 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,774 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partitions_ps_with_auth : db=default tbl=srcpart[2008-04-08,12] 2018-01-05T11:28:51,774 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions_ps_with_auth : db=default tbl=srcpart[2008-04-08,12] 2018-01-05T11:28:51,777 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,788 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,790 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,790 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,790 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcpart[2008-04-08,12] 2018-01-05T11:28:51,790 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcpart[2008-04-08,12] 2018-01-05T11:28:51,793 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,797 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,797 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,797 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validated ds=2008-04-09/hr=11 2018-01-05T11:28:51,797 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: srcpart 2018-01-05T11:28:51,798 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,798 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partitions_ps_with_auth : db=default tbl=srcpart[2008-04-09,11] 2018-01-05T11:28:51,798 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions_ps_with_auth : db=default tbl=srcpart[2008-04-09,11] 2018-01-05T11:28:51,801 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,811 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,812 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,812 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,812 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcpart[2008-04-09,11] 2018-01-05T11:28:51,812 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcpart[2008-04-09,11] 2018-01-05T11:28:51,815 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,819 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,819 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,819 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validated ds=2008-04-09/hr=12 2018-01-05T11:28:51,819 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: srcpart 2018-01-05T11:28:51,819 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,819 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partitions_ps_with_auth : db=default tbl=srcpart[2008-04-09,12] 2018-01-05T11:28:51,819 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions_ps_with_auth : db=default tbl=srcpart[2008-04-09,12] 2018-01-05T11:28:51,822 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,839 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,843 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,843 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,843 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcpart[2008-04-09,12] 2018-01-05T11:28:51,843 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcpart[2008-04-09,12] 2018-01-05T11:28:51,845 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,849 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:51,850 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:51,850 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,850 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:srcpart.key, type:string, comment:null), FieldSchema(name:srcpart.value, type:string, comment:null), FieldSchema(name:srcpart.ds, type:string, comment:null), FieldSchema(name:srcpart.hr, type:string, comment:null)], properties:null) 2018-01-05T11:28:51,850 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,850 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:51,850 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, listPartitionNames_(String, String, List, short, )=13, getTable_(String, String, )=10, flushCache_()=0, getPartitionsByNames_(String, String, List, )=57, getPartitionWithAuthInfo_(String, String, List, String, List, )=31, listPartitions_(String, String, short, )=36, listPartitionsWithAuthInfo_(String, String, List, short, String, List, )=76} 2018-01-05T11:28:51,850 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112851_c7191e3d-978b-4094-ad5d-df22da119ed8); Time taken: 0.254 seconds 2018-01-05T11:28:51,850 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:51,850 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,850 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112851_c7191e3d-978b-4094-ad5d-df22da119ed8): ANALYZE TABLE srcpart PARTITION(ds, hr) COMPUTE STATISTICS 2018-01-05T11:28:51,851 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,851 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: ANALYZE TABLE srcpart PARTITION(ds, hr) COMPUTE STATISTICS 2018-01-05T11:28:51,851 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:28:51,851 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@srcpart 2018-01-05T11:28:51,851 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@srcpart@ds=2008-04-08/hr=11 2018-01-05T11:28:51,851 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@srcpart@ds=2008-04-08/hr=12 2018-01-05T11:28:51,851 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@srcpart@ds=2008-04-09/hr=11 2018-01-05T11:28:51,851 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@srcpart@ds=2008-04-09/hr=12 2018-01-05T11:28:51,851 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@srcpart 2018-01-05T11:28:51,851 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@srcpart@ds=2008-04-08/hr=11 2018-01-05T11:28:51,851 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@srcpart@ds=2008-04-08/hr=12 2018-01-05T11:28:51,851 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@srcpart@ds=2008-04-09/hr=11 2018-01-05T11:28:51,851 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@srcpart@ds=2008-04-09/hr=12 2018-01-05T11:28:51,851 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,851 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,851 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,852 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Query ID = djaiswal_20180105112851_c7191e3d-978b-4094-ad5d-df22da119ed8 2018-01-05T11:28:51,852 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Total jobs = 1 2018-01-05T11:28:51,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,852 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:28:51,852 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MAPRED] in serial mode 2018-01-05T11:28:51,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-51_597_8151257429790130914-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:28:51,890 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:28:51,890 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:28:51,890 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:28:51,890 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112851_c7191e3d-978b-4094-ad5d-df22da119ed8 2018-01-05T11:28:51,890 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Session is already open 2018-01-05T11:28:51,890 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:28:51,890 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,890 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Dag name: ANALYZE TABLE srcpart PARTITION...STATISTICS (Stage-0) 2018-01-05T11:28:51,890 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: DagInfo: {"context":"Hive","description":"\n\nANALYZE TABLE srcpart PARTITION(ds, hr) COMPUTE STATISTICS"} 2018-01-05T11:28:51,890 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112851_c7191e3d-978b-4094-ad5d-df22da119ed8 with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:28:51,890 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,903 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:28:51,903 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,903 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:28:51,904 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,904 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 2.03KB 2018-01-05T11:28:51,910 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-51_597_8151257429790130914-1/-mr-10000 2018-01-05T11:28:51,924 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-51_597_8151257429790130914-1/-mr-10000 2018-01-05T11:28:51,926 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11 2018-01-05T11:28:51,926 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12 2018-01-05T11:28:51,926 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11 2018-01-05T11:28:51,926 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12 2018-01-05T11:28:51,926 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,926 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,926 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,926 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=ANALYZE TABLE srcpart PARTITION...STATISTICS (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112851_c7191e3d-978b-4094-ad5d-df22da119ed8 } 2018-01-05T11:28:51,926 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:51,928 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: ANALYZE TABLE srcpart PARTITION...STATISTICS (Stage-0) 2018-01-05T11:28:51,929 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:28:51,929 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_16, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_16.dot 2018-01-05T11:28:51,930 INFO [IPC Server handler 0 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_16 2018-01-05T11:28:51,930 WARN [IPC Server handler 0 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:51,930 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Running DAG: ANALYZE TABLE srcpart PARTITION...STATISTICS (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112851_c7191e3d-978b-4094-ad5d-df22da119ed8 } 2018-01-05T11:28:51,956 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_16][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_16, submitTime=1515180531928, queueName=null 2018-01-05T11:28:51,957 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_16_00 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_16_00 [Map 1], Default Resources= 2018-01-05T11:28:51,957 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_16_00 [Map 1] 2018-01-05T11:28:51,957 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_16_00 [Map 1] 2018-01-05T11:28:51,957 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:28:51,957 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_16_00 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:51,957 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:28:51,957 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_16][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_16, initTime=1515180531957 2018-01-05T11:28:51,957 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: dag_1515180518813_0001_16 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:28:51,958 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:28:51,958 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_16][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_16, startTime=1515180531958 2018-01-05T11:28:51,958 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_16 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:28:51,958 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {srcpart={InputName=srcpart}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:28:51,958 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: srcpart, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:28:51,958 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_16_00 [Map 1] 2018-01-05T11:28:51,958 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_16, dagName=ANALYZE TABLE srcpart PARTITION...STATISTICS (Stage-0) 2018-01-05T11:28:51,958 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,958 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,958 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:51,963 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_16_00 [Map 1] 2018-01-05T11:28:51,963 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_16_00 [Map 1] 2018-01-05T11:28:51,963 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:51,963 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_16_00 [Map 1] 2018-01-05T11:28:51,965 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:28:51,965 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@31a910fd 2018-01-05T11:28:51,966 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-51_597_8151257429790130914-1/djaiswal/_tez_scratch_dir/6896f735-dbb7-4bbd-a14f-f8a1d5bcb679/map.xml 2018-01-05T11:28:51,966 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-51_597_8151257429790130914-1/djaiswal/_tez_scratch_dir/6896f735-dbb7-4bbd-a14f-f8a1d5bcb679/map.xml 2018-01-05T11:28:51,966 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-51_597_8151257429790130914-1/djaiswal/_tez_scratch_dir/6896f735-dbb7-4bbd-a14f-f8a1d5bcb679/map.xml 2018-01-05T11:28:51,966 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:51,966 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:51,966 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:51,966 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:51,966 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 2.03KB 2018-01-05T11:28:51,966 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_16_00 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:28:51,966 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: srcpart on vertex vertex_1515180518813_0001_16_00 [Map 1] 2018-01-05T11:28:51,966 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:28:51,966 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:51,966 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:28:51,967 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:51,967 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-51_597_8151257429790130914-1/djaiswal/_tez_scratch_dir/6896f735-dbb7-4bbd-a14f-f8a1d5bcb679/map.xml 2018-01-05T11:28:51,967 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:51,967 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias srcpart 2018-01-05T11:28:51,967 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12 2018-01-05T11:28:51,967 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 4 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12 2018-01-05T11:28:51,967 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11 2018-01-05T11:28:51,967 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11 2018-01-05T11:28:51,967 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12 2018-01-05T11:28:51,967 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 2018-01-05T11:28:51,967 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = 2018-01-05T11:28:51,967 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12 2018-01-05T11:28:51,967 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:51,967 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:51,967 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:51,967 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:52,021 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Time taken to get FileStatuses: 53 2018-01-05T11:28:52,021 INFO [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total input files to process : 4 2018-01-05T11:28:52,021 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total # of splits generated by getSplits: 4, TimeTaken: 53 2018-01-05T11:28:52,021 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 4 2018-01-05T11:28:52,021 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:52,021 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 4. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:28:52,021 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-51_597_8151257429790130914-1/djaiswal/_tez_scratch_dir/6896f735-dbb7-4bbd-a14f-f8a1d5bcb679/map.xml 2018-01-05T11:28:52,021 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-51_597_8151257429790130914-1/djaiswal/_tez_scratch_dir/6896f735-dbb7-4bbd-a14f-f8a1d5bcb679/map.xml 2018-01-05T11:28:52,021 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-51_597_8151257429790130914-1/djaiswal/_tez_scratch_dir/6896f735-dbb7-4bbd-a14f-f8a1d5bcb679/map.xml 2018-01-05T11:28:52,021 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:52,021 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:52,021 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,022 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:52,022 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 2.03KB 2018-01-05T11:28:52,022 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11/kv1.txt to src new group? true 2018-01-05T11:28:52,022 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12/kv1.txt to src new group? false 2018-01-05T11:28:52,022 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11/kv1.txt to src new group? false 2018-01-05T11:28:52,022 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12/kv1.txt to src new group? false 2018-01-05T11:28:52,022 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:28:52,022 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:28:52,022 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:28:52,022 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 430 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 23248 Original splits: 4 2018-01-05T11:28:52,022 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired numSplits: 1 lengthPerGroup: 23248 numLocations: 1 numSplitsPerLocation: 4 numSplitsInGroup: 4 totalLength: 23248 numOriginalSplits: 4 . Grouping by length: true count: false nodeLocalOnly: false 2018-01-05T11:28:52,022 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Number of splits desired: 1 created: 1 splitsProcessed: 4 2018-01-05T11:28:52,022 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 4 grouped split count is 1, for bucket: 1 2018-01-05T11:28:52,022 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:28:52,023 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: srcpart on vertex vertex_1515180518813_0001_16_00 [Map 1] 2018-01-05T11:28:52,023 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_16_00 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:52,023 INFO [App Shared Pool - #0] impl.VertexImpl: Got updated RootInputsSpecs: {srcpart=forAllWorkUnits=true, update=[1]} 2018-01-05T11:28:52,024 INFO [App Shared Pool - #0] impl.VertexImpl: Vertex vertex_1515180518813_0001_16_00 [Map 1] parallelism set to 1 2018-01-05T11:28:52,024 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_16_00 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:52,024 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_16_00 [Map 1], numAdditionalOutputs=1 2018-01-05T11:28:52,024 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_16][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_16_00, initRequestedTime=1515180531958, initedTime=1515180532024, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:52,024 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_16_00 [Map 1] with distanceFromRoot: 0 2018-01-05T11:28:52,024 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_16_00 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:28:52,024 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_16][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_16_00, startRequestedTime=1515180531966, startedTime=1515180532024 2018-01-05T11:28:52,024 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_16][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_16_00, reconfigureDoneTime=1515180532024, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:28:52,024 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_16_00 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:52,024 INFO [App Shared Pool - #0] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:28:52,024 INFO [App Shared Pool - #0] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:28:52,024 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_16_00 [Map 1] 2018-01-05T11:28:52,024 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_16][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_16_00_000000, scheduledTime=1515180532024, launchTime=1515180532024 2018-01-05T11:28:52,024 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_16_00 [Map 1] 2018-01-05T11:28:52,024 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_16_00_000000_0, priority=2, capability=, hosts=null 2018-01-05T11:28:52,024 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=2,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:52,024 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=2,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:52,024 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:52,024 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_16_00_000000_0 on hosts=null 2018-01-05T11:28:52,024 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_16_00_000000_0 on hosts=null 2018-01-05T11:28:52,024 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_16_00_000000_0 2018-01-05T11:28:52,024 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:52,024 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #30, task=TaskInfo{task=attempt_1515180518813_0001_16_00_000000_0, priority=2, startTime=0, containerId=null, uniqueId=29, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000030 2018-01-05T11:28:52,024 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_16_00_000000_0; false 2018-01-05T11:28:52,024 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_16_00_000000_0, priority=2, startTime=311552533, containerId=container_111101111_0001_00_000030assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=29, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:52,025 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000030 for node: 10.22.8.182:49818 2018-01-05T11:28:52,026 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 16, name=ANALYZE TABLE srcpart PARTITION...STATISTICS (Stage-0), queryId=djaiswal_20180105112851_c7191e3d-978b-4094-ad5d-df22da119ed8 2018-01-05T11:28:52,026 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000030, attempt_1515180518813_0001_16_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:52,026 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000030 for node: 10.22.8.182:49818 2018-01-05T11:28:52,027 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000030 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:52,027 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000030, launchTime=1515180532027 2018-01-05T11:28:52,028 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_16_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000030] on NM: [10.22.8.182:49818] 2018-01-05T11:28:52,028 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_16_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112851_c7191e3d-978b-4094-ad5d-df22da119ed8, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000030, dagName=ANALYZE TABLE srcpart PARTITION...STATISTICS (Stage-0), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{srcpart,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{out_Map 1,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180531958, firstAttemptStartTime=1515180532024, currentAttemptStartTime=1515180532027} 2018-01-05T11:28:52,028 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:52,028 INFO [IPC Server handler 0 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000030, QueryId=djaiswal_20180105112851_c7191e3d-978b-4094-ad5d-df22da119ed8-16, DagId=16, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180532028 2018-01-05T11:28:52,028 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:52,028 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_16][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_16_00_000000_0, startTime=1515180532027, containerId=container_111101111_0001_00_000030, nodeId=10.22.8.182:49818 2018-01-05T11:28:52,028 DEBUG [IPC Server handler 0 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=16} with the ShuffleHandler 2018-01-05T11:28:52,039 INFO [IPC Server handler 0 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:28:52,039 DEBUG [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/16] 2018-01-05T11:28:52,040 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:52,040 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_16_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:52,040 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_16_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180532024, dagStartTime=1515180531958, withinDagPriority=2, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:28:52,040 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_16_00_000000_0 : ACCEPTED 2018-01-05T11:28:52,040 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_16_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:52,040 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_16_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180532024, dagStartTime=1515180531958, withinDagPriority=2, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:28:52,040 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_16_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:52,040 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_16_00_000000_0 to pre-emption queue 2018-01-05T11:28:52,040 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_16_00_000000_0: true 2018-01-05T11:28:52,040 INFO [TaskCommunicator # 4] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_16_00_000000_0 2018-01-05T11:28:52,040 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:1943871212 2018-01-05T11:28:52,041 INFO [TaskCommunicator # 4] GuaranteedTasks: Received response for attempt_1515180518813_0001_16_00_000000_0, true 2018-01-05T11:28:52,041 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : ANALYZE TABLE srcpart PARTITION...STATISTICS (Stage-0), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_16_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=srcpart, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=out_Map 1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:28:52,041 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1043333120, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:52,041 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_16_00_000000_0 vertexName: Map 1 2018-01-05T11:28:52,041 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000030, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_16_00_000000_0, eventCount=1 } 2018-01-05T11:28:52,042 DEBUG [TezTR-518813_1_16_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_16 queryId: djaiswal_20180105112851_c7191e3d-978b-4094-ad5d-df22da119ed8 instanceType: TaskRunner2Callable 2018-01-05T11:28:52,042 INFO [TezTR-518813_1_16_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_16_00_000000_0 2018-01-05T11:28:52,043 DEBUG [TezTR-518813_1_16_0_0_0] log.PerfLogger: 2018-01-05T11:28:52,044 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:52,044 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:52,046 INFO [IPC Server handler 20 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_16_00_000000_0 sent events: (0-1). 2018-01-05T11:28:52,046 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:28:52,046 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_16_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:28:52,046 DEBUG [TezTR-518813_1_16_0_0_0] log.PerfLogger: 2018-01-05T11:28:52,046 INFO [TezTR-518813_1_16_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:52,046 INFO [TezTR-518813_1_16_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:52,047 INFO [I/O Setup 0 Initialize: {srcpart}] input.MRInput: srcpart using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:28:52,047 INFO [I/O Setup 0 Initialize: {srcpart}] input.MRInputLegacy: srcpart MRInputLegacy deferring initialization 2018-01-05T11:28:52,047 INFO [TezTR-518813_1_16_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:52,047 INFO [I/O Setup 1 Initialize: {out_Map 1}] output.MROutput: out_Map 1: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:28:52,047 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:52,047 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:52,047 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:52,047 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:52,048 INFO [TezTR-518813_1_16_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:52,048 INFO [TezTR-518813_1_16_0_0_0] resources.MemoryDistributor: InitialRequests=[srcpart:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [out_Map 1:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput] 2018-01-05T11:28:52,048 INFO [TezTR-518813_1_16_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:52,048 INFO [TezTR-518813_1_16_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:52,048 INFO [TezTR-518813_1_16_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 1043333120, TotalAvailable: 699033190, TotalRequested/TotalJVMHeap:0.00 2018-01-05T11:28:52,048 INFO [TezTR-518813_1_16_0_0_0] resources.MemoryDistributor: Allocations=[srcpart:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [out_Map 1:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0] 2018-01-05T11:28:52,048 INFO [TezTR-518813_1_16_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:52,048 INFO [TezTR-518813_1_16_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: srcpart being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:28:52,048 INFO [TezTR-518813_1_16_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:28:52,048 INFO [TezTR-518813_1_16_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:28:52,048 INFO [I/O Setup 0 Start: {srcpart}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: srcpart 2018-01-05T11:28:52,048 INFO [TezTR-518813_1_16_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:52,048 INFO [TezTR-518813_1_16_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_16_00_000000_0 2018-01-05T11:28:52,048 DEBUG [TezTR-518813_1_16_0_0_0] log.PerfLogger: 2018-01-05T11:28:52,048 DEBUG [TezTR-518813_1_16_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_16_00_000000_0_10088 2018-01-05T11:28:52,048 DEBUG [TezTR-518813_1_16_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:52,048 DEBUG [TezTR-518813_1_16_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_16_00_000000_0 2018-01-05T11:28:52,048 INFO [TezTR-518813_1_16_0_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112851_c7191e3d-978b-4094-ad5d-df22da119ed8 2018-01-05T11:28:52,048 DEBUG [TezTR-518813_1_16_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:52,048 DEBUG [TezTR-518813_1_16_0_0_0] log.PerfLogger: 2018-01-05T11:28:52,048 DEBUG [TezTR-518813_1_16_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:52,048 DEBUG [TezTR-518813_1_16_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:52,048 DEBUG [TezTR-518813_1_16_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112851_c7191e3d-978b-4094-ad5d-df22da119ed8_Map 1__MAP_PLAN__ 2018-01-05T11:28:52,048 INFO [TezTR-518813_1_16_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-51_597_8151257429790130914-1/djaiswal/_tez_scratch_dir/6896f735-dbb7-4bbd-a14f-f8a1d5bcb679/map.xml 2018-01-05T11:28:52,049 DEBUG [TezTR-518813_1_16_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-51_597_8151257429790130914-1/djaiswal/_tez_scratch_dir/6896f735-dbb7-4bbd-a14f-f8a1d5bcb679/map.xml 2018-01-05T11:28:52,049 DEBUG [TezTR-518813_1_16_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-51_597_8151257429790130914-1/djaiswal/_tez_scratch_dir/6896f735-dbb7-4bbd-a14f-f8a1d5bcb679/map.xml 2018-01-05T11:28:52,049 DEBUG [TezTR-518813_1_16_0_0_0] log.PerfLogger: 2018-01-05T11:28:52,049 INFO [TezTR-518813_1_16_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:52,049 DEBUG [TezTR-518813_1_16_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,049 DEBUG [TezTR-518813_1_16_0_0_0] log.PerfLogger: 2018-01-05T11:28:52,049 INFO [TezTR-518813_1_16_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 2.03KB 2018-01-05T11:28:52,050 DEBUG [TezTR-518813_1_16_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:52,050 INFO [TezTR-518813_1_16_0_0_0] tez.MapRecordProcessor: The input names are: [srcpart] 2018-01-05T11:28:52,050 INFO [TezTR-518813_1_16_0_0_0] input.MRInput: Initialized MRInput: srcpart 2018-01-05T11:28:52,051 DEBUG [TezTR-518813_1_16_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 4 2018-01-05T11:28:52,051 INFO [TezTR-518813_1_16_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-51_597_8151257429790130914-1/djaiswal/_tez_scratch_dir/6896f735-dbb7-4bbd-a14f-f8a1d5bcb679/map.xml 2018-01-05T11:28:52,051 DEBUG [TezTR-518813_1_16_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:52,051 DEBUG [TezTR-518813_1_16_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11/kv1.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-08, hr=11}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11, columns=key,value, partition_columns.types=string:string, serialization.format=1, numRows=0, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180531, rawDataSize=0, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=11, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-09, hr=12}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12, columns=key,value, partition_columns.types=string:string, serialization.format=1, numRows=0, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180531, rawDataSize=0, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=12, vectorPartitionDesc=null], file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-09, hr=11}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11, columns=key,value, partition_columns.types=string:string, serialization.format=1, numRows=0, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180531, rawDataSize=0, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=11, vectorPartitionDesc=null], file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-08, hr=11}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11, columns=key,value, partition_columns.types=string:string, serialization.format=1, numRows=0, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180531, rawDataSize=0, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=11, vectorPartitionDesc=null], file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-08, hr=12}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12, columns=key,value, partition_columns.types=string:string, serialization.format=1, numRows=0, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180531, rawDataSize=0, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=12, vectorPartitionDesc=null]} 2018-01-05T11:28:52,051 DEBUG [TezTR-518813_1_16_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:52,051 INFO [TezTR-518813_1_16_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:52,051 DEBUG [TezTR-518813_1_16_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:52,052 DEBUG [TezTR-518813_1_16_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:52,052 INFO [TezTR-518813_1_16_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-51_597_8151257429790130914-1/djaiswal/_tez_scratch_dir/6896f735-dbb7-4bbd-a14f-f8a1d5bcb679/map.xml 2018-01-05T11:28:52,052 DEBUG [TezTR-518813_1_16_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:52,052 INFO [TezTR-518813_1_16_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_16_00_000000_0 2018-01-05T11:28:52,053 INFO [TezTR-518813_1_16_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [string, string] (isAcidRead false) 2018-01-05T11:28:52,053 WARN [TezTR-518813_1_16_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11/kv1.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:28:52,056 DEBUG [TezTR-518813_1_16_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:28:52,056 WARN [TezTR-518813_1_16_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11/kv1.txt:0+5812 2018-01-05T11:28:52,056 DEBUG [TezTR-518813_1_16_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11/kv1.txt 2018-01-05T11:28:52,056 INFO [TezTR-518813_1_16_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11/kv1.txt:0+5812, org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12/kv1.txt:0+5812, org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11/kv1.txt:0+5812, org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12/kv1.txt:0+5812], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=23248} 2018-01-05T11:28:52,056 INFO [TezTR-518813_1_16_0_0_0] input.MRInput: srcpart initialized RecordReader from event 2018-01-05T11:28:52,056 DEBUG [TezTR-518813_1_16_0_0_0] tez.MapRecordProcessor: Starting Output: out_Map 1 2018-01-05T11:28:52,056 INFO [TezTR-518813_1_16_0_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:28:52,056 INFO [TezTR-518813_1_16_0_0_0] exec.MapOperator: Initializing operator MAP[0] 2018-01-05T11:28:52,056 DEBUG [TezTR-518813_1_16_0_0_0] exec.MapOperator: Initialization Done 0 MAP 2018-01-05T11:28:52,056 DEBUG [TezTR-518813_1_16_0_0_0] exec.MapOperator: Operator 0 MAP initialized 2018-01-05T11:28:52,056 DEBUG [TezTR-518813_1_16_0_0_0] exec.MapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:28:52,056 DEBUG [TezTR-518813_1_16_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,057 DEBUG [TezTR-518813_1_16_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,057 DEBUG [TezTR-518813_1_16_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,057 DEBUG [TezTR-518813_1_16_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,057 DEBUG [TezTR-518813_1_16_0_0_0] exec.MapOperator: Adding alias srcpart to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12 2018-01-05T11:28:52,057 DEBUG [TezTR-518813_1_16_0_0_0] exec.MapOperator: Adding alias srcpart to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11 2018-01-05T11:28:52,057 DEBUG [TezTR-518813_1_16_0_0_0] exec.MapOperator: Adding alias srcpart to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11 2018-01-05T11:28:52,057 DEBUG [TezTR-518813_1_16_0_0_0] exec.MapOperator: Adding alias srcpart to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12 2018-01-05T11:28:52,058 DEBUG [TezTR-518813_1_16_0_0_0] exec.MapOperator: dump TS[0] struct 2018-01-05T11:28:52,058 DEBUG [TezTR-518813_1_16_0_0_0] exec.MapOperator: dump TS[0] struct 2018-01-05T11:28:52,058 DEBUG [TezTR-518813_1_16_0_0_0] exec.MapOperator: dump TS[0] struct 2018-01-05T11:28:52,058 DEBUG [TezTR-518813_1_16_0_0_0] exec.MapOperator: dump TS[0] struct 2018-01-05T11:28:52,058 INFO [TezTR-518813_1_16_0_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:28:52,058 INFO [TezTR-518813_1_16_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:52,058 DEBUG [TezTR-518813_1_16_0_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:28:52,058 INFO [TezTR-518813_1_16_0_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:28:52,058 DEBUG [TezTR-518813_1_16_0_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:28:52,058 DEBUG [TezTR-518813_1_16_0_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:28:52,058 DEBUG [TezTR-518813_1_16_0_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:28:52,058 DEBUG [TezTR-518813_1_16_0_0_0] log.PerfLogger: 2018-01-05T11:28:52,058 INFO [TezTR-518813_1_16_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-51_597_8151257429790130914-1/djaiswal/_tez_scratch_dir/6896f735-dbb7-4bbd-a14f-f8a1d5bcb679/map.xml 2018-01-05T11:28:52,058 DEBUG [TezTR-518813_1_16_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:52,058 DEBUG [TezTR-518813_1_16_0_0_0] exec.MapOperator: Processing alias(es) srcpart for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11/kv1.txt 2018-01-05T11:28:52,058 INFO [TezTR-518813_1_16_0_0_0] exec.TableScanOperator: Stats Gathering found a new partition spec = ds=2008-04-08/hr=11 2018-01-05T11:28:52,058 INFO [TezTR-518813_1_16_0_0_0] exec.MapOperator: MAP[0]: records read - 1 2018-01-05T11:28:52,058 INFO [TezTR-518813_1_16_0_0_0] exec.MapOperator: MAP[0]: records read - 10 2018-01-05T11:28:52,059 INFO [TezTR-518813_1_16_0_0_0] exec.MapOperator: MAP[0]: records read - 100 2018-01-05T11:28:52,059 DEBUG [TezTR-518813_1_16_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 1 of 4 2018-01-05T11:28:52,059 DEBUG [TezTR-518813_1_16_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12/kv1.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-08, hr=12}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12, columns=key,value, partition_columns.types=string:string, serialization.format=1, numRows=0, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180531, rawDataSize=0, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=12, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-09, hr=12}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12, columns=key,value, partition_columns.types=string:string, serialization.format=1, numRows=0, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180531, rawDataSize=0, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=12, vectorPartitionDesc=null], file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-09, hr=11}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11, columns=key,value, partition_columns.types=string:string, serialization.format=1, numRows=0, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180531, rawDataSize=0, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=11, vectorPartitionDesc=null], file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-08, hr=11}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11, columns=key,value, partition_columns.types=string:string, serialization.format=1, numRows=0, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180531, rawDataSize=0, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=11, vectorPartitionDesc=null], file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-08, hr=12}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12, columns=key,value, partition_columns.types=string:string, serialization.format=1, numRows=0, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180531, rawDataSize=0, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=12, vectorPartitionDesc=null]} 2018-01-05T11:28:52,060 DEBUG [TezTR-518813_1_16_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:52,060 INFO [TezTR-518813_1_16_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:52,060 DEBUG [TezTR-518813_1_16_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:52,061 DEBUG [TezTR-518813_1_16_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:52,061 INFO [TezTR-518813_1_16_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-51_597_8151257429790130914-1/djaiswal/_tez_scratch_dir/6896f735-dbb7-4bbd-a14f-f8a1d5bcb679/map.xml 2018-01-05T11:28:52,061 DEBUG [TezTR-518813_1_16_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:52,061 INFO [TezTR-518813_1_16_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_16_00_000000_0 2018-01-05T11:28:52,061 INFO [TezTR-518813_1_16_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [string, string] (isAcidRead false) 2018-01-05T11:28:52,061 WARN [TezTR-518813_1_16_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12/kv1.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:28:52,061 DEBUG [TezTR-518813_1_16_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:28:52,061 WARN [TezTR-518813_1_16_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12/kv1.txt:0+5812 2018-01-05T11:28:52,062 DEBUG [TezTR-518813_1_16_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12/kv1.txt 2018-01-05T11:28:52,062 INFO [TezTR-518813_1_16_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-51_597_8151257429790130914-1/djaiswal/_tez_scratch_dir/6896f735-dbb7-4bbd-a14f-f8a1d5bcb679/map.xml 2018-01-05T11:28:52,062 DEBUG [TezTR-518813_1_16_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:52,062 DEBUG [TezTR-518813_1_16_0_0_0] exec.MapOperator: Processing alias(es) srcpart for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12/kv1.txt 2018-01-05T11:28:52,062 INFO [TezTR-518813_1_16_0_0_0] exec.TableScanOperator: Stats Gathering found a new partition spec = ds=2008-04-08/hr=12 2018-01-05T11:28:52,063 INFO [TezTR-518813_1_16_0_0_0] exec.MapOperator: MAP[0]: records read - 1000 2018-01-05T11:28:52,063 DEBUG [TezTR-518813_1_16_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 2 of 4 2018-01-05T11:28:52,063 DEBUG [TezTR-518813_1_16_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11/kv1.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-09, hr=11}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11, columns=key,value, partition_columns.types=string:string, serialization.format=1, numRows=0, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180531, rawDataSize=0, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=11, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-09, hr=12}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12, columns=key,value, partition_columns.types=string:string, serialization.format=1, numRows=0, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180531, rawDataSize=0, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=12, vectorPartitionDesc=null], file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-09, hr=11}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11, columns=key,value, partition_columns.types=string:string, serialization.format=1, numRows=0, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180531, rawDataSize=0, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=11, vectorPartitionDesc=null], file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-08, hr=11}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11, columns=key,value, partition_columns.types=string:string, serialization.format=1, numRows=0, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180531, rawDataSize=0, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=11, vectorPartitionDesc=null], file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-08, hr=12}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12, columns=key,value, partition_columns.types=string:string, serialization.format=1, numRows=0, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180531, rawDataSize=0, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=12, vectorPartitionDesc=null]} 2018-01-05T11:28:52,063 DEBUG [TezTR-518813_1_16_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:52,063 INFO [TezTR-518813_1_16_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:52,063 DEBUG [TezTR-518813_1_16_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:52,064 DEBUG [TezTR-518813_1_16_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:52,065 INFO [TezTR-518813_1_16_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-51_597_8151257429790130914-1/djaiswal/_tez_scratch_dir/6896f735-dbb7-4bbd-a14f-f8a1d5bcb679/map.xml 2018-01-05T11:28:52,065 DEBUG [TezTR-518813_1_16_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:52,065 INFO [TezTR-518813_1_16_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_16_00_000000_0 2018-01-05T11:28:52,065 INFO [TezTR-518813_1_16_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [string, string] (isAcidRead false) 2018-01-05T11:28:52,065 WARN [TezTR-518813_1_16_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11/kv1.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:28:52,065 DEBUG [TezTR-518813_1_16_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:28:52,065 WARN [TezTR-518813_1_16_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11/kv1.txt:0+5812 2018-01-05T11:28:52,065 DEBUG [TezTR-518813_1_16_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11/kv1.txt 2018-01-05T11:28:52,065 INFO [TezTR-518813_1_16_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-51_597_8151257429790130914-1/djaiswal/_tez_scratch_dir/6896f735-dbb7-4bbd-a14f-f8a1d5bcb679/map.xml 2018-01-05T11:28:52,065 DEBUG [TezTR-518813_1_16_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:52,066 DEBUG [TezTR-518813_1_16_0_0_0] exec.MapOperator: Processing alias(es) srcpart for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11/kv1.txt 2018-01-05T11:28:52,066 INFO [TezTR-518813_1_16_0_0_0] exec.TableScanOperator: Stats Gathering found a new partition spec = ds=2008-04-09/hr=11 2018-01-05T11:28:52,066 DEBUG [TezTR-518813_1_16_0_0_0] tez.InterruptibleProcessing: Adjusting abort check rows to 100000 from 1000 2018-01-05T11:28:52,066 DEBUG [TezTR-518813_1_16_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 3 of 4 2018-01-05T11:28:52,066 DEBUG [TezTR-518813_1_16_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12/kv1.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-09, hr=12}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12, columns=key,value, partition_columns.types=string:string, serialization.format=1, numRows=0, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180531, rawDataSize=0, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=12, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-09, hr=12}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12, columns=key,value, partition_columns.types=string:string, serialization.format=1, numRows=0, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180531, rawDataSize=0, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=12, vectorPartitionDesc=null], file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-09, hr=11}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11, columns=key,value, partition_columns.types=string:string, serialization.format=1, numRows=0, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180531, rawDataSize=0, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=11, vectorPartitionDesc=null], file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-08, hr=11}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11, columns=key,value, partition_columns.types=string:string, serialization.format=1, numRows=0, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180531, rawDataSize=0, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=11, vectorPartitionDesc=null], file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-08, hr=12}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12, columns=key,value, partition_columns.types=string:string, serialization.format=1, numRows=0, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180531, rawDataSize=0, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=12, vectorPartitionDesc=null]} 2018-01-05T11:28:52,067 DEBUG [TezTR-518813_1_16_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:52,067 INFO [TezTR-518813_1_16_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:52,067 DEBUG [TezTR-518813_1_16_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:52,068 DEBUG [TezTR-518813_1_16_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:52,068 INFO [TezTR-518813_1_16_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-51_597_8151257429790130914-1/djaiswal/_tez_scratch_dir/6896f735-dbb7-4bbd-a14f-f8a1d5bcb679/map.xml 2018-01-05T11:28:52,068 DEBUG [TezTR-518813_1_16_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:52,069 INFO [TezTR-518813_1_16_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_16_00_000000_0 2018-01-05T11:28:52,069 INFO [TezTR-518813_1_16_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [string, string] (isAcidRead false) 2018-01-05T11:28:52,069 WARN [TezTR-518813_1_16_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12/kv1.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:28:52,069 DEBUG [TezTR-518813_1_16_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:28:52,069 WARN [TezTR-518813_1_16_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12/kv1.txt:0+5812 2018-01-05T11:28:52,070 DEBUG [TezTR-518813_1_16_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12/kv1.txt 2018-01-05T11:28:52,070 INFO [TezTR-518813_1_16_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-51_597_8151257429790130914-1/djaiswal/_tez_scratch_dir/6896f735-dbb7-4bbd-a14f-f8a1d5bcb679/map.xml 2018-01-05T11:28:52,070 DEBUG [TezTR-518813_1_16_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:52,070 DEBUG [TezTR-518813_1_16_0_0_0] exec.MapOperator: Processing alias(es) srcpart for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12/kv1.txt 2018-01-05T11:28:52,070 INFO [TezTR-518813_1_16_0_0_0] exec.TableScanOperator: Stats Gathering found a new partition spec = ds=2008-04-09/hr=12 2018-01-05T11:28:52,071 DEBUG [TezTR-518813_1_16_0_0_0] log.PerfLogger: 2018-01-05T11:28:52,071 DEBUG [TezTR-518813_1_16_0_0_0] mr.ObjectCache: djaiswal_20180105112851_c7191e3d-978b-4094-ad5d-df22da119ed8_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:28:52,071 DEBUG [TezTR-518813_1_16_0_0_0] exec.MapOperator: close called for operator MAP[0] 2018-01-05T11:28:52,071 INFO [TezTR-518813_1_16_0_0_0] exec.MapOperator: Closing operator MAP[0] 2018-01-05T11:28:52,071 INFO [TezTR-518813_1_16_0_0_0] exec.MapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:2000, 2018-01-05T11:28:52,071 DEBUG [TezTR-518813_1_16_0_0_0] exec.MapOperator: Closing child = TS[0] 2018-01-05T11:28:52,071 DEBUG [TezTR-518813_1_16_0_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:28:52,071 DEBUG [TezTR-518813_1_16_0_0_0] exec.TableScanOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:52,071 INFO [TezTR-518813_1_16_0_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:28:52,071 DEBUG [TezTR-518813_1_16_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,071 DEBUG [TezTR-518813_1_16_0_0_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-51_597_8151257429790130914-1/-mr-10000 2018-01-05T11:28:52,071 INFO [TezTR-518813_1_16_0_0_0] exec.TableScanOperator: publishing : default.srcpart/ds=2008-04-09/hr=11/ : {numRows=500, rawDataSize=5312} 2018-01-05T11:28:52,071 INFO [TezTR-518813_1_16_0_0_0] exec.TableScanOperator: publishing : default.srcpart/ds=2008-04-09/hr=12/ : {numRows=500, rawDataSize=5312} 2018-01-05T11:28:52,071 INFO [TezTR-518813_1_16_0_0_0] exec.TableScanOperator: publishing : default.srcpart/ds=2008-04-08/hr=11/ : {numRows=500, rawDataSize=5312} 2018-01-05T11:28:52,071 INFO [TezTR-518813_1_16_0_0_0] exec.TableScanOperator: publishing : default.srcpart/ds=2008-04-08/hr=12/ : {numRows=500, rawDataSize=5312} 2018-01-05T11:28:52,101 DEBUG [TezTR-518813_1_16_0_0_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-51_597_8151257429790130914-1/-mr-10000/tmpstats-0_0 2018-01-05T11:28:52,101 DEBUG [TezTR-518813_1_16_0_0_0] fs.FSStatsPublisher: Writing stats in it : {default.srcpart/ds=2008-04-08/hr=11/={numRows=500, rawDataSize=5312}, default.srcpart/ds=2008-04-09/hr=11/={numRows=500, rawDataSize=5312}, default.srcpart/ds=2008-04-09/hr=12/={numRows=500, rawDataSize=5312}, default.srcpart/ds=2008-04-08/hr=12/={numRows=500, rawDataSize=5312}} 2018-01-05T11:28:52,101 DEBUG [TezTR-518813_1_16_0_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:28:52,101 DEBUG [TezTR-518813_1_16_0_0_0] exec.MapOperator: 0 Close done 2018-01-05T11:28:52,101 INFO [TezTR-518813_1_16_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_16_00_000000_0 2018-01-05T11:28:52,102 INFO [TezTR-518813_1_16_0_0_0] output.MROutput: out_Map 1 closed 2018-01-05T11:28:52,102 DEBUG [TezTR-518813_1_16_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:52,102 INFO [TezTR-518813_1_16_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:52,102 INFO [TezTR-518813_1_16_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_16_00_000000_0, askedToStop=false 2018-01-05T11:28:52,102 INFO [TezTR-518813_1_16_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_16_00_000000_0, stopRequested=false 2018-01-05T11:28:52,102 INFO [TezTR-518813_1_16_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_16_00_000000_0: Counters: 5 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=2000, INPUT_SPLIT_LENGTH_BYTES=23248, OUTPUT_RECORDS=0][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=2000]] 2018-01-05T11:28:52,102 DEBUG [TezTR-518813_1_16_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_16_0_0_0 thread id: 1101 scheme: file bytesRead: 23488 bytesWritten: 277 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:52,102 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_16_00_000000_0, isTaskDone=true 2018-01-05T11:28:52,103 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000030, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_16_00_000000_0, eventCount=2 } 2018-01-05T11:28:52,103 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_16][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_16_00_000000_0, creationTime=1515180532024, allocationTime=1515180532025, startTime=1515180532027, finishTime=1515180532103, timeTaken=76, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:52,104 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_16_00_000000_0 2018-01-05T11:28:52,104 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_16][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_16_00_000000, startTime=1515180532027, finishTime=1515180532104, timeTaken=77, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_16_00_000000_0, diagnostics=, counters=Counters: 11, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=23488, FILE_BYTES_WRITTEN=277, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=2000, INPUT_SPLIT_LENGTH_BYTES=23248, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=2000 2018-01-05T11:28:52,104 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_16_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:52,104 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_16_00_000000_0; false 2018-01-05T11:28:52,104 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_16_00 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:52,104 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_16_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:52,104 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_16_00 [Map 1] 2018-01-05T11:28:52,104 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:52,104 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_16_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:52,104 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_16_00_000000_0 2018-01-05T11:28:52,104 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000030=63 2018-01-05T11:28:52,104 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:52,104 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:52,104 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_16_00_000000_0: true 2018-01-05T11:28:52,104 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_16][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_16_00, initRequestedTime=1515180531958, initedTime=1515180532024, startRequestedTime=1515180531966, startedTime=1515180532024, finishTime=1515180532104, timeTaken=80, status=SUCCEEDED, diagnostics=, counters=Counters: 11, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=23488, FILE_BYTES_WRITTEN=277, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=2000, INPUT_SPLIT_LENGTH_BYTES=23248, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=2000, vertexStats=firstTaskStartTime=1515180532027, firstTasksToStart=[ task_1515180518813_0001_16_00_000000 ], lastTaskFinishTime=1515180532103, lastTasksToFinish=[ task_1515180518813_0001_16_00_000000 ], minTaskDuration=76, maxTaskDuration=76, avgTaskDuration=76.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_16_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_16_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:52,104 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_16_00 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:52,105 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_16][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000030, stoppedTime=1515180532105, exitStatus=0 2018-01-05T11:28:52,105 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_16_00 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1 2018-01-05T11:28:52,105 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null 2018-01-05T11:28:52,105 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_16_00 [Map 1] 2018-01-05T11:28:52,105 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_16_00_000000_0 2018-01-05T11:28:52,105 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000030, QueryId=djaiswal_20180105112851_c7191e3d-978b-4094-ad5d-df22da119ed8-16, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:28:52,105 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_16_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:52,105 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_16_00_000000_0 2018-01-05T11:28:52,106 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_16, queueSize=0 2018-01-05T11:28:52,106 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_16][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_16, startTime=1515180531958, finishTime=1515180532105, timeTaken=147, status=SUCCEEDED, diagnostics=, counters=Counters: 15, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, OTHER_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=260, AM_GC_TIME_MILLIS=0, File System Counters, FILE_BYTES_READ=23488, FILE_BYTES_WRITTEN=277, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=2000, INPUT_SPLIT_LENGTH_BYTES=23248, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=2000 2018-01-05T11:28:52,106 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_16 finished with state: SUCCEEDED 2018-01-05T11:28:52,106 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_16 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:28:52,106 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000030 exited with diagnostics set to Container completed. 2018-01-05T11:28:52,106 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000030 2018-01-05T11:28:52,106 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_16, dagState=SUCCEEDED 2018-01-05T11:28:52,106 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_16_post 2018-01-05T11:28:52,107 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:52,107 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:28:52,107 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:28:52,107 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 16 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=1, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=1,NumRequestsWithoutLocation=1, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=1} 2018-01-05T11:28:52,107 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:28:52,107 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=ANALYZE TABLE srcpart PARTITION...STATISTICS (Stage-0), with id=dag_1515180518813_0001_16 2018-01-05T11:28:52,107 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 16, to 10.22.8.182:49818 2018-01-05T11:28:52,107 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=1, Remaining=0 2018-01-05T11:28:52,107 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=ANALYZE TABLE srcpart PARTITION...STATISTICS (Stage-0), with id=dag_1515180518813_0001_16 2018-01-05T11:28:52,107 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=16} 2018-01-05T11:28:52,107 INFO [IPC Server handler 2 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=16}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:28:52,107 INFO [IPC Server handler 2 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/16 after 300 seconds 2018-01-05T11:28:52,107 ERROR [IPC Server handler 2 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112851_c7191e3d-978b-4094-ad5d-df22da119ed8, dag_1515180518813_0001_16 2018-01-05T11:28:52,107 INFO [IPC Server handler 2 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112851_c7191e3d-978b-4094-ad5d-df22da119ed8 2018-01-05T11:28:52,107 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=16} = 0 2018-01-05T11:28:52,107 DEBUG [IPC Server handler 2 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=16} 2018-01-05T11:28:52,107 DEBUG [IPC Server handler 2 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:28:52,107 DEBUG [IPC Server handler 2 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=16} 2018-01-05T11:28:52,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,108 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 2018-01-05T11:28:52,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-2:STATS] in serial mode 2018-01-05T11:28:52,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@2edd4b9f, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:52,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:52,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:52,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:52,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:52,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:52,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:52,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:52,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:52,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:52,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:52,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,110 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:52,110 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-51_597_8151257429790130914-1/-mr-10000 2018-01-05T11:28:52,110 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-51_597_8151257429790130914-1/-mr-10000 2018-01-05T11:28:52,111 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Getting file stats of all partitions. threadpool size:15 2018-01-05T11:28:52,112 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Finished getting file stats of all partitions! 2018-01-05T11:28:52,113 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.srcpart/ds=2008-04-08/hr=11/ numRows 2018-01-05T11:28:52,113 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.srcpart/ds=2008-04-08/hr=11/ numRows 500 2018-01-05T11:28:52,113 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.srcpart/ds=2008-04-08/hr=11/ rawDataSize 2018-01-05T11:28:52,113 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.srcpart/ds=2008-04-08/hr=11/ rawDataSize 5312 2018-01-05T11:28:52,113 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Partition {ds=2008-04-08, hr=11} stats: [numFiles=1, numRows=500, totalSize=5812, rawDataSize=5312] 2018-01-05T11:28:52,113 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.srcpart/ds=2008-04-08/hr=12/ numRows 2018-01-05T11:28:52,113 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.srcpart/ds=2008-04-08/hr=12/ numRows 500 2018-01-05T11:28:52,113 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.srcpart/ds=2008-04-08/hr=12/ rawDataSize 2018-01-05T11:28:52,113 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.srcpart/ds=2008-04-08/hr=12/ rawDataSize 5312 2018-01-05T11:28:52,113 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Partition {ds=2008-04-08, hr=12} stats: [numFiles=1, numRows=500, totalSize=5812, rawDataSize=5312] 2018-01-05T11:28:52,113 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.srcpart/ds=2008-04-09/hr=11/ numRows 2018-01-05T11:28:52,113 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.srcpart/ds=2008-04-09/hr=11/ numRows 500 2018-01-05T11:28:52,113 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.srcpart/ds=2008-04-09/hr=11/ rawDataSize 2018-01-05T11:28:52,113 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.srcpart/ds=2008-04-09/hr=11/ rawDataSize 5312 2018-01-05T11:28:52,113 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Partition {ds=2008-04-09, hr=11} stats: [numFiles=1, numRows=500, totalSize=5812, rawDataSize=5312] 2018-01-05T11:28:52,113 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.srcpart/ds=2008-04-09/hr=12/ numRows 2018-01-05T11:28:52,113 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.srcpart/ds=2008-04-09/hr=12/ numRows 500 2018-01-05T11:28:52,113 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.srcpart/ds=2008-04-09/hr=12/ rawDataSize 2018-01-05T11:28:52,113 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.srcpart/ds=2008-04-09/hr=12/ rawDataSize 5312 2018-01-05T11:28:52,113 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Partition {ds=2008-04-09, hr=12} stats: [numFiles=1, numRows=500, totalSize=5812, rawDataSize=5312] 2018-01-05T11:28:52,114 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,115 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:52,115 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:52,115 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:52,115 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:52,115 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,115 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,115 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,116 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:52,116 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:52,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,117 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:52,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,117 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_partitions : db=default tbl=srcpart 2018-01-05T11:28:52,117 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_partitions : db=default tbl=srcpart 2018-01-05T11:28:52,117 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: New partition values:[2008-04-08, 11] 2018-01-05T11:28:52,117 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: New partition values:[2008-04-08, 12] 2018-01-05T11:28:52,117 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: New partition values:[2008-04-09, 11] 2018-01-05T11:28:52,117 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: New partition values:[2008-04-09, 12] 2018-01-05T11:28:52,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,140 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,145 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,145 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,145 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: About to delete stats tmp dir :file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-51_597_8151257429790130914-1/-mr-10000 2018-01-05T11:28:52,146 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,146 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,146 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: ANALYZE TABLE srcpart PARTITION(ds, hr) COMPUTE STATISTICS 2018-01-05T11:28:52,146 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:28:52,146 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@srcpart 2018-01-05T11:28:52,146 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@srcpart@ds=2008-04-08/hr=11 2018-01-05T11:28:52,146 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@srcpart@ds=2008-04-08/hr=12 2018-01-05T11:28:52,146 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=11 2018-01-05T11:28:52,146 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=12 2018-01-05T11:28:52,146 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@srcpart 2018-01-05T11:28:52,146 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@srcpart@ds=2008-04-08/hr=11 2018-01-05T11:28:52,146 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@srcpart@ds=2008-04-08/hr=12 2018-01-05T11:28:52,146 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@srcpart@ds=2008-04-09/hr=11 2018-01-05T11:28:52,146 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@srcpart@ds=2008-04-09/hr=12 2018-01-05T11:28:52,146 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,146 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,146 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:52,146 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_partitions_(String, String, List, EnvironmentContext, )=28} 2018-01-05T11:28:52,146 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112851_c7191e3d-978b-4094-ad5d-df22da119ed8); Time taken: 0.296 seconds 2018-01-05T11:28:52,146 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:52,146 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,146 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:52,146 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,146 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query ANALYZE TABLE srcpart PARTITION(ds, hr) COMPUTE STATISTICS 2018-01-05T11:28:52,146 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-51_597_8151257429790130914-1 2018-01-05T11:28:52,147 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.55 seconds 2018-01-05T11:28:52,147 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:52,147 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:52,147 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:52,147 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:52,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:52,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: ANALYZE TABLE srcpart PARTITION(ds, hr) COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:52,147 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab): ANALYZE TABLE srcpart PARTITION(ds, hr) COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:52,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: ANALYZE TABLE srcpart PARTITION(ds, hr) COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:52,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:52,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,148 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:52,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,148 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcpart 2018-01-05T11:28:52,148 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcpart 2018-01-05T11:28:52,150 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,150 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,151 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.BaseSemanticAnalyzer: Dynamic partitioning is used; only validating 0 columns 2018-01-05T11:28:52,151 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: select compute_stats(`key`, 'hll') , compute_stats(`value`, 'hll') , `ds` , `hr` from `default`.`srcpart` group by `ds`,`hr` 2018-01-05T11:28:52,151 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: select compute_stats(`key`, 'hll') , compute_stats(`value`, 'hll') , `ds` , `hr` from `default`.`srcpart` group by `ds`,`hr` 2018-01-05T11:28:52,153 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:52,153 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Invoking analyze on rewritten query 2018-01-05T11:28:52,153 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Starting Semantic Analysis 2018-01-05T11:28:52,153 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:52,153 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:52,153 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed phase 1 of Semantic Analysis 2018-01-05T11:28:52,153 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for source tables 2018-01-05T11:28:52,153 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,153 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcpart 2018-01-05T11:28:52,153 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcpart 2018-01-05T11:28:52,157 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,158 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,158 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for subqueries 2018-01-05T11:28:52,158 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for destination tables 2018-01-05T11:28:52,171 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed getting MetaData in Semantic Analysis 2018-01-05T11:28:52,171 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_151_4467563815339635813-1/-mr-10001 2018-01-05T11:28:52,171 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Table Plan for srcpart TS[0] 2018-01-05T11:28:52,171 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:52,172 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:52,172 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: RR before GB srcpart{(key,key: string)(value,value: string)(ds,ds: string)(hr,hr: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)(RAW__DATA__SIZE,RAW__DATA__SIZE: bigint)} after GB {((tok_table_or_col ds),_col0: string)((tok_table_or_col hr),_col1: string)((tok_function compute_stats (tok_table_or_col key) 'hll'),_col2: struct)((tok_function compute_stats (tok_table_or_col value) 'hll'),_col3: struct)} srcpart{(ds,_col0: string)(hr,_col1: string)} 2018-01-05T11:28:52,172 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: tree: (tok_select (tok_selexpr (tok_function compute_stats (tok_table_or_col key) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col value) 'hll')) (tok_selexpr (tok_table_or_col ds)) (tok_selexpr (tok_table_or_col hr))) 2018-01-05T11:28:52,172 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: genSelectPlan: input = {((tok_table_or_col ds),_col0: string)((tok_table_or_col hr),_col1: string)((tok_function compute_stats (tok_table_or_col key) 'hll'),_col2: struct)((tok_function compute_stats (tok_table_or_col value) 'hll'),_col3: struct)} srcpart{(ds,_col0: string)(hr,_col1: string)} starRr = srcpart{(key,key: string)(value,value: string)(ds,ds: string)(hr,hr: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)(RAW__DATA__SIZE,RAW__DATA__SIZE: bigint)} 2018-01-05T11:28:52,174 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Select Plan row schema: null{(_c0,_col0: struct)(_c1,_col1: struct)(ds,_col2: string)(hr,_col3: string)} srcpart{(ds,_col2: string)(hr,_col3: string)} 2018-01-05T11:28:52,174 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Select Plan for clause: insclause-0 2018-01-05T11:28:52,174 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_151_4467563815339635813-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-52_151_4467563815339635813-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_151_4467563815339635813-1/-mr-10000 2018-01-05T11:28:52,174 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_151_4467563815339635813-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-52_151_4467563815339635813-1 2018-01-05T11:28:52,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_151_4467563815339635813-1/-mr-10003 2018-01-05T11:28:52,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_151_4467563815339635813-1/-mr-10000 row schema: null{(_c0,_col0: struct)(_c1,_col1: struct)(ds,_col2: string)(hr,_col3: string)} srcpart{(ds,_col2: string)(hr,_col3: string)} 2018-01-05T11:28:52,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Body Plan for Query Block null 2018-01-05T11:28:52,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Plan for Query Block null 2018-01-05T11:28:52,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Before logical optimization TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-SEL[5]-FS[6] 2018-01-05T11:28:52,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] lineage.Generator: Time taken for lineage transform=1 2018-01-05T11:28:52,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{srcpart}key,value: string|{srcpart}value,ds: string|{srcpart}ds,hr: string|{srcpart}hr,BLOCK__OFFSET__INSIDE__FILE: bigint|{srcpart}block__offset__inside__file,INPUT__FILE__NAME: string|{srcpart}input__file__name,ROW__ID: struct|{srcpart}row__id,RAW__DATA__SIZE: bigint|{srcpart}raw__data__size) 2018-01-05T11:28:52,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: string|{srcpart}key,value: string|{srcpart}value,ds: string|{srcpart}ds,hr: string|{srcpart}hr,BLOCK__OFFSET__INSIDE__FILE: bigint|{srcpart}block__offset__inside__file,INPUT__FILE__NAME: string|{srcpart}input__file__name,ROW__ID: struct|{srcpart}row__id,RAW__DATA__SIZE: bigint|{srcpart}raw__data__size) 2018-01-05T11:28:52,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:52,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:52,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value] Column[ds] Column[hr] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:28:52,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: string|{}(tok_table_or_col ds),_col1: string|{}(tok_table_or_col hr),_col2: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:52,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:52,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:52,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(KEY._col0: string|{}(tok_table_or_col ds),KEY._col1: string|{}(tok_table_or_col hr),VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:52,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:52,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:52,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: string|{}(tok_table_or_col ds),_col1: string|{}(tok_table_or_col hr),_col2: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:52,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:52,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:52,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1,_col2: string|{null}ds,_col3: string|{null}hr) 2018-01-05T11:28:52,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:52,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:52,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col2] Column[_col3] Column[_col0] Column[_col1]) 2018-01-05T11:28:52,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{},_col2: string|{},_col3: string|{}) 2018-01-05T11:28:52,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:52,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:52,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,204 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for FS(6) 2018-01-05T11:28:52,204 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(5) 2018-01-05T11:28:52,204 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for GBY(4) 2018-01-05T11:28:52,204 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for RS(3) 2018-01-05T11:28:52,204 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for GBY(2) 2018-01-05T11:28:52,204 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(1) 2018-01-05T11:28:52,204 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.PredicatePushDown: After PPD: TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-SEL[5]-FS[6] 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{srcpart}key,value: string|{srcpart}value,ds: string|{srcpart}ds,hr: string|{srcpart}hr,BLOCK__OFFSET__INSIDE__FILE: bigint|{srcpart}block__offset__inside__file,INPUT__FILE__NAME: string|{srcpart}input__file__name,ROW__ID: struct|{srcpart}row__id,RAW__DATA__SIZE: bigint|{srcpart}raw__data__size) 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: string|{srcpart}key,value: string|{srcpart}value,ds: string|{srcpart}ds,hr: string|{srcpart}hr,BLOCK__OFFSET__INSIDE__FILE: bigint|{srcpart}block__offset__inside__file,INPUT__FILE__NAME: string|{srcpart}input__file__name,ROW__ID: struct|{srcpart}row__id,RAW__DATA__SIZE: bigint|{srcpart}raw__data__size) 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value] Column[ds] Column[hr] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: string|{}(tok_table_or_col ds),_col1: string|{}(tok_table_or_col hr),_col2: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(KEY._col0: string|{}(tok_table_or_col ds),KEY._col1: string|{}(tok_table_or_col hr),VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: string|{}(tok_table_or_col ds),_col1: string|{}(tok_table_or_col hr),_col2: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1,_col2: string|{null}ds,_col3: string|{null}hr) 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col2] Column[_col3] Column[_col0] Column[_col1]) 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{},_col2: string|{},_col3: string|{}) 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{srcpart}key,value: string|{srcpart}value,ds: string|{srcpart}ds,hr: string|{srcpart}hr,BLOCK__OFFSET__INSIDE__FILE: bigint|{srcpart}block__offset__inside__file,INPUT__FILE__NAME: string|{srcpart}input__file__name,ROW__ID: struct|{srcpart}row__id,RAW__DATA__SIZE: bigint|{srcpart}raw__data__size) 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: string|{srcpart}key,value: string|{srcpart}value,ds: string|{srcpart}ds,hr: string|{srcpart}hr,BLOCK__OFFSET__INSIDE__FILE: bigint|{srcpart}block__offset__inside__file,INPUT__FILE__NAME: string|{srcpart}input__file__name,ROW__ID: struct|{srcpart}row__id,RAW__DATA__SIZE: bigint|{srcpart}raw__data__size) 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value] Column[ds] Column[hr] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: string|{}(tok_table_or_col ds),_col1: string|{}(tok_table_or_col hr),_col2: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(KEY._col0: string|{}(tok_table_or_col ds),KEY._col1: string|{}(tok_table_or_col hr),VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: string|{}(tok_table_or_col ds),_col1: string|{}(tok_table_or_col hr),_col2: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1,_col2: string|{null}ds,_col3: string|{null}hr) 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:52,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:52,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col2] Column[_col3] Column[_col0] Column[_col1]) 2018-01-05T11:28:52,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{},_col2: string|{},_col3: string|{}) 2018-01-05T11:28:52,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:52,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:52,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,205 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,205 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partitions : db=default tbl=srcpart 2018-01-05T11:28:52,205 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions : db=default tbl=srcpart 2018-01-05T11:28:52,208 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,208 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000030, QueryId=djaiswal_20180105112851_c7191e3d-978b-4094-ad5d-df22da119ed8-16, DagId=16, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180532040, EndTime=1515180532208 2018-01-05T11:28:52,209 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.244103ms + 0.007226ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? ] 2018-01-05T11:28:52,209 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.360984ms + 0.114587ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (1,2,3,4) order by "PART_NAME" asc] 2018-01-05T11:28:52,210 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.111929ms + 0.216589ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (1,2,3,4) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:28:52,210 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.147775ms + 0.04034ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (1,2,3,4) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:52,210 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.110712ms + 0.011607ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (11,12,13,14) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:28:52,210 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.110573ms + 0.01321ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (11,12,13,14) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:52,210 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.094616ms + 0.008881ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (11,12,13,14) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:52,211 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.088971ms + 0.008717ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (11,12,13,14) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:52,211 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.120464ms + 0.028722ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (10) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:52,211 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.088015ms + 0.080967ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (11,12,13,14) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:28:52,211 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,220 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 3 key:[Column[_col0], Column[_col1]] 2018-01-05T11:28:52,222 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 3 oldColExprMap: {KEY._col0=Column[_col0], KEY._col1=Column[_col1], VALUE._col0=Column[_col2], VALUE._col1=Column[_col3]} 2018-01-05T11:28:52,222 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 3 newColExprMap: {KEY._col0=Column[_col0], KEY._col1=Column[_col1], VALUE._col0=Column[_col2], VALUE._col1=Column[_col3]} 2018-01-05T11:28:52,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: After logical optimization TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-SEL[5]-FS[6] 2018-01-05T11:28:52,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.StatsUtils: Estimated average row size: 200 2018-01-05T11:28:52,223 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,223 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_aggr_stats_for: db=default table=srcpart 2018-01-05T11:28:52,223 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_aggr_stats_for: db=default table=srcpart 2018-01-05T11:28:52,229 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.AggregateStatsCache: No aggregate stats cached for database:default, table:srcpart, column:key 2018-01-05T11:28:52,237 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 7.443674ms + 0.008624ms, the query is [select count("COLUMN_NAME") from "PART_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (?,?) and "PARTITION_NAME" in (?,?,?,?) group by "PARTITION_NAME"] 2018-01-05T11:28:52,240 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 2.204455ms + 0.009314ms, the query is [select "PARTITION_NAME", "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "PART_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (%1$s) AND "PARTITION_NAME" in (%2$s) order by "PARTITION_NAME"] 2018-01-05T11:28:52,240 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.MetaStoreUtils: No stats data found for: dbName=default tblName=srcpart partNames= [ds=2008-04-08/hr=11, ds=2008-04-08/hr=12, ds=2008-04-09/hr=11, ds=2008-04-09/hr=12] colNames=[key] 2018-01-05T11:28:52,240 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.AggregateStatsCache: No aggregate stats cached for database:default, table:srcpart, column:value 2018-01-05T11:28:52,240 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.164584ms + 0.003453ms, the query is [select "PARTITION_NAME", "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "PART_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (%1$s) AND "PARTITION_NAME" in (%2$s) order by "PARTITION_NAME"] 2018-01-05T11:28:52,240 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.MetaStoreUtils: No stats data found for: dbName=default tblName=srcpart partNames= [ds=2008-04-08/hr=11, ds=2008-04-08/hr=12, ds=2008-04-09/hr=11, ds=2008-04-09/hr=12] colNames=[value] 2018-01-05T11:28:52,240 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: useDensityFunctionForNDVEstimation = false partsFound = 0 ColumnStatisticsObj = [] 2018-01-05T11:28:52,241 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,242 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,243 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting stats (Num rows: 2000 Data size: 1456816 Basic stats: COMPLETE Column stats: PARTIAL) on TS[0] 2018-01-05T11:28:52,243 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (srcpart): numRows: 2000 dataSize: 1456816 basicStatsState: COMPLETE colStatsState: PARTIAL colStats: {hr= colName: hr colType: string countDistincts: 2 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, value= colName: value colType: string countDistincts: 400 numNulls: 100 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: string countDistincts: 400 numNulls: 100 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, ds= colName: ds colType: string countDistincts: 2 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:52,244 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting stats (Num rows: 2000 Data size: 1456816 Basic stats: COMPLETE Column stats: PARTIAL) on SEL[1] 2018-01-05T11:28:52,244 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-SEL[1]: numRows: 2000 dataSize: 1456816 basicStatsState: COMPLETE colStatsState: PARTIAL colStats: {ROW__ID= colName: ROW__ID colType: struct countDistincts: 2000 numNulls: 0 avgColLen: 76.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, RAW__DATA__SIZE= colName: RAW__DATA__SIZE colType: bigint countDistincts: 2000 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, BLOCK__OFFSET__INSIDE__FILE= colName: BLOCK__OFFSET__INSIDE__FILE colType: bigint countDistincts: 2000 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, hr= colName: hr colType: string countDistincts: 2 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, INPUT__FILE__NAME= colName: INPUT__FILE__NAME colType: string countDistincts: 2000 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, value= colName: value colType: string countDistincts: 400 numNulls: 100 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: string countDistincts: 400 numNulls: 100 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, ds= colName: ds colType: string countDistincts: 2 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:52,244 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[2]: inputSize: 1456816 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:28:52,244 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[2] hashAgg: true 2018-01-05T11:28:52,244 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [Case 3] STATS-GBY[2]: cardinality: 4 2018-01-05T11:28:52,244 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting stats (Num rows: 4 Data size: 4992 Basic stats: COMPLETE Column stats: PARTIAL) on GBY[2] 2018-01-05T11:28:52,244 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-GBY[2]: numRows: 4 dataSize: 4992 basicStatsState: COMPLETE colStatsState: PARTIAL colStats: {_col0= colName: _col0 colType: string countDistincts: 2 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: struct countDistincts: 4 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: string countDistincts: 2 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: struct countDistincts: 4 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:52,244 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting stats (Num rows: 4 Data size: 4992 Basic stats: COMPLETE Column stats: PARTIAL) on RS[3] 2018-01-05T11:28:52,244 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-RS[3]: numRows: 4 dataSize: 4992 basicStatsState: COMPLETE colStatsState: PARTIAL colStats: {KEY._col0= colName: KEY._col0 colType: string countDistincts: 2 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, KEY._col1= colName: KEY._col1 colType: string countDistincts: 2 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col0= colName: VALUE._col0 colType: struct countDistincts: 4 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col1= colName: VALUE._col1 colType: struct countDistincts: 4 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:52,244 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[4]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:28:52,244 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[4] hashAgg: false 2018-01-05T11:28:52,244 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [Case 9] STATS-GBY[4]: cardinality: 4 2018-01-05T11:28:52,244 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting stats (Num rows: 4 Data size: 4992 Basic stats: COMPLETE Column stats: PARTIAL) on GBY[4] 2018-01-05T11:28:52,244 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-GBY[4]: numRows: 4 dataSize: 4992 basicStatsState: COMPLETE colStatsState: PARTIAL colStats: {_col0= colName: _col0 colType: string countDistincts: 2 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: struct countDistincts: 4 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: string countDistincts: 2 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: struct countDistincts: 4 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:52,244 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting stats (Num rows: 4 Data size: 4992 Basic stats: COMPLETE Column stats: PARTIAL) on SEL[5] 2018-01-05T11:28:52,244 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-SEL[5]: numRows: 4 dataSize: 4992 basicStatsState: COMPLETE colStatsState: PARTIAL colStats: {_col0= colName: _col0 colType: struct countDistincts: 4 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: string countDistincts: 2 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 4 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: string countDistincts: 2 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:52,244 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-FS[6]: numRows: 4 dataSize: 4992 basicStatsState: COMPLETE colStatsState: PARTIAL colStats: {_col0= colName: _col0 colType: struct countDistincts: 4 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: string countDistincts: 2 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 4 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: string countDistincts: 2 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:52,244 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on TS[0] 2018-01-05T11:28:52,244 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on SEL[1] 2018-01-05T11:28:52,244 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting traits ({ bucket column names: [[_col0, _col1]]; sort column names: [[_col0, _col1]]; bucket count: -1 }) on GBY[2] 2018-01-05T11:28:52,244 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[KEY._col0, KEY._col1]]; sort column names: [[KEY._col0, KEY._col1]]; bucket count: -1 }) on RS[3] 2018-01-05T11:28:52,244 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting traits ({ bucket column names: [[_col0, _col1]]; sort column names: [[_col0, _col1]]; bucket count: -1 }) on GBY[4] 2018-01-05T11:28:52,244 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col2, _col3]]; sort column names: [[_col2, _col3]]; bucket count: -1 }) on SEL[5] 2018-01-05T11:28:52,244 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FileSinkOperator: Setting traits ({ bucket column names: [[_col2, _col3]]; sort column names: [[_col2, _col3]]; bucket count: -1 }) on FS[6] 2018-01-05T11:28:52,244 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,244 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,244 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.SetReducerParallelism: Set parallelism for reduce sink RS[3] to: 1 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: FS, 6 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: SEL, 5 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: GBY, 4 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: RS, 3 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: GBY, 2 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: SEL, 1 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:28:52,245 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Cycle free: true 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{srcpart}key,value: string|{srcpart}value,ds: string|{srcpart}ds,hr: string|{srcpart}hr,BLOCK__OFFSET__INSIDE__FILE: bigint|{srcpart}block__offset__inside__file,INPUT__FILE__NAME: string|{srcpart}input__file__name,ROW__ID: struct|{srcpart}row__id,RAW__DATA__SIZE: bigint|{srcpart}raw__data__size) 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: string|{srcpart}key,value: string|{srcpart}value,ds: string|{srcpart}ds,hr: string|{srcpart}hr) 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value] Column[ds] Column[hr]) 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: string|{}(tok_table_or_col ds),_col1: string|{}(tok_table_or_col hr),_col2: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(KEY._col0: string|{}(tok_table_or_col ds),KEY._col1: string|{}(tok_table_or_col hr),VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: string|{}(tok_table_or_col ds),_col1: string|{}(tok_table_or_col hr),_col2: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1,_col2: string|{null}ds,_col3: string|{null}hr) 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col2] Column[_col3] Column[_col0] Column[_col1]) 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{},_col2: string|{},_col3: string|{}) 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: TS[0] 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: RS[3] 2018-01-05T11:28:52,245 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:28:52,246 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11 of table srcpart 2018-01-05T11:28:52,246 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12 of table srcpart 2018-01-05T11:28:52,246 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11 of table srcpart 2018-01-05T11:28:52,246 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12 of table srcpart 2018-01-05T11:28:52,246 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11 2018-01-05T11:28:52,246 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12 2018-01-05T11:28:52,246 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11 2018-01-05T11:28:52,246 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12 2018-01-05T11:28:52,246 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: RS[3] 2018-01-05T11:28:52,246 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: GBY[4] 2018-01-05T11:28:52,246 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: FS[6] 2018-01-05T11:28:52,246 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding reduce work (Reducer 2) for GBY[4] 2018-01-05T11:28:52,246 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Setting up reduce sink: RS[3] with following reduce work: Reducer 2 2018-01-05T11:28:52,246 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Removing RS[3] as parent from GBY[4] 2018-01-05T11:28:52,246 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: FS[6] 2018-01-05T11:28:52,246 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:28:52,246 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,246 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,246 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:52,246 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:52,246 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:52,246 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:52,246 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:52,246 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:52,246 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:28:52,246 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping vectorization 2018-01-05T11:28:52,246 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:28:52,246 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: llap mode: only 2018-01-05T11:28:52,246 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:28:52,247 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(key, 'hll')' 2018-01-05T11:28:52,247 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'key' 2018-01-05T11:28:52,247 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:52,247 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(value, 'hll')' 2018-01-05T11:28:52,247 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'value' 2018-01-05T11:28:52,247 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:52,247 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'key' 2018-01-05T11:28:52,247 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'value' 2018-01-05T11:28:52,247 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'ds' 2018-01-05T11:28:52,247 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'hr' 2018-01-05T11:28:52,247 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:52,247 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@795cb721 can be done in LLAP 2018-01-05T11:28:52,247 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Reducer 2 2018-01-05T11:28:52,247 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking '_col2' 2018-01-05T11:28:52,247 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking '_col3' 2018-01-05T11:28:52,247 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking '_col0' 2018-01-05T11:28:52,247 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking '_col1' 2018-01-05T11:28:52,247 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col0)' 2018-01-05T11:28:52,247 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:28:52,247 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col1)' 2018-01-05T11:28:52,247 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col1' 2018-01-05T11:28:52,247 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:52,247 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@6788168c can be done in LLAP 2018-01-05T11:28:52,247 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] impl.LlapRegistryService: Using LLAP registry (client) type: Service LlapRegistryService in state LlapRegistryService: STARTED 2018-01-05T11:28:52,247 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,247 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed plan generation 2018-01-05T11:28:52,247 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:52,247 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validation start 2018-01-05T11:28:52,247 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Not a partition. 2018-01-05T11:28:52,247 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:28:52,247 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validated ds=2008-04-08/hr=11 2018-01-05T11:28:52,247 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: srcpart 2018-01-05T11:28:52,247 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,247 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partitions_ps_with_auth : db=default tbl=srcpart[2008-04-08,11] 2018-01-05T11:28:52,247 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions_ps_with_auth : db=default tbl=srcpart[2008-04-08,11] 2018-01-05T11:28:52,250 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,256 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,257 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,257 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,257 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcpart[2008-04-08,11] 2018-01-05T11:28:52,257 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcpart[2008-04-08,11] 2018-01-05T11:28:52,260 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,263 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,264 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,264 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validated ds=2008-04-08/hr=12 2018-01-05T11:28:52,264 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: srcpart 2018-01-05T11:28:52,264 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,264 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partitions_ps_with_auth : db=default tbl=srcpart[2008-04-08,12] 2018-01-05T11:28:52,264 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions_ps_with_auth : db=default tbl=srcpart[2008-04-08,12] 2018-01-05T11:28:52,267 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,273 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,273 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,273 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,273 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcpart[2008-04-08,12] 2018-01-05T11:28:52,273 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcpart[2008-04-08,12] 2018-01-05T11:28:52,276 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,279 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,280 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,280 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validated ds=2008-04-09/hr=11 2018-01-05T11:28:52,280 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: srcpart 2018-01-05T11:28:52,280 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,280 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partitions_ps_with_auth : db=default tbl=srcpart[2008-04-09,11] 2018-01-05T11:28:52,280 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions_ps_with_auth : db=default tbl=srcpart[2008-04-09,11] 2018-01-05T11:28:52,282 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,290 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,291 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,291 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,291 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcpart[2008-04-09,11] 2018-01-05T11:28:52,291 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcpart[2008-04-09,11] 2018-01-05T11:28:52,293 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,296 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,296 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,296 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validated ds=2008-04-09/hr=12 2018-01-05T11:28:52,296 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: srcpart 2018-01-05T11:28:52,296 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,297 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partitions_ps_with_auth : db=default tbl=srcpart[2008-04-09,12] 2018-01-05T11:28:52,297 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions_ps_with_auth : db=default tbl=srcpart[2008-04-09,12] 2018-01-05T11:28:52,299 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,305 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,306 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,306 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,306 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcpart[2008-04-09,12] 2018-01-05T11:28:52,306 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcpart[2008-04-09,12] 2018-01-05T11:28:52,309 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,313 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,313 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,313 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,313 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0, type:struct, comment:null), FieldSchema(name:_c1, type:struct, comment:null), FieldSchema(name:ds, type:string, comment:null), FieldSchema(name:hr, type:string, comment:null)], properties:null) 2018-01-05T11:28:52,313 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,313 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:52,313 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getAggrColStatsFor_(String, String, List, List, )=19, getTable_(String, String, )=8, flushCache_()=1, getPartitionWithAuthInfo_(String, String, List, String, List, )=26, listPartitions_(String, String, short, )=16, listPartitionsWithAuthInfo_(String, String, List, short, String, List, )=40} 2018-01-05T11:28:52,313 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab); Time taken: 0.166 seconds 2018-01-05T11:28:52,313 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:52,313 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,313 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab): ANALYZE TABLE srcpart PARTITION(ds, hr) COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:52,315 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,315 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: ANALYZE TABLE srcpart PARTITION(ds, hr) COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:52,315 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:28:52,315 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@srcpart 2018-01-05T11:28:52,315 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@srcpart@ds=2008-04-08/hr=11 2018-01-05T11:28:52,315 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@srcpart@ds=2008-04-08/hr=12 2018-01-05T11:28:52,315 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@srcpart@ds=2008-04-09/hr=11 2018-01-05T11:28:52,315 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@srcpart@ds=2008-04-09/hr=12 2018-01-05T11:28:52,315 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@srcpart 2018-01-05T11:28:52,315 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@srcpart@ds=2008-04-08/hr=11 2018-01-05T11:28:52,315 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@srcpart@ds=2008-04-08/hr=12 2018-01-05T11:28:52,315 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@srcpart@ds=2008-04-09/hr=11 2018-01-05T11:28:52,315 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@srcpart@ds=2008-04-09/hr=12 2018-01-05T11:28:52,315 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_151_4467563815339635813-1/-mr-10000 2018-01-05T11:28:52,315 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,315 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,315 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,315 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Query ID = djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab 2018-01-05T11:28:52,315 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Total jobs = 1 2018-01-05T11:28:52,315 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,315 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:28:52,315 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MAPRED] in serial mode 2018-01-05T11:28:52,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_147_6959340210354157653-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:28:52,359 DEBUG [AMReporterQueueDrainer] impl.AMReporter: Removing am 10.22.8.182:49824 with last associated dag QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=3} from heartbeat with taskCount=0, amFailed=false, isDone=true 2018-01-05T11:28:52,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:28:52,362 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:28:52,362 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:28:52,362 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab 2018-01-05T11:28:52,362 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Session is already open 2018-01-05T11:28:52,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:28:52,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,362 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Dag name: ANALYZE TABLE srcpart PARTITION(...key,value (Stage-0) 2018-01-05T11:28:52,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: DagInfo: {"context":"Hive","description":"\n\nANALYZE TABLE srcpart PARTITION(ds, hr) COMPUTE STATISTICS FOR COLUMNS key,value"} 2018-01-05T11:28:52,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:28:52,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,375 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,375 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:28:52,375 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,376 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 2 size: 2.39KB 2018-01-05T11:28:52,391 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,391 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,406 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:28:52,407 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,407 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:28:52,408 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,408 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 3.77KB 2018-01-05T11:28:52,413 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_151_4467563815339635813-1/-mr-10001 2018-01-05T11:28:52,424 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_151_4467563815339635813-1/-mr-10001 2018-01-05T11:28:52,424 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11 2018-01-05T11:28:52,424 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12 2018-01-05T11:28:52,424 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11 2018-01-05T11:28:52,424 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12 2018-01-05T11:28:52,424 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,445 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,445 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,445 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=ANALYZE TABLE srcpart PARTITION(...key,value (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab } 2018-01-05T11:28:52,446 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:52,446 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:52,448 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: ANALYZE TABLE srcpart PARTITION(...key,value (Stage-0) 2018-01-05T11:28:52,448 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:28:52,449 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_17, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_17.dot 2018-01-05T11:28:52,449 INFO [IPC Server handler 1 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_17 2018-01-05T11:28:52,449 WARN [IPC Server handler 1 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:52,449 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Running DAG: ANALYZE TABLE srcpart PARTITION(...key,value (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab } 2018-01-05T11:28:52,472 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_17][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_17, submitTime=1515180532448, queueName=null 2018-01-05T11:28:52,473 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_17_00 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_17_00 [Map 1], Default Resources= 2018-01-05T11:28:52,473 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_17_00 [Map 1] 2018-01-05T11:28:52,473 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:28:52,473 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_17_00 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:52,473 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_17_01 [Reducer 2]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_17_01 [Reducer 2], Default Resources= 2018-01-05T11:28:52,473 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_17_01 [Reducer 2] 2018-01-05T11:28:52,473 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 2 2018-01-05T11:28:52,473 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_17_01 [Reducer 2] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:52,474 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:28:52,474 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_17][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_17, initTime=1515180532474 2018-01-05T11:28:52,474 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: dag_1515180518813_0001_17 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:28:52,474 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:28:52,474 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_17][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_17, startTime=1515180532474 2018-01-05T11:28:52,474 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_17 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:28:52,474 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {srcpart={InputName=srcpart}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:28:52,474 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: srcpart, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:28:52,474 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_17_00 [Map 1] 2018-01-05T11:28:52,475 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_17, dagName=ANALYZE TABLE srcpart PARTITION(...key,value (Stage-0) 2018-01-05T11:28:52,475 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,475 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,475 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,478 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_17_00 [Map 1] 2018-01-05T11:28:52,478 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_17_00 [Map 1] 2018-01-05T11:28:52,478 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:52,478 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_17_00 [Map 1] 2018-01-05T11:28:52,481 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:28:52,481 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@171c81f5 2018-01-05T11:28:52,481 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_147_6959340210354157653-1/djaiswal/_tez_scratch_dir/fe6eb59b-a496-4119-8b2e-b1640d9c2cea/map.xml 2018-01-05T11:28:52,481 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_147_6959340210354157653-1/djaiswal/_tez_scratch_dir/fe6eb59b-a496-4119-8b2e-b1640d9c2cea/map.xml 2018-01-05T11:28:52,481 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_147_6959340210354157653-1/djaiswal/_tez_scratch_dir/fe6eb59b-a496-4119-8b2e-b1640d9c2cea/map.xml 2018-01-05T11:28:52,481 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:52,481 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:52,481 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,482 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:52,482 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.77KB 2018-01-05T11:28:52,482 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_17_00 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:28:52,482 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: srcpart on vertex vertex_1515180518813_0001_17_00 [Map 1] 2018-01-05T11:28:52,483 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to ShuffleVertexManager for vertex_1515180518813_0001_17_01 [Reducer 2] 2018-01-05T11:28:52,483 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:28:52,483 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:52,483 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:28:52,483 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:52,483 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_147_6959340210354157653-1/djaiswal/_tez_scratch_dir/fe6eb59b-a496-4119-8b2e-b1640d9c2cea/map.xml 2018-01-05T11:28:52,483 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:52,483 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias srcpart 2018-01-05T11:28:52,483 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11 2018-01-05T11:28:52,483 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 4 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11 2018-01-05T11:28:52,483 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12 2018-01-05T11:28:52,483 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11 2018-01-05T11:28:52,483 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12 2018-01-05T11:28:52,483 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 0,1 2018-01-05T11:28:52,483 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = key,value 2018-01-05T11:28:52,483 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11 2018-01-05T11:28:52,483 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:52,483 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:52,483 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:52,483 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:52,487 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.05 maxFrac: 0.75 auto: false desiredTaskIput: 104857600 2018-01-05T11:28:52,487 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:28:52,487 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 4 tasks for vertex: vertex_1515180518813_0001_17_01 [Reducer 2] 2018-01-05T11:28:52,487 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_17_01 [Reducer 2] 2018-01-05T11:28:52,487 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_17][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_17_01, reconfigureDoneTime=1515180532487, numTasks=4, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:28:52,487 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_17_01 [Reducer 2], numAdditionalOutputs=1 2018-01-05T11:28:52,487 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_17][Event:VERTEX_INITIALIZED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_17_01, initRequestedTime=1515180532482, initedTime=1515180532487, numTasks=4, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:52,487 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_17_01 [Reducer 2] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:28:52,539 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Time taken to get FileStatuses: 55 2018-01-05T11:28:52,539 INFO [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total input files to process : 4 2018-01-05T11:28:52,539 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total # of splits generated by getSplits: 4, TimeTaken: 55 2018-01-05T11:28:52,539 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 4 2018-01-05T11:28:52,539 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:52,539 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 4. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:28:52,539 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_147_6959340210354157653-1/djaiswal/_tez_scratch_dir/fe6eb59b-a496-4119-8b2e-b1640d9c2cea/map.xml 2018-01-05T11:28:52,539 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_147_6959340210354157653-1/djaiswal/_tez_scratch_dir/fe6eb59b-a496-4119-8b2e-b1640d9c2cea/map.xml 2018-01-05T11:28:52,539 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_147_6959340210354157653-1/djaiswal/_tez_scratch_dir/fe6eb59b-a496-4119-8b2e-b1640d9c2cea/map.xml 2018-01-05T11:28:52,539 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:52,539 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:52,539 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,541 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:52,541 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.77KB 2018-01-05T11:28:52,541 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11/kv1.txt to src new group? true 2018-01-05T11:28:52,541 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12/kv1.txt to src new group? false 2018-01-05T11:28:52,541 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11/kv1.txt to src new group? false 2018-01-05T11:28:52,541 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12/kv1.txt to src new group? false 2018-01-05T11:28:52,541 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:28:52,541 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:28:52,541 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:28:52,541 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 430 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 23248 Original splits: 4 2018-01-05T11:28:52,541 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired numSplits: 1 lengthPerGroup: 23248 numLocations: 1 numSplitsPerLocation: 4 numSplitsInGroup: 4 totalLength: 23248 numOriginalSplits: 4 . Grouping by length: true count: false nodeLocalOnly: false 2018-01-05T11:28:52,541 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Number of splits desired: 1 created: 1 splitsProcessed: 4 2018-01-05T11:28:52,541 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 4 grouped split count is 1, for bucket: 1 2018-01-05T11:28:52,541 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:28:52,541 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: srcpart on vertex vertex_1515180518813_0001_17_00 [Map 1] 2018-01-05T11:28:52,542 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_17_00 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:52,542 INFO [App Shared Pool - #0] impl.VertexImpl: Got updated RootInputsSpecs: {srcpart=forAllWorkUnits=true, update=[1]} 2018-01-05T11:28:52,543 INFO [App Shared Pool - #0] impl.VertexImpl: Vertex vertex_1515180518813_0001_17_00 [Map 1] parallelism set to 1 2018-01-05T11:28:52,543 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_17_00 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:52,543 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_17][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_17_00, initRequestedTime=1515180532474, initedTime=1515180532543, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:52,543 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_17_00 [Map 1] with distanceFromRoot: 0 2018-01-05T11:28:52,543 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_17_00 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:28:52,543 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_17][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_17_00, startRequestedTime=1515180532482, startedTime=1515180532543 2018-01-05T11:28:52,543 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_17][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_17_00, reconfigureDoneTime=1515180532543, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:28:52,543 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_17_00 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:52,543 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_17_00 for vertex: vertex_1515180518813_0001_17_01 [Reducer 2] numStartedSources: 1 numSources: 1 2018-01-05T11:28:52,543 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_17_01 [Reducer 2] with distanceFromRoot: 1 2018-01-05T11:28:52,543 INFO [App Shared Pool - #0] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:28:52,543 INFO [App Shared Pool - #0] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:28:52,543 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_17][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_17_01, startRequestedTime=1515180532543, startedTime=1515180532543 2018-01-05T11:28:52,543 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_17_00 [Map 1] 2018-01-05T11:28:52,543 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_17_01 [Reducer 2] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:52,544 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_17][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_17_00_000000, scheduledTime=1515180532544, launchTime=1515180532544 2018-01-05T11:28:52,544 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: OnVertexStarted vertex: Reducer 2 with 0 source tasks and 4 pending tasks 2018-01-05T11:28:52,544 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_17_00 [Map 1] 2018-01-05T11:28:52,544 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Map 1 in vertex: Reducer 2 numBipartiteSourceTasks: 1 2018-01-05T11:28:52,544 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_17_00_000000_0, priority=5, capability=, hosts=null 2018-01-05T11:28:52,544 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=5,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:52,544 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=5,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:52,544 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:52,544 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_17_00_000000_0 on hosts=null 2018-01-05T11:28:52,544 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_17_00_000000_0 on hosts=null 2018-01-05T11:28:52,544 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_17_00_000000_0 2018-01-05T11:28:52,544 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:52,544 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #31, task=TaskInfo{task=attempt_1515180518813_0001_17_00_000000_0, priority=5, startTime=0, containerId=null, uniqueId=30, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000031 2018-01-05T11:28:52,544 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_17_00_000000_0; false 2018-01-05T11:28:52,544 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_17_00_000000_0, priority=5, startTime=311553053, containerId=container_111101111_0001_00_000031assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=30, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:52,545 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000031 for node: 10.22.8.182:49818 2018-01-05T11:28:52,547 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 17, name=ANALYZE TABLE srcpart PARTITION(...key,value (Stage-0), queryId=djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab 2018-01-05T11:28:52,547 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000031, attempt_1515180518813_0001_17_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:52,547 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000031 for node: 10.22.8.182:49818 2018-01-05T11:28:52,547 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000031 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:52,547 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000031, launchTime=1515180532547 2018-01-05T11:28:52,548 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_17_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000031, dagName=ANALYZE TABLE srcpart PARTITION(...key,value (Stage-0), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{srcpart,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{Reducer 2,org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput,4}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180532474, firstAttemptStartTime=1515180532544, currentAttemptStartTime=1515180532547} 2018-01-05T11:28:52,548 INFO [IPC Server handler 4 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000031, QueryId=djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab-17, DagId=17, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180532548 2018-01-05T11:28:52,548 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_17_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000031] on NM: [10.22.8.182:49818] 2018-01-05T11:28:52,548 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:52,548 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:52,548 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_17][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_17_00_000000_0, startTime=1515180532547, containerId=container_111101111_0001_00_000031, nodeId=10.22.8.182:49818 2018-01-05T11:28:52,548 DEBUG [IPC Server handler 4 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=17} with the ShuffleHandler 2018-01-05T11:28:52,559 INFO [IPC Server handler 4 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:28:52,559 DEBUG [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/17] 2018-01-05T11:28:52,560 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:52,560 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_17_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:52,560 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_17_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180532544, dagStartTime=1515180532474, withinDagPriority=5, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:28:52,560 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_17_00_000000_0 : ACCEPTED 2018-01-05T11:28:52,560 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_17_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:52,560 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_17_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180532544, dagStartTime=1515180532474, withinDagPriority=5, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:28:52,560 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_17_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:52,560 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_17_00_000000_0 to pre-emption queue 2018-01-05T11:28:52,560 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_17_00_000000_0: true 2018-01-05T11:28:52,560 INFO [TaskCommunicator # 6] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_17_00_000000_0 2018-01-05T11:28:52,560 INFO [TaskCommunicator # 6] GuaranteedTasks: Received response for attempt_1515180518813_0001_17_00_000000_0, true 2018-01-05T11:28:52,560 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:313075933 2018-01-05T11:28:52,561 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : ANALYZE TABLE srcpart PARTITION(...key,value (Stage-0), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_17_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=srcpart, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=Reducer 2, physicalEdgeCount=4, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ] 2018-01-05T11:28:52,561 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1043333120, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:52,561 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_17_00_000000_0 vertexName: Map 1 2018-01-05T11:28:52,561 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000031, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_17_00_000000_0, eventCount=1 } 2018-01-05T11:28:52,562 DEBUG [TezTR-518813_1_17_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_17 queryId: djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab instanceType: TaskRunner2Callable 2018-01-05T11:28:52,562 INFO [TezTR-518813_1_17_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_17_00_000000_0 2018-01-05T11:28:52,562 DEBUG [TezTR-518813_1_17_0_0_0] log.PerfLogger: 2018-01-05T11:28:52,563 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:52,564 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:52,565 INFO [IPC Server handler 4 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_17_00_000000_0 sent events: (0-1). 2018-01-05T11:28:52,566 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:28:52,566 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_17_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:28:52,566 DEBUG [TezTR-518813_1_17_0_0_0] log.PerfLogger: 2018-01-05T11:28:52,566 INFO [TezTR-518813_1_17_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:52,566 INFO [TezTR-518813_1_17_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:52,567 INFO [TezTR-518813_1_17_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:52,567 INFO [I/O Setup 0 Initialize: {srcpart}] input.MRInput: srcpart using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:28:52,567 INFO [I/O Setup 0 Initialize: {srcpart}] input.MRInputLegacy: srcpart MRInputLegacy deferring initialization 2018-01-05T11:28:52,567 INFO [TezTR-518813_1_17_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:52,567 INFO [TezTR-518813_1_17_0_0_0] resources.MemoryDistributor: InitialRequests=[srcpart:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [Reducer 2:OUTPUT:10485760:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput] 2018-01-05T11:28:52,567 INFO [TezTR-518813_1_17_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:52,567 INFO [TezTR-518813_1_17_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:52,567 INFO [TezTR-518813_1_17_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 10485760, TotalRequestedScaled: 9679163.076923078, TotalJVMHeap: 1043333120, TotalAvailable: 699033190, TotalRequested/TotalJVMHeap:0.01 2018-01-05T11:28:52,567 INFO [TezTR-518813_1_17_0_0_0] resources.MemoryDistributor: Allocations=[srcpart:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:10485760:10485760] 2018-01-05T11:28:52,567 INFO [TezTR-518813_1_17_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:52,567 INFO [TezTR-518813_1_17_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: srcpart being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:28:52,567 INFO [TezTR-518813_1_17_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:28:52,567 INFO [TezTR-518813_1_17_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:28:52,567 INFO [I/O Setup 1 Start: {srcpart}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: srcpart 2018-01-05T11:28:52,567 INFO [TezTR-518813_1_17_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:52,567 INFO [TezTR-518813_1_17_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_17_00_000000_0 2018-01-05T11:28:52,567 DEBUG [TezTR-518813_1_17_0_0_0] log.PerfLogger: 2018-01-05T11:28:52,567 DEBUG [TezTR-518813_1_17_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_17_00_000000_0_10091 2018-01-05T11:28:52,567 DEBUG [TezTR-518813_1_17_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:52,567 DEBUG [TezTR-518813_1_17_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_17_00_000000_0 2018-01-05T11:28:52,567 INFO [TezTR-518813_1_17_0_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab 2018-01-05T11:28:52,567 DEBUG [TezTR-518813_1_17_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:52,568 DEBUG [TezTR-518813_1_17_0_0_0] log.PerfLogger: 2018-01-05T11:28:52,568 DEBUG [TezTR-518813_1_17_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:52,568 DEBUG [TezTR-518813_1_17_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:52,568 DEBUG [TezTR-518813_1_17_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab_Map 1__MAP_PLAN__ 2018-01-05T11:28:52,568 INFO [TezTR-518813_1_17_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_147_6959340210354157653-1/djaiswal/_tez_scratch_dir/fe6eb59b-a496-4119-8b2e-b1640d9c2cea/map.xml 2018-01-05T11:28:52,568 DEBUG [TezTR-518813_1_17_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_147_6959340210354157653-1/djaiswal/_tez_scratch_dir/fe6eb59b-a496-4119-8b2e-b1640d9c2cea/map.xml 2018-01-05T11:28:52,568 DEBUG [TezTR-518813_1_17_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_147_6959340210354157653-1/djaiswal/_tez_scratch_dir/fe6eb59b-a496-4119-8b2e-b1640d9c2cea/map.xml 2018-01-05T11:28:52,568 DEBUG [TezTR-518813_1_17_0_0_0] log.PerfLogger: 2018-01-05T11:28:52,568 INFO [TezTR-518813_1_17_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:52,568 DEBUG [TezTR-518813_1_17_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,569 DEBUG [TezTR-518813_1_17_0_0_0] log.PerfLogger: 2018-01-05T11:28:52,569 INFO [TezTR-518813_1_17_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.77KB 2018-01-05T11:28:52,569 DEBUG [TezTR-518813_1_17_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:52,569 INFO [TezTR-518813_1_17_0_0_0] tez.MapRecordProcessor: The input names are: [srcpart] 2018-01-05T11:28:52,569 INFO [TezTR-518813_1_17_0_0_0] input.MRInput: Initialized MRInput: srcpart 2018-01-05T11:28:52,569 DEBUG [TezTR-518813_1_17_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 4 2018-01-05T11:28:52,570 INFO [TezTR-518813_1_17_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_147_6959340210354157653-1/djaiswal/_tez_scratch_dir/fe6eb59b-a496-4119-8b2e-b1640d9c2cea/map.xml 2018-01-05T11:28:52,570 DEBUG [TezTR-518813_1_17_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:52,570 DEBUG [TezTR-518813_1_17_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11/kv1.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-08, hr=11}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11, columns=key,value, partition_columns.types=string:string, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180532, rawDataSize=5312, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=11, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-08, hr=11}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11, columns=key,value, partition_columns.types=string:string, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180532, rawDataSize=5312, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=11, vectorPartitionDesc=null], file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-08, hr=12}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12, columns=key,value, partition_columns.types=string:string, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180532, rawDataSize=5312, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=12, vectorPartitionDesc=null], file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-09, hr=11}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11, columns=key,value, partition_columns.types=string:string, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180532, rawDataSize=5312, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=11, vectorPartitionDesc=null], file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-09, hr=12}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12, columns=key,value, partition_columns.types=string:string, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180532, rawDataSize=5312, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=12, vectorPartitionDesc=null]} 2018-01-05T11:28:52,570 DEBUG [TezTR-518813_1_17_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:52,570 INFO [TezTR-518813_1_17_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:52,570 DEBUG [TezTR-518813_1_17_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:52,571 DEBUG [TezTR-518813_1_17_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:52,571 INFO [TezTR-518813_1_17_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_147_6959340210354157653-1/djaiswal/_tez_scratch_dir/fe6eb59b-a496-4119-8b2e-b1640d9c2cea/map.xml 2018-01-05T11:28:52,571 DEBUG [TezTR-518813_1_17_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:52,571 INFO [TezTR-518813_1_17_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_17_00_000000_0 2018-01-05T11:28:52,571 INFO [TezTR-518813_1_17_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [string, string] (isAcidRead false) 2018-01-05T11:28:52,571 DEBUG [TezTR-518813_1_17_0_0_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:52,571 DEBUG [TezTR-518813_1_17_0_0_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:52,571 DEBUG [TezTR-518813_1_17_0_0_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:52,571 DEBUG [TezTR-518813_1_17_0_0_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:52,571 WARN [TezTR-518813_1_17_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11/kv1.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:28:52,572 DEBUG [TezTR-518813_1_17_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:28:52,572 WARN [TezTR-518813_1_17_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11/kv1.txt:0+5812 2018-01-05T11:28:52,572 DEBUG [TezTR-518813_1_17_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11/kv1.txt 2018-01-05T11:28:52,572 INFO [TezTR-518813_1_17_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11/kv1.txt:0+5812, org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12/kv1.txt:0+5812, org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11/kv1.txt:0+5812, org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12/kv1.txt:0+5812], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=23248} 2018-01-05T11:28:52,572 INFO [TezTR-518813_1_17_0_0_0] input.MRInput: srcpart initialized RecordReader from event 2018-01-05T11:28:52,572 DEBUG [TezTR-518813_1_17_0_0_0] tez.MapRecordProcessor: Starting Output: Reducer 2 2018-01-05T11:28:52,577 INFO [TezTR-518813_1_17_0_0_0] impl.ExternalSorter: Reducer 2 using: memoryMb=10, keySerializerClass=class org.apache.hadoop.hive.ql.io.HiveKey, valueSerializerClass=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization$TezBytesWritableSerializer@6570acba, comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator@74890567, partitioner=org.apache.tez.mapreduce.partition.MRPartitioner, serialization=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.hadoop.io.serializer.WritableSerialization, org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization, org.apache.hadoop.io.serializer.avro.AvroReflectSerialization, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:28:52,577 INFO [TezTR-518813_1_17_0_0_0] partition.MRPartitioner: Using oldApi, MRpartitionerClass=org.apache.hadoop.hive.ql.io.DefaultHivePartitioner 2018-01-05T11:28:52,577 INFO [TezTR-518813_1_17_0_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 2: , UsingHashComparator=true 2018-01-05T11:28:52,578 INFO [TezTR-518813_1_17_0_0_0] impl.PipelinedSorter: Newly allocated block size=10485760, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=10485760, total=10485760 2018-01-05T11:28:52,578 INFO [TezTR-518813_1_17_0_0_0] impl.PipelinedSorter: Pre allocating rest of memory buffers upfront 2018-01-05T11:28:52,578 INFO [TezTR-518813_1_17_0_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 2: , UsingHashComparator=true#blocks=1, maxMemUsage=10485760, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=10485760, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=10 2018-01-05T11:28:52,582 INFO [TezTR-518813_1_17_0_0_0] impl.PipelinedSorter: Reducer 2: reserved.remaining()=10485760, reserved.metasize=5242880 2018-01-05T11:28:52,586 INFO [TezTR-518813_1_17_0_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:28:52,586 INFO [TezTR-518813_1_17_0_0_0] exec.MapOperator: Initializing operator MAP[0] 2018-01-05T11:28:52,586 DEBUG [TezTR-518813_1_17_0_0_0] exec.MapOperator: Initialization Done 0 MAP 2018-01-05T11:28:52,586 DEBUG [TezTR-518813_1_17_0_0_0] exec.MapOperator: Operator 0 MAP initialized 2018-01-05T11:28:52,586 DEBUG [TezTR-518813_1_17_0_0_0] exec.MapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:28:52,587 DEBUG [TezTR-518813_1_17_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,587 DEBUG [TezTR-518813_1_17_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,587 DEBUG [TezTR-518813_1_17_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,587 DEBUG [TezTR-518813_1_17_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,587 DEBUG [TezTR-518813_1_17_0_0_0] exec.MapOperator: Adding alias srcpart to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11 2018-01-05T11:28:52,587 DEBUG [TezTR-518813_1_17_0_0_0] exec.MapOperator: Adding alias srcpart to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12 2018-01-05T11:28:52,587 DEBUG [TezTR-518813_1_17_0_0_0] exec.MapOperator: Adding alias srcpart to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11 2018-01-05T11:28:52,587 DEBUG [TezTR-518813_1_17_0_0_0] exec.MapOperator: Adding alias srcpart to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12 2018-01-05T11:28:52,588 DEBUG [TezTR-518813_1_17_0_0_0] exec.MapOperator: dump TS[0] struct 2018-01-05T11:28:52,588 DEBUG [TezTR-518813_1_17_0_0_0] exec.MapOperator: dump TS[0] struct 2018-01-05T11:28:52,588 DEBUG [TezTR-518813_1_17_0_0_0] exec.MapOperator: dump TS[0] struct 2018-01-05T11:28:52,588 DEBUG [TezTR-518813_1_17_0_0_0] exec.MapOperator: dump TS[0] struct 2018-01-05T11:28:52,588 INFO [TezTR-518813_1_17_0_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:28:52,588 INFO [TezTR-518813_1_17_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:52,588 DEBUG [TezTR-518813_1_17_0_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:28:52,588 INFO [TezTR-518813_1_17_0_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:28:52,588 DEBUG [TezTR-518813_1_17_0_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:28:52,588 DEBUG [TezTR-518813_1_17_0_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:28:52,588 DEBUG [TezTR-518813_1_17_0_0_0] exec.TableScanOperator: Initializing children of 0 TS 2018-01-05T11:28:52,588 DEBUG [TezTR-518813_1_17_0_0_0] exec.SelectOperator: Initializing child 1 SEL 2018-01-05T11:28:52,588 INFO [TezTR-518813_1_17_0_0_0] exec.SelectOperator: Initializing operator SEL[1] 2018-01-05T11:28:52,588 INFO [TezTR-518813_1_17_0_0_0] exec.SelectOperator: SELECT struct 2018-01-05T11:28:52,588 DEBUG [TezTR-518813_1_17_0_0_0] exec.SelectOperator: Initialization Done 1 SEL 2018-01-05T11:28:52,588 DEBUG [TezTR-518813_1_17_0_0_0] exec.SelectOperator: Operator 1 SEL initialized 2018-01-05T11:28:52,588 DEBUG [TezTR-518813_1_17_0_0_0] exec.SelectOperator: Initializing children of 1 SEL 2018-01-05T11:28:52,588 DEBUG [TezTR-518813_1_17_0_0_0] exec.GroupByOperator: Initializing child 2 GBY 2018-01-05T11:28:52,588 INFO [TezTR-518813_1_17_0_0_0] exec.GroupByOperator: Initializing operator GBY[2] 2018-01-05T11:28:52,588 DEBUG [TezTR-518813_1_17_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,588 DEBUG [TezTR-518813_1_17_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,591 INFO [TezTR-518813_1_17_0_0_0] exec.GroupByOperator: Max hash table memory: 93860656 bytes 2018-01-05T11:28:52,591 INFO [TezTR-518813_1_17_0_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:52,591 DEBUG [TezTR-518813_1_17_0_0_0] exec.GroupByOperator: Initialization Done 2 GBY 2018-01-05T11:28:52,591 DEBUG [TezTR-518813_1_17_0_0_0] exec.GroupByOperator: Operator 2 GBY initialized 2018-01-05T11:28:52,591 DEBUG [TezTR-518813_1_17_0_0_0] exec.GroupByOperator: Initializing children of 2 GBY 2018-01-05T11:28:52,591 DEBUG [TezTR-518813_1_17_0_0_0] exec.ReduceSinkOperator: Initializing child 3 RS 2018-01-05T11:28:52,591 INFO [TezTR-518813_1_17_0_0_0] exec.ReduceSinkOperator: Initializing operator RS[3] 2018-01-05T11:28:52,591 DEBUG [TezTR-518813_1_17_0_0_0] exec.ReduceSinkOperator: keys size is 2 2018-01-05T11:28:52,591 DEBUG [TezTR-518813_1_17_0_0_0] exec.ReduceSinkOperator: Key exprNodeDesc _col0 2018-01-05T11:28:52,591 DEBUG [TezTR-518813_1_17_0_0_0] exec.ReduceSinkOperator: Key exprNodeDesc _col1 2018-01-05T11:28:52,591 INFO [TezTR-518813_1_17_0_0_0] exec.ReduceSinkOperator: Using tag = -1 2018-01-05T11:28:52,591 DEBUG [TezTR-518813_1_17_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,592 DEBUG [TezTR-518813_1_17_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,592 DEBUG [TezTR-518813_1_17_0_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:28:52,592 DEBUG [TezTR-518813_1_17_0_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS 2018-01-05T11:28:52,592 DEBUG [TezTR-518813_1_17_0_0_0] exec.ReduceSinkOperator: Operator 3 RS initialized 2018-01-05T11:28:52,592 DEBUG [TezTR-518813_1_17_0_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS done is reset. 2018-01-05T11:28:52,592 DEBUG [TezTR-518813_1_17_0_0_0] exec.GroupByOperator: Initialization Done 2 GBY done is reset. 2018-01-05T11:28:52,592 DEBUG [TezTR-518813_1_17_0_0_0] exec.SelectOperator: Initialization Done 1 SEL done is reset. 2018-01-05T11:28:52,592 DEBUG [TezTR-518813_1_17_0_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:28:52,592 INFO [TezTR-518813_1_17_0_0_0] exec.OperatorUtils: Setting output collector: RS[3] --> Reducer 2 2018-01-05T11:28:52,592 DEBUG [TezTR-518813_1_17_0_0_0] log.PerfLogger: 2018-01-05T11:28:52,592 INFO [TezTR-518813_1_17_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_147_6959340210354157653-1/djaiswal/_tez_scratch_dir/fe6eb59b-a496-4119-8b2e-b1640d9c2cea/map.xml 2018-01-05T11:28:52,592 DEBUG [TezTR-518813_1_17_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:52,592 DEBUG [TezTR-518813_1_17_0_0_0] exec.MapOperator: Processing alias(es) srcpart for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11/kv1.txt 2018-01-05T11:28:52,592 INFO [TezTR-518813_1_17_0_0_0] exec.TableScanOperator: Stats Gathering found a new partition spec = ds=2008-04-08/hr=11 2018-01-05T11:28:52,592 INFO [TezTR-518813_1_17_0_0_0] exec.MapOperator: MAP[0]: records read - 1 2018-01-05T11:28:52,593 INFO [TezTR-518813_1_17_0_0_0] exec.MapOperator: MAP[0]: records read - 10 2018-01-05T11:28:52,593 INFO [TezTR-518813_1_17_0_0_0] exec.MapOperator: MAP[0]: records read - 100 2018-01-05T11:28:52,594 DEBUG [TezTR-518813_1_17_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 1 of 4 2018-01-05T11:28:52,594 DEBUG [TezTR-518813_1_17_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12/kv1.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-08, hr=12}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12, columns=key,value, partition_columns.types=string:string, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180532, rawDataSize=5312, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=12, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-08, hr=11}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11, columns=key,value, partition_columns.types=string:string, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180532, rawDataSize=5312, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=11, vectorPartitionDesc=null], file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-08, hr=12}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12, columns=key,value, partition_columns.types=string:string, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180532, rawDataSize=5312, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=12, vectorPartitionDesc=null], file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-09, hr=11}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11, columns=key,value, partition_columns.types=string:string, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180532, rawDataSize=5312, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=11, vectorPartitionDesc=null], file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-09, hr=12}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12, columns=key,value, partition_columns.types=string:string, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180532, rawDataSize=5312, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=12, vectorPartitionDesc=null]} 2018-01-05T11:28:52,594 DEBUG [TezTR-518813_1_17_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:52,594 INFO [TezTR-518813_1_17_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:52,594 DEBUG [TezTR-518813_1_17_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:52,595 DEBUG [TezTR-518813_1_17_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:52,595 INFO [TezTR-518813_1_17_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_147_6959340210354157653-1/djaiswal/_tez_scratch_dir/fe6eb59b-a496-4119-8b2e-b1640d9c2cea/map.xml 2018-01-05T11:28:52,595 DEBUG [TezTR-518813_1_17_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:52,596 INFO [TezTR-518813_1_17_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_17_00_000000_0 2018-01-05T11:28:52,596 INFO [TezTR-518813_1_17_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [string, string] (isAcidRead false) 2018-01-05T11:28:52,596 WARN [TezTR-518813_1_17_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12/kv1.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:28:52,596 DEBUG [TezTR-518813_1_17_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:28:52,596 WARN [TezTR-518813_1_17_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12/kv1.txt:0+5812 2018-01-05T11:28:52,596 DEBUG [TezTR-518813_1_17_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12/kv1.txt 2018-01-05T11:28:52,596 INFO [TezTR-518813_1_17_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_147_6959340210354157653-1/djaiswal/_tez_scratch_dir/fe6eb59b-a496-4119-8b2e-b1640d9c2cea/map.xml 2018-01-05T11:28:52,596 DEBUG [TezTR-518813_1_17_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:52,596 DEBUG [TezTR-518813_1_17_0_0_0] exec.MapOperator: Processing alias(es) srcpart for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12/kv1.txt 2018-01-05T11:28:52,597 INFO [TezTR-518813_1_17_0_0_0] exec.TableScanOperator: Stats Gathering found a new partition spec = ds=2008-04-08/hr=12 2018-01-05T11:28:52,598 INFO [TezTR-518813_1_17_0_0_0] exec.MapOperator: MAP[0]: records read - 1000 2018-01-05T11:28:52,598 DEBUG [TezTR-518813_1_17_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 2 of 4 2018-01-05T11:28:52,599 DEBUG [TezTR-518813_1_17_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11/kv1.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-09, hr=11}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11, columns=key,value, partition_columns.types=string:string, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180532, rawDataSize=5312, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=11, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-08, hr=11}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11, columns=key,value, partition_columns.types=string:string, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180532, rawDataSize=5312, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=11, vectorPartitionDesc=null], file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-08, hr=12}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12, columns=key,value, partition_columns.types=string:string, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180532, rawDataSize=5312, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=12, vectorPartitionDesc=null], file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-09, hr=11}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11, columns=key,value, partition_columns.types=string:string, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180532, rawDataSize=5312, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=11, vectorPartitionDesc=null], file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-09, hr=12}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12, columns=key,value, partition_columns.types=string:string, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180532, rawDataSize=5312, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=12, vectorPartitionDesc=null]} 2018-01-05T11:28:52,599 DEBUG [TezTR-518813_1_17_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:52,599 INFO [TezTR-518813_1_17_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:52,599 DEBUG [TezTR-518813_1_17_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:52,600 DEBUG [TezTR-518813_1_17_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:52,600 INFO [TezTR-518813_1_17_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_147_6959340210354157653-1/djaiswal/_tez_scratch_dir/fe6eb59b-a496-4119-8b2e-b1640d9c2cea/map.xml 2018-01-05T11:28:52,600 DEBUG [TezTR-518813_1_17_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:52,600 INFO [TezTR-518813_1_17_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_17_00_000000_0 2018-01-05T11:28:52,600 INFO [TezTR-518813_1_17_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [string, string] (isAcidRead false) 2018-01-05T11:28:52,601 WARN [TezTR-518813_1_17_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11/kv1.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:28:52,601 DEBUG [TezTR-518813_1_17_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:28:52,601 WARN [TezTR-518813_1_17_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11/kv1.txt:0+5812 2018-01-05T11:28:52,602 DEBUG [TezTR-518813_1_17_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11/kv1.txt 2018-01-05T11:28:52,602 INFO [TezTR-518813_1_17_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_147_6959340210354157653-1/djaiswal/_tez_scratch_dir/fe6eb59b-a496-4119-8b2e-b1640d9c2cea/map.xml 2018-01-05T11:28:52,602 DEBUG [TezTR-518813_1_17_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:52,602 DEBUG [TezTR-518813_1_17_0_0_0] exec.MapOperator: Processing alias(es) srcpart for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11/kv1.txt 2018-01-05T11:28:52,602 INFO [TezTR-518813_1_17_0_0_0] exec.TableScanOperator: Stats Gathering found a new partition spec = ds=2008-04-09/hr=11 2018-01-05T11:28:52,602 DEBUG [TezTR-518813_1_17_0_0_0] tez.InterruptibleProcessing: Adjusting abort check rows to 100000 from 1000 2018-01-05T11:28:52,603 DEBUG [TezTR-518813_1_17_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 3 of 4 2018-01-05T11:28:52,603 DEBUG [TezTR-518813_1_17_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12/kv1.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-09, hr=12}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12, columns=key,value, partition_columns.types=string:string, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180532, rawDataSize=5312, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=12, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-08, hr=11}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=11, columns=key,value, partition_columns.types=string:string, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180532, rawDataSize=5312, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=11, vectorPartitionDesc=null], file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-08, hr=12}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-08/hr=12, columns=key,value, partition_columns.types=string:string, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180532, rawDataSize=5312, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=12, vectorPartitionDesc=null], file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-09, hr=11}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=11, columns=key,value, partition_columns.types=string:string, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180532, rawDataSize=5312, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=11, vectorPartitionDesc=null], file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcpart, columns.types=string:string, serialization.format=1, serialization.ddl=struct srcpart { string key, string value}, columns=key,value, columns.comments=defaultdefault, partition_columns.types=string:string, partition_columns=ds/hr, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180531}, jobProperties=null], partSpec={ds=2008-04-09, hr=12}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12, columns=key,value, partition_columns.types=string:string, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct srcpart { string key, string value}, transient_lastDdlTime=1515180532, rawDataSize=5312, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds/hr, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcpart}, baseFileName=hr=12, vectorPartitionDesc=null]} 2018-01-05T11:28:52,604 DEBUG [TezTR-518813_1_17_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:52,604 INFO [TezTR-518813_1_17_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:52,604 DEBUG [TezTR-518813_1_17_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:52,605 DEBUG [TezTR-518813_1_17_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:52,605 INFO [TezTR-518813_1_17_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_147_6959340210354157653-1/djaiswal/_tez_scratch_dir/fe6eb59b-a496-4119-8b2e-b1640d9c2cea/map.xml 2018-01-05T11:28:52,605 DEBUG [TezTR-518813_1_17_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:52,605 INFO [TezTR-518813_1_17_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_17_00_000000_0 2018-01-05T11:28:52,605 INFO [TezTR-518813_1_17_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [string, string] (isAcidRead false) 2018-01-05T11:28:52,605 WARN [TezTR-518813_1_17_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12/kv1.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:28:52,605 DEBUG [TezTR-518813_1_17_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:28:52,605 WARN [TezTR-518813_1_17_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12/kv1.txt:0+5812 2018-01-05T11:28:52,605 DEBUG [TezTR-518813_1_17_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12/kv1.txt 2018-01-05T11:28:52,605 INFO [TezTR-518813_1_17_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_147_6959340210354157653-1/djaiswal/_tez_scratch_dir/fe6eb59b-a496-4119-8b2e-b1640d9c2cea/map.xml 2018-01-05T11:28:52,605 DEBUG [TezTR-518813_1_17_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:52,605 DEBUG [TezTR-518813_1_17_0_0_0] exec.MapOperator: Processing alias(es) srcpart for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart/ds=2008-04-09/hr=12/kv1.txt 2018-01-05T11:28:52,606 INFO [TezTR-518813_1_17_0_0_0] exec.TableScanOperator: Stats Gathering found a new partition spec = ds=2008-04-09/hr=12 2018-01-05T11:28:52,607 DEBUG [TezTR-518813_1_17_0_0_0] log.PerfLogger: 2018-01-05T11:28:52,607 DEBUG [TezTR-518813_1_17_0_0_0] mr.ObjectCache: djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:28:52,607 DEBUG [TezTR-518813_1_17_0_0_0] exec.MapOperator: close called for operator MAP[0] 2018-01-05T11:28:52,607 INFO [TezTR-518813_1_17_0_0_0] exec.MapOperator: Closing operator MAP[0] 2018-01-05T11:28:52,607 INFO [TezTR-518813_1_17_0_0_0] exec.MapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:2000, 2018-01-05T11:28:52,607 DEBUG [TezTR-518813_1_17_0_0_0] exec.MapOperator: Closing child = TS[0] 2018-01-05T11:28:52,607 DEBUG [TezTR-518813_1_17_0_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:28:52,607 DEBUG [TezTR-518813_1_17_0_0_0] exec.TableScanOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:52,607 INFO [TezTR-518813_1_17_0_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:28:52,607 DEBUG [TezTR-518813_1_17_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,607 DEBUG [TezTR-518813_1_17_0_0_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_151_4467563815339635813-1/-mr-10001 2018-01-05T11:28:52,607 INFO [TezTR-518813_1_17_0_0_0] exec.TableScanOperator: publishing : default.srcpart/ds=2008-04-09/hr=11/ : {numRows=500, rawDataSize=5312} 2018-01-05T11:28:52,607 INFO [TezTR-518813_1_17_0_0_0] exec.TableScanOperator: publishing : default.srcpart/ds=2008-04-09/hr=12/ : {numRows=500, rawDataSize=5312} 2018-01-05T11:28:52,607 INFO [TezTR-518813_1_17_0_0_0] exec.TableScanOperator: publishing : default.srcpart/ds=2008-04-08/hr=11/ : {numRows=500, rawDataSize=5312} 2018-01-05T11:28:52,607 INFO [TezTR-518813_1_17_0_0_0] exec.TableScanOperator: publishing : default.srcpart/ds=2008-04-08/hr=12/ : {numRows=500, rawDataSize=5312} 2018-01-05T11:28:52,630 DEBUG [TezTR-518813_1_17_0_0_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_151_4467563815339635813-1/-mr-10001/tmpstats-0_0 2018-01-05T11:28:52,630 DEBUG [TezTR-518813_1_17_0_0_0] fs.FSStatsPublisher: Writing stats in it : {default.srcpart/ds=2008-04-08/hr=11/={numRows=500, rawDataSize=5312}, default.srcpart/ds=2008-04-09/hr=11/={numRows=500, rawDataSize=5312}, default.srcpart/ds=2008-04-09/hr=12/={numRows=500, rawDataSize=5312}, default.srcpart/ds=2008-04-08/hr=12/={numRows=500, rawDataSize=5312}} 2018-01-05T11:28:52,630 DEBUG [TezTR-518813_1_17_0_0_0] exec.TableScanOperator: Closing child = SEL[1] 2018-01-05T11:28:52,630 DEBUG [TezTR-518813_1_17_0_0_0] exec.SelectOperator: close called for operator SEL[1] 2018-01-05T11:28:52,630 DEBUG [TezTR-518813_1_17_0_0_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:52,630 INFO [TezTR-518813_1_17_0_0_0] exec.SelectOperator: Closing operator SEL[1] 2018-01-05T11:28:52,630 DEBUG [TezTR-518813_1_17_0_0_0] exec.SelectOperator: Closing child = GBY[2] 2018-01-05T11:28:52,630 DEBUG [TezTR-518813_1_17_0_0_0] exec.GroupByOperator: close called for operator GBY[2] 2018-01-05T11:28:52,630 DEBUG [TezTR-518813_1_17_0_0_0] exec.GroupByOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:52,630 INFO [TezTR-518813_1_17_0_0_0] exec.GroupByOperator: Closing operator GBY[2] 2018-01-05T11:28:52,630 INFO [TezTR-518813_1_17_0_0_0] exec.GroupByOperator: Begin Hash Table flush: size = 4 2018-01-05T11:28:52,631 INFO [TezTR-518813_1_17_0_0_0] exec.ReduceSinkOperator: keys are [_col0, _col1] num distributions: 2 2018-01-05T11:28:52,632 DEBUG [TezTR-518813_1_17_0_0_0] exec.GroupByOperator: Closing child = RS[3] 2018-01-05T11:28:52,632 DEBUG [TezTR-518813_1_17_0_0_0] exec.ReduceSinkOperator: close called for operator RS[3] 2018-01-05T11:28:52,632 DEBUG [TezTR-518813_1_17_0_0_0] exec.ReduceSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:52,632 INFO [TezTR-518813_1_17_0_0_0] exec.ReduceSinkOperator: Closing operator RS[3] 2018-01-05T11:28:52,632 INFO [TezTR-518813_1_17_0_0_0] exec.ReduceSinkOperator: RECORDS_OUT_INTERMEDIATE_Map_1:4, 2018-01-05T11:28:52,632 DEBUG [TezTR-518813_1_17_0_0_0] exec.ReduceSinkOperator: 3 Close done 2018-01-05T11:28:52,632 DEBUG [TezTR-518813_1_17_0_0_0] exec.GroupByOperator: 2 Close done 2018-01-05T11:28:52,632 DEBUG [TezTR-518813_1_17_0_0_0] exec.SelectOperator: 1 Close done 2018-01-05T11:28:52,632 DEBUG [TezTR-518813_1_17_0_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:28:52,632 DEBUG [TezTR-518813_1_17_0_0_0] exec.MapOperator: 0 Close done 2018-01-05T11:28:52,632 INFO [TezTR-518813_1_17_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_17_00_000000_0 2018-01-05T11:28:52,632 INFO [TezTR-518813_1_17_0_0_0] impl.PipelinedSorter: Reducer 2: Starting flush of map output 2018-01-05T11:28:52,632 INFO [TezTR-518813_1_17_0_0_0] impl.PipelinedSorter: Reducer 2: Span0.length = 4, perItem = 2390 2018-01-05T11:28:52,633 INFO [TezTR-518813_1_17_0_0_0] impl.PipelinedSorter: Reducer 2: done sorting span=0, length=4, time=1 2018-01-05T11:28:52,636 INFO [TezTR-518813_1_17_0_0_0] impl.PipelinedSorter: Reducer 2: Heap = SpanIterator<0:3> (span=Span[5242880,9560]), 2018-01-05T11:28:52,648 INFO [TezTR-518813_1_17_0_0_0] impl.PipelinedSorter: Reducer 2: Spilling to /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/17/output/attempt_1515180518813_0001_17_00_000000_0_10093_0/file.out 2018-01-05T11:28:52,660 INFO [TezTR-518813_1_17_0_0_0] shuffle.ShuffleUtils: EmptyPartition bitsetSize=2, numOutputs=4, emptyPartitions=2, compressedSize=3 2018-01-05T11:28:52,661 DEBUG [TezTR-518813_1_17_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:52,661 INFO [TezTR-518813_1_17_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:52,661 INFO [TezTR-518813_1_17_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_17_00_000000_0, askedToStop=false 2018-01-05T11:28:52,661 INFO [TezTR-518813_1_17_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_17_00_000000_0, stopRequested=false 2018-01-05T11:28:52,661 INFO [TezTR-518813_1_17_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_17_00_000000_0: Counters: 15 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=4, INPUT_RECORDS_PROCESSED=2000, INPUT_SPLIT_LENGTH_BYTES=23248, OUTPUT_RECORDS=4, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=9560, OUTPUT_BYTES_WITH_OVERHEAD=9588, OUTPUT_BYTES_PHYSICAL=9596, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=2000, RECORDS_OUT_INTERMEDIATE_Map_1=4]] 2018-01-05T11:28:52,661 DEBUG [TezTR-518813_1_17_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_17_0_0_0 thread id: 1138 scheme: file bytesRead: 23696 bytesWritten: 9977 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:52,661 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_17_00_000000_0, isTaskDone=true 2018-01-05T11:28:52,661 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000031, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_17_00_000000_0, eventCount=4 } 2018-01-05T11:28:52,662 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_17][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_17_00_000000_0, creationTime=1515180532544, allocationTime=1515180532545, startTime=1515180532547, finishTime=1515180532662, timeTaken=115, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:52,662 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:52,662 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_17_00_000000_0 2018-01-05T11:28:52,662 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_17_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:52,662 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_17_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:52,662 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_17_00_000000_0; false 2018-01-05T11:28:52,662 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_17_00_000000_0 2018-01-05T11:28:52,662 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_17_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:52,662 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:52,662 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_17][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_17_00_000000, startTime=1515180532547, finishTime=1515180532662, timeTaken=115, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_17_00_000000_0, diagnostics=, counters=Counters: 21, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=23696, FILE_BYTES_WRITTEN=9977, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=4, INPUT_RECORDS_PROCESSED=2000, INPUT_SPLIT_LENGTH_BYTES=23248, OUTPUT_RECORDS=4, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=9560, OUTPUT_BYTES_WITH_OVERHEAD=9588, OUTPUT_BYTES_PHYSICAL=9596, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=2000, RECORDS_OUT_INTERMEDIATE_Map_1=4 2018-01-05T11:28:52,662 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:52,662 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000031=102 2018-01-05T11:28:52,663 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_17_00_000000_0: true 2018-01-05T11:28:52,663 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_17_00_000000_0 2018-01-05T11:28:52,663 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_17_00 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:52,663 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_17_00 [Map 1] 2018-01-05T11:28:52,663 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000031, QueryId=djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab-17, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:28:52,663 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_17_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:52,663 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_17_00_000000_0 2018-01-05T11:28:52,663 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_17][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_17_00, initRequestedTime=1515180532474, initedTime=1515180532543, startRequestedTime=1515180532482, startedTime=1515180532543, finishTime=1515180532663, timeTaken=120, status=SUCCEEDED, diagnostics=, counters=Counters: 21, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=23696, FILE_BYTES_WRITTEN=9977, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=4, INPUT_RECORDS_PROCESSED=2000, INPUT_SPLIT_LENGTH_BYTES=23248, OUTPUT_RECORDS=4, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=9560, OUTPUT_BYTES_WITH_OVERHEAD=9588, OUTPUT_BYTES_PHYSICAL=9596, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=2000, RECORDS_OUT_INTERMEDIATE_Map_1=4, vertexStats=firstTaskStartTime=1515180532547, firstTasksToStart=[ task_1515180518813_0001_17_00_000000 ], lastTaskFinishTime=1515180532662, lastTasksToFinish=[ task_1515180518813_0001_17_00_000000 ], minTaskDuration=115, maxTaskDuration=115, avgTaskDuration=115.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_17_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_17_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:52,663 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_17_00 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:52,663 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_17][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000031, stoppedTime=1515180532663, exitStatus=0 2018-01-05T11:28:52,663 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000031 exited with diagnostics set to Container completed. 2018-01-05T11:28:52,663 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_17_01 [Reducer 2] attempt: attempt_1515180518813_0001_17_00_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:28:52,663 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_17_00 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2 2018-01-05T11:28:52,663 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null 2018-01-05T11:28:52,663 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000031 2018-01-05T11:28:52,663 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: All source tasks completed. Ramping up 4 remaining tasks for vertex: Reducer 2 2018-01-05T11:28:52,663 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Scheduling 4 tasks for vertex: Reducer 2 with totalTasks: 4. 1 source tasks completed out of 1. MinSourceTaskCompletedFraction: 1.0 min: 0.05 max: 0.75 2018-01-05T11:28:52,664 INFO [App Shared Pool - #0] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Map 1, destinationVertexName=Reducer 2. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:28:52,664 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_17_01 [Reducer 2] 2018-01-05T11:28:52,664 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_17][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_17_01_000001, scheduledTime=1515180532664, launchTime=1515180532664 2018-01-05T11:28:52,664 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_17][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_17_01_000003, scheduledTime=1515180532664, launchTime=1515180532664 2018-01-05T11:28:52,664 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_17][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_17_01_000000, scheduledTime=1515180532664, launchTime=1515180532664 2018-01-05T11:28:52,664 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_17][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_17_01_000002, scheduledTime=1515180532664, launchTime=1515180532664 2018-01-05T11:28:52,664 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_17_01 [Reducer 2] 2018-01-05T11:28:52,664 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_17_01_000001_0, priority=14, capability=, hosts=null 2018-01-05T11:28:52,664 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:52,664 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_17_01_000003_0, priority=14, capability=, hosts=null 2018-01-05T11:28:52,664 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=14,c=2]. totalPendingTasks=2. delayedTaskQueueSize=0 2018-01-05T11:28:52,664 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_17_01_000000_0, priority=14, capability=, hosts=null 2018-01-05T11:28:52,664 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=14,c=2]. totalPendingTasks=2. delayedTaskQueueSize=0 2018-01-05T11:28:52,664 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:52,664 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_17_01_000001_0 on hosts=null 2018-01-05T11:28:52,664 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_17_01_000001_0 on hosts=null 2018-01-05T11:28:52,664 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_17_01_000001_0 2018-01-05T11:28:52,664 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:52,664 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #32, task=TaskInfo{task=attempt_1515180518813_0001_17_01_000001_0, priority=14, startTime=0, containerId=null, uniqueId=31, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000032 2018-01-05T11:28:52,664 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_17_01_000001_0; false 2018-01-05T11:28:52,664 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_17_01_000001_0, priority=14, startTime=311553173, containerId=container_111101111_0001_00_000032assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=31, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:52,664 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_17_01_000003_0 on hosts=null 2018-01-05T11:28:52,664 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_17_01_000003_0 on hosts=null 2018-01-05T11:28:52,664 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_17_01_000003_0 2018-01-05T11:28:52,664 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:52,664 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #33, task=TaskInfo{task=attempt_1515180518813_0001_17_01_000003_0, priority=14, startTime=0, containerId=null, uniqueId=32, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, to container=container_111101111_0001_00_000033 2018-01-05T11:28:52,664 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_17_01_000003_0; false 2018-01-05T11:28:52,664 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_17_01_000003_0, priority=14, startTime=311553173, containerId=container_111101111_0001_00_000033assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=2, ac=6, commF=false, disabled=false}, uniqueId=32, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:52,665 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:52,665 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:52,665 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:52,665 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:52,665 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:52,665 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_17_01_000002_0, priority=14, capability=, hosts=null 2018-01-05T11:28:52,665 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_17_01_000000_0 on hosts=null 2018-01-05T11:28:52,665 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_17_01_000000_0 on hosts=null 2018-01-05T11:28:52,665 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_17_01_000000_0 2018-01-05T11:28:52,665 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=2, ac=6, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:52,665 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #34, task=TaskInfo{task=attempt_1515180518813_0001_17_01_000000_0, priority=14, startTime=0, containerId=null, uniqueId=33, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=2, ac=6, commF=false, disabled=false}, to container=container_111101111_0001_00_000034 2018-01-05T11:28:52,665 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_17_01_000000_0; false 2018-01-05T11:28:52,665 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_17_01_000000_0, priority=14, startTime=311553174, containerId=container_111101111_0001_00_000034assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=3, ac=5, commF=false, disabled=false}, uniqueId=33, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:52,665 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:52,665 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:52,665 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:52,665 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_17_01_000002_0 on hosts=null 2018-01-05T11:28:52,665 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_17_01_000002_0 on hosts=null 2018-01-05T11:28:52,665 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_17_01_000002_0 2018-01-05T11:28:52,665 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=3, ac=5, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:52,665 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #35, task=TaskInfo{task=attempt_1515180518813_0001_17_01_000002_0, priority=14, startTime=0, containerId=null, uniqueId=34, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=3, ac=5, commF=false, disabled=false}, to container=container_111101111_0001_00_000035 2018-01-05T11:28:52,665 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_17_01_000002_0; false 2018-01-05T11:28:52,665 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_17_01_000002_0, priority=14, startTime=311553174, containerId=container_111101111_0001_00_000035assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=4, ac=4, commF=false, disabled=false}, uniqueId=34, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:52,665 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000032 for node: 10.22.8.182:49818 2018-01-05T11:28:52,665 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000032, attempt_1515180518813_0001_17_01_000001_0 for node: 10.22.8.182:49818 2018-01-05T11:28:52,665 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000032 for node: 10.22.8.182:49818 2018-01-05T11:28:52,666 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=17}, sourceName=Map 1, state=S_SUCCEEDED 2018-01-05T11:28:52,666 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000033 for node: 10.22.8.182:49818 2018-01-05T11:28:52,666 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000033, attempt_1515180518813_0001_17_01_000003_0 for node: 10.22.8.182:49818 2018-01-05T11:28:52,666 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000033 for node: 10.22.8.182:49818 2018-01-05T11:28:52,666 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000034 for node: 10.22.8.182:49818 2018-01-05T11:28:52,666 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000034, attempt_1515180518813_0001_17_01_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:52,666 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000034 for node: 10.22.8.182:49818 2018-01-05T11:28:52,667 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000035 for node: 10.22.8.182:49818 2018-01-05T11:28:52,667 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_17_01_000001_0, am_details=10.22.8.182:49824, taskInfo= fragment 1 attempt 0, user=djaiswal, queryId=djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000032, dagName=ANALYZE TABLE srcpart PARTITION(...key,value (Stage-0), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.OrderedGroupedKVInput,1}}, Outputs={{out_Reducer 2,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=5, completedTaskCount=1, dagStartTime=1515180532474, firstAttemptStartTime=1515180532664, currentAttemptStartTime=1515180532665} 2018-01-05T11:28:52,667 INFO [IPC Server handler 1 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000032, QueryId=djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab-17, DagId=17, VertexName=Reducer 2, TaskId=1, TaskAttemptId=0, SubmitTime=1515180532667 2018-01-05T11:28:52,667 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000035, attempt_1515180518813_0001_17_01_000002_0 for node: 10.22.8.182:49818 2018-01-05T11:28:52,667 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000035 for node: 10.22.8.182:49818 2018-01-05T11:28:52,667 DEBUG [IPC Server handler 1 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=17} with the ShuffleHandler 2018-01-05T11:28:52,667 DEBUG [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/17] 2018-01-05T11:28:52,668 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000032 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:52,668 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000032, launchTime=1515180532668 2018-01-05T11:28:52,668 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:52,668 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_17_01_000001_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:52,668 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_17_01_000001_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180532664, dagStartTime=1515180532474, withinDagPriority=14, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1}] 2018-01-05T11:28:52,668 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_17_01_000001_0 : ACCEPTED 2018-01-05T11:28:52,668 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_17_01_000001_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:52,668 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_17_01_000001_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180532664, dagStartTime=1515180532474, withinDagPriority=14, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1} 2018-01-05T11:28:52,668 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_17_01_000001_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:52,668 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_17_01_000001_0 to pre-emption queue 2018-01-05T11:28:52,668 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_17_01_000001_0: true 2018-01-05T11:28:52,668 INFO [TaskCommunicator # 8] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_17_01_000001_0 2018-01-05T11:28:52,668 INFO [TaskCommunicator # 8] GuaranteedTasks: Received response for attempt_1515180518813_0001_17_01_000001_0, true 2018-01-05T11:28:52,668 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:313075933 2018-01-05T11:28:52,668 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_17_01_000001_0] submitted. Is using containerId: [container_111101111_0001_00_000032] on NM: [10.22.8.182:49818] 2018-01-05T11:28:52,668 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:52,668 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:52,668 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_17][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_17_01_000001_0, startTime=1515180532668, containerId=container_111101111_0001_00_000032, nodeId=10.22.8.182:49818 2018-01-05T11:28:52,669 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000033 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:52,669 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000033, launchTime=1515180532669 2018-01-05T11:28:52,669 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : ANALYZE TABLE srcpart PARTITION(...key,value (Stage-0), VertexName: Reducer 2, VertexParallelism: 4, TaskAttemptID:attempt_1515180518813_0001_17_01_000001_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:28:52,669 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1043333120, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:52,669 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_17_01_000001_0 vertexName: Reducer 2 2018-01-05T11:28:52,669 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000032, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_17_01_000001_0, eventCount=1 } 2018-01-05T11:28:52,669 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_17_01_000003_0, am_details=10.22.8.182:49824, taskInfo= fragment 3 attempt 0, user=djaiswal, queryId=djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000033, dagName=ANALYZE TABLE srcpart PARTITION(...key,value (Stage-0), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.OrderedGroupedKVInput,1}}, Outputs={{out_Reducer 2,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=5, completedTaskCount=1, dagStartTime=1515180532474, firstAttemptStartTime=1515180532664, currentAttemptStartTime=1515180532666} 2018-01-05T11:28:52,669 INFO [IPC Server handler 0 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000033, QueryId=djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab-17, DagId=17, VertexName=Reducer 2, TaskId=3, TaskAttemptId=0, SubmitTime=1515180532669 2018-01-05T11:28:52,669 DEBUG [TezTR-518813_1_17_1_1_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_17 queryId: djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab instanceType: TaskRunner2Callable 2018-01-05T11:28:52,669 INFO [TezTR-518813_1_17_1_1_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_17_01_000001_0 2018-01-05T11:28:52,669 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_17_01_000003_0] submitted. Is using containerId: [container_111101111_0001_00_000033] on NM: [10.22.8.182:49818] 2018-01-05T11:28:52,669 DEBUG [IPC Server handler 0 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=17} with the ShuffleHandler 2018-01-05T11:28:52,669 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:52,669 DEBUG [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/17] 2018-01-05T11:28:52,669 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:52,669 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_17][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_17_01_000003_0, startTime=1515180532669, containerId=container_111101111_0001_00_000033, nodeId=10.22.8.182:49818 2018-01-05T11:28:52,669 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000034 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:52,669 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000034, launchTime=1515180532669 2018-01-05T11:28:52,669 DEBUG [TezTR-518813_1_17_1_1_0] log.PerfLogger: 2018-01-05T11:28:52,669 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=3, runningFragmentCount=1 2018-01-05T11:28:52,670 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_17_01_000003_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:52,670 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_17_01_000003_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180532664, dagStartTime=1515180532474, withinDagPriority=14, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1}] 2018-01-05T11:28:52,670 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_17_01_000003_0 : ACCEPTED 2018-01-05T11:28:52,670 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_17_01_000003_0, canFinish=true. Current state: preemptionQueueSize=1, numSlotsAvailable=3, waitQueueSize=1 2018-01-05T11:28:52,670 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_17_01_000003_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180532664, dagStartTime=1515180532474, withinDagPriority=14, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1} 2018-01-05T11:28:52,670 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_17_01_000003_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:52,670 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_17_01_000003_0 to pre-emption queue 2018-01-05T11:28:52,670 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:52,670 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_17_01_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000034] on NM: [10.22.8.182:49818] 2018-01-05T11:28:52,670 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:52,670 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:52,670 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_17_01_000003_0: true 2018-01-05T11:28:52,670 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_17][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_17_01_000000_0, startTime=1515180532669, containerId=container_111101111_0001_00_000034, nodeId=10.22.8.182:49818 2018-01-05T11:28:52,670 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000035 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:52,670 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: taskOwner hashCode:1890521494 2018-01-05T11:28:52,670 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000035, launchTime=1515180532670 2018-01-05T11:28:52,670 INFO [TaskCommunicator # 9] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_17_01_000003_0 2018-01-05T11:28:52,670 INFO [TaskCommunicator # 9] GuaranteedTasks: Received response for attempt_1515180518813_0001_17_01_000003_0, true 2018-01-05T11:28:52,671 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_17_01_000002_0] submitted. Is using containerId: [container_111101111_0001_00_000035] on NM: [10.22.8.182:49818] 2018-01-05T11:28:52,671 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:52,671 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:52,671 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_17][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_17_01_000002_0, startTime=1515180532670, containerId=container_111101111_0001_00_000035, nodeId=10.22.8.182:49818 2018-01-05T11:28:52,671 INFO [Task-Executor-3] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : ANALYZE TABLE srcpart PARTITION(...key,value (Stage-0), VertexName: Reducer 2, VertexParallelism: 4, TaskAttemptID:attempt_1515180518813_0001_17_01_000003_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:28:52,671 INFO [Task-Executor-3] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1043333120, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:52,671 INFO [Task-Executor-3] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_17_01_000003_0 vertexName: Reducer 2 2018-01-05T11:28:52,671 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_17_01_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000034, dagName=ANALYZE TABLE srcpart PARTITION(...key,value (Stage-0), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.OrderedGroupedKVInput,1}}, Outputs={{out_Reducer 2,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=5, completedTaskCount=1, dagStartTime=1515180532474, firstAttemptStartTime=1515180532664, currentAttemptStartTime=1515180532667} 2018-01-05T11:28:52,672 INFO [IPC Server handler 2 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000034, QueryId=djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab-17, DagId=17, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, SubmitTime=1515180532672 2018-01-05T11:28:52,672 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000033, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_17_01_000003_0, eventCount=1 } 2018-01-05T11:28:52,672 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:52,672 DEBUG [IPC Server handler 2 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=17} with the ShuffleHandler 2018-01-05T11:28:52,672 DEBUG [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/17] 2018-01-05T11:28:52,672 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=2, runningFragmentCount=2 2018-01-05T11:28:52,672 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_17_01_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:52,672 DEBUG [TezTR-518813_1_17_1_3_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_17 queryId: djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab instanceType: TaskRunner2Callable 2018-01-05T11:28:52,672 INFO [TezTR-518813_1_17_1_3_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_17_01_000003_0 2018-01-05T11:28:52,673 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_17_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180532664, dagStartTime=1515180532474, withinDagPriority=14, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1}] 2018-01-05T11:28:52,673 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_17_01_000000_0 : ACCEPTED 2018-01-05T11:28:52,673 INFO [IPC Server handler 10 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_17_01_000001_0 1 events [0,1) total 1 vertex_1515180518813_0001_17_01 [Reducer 2] 2018-01-05T11:28:52,674 INFO [TaskCommunicator # 0] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_17_01_000000_0 2018-01-05T11:28:52,674 INFO [TaskCommunicator # 0] GuaranteedTasks: Received response for attempt_1515180518813_0001_17_01_000000_0, true 2018-01-05T11:28:52,674 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:52,674 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_17_01_000000_0, canFinish=true. Current state: preemptionQueueSize=2, numSlotsAvailable=2, waitQueueSize=1 2018-01-05T11:28:52,674 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_17_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180532664, dagStartTime=1515180532474, withinDagPriority=14, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1} 2018-01-05T11:28:52,674 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:28:52,674 DEBUG [TezTR-518813_1_17_1_1_0] log.PerfLogger: 2018-01-05T11:28:52,674 INFO [TezTR-518813_1_17_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:52,674 INFO [TezTR-518813_1_17_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:52,674 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_17_01_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:52,674 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_17_01_000000_0: true 2018-01-05T11:28:52,674 DEBUG [TezTR-518813_1_17_1_3_0] log.PerfLogger: 2018-01-05T11:28:52,674 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: taskOwner hashCode:1566245934 2018-01-05T11:28:52,674 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_17_01_000001_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:28:52,674 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_17_01_000000_0 to pre-emption queue 2018-01-05T11:28:52,675 INFO [I/O Setup 1 Initialize: {out_Reducer 2}] output.MROutput: out_Reducer 2: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:28:52,675 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:52,675 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:52,675 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_17_01_000002_0, am_details=10.22.8.182:49824, taskInfo= fragment 2 attempt 0, user=djaiswal, queryId=djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000035, dagName=ANALYZE TABLE srcpart PARTITION(...key,value (Stage-0), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.OrderedGroupedKVInput,1}}, Outputs={{out_Reducer 2,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=5, completedTaskCount=1, dagStartTime=1515180532474, firstAttemptStartTime=1515180532664, currentAttemptStartTime=1515180532667} 2018-01-05T11:28:52,675 INFO [IPC Server handler 4 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000035, QueryId=djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab-17, DagId=17, VertexName=Reducer 2, TaskId=2, TaskAttemptId=0, SubmitTime=1515180532675 2018-01-05T11:28:52,675 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:52,675 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:52,675 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:52,675 INFO [Task-Executor-1] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : ANALYZE TABLE srcpart PARTITION(...key,value (Stage-0), VertexName: Reducer 2, VertexParallelism: 4, TaskAttemptID:attempt_1515180518813_0001_17_01_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:28:52,675 INFO [Task-Executor-1] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1043333120, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:52,675 DEBUG [IPC Server handler 4 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=17} with the ShuffleHandler 2018-01-05T11:28:52,675 INFO [TezTR-518813_1_17_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:52,675 DEBUG [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/17] 2018-01-05T11:28:52,675 INFO [Task-Executor-1] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_17_01_000000_0 vertexName: Reducer 2 2018-01-05T11:28:52,676 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=1, runningFragmentCount=3 2018-01-05T11:28:52,676 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_17_01_000002_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:52,676 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000034, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_17_01_000000_0, eventCount=1 } 2018-01-05T11:28:52,676 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_17_01_000002_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180532664, dagStartTime=1515180532474, withinDagPriority=14, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1}] 2018-01-05T11:28:52,676 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_17_01_000002_0 : ACCEPTED 2018-01-05T11:28:52,676 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_17_01_000002_0, canFinish=true. Current state: preemptionQueueSize=3, numSlotsAvailable=1, waitQueueSize=1 2018-01-05T11:28:52,676 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_17_01_000002_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180532664, dagStartTime=1515180532474, withinDagPriority=14, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1} 2018-01-05T11:28:52,677 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_17_01_000002_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:52,677 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_17_01_000002_0 to pre-emption queue 2018-01-05T11:28:52,677 INFO [IPC Server handler 6 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_17_01_000003_0 1 events [0,1) total 1 vertex_1515180518813_0001_17_01 [Reducer 2] 2018-01-05T11:28:52,677 DEBUG [TezTR-518813_1_17_1_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_17 queryId: djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab instanceType: TaskRunner2Callable 2018-01-05T11:28:52,677 INFO [TezTR-518813_1_17_1_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_17_01_000000_0 2018-01-05T11:28:52,677 INFO [TaskCommunicator # 1] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_17_01_000002_0 2018-01-05T11:28:52,677 INFO [TaskCommunicator # 1] GuaranteedTasks: Received response for attempt_1515180518813_0001_17_01_000002_0, true 2018-01-05T11:28:52,677 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_17_01_000002_0: true 2018-01-05T11:28:52,677 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: taskOwner hashCode:1126359976 2018-01-05T11:28:52,677 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:28:52,677 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_17_01_000003_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:28:52,677 DEBUG [TezTR-518813_1_17_1_0_0] log.PerfLogger: 2018-01-05T11:28:52,677 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:52,678 INFO [Task-Executor-2] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : ANALYZE TABLE srcpart PARTITION(...key,value (Stage-0), VertexName: Reducer 2, VertexParallelism: 4, TaskAttemptID:attempt_1515180518813_0001_17_01_000002_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:28:52,678 INFO [Task-Executor-2] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1043333120, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:52,678 INFO [Task-Executor-2] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_17_01_000002_0 vertexName: Reducer 2 2018-01-05T11:28:52,678 DEBUG [TezTR-518813_1_17_1_3_0] log.PerfLogger: 2018-01-05T11:28:52,678 INFO [TezTR-518813_1_17_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:52,678 INFO [TezTR-518813_1_17_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:52,679 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000035, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_17_01_000002_0, eventCount=1 } 2018-01-05T11:28:52,679 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:52,679 DEBUG [TezTR-518813_1_17_1_2_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_17 queryId: djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab instanceType: TaskRunner2Callable 2018-01-05T11:28:52,679 INFO [I/O Setup 1 Initialize: {out_Reducer 2}] output.MROutput: out_Reducer 2: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:28:52,679 INFO [TezTR-518813_1_17_1_2_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_17_01_000002_0 2018-01-05T11:28:52,679 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:52,679 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:52,679 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:52,679 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:52,680 DEBUG [TezTR-518813_1_17_1_2_0] log.PerfLogger: 2018-01-05T11:28:52,681 DEBUG [TezTR-518813_1_17_1_0_0] log.PerfLogger: 2018-01-05T11:28:52,681 INFO [TezTR-518813_1_17_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:52,681 INFO [TezTR-518813_1_17_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:52,681 INFO [TezTR-518813_1_17_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:52,682 INFO [IPC Server handler 27 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_17_01_000000_0 1 events [0,1) total 1 vertex_1515180518813_0001_17_01 [Reducer 2] 2018-01-05T11:28:52,683 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:52,683 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:28:52,683 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_17_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:28:52,683 INFO [I/O Setup 1 Initialize: {out_Reducer 2}] output.MROutput: out_Reducer 2: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:28:52,683 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:52,683 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:52,683 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:52,683 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:52,683 INFO [TezTR-518813_1_17_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:52,684 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:52,685 DEBUG [TezTR-518813_1_17_1_2_0] log.PerfLogger: 2018-01-05T11:28:52,685 INFO [TezTR-518813_1_17_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:52,685 INFO [TezTR-518813_1_17_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:52,685 INFO [IPC Server handler 26 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_17_01_000002_0 1 events [0,1) total 1 vertex_1515180518813_0001_17_01 [Reducer 2] 2018-01-05T11:28:52,686 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:28:52,686 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_17_01_000002_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:28:52,686 INFO [I/O Setup 1 Initialize: {out_Reducer 2}] output.MROutput: out_Reducer 2: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:28:52,686 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:52,686 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:52,686 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:52,686 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:52,686 INFO [TezTR-518813_1_17_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:52,686 INFO [I/O Setup 0 Initialize: {Map 1}] orderedgrouped.MergeManager: Initial Memory required for SHUFFLE_BUFFER=131404920 based on INPUT_BUFFER_FACTOR=0.7, for final merged output=0, using factor: 0.0 2018-01-05T11:28:52,686 INFO [I/O Setup 0 Initialize: {Map 1}] orderedgrouped.MergeManager: Initial Memory required for SHUFFLE_BUFFER=131404920 based on INPUT_BUFFER_FACTOR=0.7, for final merged output=0, using factor: 0.0 2018-01-05T11:28:52,686 INFO [I/O Setup 0 Initialize: {Map 1}] orderedgrouped.MergeManager: Initial Memory required for SHUFFLE_BUFFER=131404920 based on INPUT_BUFFER_FACTOR=0.7, for final merged output=0, using factor: 0.0 2018-01-05T11:28:52,686 INFO [I/O Setup 0 Initialize: {Map 1}] orderedgrouped.MergeManager: Initial Memory required for SHUFFLE_BUFFER=131404920 based on INPUT_BUFFER_FACTOR=0.7, for final merged output=0, using factor: 0.0 2018-01-05T11:28:52,687 INFO [TezTR-518813_1_17_1_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:52,687 INFO [TezTR-518813_1_17_1_3_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:52,687 INFO [TezTR-518813_1_17_1_1_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:52,687 INFO [TezTR-518813_1_17_1_0_0] resources.MemoryDistributor: InitialRequests=[out_Reducer 2:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.OrderedGroupedKVInput] 2018-01-05T11:28:52,687 INFO [TezTR-518813_1_17_1_3_0] resources.MemoryDistributor: InitialRequests=[out_Reducer 2:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.OrderedGroupedKVInput] 2018-01-05T11:28:52,687 INFO [TezTR-518813_1_17_1_2_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:52,687 INFO [TezTR-518813_1_17_1_1_0] resources.MemoryDistributor: InitialRequests=[out_Reducer 2:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.OrderedGroupedKVInput] 2018-01-05T11:28:52,687 INFO [TezTR-518813_1_17_1_2_0] resources.MemoryDistributor: InitialRequests=[out_Reducer 2:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.OrderedGroupedKVInput] 2018-01-05T11:28:52,687 INFO [TezTR-518813_1_17_1_3_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:52,687 INFO [TezTR-518813_1_17_1_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:52,687 INFO [TezTR-518813_1_17_1_1_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:52,687 INFO [TezTR-518813_1_17_1_2_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:52,687 INFO [TezTR-518813_1_17_1_3_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:52,687 INFO [TezTR-518813_1_17_1_2_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:52,687 INFO [TezTR-518813_1_17_1_1_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:52,687 INFO [TezTR-518813_1_17_1_3_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 131404920, TotalRequestedScaled: 1.2129684923076923E8, TotalJVMHeap: 1043333120, TotalAvailable: 699033190, TotalRequested/TotalJVMHeap:0.13 2018-01-05T11:28:52,687 INFO [TezTR-518813_1_17_1_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:52,687 INFO [TezTR-518813_1_17_1_1_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 131404920, TotalRequestedScaled: 1.2129684923076923E8, TotalJVMHeap: 1043333120, TotalAvailable: 699033190, TotalRequested/TotalJVMHeap:0.13 2018-01-05T11:28:52,687 INFO [TezTR-518813_1_17_1_2_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 131404920, TotalRequestedScaled: 1.2129684923076923E8, TotalJVMHeap: 1043333120, TotalAvailable: 699033190, TotalRequested/TotalJVMHeap:0.13 2018-01-05T11:28:52,687 INFO [TezTR-518813_1_17_1_3_0] resources.MemoryDistributor: Allocations=[out_Reducer 2:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [Map 1:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:131404920:131404920] 2018-01-05T11:28:52,687 INFO [TezTR-518813_1_17_1_1_0] resources.MemoryDistributor: Allocations=[out_Reducer 2:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [Map 1:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:131404920:131404920] 2018-01-05T11:28:52,687 INFO [TezTR-518813_1_17_1_2_0] resources.MemoryDistributor: Allocations=[out_Reducer 2:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [Map 1:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:131404920:131404920] 2018-01-05T11:28:52,687 INFO [TezTR-518813_1_17_1_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 131404920, TotalRequestedScaled: 1.2129684923076923E8, TotalJVMHeap: 1043333120, TotalAvailable: 699033190, TotalRequested/TotalJVMHeap:0.13 2018-01-05T11:28:52,687 INFO [TezTR-518813_1_17_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:52,687 INFO [TezTR-518813_1_17_1_0_0] resources.MemoryDistributor: Allocations=[out_Reducer 2:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [Map 1:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:131404920:131404920] 2018-01-05T11:28:52,687 INFO [TezTR-518813_1_17_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:52,687 INFO [TezTR-518813_1_17_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:52,687 INFO [TezTR-518813_1_17_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:52,687 INFO [TezTR-518813_1_17_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Input: Map 1 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:28:52,687 INFO [TezTR-518813_1_17_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Input: Map 1 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:28:52,687 INFO [TezTR-518813_1_17_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Input: Map 1 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:28:52,687 INFO [TezTR-518813_1_17_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: Map 1 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:28:52,687 INFO [TezTR-518813_1_17_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:28:52,689 INFO [TezTR-518813_1_17_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:28:52,688 INFO [TezTR-518813_1_17_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:28:52,689 INFO [TezTR-518813_1_17_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:28:52,688 INFO [TezTR-518813_1_17_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:28:52,689 INFO [TezTR-518813_1_17_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:28:52,689 INFO [TezTR-518813_1_17_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:28:52,689 INFO [TezTR-518813_1_17_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:28:52,689 INFO [I/O Setup 1 Start: {Map 1}] orderedgrouped.Shuffle: Map_1: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true 2018-01-05T11:28:52,689 INFO [I/O Setup 1 Start: {Map 1}] orderedgrouped.Shuffle: Map_1: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true 2018-01-05T11:28:52,689 INFO [I/O Setup 1 Start: {Map 1}] orderedgrouped.Shuffle: Map_1: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true 2018-01-05T11:28:52,689 INFO [I/O Setup 1 Start: {Map 1}] orderedgrouped.Shuffle: Map_1: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true 2018-01-05T11:28:52,693 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,693 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: Status: Running (Executing on YARN cluster with App id application_1515180518813_0001) 2018-01-05T11:28:52,694 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,694 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,694 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,694 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 Reducer 2: 0(+4)/4 2018-01-05T11:28:52,695 INFO [I/O Setup 1 Start: {Map 1}] orderedgrouped.MergeManager: Map 1: MergerManager: memoryLimit=131404920, maxSingleShuffleLimit=32851230, mergeThreshold=86727248, ioSortFactor=10, postMergeMem=0, memToMemMergeOutputsThreshold=10 2018-01-05T11:28:52,695 INFO [I/O Setup 1 Start: {Map 1}] orderedgrouped.MergeManager: Map 1: MergerManager: memoryLimit=131404920, maxSingleShuffleLimit=32851230, mergeThreshold=86727248, ioSortFactor=10, postMergeMem=0, memToMemMergeOutputsThreshold=10 2018-01-05T11:28:52,695 INFO [I/O Setup 1 Start: {Map 1}] orderedgrouped.MergeManager: Map 1: MergerManager: memoryLimit=131404920, maxSingleShuffleLimit=32851230, mergeThreshold=86727248, ioSortFactor=10, postMergeMem=0, memToMemMergeOutputsThreshold=10 2018-01-05T11:28:52,695 INFO [I/O Setup 1 Start: {Map 1}] orderedgrouped.MergeManager: Map 1: MergerManager: memoryLimit=131404920, maxSingleShuffleLimit=32851230, mergeThreshold=86727248, ioSortFactor=10, postMergeMem=0, memToMemMergeOutputsThreshold=10 2018-01-05T11:28:52,707 INFO [I/O Setup 1 Start: {Map 1}] orderedgrouped.ShuffleScheduler: ShuffleScheduler running for sourceVertex: Map 1 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true 2018-01-05T11:28:52,707 INFO [I/O Setup 1 Start: {Map 1}] orderedgrouped.ShuffleScheduler: ShuffleScheduler running for sourceVertex: Map 1 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true 2018-01-05T11:28:52,707 INFO [I/O Setup 1 Start: {Map 1}] orderedgrouped.ShuffleScheduler: ShuffleScheduler running for sourceVertex: Map 1 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true 2018-01-05T11:28:52,707 INFO [I/O Setup 1 Start: {Map 1}] orderedgrouped.ShuffleScheduler: ShuffleScheduler running for sourceVertex: Map 1 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true 2018-01-05T11:28:52,712 INFO [I/O Setup 1 Start: {Map 1}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: Map 1 2018-01-05T11:28:52,712 INFO [I/O Setup 1 Start: {Map 1}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: Map 1 2018-01-05T11:28:52,712 INFO [I/O Setup 1 Start: {Map 1}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: Map 1 2018-01-05T11:28:52,712 INFO [I/O Setup 1 Start: {Map 1}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: Map 1 2018-01-05T11:28:52,713 INFO [TezTR-518813_1_17_1_3_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:52,713 INFO [TezTR-518813_1_17_1_2_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:52,712 INFO [TezTR-518813_1_17_1_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:52,713 INFO [TezTR-518813_1_17_1_1_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:52,713 INFO [TezTR-518813_1_17_1_3_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_17_01_000003_0 2018-01-05T11:28:52,713 DEBUG [TezTR-518813_1_17_1_3_0] log.PerfLogger: 2018-01-05T11:28:52,713 DEBUG [TezTR-518813_1_17_1_3_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_17_01_000003_0_10097 2018-01-05T11:28:52,713 INFO [TezTR-518813_1_17_1_1_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_17_01_000001_0 2018-01-05T11:28:52,713 DEBUG [TezTR-518813_1_17_1_1_0] log.PerfLogger: 2018-01-05T11:28:52,713 DEBUG [TezTR-518813_1_17_1_1_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_17_01_000001_0_10094 2018-01-05T11:28:52,713 DEBUG [TezTR-518813_1_17_1_3_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:52,713 INFO [TezTR-518813_1_17_1_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_17_01_000000_0 2018-01-05T11:28:52,713 DEBUG [TezTR-518813_1_17_1_0_0] log.PerfLogger: 2018-01-05T11:28:52,713 DEBUG [TezTR-518813_1_17_1_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_17_01_000000_0_10100 2018-01-05T11:28:52,713 DEBUG [TezTR-518813_1_17_1_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:52,713 DEBUG [TezTR-518813_1_17_1_0_0] mr.ObjectCache: Creating djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:28:52,713 DEBUG [TezTR-518813_1_17_1_1_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:52,713 INFO [TezTR-518813_1_17_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_147_6959340210354157653-1/djaiswal/_tez_scratch_dir/e0eb815d-bfdd-41b0-8da2-2347a3339853/reduce.xml 2018-01-05T11:28:52,713 DEBUG [TezTR-518813_1_17_1_1_0] mr.ObjectCache: Creating djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:28:52,713 DEBUG [TezTR-518813_1_17_1_3_0] mr.ObjectCache: Creating djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:28:52,713 INFO [TezTR-518813_1_17_1_1_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_147_6959340210354157653-1/djaiswal/_tez_scratch_dir/e0eb815d-bfdd-41b0-8da2-2347a3339853/reduce.xml 2018-01-05T11:28:52,713 DEBUG [TezTR-518813_1_17_1_1_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_147_6959340210354157653-1/djaiswal/_tez_scratch_dir/e0eb815d-bfdd-41b0-8da2-2347a3339853/reduce.xml 2018-01-05T11:28:52,713 DEBUG [TezTR-518813_1_17_1_1_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_147_6959340210354157653-1/djaiswal/_tez_scratch_dir/e0eb815d-bfdd-41b0-8da2-2347a3339853/reduce.xml 2018-01-05T11:28:52,713 DEBUG [TezTR-518813_1_17_1_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_147_6959340210354157653-1/djaiswal/_tez_scratch_dir/e0eb815d-bfdd-41b0-8da2-2347a3339853/reduce.xml 2018-01-05T11:28:52,713 DEBUG [TezTR-518813_1_17_1_1_0] log.PerfLogger: 2018-01-05T11:28:52,713 INFO [TezTR-518813_1_17_1_1_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:28:52,713 INFO [TezTaskEventRouter{attempt_1515180518813_0001_17_01_000000_0}] orderedgrouped.ShuffleScheduler: All inputs fetched for input vertex : Map 1 2018-01-05T11:28:52,713 INFO [TezTR-518813_1_17_1_2_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_17_01_000002_0 2018-01-05T11:28:52,713 DEBUG [TezTR-518813_1_17_1_2_0] log.PerfLogger: 2018-01-05T11:28:52,713 DEBUG [TezTR-518813_1_17_1_2_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_17_01_000002_0_10103 2018-01-05T11:28:52,713 DEBUG [TezTR-518813_1_17_1_2_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:52,713 DEBUG [TezTR-518813_1_17_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,713 DEBUG [TezTR-518813_1_17_1_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_147_6959340210354157653-1/djaiswal/_tez_scratch_dir/e0eb815d-bfdd-41b0-8da2-2347a3339853/reduce.xml 2018-01-05T11:28:52,713 DEBUG [TezTR-518813_1_17_1_2_0] mr.ObjectCache: Creating djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:28:52,713 INFO [TezTaskEventRouter{attempt_1515180518813_0001_17_01_000000_0}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:28:52,713 INFO [TezTaskEventRouter{attempt_1515180518813_0001_17_01_000002_0}] orderedgrouped.ShuffleScheduler: All inputs fetched for input vertex : Map 1 2018-01-05T11:28:52,713 INFO [TezTaskEventRouter{attempt_1515180518813_0001_17_01_000002_0}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:28:52,713 INFO [TezTaskEventRouter{attempt_1515180518813_0001_17_01_000000_0}] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=1, numObsoletionEventsSeen=0 2018-01-05T11:28:52,713 DEBUG [TezTR-518813_1_17_1_0_0] log.PerfLogger: 2018-01-05T11:28:52,713 INFO [TezTR-518813_1_17_1_0_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:28:52,713 DEBUG [TezTR-518813_1_17_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,713 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down FetchScheduler for input: Map_1, wasInterrupted=false 2018-01-05T11:28:52,714 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down FetchScheduler for input: Map_1, wasInterrupted=false 2018-01-05T11:28:52,713 INFO [TezTaskEventRouter{attempt_1515180518813_0001_17_01_000002_0}] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=1, numObsoletionEventsSeen=0 2018-01-05T11:28:52,714 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:28:52,714 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:28:52,715 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down fetchers for input: Map_1, shutdown timetaken: 0 ms, hasFetcherExecutorStopped: true 2018-01-05T11:28:52,715 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: finalMerge with #inMemoryOutputs=0, size=0 and #onDiskOutputs=0, size=0 2018-01-05T11:28:52,715 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down fetchers for input: Map_1, shutdown timetaken: 0 ms, hasFetcherExecutorStopped: true 2018-01-05T11:28:52,715 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: finalMerge with #inMemoryOutputs=0, size=0 and #onDiskOutputs=0, size=0 2018-01-05T11:28:52,715 DEBUG [TezTR-518813_1_17_1_1_0] log.PerfLogger: 2018-01-05T11:28:52,716 INFO [TezTR-518813_1_17_1_1_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 2.39KB 2018-01-05T11:28:52,716 DEBUG [TezTR-518813_1_17_1_1_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:52,716 DEBUG [TezTR-518813_1_17_1_1_0] log.PerfLogger: 2018-01-05T11:28:52,716 DEBUG [TezTR-518813_1_17_1_1_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:52,716 DEBUG [TezTR-518813_1_17_1_1_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:52,716 DEBUG [TezTR-518813_1_17_1_0_0] log.PerfLogger: 2018-01-05T11:28:52,716 INFO [TezTR-518813_1_17_1_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 2.39KB 2018-01-05T11:28:52,717 DEBUG [TezTR-518813_1_17_1_1_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:52,717 INFO [TezTR-518813_1_17_1_1_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:28:52,717 DEBUG [TezTR-518813_1_17_1_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:52,717 DEBUG [TezTR-518813_1_17_1_0_0] log.PerfLogger: 2018-01-05T11:28:52,717 DEBUG [TezTR-518813_1_17_1_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:52,717 DEBUG [TezTR-518813_1_17_1_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:52,717 INFO [TezTR-518813_1_17_1_3_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_147_6959340210354157653-1/djaiswal/_tez_scratch_dir/e0eb815d-bfdd-41b0-8da2-2347a3339853/reduce.xml 2018-01-05T11:28:52,717 DEBUG [TezTR-518813_1_17_1_3_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_147_6959340210354157653-1/djaiswal/_tez_scratch_dir/e0eb815d-bfdd-41b0-8da2-2347a3339853/reduce.xml 2018-01-05T11:28:52,717 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: . DiskSeg: 0, 0. MemSeg: 0, 0 2018-01-05T11:28:52,717 DEBUG [TezTR-518813_1_17_1_3_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_147_6959340210354157653-1/djaiswal/_tez_scratch_dir/e0eb815d-bfdd-41b0-8da2-2347a3339853/reduce.xml 2018-01-05T11:28:52,717 INFO [TezTR-518813_1_17_1_2_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_147_6959340210354157653-1/djaiswal/_tez_scratch_dir/e0eb815d-bfdd-41b0-8da2-2347a3339853/reduce.xml 2018-01-05T11:28:52,717 DEBUG [TezTR-518813_1_17_1_2_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_147_6959340210354157653-1/djaiswal/_tez_scratch_dir/e0eb815d-bfdd-41b0-8da2-2347a3339853/reduce.xml 2018-01-05T11:28:52,717 DEBUG [TezTR-518813_1_17_1_2_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_147_6959340210354157653-1/djaiswal/_tez_scratch_dir/e0eb815d-bfdd-41b0-8da2-2347a3339853/reduce.xml 2018-01-05T11:28:52,717 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: . DiskSeg: 0, 0. MemSeg: 0, 0 2018-01-05T11:28:52,717 DEBUG [TezTR-518813_1_17_1_3_0] log.PerfLogger: 2018-01-05T11:28:52,717 INFO [TezTR-518813_1_17_1_3_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:28:52,717 DEBUG [TezTR-518813_1_17_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,717 DEBUG [TezTR-518813_1_17_1_2_0] log.PerfLogger: 2018-01-05T11:28:52,718 INFO [TezTR-518813_1_17_1_2_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:28:52,718 DEBUG [TezTR-518813_1_17_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,717 DEBUG [TezTR-518813_1_17_1_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:52,718 INFO [TezTR-518813_1_17_1_0_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:28:52,719 INFO [TezTaskEventRouter{attempt_1515180518813_0001_17_01_000003_0}] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:28:52,719 INFO [TezTaskEventRouter{attempt_1515180518813_0001_17_01_000001_0}] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:28:52,721 DEBUG [AMReporterQueueDrainer] impl.AMReporter: Removing am 10.22.8.182:49824 with last associated dag QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=4} from heartbeat with taskCount=0, amFailed=false, isDone=true 2018-01-05T11:28:52,722 DEBUG [TezTR-518813_1_17_1_3_0] log.PerfLogger: 2018-01-05T11:28:52,722 INFO [TezTR-518813_1_17_1_3_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 2.39KB 2018-01-05T11:28:52,722 DEBUG [TezTR-518813_1_17_1_3_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:52,722 DEBUG [TezTR-518813_1_17_1_3_0] log.PerfLogger: 2018-01-05T11:28:52,722 DEBUG [TezTR-518813_1_17_1_3_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:52,722 DEBUG [TezTR-518813_1_17_1_3_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:52,722 DEBUG [TezTR-518813_1_17_1_2_0] log.PerfLogger: 2018-01-05T11:28:52,723 INFO [TezTR-518813_1_17_1_2_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 2.39KB 2018-01-05T11:28:52,723 DEBUG [TezTR-518813_1_17_1_2_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:52,723 DEBUG [TezTR-518813_1_17_1_2_0] log.PerfLogger: 2018-01-05T11:28:52,723 DEBUG [TezTR-518813_1_17_1_2_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:52,723 DEBUG [TezTR-518813_1_17_1_2_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:52,724 DEBUG [TezTR-518813_1_17_1_3_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:52,724 INFO [TezTR-518813_1_17_1_3_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:28:52,724 DEBUG [TezTR-518813_1_17_1_2_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:52,724 INFO [TezTR-518813_1_17_1_2_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:28:52,726 INFO [ShuffleAndMergeRunner {Map_1}] impl.TezMerger: Nothing to merge. Returning an empty iterator 2018-01-05T11:28:52,726 INFO [ShuffleAndMergeRunner {Map_1}] impl.TezMerger: Nothing to merge. Returning an empty iterator 2018-01-05T11:28:52,726 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=17&reduce=3&map=attempt_1515180518813_0001_17_00_000000_0_10093 mapId: [attempt_1515180518813_0001_17_00_000000_0_10093] reduceId: [3] jobId: [job_1515180518813_0001] dagId: [17] keepAlive: false 2018-01-05T11:28:52,726 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=17&reduce=1&map=attempt_1515180518813_0001_17_00_000000_0_10093 mapId: [attempt_1515180518813_0001_17_00_000000_0_10093] reduceId: [1] jobId: [job_1515180518813_0001] dagId: [17] keepAlive: false 2018-01-05T11:28:52,726 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=17&reduce=3&map=attempt_1515180518813_0001_17_00_000000_0_10093; hash=...jbC52KQ+E7WWk 2018-01-05T11:28:52,726 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=17&reduce=1&map=attempt_1515180518813_0001_17_00_000000_0_10093; hash=...BTl53+vuwsyhA 2018-01-05T11:28:52,726 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=17&reduce=3&map=attempt_1515180518813_0001_17_00_000000_0_10093;reply=IcrVGV2eRXnso 2018-01-05T11:28:52,726 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=17&reduce=1&map=attempt_1515180518813_0001_17_00_000000_0_10093;reply=iWaXBsZxTvhQU 2018-01-05T11:28:52,727 INFO [TezTR-518813_1_17_1_0_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:28:52,727 INFO [TezTR-518813_1_17_1_2_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:28:52,727 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: merge complete for input vertex : Map_1 2018-01-05T11:28:52,727 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: merge complete for input vertex : Map_1 2018-01-05T11:28:52,727 INFO [TezTR-518813_1_17_1_0_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:52,727 INFO [TezTR-518813_1_17_1_2_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:52,727 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=17, user='djaiswal', attemptId='attempt_1515180518813_0001_17_00_000000_0_10093'} via loader 2018-01-05T11:28:52,727 DEBUG [TezTR-518813_1_17_1_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:28:52,727 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: Map_1: Shuffle Runner thread complete 2018-01-05T11:28:52,727 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: Map_1: Shuffle Runner thread complete 2018-01-05T11:28:52,727 DEBUG [TezTR-518813_1_17_1_2_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:28:52,727 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=17, user='djaiswal', attemptId='attempt_1515180518813_0001_17_00_000000_0_10093'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:28:52,727 DEBUG [New I/O worker #2] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_17_00_000000_0_10093 not found 2018-01-05T11:28:52,727 INFO [TezTR-518813_1_17_1_0_0] input.OrderedGroupedKVInput: Map 1: creating ValuesIterator with comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator, keyClass=org.apache.hadoop.hive.ql.io.HiveKey, valClass=org.apache.hadoop.io.BytesWritable 2018-01-05T11:28:52,727 INFO [TezTR-518813_1_17_1_2_0] input.OrderedGroupedKVInput: Map 1: creating ValuesIterator with comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator, keyClass=org.apache.hadoop.hive.ql.io.HiveKey, valClass=org.apache.hadoop.io.BytesWritable 2018-01-05T11:28:52,727 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=17, user='djaiswal', attemptId='attempt_1515180518813_0001_17_00_000000_0_10093'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:28:52,728 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache HIT: MapId attempt_1515180518813_0001_17_00_000000_0_10093 found 2018-01-05T11:28:52,728 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_17_00_000000_0_10093,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/17/output/attempt_1515180518813_0001_17_00_000000_0_10093/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/17/output/attempt_1515180518813_0001_17_00_000000_0_10093/file.out.index 2018-01-05T11:28:52,728 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_17_00_000000_0_10093,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/17/output/attempt_1515180518813_0001_17_00_000000_0_10093/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/17/output/attempt_1515180518813_0001_17_00_000000_0_10093/file.out.index 2018-01-05T11:28:52,728 DEBUG [New I/O worker #2] shufflehandler.ShuffleHandler: Content Length in shuffle : 2459 2018-01-05T11:28:52,728 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Content Length in shuffle : 7247 2018-01-05T11:28:52,728 INFO [Fetcher_O {Map_1} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=17&reduce=1&map=attempt_1515180518813_0001_17_00_000000_0_10093 sent hash and receievd reply 0 ms 2018-01-05T11:28:52,728 DEBUG [New I/O worker #2] shufflehandler.FadvisedFileRegion: shuffleBufferSize: 131072, path: /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/17/output/attempt_1515180518813_0001_17_00_000000_0_10093/file.out 2018-01-05T11:28:52,728 DEBUG [New I/O worker #1] shufflehandler.FadvisedFileRegion: shuffleBufferSize: 131072, path: /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/17/output/attempt_1515180518813_0001_17_00_000000_0_10093/file.out 2018-01-05T11:28:52,728 INFO [Fetcher_O {Map_1} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=17&reduce=3&map=attempt_1515180518813_0001_17_00_000000_0_10093 sent hash and receievd reply 0 ms 2018-01-05T11:28:52,731 DEBUG [TezTR-518813_1_17_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,731 DEBUG [TezTR-518813_1_17_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,731 DEBUG [TezTR-518813_1_17_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,731 DEBUG [TezTR-518813_1_17_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,731 DEBUG [TezTR-518813_1_17_1_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:28:52,731 DEBUG [TezTR-518813_1_17_1_2_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:28:52,731 DEBUG [TezTR-518813_1_17_1_0_0] log.PerfLogger: 2018-01-05T11:28:52,731 DEBUG [TezTR-518813_1_17_1_2_0] log.PerfLogger: 2018-01-05T11:28:52,731 INFO [TezTR-518813_1_17_1_0_0] exec.GroupByOperator: Initializing operator GBY[4] 2018-01-05T11:28:52,731 INFO [TezTR-518813_1_17_1_2_0] exec.GroupByOperator: Initializing operator GBY[4] 2018-01-05T11:28:52,731 DEBUG [TezTR-518813_1_17_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,731 DEBUG [TezTR-518813_1_17_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,731 DEBUG [TezTR-518813_1_17_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,731 DEBUG [TezTR-518813_1_17_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,731 INFO [TezTR-518813_1_17_1_2_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:52,731 DEBUG [TezTR-518813_1_17_1_2_0] exec.GroupByOperator: Initialization Done 4 GBY 2018-01-05T11:28:52,731 INFO [TezTR-518813_1_17_1_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:52,731 DEBUG [TezTR-518813_1_17_1_2_0] exec.GroupByOperator: Operator 4 GBY initialized 2018-01-05T11:28:52,731 DEBUG [TezTR-518813_1_17_1_0_0] exec.GroupByOperator: Initialization Done 4 GBY 2018-01-05T11:28:52,731 DEBUG [TezTR-518813_1_17_1_0_0] exec.GroupByOperator: Operator 4 GBY initialized 2018-01-05T11:28:52,732 DEBUG [TezTR-518813_1_17_1_0_0] exec.GroupByOperator: Initializing children of 4 GBY 2018-01-05T11:28:52,731 DEBUG [TezTR-518813_1_17_1_2_0] exec.GroupByOperator: Initializing children of 4 GBY 2018-01-05T11:28:52,732 DEBUG [TezTR-518813_1_17_1_0_0] exec.SelectOperator: Initializing child 5 SEL 2018-01-05T11:28:52,732 INFO [TezTR-518813_1_17_1_0_0] exec.SelectOperator: Initializing operator SEL[5] 2018-01-05T11:28:52,732 DEBUG [TezTR-518813_1_17_1_2_0] exec.SelectOperator: Initializing child 5 SEL 2018-01-05T11:28:52,732 INFO [TezTR-518813_1_17_1_2_0] exec.SelectOperator: Initializing operator SEL[5] 2018-01-05T11:28:52,732 INFO [TezTR-518813_1_17_1_0_0] exec.SelectOperator: SELECT struct<_col0:string,_col1:string,_col2:struct,_col3:struct> 2018-01-05T11:28:52,732 INFO [TezTR-518813_1_17_1_2_0] exec.SelectOperator: SELECT struct<_col0:string,_col1:string,_col2:struct,_col3:struct> 2018-01-05T11:28:52,732 DEBUG [TezTR-518813_1_17_1_0_0] exec.SelectOperator: Initialization Done 5 SEL 2018-01-05T11:28:52,732 DEBUG [TezTR-518813_1_17_1_0_0] exec.SelectOperator: Operator 5 SEL initialized 2018-01-05T11:28:52,732 DEBUG [TezTR-518813_1_17_1_0_0] exec.SelectOperator: Initializing children of 5 SEL 2018-01-05T11:28:52,732 DEBUG [TezTR-518813_1_17_1_2_0] exec.SelectOperator: Initialization Done 5 SEL 2018-01-05T11:28:52,732 DEBUG [TezTR-518813_1_17_1_0_0] exec.FileSinkOperator: Initializing child 6 FS 2018-01-05T11:28:52,732 INFO [TezTR-518813_1_17_1_0_0] exec.FileSinkOperator: Initializing operator FS[6] 2018-01-05T11:28:52,732 DEBUG [TezTR-518813_1_17_1_2_0] exec.SelectOperator: Operator 5 SEL initialized 2018-01-05T11:28:52,732 DEBUG [TezTR-518813_1_17_1_2_0] exec.SelectOperator: Initializing children of 5 SEL 2018-01-05T11:28:52,732 DEBUG [TezTR-518813_1_17_1_2_0] exec.FileSinkOperator: Initializing child 6 FS 2018-01-05T11:28:52,732 INFO [TezTR-518813_1_17_1_2_0] exec.FileSinkOperator: Initializing operator FS[6] 2018-01-05T11:28:52,732 DEBUG [TezTR-518813_1_17_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,732 INFO [Fetcher_O {Map_1} #0] orderedgrouped.MergeManager: CloseInMemoryFile. Current state: inMemoryMapOutputs.size=1, commitMemory=0, usedMemory=2400. Since last log: count=1, min=2400, max=2400, total=2400, avg=2400.0 2018-01-05T11:28:52,732 INFO [Fetcher_O {Map_1} #0] orderedgrouped.MergeManager: CloseInMemoryFile. Current state: inMemoryMapOutputs.size=1, commitMemory=0, usedMemory=7188. Since last log: count=1, min=7188, max=7188, total=7188, avg=7188.0 2018-01-05T11:28:52,732 DEBUG [TezTR-518813_1_17_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,732 INFO [Fetcher_O {Map_1} #0] ShuffleScheduler.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_17_00_000000_0_10093} to MEMORY, csize=7192, dsize=7188, EndTime=1515180532732, TimeTaken=4, Rate=1.71 MB/s 2018-01-05T11:28:52,732 INFO [Fetcher_O {Map_1} #0] ShuffleScheduler.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_17_00_000000_0_10093} to MEMORY, csize=2404, dsize=2400, EndTime=1515180532732, TimeTaken=4, Rate=0.57 MB/s 2018-01-05T11:28:52,732 INFO [Fetcher_O {Map_1} #0] orderedgrouped.ShuffleScheduler: All inputs fetched for input vertex : Map 1 2018-01-05T11:28:52,732 INFO [Fetcher_O {Map_1} #0] orderedgrouped.ShuffleScheduler: All inputs fetched for input vertex : Map 1 2018-01-05T11:28:52,732 INFO [Fetcher_O {Map_1} #0] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.01 MB/s) 2018-01-05T11:28:52,732 INFO [Fetcher_O {Map_1} #0] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:28:52,732 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down FetchScheduler for input: Map_1, wasInterrupted=false 2018-01-05T11:28:52,732 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down FetchScheduler for input: Map_1, wasInterrupted=false 2018-01-05T11:28:52,732 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.01 MB/s) 2018-01-05T11:28:52,733 INFO [Fetcher_O {Map_1} #0] orderedgrouped.ShuffleScheduler: Map_1: Already shutdown. Ignoring fetch complete 2018-01-05T11:28:52,733 INFO [TezTR-518813_1_17_1_0_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@3cdb3c0b]:[_col0, _col1, _col2, _col3]:[struct, struct, string, string]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@2352d100 2018-01-05T11:28:52,736 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:28:52,735 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down fetchers for input: Map_1, shutdown timetaken: 0 ms, hasFetcherExecutorStopped: true 2018-01-05T11:28:52,736 INFO [TezTR-518813_1_17_1_2_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@7e52fdb9]:[_col0, _col1, _col2, _col3]:[struct, struct, string, string]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@765b2818 2018-01-05T11:28:52,736 DEBUG [TezTR-518813_1_17_1_0_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:28:52,736 DEBUG [TezTR-518813_1_17_1_2_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:28:52,736 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down fetchers for input: Map_1, shutdown timetaken: 0 ms, hasFetcherExecutorStopped: true 2018-01-05T11:28:52,736 DEBUG [TezTR-518813_1_17_1_0_0] exec.FileSinkOperator: Initialization Done 6 FS 2018-01-05T11:28:52,737 DEBUG [TezTR-518813_1_17_1_2_0] exec.FileSinkOperator: Initialization Done 6 FS 2018-01-05T11:28:52,737 DEBUG [TezTR-518813_1_17_1_2_0] exec.FileSinkOperator: Operator 6 FS initialized 2018-01-05T11:28:52,737 DEBUG [TezTR-518813_1_17_1_2_0] exec.FileSinkOperator: Initialization Done 6 FS done is reset. 2018-01-05T11:28:52,737 DEBUG [TezTR-518813_1_17_1_2_0] exec.SelectOperator: Initialization Done 5 SEL done is reset. 2018-01-05T11:28:52,737 DEBUG [TezTR-518813_1_17_1_0_0] exec.FileSinkOperator: Operator 6 FS initialized 2018-01-05T11:28:52,737 DEBUG [TezTR-518813_1_17_1_0_0] exec.FileSinkOperator: Initialization Done 6 FS done is reset. 2018-01-05T11:28:52,737 DEBUG [TezTR-518813_1_17_1_0_0] exec.SelectOperator: Initialization Done 5 SEL done is reset. 2018-01-05T11:28:52,737 DEBUG [TezTR-518813_1_17_1_0_0] exec.GroupByOperator: Initialization Done 4 GBY done is reset. 2018-01-05T11:28:52,737 DEBUG [TezTR-518813_1_17_1_2_0] exec.GroupByOperator: Initialization Done 4 GBY done is reset. 2018-01-05T11:28:52,737 INFO [TezTR-518813_1_17_1_2_0] tez.ReduceRecordProcessor: Id =4 Id =5 Id =6 <\Children> Id = 5 null<\Parent> <\FS> <\Children> Id = 4 null<\Parent> <\SEL> <\Children> <\Parent> <\GBY> 2018-01-05T11:28:52,737 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: TotalInMemFetchStats: count=1, totalSize=2400, min=2400, max=2400, avg=1.0 2018-01-05T11:28:52,737 INFO [TezTR-518813_1_17_1_0_0] tez.ReduceRecordProcessor: Id =4 Id =5 Id =6 <\Children> Id = 5 null<\Parent> <\FS> <\Children> Id = 4 null<\Parent> <\SEL> <\Children> <\Parent> <\GBY> 2018-01-05T11:28:52,737 DEBUG [TezTR-518813_1_17_1_2_0] log.PerfLogger: 2018-01-05T11:28:52,737 DEBUG [TezTR-518813_1_17_1_0_0] log.PerfLogger: 2018-01-05T11:28:52,737 INFO [TezTR-518813_1_17_1_0_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 2 2018-01-05T11:28:52,737 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: finalMerge with #inMemoryOutputs=1, size=2400 and #onDiskOutputs=0, size=0 2018-01-05T11:28:52,736 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: TotalInMemFetchStats: count=1, totalSize=7188, min=7188, max=7188, avg=1.0 2018-01-05T11:28:52,737 INFO [TezTR-518813_1_17_1_2_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 2 2018-01-05T11:28:52,737 DEBUG [TezTR-518813_1_17_1_0_0] log.PerfLogger: 2018-01-05T11:28:52,737 DEBUG [TezTR-518813_1_17_1_2_0] log.PerfLogger: 2018-01-05T11:28:52,737 DEBUG [TezTR-518813_1_17_1_0_0] mr.ObjectCache: djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:28:52,737 DEBUG [TezTR-518813_1_17_1_2_0] mr.ObjectCache: djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:28:52,737 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: finalMerge with #inMemoryOutputs=1, size=7188 and #onDiskOutputs=0, size=0 2018-01-05T11:28:52,737 DEBUG [TezTR-518813_1_17_1_2_0] exec.GroupByOperator: close called for operator GBY[4] 2018-01-05T11:28:52,737 DEBUG [TezTR-518813_1_17_1_0_0] exec.GroupByOperator: close called for operator GBY[4] 2018-01-05T11:28:52,737 INFO [TezTR-518813_1_17_1_0_0] exec.GroupByOperator: Closing operator GBY[4] 2018-01-05T11:28:52,737 INFO [TezTR-518813_1_17_1_2_0] exec.GroupByOperator: Closing operator GBY[4] 2018-01-05T11:28:52,737 DEBUG [TezTR-518813_1_17_1_2_0] exec.GroupByOperator: Closing child = SEL[5] 2018-01-05T11:28:52,737 DEBUG [TezTR-518813_1_17_1_0_0] exec.GroupByOperator: Closing child = SEL[5] 2018-01-05T11:28:52,737 DEBUG [TezTR-518813_1_17_1_0_0] exec.SelectOperator: close called for operator SEL[5] 2018-01-05T11:28:52,737 DEBUG [TezTR-518813_1_17_1_0_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:52,737 INFO [TezTR-518813_1_17_1_0_0] exec.SelectOperator: Closing operator SEL[5] 2018-01-05T11:28:52,737 DEBUG [TezTR-518813_1_17_1_2_0] exec.SelectOperator: close called for operator SEL[5] 2018-01-05T11:28:52,737 DEBUG [TezTR-518813_1_17_1_0_0] exec.SelectOperator: Closing child = FS[6] 2018-01-05T11:28:52,737 DEBUG [TezTR-518813_1_17_1_0_0] exec.FileSinkOperator: close called for operator FS[6] 2018-01-05T11:28:52,737 DEBUG [TezTR-518813_1_17_1_0_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:52,737 INFO [TezTR-518813_1_17_1_0_0] exec.FileSinkOperator: Closing operator FS[6] 2018-01-05T11:28:52,737 INFO [TezTR-518813_1_17_1_0_0] exec.FileSinkOperator: FS[6]: records written - 0 2018-01-05T11:28:52,737 DEBUG [TezTR-518813_1_17_1_2_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:52,737 INFO [TezTR-518813_1_17_1_2_0] exec.SelectOperator: Closing operator SEL[5] 2018-01-05T11:28:52,737 DEBUG [TezTR-518813_1_17_1_2_0] exec.SelectOperator: Closing child = FS[6] 2018-01-05T11:28:52,737 DEBUG [TezTR-518813_1_17_1_2_0] exec.FileSinkOperator: close called for operator FS[6] 2018-01-05T11:28:52,737 DEBUG [TezTR-518813_1_17_1_2_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:52,737 INFO [TezTR-518813_1_17_1_2_0] exec.FileSinkOperator: Closing operator FS[6] 2018-01-05T11:28:52,737 INFO [TezTR-518813_1_17_1_2_0] exec.FileSinkOperator: FS[6]: records written - 0 2018-01-05T11:28:52,737 INFO [TezTR-518813_1_17_1_2_0] exec.FileSinkOperator: RECORDS_OUT_0:0, 2018-01-05T11:28:52,737 DEBUG [TezTR-518813_1_17_1_2_0] exec.FileSinkOperator: 6 Close done 2018-01-05T11:28:52,737 DEBUG [TezTR-518813_1_17_1_2_0] exec.SelectOperator: 5 Close done 2018-01-05T11:28:52,737 DEBUG [TezTR-518813_1_17_1_2_0] exec.GroupByOperator: 4 Close done 2018-01-05T11:28:52,737 INFO [TezTR-518813_1_17_1_0_0] exec.FileSinkOperator: RECORDS_OUT_0:0, 2018-01-05T11:28:52,737 DEBUG [TezTR-518813_1_17_1_0_0] exec.FileSinkOperator: 6 Close done 2018-01-05T11:28:52,737 DEBUG [TezTR-518813_1_17_1_0_0] exec.SelectOperator: 5 Close done 2018-01-05T11:28:52,737 DEBUG [TezTR-518813_1_17_1_0_0] exec.GroupByOperator: 4 Close done 2018-01-05T11:28:52,737 INFO [TezTR-518813_1_17_1_2_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_17_01_000002_0 2018-01-05T11:28:52,737 INFO [TezTR-518813_1_17_1_2_0] orderedgrouped.Shuffle: Shutting down Shuffle for source: Map_1 2018-01-05T11:28:52,737 INFO [TezTR-518813_1_17_1_2_0] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=1, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:28:52,737 INFO [TezTR-518813_1_17_1_2_0] output.MROutput: out_Reducer 2 closed 2018-01-05T11:28:52,737 INFO [TezTR-518813_1_17_1_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_17_01_000000_0 2018-01-05T11:28:52,737 INFO [TezTR-518813_1_17_1_0_0] orderedgrouped.Shuffle: Shutting down Shuffle for source: Map_1 2018-01-05T11:28:52,737 INFO [TezTR-518813_1_17_1_0_0] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=1, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:28:52,737 INFO [TezTR-518813_1_17_1_0_0] output.MROutput: out_Reducer 2 closed 2018-01-05T11:28:52,737 DEBUG [TezTR-518813_1_17_1_2_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:52,737 INFO [TezTR-518813_1_17_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:52,737 DEBUG [TezTR-518813_1_17_1_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:52,737 INFO [TezTR-518813_1_17_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:52,738 INFO [TezTR-518813_1_17_1_2_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_17_01_000002_0, askedToStop=false 2018-01-05T11:28:52,738 INFO [TezTR-518813_1_17_1_2_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_17_01_000002_0, stopRequested=false 2018-01-05T11:28:52,738 INFO [TezTR-518813_1_17_1_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_17_01_000000_0, askedToStop=false 2018-01-05T11:28:52,738 INFO [TezTR-518813_1_17_1_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_17_01_000000_0, stopRequested=false 2018-01-05T11:28:52,738 INFO [TezTR-518813_1_17_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_17_01_000002_0: Counters: 29 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=0, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=0, NUM_SKIPPED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=0, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, SHUFFLE_BYTES=0, SHUFFLE_BYTES_DECOMPRESSED=0, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=26, MERGE_PHASE_TIME=38, FIRST_EVENT_RECEIVED=24, LAST_EVENT_RECEIVED=24][HIVE RECORDS_OUT_0=0][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0]] 2018-01-05T11:28:52,738 INFO [TezTR-518813_1_17_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_17_01_000000_0: Counters: 29 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=0, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=0, NUM_SKIPPED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=0, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, SHUFFLE_BYTES=0, SHUFFLE_BYTES_DECOMPRESSED=0, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=26, MERGE_PHASE_TIME=38, FIRST_EVENT_RECEIVED=24, LAST_EVENT_RECEIVED=24][HIVE RECORDS_OUT_0=0][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0]] 2018-01-05T11:28:52,738 DEBUG [TezTR-518813_1_17_1_2_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_17_1_2_0 thread id: 1168 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:52,738 DEBUG [TezTR-518813_1_17_1_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_17_1_0_0 thread id: 1162 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:52,738 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_17_01_000002_0, isTaskDone=true 2018-01-05T11:28:52,738 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_17_01_000000_0, isTaskDone=true 2018-01-05T11:28:52,738 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000035, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_17_01_000002_0, eventCount=2 } 2018-01-05T11:28:52,738 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000034, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_17_01_000000_0, eventCount=2 } 2018-01-05T11:28:52,739 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_17][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_17_01_000002_0, creationTime=1515180532664, allocationTime=1515180532667, startTime=1515180532670, finishTime=1515180532739, timeTaken=69, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:52,739 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_17][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_17_01_000000_0, creationTime=1515180532664, allocationTime=1515180532666, startTime=1515180532669, finishTime=1515180532739, timeTaken=70, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:52,739 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_17_01_000002_0 2018-01-05T11:28:52,740 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_17][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_17_01_000002, startTime=1515180532670, finishTime=1515180532739, timeTaken=69, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_17_01_000002_0, diagnostics=, counters=Counters: 34, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=0, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=0, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=0, NUM_SKIPPED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=0, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, SHUFFLE_BYTES=0, SHUFFLE_BYTES_DECOMPRESSED=0, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=26, MERGE_PHASE_TIME=38, FIRST_EVENT_RECEIVED=24, LAST_EVENT_RECEIVED=24, HIVE, RECORDS_OUT_0=0, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0 2018-01-05T11:28:52,740 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_17_01_000000_0 2018-01-05T11:28:52,740 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_17][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_17_01_000000, startTime=1515180532669, finishTime=1515180532740, timeTaken=71, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_17_01_000000_0, diagnostics=, counters=Counters: 34, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=0, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=0, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=0, NUM_SKIPPED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=0, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, SHUFFLE_BYTES=0, SHUFFLE_BYTES_DECOMPRESSED=0, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=26, MERGE_PHASE_TIME=38, FIRST_EVENT_RECEIVED=24, LAST_EVENT_RECEIVED=24, HIVE, RECORDS_OUT_0=0, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0 2018-01-05T11:28:52,740 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_17_01 [Reducer 2], tasks=4, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:52,740 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_17_01 [Reducer 2], tasks=4, failed=0, killed=0, success=2, completed=2, commits=0, err=null 2018-01-05T11:28:52,740 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:28:52,740 INFO [Task-Executor-2] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_17_01_000002_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:52,740 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_17_01_000002_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:52,740 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_17_01_000002_0; false 2018-01-05T11:28:52,740 INFO [Task-Executor-2] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_17_01_000002_0 2018-01-05T11:28:52,741 INFO [Task-Executor-2] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000035=63 2018-01-05T11:28:52,741 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_17_01_000002_0: true 2018-01-05T11:28:52,740 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_17_01_000002_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:52,741 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_17_01_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:52,741 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_17_01_000000_0; false 2018-01-05T11:28:52,741 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_17_01_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:52,741 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:28:52,741 INFO [Task-Executor-1] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_17_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:52,741 INFO [Task-Executor-1] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_17_01_000000_0 2018-01-05T11:28:52,741 INFO [Task-Executor-1] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000034=66 2018-01-05T11:28:52,741 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_17_01_000000_0: true 2018-01-05T11:28:52,741 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_17][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000035, stoppedTime=1515180532741, exitStatus=0 2018-01-05T11:28:52,741 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:52,741 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000035 2018-01-05T11:28:52,742 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:52,742 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000035 exited with diagnostics set to Container completed. 2018-01-05T11:28:52,742 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_17][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000034, stoppedTime=1515180532742, exitStatus=0 2018-01-05T11:28:52,742 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000034 exited with diagnostics set to Container completed. 2018-01-05T11:28:52,742 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000034 2018-01-05T11:28:52,756 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: merge complete for input vertex : Map_1 2018-01-05T11:28:52,756 INFO [TezTR-518813_1_17_1_1_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:28:52,756 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: merge complete for input vertex : Map_1 2018-01-05T11:28:52,756 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: Map_1: Shuffle Runner thread complete 2018-01-05T11:28:52,756 INFO [TezTR-518813_1_17_1_3_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:28:52,756 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: Map_1: Shuffle Runner thread complete 2018-01-05T11:28:52,756 INFO [TezTR-518813_1_17_1_1_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:52,756 DEBUG [TezTR-518813_1_17_1_1_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:28:52,756 INFO [TezTR-518813_1_17_1_3_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:52,756 DEBUG [TezTR-518813_1_17_1_3_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:28:52,756 INFO [TezTR-518813_1_17_1_1_0] input.OrderedGroupedKVInput: Map 1: creating ValuesIterator with comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator, keyClass=org.apache.hadoop.hive.ql.io.HiveKey, valClass=org.apache.hadoop.io.BytesWritable 2018-01-05T11:28:52,756 INFO [TezTR-518813_1_17_1_3_0] input.OrderedGroupedKVInput: Map 1: creating ValuesIterator with comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator, keyClass=org.apache.hadoop.hive.ql.io.HiveKey, valClass=org.apache.hadoop.io.BytesWritable 2018-01-05T11:28:52,756 DEBUG [TezTR-518813_1_17_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,756 DEBUG [TezTR-518813_1_17_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,756 DEBUG [TezTR-518813_1_17_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,756 DEBUG [TezTR-518813_1_17_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,756 DEBUG [TezTR-518813_1_17_1_1_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:28:52,756 DEBUG [TezTR-518813_1_17_1_3_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:28:52,756 DEBUG [TezTR-518813_1_17_1_1_0] log.PerfLogger: 2018-01-05T11:28:52,756 DEBUG [TezTR-518813_1_17_1_3_0] log.PerfLogger: 2018-01-05T11:28:52,756 INFO [TezTR-518813_1_17_1_1_0] exec.GroupByOperator: Initializing operator GBY[4] 2018-01-05T11:28:52,756 INFO [TezTR-518813_1_17_1_3_0] exec.GroupByOperator: Initializing operator GBY[4] 2018-01-05T11:28:52,756 DEBUG [TezTR-518813_1_17_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,756 DEBUG [TezTR-518813_1_17_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,756 DEBUG [TezTR-518813_1_17_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,756 DEBUG [TezTR-518813_1_17_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,757 INFO [TezTR-518813_1_17_1_1_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:52,757 DEBUG [TezTR-518813_1_17_1_1_0] exec.GroupByOperator: Initialization Done 4 GBY 2018-01-05T11:28:52,757 DEBUG [TezTR-518813_1_17_1_1_0] exec.GroupByOperator: Operator 4 GBY initialized 2018-01-05T11:28:52,757 INFO [TezTR-518813_1_17_1_3_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:52,757 DEBUG [TezTR-518813_1_17_1_3_0] exec.GroupByOperator: Initialization Done 4 GBY 2018-01-05T11:28:52,757 DEBUG [TezTR-518813_1_17_1_3_0] exec.GroupByOperator: Operator 4 GBY initialized 2018-01-05T11:28:52,757 DEBUG [TezTR-518813_1_17_1_1_0] exec.GroupByOperator: Initializing children of 4 GBY 2018-01-05T11:28:52,757 DEBUG [TezTR-518813_1_17_1_3_0] exec.GroupByOperator: Initializing children of 4 GBY 2018-01-05T11:28:52,757 DEBUG [TezTR-518813_1_17_1_1_0] exec.SelectOperator: Initializing child 5 SEL 2018-01-05T11:28:52,757 INFO [TezTR-518813_1_17_1_1_0] exec.SelectOperator: Initializing operator SEL[5] 2018-01-05T11:28:52,757 DEBUG [TezTR-518813_1_17_1_3_0] exec.SelectOperator: Initializing child 5 SEL 2018-01-05T11:28:52,757 INFO [TezTR-518813_1_17_1_3_0] exec.SelectOperator: Initializing operator SEL[5] 2018-01-05T11:28:52,757 INFO [TezTR-518813_1_17_1_1_0] exec.SelectOperator: SELECT struct<_col0:string,_col1:string,_col2:struct,_col3:struct> 2018-01-05T11:28:52,757 INFO [TezTR-518813_1_17_1_3_0] exec.SelectOperator: SELECT struct<_col0:string,_col1:string,_col2:struct,_col3:struct> 2018-01-05T11:28:52,757 DEBUG [TezTR-518813_1_17_1_1_0] exec.SelectOperator: Initialization Done 5 SEL 2018-01-05T11:28:52,757 DEBUG [TezTR-518813_1_17_1_1_0] exec.SelectOperator: Operator 5 SEL initialized 2018-01-05T11:28:52,757 DEBUG [TezTR-518813_1_17_1_1_0] exec.SelectOperator: Initializing children of 5 SEL 2018-01-05T11:28:52,757 DEBUG [TezTR-518813_1_17_1_1_0] exec.FileSinkOperator: Initializing child 6 FS 2018-01-05T11:28:52,757 DEBUG [TezTR-518813_1_17_1_3_0] exec.SelectOperator: Initialization Done 5 SEL 2018-01-05T11:28:52,757 INFO [TezTR-518813_1_17_1_1_0] exec.FileSinkOperator: Initializing operator FS[6] 2018-01-05T11:28:52,757 DEBUG [TezTR-518813_1_17_1_3_0] exec.SelectOperator: Operator 5 SEL initialized 2018-01-05T11:28:52,757 DEBUG [TezTR-518813_1_17_1_3_0] exec.SelectOperator: Initializing children of 5 SEL 2018-01-05T11:28:52,757 DEBUG [TezTR-518813_1_17_1_3_0] exec.FileSinkOperator: Initializing child 6 FS 2018-01-05T11:28:52,757 INFO [TezTR-518813_1_17_1_3_0] exec.FileSinkOperator: Initializing operator FS[6] 2018-01-05T11:28:52,757 DEBUG [TezTR-518813_1_17_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,757 DEBUG [TezTR-518813_1_17_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:52,758 INFO [TezTR-518813_1_17_1_3_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@35a3ed2a]:[_col0, _col1, _col2, _col3]:[struct, struct, string, string]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@5ec0bd56 2018-01-05T11:28:52,758 INFO [TezTR-518813_1_17_1_1_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@46cc9249]:[_col0, _col1, _col2, _col3]:[struct, struct, string, string]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@7eef27a7 2018-01-05T11:28:52,758 DEBUG [TezTR-518813_1_17_1_3_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:28:52,758 DEBUG [TezTR-518813_1_17_1_3_0] exec.FileSinkOperator: Initialization Done 6 FS 2018-01-05T11:28:52,758 DEBUG [TezTR-518813_1_17_1_3_0] exec.FileSinkOperator: Operator 6 FS initialized 2018-01-05T11:28:52,758 DEBUG [TezTR-518813_1_17_1_3_0] exec.FileSinkOperator: Initialization Done 6 FS done is reset. 2018-01-05T11:28:52,758 DEBUG [TezTR-518813_1_17_1_3_0] exec.SelectOperator: Initialization Done 5 SEL done is reset. 2018-01-05T11:28:52,758 DEBUG [TezTR-518813_1_17_1_1_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:28:52,758 DEBUG [TezTR-518813_1_17_1_3_0] exec.GroupByOperator: Initialization Done 4 GBY done is reset. 2018-01-05T11:28:52,758 DEBUG [TezTR-518813_1_17_1_1_0] exec.FileSinkOperator: Initialization Done 6 FS 2018-01-05T11:28:52,758 DEBUG [TezTR-518813_1_17_1_1_0] exec.FileSinkOperator: Operator 6 FS initialized 2018-01-05T11:28:52,758 DEBUG [TezTR-518813_1_17_1_1_0] exec.FileSinkOperator: Initialization Done 6 FS done is reset. 2018-01-05T11:28:52,758 INFO [TezTR-518813_1_17_1_3_0] tez.ReduceRecordProcessor: Id =4 Id =5 Id =6 <\Children> Id = 5 null<\Parent> <\FS> <\Children> Id = 4 null<\Parent> <\SEL> <\Children> <\Parent> <\GBY> 2018-01-05T11:28:52,758 DEBUG [TezTR-518813_1_17_1_1_0] exec.SelectOperator: Initialization Done 5 SEL done is reset. 2018-01-05T11:28:52,758 DEBUG [TezTR-518813_1_17_1_1_0] exec.GroupByOperator: Initialization Done 4 GBY done is reset. 2018-01-05T11:28:52,758 DEBUG [TezTR-518813_1_17_1_3_0] log.PerfLogger: 2018-01-05T11:28:52,758 INFO [TezTR-518813_1_17_1_1_0] tez.ReduceRecordProcessor: Id =4 Id =5 Id =6 <\Children> Id = 5 null<\Parent> <\FS> <\Children> Id = 4 null<\Parent> <\SEL> <\Children> <\Parent> <\GBY> 2018-01-05T11:28:52,758 INFO [TezTR-518813_1_17_1_3_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 2 2018-01-05T11:28:52,758 DEBUG [TezTR-518813_1_17_1_1_0] log.PerfLogger: 2018-01-05T11:28:52,758 INFO [TezTR-518813_1_17_1_1_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 2 2018-01-05T11:28:52,759 DEBUG [TezTR-518813_1_17_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:28:52,759 DEBUG [TezTR-518813_1_17_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:28:52,759 DEBUG [TezTR-518813_1_17_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:28:52,759 DEBUG [TezTR-518813_1_17_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:28:52,759 DEBUG [TezTR-518813_1_17_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:28:52,759 DEBUG [TezTR-518813_1_17_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:28:52,759 DEBUG [TezTR-518813_1_17_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:28:52,759 DEBUG [TezTR-518813_1_17_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:28:52,759 DEBUG [TezTR-518813_1_17_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:28:52,759 DEBUG [TezTR-518813_1_17_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:28:52,759 DEBUG [TezTR-518813_1_17_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:28:52,759 DEBUG [TezTR-518813_1_17_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:28:52,759 DEBUG [TezTR-518813_1_17_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:28:52,759 DEBUG [TezTR-518813_1_17_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:28:52,759 DEBUG [TezTR-518813_1_17_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:28:52,760 DEBUG [TezTR-518813_1_17_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:28:52,760 DEBUG [TezTR-518813_1_17_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:28:52,760 DEBUG [TezTR-518813_1_17_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:28:52,761 DEBUG [TezTR-518813_1_17_1_1_0] log.PerfLogger: 2018-01-05T11:28:52,761 DEBUG [TezTR-518813_1_17_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:28:52,761 DEBUG [TezTR-518813_1_17_1_1_0] mr.ObjectCache: djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:28:52,761 DEBUG [TezTR-518813_1_17_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:28:52,761 DEBUG [TezTR-518813_1_17_1_1_0] exec.GroupByOperator: close called for operator GBY[4] 2018-01-05T11:28:52,761 INFO [TezTR-518813_1_17_1_1_0] exec.GroupByOperator: Closing operator GBY[4] 2018-01-05T11:28:52,761 DEBUG [TezTR-518813_1_17_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:28:52,761 DEBUG [TezTR-518813_1_17_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:28:52,761 DEBUG [TezTR-518813_1_17_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:28:52,761 DEBUG [TezTR-518813_1_17_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:28:52,761 DEBUG [TezTR-518813_1_17_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:28:52,761 DEBUG [TezTR-518813_1_17_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:28:52,761 DEBUG [TezTR-518813_1_17_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:28:52,761 DEBUG [TezTR-518813_1_17_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:28:52,761 DEBUG [TezTR-518813_1_17_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:28:52,761 DEBUG [TezTR-518813_1_17_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:28:52,761 DEBUG [TezTR-518813_1_17_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:28:52,761 DEBUG [TezTR-518813_1_17_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:28:52,761 DEBUG [TezTR-518813_1_17_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:28:52,761 DEBUG [TezTR-518813_1_17_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:28:52,761 DEBUG [TezTR-518813_1_17_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:28:52,761 DEBUG [TezTR-518813_1_17_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:28:52,761 INFO [TezTR-518813_1_17_1_1_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_151_4467563815339635813-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-52_151_4467563815339635813-1/_tmp.-ext-10002/000001_0 2018-01-05T11:28:52,761 INFO [TezTR-518813_1_17_1_1_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_151_4467563815339635813-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-52_151_4467563815339635813-1/_task_tmp.-ext-10002/_tmp.000001_0 2018-01-05T11:28:52,761 INFO [TezTR-518813_1_17_1_1_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_151_4467563815339635813-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-52_151_4467563815339635813-1/_tmp.-ext-10002/000001_0 2018-01-05T11:28:52,761 INFO [TezTR-518813_1_17_1_3_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_151_4467563815339635813-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-52_151_4467563815339635813-1/_tmp.-ext-10002/000003_0 2018-01-05T11:28:52,761 INFO [TezTR-518813_1_17_1_3_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_151_4467563815339635813-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-52_151_4467563815339635813-1/_task_tmp.-ext-10002/_tmp.000003_0 2018-01-05T11:28:52,761 INFO [TezTR-518813_1_17_1_3_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_151_4467563815339635813-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-52_151_4467563815339635813-1/_tmp.-ext-10002/000003_0 2018-01-05T11:28:52,768 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000031, QueryId=djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab-17, DagId=17, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180532560, EndTime=1515180532768 2018-01-05T11:28:52,768 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_17_01_000002_0 2018-01-05T11:28:52,768 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000035, QueryId=djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab-17, Vertex=Reducer 2, FragmentNum=2, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:28:52,768 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_17_01_000002_0 complete. WaitQueueSize=0, numSlotsAvailable=1, preemptionQueueSize=3 2018-01-05T11:28:52,768 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_17_01_000002_0 2018-01-05T11:28:52,768 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000035, QueryId=djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab-17, DagId=17, VertexName=Reducer 2, TaskId=2, TaskAttemptId=0, ThreadName=Task-Executor-2, Succeeded=true, StartTime=1515180532677, EndTime=1515180532768 2018-01-05T11:28:52,768 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_17_01_000000_0 2018-01-05T11:28:52,768 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000034, QueryId=djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab-17, Vertex=Reducer 2, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:28:52,768 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_17_01_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=2, preemptionQueueSize=2 2018-01-05T11:28:52,768 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_17_01_000000_0 2018-01-05T11:28:52,768 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000034, QueryId=djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab-17, DagId=17, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-1, Succeeded=true, StartTime=1515180532674, EndTime=1515180532768 2018-01-05T11:28:52,777 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000032, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_17_01_000001_0, eventCount=1 } 2018-01-05T11:28:52,777 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:28:52,779 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000033, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_17_01_000003_0, eventCount=1 } 2018-01-05T11:28:52,780 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:28:52,784 INFO [TezTR-518813_1_17_1_3_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:28:52,785 DEBUG [TezTR-518813_1_17_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:28:52,785 DEBUG [TezTR-518813_1_17_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:28:52,785 DEBUG [TezTR-518813_1_17_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:28:52,785 DEBUG [TezTR-518813_1_17_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:28:52,785 DEBUG [TezTR-518813_1_17_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:28:52,785 DEBUG [TezTR-518813_1_17_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:28:52,785 DEBUG [TezTR-518813_1_17_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:28:52,785 DEBUG [TezTR-518813_1_17_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:28:52,785 DEBUG [TezTR-518813_1_17_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:28:52,785 DEBUG [TezTR-518813_1_17_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:28:52,785 DEBUG [TezTR-518813_1_17_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:28:52,785 DEBUG [TezTR-518813_1_17_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:28:52,785 DEBUG [TezTR-518813_1_17_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:28:52,785 DEBUG [TezTR-518813_1_17_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:28:52,785 DEBUG [TezTR-518813_1_17_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:28:52,785 DEBUG [TezTR-518813_1_17_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:28:52,785 DEBUG [TezTR-518813_1_17_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:28:52,785 DEBUG [TezTR-518813_1_17_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:28:52,786 DEBUG [TezTR-518813_1_17_1_3_0] log.PerfLogger: 2018-01-05T11:28:52,786 DEBUG [TezTR-518813_1_17_1_3_0] mr.ObjectCache: djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:28:52,786 DEBUG [TezTR-518813_1_17_1_3_0] exec.GroupByOperator: close called for operator GBY[4] 2018-01-05T11:28:52,786 INFO [TezTR-518813_1_17_1_3_0] exec.GroupByOperator: Closing operator GBY[4] 2018-01-05T11:28:52,786 DEBUG [TezTR-518813_1_17_1_3_0] exec.GroupByOperator: Closing child = SEL[5] 2018-01-05T11:28:52,786 DEBUG [TezTR-518813_1_17_1_3_0] exec.SelectOperator: close called for operator SEL[5] 2018-01-05T11:28:52,786 DEBUG [TezTR-518813_1_17_1_3_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:52,786 INFO [TezTR-518813_1_17_1_3_0] exec.SelectOperator: Closing operator SEL[5] 2018-01-05T11:28:52,786 DEBUG [TezTR-518813_1_17_1_3_0] exec.SelectOperator: Closing child = FS[6] 2018-01-05T11:28:52,786 DEBUG [TezTR-518813_1_17_1_3_0] exec.FileSinkOperator: close called for operator FS[6] 2018-01-05T11:28:52,786 DEBUG [TezTR-518813_1_17_1_3_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:52,786 INFO [TezTR-518813_1_17_1_3_0] exec.FileSinkOperator: Closing operator FS[6] 2018-01-05T11:28:52,786 INFO [TezTR-518813_1_17_1_3_0] exec.FileSinkOperator: FS[6]: records written - 3 2018-01-05T11:28:52,786 INFO [TezTR-518813_1_17_1_3_0] exec.FileSinkOperator: RECORDS_OUT_0:3, 2018-01-05T11:28:52,786 DEBUG [TezTR-518813_1_17_1_3_0] exec.FileSinkOperator: 6 Close done 2018-01-05T11:28:52,786 DEBUG [TezTR-518813_1_17_1_3_0] exec.SelectOperator: 5 Close done 2018-01-05T11:28:52,786 DEBUG [TezTR-518813_1_17_1_3_0] exec.GroupByOperator: 4 Close done 2018-01-05T11:28:52,786 INFO [TezTR-518813_1_17_1_3_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_17_01_000003_0 2018-01-05T11:28:52,786 INFO [TezTR-518813_1_17_1_3_0] orderedgrouped.Shuffle: Shutting down Shuffle for source: Map_1 2018-01-05T11:28:52,786 INFO [TezTR-518813_1_17_1_3_0] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:28:52,786 INFO [TezTR-518813_1_17_1_3_0] output.MROutput: out_Reducer 2 closed 2018-01-05T11:28:52,786 DEBUG [TezTR-518813_1_17_1_3_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:52,786 INFO [TezTR-518813_1_17_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:52,787 INFO [TezTR-518813_1_17_1_3_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_17_01_000003_0, askedToStop=false 2018-01-05T11:28:52,787 INFO [TezTR-518813_1_17_1_3_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_17_01_000003_0, stopRequested=false 2018-01-05T11:28:52,787 INFO [TezTR-518813_1_17_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_17_01_000003_0: Counters: 30 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=3, REDUCE_INPUT_RECORDS=3, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=3, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=7192, ADDITIONAL_SPILLS_BYTES_READ=7192, SHUFFLE_BYTES=7192, SHUFFLE_BYTES_DECOMPRESSED=7188, SHUFFLE_BYTES_TO_MEM=7192, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=47, MERGE_PHASE_TIME=67, FIRST_EVENT_RECEIVED=30, LAST_EVENT_RECEIVED=30][HIVE CREATED_FILES=1, RECORDS_OUT_0=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0]] 2018-01-05T11:28:52,787 DEBUG [TezTR-518813_1_17_1_3_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_17_1_3_0 thread id: 1156 scheme: file bytesRead: 3096 bytesWritten: 9699 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:52,787 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_17_01_000003_0, isTaskDone=true 2018-01-05T11:28:52,787 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000033, requestId=3, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_17_01_000003_0, eventCount=2 } 2018-01-05T11:28:52,788 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_17][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_17_01_000003_0, creationTime=1515180532664, allocationTime=1515180532666, startTime=1515180532669, finishTime=1515180532788, timeTaken=119, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:52,788 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_17_01_000003_0 2018-01-05T11:28:52,788 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_17][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_17_01_000003, startTime=1515180532669, finishTime=1515180532788, timeTaken=119, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_17_01_000003_0, diagnostics=, counters=Counters: 35, File System Counters, FILE_BYTES_READ=3096, FILE_BYTES_WRITTEN=9699, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=3, REDUCE_INPUT_RECORDS=3, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=3, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=7192, ADDITIONAL_SPILLS_BYTES_READ=7192, SHUFFLE_BYTES=7192, SHUFFLE_BYTES_DECOMPRESSED=7188, SHUFFLE_BYTES_TO_MEM=7192, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=47, MERGE_PHASE_TIME=67, FIRST_EVENT_RECEIVED=30, LAST_EVENT_RECEIVED=30, HIVE, CREATED_FILES=1, RECORDS_OUT_0=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0 2018-01-05T11:28:52,788 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_17_01_000003_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:52,788 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_17_01_000003_0; false 2018-01-05T11:28:52,788 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=3, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:28:52,788 INFO [Task-Executor-3] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_17_01_000003_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:52,788 INFO [Task-Executor-3] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_17_01_000003_0 2018-01-05T11:28:52,788 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_17_01 [Reducer 2], tasks=4, failed=0, killed=0, success=3, completed=3, commits=0, err=null 2018-01-05T11:28:52,788 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_17_01_000003_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:52,789 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:52,789 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:52,789 INFO [Task-Executor-3] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000033=118 2018-01-05T11:28:52,789 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_17][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000033, stoppedTime=1515180532789, exitStatus=0 2018-01-05T11:28:52,789 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_17_01_000003_0: true 2018-01-05T11:28:52,789 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000033 exited with diagnostics set to Container completed. 2018-01-05T11:28:52,789 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000033 2018-01-05T11:28:52,789 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_17_01_000003_0 2018-01-05T11:28:52,789 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000033, QueryId=djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab-17, Vertex=Reducer 2, FragmentNum=3, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:28:52,789 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_17_01_000003_0 complete. WaitQueueSize=0, numSlotsAvailable=3, preemptionQueueSize=1 2018-01-05T11:28:52,789 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_17_01_000003_0 2018-01-05T11:28:52,789 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000033, QueryId=djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab-17, DagId=17, VertexName=Reducer 2, TaskId=3, TaskAttemptId=0, ThreadName=Task-Executor-3, Succeeded=true, StartTime=1515180532670, EndTime=1515180532789 2018-01-05T11:28:52,796 INFO [TezTR-518813_1_17_1_1_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:28:52,796 DEBUG [TezTR-518813_1_17_1_1_0] exec.GroupByOperator: Closing child = SEL[5] 2018-01-05T11:28:52,796 DEBUG [TezTR-518813_1_17_1_1_0] exec.SelectOperator: close called for operator SEL[5] 2018-01-05T11:28:52,796 DEBUG [TezTR-518813_1_17_1_1_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:52,796 INFO [TezTR-518813_1_17_1_1_0] exec.SelectOperator: Closing operator SEL[5] 2018-01-05T11:28:52,796 DEBUG [TezTR-518813_1_17_1_1_0] exec.SelectOperator: Closing child = FS[6] 2018-01-05T11:28:52,796 DEBUG [TezTR-518813_1_17_1_1_0] exec.FileSinkOperator: close called for operator FS[6] 2018-01-05T11:28:52,796 DEBUG [TezTR-518813_1_17_1_1_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:52,796 INFO [TezTR-518813_1_17_1_1_0] exec.FileSinkOperator: Closing operator FS[6] 2018-01-05T11:28:52,796 INFO [TezTR-518813_1_17_1_1_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:28:52,797 INFO [TezTR-518813_1_17_1_1_0] exec.FileSinkOperator: RECORDS_OUT_0:1, 2018-01-05T11:28:52,797 DEBUG [TezTR-518813_1_17_1_1_0] exec.FileSinkOperator: 6 Close done 2018-01-05T11:28:52,797 DEBUG [TezTR-518813_1_17_1_1_0] exec.SelectOperator: 5 Close done 2018-01-05T11:28:52,797 DEBUG [TezTR-518813_1_17_1_1_0] exec.GroupByOperator: 4 Close done 2018-01-05T11:28:52,797 INFO [TezTR-518813_1_17_1_1_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_17_01_000001_0 2018-01-05T11:28:52,797 INFO [TezTR-518813_1_17_1_1_0] orderedgrouped.Shuffle: Shutting down Shuffle for source: Map_1 2018-01-05T11:28:52,797 INFO [TezTR-518813_1_17_1_1_0] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:28:52,797 INFO [TezTR-518813_1_17_1_1_0] output.MROutput: out_Reducer 2 closed 2018-01-05T11:28:52,797 DEBUG [TezTR-518813_1_17_1_1_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:52,797 INFO [TezTR-518813_1_17_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:52,797 INFO [TezTR-518813_1_17_1_1_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_17_01_000001_0, askedToStop=false 2018-01-05T11:28:52,797 INFO [TezTR-518813_1_17_1_1_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_17_01_000001_0, stopRequested=false 2018-01-05T11:28:52,797 INFO [TezTR-518813_1_17_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_17_01_000001_0: Counters: 30 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=1, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=2404, ADDITIONAL_SPILLS_BYTES_READ=2404, SHUFFLE_BYTES=2404, SHUFFLE_BYTES_DECOMPRESSED=2400, SHUFFLE_BYTES_TO_MEM=2404, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=48, MERGE_PHASE_TIME=67, FIRST_EVENT_RECEIVED=30, LAST_EVENT_RECEIVED=30][HIVE CREATED_FILES=1, RECORDS_OUT_0=1][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0]] 2018-01-05T11:28:52,797 DEBUG [TezTR-518813_1_17_1_1_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_17_1_1_0 thread id: 1150 scheme: file bytesRead: 0 bytesWritten: 3299 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:52,797 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_17_01_000001_0, isTaskDone=true 2018-01-05T11:28:52,798 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000032, requestId=3, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_17_01_000001_0, eventCount=2 } 2018-01-05T11:28:52,798 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_17][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_17_01_000001_0, creationTime=1515180532664, allocationTime=1515180532665, startTime=1515180532668, finishTime=1515180532798, timeTaken=130, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:52,798 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_17_01_000001_0 2018-01-05T11:28:52,798 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_17_01_000001_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:52,798 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=3, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:28:52,798 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_17_01_000001_0; false 2018-01-05T11:28:52,798 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_17_01_000001_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:52,798 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_17_01_000001_0 2018-01-05T11:28:52,798 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_17_01_000001_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:52,798 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:52,798 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_17][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_17_01_000001, startTime=1515180532668, finishTime=1515180532798, timeTaken=130, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_17_01_000001_0, diagnostics=, counters=Counters: 35, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=3299, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=1, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=2404, ADDITIONAL_SPILLS_BYTES_READ=2404, SHUFFLE_BYTES=2404, SHUFFLE_BYTES_DECOMPRESSED=2400, SHUFFLE_BYTES_TO_MEM=2404, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=48, MERGE_PHASE_TIME=67, FIRST_EVENT_RECEIVED=30, LAST_EVENT_RECEIVED=30, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0 2018-01-05T11:28:52,799 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:52,799 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000032=130 2018-01-05T11:28:52,799 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_17_01_000001_0: true 2018-01-05T11:28:52,799 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_17_01 [Reducer 2], tasks=4, failed=0, killed=0, success=4, completed=4, commits=0, err=null 2018-01-05T11:28:52,799 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_17_01 [Reducer 2] 2018-01-05T11:28:52,799 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_17_01_000001_0 2018-01-05T11:28:52,799 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000032, QueryId=djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab-17, Vertex=Reducer 2, FragmentNum=1, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:28:52,799 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_17_01_000001_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:52,799 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_17_01_000001_0 2018-01-05T11:28:52,800 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_17][Event:VERTEX_FINISHED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_17_01, initRequestedTime=1515180532482, initedTime=1515180532487, startRequestedTime=1515180532543, startedTime=1515180532543, finishTime=1515180532799, timeTaken=256, status=SUCCEEDED, diagnostics=, counters=Counters: 35, File System Counters, FILE_BYTES_READ=3096, FILE_BYTES_WRITTEN=12998, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=4, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=4, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=2, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=9596, ADDITIONAL_SPILLS_BYTES_READ=9596, SHUFFLE_BYTES=9596, SHUFFLE_BYTES_DECOMPRESSED=9588, SHUFFLE_BYTES_TO_MEM=9596, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=147, MERGE_PHASE_TIME=210, FIRST_EVENT_RECEIVED=108, LAST_EVENT_RECEIVED=108, HIVE, CREATED_FILES=2, RECORDS_OUT_0=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, vertexStats=firstTaskStartTime=1515180532668, firstTasksToStart=[ task_1515180518813_0001_17_01_000001 ], lastTaskFinishTime=1515180532798, lastTasksToFinish=[ task_1515180518813_0001_17_01_000001 ], minTaskDuration=69, maxTaskDuration=130, avgTaskDuration=97.0, numSuccessfulTasks=4, shortestDurationTasks=[ task_1515180518813_0001_17_01_000002 ], longestDurationTasks=[ task_1515180518813_0001_17_01_000001 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=4, numSucceededTasks=4, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:52,800 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_17_01 [Reducer 2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:52,800 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_17][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000032, stoppedTime=1515180532800, exitStatus=0 2018-01-05T11:28:52,800 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000032 exited with diagnostics set to Container completed. 2018-01-05T11:28:52,800 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_17_01 [Reducer 2] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2 2018-01-05T11:28:52,800 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null 2018-01-05T11:28:52,800 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_17_01 [Reducer 2] 2018-01-05T11:28:52,800 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_17_00 [Map 1] 2018-01-05T11:28:52,801 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_17, queueSize=0 2018-01-05T11:28:52,801 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_17][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_17, startTime=1515180532474, finishTime=1515180532800, timeTaken=326, status=SUCCEEDED, diagnostics=, counters=Counters: 51, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=5, TOTAL_LAUNCHED_TASKS=5, OTHER_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=676, AM_GC_TIME_MILLIS=0, File System Counters, FILE_BYTES_READ=26792, FILE_BYTES_WRITTEN=22975, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=4, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=8, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=2, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, INPUT_RECORDS_PROCESSED=2000, INPUT_SPLIT_LENGTH_BYTES=23248, OUTPUT_RECORDS=4, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=9560, OUTPUT_BYTES_WITH_OVERHEAD=9588, OUTPUT_BYTES_PHYSICAL=9596, ADDITIONAL_SPILLS_BYTES_WRITTEN=9596, ADDITIONAL_SPILLS_BYTES_READ=9596, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=9596, SHUFFLE_BYTES_DECOMPRESSED=9588, SHUFFLE_BYTES_TO_MEM=9596, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=147, MERGE_PHASE_TIME=210, FIRST_EVENT_RECEIVED=108, LAST_EVENT_RECEIVED=108, HIVE, CREATED_FILES=2, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=2000, RECORDS_OUT_0=4, RECORDS_OUT_INTERMEDIATE_Map_1=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0 2018-01-05T11:28:52,801 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_17 finished with state: SUCCEEDED 2018-01-05T11:28:52,801 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_17 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:28:52,801 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000032 2018-01-05T11:28:52,801 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_17, dagState=SUCCEEDED 2018-01-05T11:28:52,801 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_17_post 2018-01-05T11:28:52,802 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:52,802 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:28:52,802 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:28:52,802 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 17 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=5, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=5,NumRequestsWithoutLocation=5, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=5} 2018-01-05T11:28:52,802 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:28:52,802 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=ANALYZE TABLE srcpart PARTITION(...key,value (Stage-0), with id=dag_1515180518813_0001_17 2018-01-05T11:28:52,802 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 17, to 10.22.8.182:49818 2018-01-05T11:28:52,802 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=5, Remaining=0 2018-01-05T11:28:52,802 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=ANALYZE TABLE srcpart PARTITION(...key,value (Stage-0), with id=dag_1515180518813_0001_17 2018-01-05T11:28:52,802 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=17} 2018-01-05T11:28:52,802 INFO [IPC Server handler 3 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=17}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:28:52,802 INFO [IPC Server handler 3 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/17 after 300 seconds 2018-01-05T11:28:52,802 ERROR [IPC Server handler 3 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab, dag_1515180518813_0001_17 2018-01-05T11:28:52,802 INFO [IPC Server handler 3 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab 2018-01-05T11:28:52,802 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=17} = 0 2018-01-05T11:28:52,802 DEBUG [IPC Server handler 3 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=17} 2018-01-05T11:28:52,802 DEBUG [IPC Server handler 3 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:28:52,802 DEBUG [IPC Server handler 3 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=17} 2018-01-05T11:28:52,803 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,803 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 Reducer 2: 4/4 2018-01-05T11:28:52,803 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,805 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,805 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: TaskId for 000001_0 = 000001 2018-01-05T11:28:52,805 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: TaskId for 000003_0 = 000003 2018-01-05T11:28:52,805 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,805 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,805 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,806 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-2:STATS] in serial mode 2018-01-05T11:28:52,806 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@fc7308, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:52,806 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:52,806 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,806 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:52,806 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:52,806 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:52,806 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:52,806 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,806 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,806 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:52,806 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:52,806 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,806 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:52,806 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,806 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:52,806 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:52,806 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,806 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:52,806 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_151_4467563815339635813-1/-mr-10001 2018-01-05T11:28:52,806 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_151_4467563815339635813-1/-mr-10001 2018-01-05T11:28:52,807 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Getting file stats of all partitions. threadpool size:15 2018-01-05T11:28:52,808 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Finished getting file stats of all partitions! 2018-01-05T11:28:52,808 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.srcpart/ds=2008-04-08/hr=11/ numRows 2018-01-05T11:28:52,808 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.srcpart/ds=2008-04-08/hr=11/ numRows 500 2018-01-05T11:28:52,808 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.srcpart/ds=2008-04-08/hr=11/ rawDataSize 2018-01-05T11:28:52,808 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.srcpart/ds=2008-04-08/hr=11/ rawDataSize 5312 2018-01-05T11:28:52,808 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Partition {ds=2008-04-08, hr=11} stats: [numFiles=1, numRows=500, totalSize=5812, rawDataSize=5312] 2018-01-05T11:28:52,808 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.srcpart/ds=2008-04-08/hr=12/ numRows 2018-01-05T11:28:52,808 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.srcpart/ds=2008-04-08/hr=12/ numRows 500 2018-01-05T11:28:52,808 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.srcpart/ds=2008-04-08/hr=12/ rawDataSize 2018-01-05T11:28:52,808 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.srcpart/ds=2008-04-08/hr=12/ rawDataSize 5312 2018-01-05T11:28:52,809 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Partition {ds=2008-04-08, hr=12} stats: [numFiles=1, numRows=500, totalSize=5812, rawDataSize=5312] 2018-01-05T11:28:52,809 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.srcpart/ds=2008-04-09/hr=11/ numRows 2018-01-05T11:28:52,809 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.srcpart/ds=2008-04-09/hr=11/ numRows 500 2018-01-05T11:28:52,809 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.srcpart/ds=2008-04-09/hr=11/ rawDataSize 2018-01-05T11:28:52,809 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.srcpart/ds=2008-04-09/hr=11/ rawDataSize 5312 2018-01-05T11:28:52,809 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Partition {ds=2008-04-09, hr=11} stats: [numFiles=1, numRows=500, totalSize=5812, rawDataSize=5312] 2018-01-05T11:28:52,809 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.srcpart/ds=2008-04-09/hr=12/ numRows 2018-01-05T11:28:52,809 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.srcpart/ds=2008-04-09/hr=12/ numRows 500 2018-01-05T11:28:52,809 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.srcpart/ds=2008-04-09/hr=12/ rawDataSize 2018-01-05T11:28:52,809 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.srcpart/ds=2008-04-09/hr=12/ rawDataSize 5312 2018-01-05T11:28:52,809 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Partition {ds=2008-04-09, hr=12} stats: [numFiles=1, numRows=500, totalSize=5812, rawDataSize=5312] 2018-01-05T11:28:52,809 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,810 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:52,810 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:52,810 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:52,810 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:52,810 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,810 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,810 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,811 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,812 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,812 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:52,812 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:52,812 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,812 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:52,812 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,812 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_partitions : db=default tbl=srcpart 2018-01-05T11:28:52,812 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_partitions : db=default tbl=srcpart 2018-01-05T11:28:52,812 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: New partition values:[2008-04-08, 11] 2018-01-05T11:28:52,812 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: New partition values:[2008-04-08, 12] 2018-01-05T11:28:52,812 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: New partition values:[2008-04-09, 11] 2018-01-05T11:28:52,812 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: New partition values:[2008-04-09, 12] 2018-01-05T11:28:52,816 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,829 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,833 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,834 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,834 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: About to delete stats tmp dir :file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_151_4467563815339635813-1/-mr-10001 2018-01-05T11:28:52,864 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Time taken to get FileStatuses: 29 2018-01-05T11:28:52,864 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Total input files to process : 2 2018-01-05T11:28:52,865 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Total # of splits generated by getSplits: 2, TimeTaken: 29 2018-01-05T11:28:52,865 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FetchOperator: Creating fetchTask with deserializer typeinfo: struct<_col0:struct,_col1:struct,_col2:string,_col3:string> 2018-01-05T11:28:52,865 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FetchOperator: deserializer properties: table properties: {columns=_col0,_col1,_col2,_col3, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct:struct:string:string, escape.delim=\} partition properties: {columns=_col0,_col1,_col2,_col3, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct:struct:string:string, escape.delim=\} 2018-01-05T11:28:52,867 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,870 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: write_partition_column_statistics: db=default table=srcpart part=ds=2008-04-09/hr=11 2018-01-05T11:28:52,870 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=write_partition_column_statistics: db=default table=srcpart part=ds=2008-04-09/hr=11 2018-01-05T11:28:52,879 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating partition level column statistics for db=default tableName=srcpart partName=ds=2008-04-09/hr=11 colName=key 2018-01-05T11:28:52,888 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,890 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating partition level column statistics for db=default tableName=srcpart partName=ds=2008-04-09/hr=11 colName=value 2018-01-05T11:28:52,893 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,893 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: write_partition_column_statistics: db=default table=srcpart part=ds=2008-04-09/hr=12 2018-01-05T11:28:52,894 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=write_partition_column_statistics: db=default table=srcpart part=ds=2008-04-09/hr=12 2018-01-05T11:28:52,898 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating partition level column statistics for db=default tableName=srcpart partName=ds=2008-04-09/hr=12 colName=key 2018-01-05T11:28:52,899 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating partition level column statistics for db=default tableName=srcpart partName=ds=2008-04-09/hr=12 colName=value 2018-01-05T11:28:52,901 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,902 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000032, QueryId=djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab-17, DagId=17, VertexName=Reducer 2, TaskId=1, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180532668, EndTime=1515180532902 2018-01-05T11:28:52,902 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: write_partition_column_statistics: db=default table=srcpart part=ds=2008-04-08/hr=11 2018-01-05T11:28:52,902 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=write_partition_column_statistics: db=default table=srcpart part=ds=2008-04-08/hr=11 2018-01-05T11:28:52,908 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating partition level column statistics for db=default tableName=srcpart partName=ds=2008-04-08/hr=11 colName=key 2018-01-05T11:28:52,908 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating partition level column statistics for db=default tableName=srcpart partName=ds=2008-04-08/hr=11 colName=value 2018-01-05T11:28:52,909 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,912 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,913 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: write_partition_column_statistics: db=default table=srcpart part=ds=2008-04-08/hr=12 2018-01-05T11:28:52,913 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=write_partition_column_statistics: db=default table=srcpart part=ds=2008-04-08/hr=12 2018-01-05T11:28:52,918 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating partition level column statistics for db=default tableName=srcpart partName=ds=2008-04-08/hr=12 colName=key 2018-01-05T11:28:52,919 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating partition level column statistics for db=default tableName=srcpart partName=ds=2008-04-08/hr=12 colName=value 2018-01-05T11:28:52,922 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,923 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,923 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,923 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,923 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: ANALYZE TABLE srcpart PARTITION(ds, hr) COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:52,923 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:28:52,923 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@srcpart 2018-01-05T11:28:52,923 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@srcpart@ds=2008-04-08/hr=11 2018-01-05T11:28:52,923 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@srcpart@ds=2008-04-08/hr=12 2018-01-05T11:28:52,923 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=11 2018-01-05T11:28:52,923 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=12 2018-01-05T11:28:52,923 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@srcpart 2018-01-05T11:28:52,923 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@srcpart@ds=2008-04-08/hr=11 2018-01-05T11:28:52,923 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@srcpart@ds=2008-04-08/hr=12 2018-01-05T11:28:52,923 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@srcpart@ds=2008-04-09/hr=11 2018-01-05T11:28:52,923 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@srcpart@ds=2008-04-09/hr=12 2018-01-05T11:28:52,923 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_151_4467563815339635813-1/-mr-10000 2018-01-05T11:28:52,923 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,923 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,923 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:52,923 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_partitions_(String, String, List, EnvironmentContext, )=22, setPartitionColumnStatistics_(SetPartitionsStatsRequest, )=56} 2018-01-05T11:28:52,923 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112852_0fabd7e5-8aa7-49cf-b829-ce2670b2aaab); Time taken: 0.61 seconds 2018-01-05T11:28:52,923 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:52,923 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,923 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:52,923 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,923 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query ANALYZE TABLE srcpart PARTITION(ds, hr) COMPUTE STATISTICS FOR COLUMNS key,value 2018-01-05T11:28:52,923 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-52_147_6959340210354157653-1 2018-01-05T11:28:52,924 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.776 seconds 2018-01-05T11:28:52,924 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:52,924 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:52,924 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:52,924 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:52,924 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,924 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:52,924 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,924 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,924 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,924 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: -- -- Table alltypesorc -- DROP TABLE IF EXISTS alltypesorc 2018-01-05T11:28:52,924 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112852_b5d2ff9f-8e2f-4790-9c22-1ed05ea7b59c): -- -- Table alltypesorc -- DROP TABLE IF EXISTS alltypesorc 2018-01-05T11:28:52,924 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,924 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: -- -- Table alltypesorc -- DROP TABLE IF EXISTS alltypesorc 2018-01-05T11:28:52,925 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:52,925 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,925 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,925 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,925 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,925 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:52,925 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,925 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesorc 2018-01-05T11:28:52,925 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesorc 2018-01-05T11:28:52,925 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,925 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,925 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:52,925 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,925 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:52,925 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,925 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:52,925 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:28:52,925 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112852_b5d2ff9f-8e2f-4790-9c22-1ed05ea7b59c); Time taken: 0.001 seconds 2018-01-05T11:28:52,925 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:52,925 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,925 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112852_b5d2ff9f-8e2f-4790-9c22-1ed05ea7b59c): -- -- Table alltypesorc -- DROP TABLE IF EXISTS alltypesorc 2018-01-05T11:28:52,927 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,927 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: -- -- Table alltypesorc -- DROP TABLE IF EXISTS alltypesorc 2018-01-05T11:28:52,927 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:52,927 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,927 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,927 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,927 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,927 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:52,927 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,927 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesorc 2018-01-05T11:28:52,927 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesorc 2018-01-05T11:28:52,927 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,927 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,927 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table alltypesorc not found: default.alltypesorc table not found 2018-01-05T11:28:52,927 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,928 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesorc 2018-01-05T11:28:52,928 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesorc 2018-01-05T11:28:52,928 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,928 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,928 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,928 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,928 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: -- -- Table alltypesorc -- DROP TABLE IF EXISTS alltypesorc 2018-01-05T11:28:52,928 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:52,928 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,928 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,928 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:52,928 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=1} 2018-01-05T11:28:52,928 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112852_b5d2ff9f-8e2f-4790-9c22-1ed05ea7b59c); Time taken: 0.003 seconds 2018-01-05T11:28:52,928 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:52,928 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,928 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:52,928 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,928 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query -- -- Table alltypesorc -- DROP TABLE IF EXISTS alltypesorc 2018-01-05T11:28:52,928 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.004 seconds 2018-01-05T11:28:52,928 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:52,928 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:52,928 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:52,928 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:52,928 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,928 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:52,928 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,928 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,928 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,928 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: CREATE TABLE alltypesorc( ctinyint TINYINT, csmallint SMALLINT, cint INT, cbigint BIGINT, cfloat FLOAT, cdouble DOUBLE, cstring1 STRING, cstring2 STRING, ctimestamp1 TIMESTAMP, ctimestamp2 TIMESTAMP, cboolean1 BOOLEAN, cboolean2 BOOLEAN) STORED AS ORC 2018-01-05T11:28:52,929 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112852_dc26de6a-55ad-413b-a1bd-bbe6d3a2fb43): CREATE TABLE alltypesorc( ctinyint TINYINT, csmallint SMALLINT, cint INT, cbigint BIGINT, cfloat FLOAT, cdouble DOUBLE, cstring1 STRING, cstring2 STRING, ctimestamp1 TIMESTAMP, ctimestamp2 TIMESTAMP, cboolean1 BOOLEAN, cboolean2 BOOLEAN) STORED AS ORC 2018-01-05T11:28:52,929 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,929 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: CREATE TABLE alltypesorc( ctinyint TINYINT, csmallint SMALLINT, cint INT, cbigint BIGINT, cfloat FLOAT, cdouble DOUBLE, cstring1 STRING, cstring2 STRING, ctimestamp1 TIMESTAMP, ctimestamp2 TIMESTAMP, cboolean1 BOOLEAN, cboolean2 BOOLEAN) STORED AS ORC 2018-01-05T11:28:52,929 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:52,930 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,930 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,930 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,930 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,930 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:52,930 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:28:52,930 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Creating table default.alltypesorc position=13 2018-01-05T11:28:52,934 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,934 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_database: default 2018-01-05T11:28:52,934 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_database: default 2018-01-05T11:28:52,934 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:52,934 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,934 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,934 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:52,934 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: validation start 2018-01-05T11:28:52,934 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:28:52,934 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Not a partition. 2018-01-05T11:28:52,934 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,934 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:52,935 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,935 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:52,935 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getDatabase_(String, )=0, flushCache_()=0} 2018-01-05T11:28:52,935 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112852_dc26de6a-55ad-413b-a1bd-bbe6d3a2fb43); Time taken: 0.007 seconds 2018-01-05T11:28:52,935 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:52,935 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,935 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112852_dc26de6a-55ad-413b-a1bd-bbe6d3a2fb43): CREATE TABLE alltypesorc( ctinyint TINYINT, csmallint SMALLINT, cint INT, cbigint BIGINT, cfloat FLOAT, cdouble DOUBLE, cstring1 STRING, cstring2 STRING, ctimestamp1 TIMESTAMP, ctimestamp2 TIMESTAMP, cboolean1 BOOLEAN, cboolean2 BOOLEAN) STORED AS ORC 2018-01-05T11:28:52,936 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,936 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: CREATE TABLE alltypesorc( ctinyint TINYINT, csmallint SMALLINT, cint INT, cbigint BIGINT, cfloat FLOAT, cdouble DOUBLE, cstring1 STRING, cstring2 STRING, ctimestamp1 TIMESTAMP, ctimestamp2 TIMESTAMP, cboolean1 BOOLEAN, cboolean2 BOOLEAN) STORED AS ORC 2018-01-05T11:28:52,936 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: CREATETABLE 2018-01-05T11:28:52,936 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: database:default 2018-01-05T11:28:52,936 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@alltypesorc 2018-01-05T11:28:52,936 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,936 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,936 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,936 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,936 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:52,937 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Found class for org.apache.hadoop.hive.ql.io.orc.OrcSerde 2018-01-05T11:28:52,937 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: creating table default.alltypesorc on file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc 2018-01-05T11:28:52,940 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,940 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: create_table: Table(tableName:alltypesorc, dbName:default, owner:hive_test_user, createTime:1515180532, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:ctinyint, type:tinyint, comment:null), FieldSchema(name:csmallint, type:smallint, comment:null), FieldSchema(name:cint, type:int, comment:null), FieldSchema(name:cbigint, type:bigint, comment:null), FieldSchema(name:cfloat, type:float, comment:null), FieldSchema(name:cdouble, type:double, comment:null), FieldSchema(name:cstring1, type:string, comment:null), FieldSchema(name:cstring2, type:string, comment:null), FieldSchema(name:ctimestamp1, type:timestamp, comment:null), FieldSchema(name:ctimestamp2, type:timestamp, comment:null), FieldSchema(name:cboolean1, type:boolean, comment:null), FieldSchema(name:cboolean2, type:boolean, comment:null)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc, inputFormat:org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.ql.io.orc.OrcSerde, parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"cbigint":"true","cboolean1":"true","cboolean2":"true","cdouble":"true","cfloat":"true","cint":"true","csmallint":"true","cstring1":"true","cstring2":"true","ctimestamp1":"true","ctimestamp2":"true","ctinyint":"true"}}, numFiles=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:52,940 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=create_table: Table(tableName:alltypesorc, dbName:default, owner:hive_test_user, createTime:1515180532, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:ctinyint, type:tinyint, comment:null), FieldSchema(name:csmallint, type:smallint, comment:null), FieldSchema(name:cint, type:int, comment:null), FieldSchema(name:cbigint, type:bigint, comment:null), FieldSchema(name:cfloat, type:float, comment:null), FieldSchema(name:cdouble, type:double, comment:null), FieldSchema(name:cstring1, type:string, comment:null), FieldSchema(name:cstring2, type:string, comment:null), FieldSchema(name:ctimestamp1, type:timestamp, comment:null), FieldSchema(name:ctimestamp2, type:timestamp, comment:null), FieldSchema(name:cboolean1, type:boolean, comment:null), FieldSchema(name:cboolean2, type:boolean, comment:null)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc, inputFormat:org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.ql.io.orc.OrcSerde, parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"cbigint":"true","cboolean1":"true","cboolean2":"true","cdouble":"true","cfloat":"true","cint":"true","csmallint":"true","cstring1":"true","cstring2":"true","ctimestamp1":"true","ctimestamp2":"true","ctinyint":"true"}}, numFiles=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:52,941 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hive-site.xml 2018-01-05T11:28:52,941 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hivemetastore-site.xml 2018-01-05T11:28:52,942 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Unable to find config file metastore-site.xml 2018-01-05T11:28:52,942 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file null 2018-01-05T11:28:52,950 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:52,951 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: Location: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc specified for non-external table:alltypesorc 2018-01-05T11:28:52,951 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc 2018-01-05T11:28:52,965 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,967 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,970 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,973 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,973 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,973 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@alltypesorc) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@alltypesorc) Type=TABLE WriteType=DDL_NO_LOCK is present 2018-01-05T11:28:52,973 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,973 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,973 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: CREATE TABLE alltypesorc( ctinyint TINYINT, csmallint SMALLINT, cint INT, cbigint BIGINT, cfloat FLOAT, cdouble DOUBLE, cstring1 STRING, cstring2 STRING, ctimestamp1 TIMESTAMP, ctimestamp2 TIMESTAMP, cboolean1 BOOLEAN, cboolean2 BOOLEAN) STORED AS ORC 2018-01-05T11:28:52,973 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: CREATETABLE 2018-01-05T11:28:52,973 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: database:default 2018-01-05T11:28:52,973 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@alltypesorc 2018-01-05T11:28:52,973 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,973 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,973 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:52,973 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {createTable_(Table, )=33, isCompatibleWith_(Configuration, )=0} 2018-01-05T11:28:52,973 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112852_dc26de6a-55ad-413b-a1bd-bbe6d3a2fb43); Time taken: 0.038 seconds 2018-01-05T11:28:52,974 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:52,974 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,974 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:52,974 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,974 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query CREATE TABLE alltypesorc( ctinyint TINYINT, csmallint SMALLINT, cint INT, cbigint BIGINT, cfloat FLOAT, cdouble DOUBLE, cstring1 STRING, cstring2 STRING, ctimestamp1 TIMESTAMP, ctimestamp2 TIMESTAMP, cboolean1 BOOLEAN, cboolean2 BOOLEAN) STORED AS ORC 2018-01-05T11:28:52,974 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.046 seconds 2018-01-05T11:28:52,974 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:52,974 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:52,974 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:52,974 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:52,974 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,974 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:52,974 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,974 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,974 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,974 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/alltypesorc" OVERWRITE INTO TABLE alltypesorc 2018-01-05T11:28:52,974 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112852_3e813e2d-9ea0-492c-b02e-323bf4f69ab4): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/alltypesorc" OVERWRITE INTO TABLE alltypesorc 2018-01-05T11:28:52,974 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,974 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/alltypesorc" OVERWRITE INTO TABLE alltypesorc 2018-01-05T11:28:52,974 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:52,974 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,974 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:52,974 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,974 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,974 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:52,974 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.LoadSemanticAnalyzer: file@null@/Users/djaiswal/hive/hive/data/files/alltypesorc 2018-01-05T11:28:52,974 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,974 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesorc 2018-01-05T11:28:52,974 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesorc 2018-01-05T11:28:52,977 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:52,978 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:52,982 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] impl.OrcCodecPool: Got brand-new codec ZLIB 2018-01-05T11:28:53,000 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:28:53,003 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:53,003 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,003 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:53,003 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,003 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:53,003 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=4, flushCache_()=0} 2018-01-05T11:28:53,003 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112852_3e813e2d-9ea0-492c-b02e-323bf4f69ab4); Time taken: 0.029 seconds 2018-01-05T11:28:53,003 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:53,003 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,003 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112852_3e813e2d-9ea0-492c-b02e-323bf4f69ab4): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/alltypesorc" OVERWRITE INTO TABLE alltypesorc 2018-01-05T11:28:53,004 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,004 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/alltypesorc" OVERWRITE INTO TABLE alltypesorc 2018-01-05T11:28:53,004 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: LOAD 2018-01-05T11:28:53,005 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/alltypesorc 2018-01-05T11:28:53,005 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@alltypesorc 2018-01-05T11:28:53,005 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,005 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,005 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,005 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,005 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MOVE] in serial mode 2018-01-05T11:28:53,005 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@4404dee7, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:53,005 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:53,005 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,005 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:53,005 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:53,005 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:53,005 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:53,005 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,005 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:53,005 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Loading data to table default.alltypesorc from file:/Users/djaiswal/hive/hive/data/files/alltypesorc 2018-01-05T11:28:53,005 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,006 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:53,006 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:53,006 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:53,006 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:53,006 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,006 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,006 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,007 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,007 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,007 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:53,007 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:53,008 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,008 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:53,008 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,008 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesorc 2018-01-05T11:28:53,008 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesorc 2018-01-05T11:28:53,011 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,011 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,012 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:28:53,012 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:28:53,012 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,012 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesorc 2018-01-05T11:28:53,012 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesorc 2018-01-05T11:28:53,015 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,016 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,016 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] FileOperations: moving file:/Users/djaiswal/hive/hive/data/files/alltypesorc to file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc (replace = REPLACE_ALL) 2018-01-05T11:28:53,016 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] FileOperations: Deleting old paths for replace in file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc and old path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc 2018-01-05T11:28:53,016 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc/ and the destination path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc/ 2018-01-05T11:28:53,017 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc 2018-01-05T11:28:53,017 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/data/files/alltypesorc/ and the destination path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc/alltypesorc/ 2018-01-05T11:28:53,017 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc/alltypesorc/ and the destination path is /Users/djaiswal/hive/hive/data/files/alltypesorc/ 2018-01-05T11:28:53,046 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,046 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=alltypesorc newtbl=alltypesorc 2018-01-05T11:28:53,046 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=alltypesorc newtbl=alltypesorc 2018-01-05T11:28:53,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,058 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,058 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,058 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@alltypesorc) Type=TABLE WriteType=INSERT_OVERWRITE because WriteEntity(default@alltypesorc) Type=TABLE WriteType=INSERT_OVERWRITE is present 2018-01-05T11:28:53,058 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-1:STATS] in serial mode 2018-01-05T11:28:53,058 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@55c596aa, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:53,058 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:53,058 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,058 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:53,058 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:53,058 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:53,058 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:53,058 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,058 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:53,059 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:53,059 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,059 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:53,059 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:53,059 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:53,059 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:53,059 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,060 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,060 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,060 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,060 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,060 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:53,060 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:53,060 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,061 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:53,061 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,061 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesorc 2018-01-05T11:28:53,061 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesorc 2018-01-05T11:28:53,063 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,064 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,065 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=alltypesorc newtbl=alltypesorc 2018-01-05T11:28:53,065 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=alltypesorc newtbl=alltypesorc 2018-01-05T11:28:53,067 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,074 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,074 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,074 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.alltypesorc stats: [numFiles=1, numRows=0, totalSize=377237, rawDataSize=0] 2018-01-05T11:28:53,074 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,074 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,074 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/alltypesorc" OVERWRITE INTO TABLE alltypesorc 2018-01-05T11:28:53,074 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: LOAD 2018-01-05T11:28:53,074 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/alltypesorc 2018-01-05T11:28:53,074 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@alltypesorc 2018-01-05T11:28:53,074 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,074 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,074 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:53,074 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=9, getTable_(String, String, )=3} 2018-01-05T11:28:53,074 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112852_3e813e2d-9ea0-492c-b02e-323bf4f69ab4); Time taken: 0.071 seconds 2018-01-05T11:28:53,074 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:53,074 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,074 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:53,074 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,074 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/alltypesorc" OVERWRITE INTO TABLE alltypesorc 2018-01-05T11:28:53,074 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.1 seconds 2018-01-05T11:28:53,074 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:53,075 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:53,075 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:53,075 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:53,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:53,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: ANALYZE TABLE alltypesorc COMPUTE STATISTICS 2018-01-05T11:28:53,075 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112853_a8cdfcc6-2ad4-4c71-94a5-276ccef4eca9): ANALYZE TABLE alltypesorc COMPUTE STATISTICS 2018-01-05T11:28:53,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: ANALYZE TABLE alltypesorc COMPUTE STATISTICS 2018-01-05T11:28:53,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:53,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,075 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:53,075 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Invoking analyze on original query 2018-01-05T11:28:53,075 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Starting Semantic Analysis 2018-01-05T11:28:53,075 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed phase 1 of Semantic Analysis 2018-01-05T11:28:53,075 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for source tables 2018-01-05T11:28:53,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,075 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesorc 2018-01-05T11:28:53,075 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesorc 2018-01-05T11:28:53,078 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,078 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,078 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,078 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesorc 2018-01-05T11:28:53,078 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesorc 2018-01-05T11:28:53,081 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,082 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,082 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for subqueries 2018-01-05T11:28:53,082 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for destination tables 2018-01-05T11:28:53,082 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed getting MetaData in Semantic Analysis 2018-01-05T11:28:53,093 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_075_8421955314026672262-1/-mr-10000 2018-01-05T11:28:53,093 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Table Plan for alltypesorc TS[0] 2018-01-05T11:28:53,093 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Body Plan for Query Block null 2018-01-05T11:28:53,093 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Plan for Query Block null 2018-01-05T11:28:53,093 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Before logical optimization TS[0] 2018-01-05T11:28:53,093 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,093 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,093 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] lineage.Generator: Time taken for lineage transform=1 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(ctinyint: tinyint|{alltypesorc}ctinyint,csmallint: smallint|{alltypesorc}csmallint,cint: int|{alltypesorc}cint,cbigint: bigint|{alltypesorc}cbigint,cfloat: float|{alltypesorc}cfloat,cdouble: double|{alltypesorc}cdouble,cstring1: string|{alltypesorc}cstring1,cstring2: string|{alltypesorc}cstring2,ctimestamp1: timestamp|{alltypesorc}ctimestamp1,ctimestamp2: timestamp|{alltypesorc}ctimestamp2,cboolean1: boolean|{alltypesorc}cboolean1,cboolean2: boolean|{alltypesorc}cboolean2,BLOCK__OFFSET__INSIDE__FILE: bigint|{alltypesorc}block__offset__inside__file,INPUT__FILE__NAME: string|{alltypesorc}input__file__name,ROW__ID: struct|{alltypesorc}row__id,RAW__DATA__SIZE: bigint|{alltypesorc}raw__data__size) 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.PredicatePushDown: After PPD: TS[0] 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(ctinyint: tinyint|{alltypesorc}ctinyint,csmallint: smallint|{alltypesorc}csmallint,cint: int|{alltypesorc}cint,cbigint: bigint|{alltypesorc}cbigint,cfloat: float|{alltypesorc}cfloat,cdouble: double|{alltypesorc}cdouble,cstring1: string|{alltypesorc}cstring1,cstring2: string|{alltypesorc}cstring2,ctimestamp1: timestamp|{alltypesorc}ctimestamp1,ctimestamp2: timestamp|{alltypesorc}ctimestamp2,cboolean1: boolean|{alltypesorc}cboolean1,cboolean2: boolean|{alltypesorc}cboolean2,BLOCK__OFFSET__INSIDE__FILE: bigint|{alltypesorc}block__offset__inside__file,INPUT__FILE__NAME: string|{alltypesorc}input__file__name,ROW__ID: struct|{alltypesorc}row__id,RAW__DATA__SIZE: bigint|{alltypesorc}raw__data__size) 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(ctinyint: tinyint|{alltypesorc}ctinyint,csmallint: smallint|{alltypesorc}csmallint,cint: int|{alltypesorc}cint,cbigint: bigint|{alltypesorc}cbigint,cfloat: float|{alltypesorc}cfloat,cdouble: double|{alltypesorc}cdouble,cstring1: string|{alltypesorc}cstring1,cstring2: string|{alltypesorc}cstring2,ctimestamp1: timestamp|{alltypesorc}ctimestamp1,ctimestamp2: timestamp|{alltypesorc}ctimestamp2,cboolean1: boolean|{alltypesorc}cboolean1,cboolean2: boolean|{alltypesorc}cboolean2,BLOCK__OFFSET__INSIDE__FILE: bigint|{alltypesorc}block__offset__inside__file,INPUT__FILE__NAME: string|{alltypesorc}input__file__name,ROW__ID: struct|{alltypesorc}row__id,RAW__DATA__SIZE: bigint|{alltypesorc}raw__data__size) 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: After logical optimization TS[0] 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,094 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,095 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,095 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=alltypesorc 2018-01-05T11:28:53,095 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=alltypesorc 2018-01-05T11:28:53,098 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,100 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,101 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,101 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting stats (Num rows: 1 Data size: 3772370 Basic stats: COMPLETE Column stats: COMPLETE) on TS[0] 2018-01-05T11:28:53,101 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (alltypesorc): numRows: 1 dataSize: 3772370 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:28:53,101 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on TS[0] 2018-01-05T11:28:53,101 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,101 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,101 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,101 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,101 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,101 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,101 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,101 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,101 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,101 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,101 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,101 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,101 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,101 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,101 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,101 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,101 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,101 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,101 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,101 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,101 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:53,101 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:28:53,101 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Cycle free: true 2018-01-05T11:28:53,101 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,101 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,101 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,101 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(ctinyint: tinyint|{alltypesorc}ctinyint,csmallint: smallint|{alltypesorc}csmallint,cint: int|{alltypesorc}cint,cbigint: bigint|{alltypesorc}cbigint,cfloat: float|{alltypesorc}cfloat,cdouble: double|{alltypesorc}cdouble,cstring1: string|{alltypesorc}cstring1,cstring2: string|{alltypesorc}cstring2,ctimestamp1: timestamp|{alltypesorc}ctimestamp1,ctimestamp2: timestamp|{alltypesorc}ctimestamp2,cboolean1: boolean|{alltypesorc}cboolean1,cboolean2: boolean|{alltypesorc}cboolean2,BLOCK__OFFSET__INSIDE__FILE: bigint|{alltypesorc}block__offset__inside__file,INPUT__FILE__NAME: string|{alltypesorc}input__file__name,ROW__ID: struct|{alltypesorc}row__id,RAW__DATA__SIZE: bigint|{alltypesorc}raw__data__size) 2018-01-05T11:28:53,101 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,102 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:28:53,102 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,102 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,102 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:28:53,102 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping vectorization 2018-01-05T11:28:53,102 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:28:53,102 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: llap mode: only 2018-01-05T11:28:53,102 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,103 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.QueryPlanPostProcessor: Found org.apache.hadoop.hive.ql.plan.StatsWork - no FileSinkOperation can be present. executionId=hive_2018-01-05_11-28-53_075_8421955314026672262 2018-01-05T11:28:53,103 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed plan generation 2018-01-05T11:28:53,103 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:53,103 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validation start 2018-01-05T11:28:53,103 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Not a partition. 2018-01-05T11:28:53,103 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,103 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:alltypesorc.ctinyint, type:tinyint, comment:null), FieldSchema(name:alltypesorc.csmallint, type:smallint, comment:null), FieldSchema(name:alltypesorc.cint, type:int, comment:null), FieldSchema(name:alltypesorc.cbigint, type:bigint, comment:null), FieldSchema(name:alltypesorc.cfloat, type:float, comment:null), FieldSchema(name:alltypesorc.cdouble, type:double, comment:null), FieldSchema(name:alltypesorc.cstring1, type:string, comment:null), FieldSchema(name:alltypesorc.cstring2, type:string, comment:null), FieldSchema(name:alltypesorc.ctimestamp1, type:timestamp, comment:null), FieldSchema(name:alltypesorc.ctimestamp2, type:timestamp, comment:null), FieldSchema(name:alltypesorc.cboolean1, type:boolean, comment:null), FieldSchema(name:alltypesorc.cboolean2, type:boolean, comment:null)], properties:null) 2018-01-05T11:28:53,103 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,103 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:53,103 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=7, flushCache_()=0, getTableColumnStatistics_(String, String, List, )=6} 2018-01-05T11:28:53,103 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112853_a8cdfcc6-2ad4-4c71-94a5-276ccef4eca9); Time taken: 0.028 seconds 2018-01-05T11:28:53,103 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:53,103 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,103 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112853_a8cdfcc6-2ad4-4c71-94a5-276ccef4eca9): ANALYZE TABLE alltypesorc COMPUTE STATISTICS 2018-01-05T11:28:53,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,104 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: ANALYZE TABLE alltypesorc COMPUTE STATISTICS 2018-01-05T11:28:53,104 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:28:53,104 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@alltypesorc 2018-01-05T11:28:53,104 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@alltypesorc 2018-01-05T11:28:53,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,106 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-2:STATS] in serial mode 2018-01-05T11:28:53,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@786296ca, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:53,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:53,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,107 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:53,107 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:53,107 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:53,107 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:53,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:53,107 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsNoJobTask: Executing stats (no job) task 2018-01-05T11:28:53,107 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.StatsTask: Initialized threadpool for stats computation with 10 threads 2018-01-05T11:28:53,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsNoJobTask: Stats collection waiting for threadpool to shutdown.. 2018-01-05T11:28:53,109 DEBUG [StatsNoJobTask-Thread-0] stats.BasicStatsNoJobTask: Aggregating stats for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc 2018-01-05T11:28:53,109 DEBUG [StatsNoJobTask-Thread-0] stats.BasicStatsNoJobTask: Computing stats for DeprecatedRawLocalFileStatus{path=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc/alltypesorc; isDirectory=false; length=377237; replication=1; blocksize=33554432; modification_time=1515180533000; access_time=1515180533000; owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false} 2018-01-05T11:28:53,109 DEBUG [StatsNoJobTask-Thread-0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:53,112 DEBUG [StatsNoJobTask-Thread-0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:28:53,112 DEBUG [StatsNoJobTask-Thread-0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:28:53,112 DEBUG [StatsNoJobTask-Thread-0] orc.OrcInputFormat: No ORC pushdown predicate - no column names 2018-01-05T11:28:53,112 INFO [StatsNoJobTask-Thread-0] orc.ReaderImpl: Reading ORC rows from file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc/alltypesorc with {include: null, offset: 0, length: 0} 2018-01-05T11:28:53,118 INFO [StatsNoJobTask-Thread-0] impl.RecordReaderImpl: Reader schema not provided -- using file schema struct 2018-01-05T11:28:53,121 DEBUG [StatsNoJobTask-Thread-0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:28:53,133 DEBUG [StatsNoJobTask-Thread-0] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:28:53,133 DEBUG [StatsNoJobTask-Thread-0] stats.BasicStatsNoJobTask: Table default.alltypesorc stats: [numFiles=1, numRows=12288, totalSize=377237, rawDataSize=2641964] 2018-01-05T11:28:53,133 INFO [StatsNoJobTask-Thread-0] stats.BasicStatsNoJobTask: Table default.alltypesorc stats: [numFiles=1, numRows=12288, totalSize=377237, rawDataSize=2641964] 2018-01-05T11:28:53,133 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsNoJobTask: Stats collection threadpool shutdown successful. 2018-01-05T11:28:53,134 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsNoJobTask: Collectors.size(): [default@alltypesorc#PTable] 2018-01-05T11:28:53,134 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsNoJobTask: Updating stats for: default.alltypesorc 2018-01-05T11:28:53,134 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,135 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:53,135 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:53,135 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:53,135 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:53,135 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,135 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,135 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,136 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,136 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,136 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:53,136 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:53,136 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,136 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:53,136 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,136 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=alltypesorc newtbl=alltypesorc 2018-01-05T11:28:53,136 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=alltypesorc newtbl=alltypesorc 2018-01-05T11:28:53,140 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsNoJobTask: Updated stats for default.alltypesorc. 2018-01-05T11:28:53,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsNoJobTask: Updated stats for: default.alltypesorc 2018-01-05T11:28:53,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,147 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: ANALYZE TABLE alltypesorc COMPUTE STATISTICS 2018-01-05T11:28:53,147 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:28:53,147 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@alltypesorc 2018-01-05T11:28:53,147 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@alltypesorc 2018-01-05T11:28:53,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,147 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:53,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=11} 2018-01-05T11:28:53,148 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112853_a8cdfcc6-2ad4-4c71-94a5-276ccef4eca9); Time taken: 0.044 seconds 2018-01-05T11:28:53,148 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:53,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,148 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:53,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query ANALYZE TABLE alltypesorc COMPUTE STATISTICS 2018-01-05T11:28:53,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_075_8421955314026672262-1 2018-01-05T11:28:53,148 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.073 seconds 2018-01-05T11:28:53,148 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:53,148 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:53,148 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:53,148 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:53,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:53,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: ANALYZE TABLE alltypesorc COMPUTE STATISTICS FOR COLUMNS ctinyint,csmallint,cint,cbigint,cfloat,cdouble,cstring1,cstring2,ctimestamp1,ctimestamp2,cboolean1,cboolean2 2018-01-05T11:28:53,148 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112853_02da1873-4ec4-4565-8e4c-6f3b4f4735d5): ANALYZE TABLE alltypesorc COMPUTE STATISTICS FOR COLUMNS ctinyint,csmallint,cint,cbigint,cfloat,cdouble,cstring1,cstring2,ctimestamp1,ctimestamp2,cboolean1,cboolean2 2018-01-05T11:28:53,149 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,149 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: ANALYZE TABLE alltypesorc COMPUTE STATISTICS FOR COLUMNS ctinyint,csmallint,cint,cbigint,cfloat,cdouble,cstring1,cstring2,ctimestamp1,ctimestamp2,cboolean1,cboolean2 2018-01-05T11:28:53,149 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:53,149 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,149 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,149 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,149 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,149 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:53,149 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,149 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesorc 2018-01-05T11:28:53,149 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesorc 2018-01-05T11:28:53,152 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,153 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,153 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: select compute_stats(`ctinyint`, 'hll') , compute_stats(`csmallint`, 'hll') , compute_stats(`cint`, 'hll') , compute_stats(`cbigint`, 'hll') , compute_stats(`cfloat`, 'hll') , compute_stats(`cdouble`, 'hll') , compute_stats(`cstring1`, 'hll') , compute_stats(`cstring2`, 'hll') , compute_stats(`ctimestamp1`, 'hll') , compute_stats(`ctimestamp2`, 'hll') , compute_stats(`cboolean1`, 'hll') , compute_stats(`cboolean2`, 'hll') from `default`.`alltypesorc` 2018-01-05T11:28:53,153 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: select compute_stats(`ctinyint`, 'hll') , compute_stats(`csmallint`, 'hll') , compute_stats(`cint`, 'hll') , compute_stats(`cbigint`, 'hll') , compute_stats(`cfloat`, 'hll') , compute_stats(`cdouble`, 'hll') , compute_stats(`cstring1`, 'hll') , compute_stats(`cstring2`, 'hll') , compute_stats(`ctimestamp1`, 'hll') , compute_stats(`ctimestamp2`, 'hll') , compute_stats(`cboolean1`, 'hll') , compute_stats(`cboolean2`, 'hll') from `default`.`alltypesorc` 2018-01-05T11:28:53,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:53,155 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Invoking analyze on rewritten query 2018-01-05T11:28:53,155 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Starting Semantic Analysis 2018-01-05T11:28:53,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:53,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:53,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:53,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:53,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:53,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:53,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:53,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:53,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:53,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:53,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:53,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:53,155 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed phase 1 of Semantic Analysis 2018-01-05T11:28:53,155 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for source tables 2018-01-05T11:28:53,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,156 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesorc 2018-01-05T11:28:53,156 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesorc 2018-01-05T11:28:53,158 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,159 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for subqueries 2018-01-05T11:28:53,159 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for destination tables 2018-01-05T11:28:53,173 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed getting MetaData in Semantic Analysis 2018-01-05T11:28:53,174 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_153_487699818259609034-1/-mr-10001 2018-01-05T11:28:53,174 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Table Plan for alltypesorc TS[0] 2018-01-05T11:28:53,174 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:53,174 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:53,174 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:53,174 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:53,175 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:53,175 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:53,175 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:53,175 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:53,175 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:53,175 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:53,175 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:53,176 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:53,177 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: RR before GB alltypesorc{(ctinyint,ctinyint: tinyint)(csmallint,csmallint: smallint)(cint,cint: int)(cbigint,cbigint: bigint)(cfloat,cfloat: float)(cdouble,cdouble: double)(cstring1,cstring1: string)(cstring2,cstring2: string)(ctimestamp1,ctimestamp1: timestamp)(ctimestamp2,ctimestamp2: timestamp)(cboolean1,cboolean1: boolean)(cboolean2,cboolean2: boolean)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)(RAW__DATA__SIZE,RAW__DATA__SIZE: bigint)} after GB {((tok_function compute_stats (tok_table_or_col ctinyint) 'hll'),_col0: struct)((tok_function compute_stats (tok_table_or_col csmallint) 'hll'),_col1: struct)((tok_function compute_stats (tok_table_or_col cint) 'hll'),_col2: struct)((tok_function compute_stats (tok_table_or_col cbigint) 'hll'),_col3: struct)((tok_function compute_stats (tok_table_or_col cfloat) 'hll'),_col4: struct)((tok_function compute_stats (tok_table_or_col cdouble) 'hll'),_col5: struct)((tok_function compute_stats (tok_table_or_col cstring1) 'hll'),_col6: struct)((tok_function compute_stats (tok_table_or_col cstring2) 'hll'),_col7: struct)((tok_function compute_stats (tok_table_or_col ctimestamp1) 'hll'),_col8: struct)((tok_function compute_stats (tok_table_or_col ctimestamp2) 'hll'),_col9: struct)((tok_function compute_stats (tok_table_or_col cboolean1) 'hll'),_col10: struct)((tok_function compute_stats (tok_table_or_col cboolean2) 'hll'),_col11: struct)} 2018-01-05T11:28:53,177 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: tree: (tok_select (tok_selexpr (tok_function compute_stats (tok_table_or_col ctinyint) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col csmallint) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col cint) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col cbigint) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col cfloat) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col cdouble) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col cstring1) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col cstring2) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col ctimestamp1) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col ctimestamp2) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col cboolean1) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col cboolean2) 'hll'))) 2018-01-05T11:28:53,177 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: genSelectPlan: input = {((tok_function compute_stats (tok_table_or_col ctinyint) 'hll'),_col0: struct)((tok_function compute_stats (tok_table_or_col csmallint) 'hll'),_col1: struct)((tok_function compute_stats (tok_table_or_col cint) 'hll'),_col2: struct)((tok_function compute_stats (tok_table_or_col cbigint) 'hll'),_col3: struct)((tok_function compute_stats (tok_table_or_col cfloat) 'hll'),_col4: struct)((tok_function compute_stats (tok_table_or_col cdouble) 'hll'),_col5: struct)((tok_function compute_stats (tok_table_or_col cstring1) 'hll'),_col6: struct)((tok_function compute_stats (tok_table_or_col cstring2) 'hll'),_col7: struct)((tok_function compute_stats (tok_table_or_col ctimestamp1) 'hll'),_col8: struct)((tok_function compute_stats (tok_table_or_col ctimestamp2) 'hll'),_col9: struct)((tok_function compute_stats (tok_table_or_col cboolean1) 'hll'),_col10: struct)((tok_function compute_stats (tok_table_or_col cboolean2) 'hll'),_col11: struct)} starRr = alltypesorc{(ctinyint,ctinyint: tinyint)(csmallint,csmallint: smallint)(cint,cint: int)(cbigint,cbigint: bigint)(cfloat,cfloat: float)(cdouble,cdouble: double)(cstring1,cstring1: string)(cstring2,cstring2: string)(ctimestamp1,ctimestamp1: timestamp)(ctimestamp2,ctimestamp2: timestamp)(cboolean1,cboolean1: boolean)(cboolean2,cboolean2: boolean)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)(RAW__DATA__SIZE,RAW__DATA__SIZE: bigint)} 2018-01-05T11:28:53,177 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Select Plan row schema: null{(_c0,_col0: struct)(_c1,_col1: struct)(_c2,_col2: struct)(_c3,_col3: struct)(_c4,_col4: struct)(_c5,_col5: struct)(_c6,_col6: struct)(_c7,_col7: struct)(_c8,_col8: struct)(_c9,_col9: struct)(_c10,_col10: struct)(_c11,_col11: struct)} 2018-01-05T11:28:53,177 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Select Plan for clause: insclause-0 2018-01-05T11:28:53,177 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_153_487699818259609034-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-53_153_487699818259609034-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_153_487699818259609034-1/-mr-10000 2018-01-05T11:28:53,177 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_153_487699818259609034-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-53_153_487699818259609034-1 2018-01-05T11:28:53,200 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_153_487699818259609034-1/-mr-10003 2018-01-05T11:28:53,200 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_153_487699818259609034-1/-mr-10000 row schema: null{(_c0,_col0: struct)(_c1,_col1: struct)(_c2,_col2: struct)(_c3,_col3: struct)(_c4,_col4: struct)(_c5,_col5: struct)(_c6,_col6: struct)(_c7,_col7: struct)(_c8,_col8: struct)(_c9,_col9: struct)(_c10,_col10: struct)(_c11,_col11: struct)} 2018-01-05T11:28:53,200 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Body Plan for Query Block null 2018-01-05T11:28:53,200 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Plan for Query Block null 2018-01-05T11:28:53,200 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Before logical optimization TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-SEL[5]-FS[6] 2018-01-05T11:28:53,200 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,200 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,200 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,201 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] lineage.Generator: Time taken for lineage transform=1 2018-01-05T11:28:53,201 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,201 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,201 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,201 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,201 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,201 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,201 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,201 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,201 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(ctinyint: tinyint|{alltypesorc}ctinyint,csmallint: smallint|{alltypesorc}csmallint,cint: int|{alltypesorc}cint,cbigint: bigint|{alltypesorc}cbigint,cfloat: float|{alltypesorc}cfloat,cdouble: double|{alltypesorc}cdouble,cstring1: string|{alltypesorc}cstring1,cstring2: string|{alltypesorc}cstring2,ctimestamp1: timestamp|{alltypesorc}ctimestamp1,ctimestamp2: timestamp|{alltypesorc}ctimestamp2,cboolean1: boolean|{alltypesorc}cboolean1,cboolean2: boolean|{alltypesorc}cboolean2,BLOCK__OFFSET__INSIDE__FILE: bigint|{alltypesorc}block__offset__inside__file,INPUT__FILE__NAME: string|{alltypesorc}input__file__name,ROW__ID: struct|{alltypesorc}row__id,RAW__DATA__SIZE: bigint|{alltypesorc}raw__data__size) 2018-01-05T11:28:53,201 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(ctinyint: tinyint|{alltypesorc}ctinyint,csmallint: smallint|{alltypesorc}csmallint,cint: int|{alltypesorc}cint,cbigint: bigint|{alltypesorc}cbigint,cfloat: float|{alltypesorc}cfloat,cdouble: double|{alltypesorc}cdouble,cstring1: string|{alltypesorc}cstring1,cstring2: string|{alltypesorc}cstring2,ctimestamp1: timestamp|{alltypesorc}ctimestamp1,ctimestamp2: timestamp|{alltypesorc}ctimestamp2,cboolean1: boolean|{alltypesorc}cboolean1,cboolean2: boolean|{alltypesorc}cboolean2,BLOCK__OFFSET__INSIDE__FILE: bigint|{alltypesorc}block__offset__inside__file,INPUT__FILE__NAME: string|{alltypesorc}input__file__name,ROW__ID: struct|{alltypesorc}row__id,RAW__DATA__SIZE: bigint|{alltypesorc}raw__data__size) 2018-01-05T11:28:53,201 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:53,201 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:53,201 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[ctinyint] Column[csmallint] Column[cint] Column[cbigint] Column[cfloat] Column[cdouble] Column[cstring1] Column[cstring2] Column[ctimestamp1] Column[ctimestamp2] Column[cboolean1] Column[cboolean2] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:28:53,201 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col ctinyint) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col csmallint) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col cint) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col cbigint) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col cfloat) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col cdouble) 'hll'),_col6: struct|{}(tok_function compute_stats (tok_table_or_col cstring1) 'hll'),_col7: struct|{}(tok_function compute_stats (tok_table_or_col cstring2) 'hll'),_col8: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp1) 'hll'),_col9: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp2) 'hll'),_col10: struct|{}(tok_function compute_stats (tok_table_or_col cboolean1) 'hll'),_col11: struct|{}(tok_function compute_stats (tok_table_or_col cboolean2) 'hll')) 2018-01-05T11:28:53,201 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:53,201 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col ctinyint) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col csmallint) 'hll'),VALUE._col2: struct|{}(tok_function compute_stats (tok_table_or_col cint) 'hll'),VALUE._col3: struct|{}(tok_function compute_stats (tok_table_or_col cbigint) 'hll'),VALUE._col4: struct|{}(tok_function compute_stats (tok_table_or_col cfloat) 'hll'),VALUE._col5: struct|{}(tok_function compute_stats (tok_table_or_col cdouble) 'hll'),VALUE._col6: struct|{}(tok_function compute_stats (tok_table_or_col cstring1) 'hll'),VALUE._col7: struct|{}(tok_function compute_stats (tok_table_or_col cstring2) 'hll'),VALUE._col8: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp1) 'hll'),VALUE._col9: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp2) 'hll'),VALUE._col10: struct|{}(tok_function compute_stats (tok_table_or_col cboolean1) 'hll'),VALUE._col11: struct|{}(tok_function compute_stats (tok_table_or_col cboolean2) 'hll')) 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col ctinyint) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col csmallint) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col cint) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col cbigint) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col cfloat) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col cdouble) 'hll'),_col6: struct|{}(tok_function compute_stats (tok_table_or_col cstring1) 'hll'),_col7: struct|{}(tok_function compute_stats (tok_table_or_col cstring2) 'hll'),_col8: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp1) 'hll'),_col9: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp2) 'hll'),_col10: struct|{}(tok_function compute_stats (tok_table_or_col cboolean1) 'hll'),_col11: struct|{}(tok_function compute_stats (tok_table_or_col cboolean2) 'hll')) 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1,_col2: struct|{null}_c2,_col3: struct|{null}_c3,_col4: struct|{null}_c4,_col5: struct|{null}_c5,_col6: struct|{null}_c6,_col7: struct|{null}_c7,_col8: struct|{null}_c8,_col9: struct|{null}_c9,_col10: struct|{null}_c10,_col11: struct|{null}_c11) 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0] Column[_col1] Column[_col2] Column[_col3] Column[_col4] Column[_col5] Column[_col6] Column[_col7] Column[_col8] Column[_col9] Column[_col10] Column[_col11]) 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{},_col2: struct|{},_col3: struct|{},_col4: struct|{},_col5: struct|{},_col6: struct|{},_col7: struct|{},_col8: struct|{},_col9: struct|{},_col10: struct|{},_col11: struct|{}) 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,202 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for FS(6) 2018-01-05T11:28:53,202 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(5) 2018-01-05T11:28:53,202 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for GBY(4) 2018-01-05T11:28:53,202 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for RS(3) 2018-01-05T11:28:53,202 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for GBY(2) 2018-01-05T11:28:53,202 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(1) 2018-01-05T11:28:53,202 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.PredicatePushDown: After PPD: TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-SEL[5]-FS[6] 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(ctinyint: tinyint|{alltypesorc}ctinyint,csmallint: smallint|{alltypesorc}csmallint,cint: int|{alltypesorc}cint,cbigint: bigint|{alltypesorc}cbigint,cfloat: float|{alltypesorc}cfloat,cdouble: double|{alltypesorc}cdouble,cstring1: string|{alltypesorc}cstring1,cstring2: string|{alltypesorc}cstring2,ctimestamp1: timestamp|{alltypesorc}ctimestamp1,ctimestamp2: timestamp|{alltypesorc}ctimestamp2,cboolean1: boolean|{alltypesorc}cboolean1,cboolean2: boolean|{alltypesorc}cboolean2,BLOCK__OFFSET__INSIDE__FILE: bigint|{alltypesorc}block__offset__inside__file,INPUT__FILE__NAME: string|{alltypesorc}input__file__name,ROW__ID: struct|{alltypesorc}row__id,RAW__DATA__SIZE: bigint|{alltypesorc}raw__data__size) 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(ctinyint: tinyint|{alltypesorc}ctinyint,csmallint: smallint|{alltypesorc}csmallint,cint: int|{alltypesorc}cint,cbigint: bigint|{alltypesorc}cbigint,cfloat: float|{alltypesorc}cfloat,cdouble: double|{alltypesorc}cdouble,cstring1: string|{alltypesorc}cstring1,cstring2: string|{alltypesorc}cstring2,ctimestamp1: timestamp|{alltypesorc}ctimestamp1,ctimestamp2: timestamp|{alltypesorc}ctimestamp2,cboolean1: boolean|{alltypesorc}cboolean1,cboolean2: boolean|{alltypesorc}cboolean2,BLOCK__OFFSET__INSIDE__FILE: bigint|{alltypesorc}block__offset__inside__file,INPUT__FILE__NAME: string|{alltypesorc}input__file__name,ROW__ID: struct|{alltypesorc}row__id,RAW__DATA__SIZE: bigint|{alltypesorc}raw__data__size) 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[ctinyint] Column[csmallint] Column[cint] Column[cbigint] Column[cfloat] Column[cdouble] Column[cstring1] Column[cstring2] Column[ctimestamp1] Column[ctimestamp2] Column[cboolean1] Column[cboolean2] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col ctinyint) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col csmallint) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col cint) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col cbigint) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col cfloat) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col cdouble) 'hll'),_col6: struct|{}(tok_function compute_stats (tok_table_or_col cstring1) 'hll'),_col7: struct|{}(tok_function compute_stats (tok_table_or_col cstring2) 'hll'),_col8: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp1) 'hll'),_col9: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp2) 'hll'),_col10: struct|{}(tok_function compute_stats (tok_table_or_col cboolean1) 'hll'),_col11: struct|{}(tok_function compute_stats (tok_table_or_col cboolean2) 'hll')) 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col ctinyint) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col csmallint) 'hll'),VALUE._col2: struct|{}(tok_function compute_stats (tok_table_or_col cint) 'hll'),VALUE._col3: struct|{}(tok_function compute_stats (tok_table_or_col cbigint) 'hll'),VALUE._col4: struct|{}(tok_function compute_stats (tok_table_or_col cfloat) 'hll'),VALUE._col5: struct|{}(tok_function compute_stats (tok_table_or_col cdouble) 'hll'),VALUE._col6: struct|{}(tok_function compute_stats (tok_table_or_col cstring1) 'hll'),VALUE._col7: struct|{}(tok_function compute_stats (tok_table_or_col cstring2) 'hll'),VALUE._col8: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp1) 'hll'),VALUE._col9: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp2) 'hll'),VALUE._col10: struct|{}(tok_function compute_stats (tok_table_or_col cboolean1) 'hll'),VALUE._col11: struct|{}(tok_function compute_stats (tok_table_or_col cboolean2) 'hll')) 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col ctinyint) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col csmallint) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col cint) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col cbigint) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col cfloat) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col cdouble) 'hll'),_col6: struct|{}(tok_function compute_stats (tok_table_or_col cstring1) 'hll'),_col7: struct|{}(tok_function compute_stats (tok_table_or_col cstring2) 'hll'),_col8: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp1) 'hll'),_col9: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp2) 'hll'),_col10: struct|{}(tok_function compute_stats (tok_table_or_col cboolean1) 'hll'),_col11: struct|{}(tok_function compute_stats (tok_table_or_col cboolean2) 'hll')) 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1,_col2: struct|{null}_c2,_col3: struct|{null}_c3,_col4: struct|{null}_c4,_col5: struct|{null}_c5,_col6: struct|{null}_c6,_col7: struct|{null}_c7,_col8: struct|{null}_c8,_col9: struct|{null}_c9,_col10: struct|{null}_c10,_col11: struct|{null}_c11) 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0] Column[_col1] Column[_col2] Column[_col3] Column[_col4] Column[_col5] Column[_col6] Column[_col7] Column[_col8] Column[_col9] Column[_col10] Column[_col11]) 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{},_col2: struct|{},_col3: struct|{},_col4: struct|{},_col5: struct|{},_col6: struct|{},_col7: struct|{},_col8: struct|{},_col9: struct|{},_col10: struct|{},_col11: struct|{}) 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(ctinyint: tinyint|{alltypesorc}ctinyint,csmallint: smallint|{alltypesorc}csmallint,cint: int|{alltypesorc}cint,cbigint: bigint|{alltypesorc}cbigint,cfloat: float|{alltypesorc}cfloat,cdouble: double|{alltypesorc}cdouble,cstring1: string|{alltypesorc}cstring1,cstring2: string|{alltypesorc}cstring2,ctimestamp1: timestamp|{alltypesorc}ctimestamp1,ctimestamp2: timestamp|{alltypesorc}ctimestamp2,cboolean1: boolean|{alltypesorc}cboolean1,cboolean2: boolean|{alltypesorc}cboolean2,BLOCK__OFFSET__INSIDE__FILE: bigint|{alltypesorc}block__offset__inside__file,INPUT__FILE__NAME: string|{alltypesorc}input__file__name,ROW__ID: struct|{alltypesorc}row__id,RAW__DATA__SIZE: bigint|{alltypesorc}raw__data__size) 2018-01-05T11:28:53,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(ctinyint: tinyint|{alltypesorc}ctinyint,csmallint: smallint|{alltypesorc}csmallint,cint: int|{alltypesorc}cint,cbigint: bigint|{alltypesorc}cbigint,cfloat: float|{alltypesorc}cfloat,cdouble: double|{alltypesorc}cdouble,cstring1: string|{alltypesorc}cstring1,cstring2: string|{alltypesorc}cstring2,ctimestamp1: timestamp|{alltypesorc}ctimestamp1,ctimestamp2: timestamp|{alltypesorc}ctimestamp2,cboolean1: boolean|{alltypesorc}cboolean1,cboolean2: boolean|{alltypesorc}cboolean2,BLOCK__OFFSET__INSIDE__FILE: bigint|{alltypesorc}block__offset__inside__file,INPUT__FILE__NAME: string|{alltypesorc}input__file__name,ROW__ID: struct|{alltypesorc}row__id,RAW__DATA__SIZE: bigint|{alltypesorc}raw__data__size) 2018-01-05T11:28:53,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:53,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:53,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[ctinyint] Column[csmallint] Column[cint] Column[cbigint] Column[cfloat] Column[cdouble] Column[cstring1] Column[cstring2] Column[ctimestamp1] Column[ctimestamp2] Column[cboolean1] Column[cboolean2] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:28:53,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col ctinyint) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col csmallint) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col cint) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col cbigint) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col cfloat) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col cdouble) 'hll'),_col6: struct|{}(tok_function compute_stats (tok_table_or_col cstring1) 'hll'),_col7: struct|{}(tok_function compute_stats (tok_table_or_col cstring2) 'hll'),_col8: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp1) 'hll'),_col9: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp2) 'hll'),_col10: struct|{}(tok_function compute_stats (tok_table_or_col cboolean1) 'hll'),_col11: struct|{}(tok_function compute_stats (tok_table_or_col cboolean2) 'hll')) 2018-01-05T11:28:53,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:53,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:53,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col ctinyint) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col csmallint) 'hll'),VALUE._col2: struct|{}(tok_function compute_stats (tok_table_or_col cint) 'hll'),VALUE._col3: struct|{}(tok_function compute_stats (tok_table_or_col cbigint) 'hll'),VALUE._col4: struct|{}(tok_function compute_stats (tok_table_or_col cfloat) 'hll'),VALUE._col5: struct|{}(tok_function compute_stats (tok_table_or_col cdouble) 'hll'),VALUE._col6: struct|{}(tok_function compute_stats (tok_table_or_col cstring1) 'hll'),VALUE._col7: struct|{}(tok_function compute_stats (tok_table_or_col cstring2) 'hll'),VALUE._col8: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp1) 'hll'),VALUE._col9: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp2) 'hll'),VALUE._col10: struct|{}(tok_function compute_stats (tok_table_or_col cboolean1) 'hll'),VALUE._col11: struct|{}(tok_function compute_stats (tok_table_or_col cboolean2) 'hll')) 2018-01-05T11:28:53,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:53,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:53,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col ctinyint) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col csmallint) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col cint) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col cbigint) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col cfloat) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col cdouble) 'hll'),_col6: struct|{}(tok_function compute_stats (tok_table_or_col cstring1) 'hll'),_col7: struct|{}(tok_function compute_stats (tok_table_or_col cstring2) 'hll'),_col8: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp1) 'hll'),_col9: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp2) 'hll'),_col10: struct|{}(tok_function compute_stats (tok_table_or_col cboolean1) 'hll'),_col11: struct|{}(tok_function compute_stats (tok_table_or_col cboolean2) 'hll')) 2018-01-05T11:28:53,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:53,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:53,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1,_col2: struct|{null}_c2,_col3: struct|{null}_c3,_col4: struct|{null}_c4,_col5: struct|{null}_c5,_col6: struct|{null}_c6,_col7: struct|{null}_c7,_col8: struct|{null}_c8,_col9: struct|{null}_c9,_col10: struct|{null}_c10,_col11: struct|{null}_c11) 2018-01-05T11:28:53,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:53,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:53,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0] Column[_col1] Column[_col2] Column[_col3] Column[_col4] Column[_col5] Column[_col6] Column[_col7] Column[_col8] Column[_col9] Column[_col10] Column[_col11]) 2018-01-05T11:28:53,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{},_col2: struct|{},_col3: struct|{},_col4: struct|{},_col5: struct|{},_col6: struct|{},_col7: struct|{},_col8: struct|{},_col9: struct|{},_col10: struct|{},_col11: struct|{}) 2018-01-05T11:28:53,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:53,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:53,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 3 key:[] 2018-01-05T11:28:53,203 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 3 oldColExprMap: {VALUE._col2=Column[_col2], VALUE._col3=Column[_col3], VALUE._col4=Column[_col4], VALUE._col5=Column[_col5], VALUE._col10=Column[_col10], VALUE._col0=Column[_col0], VALUE._col11=Column[_col11], VALUE._col1=Column[_col1], VALUE._col6=Column[_col6], VALUE._col7=Column[_col7], VALUE._col8=Column[_col8], VALUE._col9=Column[_col9]} 2018-01-05T11:28:53,203 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 3 newColExprMap: {VALUE._col2=Column[_col2], VALUE._col3=Column[_col3], VALUE._col4=Column[_col4], VALUE._col5=Column[_col5], VALUE._col10=Column[_col10], VALUE._col0=Column[_col0], VALUE._col11=Column[_col11], VALUE._col1=Column[_col1], VALUE._col6=Column[_col6], VALUE._col7=Column[_col7], VALUE._col8=Column[_col8], VALUE._col9=Column[_col9]} 2018-01-05T11:28:53,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.IdentityProjectRemover: Identity project remover optimization removed : SEL[5] 2018-01-05T11:28:53,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: After logical optimization TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-FS[6] 2018-01-05T11:28:53,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,204 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=alltypesorc 2018-01-05T11:28:53,204 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=alltypesorc 2018-01-05T11:28:53,209 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 2.40749ms + 0.007537ms, the query is [select "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "TAB_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (...)] 2018-01-05T11:28:53,210 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,218 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,218 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,219 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting stats (Num rows: 12288 Data size: 5697400 Basic stats: COMPLETE Column stats: NONE) on TS[0] 2018-01-05T11:28:53,219 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (alltypesorc): numRows: 12288 dataSize: 5697400 basicStatsState: COMPLETE colStatsState: NONE colStats: {cint= colName: cint colType: int countDistincts: 2457 numNulls: 614 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, ctimestamp2= colName: ctimestamp2 colType: timestamp countDistincts: 2457 numNulls: 614 avgColLen: 40.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, ctimestamp1= colName: ctimestamp1 colType: timestamp countDistincts: 2457 numNulls: 614 avgColLen: 40.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, csmallint= colName: csmallint colType: smallint countDistincts: 2457 numNulls: 614 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -32768 max: 32767 ] isPrimaryKey: false isEstimated: true, cboolean2= colName: cboolean2 colType: boolean countDistincts: 2 numNulls: 614 avgColLen: 4.0 numTrues: 6144 numFalses: 6144 isPrimaryKey: false isEstimated: true, cdouble= colName: cdouble colType: double countDistincts: 2457 numNulls: 614 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: 4.9E-324 max: 1.7976931348623157E308 ] isPrimaryKey: false isEstimated: true, cstring2= colName: cstring2 colType: string countDistincts: 2457 numNulls: 614 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, cboolean1= colName: cboolean1 colType: boolean countDistincts: 2 numNulls: 614 avgColLen: 4.0 numTrues: 6144 numFalses: 6144 isPrimaryKey: false isEstimated: true, cfloat= colName: cfloat colType: float countDistincts: 2457 numNulls: 614 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 1.4E-45 max: 3.4028235E38 ] isPrimaryKey: false isEstimated: true, ctinyint= colName: ctinyint colType: tinyint countDistincts: 2457 numNulls: 614 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -128 max: 127 ] isPrimaryKey: false isEstimated: true, cstring1= colName: cstring1 colType: string countDistincts: 2457 numNulls: 614 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, cbigint= colName: cbigint colType: bigint countDistincts: 2457 numNulls: 614 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: -2147483648 max: 2147483647 ] isPrimaryKey: false isEstimated: true} 2018-01-05T11:28:53,219 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting stats (Num rows: 12288 Data size: 5697400 Basic stats: COMPLETE Column stats: NONE) on SEL[1] 2018-01-05T11:28:53,219 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [1] STATS-SEL[1]: numRows: 12288 dataSize: 5697400 basicStatsState: COMPLETE colStatsState: NONE colStats: {cint= colName: cint colType: int countDistincts: 2457 numNulls: 614 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, ctimestamp2= colName: ctimestamp2 colType: timestamp countDistincts: 2457 numNulls: 614 avgColLen: 40.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, ctimestamp1= colName: ctimestamp1 colType: timestamp countDistincts: 2457 numNulls: 614 avgColLen: 40.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, csmallint= colName: csmallint colType: smallint countDistincts: 2457 numNulls: 614 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -32768 max: 32767 ] isPrimaryKey: false isEstimated: true, cboolean2= colName: cboolean2 colType: boolean countDistincts: 2 numNulls: 614 avgColLen: 4.0 numTrues: 6144 numFalses: 6144 isPrimaryKey: false isEstimated: true, cdouble= colName: cdouble colType: double countDistincts: 2457 numNulls: 614 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: 4.9E-324 max: 1.7976931348623157E308 ] isPrimaryKey: false isEstimated: true, cstring2= colName: cstring2 colType: string countDistincts: 2457 numNulls: 614 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, cboolean1= colName: cboolean1 colType: boolean countDistincts: 2 numNulls: 614 avgColLen: 4.0 numTrues: 6144 numFalses: 6144 isPrimaryKey: false isEstimated: true, cfloat= colName: cfloat colType: float countDistincts: 2457 numNulls: 614 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 1.4E-45 max: 3.4028235E38 ] isPrimaryKey: false isEstimated: true, ctinyint= colName: ctinyint colType: tinyint countDistincts: 2457 numNulls: 614 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -128 max: 127 ] isPrimaryKey: false isEstimated: true, cstring1= colName: cstring1 colType: string countDistincts: 2457 numNulls: 614 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, cbigint= colName: cbigint colType: bigint countDistincts: 2457 numNulls: 614 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: -2147483648 max: 2147483647 ] isPrimaryKey: false isEstimated: true} 2018-01-05T11:28:53,219 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[2]: inputSize: 5697400 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:28:53,219 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [Case 1] STATS-GBY[2]: cardinality: 12288 2018-01-05T11:28:53,219 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 5304 Basic stats: COMPLETE Column stats: NONE) on GBY[2] 2018-01-05T11:28:53,220 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-GBY[2]: numRows: 1 dataSize: 5304 basicStatsState: COMPLETE colStatsState: NONE colStats: {cint= colName: cint colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, _col11= colName: _col11 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 272.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col10= colName: _col10 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 272.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, csmallint= colName: csmallint colType: smallint countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -32768 max: 32767 ] isPrimaryKey: false isEstimated: true, cboolean2= colName: cboolean2 colType: boolean countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 6144 numFalses: 6144 isPrimaryKey: false isEstimated: true, cboolean1= colName: cboolean1 colType: boolean countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 6144 numFalses: 6144 isPrimaryKey: false isEstimated: true, cfloat= colName: cfloat colType: float countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 1.4E-45 max: 3.4028235E38 ] isPrimaryKey: false isEstimated: true, _col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, cbigint= colName: cbigint colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: -2147483648 max: 2147483647 ] isPrimaryKey: false isEstimated: true, _col9= colName: _col9 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, ctimestamp2= colName: ctimestamp2 colType: timestamp countDistincts: 1 numNulls: 0 avgColLen: 40.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, ctimestamp1= colName: ctimestamp1 colType: timestamp countDistincts: 1 numNulls: 0 avgColLen: 40.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, cdouble= colName: cdouble colType: double countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: 4.9E-324 max: 1.7976931348623157E308 ] isPrimaryKey: false isEstimated: true, cstring2= colName: cstring2 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, ctinyint= colName: ctinyint colType: tinyint countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -128 max: 127 ] isPrimaryKey: false isEstimated: true, cstring1= colName: cstring1 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, _col2= colName: _col2 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col4= colName: _col4 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col6= colName: _col6 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col5= colName: _col5 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col8= colName: _col8 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col7= colName: _col7 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:53,220 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 5304 Basic stats: COMPLETE Column stats: NONE) on RS[3] 2018-01-05T11:28:53,220 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-RS[3]: numRows: 1 dataSize: 5304 basicStatsState: COMPLETE colStatsState: NONE colStats: {cint= colName: cint colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, _col11= colName: _col11 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 272.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col10= colName: _col10 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 272.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, csmallint= colName: csmallint colType: smallint countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -32768 max: 32767 ] isPrimaryKey: false isEstimated: true, cboolean2= colName: cboolean2 colType: boolean countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 6144 numFalses: 6144 isPrimaryKey: false isEstimated: true, cboolean1= colName: cboolean1 colType: boolean countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 6144 numFalses: 6144 isPrimaryKey: false isEstimated: true, cfloat= colName: cfloat colType: float countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 1.4E-45 max: 3.4028235E38 ] isPrimaryKey: false isEstimated: true, _col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, cbigint= colName: cbigint colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: -2147483648 max: 2147483647 ] isPrimaryKey: false isEstimated: true, _col9= colName: _col9 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, ctimestamp2= colName: ctimestamp2 colType: timestamp countDistincts: 1 numNulls: 0 avgColLen: 40.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, ctimestamp1= colName: ctimestamp1 colType: timestamp countDistincts: 1 numNulls: 0 avgColLen: 40.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, cdouble= colName: cdouble colType: double countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: 4.9E-324 max: 1.7976931348623157E308 ] isPrimaryKey: false isEstimated: true, cstring2= colName: cstring2 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, ctinyint= colName: ctinyint colType: tinyint countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -128 max: 127 ] isPrimaryKey: false isEstimated: true, cstring1= colName: cstring1 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, _col2= colName: _col2 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col4= colName: _col4 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col6= colName: _col6 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col5= colName: _col5 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col8= colName: _col8 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col7= colName: _col7 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:53,220 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[4]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:28:53,220 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [Case 7] STATS-GBY[4]: cardinality: 0 2018-01-05T11:28:53,220 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[4]: Equals 0 in number of rows. 0 rows will be set to 1 2018-01-05T11:28:53,220 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 5432 Basic stats: COMPLETE Column stats: NONE) on GBY[4] 2018-01-05T11:28:53,220 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-GBY[4]: numRows: 1 dataSize: 5432 basicStatsState: COMPLETE colStatsState: NONE colStats: {cint= colName: cint colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, _col11= colName: _col11 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 272.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col10= colName: _col10 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 272.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, csmallint= colName: csmallint colType: smallint countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -32768 max: 32767 ] isPrimaryKey: false isEstimated: true, cboolean2= colName: cboolean2 colType: boolean countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 6144 numFalses: 6144 isPrimaryKey: false isEstimated: true, cboolean1= colName: cboolean1 colType: boolean countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 6144 numFalses: 6144 isPrimaryKey: false isEstimated: true, cfloat= colName: cfloat colType: float countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 1.4E-45 max: 3.4028235E38 ] isPrimaryKey: false isEstimated: true, _col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, cbigint= colName: cbigint colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: -2147483648 max: 2147483647 ] isPrimaryKey: false isEstimated: true, _col9= colName: _col9 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, ctimestamp2= colName: ctimestamp2 colType: timestamp countDistincts: 1 numNulls: 0 avgColLen: 40.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, ctimestamp1= colName: ctimestamp1 colType: timestamp countDistincts: 1 numNulls: 0 avgColLen: 40.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, cdouble= colName: cdouble colType: double countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: 4.9E-324 max: 1.7976931348623157E308 ] isPrimaryKey: false isEstimated: true, cstring2= colName: cstring2 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, ctinyint= colName: ctinyint colType: tinyint countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -128 max: 127 ] isPrimaryKey: false isEstimated: true, cstring1= colName: cstring1 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, _col2= colName: _col2 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col4= colName: _col4 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col6= colName: _col6 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col5= colName: _col5 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col8= colName: _col8 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col7= colName: _col7 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-FS[6]: numRows: 1 dataSize: 5432 basicStatsState: COMPLETE colStatsState: NONE colStats: {cint= colName: cint colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, _col11= colName: _col11 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 272.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col10= colName: _col10 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 272.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, csmallint= colName: csmallint colType: smallint countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -32768 max: 32767 ] isPrimaryKey: false isEstimated: true, cboolean2= colName: cboolean2 colType: boolean countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 6144 numFalses: 6144 isPrimaryKey: false isEstimated: true, cboolean1= colName: cboolean1 colType: boolean countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 6144 numFalses: 6144 isPrimaryKey: false isEstimated: true, cfloat= colName: cfloat colType: float countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 1.4E-45 max: 3.4028235E38 ] isPrimaryKey: false isEstimated: true, _col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, cbigint= colName: cbigint colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: -2147483648 max: 2147483647 ] isPrimaryKey: false isEstimated: true, _col9= colName: _col9 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, ctimestamp2= colName: ctimestamp2 colType: timestamp countDistincts: 1 numNulls: 0 avgColLen: 40.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, ctimestamp1= colName: ctimestamp1 colType: timestamp countDistincts: 1 numNulls: 0 avgColLen: 40.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, cdouble= colName: cdouble colType: double countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: 4.9E-324 max: 1.7976931348623157E308 ] isPrimaryKey: false isEstimated: true, cstring2= colName: cstring2 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, ctinyint= colName: ctinyint colType: tinyint countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -128 max: 127 ] isPrimaryKey: false isEstimated: true, cstring1= colName: cstring1 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, _col2= colName: _col2 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col4= colName: _col4 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col6= colName: _col6 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col5= colName: _col5 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col8= colName: _col8 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col7= colName: _col7 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on TS[0] 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on SEL[1] 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[2] 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[3] 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[4] 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FileSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on FS[6] 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,221 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: FS, 6 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: GBY, 4 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: RS, 3 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: GBY, 2 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: SEL, 1 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:28:53,221 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Cycle free: true 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(ctinyint: tinyint|{alltypesorc}ctinyint,csmallint: smallint|{alltypesorc}csmallint,cint: int|{alltypesorc}cint,cbigint: bigint|{alltypesorc}cbigint,cfloat: float|{alltypesorc}cfloat,cdouble: double|{alltypesorc}cdouble,cstring1: string|{alltypesorc}cstring1,cstring2: string|{alltypesorc}cstring2,ctimestamp1: timestamp|{alltypesorc}ctimestamp1,ctimestamp2: timestamp|{alltypesorc}ctimestamp2,cboolean1: boolean|{alltypesorc}cboolean1,cboolean2: boolean|{alltypesorc}cboolean2,BLOCK__OFFSET__INSIDE__FILE: bigint|{alltypesorc}block__offset__inside__file,INPUT__FILE__NAME: string|{alltypesorc}input__file__name,ROW__ID: struct|{alltypesorc}row__id,RAW__DATA__SIZE: bigint|{alltypesorc}raw__data__size) 2018-01-05T11:28:53,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(ctinyint: tinyint|{alltypesorc}ctinyint,csmallint: smallint|{alltypesorc}csmallint,cint: int|{alltypesorc}cint,cbigint: bigint|{alltypesorc}cbigint,cfloat: float|{alltypesorc}cfloat,cdouble: double|{alltypesorc}cdouble,cstring1: string|{alltypesorc}cstring1,cstring2: string|{alltypesorc}cstring2,ctimestamp1: timestamp|{alltypesorc}ctimestamp1,ctimestamp2: timestamp|{alltypesorc}ctimestamp2,cboolean1: boolean|{alltypesorc}cboolean1,cboolean2: boolean|{alltypesorc}cboolean2) 2018-01-05T11:28:53,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:53,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:53,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[ctinyint] Column[csmallint] Column[cint] Column[cbigint] Column[cfloat] Column[cdouble] Column[cstring1] Column[cstring2] Column[ctimestamp1] Column[ctimestamp2] Column[cboolean1] Column[cboolean2]) 2018-01-05T11:28:53,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col ctinyint) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col csmallint) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col cint) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col cbigint) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col cfloat) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col cdouble) 'hll'),_col6: struct|{}(tok_function compute_stats (tok_table_or_col cstring1) 'hll'),_col7: struct|{}(tok_function compute_stats (tok_table_or_col cstring2) 'hll'),_col8: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp1) 'hll'),_col9: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp2) 'hll'),_col10: struct|{}(tok_function compute_stats (tok_table_or_col cboolean1) 'hll'),_col11: struct|{}(tok_function compute_stats (tok_table_or_col cboolean2) 'hll')) 2018-01-05T11:28:53,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:53,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:53,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col ctinyint) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col csmallint) 'hll'),VALUE._col2: struct|{}(tok_function compute_stats (tok_table_or_col cint) 'hll'),VALUE._col3: struct|{}(tok_function compute_stats (tok_table_or_col cbigint) 'hll'),VALUE._col4: struct|{}(tok_function compute_stats (tok_table_or_col cfloat) 'hll'),VALUE._col5: struct|{}(tok_function compute_stats (tok_table_or_col cdouble) 'hll'),VALUE._col6: struct|{}(tok_function compute_stats (tok_table_or_col cstring1) 'hll'),VALUE._col7: struct|{}(tok_function compute_stats (tok_table_or_col cstring2) 'hll'),VALUE._col8: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp1) 'hll'),VALUE._col9: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp2) 'hll'),VALUE._col10: struct|{}(tok_function compute_stats (tok_table_or_col cboolean1) 'hll'),VALUE._col11: struct|{}(tok_function compute_stats (tok_table_or_col cboolean2) 'hll')) 2018-01-05T11:28:53,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:53,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:53,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col ctinyint) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col csmallint) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col cint) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col cbigint) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col cfloat) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col cdouble) 'hll'),_col6: struct|{}(tok_function compute_stats (tok_table_or_col cstring1) 'hll'),_col7: struct|{}(tok_function compute_stats (tok_table_or_col cstring2) 'hll'),_col8: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp1) 'hll'),_col9: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp2) 'hll'),_col10: struct|{}(tok_function compute_stats (tok_table_or_col cboolean1) 'hll'),_col11: struct|{}(tok_function compute_stats (tok_table_or_col cboolean2) 'hll')) 2018-01-05T11:28:53,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:53,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:53,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{},_col2: struct|{},_col3: struct|{},_col4: struct|{},_col5: struct|{},_col6: struct|{},_col7: struct|{},_col8: struct|{},_col9: struct|{},_col10: struct|{},_col11: struct|{}) 2018-01-05T11:28:53,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:53,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:53,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: TS[0] 2018-01-05T11:28:53,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: RS[3] 2018-01-05T11:28:53,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:28:53,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc of table alltypesorc 2018-01-05T11:28:53,223 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc 2018-01-05T11:28:53,223 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: RS[3] 2018-01-05T11:28:53,223 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: GBY[4] 2018-01-05T11:28:53,223 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: FS[6] 2018-01-05T11:28:53,223 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding reduce work (Reducer 2) for GBY[4] 2018-01-05T11:28:53,223 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Setting up reduce sink: RS[3] with following reduce work: Reducer 2 2018-01-05T11:28:53,223 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Removing RS[3] as parent from GBY[4] 2018-01-05T11:28:53,223 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: FS[6] 2018-01-05T11:28:53,223 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:28:53,223 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,223 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,223 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:53,223 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:53,223 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:53,223 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:53,223 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:53,223 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:53,223 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:28:53,223 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping vectorization 2018-01-05T11:28:53,223 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:28:53,223 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: llap mode: only 2018-01-05T11:28:53,223 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:28:53,223 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(ctinyint, 'hll')' 2018-01-05T11:28:53,223 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'ctinyint' 2018-01-05T11:28:53,223 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:53,223 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(csmallint, 'hll')' 2018-01-05T11:28:53,223 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'csmallint' 2018-01-05T11:28:53,223 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:53,223 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(cint, 'hll')' 2018-01-05T11:28:53,223 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cint' 2018-01-05T11:28:53,223 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:53,223 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(cbigint, 'hll')' 2018-01-05T11:28:53,223 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cbigint' 2018-01-05T11:28:53,223 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:53,223 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(cfloat, 'hll')' 2018-01-05T11:28:53,223 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cfloat' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(cdouble, 'hll')' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cdouble' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(cstring1, 'hll')' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cstring1' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(cstring2, 'hll')' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cstring2' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(ctimestamp1, 'hll')' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'ctimestamp1' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(ctimestamp2, 'hll')' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'ctimestamp2' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(cboolean1, 'hll')' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cboolean1' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(cboolean2, 'hll')' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cboolean2' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'ctinyint' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'csmallint' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cint' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cbigint' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cfloat' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cdouble' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cstring1' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cstring2' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'ctimestamp1' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'ctimestamp2' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cboolean1' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cboolean2' 2018-01-05T11:28:53,224 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@7e81b882 can be done in LLAP 2018-01-05T11:28:53,224 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Reducer 2 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col0)' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col1)' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col1' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col2)' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col2' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col3)' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col3' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col4)' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col4' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col5)' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col5' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col6)' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col6' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col7)' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col7' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col8)' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col8' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col9)' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col9' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col10)' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col10' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col11)' 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col11' 2018-01-05T11:28:53,224 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@4d2ad535 can be done in LLAP 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,224 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed plan generation 2018-01-05T11:28:53,224 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validation start 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Not a partition. 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,224 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0, type:struct, comment:null), FieldSchema(name:_c1, type:struct, comment:null), FieldSchema(name:_c2, type:struct, comment:null), FieldSchema(name:_c3, type:struct, comment:null), FieldSchema(name:_c4, type:struct, comment:null), FieldSchema(name:_c5, type:struct, comment:null), FieldSchema(name:_c6, type:struct, comment:null), FieldSchema(name:_c7, type:struct, comment:null), FieldSchema(name:_c8, type:struct, comment:null), FieldSchema(name:_c9, type:struct, comment:null), FieldSchema(name:_c10, type:struct, comment:null), FieldSchema(name:_c11, type:struct, comment:null)], properties:null) 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,224 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=8, flushCache_()=0, getTableColumnStatistics_(String, String, List, )=14} 2018-01-05T11:28:53,224 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112853_02da1873-4ec4-4565-8e4c-6f3b4f4735d5); Time taken: 0.076 seconds 2018-01-05T11:28:53,224 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:53,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,224 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112853_02da1873-4ec4-4565-8e4c-6f3b4f4735d5): ANALYZE TABLE alltypesorc COMPUTE STATISTICS FOR COLUMNS ctinyint,csmallint,cint,cbigint,cfloat,cdouble,cstring1,cstring2,ctimestamp1,ctimestamp2,cboolean1,cboolean2 2018-01-05T11:28:53,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,226 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: ANALYZE TABLE alltypesorc COMPUTE STATISTICS FOR COLUMNS ctinyint,csmallint,cint,cbigint,cfloat,cdouble,cstring1,cstring2,ctimestamp1,ctimestamp2,cboolean1,cboolean2 2018-01-05T11:28:53,226 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:28:53,226 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@alltypesorc 2018-01-05T11:28:53,226 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@alltypesorc 2018-01-05T11:28:53,226 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_153_487699818259609034-1/-mr-10000 2018-01-05T11:28:53,226 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,226 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,226 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,226 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Query ID = djaiswal_20180105112853_02da1873-4ec4-4565-8e4c-6f3b4f4735d5 2018-01-05T11:28:53,226 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Total jobs = 1 2018-01-05T11:28:53,226 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,226 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:28:53,226 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MAPRED] in serial mode 2018-01-05T11:28:53,247 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_148_2547157737114910598-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:28:53,264 DEBUG [AMReporterQueueDrainer] impl.AMReporter: Removing am 10.22.8.182:49824 with last associated dag QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=5} from heartbeat with taskCount=0, amFailed=false, isDone=true 2018-01-05T11:28:53,274 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:28:53,274 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:28:53,274 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:28:53,274 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112853_02da1873-4ec4-4565-8e4c-6f3b4f4735d5 2018-01-05T11:28:53,274 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Session is already open 2018-01-05T11:28:53,274 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:28:53,274 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,274 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Dag name: ANALYZE TABLE alltype...,cboolean1,cboolean2 (Stage-0) 2018-01-05T11:28:53,274 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: DagInfo: {"context":"Hive","description":"\n\nANALYZE TABLE alltypesorc COMPUTE STATISTICS FOR COLUMNS ctinyint,csmallint,cint,cbigint,cfloat,cdouble,cstring1,cstring2,ctimestamp1,ctimestamp2,cboolean1,cboolean2"} 2018-01-05T11:28:53,274 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112853_02da1873-4ec4-4565-8e4c-6f3b4f4735d5 with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:28:53,274 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,286 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,286 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:28:53,292 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,292 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 2 size: 3.50KB 2018-01-05T11:28:53,309 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,309 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,323 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:28:53,323 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,323 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:28:53,326 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,326 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 5.72KB 2018-01-05T11:28:53,332 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_153_487699818259609034-1/-mr-10001 2018-01-05T11:28:53,344 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_153_487699818259609034-1/-mr-10001 2018-01-05T11:28:53,345 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc 2018-01-05T11:28:53,345 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,346 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,346 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,346 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=ANALYZE TABLE alltype...,cboolean1,cboolean2 (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112853_02da1873-4ec4-4565-8e4c-6f3b4f4735d5 } 2018-01-05T11:28:53,347 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:53,347 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:53,349 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: ANALYZE TABLE alltype...,cboolean1,cboolean2 (Stage-0) 2018-01-05T11:28:53,349 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:28:53,350 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_18, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_18.dot 2018-01-05T11:28:53,350 INFO [IPC Server handler 0 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_18 2018-01-05T11:28:53,350 WARN [IPC Server handler 0 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:53,350 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Running DAG: ANALYZE TABLE alltype...,cboolean1,cboolean2 (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112853_02da1873-4ec4-4565-8e4c-6f3b4f4735d5 } 2018-01-05T11:28:53,375 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_18][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_18, submitTime=1515180533349, queueName=null 2018-01-05T11:28:53,376 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_18_00 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_18_00 [Map 1], Default Resources= 2018-01-05T11:28:53,376 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_18_00 [Map 1] 2018-01-05T11:28:53,376 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:28:53,376 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_18_00 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:53,377 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_18_01 [Reducer 2]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_18_01 [Reducer 2], Default Resources= 2018-01-05T11:28:53,377 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_18_01 [Reducer 2] 2018-01-05T11:28:53,377 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 2 2018-01-05T11:28:53,377 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_18_01 [Reducer 2] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:53,377 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:28:53,377 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_18][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_18, initTime=1515180533377 2018-01-05T11:28:53,377 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: dag_1515180518813_0001_18 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:28:53,378 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:28:53,378 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_18][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_18, startTime=1515180533378 2018-01-05T11:28:53,378 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_18 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:28:53,378 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {alltypesorc={InputName=alltypesorc}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:28:53,378 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: alltypesorc, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:28:53,378 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_18_00 [Map 1] 2018-01-05T11:28:53,378 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_18, dagName=ANALYZE TABLE alltype...,cboolean1,cboolean2 (Stage-0) 2018-01-05T11:28:53,378 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,378 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,378 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,382 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_18_00 [Map 1] 2018-01-05T11:28:53,382 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_18_00 [Map 1] 2018-01-05T11:28:53,382 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:53,382 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_18_00 [Map 1] 2018-01-05T11:28:53,385 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:28:53,385 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@59c01a57 2018-01-05T11:28:53,385 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_148_2547157737114910598-1/djaiswal/_tez_scratch_dir/079b5d6f-b8a0-4612-9b04-fbe438977be3/map.xml 2018-01-05T11:28:53,385 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_148_2547157737114910598-1/djaiswal/_tez_scratch_dir/079b5d6f-b8a0-4612-9b04-fbe438977be3/map.xml 2018-01-05T11:28:53,385 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_148_2547157737114910598-1/djaiswal/_tez_scratch_dir/079b5d6f-b8a0-4612-9b04-fbe438977be3/map.xml 2018-01-05T11:28:53,385 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:53,385 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:53,385 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:53,395 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:53,395 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 5.72KB 2018-01-05T11:28:53,395 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_18_00 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:28:53,395 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: alltypesorc on vertex vertex_1515180518813_0001_18_00 [Map 1] 2018-01-05T11:28:53,396 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:28:53,396 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:53,396 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to ShuffleVertexManager for vertex_1515180518813_0001_18_01 [Reducer 2] 2018-01-05T11:28:53,396 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:28:53,396 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:53,396 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_148_2547157737114910598-1/djaiswal/_tez_scratch_dir/079b5d6f-b8a0-4612-9b04-fbe438977be3/map.xml 2018-01-05T11:28:53,396 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:53,396 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias alltypesorc 2018-01-05T11:28:53,396 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc 2018-01-05T11:28:53,396 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc 2018-01-05T11:28:53,396 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 0,1,2,3,4,5,6,7,8,9,10,11 2018-01-05T11:28:53,396 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = ctinyint,csmallint,cint,cbigint,cfloat,cdouble,cstring1,cstring2,ctimestamp1,ctimestamp2,cboolean1,cboolean2 2018-01-05T11:28:53,396 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc 2018-01-05T11:28:53,397 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:53,397 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:53,397 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:53,397 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:53,397 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: getSplits started 2018-01-05T11:28:53,399 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: Number of buckets specified by conf file is 0 2018-01-05T11:28:53,402 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.05 maxFrac: 0.75 auto: false desiredTaskIput: 104857600 2018-01-05T11:28:53,402 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:28:53,402 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 1 tasks for vertex: vertex_1515180518813_0001_18_01 [Reducer 2] 2018-01-05T11:28:53,402 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_18_01 [Reducer 2] 2018-01-05T11:28:53,402 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_18][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_18_01, reconfigureDoneTime=1515180533402, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:28:53,402 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_18_01 [Reducer 2], numAdditionalOutputs=1 2018-01-05T11:28:53,402 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_18][Event:VERTEX_INITIALIZED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_18_01, initRequestedTime=1515180533395, initedTime=1515180533402, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:53,402 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_18_01 [Reducer 2] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:28:53,402 INFO [InputInitializer {Map 1} #0] orc.OrcInputFormat: ORC pushdown predicate: null 2018-01-05T11:28:53,404 INFO [InputInitializer {Map 1} #0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [ctinyint, csmallint, cint, cbigint, cfloat, cdouble, cstring1, cstring2, ctimestamp1, ctimestamp2, cboolean1, cboolean2] / schema.evolution.columns.types [tinyint, smallint, int, bigint, float, double, string, string, timestamp, timestamp, boolean, boolean] (isAcidRead false) 2018-01-05T11:28:53,404 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: Generate splits schema evolution property true reader schema struct transactional scan property false 2018-01-05T11:28:53,405 ERROR [ORC_GET_SPLITS #0] io.AcidUtils: Failed to get files with ID; using regular API: Only supported for DFS; got class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:53,426 DEBUG [ORC_GET_SPLITS #0] io.AcidUtils: in directory file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc base = null deltas = 0 2018-01-05T11:28:53,431 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: Split strategy: ETLSplitStrategy strategy for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc 2018-01-05T11:28:53,435 DEBUG [ORC_GET_SPLITS #1] orc.LocalCache: Serialized tail not cached for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc/alltypesorc 2018-01-05T11:28:53,441 DEBUG [ORC_GET_SPLITS #2] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:28:53,441 DEBUG [ORC_GET_SPLITS #2] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:28:53,442 DEBUG [ORC_GET_SPLITS #2] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:28:53,442 DEBUG [ORC_GET_SPLITS #2] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:28:53,444 INFO [InputInitializer {Map 1} #0] orc.OrcInputFormat: FooterCacheHitRatio: 0/1 2018-01-05T11:28:53,444 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc/alltypesorc, start=3, length=376777, isOriginal=true, fileLength=377237, hasFooter=false, hasBase=true, deltas=0] projected_columns_uncompressed_size: 2638742 2018-01-05T11:28:53,444 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: getSplits finished 2018-01-05T11:28:53,444 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 1 2018-01-05T11:28:53,444 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:53,444 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 1. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:28:53,444 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_148_2547157737114910598-1/djaiswal/_tez_scratch_dir/079b5d6f-b8a0-4612-9b04-fbe438977be3/map.xml 2018-01-05T11:28:53,444 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_148_2547157737114910598-1/djaiswal/_tez_scratch_dir/079b5d6f-b8a0-4612-9b04-fbe438977be3/map.xml 2018-01-05T11:28:53,444 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_148_2547157737114910598-1/djaiswal/_tez_scratch_dir/079b5d6f-b8a0-4612-9b04-fbe438977be3/map.xml 2018-01-05T11:28:53,444 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:53,444 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:53,444 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:53,446 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:53,446 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 5.72KB 2018-01-05T11:28:53,446 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc/alltypesorc to src new group? true 2018-01-05T11:28:53,446 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:28:53,446 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:28:53,446 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:28:53,446 DEBUG [InputInitializer {Map 1} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 2638742 2018-01-05T11:28:53,446 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 48865 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 2638742 Original splits: 1 2018-01-05T11:28:53,446 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Using original number of splits: 1 desired splits: 1 2018-01-05T11:28:53,446 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 1 grouped split count is 1, for bucket: 1 2018-01-05T11:28:53,446 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:28:53,446 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: alltypesorc on vertex vertex_1515180518813_0001_18_00 [Map 1] 2018-01-05T11:28:53,447 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_18_00 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:53,447 INFO [App Shared Pool - #0] impl.VertexImpl: Got updated RootInputsSpecs: {alltypesorc=forAllWorkUnits=true, update=[1]} 2018-01-05T11:28:53,447 INFO [App Shared Pool - #0] impl.VertexImpl: Vertex vertex_1515180518813_0001_18_00 [Map 1] parallelism set to 1 2018-01-05T11:28:53,447 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_18_00 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:53,447 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_18][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_18_00, initRequestedTime=1515180533378, initedTime=1515180533447, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:53,447 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_18_00 [Map 1] with distanceFromRoot: 0 2018-01-05T11:28:53,447 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_18_00 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:28:53,448 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_18][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_18_00, startRequestedTime=1515180533395, startedTime=1515180533447 2018-01-05T11:28:53,448 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_18][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_18_00, reconfigureDoneTime=1515180533448, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:28:53,448 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_18_00 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:53,448 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_18_00 for vertex: vertex_1515180518813_0001_18_01 [Reducer 2] numStartedSources: 1 numSources: 1 2018-01-05T11:28:53,448 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_18_01 [Reducer 2] with distanceFromRoot: 1 2018-01-05T11:28:53,448 INFO [App Shared Pool - #0] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:28:53,448 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_18][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_18_01, startRequestedTime=1515180533448, startedTime=1515180533448 2018-01-05T11:28:53,448 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_18_01 [Reducer 2] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:53,448 INFO [App Shared Pool - #0] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:28:53,448 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_18_00 [Map 1] 2018-01-05T11:28:53,448 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: OnVertexStarted vertex: Reducer 2 with 0 source tasks and 1 pending tasks 2018-01-05T11:28:53,449 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_18][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_18_00_000000, scheduledTime=1515180533449, launchTime=1515180533449 2018-01-05T11:28:53,449 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Map 1 in vertex: Reducer 2 numBipartiteSourceTasks: 1 2018-01-05T11:28:53,449 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_18_00 [Map 1] 2018-01-05T11:28:53,449 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_18_00_000000_0, priority=5, capability=, hosts=null 2018-01-05T11:28:53,449 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=5,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:53,449 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=5,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:53,449 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:53,449 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_18_00_000000_0 on hosts=null 2018-01-05T11:28:53,449 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_18_00_000000_0 on hosts=null 2018-01-05T11:28:53,449 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_18_00_000000_0 2018-01-05T11:28:53,449 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:53,449 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #36, task=TaskInfo{task=attempt_1515180518813_0001_18_00_000000_0, priority=5, startTime=0, containerId=null, uniqueId=35, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000036 2018-01-05T11:28:53,449 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_18_00_000000_0; false 2018-01-05T11:28:53,449 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_18_00_000000_0, priority=5, startTime=311553958, containerId=container_111101111_0001_00_000036assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=35, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:53,450 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000036 for node: 10.22.8.182:49818 2018-01-05T11:28:53,452 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 18, name=ANALYZE TABLE alltype...,cboolean1,cboolean2 (Stage-0), queryId=djaiswal_20180105112853_02da1873-4ec4-4565-8e4c-6f3b4f4735d5 2018-01-05T11:28:53,452 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000036, attempt_1515180518813_0001_18_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:53,452 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000036 for node: 10.22.8.182:49818 2018-01-05T11:28:53,452 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000036 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:53,452 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000036, launchTime=1515180533452 2018-01-05T11:28:53,453 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_18_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000036] on NM: [10.22.8.182:49818] 2018-01-05T11:28:53,453 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:53,453 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:53,453 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_18_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112853_02da1873-4ec4-4565-8e4c-6f3b4f4735d5, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000036, dagName=ANALYZE TABLE alltype...,cboolean1,cboolean2 (Stage-0), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{alltypesorc,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{Reducer 2,org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput,1}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180533378, firstAttemptStartTime=1515180533449, currentAttemptStartTime=1515180533452} 2018-01-05T11:28:53,453 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_18][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_18_00_000000_0, startTime=1515180533453, containerId=container_111101111_0001_00_000036, nodeId=10.22.8.182:49818 2018-01-05T11:28:53,453 INFO [IPC Server handler 1 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000036, QueryId=djaiswal_20180105112853_02da1873-4ec4-4565-8e4c-6f3b4f4735d5-18, DagId=18, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180533453 2018-01-05T11:28:53,453 DEBUG [IPC Server handler 1 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=18} with the ShuffleHandler 2018-01-05T11:28:53,466 INFO [IPC Server handler 1 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:28:53,466 DEBUG [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/18] 2018-01-05T11:28:53,467 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:53,467 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_18_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:53,467 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_18_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180533449, dagStartTime=1515180533378, withinDagPriority=5, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:28:53,467 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_18_00_000000_0 : ACCEPTED 2018-01-05T11:28:53,467 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_18_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:53,467 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_18_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180533449, dagStartTime=1515180533378, withinDagPriority=5, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:28:53,467 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_18_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:53,467 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_18_00_000000_0 to pre-emption queue 2018-01-05T11:28:53,467 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_18_00_000000_0: true 2018-01-05T11:28:53,467 INFO [TaskCommunicator # 3] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_18_00_000000_0 2018-01-05T11:28:53,467 INFO [TaskCommunicator # 3] GuaranteedTasks: Received response for attempt_1515180518813_0001_18_00_000000_0, true 2018-01-05T11:28:53,467 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:1247923892 2018-01-05T11:28:53,468 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : ANALYZE TABLE alltype...,cboolean1,cboolean2 (Stage-0), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_18_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=alltypesorc, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=Reducer 2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput }}, ] 2018-01-05T11:28:53,468 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1043333120, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:53,468 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_18_00_000000_0 vertexName: Map 1 2018-01-05T11:28:53,469 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000036, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_18_00_000000_0, eventCount=1 } 2018-01-05T11:28:53,469 DEBUG [TezTR-518813_1_18_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_18 queryId: djaiswal_20180105112853_02da1873-4ec4-4565-8e4c-6f3b4f4735d5 instanceType: TaskRunner2Callable 2018-01-05T11:28:53,469 INFO [TezTR-518813_1_18_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_18_00_000000_0 2018-01-05T11:28:53,469 DEBUG [TezTR-518813_1_18_0_0_0] log.PerfLogger: 2018-01-05T11:28:53,470 INFO [I/O Setup 1 Initialize: {Reducer 2}] writers.UnorderedPartitionedKVWriter: Requested BufferSize (tez.runtime.unordered.output.buffer.size-mb) : 100 2018-01-05T11:28:53,470 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:53,471 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:53,473 INFO [IPC Server handler 4 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_18_00_000000_0 sent events: (0-1). 2018-01-05T11:28:53,473 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:28:53,473 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_18_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:28:53,474 DEBUG [TezTR-518813_1_18_0_0_0] log.PerfLogger: 2018-01-05T11:28:53,474 INFO [TezTR-518813_1_18_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:53,474 INFO [TezTR-518813_1_18_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:53,474 INFO [TezTR-518813_1_18_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:53,476 INFO [I/O Setup 0 Initialize: {alltypesorc}] input.MRInput: alltypesorc using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:28:53,476 INFO [I/O Setup 0 Initialize: {alltypesorc}] input.MRInputLegacy: alltypesorc MRInputLegacy deferring initialization 2018-01-05T11:28:53,476 INFO [TezTR-518813_1_18_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:53,476 INFO [TezTR-518813_1_18_0_0_0] resources.MemoryDistributor: InitialRequests=[alltypesorc:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [Reducer 2:OUTPUT:104857600:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput] 2018-01-05T11:28:53,476 INFO [TezTR-518813_1_18_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:53,476 INFO [TezTR-518813_1_18_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:53,476 INFO [TezTR-518813_1_18_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 104857600, TotalRequestedScaled: 5.24288E7, TotalJVMHeap: 1043333120, TotalAvailable: 699033190, TotalRequested/TotalJVMHeap:0.10 2018-01-05T11:28:53,476 INFO [TezTR-518813_1_18_0_0_0] resources.MemoryDistributor: Allocations=[alltypesorc:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput:OUTPUT:104857600:104857600] 2018-01-05T11:28:53,476 INFO [TezTR-518813_1_18_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:53,476 INFO [TezTR-518813_1_18_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: alltypesorc being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:28:53,476 INFO [TezTR-518813_1_18_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:28:53,476 INFO [I/O Setup 1 Start: {alltypesorc}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: alltypesorc 2018-01-05T11:28:53,476 INFO [TezTR-518813_1_18_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:28:53,476 INFO [TezTR-518813_1_18_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:53,476 INFO [TezTR-518813_1_18_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_18_00_000000_0 2018-01-05T11:28:53,476 DEBUG [TezTR-518813_1_18_0_0_0] log.PerfLogger: 2018-01-05T11:28:53,476 DEBUG [TezTR-518813_1_18_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_18_00_000000_0_10106 2018-01-05T11:28:53,476 DEBUG [TezTR-518813_1_18_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:53,476 DEBUG [TezTR-518813_1_18_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_18_00_000000_0 2018-01-05T11:28:53,476 INFO [TezTR-518813_1_18_0_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112853_02da1873-4ec4-4565-8e4c-6f3b4f4735d5 2018-01-05T11:28:53,476 DEBUG [TezTR-518813_1_18_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:53,476 DEBUG [TezTR-518813_1_18_0_0_0] log.PerfLogger: 2018-01-05T11:28:53,476 DEBUG [TezTR-518813_1_18_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:53,476 DEBUG [TezTR-518813_1_18_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:53,477 DEBUG [TezTR-518813_1_18_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112853_02da1873-4ec4-4565-8e4c-6f3b4f4735d5_Map 1__MAP_PLAN__ 2018-01-05T11:28:53,477 INFO [TezTR-518813_1_18_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_148_2547157737114910598-1/djaiswal/_tez_scratch_dir/079b5d6f-b8a0-4612-9b04-fbe438977be3/map.xml 2018-01-05T11:28:53,477 DEBUG [TezTR-518813_1_18_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_148_2547157737114910598-1/djaiswal/_tez_scratch_dir/079b5d6f-b8a0-4612-9b04-fbe438977be3/map.xml 2018-01-05T11:28:53,477 DEBUG [TezTR-518813_1_18_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_148_2547157737114910598-1/djaiswal/_tez_scratch_dir/079b5d6f-b8a0-4612-9b04-fbe438977be3/map.xml 2018-01-05T11:28:53,477 DEBUG [TezTR-518813_1_18_0_0_0] log.PerfLogger: 2018-01-05T11:28:53,477 INFO [TezTR-518813_1_18_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:53,477 DEBUG [TezTR-518813_1_18_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:53,479 DEBUG [TezTR-518813_1_18_0_0_0] log.PerfLogger: 2018-01-05T11:28:53,479 INFO [TezTR-518813_1_18_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 5.72KB 2018-01-05T11:28:53,480 DEBUG [TezTR-518813_1_18_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:53,480 INFO [TezTR-518813_1_18_0_0_0] tez.MapRecordProcessor: The input names are: [alltypesorc] 2018-01-05T11:28:53,480 INFO [TezTR-518813_1_18_0_0_0] input.MRInput: Initialized MRInput: alltypesorc 2018-01-05T11:28:53,480 DEBUG [TezTR-518813_1_18_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 1 2018-01-05T11:28:53,480 INFO [TezTR-518813_1_18_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_148_2547157737114910598-1/djaiswal/_tez_scratch_dir/079b5d6f-b8a0-4612-9b04-fbe438977be3/map.xml 2018-01-05T11:28:53,480 DEBUG [TezTR-518813_1_18_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:53,480 DEBUG [TezTR-518813_1_18_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc/alltypesorc PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={columns.types=tinyint:smallint:int:bigint:float:double:string:string:timestamp:timestamp:boolean:boolean, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc, columns=ctinyint,csmallint,cint,cbigint,cfloat,cdouble,cstring1,cstring2,ctimestamp1,ctimestamp2,cboolean1,cboolean2, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=12288, numFiles=1, serialization.ddl=struct alltypesorc { byte ctinyint, i16 csmallint, i32 cint, i64 cbigint, float cfloat, double cdouble, string cstring1, string cstring2, timestamp ctimestamp1, timestamp ctimestamp2, bool cboolean1, bool cboolean2}, transient_lastDdlTime=1515180533, rawDataSize=2641964, columns.comments=, totalSize=377237, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, column.name.delimiter=,, name=default.alltypesorc}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={columns.types=tinyint:smallint:int:bigint:float:double:string:string:timestamp:timestamp:boolean:boolean, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc, columns=ctinyint,csmallint,cint,cbigint,cfloat,cdouble,cstring1,cstring2,ctimestamp1,ctimestamp2,cboolean1,cboolean2, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=12288, numFiles=1, serialization.ddl=struct alltypesorc { byte ctinyint, i16 csmallint, i32 cint, i64 cbigint, float cfloat, double cdouble, string cstring1, string cstring2, timestamp ctimestamp1, timestamp ctimestamp2, bool cboolean1, bool cboolean2}, transient_lastDdlTime=1515180533, rawDataSize=2641964, columns.comments=, totalSize=377237, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, column.name.delimiter=,, name=default.alltypesorc}, baseFileName=alltypesorc, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={columns.types=tinyint:smallint:int:bigint:float:double:string:string:timestamp:timestamp:boolean:boolean, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc, columns=ctinyint,csmallint,cint,cbigint,cfloat,cdouble,cstring1,cstring2,ctimestamp1,ctimestamp2,cboolean1,cboolean2, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=12288, numFiles=1, serialization.ddl=struct alltypesorc { byte ctinyint, i16 csmallint, i32 cint, i64 cbigint, float cfloat, double cdouble, string cstring1, string cstring2, timestamp ctimestamp1, timestamp ctimestamp2, bool cboolean1, bool cboolean2}, transient_lastDdlTime=1515180533, rawDataSize=2641964, columns.comments=, totalSize=377237, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, column.name.delimiter=,, name=default.alltypesorc}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={columns.types=tinyint:smallint:int:bigint:float:double:string:string:timestamp:timestamp:boolean:boolean, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc, columns=ctinyint,csmallint,cint,cbigint,cfloat,cdouble,cstring1,cstring2,ctimestamp1,ctimestamp2,cboolean1,cboolean2, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=12288, numFiles=1, serialization.ddl=struct alltypesorc { byte ctinyint, i16 csmallint, i32 cint, i64 cbigint, float cfloat, double cdouble, string cstring1, string cstring2, timestamp ctimestamp1, timestamp ctimestamp2, bool cboolean1, bool cboolean2}, transient_lastDdlTime=1515180533, rawDataSize=2641964, columns.comments=, totalSize=377237, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, column.name.delimiter=,, name=default.alltypesorc}, baseFileName=alltypesorc, vectorPartitionDesc=null]} 2018-01-05T11:28:53,480 DEBUG [TezTR-518813_1_18_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.hive.ql.io.orc.OrcInputFormat 2018-01-05T11:28:53,482 DEBUG [TezTR-518813_1_18_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:53,482 INFO [TezTR-518813_1_18_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_148_2547157737114910598-1/djaiswal/_tez_scratch_dir/079b5d6f-b8a0-4612-9b04-fbe438977be3/map.xml 2018-01-05T11:28:53,482 DEBUG [TezTR-518813_1_18_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:53,482 INFO [TezTR-518813_1_18_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_18_00_000000_0 2018-01-05T11:28:53,482 INFO [TezTR-518813_1_18_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [ctinyint, csmallint, cint, cbigint, cfloat, cdouble, cstring1, cstring2, ctimestamp1, ctimestamp2, cboolean1, cboolean2] / schema.evolution.columns.types [tinyint, smallint, int, bigint, float, double, string, string, timestamp, timestamp, boolean, boolean] (isAcidRead false) 2018-01-05T11:28:53,491 DEBUG [TezTR-518813_1_18_0_0_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:53,491 DEBUG [TezTR-518813_1_18_0_0_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:53,491 DEBUG [TezTR-518813_1_18_0_0_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:53,491 DEBUG [TezTR-518813_1_18_0_0_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:53,492 DEBUG [TezTR-518813_1_18_0_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:28:53,493 DEBUG [TezTR-518813_1_18_0_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:28:53,495 INFO [TezTR-518813_1_18_0_0_0] LlapIoImpl: Using batch-to-row converter for split: OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc/alltypesorc, start=3, length=376777, isOriginal=true, fileLength=377237, hasFooter=false, hasBase=true, deltas=0] 2018-01-05T11:28:53,502 DEBUG [TezTR-518813_1_18_0_0_0] io.BatchToRowReader: Including the columns [1111111111110] 2018-01-05T11:28:53,502 DEBUG [TezTR-518813_1_18_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc/alltypesorc 2018-01-05T11:28:53,502 INFO [TezTR-518813_1_18_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc/alltypesorc, start=3, length=376777, isOriginal=true, fileLength=377237, hasFooter=false, hasBase=true, deltas=0]], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=376777} 2018-01-05T11:28:53,502 DEBUG [IO-Elevator-Thread-0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_18 queryId: djaiswal_20180105112853_02da1873-4ec4-4565-8e4c-6f3b4f4735d5 instanceType: OrcEncodedDataReader 2018-01-05T11:28:53,502 INFO [TezTR-518813_1_18_0_0_0] input.MRInput: alltypesorc initialized RecordReader from event 2018-01-05T11:28:53,502 DEBUG [TezTR-518813_1_18_0_0_0] tez.MapRecordProcessor: Starting Output: Reducer 2 2018-01-05T11:28:53,503 INFO [TezTR-518813_1_18_0_0_0] writers.BaseUnorderedPartitionedKVWriter: Instantiating Partitioner: [org.apache.tez.mapreduce.partition.MRPartitioner] 2018-01-05T11:28:53,514 INFO [IO-Elevator-Thread-0] LlapIoImpl: Processing data for file [-2724874657402502449, 1515180533000, 377237]: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc/alltypesorc 2018-01-05T11:28:53,514 WARN [IO-Elevator-Thread-0] encoded.OrcEncodedDataReader: ORC compression buffer size (10000) is smaller than LLAP low-level cache minimum allocation size (16384). Decrease the value for hive.llap.io.allocator.alloc.min to avoid wasting memory 2018-01-05T11:28:53,514 DEBUG [IO-Elevator-Thread-0] LlapIoOrc: FileSplit {3, 376777}; stripes {3, 106930}, {200000, 114466}, {314466, 62314}, 2018-01-05T11:28:53,520 DEBUG [IO-Elevator-Thread-0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:28:53,529 INFO [IO-Elevator-Thread-0] impl.OrcCodecPool: Got brand-new codec ZLIB 2018-01-05T11:28:53,531 INFO [IO-Elevator-Thread-0] impl.OrcCodecPool: Got brand-new codec ZLIB 2018-01-05T11:28:53,534 INFO [TezTR-518813_1_18_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: numBuffers=2, sizePerBuffer=52428800, skipBuffers=true, numPartitions=1, availableMemory=104857600, maxSingleBufferSizeBytes=2147483647, pipelinedShuffle=false, isFinalMergeEnabled=true, numPartitions=1, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:28:53,534 INFO [TezTR-518813_1_18_0_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:28:53,534 INFO [TezTR-518813_1_18_0_0_0] exec.MapOperator: Initializing operator MAP[0] 2018-01-05T11:28:53,534 DEBUG [TezTR-518813_1_18_0_0_0] exec.MapOperator: Initialization Done 0 MAP 2018-01-05T11:28:53,534 DEBUG [TezTR-518813_1_18_0_0_0] exec.MapOperator: Operator 0 MAP initialized 2018-01-05T11:28:53,534 DEBUG [TezTR-518813_1_18_0_0_0] exec.MapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:28:53,534 DEBUG [TezTR-518813_1_18_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:53,534 DEBUG [TezTR-518813_1_18_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:53,534 DEBUG [TezTR-518813_1_18_0_0_0] exec.MapOperator: Adding alias alltypesorc to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc 2018-01-05T11:28:53,534 DEBUG [TezTR-518813_1_18_0_0_0] exec.MapOperator: dump TS[0] struct 2018-01-05T11:28:53,534 INFO [TezTR-518813_1_18_0_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:28:53,534 INFO [TezTR-518813_1_18_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:53,534 DEBUG [TezTR-518813_1_18_0_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:28:53,534 INFO [TezTR-518813_1_18_0_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:28:53,534 DEBUG [TezTR-518813_1_18_0_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:28:53,535 DEBUG [TezTR-518813_1_18_0_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:28:53,535 DEBUG [TezTR-518813_1_18_0_0_0] exec.TableScanOperator: Initializing children of 0 TS 2018-01-05T11:28:53,535 DEBUG [TezTR-518813_1_18_0_0_0] exec.SelectOperator: Initializing child 1 SEL 2018-01-05T11:28:53,535 INFO [TezTR-518813_1_18_0_0_0] exec.SelectOperator: Initializing operator SEL[1] 2018-01-05T11:28:53,535 INFO [TezTR-518813_1_18_0_0_0] exec.SelectOperator: SELECT struct 2018-01-05T11:28:53,535 DEBUG [TezTR-518813_1_18_0_0_0] exec.SelectOperator: Initialization Done 1 SEL 2018-01-05T11:28:53,535 DEBUG [TezTR-518813_1_18_0_0_0] exec.SelectOperator: Operator 1 SEL initialized 2018-01-05T11:28:53,535 DEBUG [TezTR-518813_1_18_0_0_0] exec.SelectOperator: Initializing children of 1 SEL 2018-01-05T11:28:53,535 DEBUG [TezTR-518813_1_18_0_0_0] exec.GroupByOperator: Initializing child 2 GBY 2018-01-05T11:28:53,535 INFO [TezTR-518813_1_18_0_0_0] exec.GroupByOperator: Initializing operator GBY[2] 2018-01-05T11:28:53,535 DEBUG [TezTR-518813_1_18_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:53,535 DEBUG [TezTR-518813_1_18_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:53,535 DEBUG [TezTR-518813_1_18_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:53,535 DEBUG [TezTR-518813_1_18_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:53,535 DEBUG [TezTR-518813_1_18_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:53,535 DEBUG [TezTR-518813_1_18_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:53,535 DEBUG [TezTR-518813_1_18_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:53,535 DEBUG [TezTR-518813_1_18_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:53,535 DEBUG [TezTR-518813_1_18_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:53,535 DEBUG [TezTR-518813_1_18_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:53,535 DEBUG [TezTR-518813_1_18_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:53,535 DEBUG [TezTR-518813_1_18_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:53,535 DEBUG [IO-Elevator-Thread-0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:28:53,538 INFO [TezTR-518813_1_18_0_0_0] exec.GroupByOperator: Max hash table memory: 93860656 bytes 2018-01-05T11:28:53,538 INFO [TezTR-518813_1_18_0_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:53,538 DEBUG [TezTR-518813_1_18_0_0_0] exec.GroupByOperator: Initialization Done 2 GBY 2018-01-05T11:28:53,538 DEBUG [TezTR-518813_1_18_0_0_0] exec.GroupByOperator: Operator 2 GBY initialized 2018-01-05T11:28:53,538 DEBUG [TezTR-518813_1_18_0_0_0] exec.GroupByOperator: Initializing children of 2 GBY 2018-01-05T11:28:53,538 DEBUG [TezTR-518813_1_18_0_0_0] exec.ReduceSinkOperator: Initializing child 3 RS 2018-01-05T11:28:53,538 INFO [TezTR-518813_1_18_0_0_0] exec.ReduceSinkOperator: Initializing operator RS[3] 2018-01-05T11:28:53,538 DEBUG [TezTR-518813_1_18_0_0_0] exec.ReduceSinkOperator: keys size is 0 2018-01-05T11:28:53,538 INFO [TezTR-518813_1_18_0_0_0] exec.ReduceSinkOperator: Using tag = -1 2018-01-05T11:28:53,538 DEBUG [TezTR-518813_1_18_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:53,538 DEBUG [TezTR-518813_1_18_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:53,538 DEBUG [TezTR-518813_1_18_0_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11] columnTypes=[struct, struct, struct, struct, struct, struct, struct, struct, struct, struct, struct, struct] 2018-01-05T11:28:53,538 DEBUG [TezTR-518813_1_18_0_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS 2018-01-05T11:28:53,538 DEBUG [TezTR-518813_1_18_0_0_0] exec.ReduceSinkOperator: Operator 3 RS initialized 2018-01-05T11:28:53,538 DEBUG [TezTR-518813_1_18_0_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS done is reset. 2018-01-05T11:28:53,538 DEBUG [TezTR-518813_1_18_0_0_0] exec.GroupByOperator: Initialization Done 2 GBY done is reset. 2018-01-05T11:28:53,538 DEBUG [TezTR-518813_1_18_0_0_0] exec.SelectOperator: Initialization Done 1 SEL done is reset. 2018-01-05T11:28:53,538 DEBUG [TezTR-518813_1_18_0_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:28:53,538 INFO [TezTR-518813_1_18_0_0_0] exec.OperatorUtils: Setting output collector: RS[3] --> Reducer 2 2018-01-05T11:28:53,538 DEBUG [TezTR-518813_1_18_0_0_0] log.PerfLogger: 2018-01-05T11:28:53,539 INFO [IO-Elevator-Thread-0] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [-2724874657402502449, 1515180533000, 377237]): [{range start: 0 end: 402}] 2018-01-05T11:28:53,539 INFO [IO-Elevator-Thread-0] encoded.EncodedReaderImpl: Disk ranges after cache (found everything false; file [-2724874657402502449, 1515180533000, 377237], base offset 3): [{range start: 0 end: 402}] 2018-01-05T11:28:53,545 INFO [IO-Elevator-Thread-0] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [-2724874657402502449, 1515180533000, 377237]): [{range start: 402 end: 106715}] 2018-01-05T11:28:53,545 INFO [IO-Elevator-Thread-0] encoded.EncodedReaderImpl: Disk ranges after cache (found everything false; file [-2724874657402502449, 1515180533000, 377237], base offset 3): [{range start: 402 end: 106715}] 2018-01-05T11:28:53,557 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: tinyint streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 2964 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:53,564 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 2 columnType: smallint streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 5531 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:53,566 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 3 columnType: int streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 8172 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:53,567 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 4 columnType: bigint streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 8156 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:53,568 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 5 columnType: float streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 10000 bytes, LlapDataBuffer with 5796 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:53,569 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 6 columnType: double streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 10000 bytes, LlapDataBuffer with 10000 bytes, LlapDataBuffer with 10000 bytes, LlapDataBuffer with 1696 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:53,569 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 7 columnType: string streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DICTIONARY_V2 dictionarySize: 2031 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 2825 bytes, ], indexBaseOffset=0] dictionary: true lengths: true secondary: false tz: 2018-01-05T11:28:53,572 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 8 columnType: string streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DICTIONARY_V2 dictionarySize: 2029 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 2814 bytes, ], indexBaseOffset=0] dictionary: true lengths: true secondary: false tz: 2018-01-05T11:28:53,573 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 9 columnType: timestamp streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 10000 bytes, LlapDataBuffer with 1713 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: true tz: 2018-01-05T11:28:53,574 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 10 columnType: timestamp streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 10000 bytes, LlapDataBuffer with 1765 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: true tz: 2018-01-05T11:28:53,574 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 11 columnType: boolean streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 264 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:53,574 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 12 columnType: boolean streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 262 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:53,577 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000036, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_18_00_000000_0, eventCount=1 } 2018-01-05T11:28:53,578 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:53,583 DEBUG [TezTR-518813_1_18_0_0_0] exec.MapOperator: Processing alias(es) alltypesorc for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc/alltypesorc 2018-01-05T11:28:53,583 INFO [TezTR-518813_1_18_0_0_0] exec.MapOperator: MAP[0]: records read - 1 2018-01-05T11:28:53,584 INFO [TezTR-518813_1_18_0_0_0] exec.MapOperator: MAP[0]: records read - 10 2018-01-05T11:28:53,587 INFO [TezTR-518813_1_18_0_0_0] exec.MapOperator: MAP[0]: records read - 100 2018-01-05T11:28:53,589 DEBUG [IO-Elevator-Thread-0] LlapIoOrc: Done with decode 2018-01-05T11:28:53,591 INFO [IO-Elevator-Thread-0] impl.OrcCodecPool: Got brand-new codec ZLIB 2018-01-05T11:28:53,591 DEBUG [IO-Elevator-Thread-0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:28:53,591 INFO [IO-Elevator-Thread-0] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [-2724874657402502449, 1515180533000, 377237]): [{range start: 0 end: 415}] 2018-01-05T11:28:53,591 INFO [IO-Elevator-Thread-0] encoded.EncodedReaderImpl: Disk ranges after cache (found everything false; file [-2724874657402502449, 1515180533000, 377237], base offset 200000): [{range start: 0 end: 415}] 2018-01-05T11:28:53,593 INFO [IO-Elevator-Thread-0] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [-2724874657402502449, 1515180533000, 377237]): [{range start: 415 end: 114240}] 2018-01-05T11:28:53,593 INFO [IO-Elevator-Thread-0] encoded.EncodedReaderImpl: Disk ranges after cache (found everything false; file [-2724874657402502449, 1515180533000, 377237], base offset 200000): [{range start: 415 end: 114240}] 2018-01-05T11:28:53,596 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: tinyint streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 2976 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:53,597 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 2 columnType: smallint streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 4071 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:53,597 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 3 columnType: int streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 8168 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:53,597 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 4 columnType: bigint streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 10000 bytes, LlapDataBuffer with 1260 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:53,598 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 5 columnType: float streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 10000 bytes, LlapDataBuffer with 5840 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:53,598 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 6 columnType: double streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 10000 bytes, LlapDataBuffer with 10000 bytes, LlapDataBuffer with 10000 bytes, LlapDataBuffer with 1696 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:53,598 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 7 columnType: string streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DICTIONARY_V2 dictionarySize: 2032 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 2820 bytes, ], indexBaseOffset=0] dictionary: true lengths: true secondary: false tz: 2018-01-05T11:28:53,599 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 8 columnType: string streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DICTIONARY_V2 dictionarySize: 2800 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 4234 bytes, ], indexBaseOffset=0] dictionary: true lengths: true secondary: false tz: 2018-01-05T11:28:53,599 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 9 columnType: timestamp streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 10000 bytes, LlapDataBuffer with 1774 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: true tz: 2018-01-05T11:28:53,600 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 10 columnType: timestamp streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 8654 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: true tz: 2018-01-05T11:28:53,600 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 11 columnType: boolean streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 264 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:53,600 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 12 columnType: boolean streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 358 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:53,604 DEBUG [AMReporterQueueDrainer] impl.AMReporter: Removing am 10.22.8.182:49824 with last associated dag QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=6} from heartbeat with taskCount=0, amFailed=false, isDone=true 2018-01-05T11:28:53,604 INFO [TezTR-518813_1_18_0_0_0] exec.MapOperator: MAP[0]: records read - 1000 2018-01-05T11:28:53,604 DEBUG [TezTR-518813_1_18_0_0_0] tez.InterruptibleProcessing: Adjusting abort check rows to 45223 from 1000 2018-01-05T11:28:53,607 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,607 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: Status: Running (Executing on YARN cluster with App id application_1515180518813_0001) 2018-01-05T11:28:53,609 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,609 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 0(+1)/1 Reducer 2: 0/1 2018-01-05T11:28:53,611 DEBUG [IO-Elevator-Thread-0] LlapIoOrc: Done with decode 2018-01-05T11:28:53,612 INFO [IO-Elevator-Thread-0] impl.OrcCodecPool: Got brand-new codec ZLIB 2018-01-05T11:28:53,613 DEBUG [IO-Elevator-Thread-0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:28:53,613 INFO [IO-Elevator-Thread-0] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [-2724874657402502449, 1515180533000, 377237]): [{range start: 0 end: 373}] 2018-01-05T11:28:53,613 INFO [IO-Elevator-Thread-0] encoded.EncodedReaderImpl: Disk ranges after cache (found everything false; file [-2724874657402502449, 1515180533000, 377237], base offset 314466): [{range start: 0 end: 373}] 2018-01-05T11:28:53,614 INFO [IO-Elevator-Thread-0] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [-2724874657402502449, 1515180533000, 377237]): [{range start: 373 end: 62097}] 2018-01-05T11:28:53,614 INFO [IO-Elevator-Thread-0] encoded.EncodedReaderImpl: Disk ranges after cache (found everything false; file [-2724874657402502449, 1515180533000, 377237], base offset 314466): [{range start: 373 end: 62097}] 2018-01-05T11:28:53,617 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: tinyint streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 258 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:53,617 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 2 columnType: smallint streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 1903 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:53,617 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 3 columnType: int streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 8144 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:53,617 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 4 columnType: bigint streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 5062 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:53,617 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 5 columnType: float streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 5056 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:53,617 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 6 columnType: double streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 10000 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:53,617 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 7 columnType: string streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 10000 bytes, LlapDataBuffer with 10000 bytes, LlapDataBuffer with 8853 bytes, ], indexBaseOffset=0] dictionary: false lengths: true secondary: false tz: 2018-01-05T11:28:53,617 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 8 columnType: string streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DICTIONARY_V2 dictionarySize: 1259 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 1749 bytes, ], indexBaseOffset=0] dictionary: true lengths: true secondary: false tz: 2018-01-05T11:28:53,617 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 9 columnType: timestamp streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 973 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: true tz: 2018-01-05T11:28:53,618 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 10 columnType: timestamp streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 4048 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: true tz: 2018-01-05T11:28:53,618 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 11 columnType: boolean streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 257 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:53,618 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 12 columnType: boolean streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 162 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:53,624 DEBUG [IO-Elevator-Thread-0] LlapIoOrc: Done with decode 2018-01-05T11:28:53,624 DEBUG [IO-Elevator-Thread-0] LlapIoImpl: setDone called; closed false, done false, err null, pending 10 2018-01-05T11:28:53,624 DEBUG [IO-Elevator-Thread-0] impl.StatsRecordingThreadPool: Updated stats: instance: OrcEncodedDataReader thread name: IO-Elevator-Thread-0 thread id: 1252 scheme: file bytesRead: 356958 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:53,670 INFO [TezTR-518813_1_18_0_0_0] exec.MapOperator: MAP[0]: records read - 10000 2018-01-05T11:28:53,680 INFO [TezTR-518813_1_18_0_0_0] LlapIoImpl: Maximum queue length observed 10 2018-01-05T11:28:53,680 INFO [TezTR-518813_1_18_0_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, warehouse.alltypesorc, file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc/alltypesorc ([-2724874657402502449, 1515180533000, 377237]), 0,3]: [ NUM_VECTOR_BATCHES=13, NUM_DECODED_BATCHES=3, SELECTED_ROWGROUPS=3, NUM_ERRORS=0, ROWS_EMITTED=12288, METADATA_CACHE_HIT=0, METADATA_CACHE_MISS=4, CACHE_HIT_BYTES=0, CACHE_MISS_BYTES=283052, ALLOCATED_BYTES=2523136, ALLOCATED_USED_BYTES=447544, TOTAL_IO_TIME_NS=110428650, DECODE_TIME_NS=62725786, HDFS_TIME_NS=6707101, CONSUMER_TIME_NS=98829683 ] 2018-01-05T11:28:53,680 DEBUG [TezTR-518813_1_18_0_0_0] encoded.OrcEncodedDataReader: Encoded reader is being stopped 2018-01-05T11:28:53,680 DEBUG [TezTR-518813_1_18_0_0_0] log.PerfLogger: 2018-01-05T11:28:53,681 DEBUG [TezTR-518813_1_18_0_0_0] mr.ObjectCache: djaiswal_20180105112853_02da1873-4ec4-4565-8e4c-6f3b4f4735d5_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:28:53,681 DEBUG [TezTR-518813_1_18_0_0_0] exec.MapOperator: close called for operator MAP[0] 2018-01-05T11:28:53,681 INFO [TezTR-518813_1_18_0_0_0] exec.MapOperator: Closing operator MAP[0] 2018-01-05T11:28:53,681 INFO [TezTR-518813_1_18_0_0_0] exec.MapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:12288, 2018-01-05T11:28:53,681 DEBUG [TezTR-518813_1_18_0_0_0] exec.MapOperator: Closing child = TS[0] 2018-01-05T11:28:53,681 DEBUG [TezTR-518813_1_18_0_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:28:53,681 DEBUG [TezTR-518813_1_18_0_0_0] exec.TableScanOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:53,681 INFO [TezTR-518813_1_18_0_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:28:53,681 DEBUG [TezTR-518813_1_18_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:53,681 DEBUG [TezTR-518813_1_18_0_0_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_153_487699818259609034-1/-mr-10001 2018-01-05T11:28:53,681 INFO [TezTR-518813_1_18_0_0_0] exec.TableScanOperator: publishing : default.alltypesorc/ : {numRows=12288, rawDataSize=0} 2018-01-05T11:28:53,682 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000036, requestId=3, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_18_00_000000_0, eventCount=1 } 2018-01-05T11:28:53,683 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=3, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:53,706 DEBUG [TezTR-518813_1_18_0_0_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_153_487699818259609034-1/-mr-10001/tmpstats-0_0 2018-01-05T11:28:53,706 DEBUG [TezTR-518813_1_18_0_0_0] fs.FSStatsPublisher: Writing stats in it : {default.alltypesorc/={numRows=12288, rawDataSize=0}} 2018-01-05T11:28:53,706 DEBUG [TezTR-518813_1_18_0_0_0] exec.TableScanOperator: Closing child = SEL[1] 2018-01-05T11:28:53,706 DEBUG [TezTR-518813_1_18_0_0_0] exec.SelectOperator: close called for operator SEL[1] 2018-01-05T11:28:53,706 DEBUG [TezTR-518813_1_18_0_0_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:53,706 INFO [TezTR-518813_1_18_0_0_0] exec.SelectOperator: Closing operator SEL[1] 2018-01-05T11:28:53,706 DEBUG [TezTR-518813_1_18_0_0_0] exec.SelectOperator: Closing child = GBY[2] 2018-01-05T11:28:53,706 DEBUG [TezTR-518813_1_18_0_0_0] exec.GroupByOperator: close called for operator GBY[2] 2018-01-05T11:28:53,706 DEBUG [TezTR-518813_1_18_0_0_0] exec.GroupByOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:53,706 INFO [TezTR-518813_1_18_0_0_0] exec.GroupByOperator: Closing operator GBY[2] 2018-01-05T11:28:53,706 INFO [TezTR-518813_1_18_0_0_0] exec.GroupByOperator: Begin Hash Table flush: size = 1 2018-01-05T11:28:53,727 INFO [TezTR-518813_1_18_0_0_0] exec.ReduceSinkOperator: keys are [] num distributions: 0 2018-01-05T11:28:53,727 WARN [TezTR-518813_1_18_0_0_0] objectinspector.StandardStructObjectInspector: Trying to access 5 fields inside a list of 6 elements: [Double, -64.0, 79.5530014038086, 3115, 48 4c 4c e0 83 01 83 01 82 8a 96 09 ff bc 5d c4 8c 9b 0e 82 b5 83 08 80 b0 24 bb f6 cf 0c 80 86 9f 0a c1 e4 82 17 c0 83 e0 07 be b1 f6 01 81 d5 86 07 c3 b8 fc 08 c0 8b 14 be 85 95 02 bf c2 f8 08 81 e4 80 03 be d9 c3 09 80 bb f5 13 c0 d8 f9 08 c2 8c d8 05 be 89 b4 01 80 d2 8c 02 80 9e db 04 c2 d3 c9 0f fe 8f c5 07 c1 e7 ad 03 80 b7 35 bf de 87 06 80 87 ba 11 c0 e7 52 c6 f1 c5 02 fc c8 a4 02 be c3 8e 04 c4 86 e3 11 be a8 ec 01 ff 91 8a 1d 81 82 8e 07 ff fb d3 0c bf c1 d5 02 c0 f7 a1 15 c4 dd a8 05 ff c2 e3 0b bd 8e bc 07 86 f4 e9 04 bb 9b fe 06 c0 bd db 07 bf d1 c5 03 81 d2 dc 02 80 ba 13 c3 9f e6 02 c4 bf eb 07 b9 f5 89 03 c1 dd 86 0b c1 83 ee 05 bd c4 b8 07 c1 da ec 03 85 b1 7c fd cb ec 0b bd ba c1 0d 80 cf ed 0d 82 f1 e0 0a fe 9f b3 09 86 8f a4 04 fa e1 d9 13 80 b3 ee 02 c1 cf b4 0a 82 9b ac 03 fe d9 ff 04 bf c9 ad 15 c0 ba 96 05 80 8a f5 02 c0 a1 f7 07 87 df d1 11 fa ea ed 0e c0 cf fb 0b 83 df 94 0d bd c9 85 01 80 8c 85 04 81 ed dd 0c c1 d9 8b 0a fd a0 c5 12 80 93 e1 03 c0 d9 b6 15 c0 d5 d1 07 c1 9c d2 03 bf c4 b7 0c c1 e6 bf 03 c1 9a 8f 03 fe ab 9f 02 81 ad e1 0a ff ef 9f 04 c3 d0 bb 04 fd fb e5 06 80 83 6f c1 81 ca 05 82 fb fa 0a fe 86 97 05 c0 be 08 83 93 df 06 fe c2 ef 01 ff c8 fd 14 c0 8c f4 0d ff c0 f8 03 82 ac 9b 01 be b9 c4 02 80 9b f2 02 c0 d1 2b 80 86 ea 0a c0 b0 89 07 c0 cb df 08 c0 fb a6 01 84 c9 af 04 fc 8b a1 02 82 8f eb 03 82 97 dc 09 80 d0 0f fc e4 50 c1 d2 ab 02 81 cd 91 03 80 b0 ea 0f fe df e1 06 81 94 a3 12 c0 bf f0 09 ff f9 cc 1b c2 f8 e5 05 c0 af e1 07 be f2 d2 0b c0 80 e1 01 81 83 c6 0b c0 ab a1 02 bf aa b6 09, null] 2018-01-05T11:28:53,727 WARN [TezTR-518813_1_18_0_0_0] objectinspector.StandardStructObjectInspector: ignoring similar errors. 2018-01-05T11:28:53,727 DEBUG [TezTR-518813_1_18_0_0_0] exec.GroupByOperator: Closing child = RS[3] 2018-01-05T11:28:53,727 DEBUG [TezTR-518813_1_18_0_0_0] exec.ReduceSinkOperator: close called for operator RS[3] 2018-01-05T11:28:53,727 DEBUG [TezTR-518813_1_18_0_0_0] exec.ReduceSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:53,727 INFO [TezTR-518813_1_18_0_0_0] exec.ReduceSinkOperator: Closing operator RS[3] 2018-01-05T11:28:53,727 INFO [TezTR-518813_1_18_0_0_0] exec.ReduceSinkOperator: RECORDS_OUT_INTERMEDIATE_Map_1:1, 2018-01-05T11:28:53,727 DEBUG [TezTR-518813_1_18_0_0_0] exec.ReduceSinkOperator: 3 Close done 2018-01-05T11:28:53,727 DEBUG [TezTR-518813_1_18_0_0_0] exec.GroupByOperator: 2 Close done 2018-01-05T11:28:53,727 DEBUG [TezTR-518813_1_18_0_0_0] exec.SelectOperator: 1 Close done 2018-01-05T11:28:53,727 DEBUG [TezTR-518813_1_18_0_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:28:53,727 DEBUG [TezTR-518813_1_18_0_0_0] exec.MapOperator: 0 Close done 2018-01-05T11:28:53,727 INFO [TezTR-518813_1_18_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_18_00_000000_0 2018-01-05T11:28:53,727 INFO [TezTR-518813_1_18_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: Waiting for all spills to complete : Pending : 0 2018-01-05T11:28:53,727 INFO [TezTR-518813_1_18_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: All spills complete 2018-01-05T11:28:53,741 DEBUG [TezTR-518813_1_18_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:53,741 INFO [TezTR-518813_1_18_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:53,741 INFO [TezTR-518813_1_18_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_18_00_000000_0, askedToStop=false 2018-01-05T11:28:53,741 INFO [TezTR-518813_1_18_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_18_00_000000_0, stopRequested=false 2018-01-05T11:28:53,741 INFO [TezTR-518813_1_18_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_18_00_000000_0: Counters: 32 [[File System Counters FILE_BYTES_READ=356958, FILE_BYTES_WRITTEN=0, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=12288, INPUT_SPLIT_LENGTH_BYTES=376777, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=54893, OUTPUT_BYTES_WITH_OVERHEAD=54903, OUTPUT_BYTES_PHYSICAL=54931, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=12288, RECORDS_OUT_INTERMEDIATE_Map_1=1][org.apache.hadoop.hive.llap.counters.LlapIOCounters ALLOCATED_BYTES=2523136, ALLOCATED_USED_BYTES=447544, CACHE_HIT_BYTES=0, CACHE_MISS_BYTES=283052, CONSUMER_TIME_NS=98829683, DECODE_TIME_NS=62725786, HDFS_TIME_NS=6707101, METADATA_CACHE_MISS=4, NUM_DECODED_BATCHES=3, NUM_VECTOR_BATCHES=13, ROWS_EMITTED=12288, SELECTED_ROWGROUPS=3, TOTAL_IO_TIME_NS=110428650]] 2018-01-05T11:28:53,742 DEBUG [TezTR-518813_1_18_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_18_0_0_0 thread id: 1246 scheme: file bytesRead: 22701 bytesWritten: 55029 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:53,742 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_18_00_000000_0, isTaskDone=true 2018-01-05T11:28:53,742 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000036, requestId=4, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_18_00_000000_0, eventCount=4 } 2018-01-05T11:28:53,743 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_18][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_18_00_000000_0, creationTime=1515180533449, allocationTime=1515180533450, startTime=1515180533453, finishTime=1515180533743, timeTaken=290, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:53,743 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_18_00_000000_0 2018-01-05T11:28:53,743 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_18_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:53,743 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=4, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:53,744 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_18_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:53,744 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_18_00_000000_0 2018-01-05T11:28:53,744 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_18_00_000000_0; false 2018-01-05T11:28:53,744 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_18_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:53,744 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000036=276 2018-01-05T11:28:53,744 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_18_00_000000_0: true 2018-01-05T11:28:53,744 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:53,744 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_18][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_18_00_000000, startTime=1515180533453, finishTime=1515180533743, timeTaken=290, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_18_00_000000_0, diagnostics=, counters=Counters: 32, File System Counters, FILE_BYTES_READ=379659, FILE_BYTES_WRITTEN=55029, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=12288, INPUT_SPLIT_LENGTH_BYTES=376777, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=54893, OUTPUT_BYTES_WITH_OVERHEAD=54903, OUTPUT_BYTES_PHYSICAL=54931, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=12288, RECORDS_OUT_INTERMEDIATE_Map_1=1, org.apache.hadoop.hive.llap.counters.LlapIOCounters, ALLOCATED_BYTES=2523136, ALLOCATED_USED_BYTES=447544, CACHE_HIT_BYTES=0, CACHE_MISS_BYTES=283052, CONSUMER_TIME_NS=98829683, DECODE_TIME_NS=62725786, HDFS_TIME_NS=6707101, METADATA_CACHE_MISS=4, NUM_DECODED_BATCHES=3, NUM_VECTOR_BATCHES=13, ROWS_EMITTED=12288, SELECTED_ROWGROUPS=3, TOTAL_IO_TIME_NS=110428650 2018-01-05T11:28:53,744 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_18_00_000000_0 2018-01-05T11:28:53,744 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:53,744 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000036, QueryId=djaiswal_20180105112853_02da1873-4ec4-4565-8e4c-6f3b4f4735d5-18, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:28:53,744 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_18_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:53,744 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_18_00_000000_0 2018-01-05T11:28:53,744 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_18_00 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:53,745 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_18_00 [Map 1] 2018-01-05T11:28:53,745 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_18][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_18_00, initRequestedTime=1515180533378, initedTime=1515180533447, startRequestedTime=1515180533395, startedTime=1515180533447, finishTime=1515180533745, timeTaken=298, status=SUCCEEDED, diagnostics=, counters=Counters: 32, File System Counters, FILE_BYTES_READ=379659, FILE_BYTES_WRITTEN=55029, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=12288, INPUT_SPLIT_LENGTH_BYTES=376777, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=54893, OUTPUT_BYTES_WITH_OVERHEAD=54903, OUTPUT_BYTES_PHYSICAL=54931, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=12288, RECORDS_OUT_INTERMEDIATE_Map_1=1, org.apache.hadoop.hive.llap.counters.LlapIOCounters, ALLOCATED_BYTES=2523136, ALLOCATED_USED_BYTES=447544, CACHE_HIT_BYTES=0, CACHE_MISS_BYTES=283052, CONSUMER_TIME_NS=98829683, DECODE_TIME_NS=62725786, HDFS_TIME_NS=6707101, METADATA_CACHE_MISS=4, NUM_DECODED_BATCHES=3, NUM_VECTOR_BATCHES=13, ROWS_EMITTED=12288, SELECTED_ROWGROUPS=3, TOTAL_IO_TIME_NS=110428650, vertexStats=firstTaskStartTime=1515180533453, firstTasksToStart=[ task_1515180518813_0001_18_00_000000 ], lastTaskFinishTime=1515180533743, lastTasksToFinish=[ task_1515180518813_0001_18_00_000000 ], minTaskDuration=290, maxTaskDuration=290, avgTaskDuration=290.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_18_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_18_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:53,745 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_18_00 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:53,745 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_18][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000036, stoppedTime=1515180533745, exitStatus=0 2018-01-05T11:28:53,745 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000036 exited with diagnostics set to Container completed. 2018-01-05T11:28:53,745 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_18_01 [Reducer 2] attempt: attempt_1515180518813_0001_18_00_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:28:53,745 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_18_00 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2 2018-01-05T11:28:53,745 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null 2018-01-05T11:28:53,745 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000036 2018-01-05T11:28:53,745 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: All source tasks completed. Ramping up 1 remaining tasks for vertex: Reducer 2 2018-01-05T11:28:53,745 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Scheduling 1 tasks for vertex: Reducer 2 with totalTasks: 1. 1 source tasks completed out of 1. MinSourceTaskCompletedFraction: 1.0 min: 0.05 max: 0.75 2018-01-05T11:28:53,745 INFO [App Shared Pool - #0] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Map 1, destinationVertexName=Reducer 2. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.UnorderedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:28:53,745 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_18_01 [Reducer 2] 2018-01-05T11:28:53,745 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_18][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_18_01_000000, scheduledTime=1515180533745, launchTime=1515180533745 2018-01-05T11:28:53,746 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_18_01 [Reducer 2] 2018-01-05T11:28:53,746 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_18_01_000000_0, priority=14, capability=, hosts=null 2018-01-05T11:28:53,746 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:53,746 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:53,746 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:53,746 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_18_01_000000_0 on hosts=null 2018-01-05T11:28:53,746 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_18_01_000000_0 on hosts=null 2018-01-05T11:28:53,746 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_18_01_000000_0 2018-01-05T11:28:53,746 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:53,746 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #37, task=TaskInfo{task=attempt_1515180518813_0001_18_01_000000_0, priority=14, startTime=0, containerId=null, uniqueId=36, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000037 2018-01-05T11:28:53,746 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_18_01_000000_0; false 2018-01-05T11:28:53,746 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_18_01_000000_0, priority=14, startTime=311554255, containerId=container_111101111_0001_00_000037assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=36, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:53,746 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000037 for node: 10.22.8.182:49818 2018-01-05T11:28:53,746 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000037, attempt_1515180518813_0001_18_01_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:53,746 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000037 for node: 10.22.8.182:49818 2018-01-05T11:28:53,747 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000037 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:53,747 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000037, launchTime=1515180533747 2018-01-05T11:28:53,747 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=18}, sourceName=Map 1, state=S_SUCCEEDED 2018-01-05T11:28:53,748 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_18_01_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000037] on NM: [10.22.8.182:49818] 2018-01-05T11:28:53,748 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_18_01_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112853_02da1873-4ec4-4565-8e4c-6f3b4f4735d5, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000037, dagName=ANALYZE TABLE alltype...,cboolean1,cboolean2 (Stage-0), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.UnorderedKVInput,1}}, Outputs={{out_Reducer 2,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=2, completedTaskCount=1, dagStartTime=1515180533378, firstAttemptStartTime=1515180533746, currentAttemptStartTime=1515180533746} 2018-01-05T11:28:53,748 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:53,748 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:53,748 INFO [IPC Server handler 1 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000037, QueryId=djaiswal_20180105112853_02da1873-4ec4-4565-8e4c-6f3b4f4735d5-18, DagId=18, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, SubmitTime=1515180533748 2018-01-05T11:28:53,748 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_18][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_18_01_000000_0, startTime=1515180533747, containerId=container_111101111_0001_00_000037, nodeId=10.22.8.182:49818 2018-01-05T11:28:53,748 DEBUG [IPC Server handler 1 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=18} with the ShuffleHandler 2018-01-05T11:28:53,748 DEBUG [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/18] 2018-01-05T11:28:53,748 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:53,748 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_18_01_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:53,748 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_18_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180533746, dagStartTime=1515180533378, withinDagPriority=14, vertexParallelism= 1, selfAndUpstreamParallelism= 2, selfAndUpstreamComplete= 1}] 2018-01-05T11:28:53,748 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_18_01_000000_0 : ACCEPTED 2018-01-05T11:28:53,748 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_18_01_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:53,748 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_18_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180533746, dagStartTime=1515180533378, withinDagPriority=14, vertexParallelism= 1, selfAndUpstreamParallelism= 2, selfAndUpstreamComplete= 1} 2018-01-05T11:28:53,748 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_18_01_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:53,748 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_18_01_000000_0 to pre-emption queue 2018-01-05T11:28:53,748 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_18_01_000000_0: true 2018-01-05T11:28:53,748 INFO [TaskCommunicator # 5] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_18_01_000000_0 2018-01-05T11:28:53,748 INFO [TaskCommunicator # 5] GuaranteedTasks: Received response for attempt_1515180518813_0001_18_01_000000_0, true 2018-01-05T11:28:53,748 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:1247923892 2018-01-05T11:28:53,749 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : ANALYZE TABLE alltype...,cboolean1,cboolean2 (Stage-0), VertexName: Reducer 2, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_18_01_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:28:53,749 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1027604480, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:53,749 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_18_01_000000_0 vertexName: Reducer 2 2018-01-05T11:28:53,749 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000037, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_18_01_000000_0, eventCount=1 } 2018-01-05T11:28:53,749 DEBUG [TezTR-518813_1_18_1_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_18 queryId: djaiswal_20180105112853_02da1873-4ec4-4565-8e4c-6f3b4f4735d5 instanceType: TaskRunner2Callable 2018-01-05T11:28:53,749 INFO [TezTR-518813_1_18_1_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_18_01_000000_0 2018-01-05T11:28:53,750 DEBUG [TezTR-518813_1_18_1_0_0] log.PerfLogger: 2018-01-05T11:28:53,750 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:53,751 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:53,753 INFO [IPC Server handler 29 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_18_01_000000_0 1 events [0,1) total 1 vertex_1515180518813_0001_18_01 [Reducer 2] 2018-01-05T11:28:53,753 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:28:53,753 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_18_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:28:53,754 DEBUG [TezTR-518813_1_18_1_0_0] log.PerfLogger: 2018-01-05T11:28:53,754 INFO [TezTR-518813_1_18_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:53,754 INFO [TezTR-518813_1_18_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:53,754 INFO [TezTR-518813_1_18_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:53,754 INFO [I/O Setup 1 Initialize: {out_Reducer 2}] output.MROutput: out_Reducer 2: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:28:53,754 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:53,754 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:53,754 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:53,754 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:53,754 INFO [TezTR-518813_1_18_1_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:53,754 INFO [TezTR-518813_1_18_1_0_0] resources.MemoryDistributor: InitialRequests=[out_Reducer 2:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.UnorderedKVInput] 2018-01-05T11:28:53,754 INFO [TezTR-518813_1_18_1_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:53,754 INFO [TezTR-518813_1_18_1_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:53,754 INFO [TezTR-518813_1_18_1_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 131404920, TotalRequestedScaled: 6.570246E7, TotalJVMHeap: 1027604480, TotalAvailable: 688495001, TotalRequested/TotalJVMHeap:0.13 2018-01-05T11:28:53,754 INFO [TezTR-518813_1_18_1_0_0] resources.MemoryDistributor: Allocations=[out_Reducer 2:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [Map 1:org.apache.tez.runtime.library.input.UnorderedKVInput:INPUT:131404920:131404920] 2018-01-05T11:28:53,754 INFO [TezTR-518813_1_18_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:53,754 INFO [TezTR-518813_1_18_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 0 2018-01-05T11:28:53,754 INFO [TezTR-518813_1_18_1_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:53,754 INFO [TezTR-518813_1_18_1_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_18_01_000000_0 2018-01-05T11:28:53,754 DEBUG [TezTR-518813_1_18_1_0_0] log.PerfLogger: 2018-01-05T11:28:53,754 DEBUG [TezTR-518813_1_18_1_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_18_01_000000_0_10109 2018-01-05T11:28:53,754 DEBUG [TezTR-518813_1_18_1_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:53,754 DEBUG [TezTR-518813_1_18_1_0_0] mr.ObjectCache: Creating djaiswal_20180105112853_02da1873-4ec4-4565-8e4c-6f3b4f4735d5_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:28:53,754 INFO [TezTR-518813_1_18_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_148_2547157737114910598-1/djaiswal/_tez_scratch_dir/b424eb5b-b973-4974-a10f-4605c7a770f6/reduce.xml 2018-01-05T11:28:53,755 DEBUG [TezTR-518813_1_18_1_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_148_2547157737114910598-1/djaiswal/_tez_scratch_dir/b424eb5b-b973-4974-a10f-4605c7a770f6/reduce.xml 2018-01-05T11:28:53,755 DEBUG [TezTR-518813_1_18_1_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_148_2547157737114910598-1/djaiswal/_tez_scratch_dir/b424eb5b-b973-4974-a10f-4605c7a770f6/reduce.xml 2018-01-05T11:28:53,755 DEBUG [TezTR-518813_1_18_1_0_0] log.PerfLogger: 2018-01-05T11:28:53,755 INFO [TezTR-518813_1_18_1_0_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:28:53,755 DEBUG [TezTR-518813_1_18_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:53,756 DEBUG [TezTR-518813_1_18_1_0_0] log.PerfLogger: 2018-01-05T11:28:53,756 INFO [TezTR-518813_1_18_1_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 3.50KB 2018-01-05T11:28:53,756 DEBUG [TezTR-518813_1_18_1_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:53,756 DEBUG [TezTR-518813_1_18_1_0_0] log.PerfLogger: 2018-01-05T11:28:53,756 DEBUG [TezTR-518813_1_18_1_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:53,756 DEBUG [TezTR-518813_1_18_1_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:53,756 DEBUG [TezTR-518813_1_18_1_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:53,756 INFO [TezTR-518813_1_18_1_0_0] impl.SimpleFetchedInputAllocator: Map_1: RequestedMemory=131404920, AssignedMemory=131404920, maxSingleShuffleLimit=32851230 2018-01-05T11:28:53,756 INFO [TezTR-518813_1_18_1_0_0] impl.ShuffleManager: Map_1: numInputs=1, compressionCodec=NoCompressionCodec, numFetchers=1, ifileBufferSize=4096, ifileReadAheadEnabled=true, ifileReadAheadLength=4194304, localDiskFetchEnabled=false, sharedFetchEnabled=false, keepAlive=false, keepAliveMaxConnections=20, connectionTimeout=20000, readTimeout=180000, bufferSize=8192, bufferSize=8192, maxTaskOutputAtOnce=20 2018-01-05T11:28:53,757 INFO [TezTR-518813_1_18_1_0_0] impl.ShuffleInputEventHandlerImpl: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:28:53,757 INFO [TezTR-518813_1_18_1_0_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:28:53,758 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=18&reduce=0&map=attempt_1515180518813_0001_18_00_000000_0_10108 mapId: [attempt_1515180518813_0001_18_00_000000_0_10108] reduceId: [0] jobId: [job_1515180518813_0001] dagId: [18] keepAlive: false 2018-01-05T11:28:53,758 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=18&reduce=0&map=attempt_1515180518813_0001_18_00_000000_0_10108; hash=...5DAFKdzUAGn+I 2018-01-05T11:28:53,758 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=18&reduce=0&map=attempt_1515180518813_0001_18_00_000000_0_10108;reply=KzXWweXxry2MM 2018-01-05T11:28:53,758 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=18, user='djaiswal', attemptId='attempt_1515180518813_0001_18_00_000000_0_10108'} via loader 2018-01-05T11:28:53,758 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=18, user='djaiswal', attemptId='attempt_1515180518813_0001_18_00_000000_0_10108'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:28:53,758 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_18_00_000000_0_10108 not found 2018-01-05T11:28:53,758 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_18_00_000000_0_10108,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/18/output/attempt_1515180518813_0001_18_00_000000_0_10108/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/18/output/attempt_1515180518813_0001_18_00_000000_0_10108/file.out.index 2018-01-05T11:28:53,758 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Content Length in shuffle : 54962 2018-01-05T11:28:53,759 INFO [Fetcher_B {Map_1} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=18&reduce=0&map=attempt_1515180518813_0001_18_00_000000_0_10108 sent hash and receievd reply 0 ms 2018-01-05T11:28:53,759 INFO [Fetcher_B {Map_1} #0] ShuffleManager.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_18_00_000000_0_10108} to MEMORY, csize=54907, dsize=54903, EndTime=1515180533759, TimeTaken=0, Rate=0.00 MB/s 2018-01-05T11:28:53,759 INFO [Fetcher_B {Map_1} #0] impl.ShuffleManager: All inputs fetched for input vertex : Map 1 2018-01-05T11:28:53,759 INFO [Fetcher_B {Map_1} #0] impl.ShuffleManager: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.05 MB/s) 2018-01-05T11:28:53,759 INFO [TezTR-518813_1_18_1_0_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:28:53,759 INFO [ShuffleRunner {Map_1}] impl.ShuffleManager: Map_1: Shutting down FetchScheduler, Was Interrupted: false 2018-01-05T11:28:53,759 INFO [ShuffleRunner {Map_1}] impl.ShuffleManager: Map_1: Scheduler thread completed 2018-01-05T11:28:53,759 INFO [TezTR-518813_1_18_1_0_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:53,759 DEBUG [TezTR-518813_1_18_1_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:28:53,759 DEBUG [TezTR-518813_1_18_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:53,759 DEBUG [TezTR-518813_1_18_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:53,759 DEBUG [TezTR-518813_1_18_1_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11] columnTypes=[struct, struct, struct, struct, struct, struct, struct, struct, struct, struct, struct, struct] 2018-01-05T11:28:53,760 DEBUG [TezTR-518813_1_18_1_0_0] log.PerfLogger: 2018-01-05T11:28:53,760 INFO [TezTR-518813_1_18_1_0_0] exec.GroupByOperator: Initializing operator GBY[4] 2018-01-05T11:28:53,760 DEBUG [TezTR-518813_1_18_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:53,760 DEBUG [TezTR-518813_1_18_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:53,760 DEBUG [TezTR-518813_1_18_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:53,760 DEBUG [TezTR-518813_1_18_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:53,760 DEBUG [TezTR-518813_1_18_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:53,760 DEBUG [TezTR-518813_1_18_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:53,760 DEBUG [TezTR-518813_1_18_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:53,760 DEBUG [TezTR-518813_1_18_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:53,760 DEBUG [TezTR-518813_1_18_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:53,760 DEBUG [TezTR-518813_1_18_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:53,760 DEBUG [TezTR-518813_1_18_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:53,760 DEBUG [TezTR-518813_1_18_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:53,760 INFO [TezTR-518813_1_18_1_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:53,760 DEBUG [TezTR-518813_1_18_1_0_0] exec.GroupByOperator: Initialization Done 4 GBY 2018-01-05T11:28:53,760 DEBUG [TezTR-518813_1_18_1_0_0] exec.GroupByOperator: Operator 4 GBY initialized 2018-01-05T11:28:53,760 DEBUG [TezTR-518813_1_18_1_0_0] exec.GroupByOperator: Initializing children of 4 GBY 2018-01-05T11:28:53,760 DEBUG [TezTR-518813_1_18_1_0_0] exec.FileSinkOperator: Initializing child 6 FS 2018-01-05T11:28:53,760 INFO [TezTR-518813_1_18_1_0_0] exec.FileSinkOperator: Initializing operator FS[6] 2018-01-05T11:28:53,760 DEBUG [TezTR-518813_1_18_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:53,761 INFO [TezTR-518813_1_18_1_0_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@2ad74aab]:[_col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11]:[struct, struct, struct, struct, struct, struct, struct, struct, struct, struct, struct, struct]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@2d7d6b93 2018-01-05T11:28:53,761 DEBUG [TezTR-518813_1_18_1_0_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:28:53,761 DEBUG [TezTR-518813_1_18_1_0_0] exec.FileSinkOperator: Initialization Done 6 FS 2018-01-05T11:28:53,761 DEBUG [TezTR-518813_1_18_1_0_0] exec.FileSinkOperator: Operator 6 FS initialized 2018-01-05T11:28:53,761 DEBUG [TezTR-518813_1_18_1_0_0] exec.FileSinkOperator: Initialization Done 6 FS done is reset. 2018-01-05T11:28:53,761 DEBUG [TezTR-518813_1_18_1_0_0] exec.GroupByOperator: Initialization Done 4 GBY done is reset. 2018-01-05T11:28:53,761 INFO [TezTR-518813_1_18_1_0_0] tez.ReduceRecordProcessor: Id =4 Id =6 <\Children> Id = 4 null<\Parent> <\FS> <\Children> <\Parent> <\GBY> 2018-01-05T11:28:53,761 DEBUG [TezTR-518813_1_18_1_0_0] log.PerfLogger: 2018-01-05T11:28:53,761 INFO [TezTR-518813_1_18_1_0_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 2 2018-01-05T11:28:53,761 DEBUG [TezTR-518813_1_18_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:28:53,761 DEBUG [TezTR-518813_1_18_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:28:53,761 DEBUG [TezTR-518813_1_18_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:28:53,761 DEBUG [TezTR-518813_1_18_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:28:53,761 DEBUG [TezTR-518813_1_18_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:28:53,761 DEBUG [TezTR-518813_1_18_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:28:53,769 INFO [TezTR-518813_1_18_1_0_0] readers.UnorderedKVReader: Num Records read: 1 2018-01-05T11:28:53,769 DEBUG [TezTR-518813_1_18_1_0_0] log.PerfLogger: 2018-01-05T11:28:53,769 DEBUG [TezTR-518813_1_18_1_0_0] mr.ObjectCache: djaiswal_20180105112853_02da1873-4ec4-4565-8e4c-6f3b4f4735d5_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:28:53,769 DEBUG [TezTR-518813_1_18_1_0_0] exec.GroupByOperator: close called for operator GBY[4] 2018-01-05T11:28:53,769 INFO [TezTR-518813_1_18_1_0_0] exec.GroupByOperator: Closing operator GBY[4] 2018-01-05T11:28:53,771 INFO [TezTR-518813_1_18_1_0_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_153_487699818259609034-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-53_153_487699818259609034-1/_tmp.-ext-10002/000000_0 2018-01-05T11:28:53,771 INFO [TezTR-518813_1_18_1_0_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_153_487699818259609034-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-53_153_487699818259609034-1/_task_tmp.-ext-10002/_tmp.000000_0 2018-01-05T11:28:53,771 INFO [TezTR-518813_1_18_1_0_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_153_487699818259609034-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-53_153_487699818259609034-1/_tmp.-ext-10002/000000_0 2018-01-05T11:28:53,808 INFO [TezTR-518813_1_18_1_0_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:28:53,808 DEBUG [TezTR-518813_1_18_1_0_0] exec.GroupByOperator: Closing child = FS[6] 2018-01-05T11:28:53,808 DEBUG [TezTR-518813_1_18_1_0_0] exec.FileSinkOperator: close called for operator FS[6] 2018-01-05T11:28:53,808 DEBUG [TezTR-518813_1_18_1_0_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:53,808 INFO [TezTR-518813_1_18_1_0_0] exec.FileSinkOperator: Closing operator FS[6] 2018-01-05T11:28:53,808 INFO [TezTR-518813_1_18_1_0_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:28:53,808 INFO [TezTR-518813_1_18_1_0_0] exec.FileSinkOperator: RECORDS_OUT_0:1, 2018-01-05T11:28:53,808 DEBUG [TezTR-518813_1_18_1_0_0] exec.FileSinkOperator: 6 Close done 2018-01-05T11:28:53,809 DEBUG [TezTR-518813_1_18_1_0_0] exec.GroupByOperator: 4 Close done 2018-01-05T11:28:53,809 INFO [TezTR-518813_1_18_1_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_18_01_000000_0 2018-01-05T11:28:53,809 INFO [TezTR-518813_1_18_1_0_0] impl.ShuffleInputEventHandlerImpl: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:28:53,809 INFO [TezTR-518813_1_18_1_0_0] impl.ShuffleManager: Shutting down pending fetchers on sourceMap_1: 0 2018-01-05T11:28:53,809 INFO [TezTR-518813_1_18_1_0_0] output.MROutput: out_Reducer 2 closed 2018-01-05T11:28:53,809 DEBUG [TezTR-518813_1_18_1_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:53,809 INFO [TezTR-518813_1_18_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:53,809 INFO [TezTR-518813_1_18_1_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_18_01_000000_0, askedToStop=false 2018-01-05T11:28:53,809 INFO [TezTR-518813_1_18_1_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_18_01_000000_0, stopRequested=false 2018-01-05T11:28:53,809 INFO [TezTR-518813_1_18_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_18_01_000000_0: Counters: 14 [[org.apache.tez.common.counters.TaskCounter NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=54907, SHUFFLE_BYTES_DECOMPRESSED=54903, SHUFFLE_BYTES_TO_MEM=54907, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=3, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1][HIVE CREATED_FILES=1, RECORDS_OUT_0=1]] 2018-01-05T11:28:53,809 DEBUG [TezTR-518813_1_18_1_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_18_1_0_0 thread id: 1259 scheme: file bytesRead: 0 bytesWritten: 73876 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:53,810 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_18_01_000000_0, isTaskDone=true 2018-01-05T11:28:53,810 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000037, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_18_01_000000_0, eventCount=2 } 2018-01-05T11:28:53,811 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_18][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_18_01_000000_0, creationTime=1515180533746, allocationTime=1515180533746, startTime=1515180533747, finishTime=1515180533811, timeTaken=64, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:53,811 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:28:53,811 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_18_01_000000_0 2018-01-05T11:28:53,811 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_18_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:53,811 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_18_01_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:53,811 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_18_01_000000_0 2018-01-05T11:28:53,811 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_18_01_000000_0; false 2018-01-05T11:28:53,811 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_18_01_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:53,811 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_18][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_18_01_000000, startTime=1515180533747, finishTime=1515180533811, timeTaken=64, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_18_01_000000_0, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=73876, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=54907, SHUFFLE_BYTES_DECOMPRESSED=54903, SHUFFLE_BYTES_TO_MEM=54907, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=3, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1 2018-01-05T11:28:53,812 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:53,812 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000037=63 2018-01-05T11:28:53,812 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_18_01 [Reducer 2], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:53,812 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_18_01_000000_0: true 2018-01-05T11:28:53,812 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_18_01 [Reducer 2] 2018-01-05T11:28:53,812 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:53,812 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_18][Event:VERTEX_FINISHED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_18_01, initRequestedTime=1515180533395, initedTime=1515180533402, startRequestedTime=1515180533448, startedTime=1515180533448, finishTime=1515180533812, timeTaken=364, status=SUCCEEDED, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=73876, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=54907, SHUFFLE_BYTES_DECOMPRESSED=54903, SHUFFLE_BYTES_TO_MEM=54907, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=3, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1, vertexStats=firstTaskStartTime=1515180533747, firstTasksToStart=[ task_1515180518813_0001_18_01_000000 ], lastTaskFinishTime=1515180533811, lastTasksToFinish=[ task_1515180518813_0001_18_01_000000 ], minTaskDuration=64, maxTaskDuration=64, avgTaskDuration=64.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_18_01_000000 ], longestDurationTasks=[ task_1515180518813_0001_18_01_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:53,812 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_18_01 [Reducer 2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:53,812 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_18][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000037, stoppedTime=1515180533812, exitStatus=0 2018-01-05T11:28:53,812 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000037 exited with diagnostics set to Container completed. 2018-01-05T11:28:53,812 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_18_01 [Reducer 2] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2 2018-01-05T11:28:53,812 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null 2018-01-05T11:28:53,812 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_18_00 [Map 1] 2018-01-05T11:28:53,812 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_18_01 [Reducer 2] 2018-01-05T11:28:53,813 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_18, queueSize=0 2018-01-05T11:28:53,813 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_18][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_18, startTime=1515180533378, finishTime=1515180533812, timeTaken=434, status=SUCCEEDED, diagnostics=, counters=Counters: 48, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, AM_CPU_MILLISECONDS=1434, AM_GC_TIME_MILLIS=10, File System Counters, FILE_BYTES_READ=379659, FILE_BYTES_WRITTEN=128905, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=12289, INPUT_SPLIT_LENGTH_BYTES=376777, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=54893, OUTPUT_BYTES_WITH_OVERHEAD=54903, OUTPUT_BYTES_PHYSICAL=54931, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=54907, SHUFFLE_BYTES_DECOMPRESSED=54903, SHUFFLE_BYTES_TO_MEM=54907, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=3, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1, HIVE, CREATED_FILES=1, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=12288, RECORDS_OUT_0=1, RECORDS_OUT_INTERMEDIATE_Map_1=1, org.apache.hadoop.hive.llap.counters.LlapIOCounters, ALLOCATED_BYTES=2523136, ALLOCATED_USED_BYTES=447544, CACHE_HIT_BYTES=0, CACHE_MISS_BYTES=283052, CONSUMER_TIME_NS=98829683, DECODE_TIME_NS=62725786, HDFS_TIME_NS=6707101, METADATA_CACHE_MISS=4, NUM_DECODED_BATCHES=3, NUM_VECTOR_BATCHES=13, ROWS_EMITTED=12288, SELECTED_ROWGROUPS=3, TOTAL_IO_TIME_NS=110428650 2018-01-05T11:28:53,813 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_18 finished with state: SUCCEEDED 2018-01-05T11:28:53,813 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_18 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:28:53,813 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000037 2018-01-05T11:28:53,814 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_18, dagState=SUCCEEDED 2018-01-05T11:28:53,814 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_18_post 2018-01-05T11:28:53,814 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:53,814 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:28:53,814 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:28:53,814 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 18 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=2, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=2,NumRequestsWithoutLocation=2, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=2} 2018-01-05T11:28:53,814 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:28:53,814 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=ANALYZE TABLE alltype...,cboolean1,cboolean2 (Stage-0), with id=dag_1515180518813_0001_18 2018-01-05T11:28:53,814 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 18, to 10.22.8.182:49818 2018-01-05T11:28:53,814 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=2, Remaining=0 2018-01-05T11:28:53,814 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=ANALYZE TABLE alltype...,cboolean1,cboolean2 (Stage-0), with id=dag_1515180518813_0001_18 2018-01-05T11:28:53,814 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=18} 2018-01-05T11:28:53,814 INFO [IPC Server handler 0 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=18}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:28:53,814 INFO [IPC Server handler 0 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/18 after 300 seconds 2018-01-05T11:28:53,814 ERROR [IPC Server handler 0 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112853_02da1873-4ec4-4565-8e4c-6f3b4f4735d5, dag_1515180518813_0001_18 2018-01-05T11:28:53,814 INFO [IPC Server handler 0 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112853_02da1873-4ec4-4565-8e4c-6f3b4f4735d5 2018-01-05T11:28:53,814 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=18} = 1 2018-01-05T11:28:53,814 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Issuing killFragment for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=18} attempt_1515180518813_0001_18_01_000000_0 2018-01-05T11:28:53,814 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Removing attempt_1515180518813_0001_18_01_000000_0 from preemptionQueue 2018-01-05T11:28:53,814 INFO [IPC Server handler 0 on 49818] impl.TaskRunnerCallable: Ignoring kill request for task attempt_1515180518813_0001_18_01_000000_0 since it's already complete 2018-01-05T11:28:53,814 DEBUG [IPC Server handler 0 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=18} 2018-01-05T11:28:53,814 DEBUG [IPC Server handler 0 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:28:53,814 DEBUG [IPC Server handler 0 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=18} 2018-01-05T11:28:53,815 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,815 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,815 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,815 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 Reducer 2: 1/1 2018-01-05T11:28:53,815 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,817 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,817 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: TaskId for 000000_0 = 000000 2018-01-05T11:28:53,817 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,817 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,817 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,819 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-2:STATS] in serial mode 2018-01-05T11:28:53,819 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@77c60c78, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:53,819 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:53,819 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,819 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:53,819 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:53,819 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:53,819 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:53,819 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,819 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:53,819 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsNoJobTask: Executing stats (no job) task 2018-01-05T11:28:53,819 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.StatsTask: Initialized threadpool for stats computation with 10 threads 2018-01-05T11:28:53,820 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsNoJobTask: Stats collection waiting for threadpool to shutdown.. 2018-01-05T11:28:53,820 DEBUG [StatsNoJobTask-Thread-0] stats.BasicStatsNoJobTask: Aggregating stats for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc 2018-01-05T11:28:53,820 DEBUG [StatsNoJobTask-Thread-0] stats.BasicStatsNoJobTask: Computing stats for DeprecatedRawLocalFileStatus{path=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc/alltypesorc; isDirectory=false; length=377237; replication=1; blocksize=33554432; modification_time=1515180533000; access_time=1515180533000; owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false} 2018-01-05T11:28:53,821 DEBUG [StatsNoJobTask-Thread-0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:28:53,821 DEBUG [StatsNoJobTask-Thread-0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:28:53,821 DEBUG [StatsNoJobTask-Thread-0] orc.OrcInputFormat: No ORC pushdown predicate - no column names 2018-01-05T11:28:53,821 INFO [StatsNoJobTask-Thread-0] orc.ReaderImpl: Reading ORC rows from file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc/alltypesorc with {include: null, offset: 0, length: 0} 2018-01-05T11:28:53,821 INFO [StatsNoJobTask-Thread-0] impl.RecordReaderImpl: Reader schema not provided -- using file schema struct 2018-01-05T11:28:53,821 DEBUG [StatsNoJobTask-Thread-0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:28:53,822 DEBUG [StatsNoJobTask-Thread-0] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:28:53,822 DEBUG [StatsNoJobTask-Thread-0] stats.BasicStatsNoJobTask: Table default.alltypesorc stats: [numFiles=1, numRows=12288, totalSize=377237, rawDataSize=2641964] 2018-01-05T11:28:53,822 INFO [StatsNoJobTask-Thread-0] stats.BasicStatsNoJobTask: Table default.alltypesorc stats: [numFiles=1, numRows=12288, totalSize=377237, rawDataSize=2641964] 2018-01-05T11:28:53,822 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsNoJobTask: Stats collection threadpool shutdown successful. 2018-01-05T11:28:53,822 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsNoJobTask: Collectors.size(): [default@alltypesorc#PTable] 2018-01-05T11:28:53,822 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsNoJobTask: Updating stats for: default.alltypesorc 2018-01-05T11:28:53,823 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,823 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:53,823 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:53,823 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:53,824 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:53,824 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,824 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,824 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,825 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,825 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,825 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:53,825 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:53,825 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,825 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:53,825 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,825 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=alltypesorc newtbl=alltypesorc 2018-01-05T11:28:53,825 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=alltypesorc newtbl=alltypesorc 2018-01-05T11:28:53,828 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,832 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,833 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,833 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsNoJobTask: Updated stats for default.alltypesorc. 2018-01-05T11:28:53,833 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsNoJobTask: Updated stats for: default.alltypesorc 2018-01-05T11:28:53,845 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000036, QueryId=djaiswal_20180105112853_02da1873-4ec4-4565-8e4c-6f3b4f4735d5-18, DagId=18, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180533467, EndTime=1515180533845 2018-01-05T11:28:53,845 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_18_01_000000_0 2018-01-05T11:28:53,845 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_18_01_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:53,845 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_18_01_000000_0 2018-01-05T11:28:53,845 INFO [ExecutionCompletionThread #0] impl.QueryTracker: Ignoring fragmentComplete message for unknown query: QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=18} 2018-01-05T11:28:53,848 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Time taken to get FileStatuses: 14 2018-01-05T11:28:53,848 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:28:53,848 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 14 2018-01-05T11:28:53,848 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FetchOperator: Creating fetchTask with deserializer typeinfo: struct<_col0:struct,_col1:struct,_col2:struct,_col3:struct,_col4:struct,_col5:struct,_col6:struct,_col7:struct,_col8:struct,_col9:struct,_col10:struct,_col11:struct> 2018-01-05T11:28:53,848 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FetchOperator: deserializer properties: table properties: {columns=_col0,_col1,_col2,_col3,_col4,_col5,_col6,_col7,_col8,_col9,_col10,_col11, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct:struct:struct:struct:struct:struct:struct:struct:struct:struct:struct:struct, escape.delim=\} partition properties: {columns=_col0,_col1,_col2,_col3,_col4,_col5,_col6,_col7,_col8,_col9,_col10,_col11, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct:struct:struct:struct:struct:struct:struct:struct:struct:struct:struct:struct, escape.delim=\} 2018-01-05T11:28:53,855 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,856 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: write_column_statistics: db=default table=alltypesorc 2018-01-05T11:28:53,856 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=write_column_statistics: db=default table=alltypesorc 2018-01-05T11:28:53,858 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=alltypesorc tableName=default colName=ctinyint 2018-01-05T11:28:53,860 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=alltypesorc tableName=default colName=csmallint 2018-01-05T11:28:53,861 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=alltypesorc tableName=default colName=cint 2018-01-05T11:28:53,861 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=alltypesorc tableName=default colName=cbigint 2018-01-05T11:28:53,862 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=alltypesorc tableName=default colName=cfloat 2018-01-05T11:28:53,864 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,865 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=alltypesorc tableName=default colName=cdouble 2018-01-05T11:28:53,865 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=alltypesorc tableName=default colName=cstring1 2018-01-05T11:28:53,866 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=alltypesorc tableName=default colName=cstring2 2018-01-05T11:28:53,867 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=alltypesorc tableName=default colName=ctimestamp1 2018-01-05T11:28:53,867 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=alltypesorc tableName=default colName=ctimestamp2 2018-01-05T11:28:53,868 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,868 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=alltypesorc tableName=default colName=cboolean1 2018-01-05T11:28:53,869 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=alltypesorc tableName=default colName=cboolean2 2018-01-05T11:28:53,872 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,872 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,872 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,872 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,872 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: ANALYZE TABLE alltypesorc COMPUTE STATISTICS FOR COLUMNS ctinyint,csmallint,cint,cbigint,cfloat,cdouble,cstring1,cstring2,ctimestamp1,ctimestamp2,cboolean1,cboolean2 2018-01-05T11:28:53,872 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:28:53,872 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@alltypesorc 2018-01-05T11:28:53,872 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@alltypesorc 2018-01-05T11:28:53,872 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_153_487699818259609034-1/-mr-10000 2018-01-05T11:28:53,872 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,872 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,872 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:53,872 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=8, setPartitionColumnStatistics_(SetPartitionsStatsRequest, )=17} 2018-01-05T11:28:53,872 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112853_02da1873-4ec4-4565-8e4c-6f3b4f4735d5); Time taken: 0.648 seconds 2018-01-05T11:28:53,872 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:53,872 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,872 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:53,872 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,872 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query ANALYZE TABLE alltypesorc COMPUTE STATISTICS FOR COLUMNS ctinyint,csmallint,cint,cbigint,cfloat,cdouble,cstring1,cstring2,ctimestamp1,ctimestamp2,cboolean1,cboolean2 2018-01-05T11:28:53,872 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_148_2547157737114910598-1 2018-01-05T11:28:53,873 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.724 seconds 2018-01-05T11:28:53,873 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:53,873 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:53,873 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:53,873 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:53,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:53,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: -- -- Table alltypesparquet -- DROP TABLE IF EXISTS alltypesparquet 2018-01-05T11:28:53,873 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112853_daf60693-bfa6-4d20-ac16-59b5181dbc4d): -- -- Table alltypesparquet -- DROP TABLE IF EXISTS alltypesparquet 2018-01-05T11:28:53,874 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,874 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: -- -- Table alltypesparquet -- DROP TABLE IF EXISTS alltypesparquet 2018-01-05T11:28:53,874 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:53,874 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,874 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,874 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,874 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,874 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:53,874 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,874 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesparquet 2018-01-05T11:28:53,874 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesparquet 2018-01-05T11:28:53,874 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,874 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,874 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:53,874 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,875 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:53,875 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,875 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:53,875 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:28:53,875 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112853_daf60693-bfa6-4d20-ac16-59b5181dbc4d); Time taken: 0.002 seconds 2018-01-05T11:28:53,875 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:53,875 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,875 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112853_daf60693-bfa6-4d20-ac16-59b5181dbc4d): -- -- Table alltypesparquet -- DROP TABLE IF EXISTS alltypesparquet 2018-01-05T11:28:53,876 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,876 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: -- -- Table alltypesparquet -- DROP TABLE IF EXISTS alltypesparquet 2018-01-05T11:28:53,876 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:53,876 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,876 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,876 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,876 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,876 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:53,876 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,876 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesparquet 2018-01-05T11:28:53,876 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesparquet 2018-01-05T11:28:53,876 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,876 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,876 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table alltypesparquet not found: default.alltypesparquet table not found 2018-01-05T11:28:53,876 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,876 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesparquet 2018-01-05T11:28:53,877 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesparquet 2018-01-05T11:28:53,877 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,877 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,877 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,877 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,877 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: -- -- Table alltypesparquet -- DROP TABLE IF EXISTS alltypesparquet 2018-01-05T11:28:53,877 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:53,877 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,877 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,877 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:53,877 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=1} 2018-01-05T11:28:53,877 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112853_daf60693-bfa6-4d20-ac16-59b5181dbc4d); Time taken: 0.002 seconds 2018-01-05T11:28:53,877 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:53,877 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,877 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:53,877 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,877 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query -- -- Table alltypesparquet -- DROP TABLE IF EXISTS alltypesparquet 2018-01-05T11:28:53,877 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.004 seconds 2018-01-05T11:28:53,877 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:53,877 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:53,877 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:53,877 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:53,877 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,877 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:53,877 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,877 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,877 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,877 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: CREATE TABLE alltypesparquet( ctinyint TINYINT, csmallint SMALLINT, cint INT, cbigint BIGINT, cfloat FLOAT, cdouble DOUBLE, cstring1 STRING, cstring2 STRING, ctimestamp1 TIMESTAMP, ctimestamp2 TIMESTAMP, cboolean1 BOOLEAN, cboolean2 BOOLEAN) STORED AS PARQUET 2018-01-05T11:28:53,877 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112853_6ea9a679-9267-42c2-bbbf-0b8e25d5f756): CREATE TABLE alltypesparquet( ctinyint TINYINT, csmallint SMALLINT, cint INT, cbigint BIGINT, cfloat FLOAT, cdouble DOUBLE, cstring1 STRING, cstring2 STRING, ctimestamp1 TIMESTAMP, ctimestamp2 TIMESTAMP, cboolean1 BOOLEAN, cboolean2 BOOLEAN) STORED AS PARQUET 2018-01-05T11:28:53,877 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,877 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: CREATE TABLE alltypesparquet( ctinyint TINYINT, csmallint SMALLINT, cint INT, cbigint BIGINT, cfloat FLOAT, cdouble DOUBLE, cstring1 STRING, cstring2 STRING, ctimestamp1 TIMESTAMP, ctimestamp2 TIMESTAMP, cboolean1 BOOLEAN, cboolean2 BOOLEAN) STORED AS PARQUET 2018-01-05T11:28:53,878 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:53,878 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,878 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,878 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,878 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,878 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:53,878 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:28:53,878 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Creating table default.alltypesparquet position=13 2018-01-05T11:28:53,883 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,883 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_database: default 2018-01-05T11:28:53,883 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_database: default 2018-01-05T11:28:53,884 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:53,884 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,884 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,884 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:53,884 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: validation start 2018-01-05T11:28:53,884 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:28:53,884 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Not a partition. 2018-01-05T11:28:53,884 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,884 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:53,884 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,884 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:53,884 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getDatabase_(String, )=1, flushCache_()=0} 2018-01-05T11:28:53,884 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112853_6ea9a679-9267-42c2-bbbf-0b8e25d5f756); Time taken: 0.007 seconds 2018-01-05T11:28:53,884 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:53,884 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,884 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112853_6ea9a679-9267-42c2-bbbf-0b8e25d5f756): CREATE TABLE alltypesparquet( ctinyint TINYINT, csmallint SMALLINT, cint INT, cbigint BIGINT, cfloat FLOAT, cdouble DOUBLE, cstring1 STRING, cstring2 STRING, ctimestamp1 TIMESTAMP, ctimestamp2 TIMESTAMP, cboolean1 BOOLEAN, cboolean2 BOOLEAN) STORED AS PARQUET 2018-01-05T11:28:53,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,885 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: CREATE TABLE alltypesparquet( ctinyint TINYINT, csmallint SMALLINT, cint INT, cbigint BIGINT, cfloat FLOAT, cdouble DOUBLE, cstring1 STRING, cstring2 STRING, ctimestamp1 TIMESTAMP, ctimestamp2 TIMESTAMP, cboolean1 BOOLEAN, cboolean2 BOOLEAN) STORED AS PARQUET 2018-01-05T11:28:53,885 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: CREATETABLE 2018-01-05T11:28:53,885 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: database:default 2018-01-05T11:28:53,885 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@alltypesparquet 2018-01-05T11:28:53,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,885 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,885 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:53,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Found class for org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe 2018-01-05T11:28:53,886 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: creating table default.alltypesparquet on file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet 2018-01-05T11:28:53,890 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,890 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: create_table: Table(tableName:alltypesparquet, dbName:default, owner:hive_test_user, createTime:1515180533, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:ctinyint, type:tinyint, comment:null), FieldSchema(name:csmallint, type:smallint, comment:null), FieldSchema(name:cint, type:int, comment:null), FieldSchema(name:cbigint, type:bigint, comment:null), FieldSchema(name:cfloat, type:float, comment:null), FieldSchema(name:cdouble, type:double, comment:null), FieldSchema(name:cstring1, type:string, comment:null), FieldSchema(name:cstring2, type:string, comment:null), FieldSchema(name:ctimestamp1, type:timestamp, comment:null), FieldSchema(name:ctimestamp2, type:timestamp, comment:null), FieldSchema(name:cboolean1, type:boolean, comment:null), FieldSchema(name:cboolean2, type:boolean, comment:null)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet, inputFormat:org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe, parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"cbigint":"true","cboolean1":"true","cboolean2":"true","cdouble":"true","cfloat":"true","cint":"true","csmallint":"true","cstring1":"true","cstring2":"true","ctimestamp1":"true","ctimestamp2":"true","ctinyint":"true"}}, numFiles=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:53,890 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=create_table: Table(tableName:alltypesparquet, dbName:default, owner:hive_test_user, createTime:1515180533, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:ctinyint, type:tinyint, comment:null), FieldSchema(name:csmallint, type:smallint, comment:null), FieldSchema(name:cint, type:int, comment:null), FieldSchema(name:cbigint, type:bigint, comment:null), FieldSchema(name:cfloat, type:float, comment:null), FieldSchema(name:cdouble, type:double, comment:null), FieldSchema(name:cstring1, type:string, comment:null), FieldSchema(name:cstring2, type:string, comment:null), FieldSchema(name:ctimestamp1, type:timestamp, comment:null), FieldSchema(name:ctimestamp2, type:timestamp, comment:null), FieldSchema(name:cboolean1, type:boolean, comment:null), FieldSchema(name:cboolean2, type:boolean, comment:null)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet, inputFormat:org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe, parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"cbigint":"true","cboolean1":"true","cboolean2":"true","cdouble":"true","cfloat":"true","cint":"true","csmallint":"true","cstring1":"true","cstring2":"true","ctimestamp1":"true","ctimestamp2":"true","ctinyint":"true"}}, numFiles=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:53,890 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hive-site.xml 2018-01-05T11:28:53,891 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hivemetastore-site.xml 2018-01-05T11:28:53,891 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Unable to find config file metastore-site.xml 2018-01-05T11:28:53,891 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file null 2018-01-05T11:28:53,900 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:53,900 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: Location: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet specified for non-external table:alltypesparquet 2018-01-05T11:28:53,900 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet 2018-01-05T11:28:53,915 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,915 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,921 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,924 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,925 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,925 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@alltypesparquet) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@alltypesparquet) Type=TABLE WriteType=DDL_NO_LOCK is present 2018-01-05T11:28:53,925 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,925 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,925 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: CREATE TABLE alltypesparquet( ctinyint TINYINT, csmallint SMALLINT, cint INT, cbigint BIGINT, cfloat FLOAT, cdouble DOUBLE, cstring1 STRING, cstring2 STRING, ctimestamp1 TIMESTAMP, ctimestamp2 TIMESTAMP, cboolean1 BOOLEAN, cboolean2 BOOLEAN) STORED AS PARQUET 2018-01-05T11:28:53,925 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: CREATETABLE 2018-01-05T11:28:53,925 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: database:default 2018-01-05T11:28:53,925 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@alltypesparquet 2018-01-05T11:28:53,925 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,925 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,925 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:53,925 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {createTable_(Table, )=35, isCompatibleWith_(Configuration, )=0} 2018-01-05T11:28:53,925 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112853_6ea9a679-9267-42c2-bbbf-0b8e25d5f756); Time taken: 0.041 seconds 2018-01-05T11:28:53,925 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:53,925 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,925 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:53,925 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,925 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query CREATE TABLE alltypesparquet( ctinyint TINYINT, csmallint SMALLINT, cint INT, cbigint BIGINT, cfloat FLOAT, cdouble DOUBLE, cstring1 STRING, cstring2 STRING, ctimestamp1 TIMESTAMP, ctimestamp2 TIMESTAMP, cboolean1 BOOLEAN, cboolean2 BOOLEAN) STORED AS PARQUET 2018-01-05T11:28:53,925 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.048 seconds 2018-01-05T11:28:53,925 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:53,925 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:53,925 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:53,925 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:53,925 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,925 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:53,925 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,925 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,925 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,925 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: INSERT OVERWRITE TABLE alltypesparquet SELECT * FROM alltypesorc 2018-01-05T11:28:53,925 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112853_18452c72-dc2a-4031-8352-113fe0f24ba2): INSERT OVERWRITE TABLE alltypesparquet SELECT * FROM alltypesorc 2018-01-05T11:28:53,925 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,925 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: INSERT OVERWRITE TABLE alltypesparquet SELECT * FROM alltypesorc 2018-01-05T11:28:53,926 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:53,926 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,926 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,926 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,926 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,926 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:53,926 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:28:53,926 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Completed phase 1 of Semantic Analysis 2018-01-05T11:28:53,926 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:28:53,926 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,926 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesorc 2018-01-05T11:28:53,926 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesorc 2018-01-05T11:28:53,930 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,930 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,930 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:28:53,930 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:28:53,930 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,930 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesparquet 2018-01-05T11:28:53,931 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesparquet 2018-01-05T11:28:53,933 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,934 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,934 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Completed getting MetaData in Semantic Analysis 2018-01-05T11:28:53,936 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,937 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,937 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_not_null_constraints : db=default tbl=alltypesorc 2018-01-05T11:28:53,937 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_not_null_constraints : db=default tbl=alltypesorc 2018-01-05T11:28:53,943 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,943 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,943 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,943 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=alltypesorc 2018-01-05T11:28:53,944 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=alltypesorc 2018-01-05T11:28:53,949 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,949 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,949 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,949 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=alltypesorc 2018-01-05T11:28:53,949 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=alltypesorc 2018-01-05T11:28:53,949 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,949 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000037, QueryId=djaiswal_20180105112853_02da1873-4ec4-4565-8e4c-6f3b4f4735d5-18, DagId=18, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180533748, EndTime=1515180533949 2018-01-05T11:28:53,949 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,949 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,949 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_unique_constraints : db=default tbl=alltypesorc 2018-01-05T11:28:53,949 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_unique_constraints : db=default tbl=alltypesorc 2018-01-05T11:28:53,956 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,956 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,956 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,956 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=alltypesorc 2018-01-05T11:28:53,956 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=alltypesorc 2018-01-05T11:28:53,967 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:53,967 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:53,968 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:28:53,968 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,994 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] calcite.sql2rel: Plan after trimming unused fields HiveProject(ctinyint=[$0], csmallint=[$1], cint=[$2], cbigint=[$3], cfloat=[$4], cdouble=[$5], cstring1=[$6], cstring2=[$7], ctimestamp1=[$8], ctimestamp2=[$9], cboolean1=[$10], cboolean2=[$11]) HiveTableScan(table=[[default.alltypesorc]], table:alias=[alltypesorc]) 2018-01-05T11:28:53,995 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Plan before removing subquery: HiveProject(ctinyint=[$0], csmallint=[$1], cint=[$2], cbigint=[$3], cfloat=[$4], cdouble=[$5], cstring1=[$6], cstring2=[$7], ctimestamp1=[$8], ctimestamp2=[$9], cboolean1=[$10], cboolean2=[$11]) HiveTableScan(table=[[default.alltypesorc]], table:alias=[alltypesorc]) 2018-01-05T11:28:53,995 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Plan just after removing subquery: HiveProject(ctinyint=[$0], csmallint=[$1], cint=[$2], cbigint=[$3], cfloat=[$4], cdouble=[$5], cstring1=[$6], cstring2=[$7], ctimestamp1=[$8], ctimestamp2=[$9], cboolean1=[$10], cboolean2=[$11]) HiveTableScan(table=[[default.alltypesorc]], table:alias=[alltypesorc]) 2018-01-05T11:28:53,995 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Plan after decorrelation: HiveProject(ctinyint=[$0], csmallint=[$1], cint=[$2], cbigint=[$3], cfloat=[$4], cdouble=[$5], cstring1=[$6], cstring2=[$7], ctimestamp1=[$8], ctimestamp2=[$9], cboolean1=[$10], cboolean2=[$11]) HiveTableScan(table=[[default.alltypesorc]], table:alias=[alltypesorc]) 2018-01-05T11:28:53,995 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,996 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,996 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,996 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,996 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,996 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,996 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,996 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,996 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,997 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:53,997 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,010 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,010 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,010 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,010 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,011 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] calcite.sql2rel: Plan after trimming unused fields HiveProject(ctinyint=[$0], csmallint=[$1], cint=[$2], cbigint=[$3], cfloat=[$4], cdouble=[$5], cstring1=[$6], cstring2=[$7], ctimestamp1=[$8], ctimestamp2=[$9], cboolean1=[$10], cboolean2=[$11]) HiveTableScan(table=[[default.alltypesorc]], table:alias=[alltypesorc]) 2018-01-05T11:28:54,011 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,011 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,011 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,011 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,011 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,011 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,011 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,012 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,012 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,012 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,012 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,012 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,012 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,012 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,013 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,013 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] translator.PlanModifierForASTConv: Original plan for PlanModifier HiveProject(ctinyint=[$0], csmallint=[$1], cint=[$2], cbigint=[$3], cfloat=[$4], cdouble=[$5], cstring1=[$6], cstring2=[$7], ctimestamp1=[$8], ctimestamp2=[$9], cboolean1=[$10], cboolean2=[$11]) HiveTableScan(table=[[default.alltypesorc]], table:alias=[alltypesorc]) 2018-01-05T11:28:54,013 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] translator.PlanModifierForASTConv: Plan after nested convertOpTree HiveProject(ctinyint=[$0], csmallint=[$1], cint=[$2], cbigint=[$3], cfloat=[$4], cdouble=[$5], cstring1=[$6], cstring2=[$7], ctimestamp1=[$8], ctimestamp2=[$9], cboolean1=[$10], cboolean2=[$11]) HiveTableScan(table=[[default.alltypesorc]], table:alias=[alltypesorc]) 2018-01-05T11:28:54,013 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] translator.PlanModifierForASTConv: Plan after propagating order HiveProject(ctinyint=[$0], csmallint=[$1], cint=[$2], cbigint=[$3], cfloat=[$4], cdouble=[$5], cstring1=[$6], cstring2=[$7], ctimestamp1=[$8], ctimestamp2=[$9], cboolean1=[$10], cboolean2=[$11]) HiveTableScan(table=[[default.alltypesorc]], table:alias=[alltypesorc]) 2018-01-05T11:28:54,013 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] translator.PlanModifierForASTConv: Plan after fixTopOBSchema HiveProject(ctinyint=[$0], csmallint=[$1], cint=[$2], cbigint=[$3], cfloat=[$4], cdouble=[$5], cstring1=[$6], cstring2=[$7], ctimestamp1=[$8], ctimestamp2=[$9], cboolean1=[$10], cboolean2=[$11]) HiveTableScan(table=[[default.alltypesorc]], table:alias=[alltypesorc]) 2018-01-05T11:28:54,013 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] translator.PlanModifierForASTConv: Final plan after modifier HiveProject(alltypesorc.ctinyint=[$0], alltypesorc.csmallint=[$1], alltypesorc.cint=[$2], alltypesorc.cbigint=[$3], alltypesorc.cfloat=[$4], alltypesorc.cdouble=[$5], alltypesorc.cstring1=[$6], alltypesorc.cstring2=[$7], alltypesorc.ctimestamp1=[$8], alltypesorc.ctimestamp2=[$9], alltypesorc.cboolean1=[$10], alltypesorc.cboolean2=[$11]) HiveTableScan(table=[[default.alltypesorc]], table:alias=[alltypesorc]) 2018-01-05T11:28:54,013 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:28:54,014 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:54,014 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesorc 2018-01-05T11:28:54,014 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesorc 2018-01-05T11:28:54,017 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:54,018 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:54,018 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:28:54,018 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:28:54,018 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:54,018 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesparquet 2018-01-05T11:28:54,018 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesparquet 2018-01-05T11:28:54,020 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:54,021 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:54,021 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Table Plan for alltypesorc TS[0] 2018-01-05T11:28:54,021 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: RR before GB alltypesorc{(ctinyint,ctinyint: tinyint)(csmallint,csmallint: smallint)(cint,cint: int)(cbigint,cbigint: bigint)(cfloat,cfloat: float)(cdouble,cdouble: double)(cstring1,cstring1: string)(cstring2,cstring2: string)(ctimestamp1,ctimestamp1: timestamp)(ctimestamp2,ctimestamp2: timestamp)(cboolean1,cboolean1: boolean)(cboolean2,cboolean2: boolean)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB alltypesorc{(ctinyint,ctinyint: tinyint)(csmallint,csmallint: smallint)(cint,cint: int)(cbigint,cbigint: bigint)(cfloat,cfloat: float)(cdouble,cdouble: double)(cstring1,cstring1: string)(cstring2,cstring2: string)(ctimestamp1,ctimestamp1: timestamp)(ctimestamp2,ctimestamp2: timestamp)(cboolean1,cboolean1: boolean)(cboolean2,cboolean2: boolean)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:28:54,021 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (. (tok_table_or_col alltypesorc) ctinyint) alltypesorc.ctinyint) (tok_selexpr (. (tok_table_or_col alltypesorc) csmallint) alltypesorc.csmallint) (tok_selexpr (. (tok_table_or_col alltypesorc) cint) alltypesorc.cint) (tok_selexpr (. (tok_table_or_col alltypesorc) cbigint) alltypesorc.cbigint) (tok_selexpr (. (tok_table_or_col alltypesorc) cfloat) alltypesorc.cfloat) (tok_selexpr (. (tok_table_or_col alltypesorc) cdouble) alltypesorc.cdouble) (tok_selexpr (. (tok_table_or_col alltypesorc) cstring1) alltypesorc.cstring1) (tok_selexpr (. (tok_table_or_col alltypesorc) cstring2) alltypesorc.cstring2) (tok_selexpr (. (tok_table_or_col alltypesorc) ctimestamp1) alltypesorc.ctimestamp1) (tok_selexpr (. (tok_table_or_col alltypesorc) ctimestamp2) alltypesorc.ctimestamp2) (tok_selexpr (. (tok_table_or_col alltypesorc) cboolean1) alltypesorc.cboolean1) (tok_selexpr (. (tok_table_or_col alltypesorc) cboolean2) alltypesorc.cboolean2)) 2018-01-05T11:28:54,021 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: genSelectPlan: input = alltypesorc{(ctinyint,ctinyint: tinyint)(csmallint,csmallint: smallint)(cint,cint: int)(cbigint,cbigint: bigint)(cfloat,cfloat: float)(cdouble,cdouble: double)(cstring1,cstring1: string)(cstring2,cstring2: string)(ctimestamp1,ctimestamp1: timestamp)(ctimestamp2,ctimestamp2: timestamp)(cboolean1,cboolean1: boolean)(cboolean2,cboolean2: boolean)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} starRr = null 2018-01-05T11:28:54,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Select Plan row schema: null{(alltypesorc.ctinyint,_col0: tinyint)(alltypesorc.csmallint,_col1: smallint)(alltypesorc.cint,_col2: int)(alltypesorc.cbigint,_col3: bigint)(alltypesorc.cfloat,_col4: float)(alltypesorc.cdouble,_col5: double)(alltypesorc.cstring1,_col6: string)(alltypesorc.cstring2,_col7: string)(alltypesorc.ctimestamp1,_col8: timestamp)(alltypesorc.ctimestamp2,_col9: timestamp)(alltypesorc.cboolean1,_col10: boolean)(alltypesorc.cboolean2,_col11: boolean)} 2018-01-05T11:28:54,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:28:54,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet/.hive-staging_hive_2018-01-05_11-28-53_925_2323731700822862447-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet 2018-01-05T11:28:54,023 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet/.hive-staging_hive_2018-01-05_11-28-53_925_2323731700822862447-1 2018-01-05T11:28:54,047 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_925_2323731700822862447-1/-mr-10001 2018-01-05T11:28:54,048 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet row schema: null{(alltypesorc.ctinyint,_col0: tinyint)(alltypesorc.csmallint,_col1: smallint)(alltypesorc.cint,_col2: int)(alltypesorc.cbigint,_col3: bigint)(alltypesorc.cfloat,_col4: float)(alltypesorc.cdouble,_col5: double)(alltypesorc.cstring1,_col6: string)(alltypesorc.cstring2,_col7: string)(alltypesorc.ctimestamp1,_col8: timestamp)(alltypesorc.ctimestamp2,_col9: timestamp)(alltypesorc.cboolean1,_col10: boolean)(alltypesorc.cboolean2,_col11: boolean)} 2018-01-05T11:28:54,048 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:54,048 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesparquet 2018-01-05T11:28:54,048 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesparquet 2018-01-05T11:28:54,051 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:54,051 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:54,051 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Generate an operator pipeline to autogather column stats for table default.alltypesparquet in query INSERT OVERWRITE TABLE alltypesparquet SELECT * FROM alltypesorc 2018-01-05T11:28:54,052 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: analyze table `default`.`alltypesparquet` compute statistics for columns 2018-01-05T11:28:54,052 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:54,052 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:54,052 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesparquet 2018-01-05T11:28:54,052 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesparquet 2018-01-05T11:28:54,056 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:54,057 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:54,057 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: select compute_stats(`ctinyint`, 'hll') , compute_stats(`csmallint`, 'hll') , compute_stats(`cint`, 'hll') , compute_stats(`cbigint`, 'hll') , compute_stats(`cfloat`, 'hll') , compute_stats(`cdouble`, 'hll') , compute_stats(`cstring1`, 'hll') , compute_stats(`cstring2`, 'hll') , compute_stats(`ctimestamp1`, 'hll') , compute_stats(`ctimestamp2`, 'hll') , compute_stats(`cboolean1`, 'hll') , compute_stats(`cboolean2`, 'hll') from `default`.`alltypesparquet` 2018-01-05T11:28:54,057 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: select compute_stats(`ctinyint`, 'hll') , compute_stats(`csmallint`, 'hll') , compute_stats(`cint`, 'hll') , compute_stats(`cbigint`, 'hll') , compute_stats(`cfloat`, 'hll') , compute_stats(`cdouble`, 'hll') , compute_stats(`cstring1`, 'hll') , compute_stats(`cstring2`, 'hll') , compute_stats(`ctimestamp1`, 'hll') , compute_stats(`ctimestamp2`, 'hll') , compute_stats(`cboolean1`, 'hll') , compute_stats(`cboolean2`, 'hll') from `default`.`alltypesparquet` 2018-01-05T11:28:54,059 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:54,059 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:54,059 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:54,059 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:54,059 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:54,060 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:54,060 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:54,060 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:54,060 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:54,060 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:54,060 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:54,060 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:54,060 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:54,060 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:28:54,060 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:54,060 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesparquet 2018-01-05T11:28:54,060 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesparquet 2018-01-05T11:28:54,062 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:54,063 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:54,063 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:28:54,063 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:28:54,076 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Table Plan for alltypesparquet TS[0] 2018-01-05T11:28:54,076 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:54,076 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:54,076 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:54,076 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:54,076 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:54,077 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:54,077 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:54,077 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:54,077 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:54,077 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:54,077 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:54,077 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:54,078 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: RR before GB alltypesparquet{(ctinyint,ctinyint: tinyint)(csmallint,csmallint: smallint)(cint,cint: int)(cbigint,cbigint: bigint)(cfloat,cfloat: float)(cdouble,cdouble: double)(cstring1,cstring1: string)(cstring2,cstring2: string)(ctimestamp1,ctimestamp1: timestamp)(ctimestamp2,ctimestamp2: timestamp)(cboolean1,cboolean1: boolean)(cboolean2,cboolean2: boolean)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB {((tok_function compute_stats (tok_table_or_col ctinyint) 'hll'),_col0: struct)((tok_function compute_stats (tok_table_or_col csmallint) 'hll'),_col1: struct)((tok_function compute_stats (tok_table_or_col cint) 'hll'),_col2: struct)((tok_function compute_stats (tok_table_or_col cbigint) 'hll'),_col3: struct)((tok_function compute_stats (tok_table_or_col cfloat) 'hll'),_col4: struct)((tok_function compute_stats (tok_table_or_col cdouble) 'hll'),_col5: struct)((tok_function compute_stats (tok_table_or_col cstring1) 'hll'),_col6: struct)((tok_function compute_stats (tok_table_or_col cstring2) 'hll'),_col7: struct)((tok_function compute_stats (tok_table_or_col ctimestamp1) 'hll'),_col8: struct)((tok_function compute_stats (tok_table_or_col ctimestamp2) 'hll'),_col9: struct)((tok_function compute_stats (tok_table_or_col cboolean1) 'hll'),_col10: struct)((tok_function compute_stats (tok_table_or_col cboolean2) 'hll'),_col11: struct)} 2018-01-05T11:28:54,078 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (tok_function compute_stats (tok_table_or_col ctinyint) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col csmallint) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col cint) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col cbigint) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col cfloat) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col cdouble) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col cstring1) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col cstring2) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col ctimestamp1) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col ctimestamp2) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col cboolean1) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col cboolean2) 'hll'))) 2018-01-05T11:28:54,078 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: genSelectPlan: input = {((tok_function compute_stats (tok_table_or_col ctinyint) 'hll'),_col0: struct)((tok_function compute_stats (tok_table_or_col csmallint) 'hll'),_col1: struct)((tok_function compute_stats (tok_table_or_col cint) 'hll'),_col2: struct)((tok_function compute_stats (tok_table_or_col cbigint) 'hll'),_col3: struct)((tok_function compute_stats (tok_table_or_col cfloat) 'hll'),_col4: struct)((tok_function compute_stats (tok_table_or_col cdouble) 'hll'),_col5: struct)((tok_function compute_stats (tok_table_or_col cstring1) 'hll'),_col6: struct)((tok_function compute_stats (tok_table_or_col cstring2) 'hll'),_col7: struct)((tok_function compute_stats (tok_table_or_col ctimestamp1) 'hll'),_col8: struct)((tok_function compute_stats (tok_table_or_col ctimestamp2) 'hll'),_col9: struct)((tok_function compute_stats (tok_table_or_col cboolean1) 'hll'),_col10: struct)((tok_function compute_stats (tok_table_or_col cboolean2) 'hll'),_col11: struct)} starRr = alltypesparquet{(ctinyint,ctinyint: tinyint)(csmallint,csmallint: smallint)(cint,cint: int)(cbigint,cbigint: bigint)(cfloat,cfloat: float)(cdouble,cdouble: double)(cstring1,cstring1: string)(cstring2,cstring2: string)(ctimestamp1,ctimestamp1: timestamp)(ctimestamp2,ctimestamp2: timestamp)(cboolean1,cboolean1: boolean)(cboolean2,cboolean2: boolean)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:28:54,078 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Select Plan row schema: null{(_c0,_col0: struct)(_c1,_col1: struct)(_c2,_col2: struct)(_c3,_col3: struct)(_c4,_col4: struct)(_c5,_col5: struct)(_c6,_col6: struct)(_c7,_col7: struct)(_c8,_col8: struct)(_c9,_col9: struct)(_c10,_col10: struct)(_c11,_col11: struct)} 2018-01-05T11:28:54,078 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:28:54,078 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-54_051_544946193635317641-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-54_051_544946193635317641-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-54_051_544946193635317641-1/-mr-10000 2018-01-05T11:28:54,078 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-54_051_544946193635317641-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-54_051_544946193635317641-1 2018-01-05T11:28:54,102 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-54_051_544946193635317641-1/-mr-10002 2018-01-05T11:28:54,102 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-54_051_544946193635317641-1/-mr-10000 row schema: null{(_c0,_col0: struct)(_c1,_col1: struct)(_c2,_col2: struct)(_c3,_col3: struct)(_c4,_col4: struct)(_c5,_col5: struct)(_c6,_col6: struct)(_c7,_col7: struct)(_c8,_col8: struct)(_c9,_col9: struct)(_c10,_col10: struct)(_c11,_col11: struct)} 2018-01-05T11:28:54,102 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Body Plan for Query Block null 2018-01-05T11:28:54,102 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:28:54,102 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Body Plan for Query Block null 2018-01-05T11:28:54,102 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:28:54,102 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: CBO Succeeded; optimized logical plan. 2018-01-05T11:28:54,102 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Before logical optimization TS[0]-SEL[1]-FS[2] -SEL[1] 2018-01-05T11:28:54,103 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,103 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,103 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] lineage.Generator: Time taken for lineage transform=2 2018-01-05T11:28:54,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,105 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for FS(2) 2018-01-05T11:28:54,105 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for FS(6) 2018-01-05T11:28:54,105 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(5) 2018-01-05T11:28:54,105 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for GBY(4) 2018-01-05T11:28:54,105 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for RS(3) 2018-01-05T11:28:54,105 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for GBY(2) 2018-01-05T11:28:54,105 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(1) 2018-01-05T11:28:54,105 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(1) 2018-01-05T11:28:54,105 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:28:54,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.SimplePredicatePushDown: After PPD: TS[0]-SEL[1]-FS[2] -SEL[1] 2018-01-05T11:28:54,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 3 key:[] 2018-01-05T11:28:54,106 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 3 oldColExprMap: {VALUE._col2=Column[_col2], VALUE._col3=Column[_col3], VALUE._col4=Column[_col4], VALUE._col5=Column[_col5], VALUE._col10=Column[_col10], VALUE._col0=Column[_col0], VALUE._col11=Column[_col11], VALUE._col1=Column[_col1], VALUE._col6=Column[_col6], VALUE._col7=Column[_col7], VALUE._col8=Column[_col8], VALUE._col9=Column[_col9]} 2018-01-05T11:28:54,106 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 3 newColExprMap: {VALUE._col2=Column[_col2], VALUE._col3=Column[_col3], VALUE._col4=Column[_col4], VALUE._col5=Column[_col5], VALUE._col10=Column[_col10], VALUE._col0=Column[_col0], VALUE._col11=Column[_col11], VALUE._col1=Column[_col1], VALUE._col6=Column[_col6], VALUE._col7=Column[_col7], VALUE._col8=Column[_col8], VALUE._col9=Column[_col9]} 2018-01-05T11:28:54,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.IdentityProjectRemover: Identity project remover optimization removed : SEL[5] 2018-01-05T11:28:54,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: After logical optimization TS[0]-SEL[1]-FS[2] -SEL[1] 2018-01-05T11:28:54,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:54,107 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=alltypesorc 2018-01-05T11:28:54,107 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=alltypesorc 2018-01-05T11:28:54,110 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.216519ms + 0.006879ms, the query is [select "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "TAB_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (...)] 2018-01-05T11:28:54,111 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:54,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:54,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:54,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting stats (Num rows: 12288 Data size: 3093170 Basic stats: COMPLETE Column stats: COMPLETE) on TS[0] 2018-01-05T11:28:54,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (alltypesorc): numRows: 12288 dataSize: 3093170 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {cint= colName: cint colType: int countDistincts: 6029 numNulls: 3115 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -1073279343 max: 1073680599 ] isPrimaryKey: false isEstimated: false, ctimestamp2= colName: ctimestamp2 colType: timestamp countDistincts: 0 numNulls: 0 avgColLen: 40.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, ctimestamp1= colName: ctimestamp1 colType: timestamp countDistincts: 0 numNulls: 0 avgColLen: 40.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, csmallint= colName: csmallint colType: smallint countDistincts: 5594 numNulls: 3114 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -16379 max: 16376 ] isPrimaryKey: false isEstimated: false, cdouble= colName: cdouble colType: double countDistincts: 5569 numNulls: 3114 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: -16379.0 max: 9763215.5639 ] isPrimaryKey: false isEstimated: false, cstring2= colName: cstring2 colType: string countDistincts: 6040 numNulls: 3115 avgColLen: 12.136555989583334 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, cboolean2= colName: cboolean2 colType: boolean countDistincts: 2 numNulls: 3115 avgColLen: 4.0 numTrues: 3983 numFalses: 5190 isPrimaryKey: false isEstimated: false, cboolean1= colName: cboolean1 colType: boolean countDistincts: 2 numNulls: 3114 avgColLen: 4.0 numTrues: 6138 numFalses: 3036 isPrimaryKey: false isEstimated: false, ctinyint= colName: ctinyint colType: tinyint countDistincts: 127 numNulls: 3115 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -64 max: 62 ] isPrimaryKey: false isEstimated: false, cfloat= colName: cfloat colType: float countDistincts: 131 numNulls: 3115 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -64.0 max: 79.5530014038086 ] isPrimaryKey: false isEstimated: false, cstring1= colName: cstring1 colType: string countDistincts: 6058 numNulls: 3114 avgColLen: 10.406982421875 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, cbigint= colName: cbigint colType: bigint countDistincts: 6064 numNulls: 3115 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: -2147311592 max: 2145498388 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:54,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting stats (Num rows: 12288 Data size: 3093170 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[1] 2018-01-05T11:28:54,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-SEL[1]: numRows: 12288 dataSize: 3093170 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col9= colName: _col9 colType: timestamp countDistincts: 0 numNulls: 0 avgColLen: 40.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col11= colName: _col11 colType: boolean countDistincts: 2 numNulls: 3115 avgColLen: 4.0 numTrues: 3983 numFalses: 5190 isPrimaryKey: false isEstimated: false, _col10= colName: _col10 colType: boolean countDistincts: 2 numNulls: 3114 avgColLen: 4.0 numTrues: 6138 numFalses: 3036 isPrimaryKey: false isEstimated: false, _col0= colName: _col0 colType: tinyint countDistincts: 127 numNulls: 3115 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -64 max: 62 ] isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: int countDistincts: 6029 numNulls: 3115 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -1073279343 max: 1073680599 ] isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: smallint countDistincts: 5594 numNulls: 3114 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -16379 max: 16376 ] isPrimaryKey: false isEstimated: false, _col4= colName: _col4 colType: float countDistincts: 131 numNulls: 3115 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -64.0 max: 79.5530014038086 ] isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: bigint countDistincts: 6064 numNulls: 3115 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: -2147311592 max: 2145498388 ] isPrimaryKey: false isEstimated: false, _col6= colName: _col6 colType: string countDistincts: 6058 numNulls: 3114 avgColLen: 10.406982421875 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col5= colName: _col5 colType: double countDistincts: 5569 numNulls: 3114 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: -16379.0 max: 9763215.5639 ] isPrimaryKey: false isEstimated: false, _col8= colName: _col8 colType: timestamp countDistincts: 0 numNulls: 0 avgColLen: 40.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col7= colName: _col7 colType: string countDistincts: 6040 numNulls: 3115 avgColLen: 12.136555989583334 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:54,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-FS[2]: numRows: 12288 dataSize: 3093170 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col9= colName: _col9 colType: timestamp countDistincts: 0 numNulls: 0 avgColLen: 40.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col11= colName: _col11 colType: boolean countDistincts: 2 numNulls: 6230 avgColLen: 4.0 numTrues: 3983 numFalses: 5190 isPrimaryKey: false isEstimated: false, _col10= colName: _col10 colType: boolean countDistincts: 2 numNulls: 6228 avgColLen: 4.0 numTrues: 6138 numFalses: 3036 isPrimaryKey: false isEstimated: false, _col0= colName: _col0 colType: tinyint countDistincts: 127 numNulls: 6230 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -64 max: 62 ] isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: int countDistincts: 6029 numNulls: 6230 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -1073279343 max: 1073680599 ] isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: smallint countDistincts: 5594 numNulls: 6228 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -16379 max: 16376 ] isPrimaryKey: false isEstimated: false, _col4= colName: _col4 colType: float countDistincts: 131 numNulls: 6230 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -64.0 max: 79.5530014038086 ] isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: bigint countDistincts: 6064 numNulls: 6230 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: -2147311592 max: 2145498388 ] isPrimaryKey: false isEstimated: false, _col6= colName: _col6 colType: string countDistincts: 6058 numNulls: 6228 avgColLen: 10.406982421875 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col5= colName: _col5 colType: double countDistincts: 5569 numNulls: 6228 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: -16379.0 max: 9763215.5639 ] isPrimaryKey: false isEstimated: false, _col8= colName: _col8 colType: timestamp countDistincts: 0 numNulls: 0 avgColLen: 40.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col7= colName: _col7 colType: string countDistincts: 6040 numNulls: 6230 avgColLen: 12.136555989583334 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:54,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting stats (Num rows: 12288 Data size: 3093170 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[1] 2018-01-05T11:28:54,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-SEL[1]: numRows: 12288 dataSize: 3093170 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {cint= colName: cint colType: int countDistincts: 6029 numNulls: 3115 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -1073279343 max: 1073680599 ] isPrimaryKey: false isEstimated: false, ctimestamp2= colName: ctimestamp2 colType: timestamp countDistincts: 0 numNulls: 0 avgColLen: 40.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, ctimestamp1= colName: ctimestamp1 colType: timestamp countDistincts: 0 numNulls: 0 avgColLen: 40.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, csmallint= colName: csmallint colType: smallint countDistincts: 5594 numNulls: 3114 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -16379 max: 16376 ] isPrimaryKey: false isEstimated: false, cdouble= colName: cdouble colType: double countDistincts: 5569 numNulls: 3114 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: -16379.0 max: 9763215.5639 ] isPrimaryKey: false isEstimated: false, cstring2= colName: cstring2 colType: string countDistincts: 6040 numNulls: 3115 avgColLen: 12.136555989583334 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, cboolean2= colName: cboolean2 colType: boolean countDistincts: 2 numNulls: 3115 avgColLen: 4.0 numTrues: 3983 numFalses: 5190 isPrimaryKey: false isEstimated: false, cboolean1= colName: cboolean1 colType: boolean countDistincts: 2 numNulls: 3114 avgColLen: 4.0 numTrues: 6138 numFalses: 3036 isPrimaryKey: false isEstimated: false, ctinyint= colName: ctinyint colType: tinyint countDistincts: 127 numNulls: 3115 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -64 max: 62 ] isPrimaryKey: false isEstimated: false, cfloat= colName: cfloat colType: float countDistincts: 131 numNulls: 3115 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -64.0 max: 79.5530014038086 ] isPrimaryKey: false isEstimated: false, cstring1= colName: cstring1 colType: string countDistincts: 6058 numNulls: 3114 avgColLen: 10.406982421875 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, cbigint= colName: cbigint colType: bigint countDistincts: 6064 numNulls: 3115 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: -2147311592 max: 2145498388 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:54,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[2]: inputSize: 3093170 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:28:54,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[2] hashAgg: true 2018-01-05T11:28:54,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [Case 3] STATS-GBY[2]: cardinality: 1 2018-01-05T11:28:54,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 4816 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[2] 2018-01-05T11:28:54,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-GBY[2]: numRows: 1 dataSize: 4816 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col9= colName: _col9 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col11= colName: _col11 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 272.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col10= colName: _col10 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 272.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col4= colName: _col4 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col6= colName: _col6 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col5= colName: _col5 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col8= colName: _col8 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col7= colName: _col7 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:54,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 4816 Basic stats: COMPLETE Column stats: COMPLETE) on RS[3] 2018-01-05T11:28:54,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-RS[3]: numRows: 1 dataSize: 4816 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {VALUE._col2= colName: VALUE._col2 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col3= colName: VALUE._col3 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col4= colName: VALUE._col4 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col5= colName: VALUE._col5 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col10= colName: VALUE._col10 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 272.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col0= colName: VALUE._col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col11= colName: VALUE._col11 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 272.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col1= colName: VALUE._col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col6= colName: VALUE._col6 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col7= colName: VALUE._col7 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col8= colName: VALUE._col8 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col9= colName: VALUE._col9 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:54,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[4]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:28:54,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[4] hashAgg: false 2018-01-05T11:28:54,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [Case 9] STATS-GBY[4]: cardinality: 1 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 4944 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[4] 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-GBY[4]: numRows: 1 dataSize: 4944 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col9= colName: _col9 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col11= colName: _col11 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 272.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col10= colName: _col10 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 272.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col4= colName: _col4 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col6= colName: _col6 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col5= colName: _col5 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col8= colName: _col8 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col7= colName: _col7 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-FS[6]: numRows: 1 dataSize: 4944 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col9= colName: _col9 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col11= colName: _col11 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 272.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col10= colName: _col10 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 272.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col4= colName: _col4 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col6= colName: _col6 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col5= colName: _col5 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col8= colName: _col8 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col7= colName: _col7 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on TS[0] 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on SEL[1] 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FileSinkOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on FS[2] 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on SEL[1] 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[2] 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[3] 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[4] 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FileSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on FS[6] 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,122 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: FS, 2 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: FS, 6 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: GBY, 4 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: RS, 3 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: GBY, 2 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: SEL, 1 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: SEL, 1 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:28:54,122 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Cycle free: true 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(ctinyint: tinyint|{alltypesorc}ctinyint,csmallint: smallint|{alltypesorc}csmallint,cint: int|{alltypesorc}cint,cbigint: bigint|{alltypesorc}cbigint,cfloat: float|{alltypesorc}cfloat,cdouble: double|{alltypesorc}cdouble,cstring1: string|{alltypesorc}cstring1,cstring2: string|{alltypesorc}cstring2,ctimestamp1: timestamp|{alltypesorc}ctimestamp1,ctimestamp2: timestamp|{alltypesorc}ctimestamp2,cboolean1: boolean|{alltypesorc}cboolean1,cboolean2: boolean|{alltypesorc}cboolean2,BLOCK__OFFSET__INSIDE__FILE: bigint|{alltypesorc}block__offset__inside__file,INPUT__FILE__NAME: string|{alltypesorc}input__file__name,ROW__ID: struct|{alltypesorc}row__id) 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(_col0: tinyint|{null}alltypesorc.ctinyint,_col1: smallint|{null}alltypesorc.csmallint,_col2: int|{null}alltypesorc.cint,_col3: bigint|{null}alltypesorc.cbigint,_col4: float|{null}alltypesorc.cfloat,_col5: double|{null}alltypesorc.cdouble,_col6: string|{null}alltypesorc.cstring1,_col7: string|{null}alltypesorc.cstring2,_col8: timestamp|{null}alltypesorc.ctimestamp1,_col9: timestamp|{null}alltypesorc.ctimestamp2,_col10: boolean|{null}alltypesorc.cboolean1,_col11: boolean|{null}alltypesorc.cboolean2) 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[ctinyint] Column[csmallint] Column[cint] Column[cbigint] Column[cfloat] Column[cdouble] Column[cstring1] Column[cstring2] Column[ctimestamp1] Column[ctimestamp2] Column[cboolean1] Column[cboolean2]) 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[2] with rs:(ctinyint: tinyint|{},csmallint: smallint|{},cint: int|{},cbigint: bigint|{},cfloat: float|{},cdouble: double|{},cstring1: string|{},cstring2: string|{},ctimestamp1: timestamp|{},ctimestamp2: timestamp|{},cboolean1: boolean|{},cboolean2: boolean|{}) 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[2] 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(ctinyint: tinyint|{alltypesparquet}ctinyint,csmallint: smallint|{alltypesparquet}csmallint,cint: int|{alltypesparquet}cint,cbigint: bigint|{alltypesparquet}cbigint,cfloat: float|{alltypesparquet}cfloat,cdouble: double|{alltypesparquet}cdouble,cstring1: string|{alltypesparquet}cstring1,cstring2: string|{alltypesparquet}cstring2,ctimestamp1: timestamp|{alltypesparquet}ctimestamp1,ctimestamp2: timestamp|{alltypesparquet}ctimestamp2,cboolean1: boolean|{alltypesparquet}cboolean1,cboolean2: boolean|{alltypesparquet}cboolean2) 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0] Column[_col1] Column[_col2] Column[_col3] Column[_col4] Column[_col5] Column[_col6] Column[_col7] Column[_col8] Column[_col9] Column[_col10] Column[_col11]) 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col ctinyint) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col csmallint) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col cint) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col cbigint) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col cfloat) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col cdouble) 'hll'),_col6: struct|{}(tok_function compute_stats (tok_table_or_col cstring1) 'hll'),_col7: struct|{}(tok_function compute_stats (tok_table_or_col cstring2) 'hll'),_col8: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp1) 'hll'),_col9: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp2) 'hll'),_col10: struct|{}(tok_function compute_stats (tok_table_or_col cboolean1) 'hll'),_col11: struct|{}(tok_function compute_stats (tok_table_or_col cboolean2) 'hll')) 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col ctinyint) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col csmallint) 'hll'),VALUE._col2: struct|{}(tok_function compute_stats (tok_table_or_col cint) 'hll'),VALUE._col3: struct|{}(tok_function compute_stats (tok_table_or_col cbigint) 'hll'),VALUE._col4: struct|{}(tok_function compute_stats (tok_table_or_col cfloat) 'hll'),VALUE._col5: struct|{}(tok_function compute_stats (tok_table_or_col cdouble) 'hll'),VALUE._col6: struct|{}(tok_function compute_stats (tok_table_or_col cstring1) 'hll'),VALUE._col7: struct|{}(tok_function compute_stats (tok_table_or_col cstring2) 'hll'),VALUE._col8: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp1) 'hll'),VALUE._col9: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp2) 'hll'),VALUE._col10: struct|{}(tok_function compute_stats (tok_table_or_col cboolean1) 'hll'),VALUE._col11: struct|{}(tok_function compute_stats (tok_table_or_col cboolean2) 'hll')) 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col ctinyint) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col csmallint) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col cint) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col cbigint) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col cfloat) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col cdouble) 'hll'),_col6: struct|{}(tok_function compute_stats (tok_table_or_col cstring1) 'hll'),_col7: struct|{}(tok_function compute_stats (tok_table_or_col cstring2) 'hll'),_col8: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp1) 'hll'),_col9: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp2) 'hll'),_col10: struct|{}(tok_function compute_stats (tok_table_or_col cboolean1) 'hll'),_col11: struct|{}(tok_function compute_stats (tok_table_or_col cboolean2) 'hll')) 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{},_col2: struct|{},_col3: struct|{},_col4: struct|{},_col5: struct|{},_col6: struct|{},_col7: struct|{},_col8: struct|{},_col9: struct|{},_col10: struct|{},_col11: struct|{}) 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:54,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: TS[0] 2018-01-05T11:28:54,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: FS[2] 2018-01-05T11:28:54,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:28:54,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc of table alltypesorc 2018-01-05T11:28:54,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc 2018-01-05T11:28:54,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: FS[2] 2018-01-05T11:28:54,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: TS[0] 2018-01-05T11:28:54,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: RS[3] 2018-01-05T11:28:54,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: RS[3] 2018-01-05T11:28:54,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: GBY[4] 2018-01-05T11:28:54,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: FS[6] 2018-01-05T11:28:54,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding reduce work (Reducer 2) for GBY[4] 2018-01-05T11:28:54,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Setting up reduce sink: RS[3] with following reduce work: Reducer 2 2018-01-05T11:28:54,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Removing RS[3] as parent from GBY[4] 2018-01-05T11:28:54,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: FS[6] 2018-01-05T11:28:54,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:54,123 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesparquet 2018-01-05T11:28:54,123 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesparquet 2018-01-05T11:28:54,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:54,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:54,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:28:54,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:54,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:54,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:54,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:54,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:54,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:54,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:28:54,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping vectorization 2018-01-05T11:28:54,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:28:54,127 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: llap mode: only 2018-01-05T11:28:54,127 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(ctinyint, 'hll')' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'ctinyint' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(csmallint, 'hll')' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'csmallint' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(cint, 'hll')' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cint' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(cbigint, 'hll')' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cbigint' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(cfloat, 'hll')' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cfloat' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(cdouble, 'hll')' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cdouble' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(cstring1, 'hll')' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cstring1' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(cstring2, 'hll')' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cstring2' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(ctimestamp1, 'hll')' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'ctimestamp1' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(ctimestamp2, 'hll')' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'ctimestamp2' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(cboolean1, 'hll')' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cboolean1' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(cboolean2, 'hll')' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cboolean2' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking '_col0' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking '_col1' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking '_col2' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking '_col3' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking '_col4' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking '_col5' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking '_col6' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking '_col7' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking '_col8' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking '_col9' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking '_col10' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking '_col11' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'ctinyint' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'csmallint' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cint' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cbigint' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cfloat' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cdouble' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cstring1' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cstring2' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'ctimestamp1' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'ctimestamp2' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cboolean1' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cboolean2' 2018-01-05T11:28:54,127 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@1f74c784 can be done in LLAP 2018-01-05T11:28:54,127 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Reducer 2 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col0)' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col1)' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col1' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col2)' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col2' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col3)' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col3' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col4)' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col4' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col5)' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col5' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col6)' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col6' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col7)' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col7' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col8)' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col8' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col9)' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col9' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col10)' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col10' 2018-01-05T11:28:54,127 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col11)' 2018-01-05T11:28:54,128 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col11' 2018-01-05T11:28:54,128 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:54,128 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@7e04a93f can be done in LLAP 2018-01-05T11:28:54,128 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,128 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Completed plan generation 2018-01-05T11:28:54,128 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:54,128 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: validation start 2018-01-05T11:28:54,128 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Not a partition. 2018-01-05T11:28:54,128 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,128 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:alltypesorc.ctinyint, type:tinyint, comment:null), FieldSchema(name:alltypesorc.csmallint, type:smallint, comment:null), FieldSchema(name:alltypesorc.cint, type:int, comment:null), FieldSchema(name:alltypesorc.cbigint, type:bigint, comment:null), FieldSchema(name:alltypesorc.cfloat, type:float, comment:null), FieldSchema(name:alltypesorc.cdouble, type:double, comment:null), FieldSchema(name:alltypesorc.cstring1, type:string, comment:null), FieldSchema(name:alltypesorc.cstring2, type:string, comment:null), FieldSchema(name:alltypesorc.ctimestamp1, type:timestamp, comment:null), FieldSchema(name:alltypesorc.ctimestamp2, type:timestamp, comment:null), FieldSchema(name:alltypesorc.cboolean1, type:boolean, comment:null), FieldSchema(name:alltypesorc.cboolean2, type:boolean, comment:null)], properties:null) 2018-01-05T11:28:54,128 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,128 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:54,128 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {getNotNullConstraints_(NotNullConstraintsRequest, )=6, isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=30, flushCache_()=0, getUniqueConstraints_(UniqueConstraintsRequest, )=7, getPrimaryKeys_(PrimaryKeysRequest, )=6, getTableColumnStatistics_(String, String, List, )=13, getForeignKeys_(ForeignKeysRequest, )=11} 2018-01-05T11:28:54,128 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112853_18452c72-dc2a-4031-8352-113fe0f24ba2); Time taken: 0.203 seconds 2018-01-05T11:28:54,128 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:54,128 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,128 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112853_18452c72-dc2a-4031-8352-113fe0f24ba2): INSERT OVERWRITE TABLE alltypesparquet SELECT * FROM alltypesorc 2018-01-05T11:28:54,129 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,129 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: INSERT OVERWRITE TABLE alltypesparquet SELECT * FROM alltypesorc 2018-01-05T11:28:54,129 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:28:54,129 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@alltypesorc 2018-01-05T11:28:54,129 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@alltypesparquet 2018-01-05T11:28:54,129 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,129 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,129 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,129 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Query ID = djaiswal_20180105112853_18452c72-dc2a-4031-8352-113fe0f24ba2 2018-01-05T11:28:54,129 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Total jobs = 1 2018-01-05T11:28:54,129 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,130 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:28:54,130 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-1:MAPRED] in serial mode 2018-01-05T11:28:54,141 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_925_2323731700822862447-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:28:54,157 DEBUG [AMReporterQueueDrainer] impl.AMReporter: Removing am 10.22.8.182:49824 with last associated dag QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=7} from heartbeat with taskCount=0, amFailed=false, isDone=true 2018-01-05T11:28:54,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:28:54,163 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:28:54,163 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:28:54,163 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112853_18452c72-dc2a-4031-8352-113fe0f24ba2 2018-01-05T11:28:54,163 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Session is already open 2018-01-05T11:28:54,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:28:54,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,163 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Dag name: INSERT OVERWRITE TABLE alltype...alltypesorc (Stage-1) 2018-01-05T11:28:54,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: DagInfo: {"context":"Hive","description":"\n\nINSERT OVERWRITE TABLE alltypesparquet SELECT * FROM alltypesorc"} 2018-01-05T11:28:54,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112853_18452c72-dc2a-4031-8352-113fe0f24ba2 with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:28:54,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,175 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,175 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:28:54,176 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,176 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 2 size: 2.92KB 2018-01-05T11:28:54,189 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-54_051_544946193635317641-1/-mr-10002 2018-01-05T11:28:54,200 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-54_051_544946193635317641-1/-mr-10002 2018-01-05T11:28:54,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,230 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:28:54,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,230 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:28:54,233 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,233 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 7.27KB 2018-01-05T11:28:54,240 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_925_2323731700822862447-1/-mr-10001 2018-01-05T11:28:54,252 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_925_2323731700822862447-1/-mr-10001 2018-01-05T11:28:54,252 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc 2018-01-05T11:28:54,252 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,254 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,254 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,254 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=INSERT OVERWRITE TABLE alltype...alltypesorc (Stage-1), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112853_18452c72-dc2a-4031-8352-113fe0f24ba2 } 2018-01-05T11:28:54,255 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:54,255 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:54,258 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: INSERT OVERWRITE TABLE alltype...alltypesorc (Stage-1) 2018-01-05T11:28:54,258 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:28:54,259 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_19, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_19.dot 2018-01-05T11:28:54,259 INFO [IPC Server handler 1 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_19 2018-01-05T11:28:54,259 WARN [IPC Server handler 1 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:54,259 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Running DAG: INSERT OVERWRITE TABLE alltype...alltypesorc (Stage-1), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112853_18452c72-dc2a-4031-8352-113fe0f24ba2 } 2018-01-05T11:28:54,284 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_19][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_19, submitTime=1515180534258, queueName=null 2018-01-05T11:28:54,286 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_19_00 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_19_00 [Map 1], Default Resources= 2018-01-05T11:28:54,287 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_19_00 [Map 1] 2018-01-05T11:28:54,287 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:28:54,287 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_19_00 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:54,287 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_19_01 [Reducer 2]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_19_01 [Reducer 2], Default Resources= 2018-01-05T11:28:54,287 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_19_01 [Reducer 2] 2018-01-05T11:28:54,287 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 2 2018-01-05T11:28:54,288 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_19_01 [Reducer 2] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:54,288 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:28:54,288 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_19][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_19, initTime=1515180534288 2018-01-05T11:28:54,288 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: dag_1515180518813_0001_19 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:28:54,289 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:28:54,289 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_19][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_19, startTime=1515180534289 2018-01-05T11:28:54,289 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_19 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:28:54,289 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {alltypesorc={InputName=alltypesorc}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:28:54,289 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: alltypesorc, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:28:54,289 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_19_00 [Map 1] 2018-01-05T11:28:54,289 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_19, dagName=INSERT OVERWRITE TABLE alltype...alltypesorc (Stage-1) 2018-01-05T11:28:54,289 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,289 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,289 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,293 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_19_00 [Map 1] 2018-01-05T11:28:54,293 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_19_00 [Map 1] 2018-01-05T11:28:54,293 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:54,293 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_19_00 [Map 1] 2018-01-05T11:28:54,296 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:28:54,296 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@7051dfb2 2018-01-05T11:28:54,296 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_925_2323731700822862447-1/djaiswal/_tez_scratch_dir/6b8e6f5b-5526-401d-ad9b-cb8ea50d79c9/map.xml 2018-01-05T11:28:54,296 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_925_2323731700822862447-1/djaiswal/_tez_scratch_dir/6b8e6f5b-5526-401d-ad9b-cb8ea50d79c9/map.xml 2018-01-05T11:28:54,296 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_925_2323731700822862447-1/djaiswal/_tez_scratch_dir/6b8e6f5b-5526-401d-ad9b-cb8ea50d79c9/map.xml 2018-01-05T11:28:54,296 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:54,296 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:54,296 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:54,298 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:54,298 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 7.27KB 2018-01-05T11:28:54,298 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_19_00 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:28:54,298 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to ShuffleVertexManager for vertex_1515180518813_0001_19_01 [Reducer 2] 2018-01-05T11:28:54,298 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: alltypesorc on vertex vertex_1515180518813_0001_19_00 [Map 1] 2018-01-05T11:28:54,298 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:28:54,298 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:54,298 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:28:54,298 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:54,299 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_925_2323731700822862447-1/djaiswal/_tez_scratch_dir/6b8e6f5b-5526-401d-ad9b-cb8ea50d79c9/map.xml 2018-01-05T11:28:54,299 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:54,299 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias alltypesorc 2018-01-05T11:28:54,299 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc 2018-01-05T11:28:54,299 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc 2018-01-05T11:28:54,299 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 0,1,2,3,4,5,6,7,8,9,10,11 2018-01-05T11:28:54,299 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = ctinyint,csmallint,cint,cbigint,cfloat,cdouble,cstring1,cstring2,ctimestamp1,ctimestamp2,cboolean1,cboolean2 2018-01-05T11:28:54,299 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc 2018-01-05T11:28:54,299 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:54,299 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:54,299 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:54,299 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:54,299 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: getSplits started 2018-01-05T11:28:54,299 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: Number of buckets specified by conf file is 0 2018-01-05T11:28:54,300 INFO [InputInitializer {Map 1} #0] orc.OrcInputFormat: ORC pushdown predicate: null 2018-01-05T11:28:54,300 INFO [InputInitializer {Map 1} #0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [ctinyint, csmallint, cint, cbigint, cfloat, cdouble, cstring1, cstring2, ctimestamp1, ctimestamp2, cboolean1, cboolean2] / schema.evolution.columns.types [tinyint, smallint, int, bigint, float, double, string, string, timestamp, timestamp, boolean, boolean] (isAcidRead false) 2018-01-05T11:28:54,300 ERROR [ORC_GET_SPLITS #3] io.AcidUtils: Failed to get files with ID; using regular API: Only supported for DFS; got class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:54,300 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: Generate splits schema evolution property true reader schema struct transactional scan property false 2018-01-05T11:28:54,305 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.05 maxFrac: 0.75 auto: false desiredTaskIput: 104857600 2018-01-05T11:28:54,305 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:28:54,305 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 1 tasks for vertex: vertex_1515180518813_0001_19_01 [Reducer 2] 2018-01-05T11:28:54,305 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_19_01 [Reducer 2] 2018-01-05T11:28:54,305 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_19][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_19_01, reconfigureDoneTime=1515180534305, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:28:54,305 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_19_01 [Reducer 2], numAdditionalOutputs=1 2018-01-05T11:28:54,305 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_19][Event:VERTEX_INITIALIZED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_19_01, initRequestedTime=1515180534298, initedTime=1515180534305, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:54,305 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_19_01 [Reducer 2] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:28:54,316 DEBUG [ORC_GET_SPLITS #3] io.AcidUtils: in directory file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc base = null deltas = 0 2018-01-05T11:28:54,316 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: Split strategy: ETLSplitStrategy strategy for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc 2018-01-05T11:28:54,316 DEBUG [ORC_GET_SPLITS #4] orc.LocalCache: Serialized tail cached for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc/alltypesorc 2018-01-05T11:28:54,316 DEBUG [ORC_GET_SPLITS #4] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:28:54,317 DEBUG [ORC_GET_SPLITS #5] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:28:54,317 DEBUG [ORC_GET_SPLITS #5] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:28:54,317 INFO [InputInitializer {Map 1} #0] orc.OrcInputFormat: FooterCacheHitRatio: 1/1 2018-01-05T11:28:54,317 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc/alltypesorc, start=3, length=376777, isOriginal=true, fileLength=377237, hasFooter=false, hasBase=true, deltas=0] projected_columns_uncompressed_size: 2638742 2018-01-05T11:28:54,317 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: getSplits finished 2018-01-05T11:28:54,317 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 1 2018-01-05T11:28:54,317 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:54,317 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 1. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:28:54,317 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_925_2323731700822862447-1/djaiswal/_tez_scratch_dir/6b8e6f5b-5526-401d-ad9b-cb8ea50d79c9/map.xml 2018-01-05T11:28:54,317 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_925_2323731700822862447-1/djaiswal/_tez_scratch_dir/6b8e6f5b-5526-401d-ad9b-cb8ea50d79c9/map.xml 2018-01-05T11:28:54,317 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_925_2323731700822862447-1/djaiswal/_tez_scratch_dir/6b8e6f5b-5526-401d-ad9b-cb8ea50d79c9/map.xml 2018-01-05T11:28:54,317 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:54,317 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:54,317 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:54,319 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:54,319 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 7.27KB 2018-01-05T11:28:54,319 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc/alltypesorc to src new group? true 2018-01-05T11:28:54,319 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:28:54,319 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:28:54,319 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:28:54,319 DEBUG [InputInitializer {Map 1} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 2638742 2018-01-05T11:28:54,319 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 48865 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 2638742 Original splits: 1 2018-01-05T11:28:54,319 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Using original number of splits: 1 desired splits: 1 2018-01-05T11:28:54,319 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 1 grouped split count is 1, for bucket: 1 2018-01-05T11:28:54,319 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:28:54,319 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: alltypesorc on vertex vertex_1515180518813_0001_19_00 [Map 1] 2018-01-05T11:28:54,320 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_19_00 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:54,320 INFO [App Shared Pool - #0] impl.VertexImpl: Got updated RootInputsSpecs: {alltypesorc=forAllWorkUnits=true, update=[1]} 2018-01-05T11:28:54,321 INFO [App Shared Pool - #0] impl.VertexImpl: Vertex vertex_1515180518813_0001_19_00 [Map 1] parallelism set to 1 2018-01-05T11:28:54,321 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_19_00 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:54,321 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_19][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_19_00, initRequestedTime=1515180534289, initedTime=1515180534321, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:54,321 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_19_00 [Map 1] with distanceFromRoot: 0 2018-01-05T11:28:54,321 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_19_00 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:28:54,321 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_19][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_19_00, startRequestedTime=1515180534298, startedTime=1515180534321 2018-01-05T11:28:54,321 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_19][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_19_00, reconfigureDoneTime=1515180534321, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:28:54,321 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_19_00 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:54,321 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_19_00 for vertex: vertex_1515180518813_0001_19_01 [Reducer 2] numStartedSources: 1 numSources: 1 2018-01-05T11:28:54,321 INFO [App Shared Pool - #0] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:28:54,321 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_19_01 [Reducer 2] with distanceFromRoot: 1 2018-01-05T11:28:54,321 INFO [App Shared Pool - #0] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:28:54,321 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_19_00 [Map 1] 2018-01-05T11:28:54,321 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_19][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_19_01, startRequestedTime=1515180534321, startedTime=1515180534321 2018-01-05T11:28:54,322 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_19_01 [Reducer 2] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:54,322 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: OnVertexStarted vertex: Reducer 2 with 0 source tasks and 1 pending tasks 2018-01-05T11:28:54,322 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_19][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_19_00_000000, scheduledTime=1515180534322, launchTime=1515180534322 2018-01-05T11:28:54,322 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_19_00 [Map 1] 2018-01-05T11:28:54,322 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Map 1 in vertex: Reducer 2 numBipartiteSourceTasks: 1 2018-01-05T11:28:54,323 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_19_00_000000_0, priority=5, capability=, hosts=null 2018-01-05T11:28:54,323 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=5,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:54,323 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=5,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:54,323 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:54,323 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_19_00_000000_0 on hosts=null 2018-01-05T11:28:54,323 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_19_00_000000_0 on hosts=null 2018-01-05T11:28:54,323 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_19_00_000000_0 2018-01-05T11:28:54,323 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:54,323 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #38, task=TaskInfo{task=attempt_1515180518813_0001_19_00_000000_0, priority=5, startTime=0, containerId=null, uniqueId=37, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000038 2018-01-05T11:28:54,323 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_19_00_000000_0; false 2018-01-05T11:28:54,323 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_19_00_000000_0, priority=5, startTime=311554832, containerId=container_111101111_0001_00_000038assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=37, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:54,323 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000038 for node: 10.22.8.182:49818 2018-01-05T11:28:54,325 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 19, name=INSERT OVERWRITE TABLE alltype...alltypesorc (Stage-1), queryId=djaiswal_20180105112853_18452c72-dc2a-4031-8352-113fe0f24ba2 2018-01-05T11:28:54,325 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000038, attempt_1515180518813_0001_19_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:54,325 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000038 for node: 10.22.8.182:49818 2018-01-05T11:28:54,326 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000038 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:54,326 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000038, launchTime=1515180534326 2018-01-05T11:28:54,327 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_19_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112853_18452c72-dc2a-4031-8352-113fe0f24ba2, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000038, dagName=INSERT OVERWRITE TABLE alltype...alltypesorc (Stage-1), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{alltypesorc,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{Reducer 2,org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput,1}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180534289, firstAttemptStartTime=1515180534322, currentAttemptStartTime=1515180534325} 2018-01-05T11:28:54,327 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_19_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000038] on NM: [10.22.8.182:49818] 2018-01-05T11:28:54,327 INFO [IPC Server handler 2 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000038, QueryId=djaiswal_20180105112853_18452c72-dc2a-4031-8352-113fe0f24ba2-19, DagId=19, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180534327 2018-01-05T11:28:54,327 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:54,327 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:54,327 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_19][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_19_00_000000_0, startTime=1515180534326, containerId=container_111101111_0001_00_000038, nodeId=10.22.8.182:49818 2018-01-05T11:28:54,327 DEBUG [IPC Server handler 2 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=19} with the ShuffleHandler 2018-01-05T11:28:54,339 INFO [IPC Server handler 2 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:28:54,339 DEBUG [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/19] 2018-01-05T11:28:54,340 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:54,340 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_19_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:54,340 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_19_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180534322, dagStartTime=1515180534289, withinDagPriority=5, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:28:54,340 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_19_00_000000_0 : ACCEPTED 2018-01-05T11:28:54,340 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_19_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:54,340 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_19_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180534322, dagStartTime=1515180534289, withinDagPriority=5, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:28:54,340 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_19_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:54,340 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_19_00_000000_0 to pre-emption queue 2018-01-05T11:28:54,340 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_19_00_000000_0: true 2018-01-05T11:28:54,340 INFO [TaskCommunicator # 7] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_19_00_000000_0 2018-01-05T11:28:54,340 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:6971875 2018-01-05T11:28:54,340 INFO [TaskCommunicator # 7] GuaranteedTasks: Received response for attempt_1515180518813_0001_19_00_000000_0, true 2018-01-05T11:28:54,341 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : INSERT OVERWRITE TABLE alltype...alltypesorc (Stage-1), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_19_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=alltypesorc, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=Reducer 2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput }}, ] 2018-01-05T11:28:54,341 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1027604480, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:54,341 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_19_00_000000_0 vertexName: Map 1 2018-01-05T11:28:54,341 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000038, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_19_00_000000_0, eventCount=1 } 2018-01-05T11:28:54,341 DEBUG [TezTR-518813_1_19_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_19 queryId: djaiswal_20180105112853_18452c72-dc2a-4031-8352-113fe0f24ba2 instanceType: TaskRunner2Callable 2018-01-05T11:28:54,342 INFO [TezTR-518813_1_19_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_19_00_000000_0 2018-01-05T11:28:54,342 DEBUG [TezTR-518813_1_19_0_0_0] log.PerfLogger: 2018-01-05T11:28:54,342 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:54,343 INFO [I/O Setup 1 Initialize: {Reducer 2}] writers.UnorderedPartitionedKVWriter: Requested BufferSize (tez.runtime.unordered.output.buffer.size-mb) : 100 2018-01-05T11:28:54,344 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:54,347 INFO [IPC Server handler 18 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_19_00_000000_0 sent events: (0-1). 2018-01-05T11:28:54,347 DEBUG [TezTR-518813_1_19_0_0_0] log.PerfLogger: 2018-01-05T11:28:54,347 INFO [TezTR-518813_1_19_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:54,347 INFO [TezTR-518813_1_19_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:54,347 INFO [TezTR-518813_1_19_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:54,347 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:28:54,347 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_19_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:28:54,347 INFO [I/O Setup 0 Initialize: {alltypesorc}] input.MRInput: alltypesorc using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:28:54,347 INFO [I/O Setup 0 Initialize: {alltypesorc}] input.MRInputLegacy: alltypesorc MRInputLegacy deferring initialization 2018-01-05T11:28:54,347 INFO [TezTR-518813_1_19_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:54,347 INFO [TezTR-518813_1_19_0_0_0] resources.MemoryDistributor: InitialRequests=[alltypesorc:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [Reducer 2:OUTPUT:104857600:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput] 2018-01-05T11:28:54,347 INFO [TezTR-518813_1_19_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:54,347 INFO [TezTR-518813_1_19_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:54,348 INFO [TezTR-518813_1_19_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 104857600, TotalRequestedScaled: 5.24288E7, TotalJVMHeap: 1027604480, TotalAvailable: 688495001, TotalRequested/TotalJVMHeap:0.10 2018-01-05T11:28:54,348 INFO [TezTR-518813_1_19_0_0_0] resources.MemoryDistributor: Allocations=[alltypesorc:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput:OUTPUT:104857600:104857600] 2018-01-05T11:28:54,348 INFO [TezTR-518813_1_19_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:54,348 INFO [TezTR-518813_1_19_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: alltypesorc being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:28:54,348 INFO [TezTR-518813_1_19_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:28:54,348 INFO [TezTR-518813_1_19_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:28:54,348 INFO [I/O Setup 1 Start: {alltypesorc}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: alltypesorc 2018-01-05T11:28:54,348 INFO [TezTR-518813_1_19_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:54,348 INFO [TezTR-518813_1_19_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_19_00_000000_0 2018-01-05T11:28:54,348 DEBUG [TezTR-518813_1_19_0_0_0] log.PerfLogger: 2018-01-05T11:28:54,348 DEBUG [TezTR-518813_1_19_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_19_00_000000_0_10112 2018-01-05T11:28:54,348 DEBUG [TezTR-518813_1_19_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:54,348 DEBUG [TezTR-518813_1_19_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_19_00_000000_0 2018-01-05T11:28:54,348 INFO [TezTR-518813_1_19_0_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112853_18452c72-dc2a-4031-8352-113fe0f24ba2 2018-01-05T11:28:54,348 DEBUG [TezTR-518813_1_19_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:54,348 DEBUG [TezTR-518813_1_19_0_0_0] log.PerfLogger: 2018-01-05T11:28:54,348 DEBUG [TezTR-518813_1_19_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:54,348 DEBUG [TezTR-518813_1_19_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:54,348 DEBUG [TezTR-518813_1_19_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112853_18452c72-dc2a-4031-8352-113fe0f24ba2_Map 1__MAP_PLAN__ 2018-01-05T11:28:54,349 INFO [TezTR-518813_1_19_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_925_2323731700822862447-1/djaiswal/_tez_scratch_dir/6b8e6f5b-5526-401d-ad9b-cb8ea50d79c9/map.xml 2018-01-05T11:28:54,349 DEBUG [TezTR-518813_1_19_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_925_2323731700822862447-1/djaiswal/_tez_scratch_dir/6b8e6f5b-5526-401d-ad9b-cb8ea50d79c9/map.xml 2018-01-05T11:28:54,349 DEBUG [TezTR-518813_1_19_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_925_2323731700822862447-1/djaiswal/_tez_scratch_dir/6b8e6f5b-5526-401d-ad9b-cb8ea50d79c9/map.xml 2018-01-05T11:28:54,349 DEBUG [TezTR-518813_1_19_0_0_0] log.PerfLogger: 2018-01-05T11:28:54,349 INFO [TezTR-518813_1_19_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:54,349 DEBUG [TezTR-518813_1_19_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:54,350 DEBUG [TezTR-518813_1_19_0_0_0] log.PerfLogger: 2018-01-05T11:28:54,350 INFO [TezTR-518813_1_19_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 7.27KB 2018-01-05T11:28:54,351 DEBUG [TezTR-518813_1_19_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:54,351 INFO [TezTR-518813_1_19_0_0_0] tez.MapRecordProcessor: The input names are: [alltypesorc] 2018-01-05T11:28:54,351 INFO [TezTR-518813_1_19_0_0_0] input.MRInput: Initialized MRInput: alltypesorc 2018-01-05T11:28:54,351 DEBUG [TezTR-518813_1_19_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 1 2018-01-05T11:28:54,351 INFO [TezTR-518813_1_19_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_925_2323731700822862447-1/djaiswal/_tez_scratch_dir/6b8e6f5b-5526-401d-ad9b-cb8ea50d79c9/map.xml 2018-01-05T11:28:54,351 DEBUG [TezTR-518813_1_19_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:54,351 DEBUG [TezTR-518813_1_19_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc/alltypesorc PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={columns.types=tinyint:smallint:int:bigint:float:double:string:string:timestamp:timestamp:boolean:boolean, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc, columns=ctinyint,csmallint,cint,cbigint,cfloat,cdouble,cstring1,cstring2,ctimestamp1,ctimestamp2,cboolean1,cboolean2, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"cbigint":"true","cboolean1":"true","cboolean2":"true","cdouble":"true","cfloat":"true","cint":"true","csmallint":"true","cstring1":"true","cstring2":"true","ctimestamp1":"true","ctimestamp2":"true","ctinyint":"true"}}, serialization.format=1, numRows=12288, numFiles=1, serialization.ddl=struct alltypesorc { byte ctinyint, i16 csmallint, i32 cint, i64 cbigint, float cfloat, double cdouble, string cstring1, string cstring2, timestamp ctimestamp1, timestamp ctimestamp2, bool cboolean1, bool cboolean2}, transient_lastDdlTime=1515180533, rawDataSize=2641964, columns.comments=, totalSize=377237, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, column.name.delimiter=,, name=default.alltypesorc}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={columns.types=tinyint:smallint:int:bigint:float:double:string:string:timestamp:timestamp:boolean:boolean, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc, columns=ctinyint,csmallint,cint,cbigint,cfloat,cdouble,cstring1,cstring2,ctimestamp1,ctimestamp2,cboolean1,cboolean2, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"cbigint":"true","cboolean1":"true","cboolean2":"true","cdouble":"true","cfloat":"true","cint":"true","csmallint":"true","cstring1":"true","cstring2":"true","ctimestamp1":"true","ctimestamp2":"true","ctinyint":"true"}}, serialization.format=1, numRows=12288, numFiles=1, serialization.ddl=struct alltypesorc { byte ctinyint, i16 csmallint, i32 cint, i64 cbigint, float cfloat, double cdouble, string cstring1, string cstring2, timestamp ctimestamp1, timestamp ctimestamp2, bool cboolean1, bool cboolean2}, transient_lastDdlTime=1515180533, rawDataSize=2641964, columns.comments=, totalSize=377237, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, column.name.delimiter=,, name=default.alltypesorc}, baseFileName=alltypesorc, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={columns.types=tinyint:smallint:int:bigint:float:double:string:string:timestamp:timestamp:boolean:boolean, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc, columns=ctinyint,csmallint,cint,cbigint,cfloat,cdouble,cstring1,cstring2,ctimestamp1,ctimestamp2,cboolean1,cboolean2, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"cbigint":"true","cboolean1":"true","cboolean2":"true","cdouble":"true","cfloat":"true","cint":"true","csmallint":"true","cstring1":"true","cstring2":"true","ctimestamp1":"true","ctimestamp2":"true","ctinyint":"true"}}, serialization.format=1, numRows=12288, numFiles=1, serialization.ddl=struct alltypesorc { byte ctinyint, i16 csmallint, i32 cint, i64 cbigint, float cfloat, double cdouble, string cstring1, string cstring2, timestamp ctimestamp1, timestamp ctimestamp2, bool cboolean1, bool cboolean2}, transient_lastDdlTime=1515180533, rawDataSize=2641964, columns.comments=, totalSize=377237, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, column.name.delimiter=,, name=default.alltypesorc}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={columns.types=tinyint:smallint:int:bigint:float:double:string:string:timestamp:timestamp:boolean:boolean, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc, columns=ctinyint,csmallint,cint,cbigint,cfloat,cdouble,cstring1,cstring2,ctimestamp1,ctimestamp2,cboolean1,cboolean2, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"cbigint":"true","cboolean1":"true","cboolean2":"true","cdouble":"true","cfloat":"true","cint":"true","csmallint":"true","cstring1":"true","cstring2":"true","ctimestamp1":"true","ctimestamp2":"true","ctinyint":"true"}}, serialization.format=1, numRows=12288, numFiles=1, serialization.ddl=struct alltypesorc { byte ctinyint, i16 csmallint, i32 cint, i64 cbigint, float cfloat, double cdouble, string cstring1, string cstring2, timestamp ctimestamp1, timestamp ctimestamp2, bool cboolean1, bool cboolean2}, transient_lastDdlTime=1515180533, rawDataSize=2641964, columns.comments=, totalSize=377237, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, column.name.delimiter=,, name=default.alltypesorc}, baseFileName=alltypesorc, vectorPartitionDesc=null]} 2018-01-05T11:28:54,351 DEBUG [TezTR-518813_1_19_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.hive.ql.io.orc.OrcInputFormat 2018-01-05T11:28:54,353 DEBUG [TezTR-518813_1_19_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:54,353 INFO [TezTR-518813_1_19_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_925_2323731700822862447-1/djaiswal/_tez_scratch_dir/6b8e6f5b-5526-401d-ad9b-cb8ea50d79c9/map.xml 2018-01-05T11:28:54,353 DEBUG [TezTR-518813_1_19_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:54,353 INFO [TezTR-518813_1_19_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_19_00_000000_0 2018-01-05T11:28:54,353 INFO [TezTR-518813_1_19_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [ctinyint, csmallint, cint, cbigint, cfloat, cdouble, cstring1, cstring2, ctimestamp1, ctimestamp2, cboolean1, cboolean2] / schema.evolution.columns.types [tinyint, smallint, int, bigint, float, double, string, string, timestamp, timestamp, boolean, boolean] (isAcidRead false) 2018-01-05T11:28:54,353 DEBUG [TezTR-518813_1_19_0_0_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:54,353 DEBUG [TezTR-518813_1_19_0_0_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:54,353 DEBUG [TezTR-518813_1_19_0_0_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:54,353 DEBUG [TezTR-518813_1_19_0_0_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:54,353 DEBUG [TezTR-518813_1_19_0_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:28:54,353 DEBUG [TezTR-518813_1_19_0_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:28:54,354 INFO [TezTR-518813_1_19_0_0_0] LlapIoImpl: Using batch-to-row converter for split: OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc/alltypesorc, start=3, length=376777, isOriginal=true, fileLength=377237, hasFooter=false, hasBase=true, deltas=0] 2018-01-05T11:28:54,354 DEBUG [TezTR-518813_1_19_0_0_0] io.BatchToRowReader: Including the columns [1111111111110] 2018-01-05T11:28:54,354 DEBUG [TezTR-518813_1_19_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc/alltypesorc 2018-01-05T11:28:54,354 INFO [TezTR-518813_1_19_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc/alltypesorc, start=3, length=376777, isOriginal=true, fileLength=377237, hasFooter=false, hasBase=true, deltas=0]], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=376777} 2018-01-05T11:28:54,354 INFO [TezTR-518813_1_19_0_0_0] input.MRInput: alltypesorc initialized RecordReader from event 2018-01-05T11:28:54,354 DEBUG [TezTR-518813_1_19_0_0_0] tez.MapRecordProcessor: Starting Output: Reducer 2 2018-01-05T11:28:54,354 DEBUG [IO-Elevator-Thread-1] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_19 queryId: djaiswal_20180105112853_18452c72-dc2a-4031-8352-113fe0f24ba2 instanceType: OrcEncodedDataReader 2018-01-05T11:28:54,354 INFO [IO-Elevator-Thread-1] LlapIoImpl: Processing data for file [-2724874657402502449, 1515180533000, 377237]: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc/alltypesorc 2018-01-05T11:28:54,354 INFO [TezTR-518813_1_19_0_0_0] writers.BaseUnorderedPartitionedKVWriter: Instantiating Partitioner: [org.apache.tez.mapreduce.partition.MRPartitioner] 2018-01-05T11:28:54,354 WARN [IO-Elevator-Thread-1] encoded.OrcEncodedDataReader: ORC compression buffer size (10000) is smaller than LLAP low-level cache minimum allocation size (16384). Decrease the value for hive.llap.io.allocator.alloc.min to avoid wasting memory 2018-01-05T11:28:54,354 DEBUG [IO-Elevator-Thread-1] LlapIoOrc: FileSplit {3, 376777}; stripes {3, 106930}, {200000, 114466}, {314466, 62314}, 2018-01-05T11:28:54,360 DEBUG [IO-Elevator-Thread-1] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:28:54,360 DEBUG [IO-Elevator-Thread-1] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:28:54,360 INFO [IO-Elevator-Thread-1] impl.OrcCodecPool: Got brand-new codec ZLIB 2018-01-05T11:28:54,360 INFO [IO-Elevator-Thread-1] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [-2724874657402502449, 1515180533000, 377237]): [{range start: 0 end: 402}] 2018-01-05T11:28:54,361 INFO [IO-Elevator-Thread-1] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [-2724874657402502449, 1515180533000, 377237], base offset 3): [{start: 0 end: 10 cache buffer: 0x1390fc39(1)}, {start: 10 end: 37 cache buffer: 0x527853ba(1)}, {start: 37 end: 68 cache buffer: 0x58040013(1)}, {start: 68 end: 107 cache buffer: 0x11cd496c(1)}, {start: 107 end: 146 cache buffer: 0xa00fcb6(1)}, {start: 146 end: 186 cache buffer: 0x1a758bcf(1)}, {start: 186 end: 230 cache buffer: 0x6209140a(1)}, {start: 230 end: 287 cache buffer: 0x19f4b70e(1)}, {start: 287 end: 326 cache buffer: 0x149c1dca(1)}, {start: 326 end: 342 cache buffer: 0x2201d765(1)}, {start: 342 end: 358 cache buffer: 0x8823e6b(1)}, {start: 358 end: 380 cache buffer: 0x3472b4b2(1)}, {start: 380 end: 402 cache buffer: 0x40749dbb(1)}] 2018-01-05T11:28:54,361 INFO [IO-Elevator-Thread-1] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [-2724874657402502449, 1515180533000, 377237]): [{range start: 402 end: 106715}] 2018-01-05T11:28:54,362 INFO [IO-Elevator-Thread-1] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [-2724874657402502449, 1515180533000, 377237], base offset 3): [{start: 402 end: 447 cache buffer: 0x4a98ac6f(1)}, {start: 447 end: 3079 cache buffer: 0x69e440b8(1)}, {start: 3079 end: 3119 cache buffer: 0x5c91f2f2(1)}, {start: 3119 end: 8653 cache buffer: 0x63a0c629(1)}, {start: 8653 end: 8694 cache buffer: 0x6cd4e5e6(1)}, {start: 8694 end: 16869 cache buffer: 0x27fe9944(1)}, {start: 16869 end: 16910 cache buffer: 0x74abf92b(1)}, {start: 16910 end: 25069 cache buffer: 0x115fe819(1)}, {start: 25069 end: 25114 cache buffer: 0x5aefc401(1)}, {start: 25114 end: 28191 cache buffer: 0x90842a1(1)}, {start: 28191 end: 29632 cache buffer: 0x2241900a(1)}, {start: 29632 end: 29672 cache buffer: 0x4cb0bb7b(1)}, {start: 29672 end: 32857 cache buffer: 0x685bb120(1)}, {start: 32857 end: 34346 cache buffer: 0x238debc0(1)}, {start: 34346 end: 38180 cache buffer: 0xde2fe7b(1)}, {start: 38180 end: 38905 cache buffer: 0x4440f7df(1)}, {start: 38905 end: 38945 cache buffer: 0x71278cc6(1)}, {start: 38945 end: 41773 cache buffer: 0x3d38bac9(1)}, {start: 41773 end: 43060 cache buffer: 0x27339a33(1)}, {start: 43060 end: 50328 cache buffer: 0x640e7f05(1)}, {start: 50328 end: 57659 cache buffer: 0x29028efe(1)}, {start: 57659 end: 64332 cache buffer: 0x519497e5(1)}, {start: 64332 end: 64373 cache buffer: 0x6a4d7de6(1)}, {start: 64373 end: 67190 cache buffer: 0x7dacd1fd(1)}, {start: 67190 end: 68477 cache buffer: 0x40e4d5c8(1)}, {start: 68477 end: 75753 cache buffer: 0x353a498e(1)}, {start: 75753 end: 83091 cache buffer: 0x3499b082(1)}, {start: 83091 end: 89706 cache buffer: 0x2fa1bcd5(1)}, {start: 89706 end: 89749 cache buffer: 0x498d767c(1)}, {start: 89749 end: 92570 cache buffer: 0x2e9c20fa(1)}, {start: 92570 end: 93120 cache buffer: 0x57f8880b(1)}, {start: 93120 end: 97885 cache buffer: 0x751dc135(1)}, {start: 97885 end: 97925 cache buffer: 0x5781677c(1)}, {start: 97925 end: 100750 cache buffer: 0x18551686(1)}, {start: 100750 end: 101314 cache buffer: 0xbfe6d57(1)}, {start: 101314 end: 106102 cache buffer: 0x10710174(1)}, {start: 106102 end: 106142 cache buffer: 0x4c173e81(1)}, {start: 106142 end: 106409 cache buffer: 0x7cc6e0a3(1)}, {start: 106409 end: 106450 cache buffer: 0x42abeb9(1)}, {start: 106450 end: 106715 cache buffer: 0x1cc875e9(1)}] 2018-01-05T11:28:54,362 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: tinyint streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 2964 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:54,362 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 2 columnType: smallint streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 5531 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:54,362 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 3 columnType: int streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 8172 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:54,362 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 4 columnType: bigint streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 8156 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:54,363 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 5 columnType: float streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 10000 bytes, LlapDataBuffer with 5796 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:54,363 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 6 columnType: double streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 10000 bytes, LlapDataBuffer with 10000 bytes, LlapDataBuffer with 10000 bytes, LlapDataBuffer with 1696 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:54,363 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 7 columnType: string streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DICTIONARY_V2 dictionarySize: 2031 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 2825 bytes, ], indexBaseOffset=0] dictionary: true lengths: true secondary: false tz: 2018-01-05T11:28:54,363 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 8 columnType: string streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DICTIONARY_V2 dictionarySize: 2029 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 2814 bytes, ], indexBaseOffset=0] dictionary: true lengths: true secondary: false tz: 2018-01-05T11:28:54,363 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 9 columnType: timestamp streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 10000 bytes, LlapDataBuffer with 1713 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: true tz: 2018-01-05T11:28:54,363 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 10 columnType: timestamp streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 10000 bytes, LlapDataBuffer with 1765 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: true tz: 2018-01-05T11:28:54,364 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 11 columnType: boolean streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 264 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:54,364 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 12 columnType: boolean streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 262 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:54,371 DEBUG [IO-Elevator-Thread-1] LlapIoOrc: Done with decode 2018-01-05T11:28:54,372 INFO [IO-Elevator-Thread-1] impl.OrcCodecPool: Got brand-new codec ZLIB 2018-01-05T11:28:54,372 INFO [IO-Elevator-Thread-1] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [-2724874657402502449, 1515180533000, 377237]): [{range start: 0 end: 415}] 2018-01-05T11:28:54,372 INFO [IO-Elevator-Thread-1] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [-2724874657402502449, 1515180533000, 377237], base offset 200000): [{start: 0 end: 10 cache buffer: 0x1fea429e(1)}, {start: 10 end: 35 cache buffer: 0x283a61a3(1)}, {start: 35 end: 67 cache buffer: 0x4db6f366(1)}, {start: 67 end: 106 cache buffer: 0xd60ba4a(1)}, {start: 106 end: 145 cache buffer: 0x40d3e7ca(1)}, {start: 145 end: 185 cache buffer: 0x3b9fb10a(1)}, {start: 185 end: 229 cache buffer: 0x414848dc(1)}, {start: 229 end: 295 cache buffer: 0x49721855(1)}, {start: 295 end: 339 cache buffer: 0x16c05b06(1)}, {start: 339 end: 355 cache buffer: 0x5091bf35(1)}, {start: 355 end: 371 cache buffer: 0x539c0353(1)}, {start: 371 end: 393 cache buffer: 0xa7519c5(1)}, {start: 393 end: 415 cache buffer: 0x1ef97be6(1)}] 2018-01-05T11:28:54,372 INFO [IO-Elevator-Thread-1] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [-2724874657402502449, 1515180533000, 377237]): [{range start: 415 end: 114240}] 2018-01-05T11:28:54,373 INFO [IO-Elevator-Thread-1] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [-2724874657402502449, 1515180533000, 377237], base offset 200000): [{start: 415 end: 459 cache buffer: 0x707912aa(1)}, {start: 459 end: 3099 cache buffer: 0x52e09651(1)}, {start: 3099 end: 3141 cache buffer: 0x713f1a0(1)}, {start: 3141 end: 7215 cache buffer: 0x24c4ca86(1)}, {start: 7215 end: 7257 cache buffer: 0x1c9070f6(1)}, {start: 7257 end: 15428 cache buffer: 0x4a943f2b(1)}, {start: 15428 end: 15472 cache buffer: 0x52125928(1)}, {start: 15472 end: 25475 cache buffer: 0x50003916(1)}, {start: 25475 end: 26738 cache buffer: 0x4f88f420(1)}, {start: 26738 end: 26782 cache buffer: 0x240ef79d(1)}, {start: 26782 end: 29159 cache buffer: 0x6415212d(1)}, {start: 29159 end: 31394 cache buffer: 0x350c6c30(1)}, {start: 31394 end: 31436 cache buffer: 0x3d2da745(1)}, {start: 31436 end: 32254 cache buffer: 0x6e4c1d62(1)}, {start: 32254 end: 36061 cache buffer: 0x239d45cf(1)}, {start: 36061 end: 38221 cache buffer: 0x7526d060(1)}, {start: 38221 end: 38246 cache buffer: 0x25e1460e(1)}, {start: 38246 end: 38287 cache buffer: 0x2b48f8dd(1)}, {start: 38287 end: 41110 cache buffer: 0x62b26346(1)}, {start: 41110 end: 42402 cache buffer: 0x21540aa0(1)}, {start: 42402 end: 49670 cache buffer: 0xe2af9b2(1)}, {start: 49670 end: 57019 cache buffer: 0x624866ba(1)}, {start: 57019 end: 63645 cache buffer: 0x139181e6(1)}, {start: 63645 end: 63688 cache buffer: 0x1291983e(1)}, {start: 63688 end: 67925 cache buffer: 0x4b2f04b7(1)}, {start: 67925 end: 69705 cache buffer: 0x6af3d6de(1)}, {start: 69705 end: 76980 cache buffer: 0x7572f4d7(1)}, {start: 76980 end: 84288 cache buffer: 0x6b3cc22f(1)}, {start: 84288 end: 91632 cache buffer: 0x58857943(1)}, {start: 91632 end: 98964 cache buffer: 0x2c3401ab(1)}, {start: 98964 end: 99335 cache buffer: 0x4e029cb6(1)}, {start: 99335 end: 99382 cache buffer: 0x43a212e3(1)}, {start: 99382 end: 102169 cache buffer: 0x301e34b5(1)}, {start: 102169 end: 102731 cache buffer: 0x3cab4244(1)}, {start: 102731 end: 107509 cache buffer: 0x66968e4b(1)}, {start: 107509 end: 107554 cache buffer: 0x35f0ad70(1)}, {start: 107554 end: 109994 cache buffer: 0x740a9ce7(1)}, {start: 109994 end: 113528 cache buffer: 0x12d55940(1)}, {start: 113528 end: 113569 cache buffer: 0x40637987(1)}, {start: 113569 end: 113836 cache buffer: 0x70c8d7d6(1)}, {start: 113836 end: 113879 cache buffer: 0x5085808c(1)}, {start: 113879 end: 114240 cache buffer: 0x68a2cc85(1)}] 2018-01-05T11:28:54,373 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: tinyint streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 2976 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:54,373 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 2 columnType: smallint streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 4071 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:54,373 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 3 columnType: int streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 8168 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:54,373 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 4 columnType: bigint streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 10000 bytes, LlapDataBuffer with 1260 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:54,373 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 5 columnType: float streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 10000 bytes, LlapDataBuffer with 5840 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:54,374 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 6 columnType: double streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 10000 bytes, LlapDataBuffer with 10000 bytes, LlapDataBuffer with 10000 bytes, LlapDataBuffer with 1696 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:54,374 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 7 columnType: string streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DICTIONARY_V2 dictionarySize: 2032 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 2820 bytes, ], indexBaseOffset=0] dictionary: true lengths: true secondary: false tz: 2018-01-05T11:28:54,374 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 8 columnType: string streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DICTIONARY_V2 dictionarySize: 2800 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 4234 bytes, ], indexBaseOffset=0] dictionary: true lengths: true secondary: false tz: 2018-01-05T11:28:54,374 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 9 columnType: timestamp streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 10000 bytes, LlapDataBuffer with 1774 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: true tz: 2018-01-05T11:28:54,374 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 10 columnType: timestamp streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 8654 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: true tz: 2018-01-05T11:28:54,375 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 11 columnType: boolean streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 264 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:54,375 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 12 columnType: boolean streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 358 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:54,379 INFO [TezTR-518813_1_19_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: numBuffers=2, sizePerBuffer=52428800, skipBuffers=true, numPartitions=1, availableMemory=104857600, maxSingleBufferSizeBytes=2147483647, pipelinedShuffle=false, isFinalMergeEnabled=true, numPartitions=1, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:28:54,379 INFO [TezTR-518813_1_19_0_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:28:54,379 INFO [TezTR-518813_1_19_0_0_0] exec.MapOperator: Initializing operator MAP[0] 2018-01-05T11:28:54,379 DEBUG [TezTR-518813_1_19_0_0_0] exec.MapOperator: Initialization Done 0 MAP 2018-01-05T11:28:54,379 DEBUG [TezTR-518813_1_19_0_0_0] exec.MapOperator: Operator 0 MAP initialized 2018-01-05T11:28:54,379 DEBUG [TezTR-518813_1_19_0_0_0] exec.MapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:28:54,380 DEBUG [TezTR-518813_1_19_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:54,380 DEBUG [TezTR-518813_1_19_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:54,380 DEBUG [TezTR-518813_1_19_0_0_0] exec.MapOperator: Adding alias alltypesorc to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc 2018-01-05T11:28:54,380 DEBUG [TezTR-518813_1_19_0_0_0] exec.MapOperator: dump TS[0] struct 2018-01-05T11:28:54,380 INFO [TezTR-518813_1_19_0_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:28:54,380 INFO [TezTR-518813_1_19_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:54,380 DEBUG [TezTR-518813_1_19_0_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:28:54,380 INFO [TezTR-518813_1_19_0_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:28:54,380 DEBUG [TezTR-518813_1_19_0_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:28:54,380 DEBUG [TezTR-518813_1_19_0_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:28:54,380 DEBUG [TezTR-518813_1_19_0_0_0] exec.TableScanOperator: Initializing children of 0 TS 2018-01-05T11:28:54,380 DEBUG [TezTR-518813_1_19_0_0_0] exec.SelectOperator: Initializing child 1 SEL 2018-01-05T11:28:54,380 INFO [TezTR-518813_1_19_0_0_0] exec.SelectOperator: Initializing operator SEL[1] 2018-01-05T11:28:54,380 INFO [TezTR-518813_1_19_0_0_0] exec.SelectOperator: SELECT struct 2018-01-05T11:28:54,381 DEBUG [TezTR-518813_1_19_0_0_0] exec.SelectOperator: Initialization Done 1 SEL 2018-01-05T11:28:54,381 DEBUG [TezTR-518813_1_19_0_0_0] exec.SelectOperator: Operator 1 SEL initialized 2018-01-05T11:28:54,381 DEBUG [TezTR-518813_1_19_0_0_0] exec.SelectOperator: Initializing children of 1 SEL 2018-01-05T11:28:54,381 DEBUG [TezTR-518813_1_19_0_0_0] exec.FileSinkOperator: Initializing child 2 FS 2018-01-05T11:28:54,381 INFO [TezTR-518813_1_19_0_0_0] exec.FileSinkOperator: Initializing operator FS[2] 2018-01-05T11:28:54,381 DEBUG [TezTR-518813_1_19_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:54,381 INFO [TezTR-518813_1_19_0_0_0] exec.FileSinkOperator: Using serializer : org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe@1aa8777 and formatter : org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat@3604819a 2018-01-05T11:28:54,381 DEBUG [TezTR-518813_1_19_0_0_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:28:54,381 DEBUG [TezTR-518813_1_19_0_0_0] exec.FileSinkOperator: Initialization Done 2 FS 2018-01-05T11:28:54,381 DEBUG [TezTR-518813_1_19_0_0_0] exec.FileSinkOperator: Operator 2 FS initialized 2018-01-05T11:28:54,381 DEBUG [TezTR-518813_1_19_0_0_0] exec.FileSinkOperator: Initialization Done 2 FS done is reset. 2018-01-05T11:28:54,381 DEBUG [TezTR-518813_1_19_0_0_0] exec.SelectOperator: Initializing child 1 SEL 2018-01-05T11:28:54,381 INFO [TezTR-518813_1_19_0_0_0] exec.SelectOperator: Initializing operator SEL[1] 2018-01-05T11:28:54,381 INFO [TezTR-518813_1_19_0_0_0] exec.SelectOperator: SELECT struct<_col0:tinyint,_col1:smallint,_col2:int,_col3:bigint,_col4:float,_col5:double,_col6:string,_col7:string,_col8:timestamp,_col9:timestamp,_col10:boolean,_col11:boolean> 2018-01-05T11:28:54,381 DEBUG [TezTR-518813_1_19_0_0_0] exec.SelectOperator: Initialization Done 1 SEL 2018-01-05T11:28:54,381 DEBUG [TezTR-518813_1_19_0_0_0] exec.SelectOperator: Operator 1 SEL initialized 2018-01-05T11:28:54,381 DEBUG [TezTR-518813_1_19_0_0_0] exec.SelectOperator: Initializing children of 1 SEL 2018-01-05T11:28:54,381 DEBUG [TezTR-518813_1_19_0_0_0] exec.GroupByOperator: Initializing child 2 GBY 2018-01-05T11:28:54,381 INFO [TezTR-518813_1_19_0_0_0] exec.GroupByOperator: Initializing operator GBY[2] 2018-01-05T11:28:54,381 DEBUG [TezTR-518813_1_19_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:54,381 DEBUG [TezTR-518813_1_19_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:54,381 DEBUG [TezTR-518813_1_19_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:54,381 DEBUG [TezTR-518813_1_19_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:54,381 DEBUG [TezTR-518813_1_19_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:54,381 DEBUG [TezTR-518813_1_19_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:54,381 DEBUG [TezTR-518813_1_19_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:54,381 DEBUG [TezTR-518813_1_19_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:54,381 DEBUG [TezTR-518813_1_19_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:54,381 DEBUG [TezTR-518813_1_19_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:54,381 DEBUG [TezTR-518813_1_19_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:54,381 DEBUG [TezTR-518813_1_19_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:54,382 INFO [TezTR-518813_1_19_0_0_0] exec.GroupByOperator: Max hash table memory: 93860656 bytes 2018-01-05T11:28:54,382 INFO [TezTR-518813_1_19_0_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:54,382 DEBUG [TezTR-518813_1_19_0_0_0] exec.GroupByOperator: Initialization Done 2 GBY 2018-01-05T11:28:54,382 DEBUG [TezTR-518813_1_19_0_0_0] exec.GroupByOperator: Operator 2 GBY initialized 2018-01-05T11:28:54,382 DEBUG [TezTR-518813_1_19_0_0_0] exec.GroupByOperator: Initializing children of 2 GBY 2018-01-05T11:28:54,382 DEBUG [TezTR-518813_1_19_0_0_0] exec.ReduceSinkOperator: Initializing child 3 RS 2018-01-05T11:28:54,382 INFO [TezTR-518813_1_19_0_0_0] exec.ReduceSinkOperator: Initializing operator RS[3] 2018-01-05T11:28:54,382 DEBUG [TezTR-518813_1_19_0_0_0] exec.ReduceSinkOperator: keys size is 0 2018-01-05T11:28:54,382 INFO [TezTR-518813_1_19_0_0_0] exec.ReduceSinkOperator: Using tag = -1 2018-01-05T11:28:54,382 DEBUG [TezTR-518813_1_19_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:54,382 DEBUG [TezTR-518813_1_19_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:54,382 DEBUG [TezTR-518813_1_19_0_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11] columnTypes=[struct, struct, struct, struct, struct, struct, struct, struct, struct, struct, struct, struct] 2018-01-05T11:28:54,382 DEBUG [TezTR-518813_1_19_0_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS 2018-01-05T11:28:54,382 DEBUG [TezTR-518813_1_19_0_0_0] exec.ReduceSinkOperator: Operator 3 RS initialized 2018-01-05T11:28:54,382 DEBUG [TezTR-518813_1_19_0_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS done is reset. 2018-01-05T11:28:54,382 DEBUG [TezTR-518813_1_19_0_0_0] exec.GroupByOperator: Initialization Done 2 GBY done is reset. 2018-01-05T11:28:54,382 DEBUG [TezTR-518813_1_19_0_0_0] exec.SelectOperator: Initialization Done 1 SEL done is reset. 2018-01-05T11:28:54,382 DEBUG [TezTR-518813_1_19_0_0_0] exec.SelectOperator: Initialization Done 1 SEL done is reset. 2018-01-05T11:28:54,382 DEBUG [TezTR-518813_1_19_0_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:28:54,382 INFO [TezTR-518813_1_19_0_0_0] exec.OperatorUtils: Setting output collector: RS[3] --> Reducer 2 2018-01-05T11:28:54,382 DEBUG [TezTR-518813_1_19_0_0_0] log.PerfLogger: 2018-01-05T11:28:54,382 DEBUG [TezTR-518813_1_19_0_0_0] exec.MapOperator: Processing alias(es) alltypesorc for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc/alltypesorc 2018-01-05T11:28:54,383 INFO [TezTR-518813_1_19_0_0_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet/.hive-staging_hive_2018-01-05_11-28-53_925_2323731700822862447-1/_tmp.-ext-10000/000000_0 2018-01-05T11:28:54,383 INFO [TezTR-518813_1_19_0_0_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet/.hive-staging_hive_2018-01-05_11-28-53_925_2323731700822862447-1/_task_tmp.-ext-10000/_tmp.000000_0 2018-01-05T11:28:54,383 INFO [TezTR-518813_1_19_0_0_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet/.hive-staging_hive_2018-01-05_11-28-53_925_2323731700822862447-1/_tmp.-ext-10000/000000_0 2018-01-05T11:28:54,383 INFO [TezTR-518813_1_19_0_0_0] parquet.MapredParquetOutputFormat: creating new record writer...org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat@36854f81 2018-01-05T11:28:54,383 DEBUG [IO-Elevator-Thread-1] LlapIoOrc: Done with decode 2018-01-05T11:28:54,384 INFO [IO-Elevator-Thread-1] impl.OrcCodecPool: Got brand-new codec ZLIB 2018-01-05T11:28:54,384 INFO [IO-Elevator-Thread-1] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [-2724874657402502449, 1515180533000, 377237]): [{range start: 0 end: 373}] 2018-01-05T11:28:54,384 INFO [IO-Elevator-Thread-1] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [-2724874657402502449, 1515180533000, 377237], base offset 314466): [{start: 0 end: 10 cache buffer: 0xe76de2e(1)}, {start: 10 end: 37 cache buffer: 0x619da4bb(1)}, {start: 37 end: 69 cache buffer: 0xec730fa(1)}, {start: 69 end: 108 cache buffer: 0x515a70d6(1)}, {start: 108 end: 147 cache buffer: 0x6965437d(1)}, {start: 147 end: 184 cache buffer: 0x65d18a96(1)}, {start: 184 end: 228 cache buffer: 0x52262636(1)}, {start: 228 end: 264 cache buffer: 0x50f82064(1)}, {start: 264 end: 297 cache buffer: 0x13c5efad(1)}, {start: 297 end: 313 cache buffer: 0x7aa94279(1)}, {start: 313 end: 329 cache buffer: 0x195dcc8c(1)}, {start: 329 end: 351 cache buffer: 0x8b724c3(1)}, {start: 351 end: 373 cache buffer: 0x34b30aff(1)}] 2018-01-05T11:28:54,385 INFO [IO-Elevator-Thread-1] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [-2724874657402502449, 1515180533000, 377237]): [{range start: 373 end: 62097}] 2018-01-05T11:28:54,385 INFO [IO-Elevator-Thread-1] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [-2724874657402502449, 1515180533000, 377237], base offset 314466): [{start: 373 end: 382 cache buffer: 0x7377a93d(1)}, {start: 382 end: 643 cache buffer: 0x42d70aac(1)}, {start: 643 end: 681 cache buffer: 0x6e9f14fb(1)}, {start: 681 end: 2587 cache buffer: 0x238f385e(1)}, {start: 2587 end: 2622 cache buffer: 0x6d7f4353(1)}, {start: 2622 end: 10769 cache buffer: 0x1707b63a(1)}, {start: 10769 end: 10791 cache buffer: 0x12940ed(1)}, {start: 10791 end: 15856 cache buffer: 0x5b3ec55e(1)}, {start: 15856 end: 15865 cache buffer: 0x4c4c70b1(1)}, {start: 15865 end: 16345 cache buffer: 0x66d0a964(1)}, {start: 16345 end: 16383 cache buffer: 0x2db5a37b(1)}, {start: 16383 end: 19522 cache buffer: 0xc87a6(1)}, {start: 19522 end: 19558 cache buffer: 0x2feaa6ee(1)}, {start: 19558 end: 26894 cache buffer: 0x4c92a1fe(1)}, {start: 26894 end: 34222 cache buffer: 0x4cbc7d6c(1)}, {start: 34222 end: 40717 cache buffer: 0x2869e2ea(1)}, {start: 40717 end: 41820 cache buffer: 0x524eecdf(1)}, {start: 41820 end: 42010 cache buffer: 0xb0a3a39(1)}, {start: 42010 end: 42031 cache buffer: 0x24727129(1)}, {start: 42031 end: 43783 cache buffer: 0x436bb9ff(1)}, {start: 43783 end: 44582 cache buffer: 0x66e68d59(1)}, {start: 44582 end: 51883 cache buffer: 0x3c67b84e(1)}, {start: 51883 end: 57935 cache buffer: 0xb2b6dd2(1)}, {start: 57935 end: 57944 cache buffer: 0x15918909(1)}, {start: 57944 end: 58301 cache buffer: 0x47cf42a7(1)}, {start: 58301 end: 58706 cache buffer: 0x46eb37d8(1)}, {start: 58706 end: 58749 cache buffer: 0x16fe872a(1)}, {start: 58749 end: 59961 cache buffer: 0xfbe9d3d(1)}, {start: 59961 end: 61615 cache buffer: 0x32d92c17(1)}, {start: 61615 end: 61651 cache buffer: 0x2d938863(1)}, {start: 61651 end: 61911 cache buffer: 0x4fb69d2c(1)}, {start: 61911 end: 61932 cache buffer: 0x349aa692(1)}, {start: 61932 end: 62097 cache buffer: 0x4fd89d82(1)}] 2018-01-05T11:28:54,386 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: tinyint streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 258 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:54,386 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 2 columnType: smallint streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 1903 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:54,386 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 3 columnType: int streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 8144 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:54,386 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 4 columnType: bigint streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 5062 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:54,386 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 5 columnType: float streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 5056 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:54,386 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 6 columnType: double streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 10000 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:54,386 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 7 columnType: string streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 10000 bytes, LlapDataBuffer with 10000 bytes, LlapDataBuffer with 8853 bytes, ], indexBaseOffset=0] dictionary: false lengths: true secondary: false tz: 2018-01-05T11:28:54,386 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 8 columnType: string streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DICTIONARY_V2 dictionarySize: 1259 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 1749 bytes, ], indexBaseOffset=0] dictionary: true lengths: true secondary: false tz: 2018-01-05T11:28:54,386 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 9 columnType: timestamp streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 973 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: true tz: 2018-01-05T11:28:54,387 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 10 columnType: timestamp streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 4048 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: true tz: 2018-01-05T11:28:54,387 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 11 columnType: boolean streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 257 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:54,387 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 12 columnType: boolean streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT present: true data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 162 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:28:54,391 DEBUG [IO-Elevator-Thread-1] LlapIoOrc: Done with decode 2018-01-05T11:28:54,391 DEBUG [IO-Elevator-Thread-1] LlapIoImpl: setDone called; closed false, done false, err null, pending 12 2018-01-05T11:28:54,391 DEBUG [IO-Elevator-Thread-1] impl.StatsRecordingThreadPool: Updated stats: instance: OrcEncodedDataReader thread name: IO-Elevator-Thread-1 thread id: 1309 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:54,408 INFO [TezTR-518813_1_19_0_0_0] write.ParquetRecordWriterWrapper: initialize serde with table properties. 2018-01-05T11:28:54,408 INFO [TezTR-518813_1_19_0_0_0] write.ParquetRecordWriterWrapper: creating real writer to write at file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet/.hive-staging_hive_2018-01-05_11-28-53_925_2323731700822862447-1/_task_tmp.-ext-10000/_tmp.000000_0 2018-01-05T11:28:54,411 INFO [TezTR-518813_1_19_0_0_0] codec.CodecConfig: Compression set to false 2018-01-05T11:28:54,412 INFO [TezTR-518813_1_19_0_0_0] codec.CodecConfig: Compression: UNCOMPRESSED 2018-01-05T11:28:54,425 INFO [TezTR-518813_1_19_0_0_0] hadoop.ParquetOutputFormat: Parquet block size to 134217728 2018-01-05T11:28:54,425 INFO [TezTR-518813_1_19_0_0_0] hadoop.ParquetOutputFormat: Parquet page size to 1048576 2018-01-05T11:28:54,425 INFO [TezTR-518813_1_19_0_0_0] hadoop.ParquetOutputFormat: Parquet dictionary page size to 1048576 2018-01-05T11:28:54,425 INFO [TezTR-518813_1_19_0_0_0] hadoop.ParquetOutputFormat: Dictionary is on 2018-01-05T11:28:54,425 INFO [TezTR-518813_1_19_0_0_0] hadoop.ParquetOutputFormat: Validation is off 2018-01-05T11:28:54,425 INFO [TezTR-518813_1_19_0_0_0] hadoop.ParquetOutputFormat: Writer version is: PARQUET_1_0 2018-01-05T11:28:54,426 INFO [TezTR-518813_1_19_0_0_0] hadoop.ParquetOutputFormat: Maximum row group padding size is 0 bytes 2018-01-05T11:28:54,426 INFO [TezTR-518813_1_19_0_0_0] hadoop.ParquetOutputFormat: Page size checking is: estimated 2018-01-05T11:28:54,426 INFO [TezTR-518813_1_19_0_0_0] hadoop.ParquetOutputFormat: Min row count for page size check is: 100 2018-01-05T11:28:54,426 INFO [TezTR-518813_1_19_0_0_0] hadoop.ParquetOutputFormat: Max row count for page size check is: 10000 2018-01-05T11:28:54,452 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000038, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_19_00_000000_0, eventCount=1 } 2018-01-05T11:28:54,452 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:54,481 DEBUG [TezTR-518813_1_19_0_0_0] hadoop.MemoryManager: Allocated total memory pool is: 513,802,240 2018-01-05T11:28:54,500 DEBUG [AMReporterQueueDrainer] impl.AMReporter: Removing am 10.22.8.182:49824 with last associated dag QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=8} from heartbeat with taskCount=0, amFailed=false, isDone=true 2018-01-05T11:28:54,508 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,508 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: Status: Running (Executing on YARN cluster with App id application_1515180518813_0001) 2018-01-05T11:28:54,510 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:54,510 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 0(+1)/1 Reducer 2: 0/1 2018-01-05T11:28:54,555 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000038, requestId=3, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_19_00_000000_0, eventCount=1 } 2018-01-05T11:28:54,556 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=3, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:54,661 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000038, requestId=4, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_19_00_000000_0, eventCount=1 } 2018-01-05T11:28:54,662 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=4, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:54,745 DEBUG [TezTR-518813_1_19_0_0_0] hadoop.MemoryManager: Adjust block size from 134,217,728 to 134,217,728 for writer: org.apache.parquet.hadoop.InternalParquetRecordWriter@3d62524a 2018-01-05T11:28:54,745 INFO [TezTR-518813_1_19_0_0_0] write.ParquetRecordWriterWrapper: real writer: org.apache.parquet.hadoop.ParquetRecordWriter@38372722 2018-01-05T11:28:54,745 INFO [TezTR-518813_1_19_0_0_0] exec.FileSinkOperator: FS[2]: records written - 1 2018-01-05T11:28:54,760 INFO [TezTR-518813_1_19_0_0_0] exec.MapOperator: MAP[0]: records read - 1 2018-01-05T11:28:54,761 INFO [TezTR-518813_1_19_0_0_0] exec.FileSinkOperator: FS[2]: records written - 10 2018-01-05T11:28:54,761 INFO [TezTR-518813_1_19_0_0_0] exec.MapOperator: MAP[0]: records read - 10 2018-01-05T11:28:54,766 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000038, requestId=5, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_19_00_000000_0, eventCount=1 } 2018-01-05T11:28:54,767 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=5, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:54,771 INFO [TezTR-518813_1_19_0_0_0] exec.FileSinkOperator: FS[2]: records written - 100 2018-01-05T11:28:54,771 INFO [TezTR-518813_1_19_0_0_0] exec.MapOperator: MAP[0]: records read - 100 2018-01-05T11:28:54,837 INFO [TezTR-518813_1_19_0_0_0] exec.FileSinkOperator: FS[2]: records written - 1000 2018-01-05T11:28:54,838 INFO [TezTR-518813_1_19_0_0_0] exec.MapOperator: MAP[0]: records read - 1000 2018-01-05T11:28:54,838 DEBUG [TezTR-518813_1_19_0_0_0] tez.InterruptibleProcessing: Adjusting abort check rows to 6589 from 1000 2018-01-05T11:28:54,869 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000038, requestId=6, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_19_00_000000_0, eventCount=1 } 2018-01-05T11:28:54,870 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=6, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:54,949 DEBUG [TezTR-518813_1_19_0_0_0] tez.InterruptibleProcessing: Adjusting abort check rows to 100000 from 6589 2018-01-05T11:28:54,971 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000038, requestId=7, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_19_00_000000_0, eventCount=1 } 2018-01-05T11:28:54,972 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=7, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:54,974 INFO [TezTR-518813_1_19_0_0_0] exec.FileSinkOperator: FS[2]: records written - 10000 2018-01-05T11:28:54,974 INFO [TezTR-518813_1_19_0_0_0] exec.MapOperator: MAP[0]: records read - 10000 2018-01-05T11:28:54,996 INFO [TezTR-518813_1_19_0_0_0] LlapIoImpl: Maximum queue length observed 12 2018-01-05T11:28:54,996 INFO [TezTR-518813_1_19_0_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, warehouse.alltypesorc, file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc/alltypesorc ([-2724874657402502449, 1515180533000, 377237]), 0,3]: [ NUM_VECTOR_BATCHES=13, NUM_DECODED_BATCHES=3, SELECTED_ROWGROUPS=3, NUM_ERRORS=0, ROWS_EMITTED=12288, METADATA_CACHE_HIT=4, METADATA_CACHE_MISS=0, CACHE_HIT_BYTES=283052, CACHE_MISS_BYTES=0, ALLOCATED_BYTES=0, ALLOCATED_USED_BYTES=0, TOTAL_IO_TIME_NS=36810377, DECODE_TIME_NS=24340296, HDFS_TIME_NS=142615, CONSUMER_TIME_NS=613186356 ] 2018-01-05T11:28:54,996 DEBUG [TezTR-518813_1_19_0_0_0] encoded.OrcEncodedDataReader: Encoded reader is being stopped 2018-01-05T11:28:54,996 DEBUG [TezTR-518813_1_19_0_0_0] log.PerfLogger: 2018-01-05T11:28:54,996 DEBUG [TezTR-518813_1_19_0_0_0] mr.ObjectCache: djaiswal_20180105112853_18452c72-dc2a-4031-8352-113fe0f24ba2_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:28:54,996 DEBUG [TezTR-518813_1_19_0_0_0] exec.MapOperator: close called for operator MAP[0] 2018-01-05T11:28:54,996 INFO [TezTR-518813_1_19_0_0_0] exec.MapOperator: Closing operator MAP[0] 2018-01-05T11:28:54,996 INFO [TezTR-518813_1_19_0_0_0] exec.MapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:12288, 2018-01-05T11:28:54,996 DEBUG [TezTR-518813_1_19_0_0_0] exec.MapOperator: Closing child = TS[0] 2018-01-05T11:28:54,996 DEBUG [TezTR-518813_1_19_0_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:28:54,996 DEBUG [TezTR-518813_1_19_0_0_0] exec.TableScanOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:54,996 INFO [TezTR-518813_1_19_0_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:28:54,996 DEBUG [TezTR-518813_1_19_0_0_0] exec.TableScanOperator: Closing child = SEL[1] 2018-01-05T11:28:54,996 DEBUG [TezTR-518813_1_19_0_0_0] exec.SelectOperator: close called for operator SEL[1] 2018-01-05T11:28:54,996 DEBUG [TezTR-518813_1_19_0_0_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:54,996 INFO [TezTR-518813_1_19_0_0_0] exec.SelectOperator: Closing operator SEL[1] 2018-01-05T11:28:54,996 DEBUG [TezTR-518813_1_19_0_0_0] exec.SelectOperator: Closing child = FS[2] 2018-01-05T11:28:54,996 DEBUG [TezTR-518813_1_19_0_0_0] exec.FileSinkOperator: close called for operator FS[2] 2018-01-05T11:28:54,996 DEBUG [TezTR-518813_1_19_0_0_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:54,996 INFO [TezTR-518813_1_19_0_0_0] exec.FileSinkOperator: Closing operator FS[2] 2018-01-05T11:28:54,996 INFO [TezTR-518813_1_19_0_0_0] exec.FileSinkOperator: FS[2]: records written - 12288 2018-01-05T11:28:54,996 INFO [TezTR-518813_1_19_0_0_0] hadoop.InternalParquetRecordWriter: Flushing mem columnStore to file. allocated memory: 997,723 2018-01-05T11:28:55,037 DEBUG [AMReporterQueueDrainer] impl.AMReporter: Removing am 10.22.8.182:49824 with last associated dag QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=9} from heartbeat with taskCount=0, amFailed=false, isDone=true 2018-01-05T11:28:55,076 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000038, requestId=8, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_19_00_000000_0, eventCount=1 } 2018-01-05T11:28:55,076 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=8, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:55,079 INFO [TezTR-518813_1_19_0_0_0] hadoop.ColumnChunkPageWriteStore: written 5,652B for [ctinyint] INT32: 12,288 values, 5,614B raw, 5,614B comp, 1 pages, encodings: [PLAIN_DICTIONARY], dic { 127 entries, 508B raw, 127B comp} 2018-01-05T11:28:55,079 INFO [TezTR-518813_1_19_0_0_0] hadoop.ColumnChunkPageWriteStore: written 10,247B for [csmallint] INT32: 12,288 values, 10,207B raw, 10,207B comp, 1 pages, encodings: [PLAIN_DICTIONARY], dic { 5,575 entries, 22,300B raw, 5,575B comp} 2018-01-05T11:28:55,079 INFO [TezTR-518813_1_19_0_0_0] hadoop.ColumnChunkPageWriteStore: written 10,250B for [cint] INT32: 12,288 values, 10,210B raw, 10,210B comp, 1 pages, encodings: [PLAIN_DICTIONARY], dic { 6,082 entries, 24,328B raw, 6,082B comp} 2018-01-05T11:28:55,079 INFO [TezTR-518813_1_19_0_0_0] hadoop.ColumnChunkPageWriteStore: written 10,255B for [cbigint] INT64: 12,288 values, 10,207B raw, 10,207B comp, 1 pages, encodings: [PLAIN_DICTIONARY], dic { 6,082 entries, 48,656B raw, 6,082B comp} 2018-01-05T11:28:55,079 INFO [TezTR-518813_1_19_0_0_0] hadoop.ColumnChunkPageWriteStore: written 6,417B for [cfloat] FLOAT: 12,288 values, 6,379B raw, 6,379B comp, 1 pages, encodings: [PLAIN_DICTIONARY], dic { 131 entries, 524B raw, 131B comp} 2018-01-05T11:28:55,080 INFO [TezTR-518813_1_19_0_0_0] hadoop.ColumnChunkPageWriteStore: written 10,255B for [cdouble] DOUBLE: 12,288 values, 10,207B raw, 10,207B comp, 1 pages, encodings: [PLAIN_DICTIONARY], dic { 5,576 entries, 44,608B raw, 5,576B comp} 2018-01-05T11:28:55,080 INFO [TezTR-518813_1_19_0_0_0] hadoop.ColumnChunkPageWriteStore: written 10,267B for [cstring1] BINARY: 12,288 values, 10,207B raw, 10,207B comp, 1 pages, encodings: [PLAIN_DICTIONARY], dic { 6,083 entries, 111,263B raw, 6,083B comp} 2018-01-05T11:28:55,080 INFO [TezTR-518813_1_19_0_0_0] hadoop.ColumnChunkPageWriteStore: written 10,260B for [cstring2] BINARY: 12,288 values, 10,206B raw, 10,206B comp, 1 pages, encodings: [PLAIN_DICTIONARY], dic { 6,081 entries, 112,044B raw, 6,081B comp} 2018-01-05T11:28:55,080 INFO [TezTR-518813_1_19_0_0_0] hadoop.ColumnChunkPageWriteStore: written 10,262B for [ctimestamp1] INT96: 12,288 values, 10,206B raw, 10,206B comp, 1 pages, encodings: [PLAIN_DICTIONARY], dic { 5,523 entries, 66,276B raw, 5,523B comp} 2018-01-05T11:28:55,081 INFO [TezTR-518813_1_19_0_0_0] hadoop.ColumnChunkPageWriteStore: written 10,262B for [ctimestamp2] INT96: 12,288 values, 10,206B raw, 10,206B comp, 1 pages, encodings: [PLAIN_DICTIONARY], dic { 5,506 entries, 66,072B raw, 5,506B comp} 2018-01-05T11:28:55,081 INFO [TezTR-518813_1_19_0_0_0] hadoop.ColumnChunkPageWriteStore: written 1,413B for [cboolean1] BOOLEAN: 12,288 values, 1,381B raw, 1,381B comp, 1 pages, encodings: [PLAIN] 2018-01-05T11:28:55,081 INFO [TezTR-518813_1_19_0_0_0] hadoop.ColumnChunkPageWriteStore: written 1,412B for [cboolean2] BOOLEAN: 12,288 values, 1,380B raw, 1,380B comp, 1 pages, encodings: [PLAIN] 2018-01-05T11:28:55,180 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000038, requestId=9, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_19_00_000000_0, eventCount=1 } 2018-01-05T11:28:55,181 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=9, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:55,221 DEBUG [TezTR-518813_1_19_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:55,221 DEBUG [TezTR-518813_1_19_0_0_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_925_2323731700822862447-1/-mr-10001 2018-01-05T11:28:55,244 DEBUG [TezTR-518813_1_19_0_0_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_925_2323731700822862447-1/-mr-10001/tmpstats-0 2018-01-05T11:28:55,244 DEBUG [TezTR-518813_1_19_0_0_0] fs.FSStatsPublisher: Writing stats in it : {default.alltypesparquet/={rawDataSize=147456, numRows=12288}} 2018-01-05T11:28:55,244 INFO [TezTR-518813_1_19_0_0_0] exec.FileSinkOperator: RECORDS_OUT_1_default.alltypesparquet:12288, 2018-01-05T11:28:55,244 DEBUG [TezTR-518813_1_19_0_0_0] exec.FileSinkOperator: 2 Close done 2018-01-05T11:28:55,244 DEBUG [TezTR-518813_1_19_0_0_0] exec.SelectOperator: Closing child = SEL[1] 2018-01-05T11:28:55,244 DEBUG [TezTR-518813_1_19_0_0_0] exec.SelectOperator: close called for operator SEL[1] 2018-01-05T11:28:55,244 DEBUG [TezTR-518813_1_19_0_0_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:55,244 INFO [TezTR-518813_1_19_0_0_0] exec.SelectOperator: Closing operator SEL[1] 2018-01-05T11:28:55,244 DEBUG [TezTR-518813_1_19_0_0_0] exec.SelectOperator: Closing child = GBY[2] 2018-01-05T11:28:55,244 DEBUG [TezTR-518813_1_19_0_0_0] exec.GroupByOperator: close called for operator GBY[2] 2018-01-05T11:28:55,244 DEBUG [TezTR-518813_1_19_0_0_0] exec.GroupByOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:55,244 INFO [TezTR-518813_1_19_0_0_0] exec.GroupByOperator: Closing operator GBY[2] 2018-01-05T11:28:55,244 INFO [TezTR-518813_1_19_0_0_0] exec.GroupByOperator: Begin Hash Table flush: size = 1 2018-01-05T11:28:55,247 INFO [TezTR-518813_1_19_0_0_0] exec.ReduceSinkOperator: keys are [] num distributions: 0 2018-01-05T11:28:55,247 DEBUG [TezTR-518813_1_19_0_0_0] exec.GroupByOperator: Closing child = RS[3] 2018-01-05T11:28:55,247 DEBUG [TezTR-518813_1_19_0_0_0] exec.ReduceSinkOperator: close called for operator RS[3] 2018-01-05T11:28:55,247 DEBUG [TezTR-518813_1_19_0_0_0] exec.ReduceSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:55,247 INFO [TezTR-518813_1_19_0_0_0] exec.ReduceSinkOperator: Closing operator RS[3] 2018-01-05T11:28:55,247 INFO [TezTR-518813_1_19_0_0_0] exec.ReduceSinkOperator: RECORDS_OUT_INTERMEDIATE_Map_1:1, 2018-01-05T11:28:55,247 DEBUG [TezTR-518813_1_19_0_0_0] exec.ReduceSinkOperator: 3 Close done 2018-01-05T11:28:55,247 DEBUG [TezTR-518813_1_19_0_0_0] exec.GroupByOperator: 2 Close done 2018-01-05T11:28:55,247 DEBUG [TezTR-518813_1_19_0_0_0] exec.SelectOperator: 1 Close done 2018-01-05T11:28:55,247 DEBUG [TezTR-518813_1_19_0_0_0] exec.SelectOperator: 1 Close done 2018-01-05T11:28:55,247 DEBUG [TezTR-518813_1_19_0_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:28:55,247 DEBUG [TezTR-518813_1_19_0_0_0] exec.MapOperator: 0 Close done 2018-01-05T11:28:55,247 INFO [TezTR-518813_1_19_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_19_00_000000_0 2018-01-05T11:28:55,247 INFO [TezTR-518813_1_19_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: Waiting for all spills to complete : Pending : 0 2018-01-05T11:28:55,247 INFO [TezTR-518813_1_19_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: All spills complete 2018-01-05T11:28:55,259 DEBUG [TezTR-518813_1_19_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:55,259 INFO [TezTR-518813_1_19_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:55,259 INFO [TezTR-518813_1_19_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_19_00_000000_0, askedToStop=false 2018-01-05T11:28:55,259 INFO [TezTR-518813_1_19_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_19_00_000000_0, stopRequested=false 2018-01-05T11:28:55,259 INFO [TezTR-518813_1_19_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_19_00_000000_0: Counters: 32 [[File System Counters FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=0, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=12288, INPUT_SPLIT_LENGTH_BYTES=376777, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=54893, OUTPUT_BYTES_WITH_OVERHEAD=54903, OUTPUT_BYTES_PHYSICAL=54931, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][HIVE CREATED_FILES=1, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=12288, RECORDS_OUT_1_default.alltypesparquet=12288, RECORDS_OUT_INTERMEDIATE_Map_1=1][org.apache.hadoop.hive.llap.counters.LlapIOCounters CACHE_HIT_BYTES=283052, CACHE_MISS_BYTES=0, CONSUMER_TIME_NS=613186356, DECODE_TIME_NS=24340296, HDFS_TIME_NS=142615, METADATA_CACHE_HIT=4, NUM_DECODED_BATCHES=3, NUM_VECTOR_BATCHES=13, ROWS_EMITTED=12288, SELECTED_ROWGROUPS=3, TOTAL_IO_TIME_NS=36810377]] 2018-01-05T11:28:55,259 DEBUG [TezTR-518813_1_19_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_19_0_0_0 thread id: 1303 scheme: file bytesRead: 0 bytesWritten: 654673 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:55,259 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_19_00_000000_0, isTaskDone=true 2018-01-05T11:28:55,260 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000038, requestId=10, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_19_00_000000_0, eventCount=4 } 2018-01-05T11:28:55,261 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_19][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_19_00_000000_0, creationTime=1515180534322, allocationTime=1515180534323, startTime=1515180534326, finishTime=1515180535261, timeTaken=935, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:55,261 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=10, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:55,261 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_19_00_000000_0 2018-01-05T11:28:55,261 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_19_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:55,261 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_19_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:55,261 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_19_00_000000_0 2018-01-05T11:28:55,261 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_19_00_000000_0; false 2018-01-05T11:28:55,261 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_19_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:55,262 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:55,262 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000038=921 2018-01-05T11:28:55,262 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:55,262 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_19_00_000000_0: true 2018-01-05T11:28:55,262 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_19][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_19_00_000000, startTime=1515180534326, finishTime=1515180535261, timeTaken=935, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_19_00_000000_0, diagnostics=, counters=Counters: 32, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=654673, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=12288, INPUT_SPLIT_LENGTH_BYTES=376777, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=54893, OUTPUT_BYTES_WITH_OVERHEAD=54903, OUTPUT_BYTES_PHYSICAL=54931, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, HIVE, CREATED_FILES=1, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=12288, RECORDS_OUT_1_default.alltypesparquet=12288, RECORDS_OUT_INTERMEDIATE_Map_1=1, org.apache.hadoop.hive.llap.counters.LlapIOCounters, CACHE_HIT_BYTES=283052, CACHE_MISS_BYTES=0, CONSUMER_TIME_NS=613186356, DECODE_TIME_NS=24340296, HDFS_TIME_NS=142615, METADATA_CACHE_HIT=4, NUM_DECODED_BATCHES=3, NUM_VECTOR_BATCHES=13, ROWS_EMITTED=12288, SELECTED_ROWGROUPS=3, TOTAL_IO_TIME_NS=36810377 2018-01-05T11:28:55,262 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_19_00_000000_0 2018-01-05T11:28:55,262 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000038, QueryId=djaiswal_20180105112853_18452c72-dc2a-4031-8352-113fe0f24ba2-19, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:28:55,262 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_19_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:55,262 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_19_00_000000_0 2018-01-05T11:28:55,262 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_19_00 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:55,262 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_19_00 [Map 1] 2018-01-05T11:28:55,263 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_19][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_19_00, initRequestedTime=1515180534289, initedTime=1515180534321, startRequestedTime=1515180534298, startedTime=1515180534321, finishTime=1515180535262, timeTaken=941, status=SUCCEEDED, diagnostics=, counters=Counters: 32, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=654673, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=12288, INPUT_SPLIT_LENGTH_BYTES=376777, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=54893, OUTPUT_BYTES_WITH_OVERHEAD=54903, OUTPUT_BYTES_PHYSICAL=54931, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, HIVE, CREATED_FILES=1, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=12288, RECORDS_OUT_1_default.alltypesparquet=12288, RECORDS_OUT_INTERMEDIATE_Map_1=1, org.apache.hadoop.hive.llap.counters.LlapIOCounters, CACHE_HIT_BYTES=283052, CACHE_MISS_BYTES=0, CONSUMER_TIME_NS=613186356, DECODE_TIME_NS=24340296, HDFS_TIME_NS=142615, METADATA_CACHE_HIT=4, NUM_DECODED_BATCHES=3, NUM_VECTOR_BATCHES=13, ROWS_EMITTED=12288, SELECTED_ROWGROUPS=3, TOTAL_IO_TIME_NS=36810377, vertexStats=firstTaskStartTime=1515180534326, firstTasksToStart=[ task_1515180518813_0001_19_00_000000 ], lastTaskFinishTime=1515180535261, lastTasksToFinish=[ task_1515180518813_0001_19_00_000000 ], minTaskDuration=935, maxTaskDuration=935, avgTaskDuration=935.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_19_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_19_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:55,263 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_19_00 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:55,263 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_19][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000038, stoppedTime=1515180535263, exitStatus=0 2018-01-05T11:28:55,263 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000038 exited with diagnostics set to Container completed. 2018-01-05T11:28:55,263 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_19_01 [Reducer 2] attempt: attempt_1515180518813_0001_19_00_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:28:55,263 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_19_00 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2 2018-01-05T11:28:55,263 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null 2018-01-05T11:28:55,263 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000038 2018-01-05T11:28:55,263 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: All source tasks completed. Ramping up 1 remaining tasks for vertex: Reducer 2 2018-01-05T11:28:55,263 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Scheduling 1 tasks for vertex: Reducer 2 with totalTasks: 1. 1 source tasks completed out of 1. MinSourceTaskCompletedFraction: 1.0 min: 0.05 max: 0.75 2018-01-05T11:28:55,263 INFO [App Shared Pool - #0] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Map 1, destinationVertexName=Reducer 2. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.UnorderedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:28:55,263 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_19_01 [Reducer 2] 2018-01-05T11:28:55,263 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_19][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_19_01_000000, scheduledTime=1515180535263, launchTime=1515180535263 2018-01-05T11:28:55,263 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_19_01 [Reducer 2] 2018-01-05T11:28:55,263 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_19_01_000000_0, priority=14, capability=, hosts=null 2018-01-05T11:28:55,263 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:55,263 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:55,263 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:55,263 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_19_01_000000_0 on hosts=null 2018-01-05T11:28:55,263 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_19_01_000000_0 on hosts=null 2018-01-05T11:28:55,263 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_19_01_000000_0 2018-01-05T11:28:55,263 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:55,263 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #39, task=TaskInfo{task=attempt_1515180518813_0001_19_01_000000_0, priority=14, startTime=0, containerId=null, uniqueId=38, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000039 2018-01-05T11:28:55,263 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_19_01_000000_0; false 2018-01-05T11:28:55,263 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_19_01_000000_0, priority=14, startTime=311555772, containerId=container_111101111_0001_00_000039assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=38, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:55,264 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000039 for node: 10.22.8.182:49818 2018-01-05T11:28:55,264 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000039, attempt_1515180518813_0001_19_01_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:55,264 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000039 for node: 10.22.8.182:49818 2018-01-05T11:28:55,265 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000039 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:55,265 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000039, launchTime=1515180535265 2018-01-05T11:28:55,265 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=19}, sourceName=Map 1, state=S_SUCCEEDED 2018-01-05T11:28:55,266 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_19_01_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000039] on NM: [10.22.8.182:49818] 2018-01-05T11:28:55,266 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:55,266 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:55,266 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_19][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_19_01_000000_0, startTime=1515180535265, containerId=container_111101111_0001_00_000039, nodeId=10.22.8.182:49818 2018-01-05T11:28:55,266 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_19_01_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112853_18452c72-dc2a-4031-8352-113fe0f24ba2, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000039, dagName=INSERT OVERWRITE TABLE alltype...alltypesorc (Stage-1), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.UnorderedKVInput,1}}, Outputs={{out_Reducer 2,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=2, completedTaskCount=1, dagStartTime=1515180534289, firstAttemptStartTime=1515180535263, currentAttemptStartTime=1515180535264} 2018-01-05T11:28:55,266 INFO [IPC Server handler 4 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000039, QueryId=djaiswal_20180105112853_18452c72-dc2a-4031-8352-113fe0f24ba2-19, DagId=19, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, SubmitTime=1515180535266 2018-01-05T11:28:55,266 DEBUG [IPC Server handler 4 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=19} with the ShuffleHandler 2018-01-05T11:28:55,266 DEBUG [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/19] 2018-01-05T11:28:55,266 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:55,266 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_19_01_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:55,266 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_19_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180535263, dagStartTime=1515180534289, withinDagPriority=14, vertexParallelism= 1, selfAndUpstreamParallelism= 2, selfAndUpstreamComplete= 1}] 2018-01-05T11:28:55,267 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_19_01_000000_0 : ACCEPTED 2018-01-05T11:28:55,267 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_19_01_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:55,267 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_19_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180535263, dagStartTime=1515180534289, withinDagPriority=14, vertexParallelism= 1, selfAndUpstreamParallelism= 2, selfAndUpstreamComplete= 1} 2018-01-05T11:28:55,267 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_19_01_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:55,267 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_19_01_000000_0 to pre-emption queue 2018-01-05T11:28:55,267 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_19_01_000000_0: true 2018-01-05T11:28:55,267 INFO [TaskCommunicator # 9] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_19_01_000000_0 2018-01-05T11:28:55,267 INFO [TaskCommunicator # 9] GuaranteedTasks: Received response for attempt_1515180518813_0001_19_01_000000_0, true 2018-01-05T11:28:55,267 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:6971875 2018-01-05T11:28:55,268 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : INSERT OVERWRITE TABLE alltype...alltypesorc (Stage-1), VertexName: Reducer 2, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_19_01_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:28:55,268 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1039138816, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:55,268 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_19_01_000000_0 vertexName: Reducer 2 2018-01-05T11:28:55,268 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000039, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_19_01_000000_0, eventCount=1 } 2018-01-05T11:28:55,268 DEBUG [TezTR-518813_1_19_1_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_19 queryId: djaiswal_20180105112853_18452c72-dc2a-4031-8352-113fe0f24ba2 instanceType: TaskRunner2Callable 2018-01-05T11:28:55,268 INFO [TezTR-518813_1_19_1_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_19_01_000000_0 2018-01-05T11:28:55,269 DEBUG [TezTR-518813_1_19_1_0_0] log.PerfLogger: 2018-01-05T11:28:55,269 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:55,271 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:55,273 INFO [IPC Server handler 21 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_19_01_000000_0 1 events [0,1) total 1 vertex_1515180518813_0001_19_01 [Reducer 2] 2018-01-05T11:28:55,273 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:28:55,273 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_19_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:28:55,273 DEBUG [TezTR-518813_1_19_1_0_0] log.PerfLogger: 2018-01-05T11:28:55,273 INFO [TezTR-518813_1_19_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:55,273 INFO [TezTR-518813_1_19_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:55,273 INFO [TezTR-518813_1_19_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:55,273 INFO [I/O Setup 1 Initialize: {out_Reducer 2}] output.MROutput: out_Reducer 2: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:28:55,274 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:55,274 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:55,274 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:55,274 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:55,274 INFO [TezTR-518813_1_19_1_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:55,274 INFO [TezTR-518813_1_19_1_0_0] resources.MemoryDistributor: InitialRequests=[out_Reducer 2:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.UnorderedKVInput] 2018-01-05T11:28:55,274 INFO [TezTR-518813_1_19_1_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:55,274 INFO [TezTR-518813_1_19_1_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:55,274 INFO [TezTR-518813_1_19_1_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 131404920, TotalRequestedScaled: 6.570246E7, TotalJVMHeap: 1039138816, TotalAvailable: 696223006, TotalRequested/TotalJVMHeap:0.13 2018-01-05T11:28:55,274 INFO [TezTR-518813_1_19_1_0_0] resources.MemoryDistributor: Allocations=[out_Reducer 2:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [Map 1:org.apache.tez.runtime.library.input.UnorderedKVInput:INPUT:131404920:131404920] 2018-01-05T11:28:55,274 INFO [TezTR-518813_1_19_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:55,274 INFO [TezTR-518813_1_19_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 0 2018-01-05T11:28:55,274 INFO [TezTR-518813_1_19_1_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:55,274 INFO [TezTR-518813_1_19_1_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_19_01_000000_0 2018-01-05T11:28:55,274 DEBUG [TezTR-518813_1_19_1_0_0] log.PerfLogger: 2018-01-05T11:28:55,274 DEBUG [TezTR-518813_1_19_1_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_19_01_000000_0_10115 2018-01-05T11:28:55,274 DEBUG [TezTR-518813_1_19_1_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:55,274 DEBUG [TezTR-518813_1_19_1_0_0] mr.ObjectCache: Creating djaiswal_20180105112853_18452c72-dc2a-4031-8352-113fe0f24ba2_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:28:55,274 INFO [TezTR-518813_1_19_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_925_2323731700822862447-1/djaiswal/_tez_scratch_dir/fde0a2ab-cd42-4330-b243-0e16f6b594d2/reduce.xml 2018-01-05T11:28:55,274 DEBUG [TezTR-518813_1_19_1_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_925_2323731700822862447-1/djaiswal/_tez_scratch_dir/fde0a2ab-cd42-4330-b243-0e16f6b594d2/reduce.xml 2018-01-05T11:28:55,274 DEBUG [TezTR-518813_1_19_1_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_925_2323731700822862447-1/djaiswal/_tez_scratch_dir/fde0a2ab-cd42-4330-b243-0e16f6b594d2/reduce.xml 2018-01-05T11:28:55,274 DEBUG [TezTR-518813_1_19_1_0_0] log.PerfLogger: 2018-01-05T11:28:55,274 INFO [TezTR-518813_1_19_1_0_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:28:55,274 DEBUG [TezTR-518813_1_19_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:55,275 DEBUG [TezTR-518813_1_19_1_0_0] log.PerfLogger: 2018-01-05T11:28:55,275 INFO [TezTR-518813_1_19_1_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 2.92KB 2018-01-05T11:28:55,276 DEBUG [TezTR-518813_1_19_1_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:55,276 DEBUG [TezTR-518813_1_19_1_0_0] log.PerfLogger: 2018-01-05T11:28:55,276 DEBUG [TezTR-518813_1_19_1_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:55,276 DEBUG [TezTR-518813_1_19_1_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:55,276 DEBUG [TezTR-518813_1_19_1_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:55,276 INFO [TezTR-518813_1_19_1_0_0] impl.SimpleFetchedInputAllocator: Map_1: RequestedMemory=131404920, AssignedMemory=131404920, maxSingleShuffleLimit=32851230 2018-01-05T11:28:55,276 INFO [TezTR-518813_1_19_1_0_0] impl.ShuffleManager: Map_1: numInputs=1, compressionCodec=NoCompressionCodec, numFetchers=1, ifileBufferSize=4096, ifileReadAheadEnabled=true, ifileReadAheadLength=4194304, localDiskFetchEnabled=false, sharedFetchEnabled=false, keepAlive=false, keepAliveMaxConnections=20, connectionTimeout=20000, readTimeout=180000, bufferSize=8192, bufferSize=8192, maxTaskOutputAtOnce=20 2018-01-05T11:28:55,276 INFO [TezTR-518813_1_19_1_0_0] impl.ShuffleInputEventHandlerImpl: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:28:55,276 INFO [TezTR-518813_1_19_1_0_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:28:55,277 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=19&reduce=0&map=attempt_1515180518813_0001_19_00_000000_0_10114 mapId: [attempt_1515180518813_0001_19_00_000000_0_10114] reduceId: [0] jobId: [job_1515180518813_0001] dagId: [19] keepAlive: false 2018-01-05T11:28:55,277 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=19&reduce=0&map=attempt_1515180518813_0001_19_00_000000_0_10114; hash=...P6CuQkhxzh2OA 2018-01-05T11:28:55,277 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=19&reduce=0&map=attempt_1515180518813_0001_19_00_000000_0_10114;reply=NNv+BuOIHj6dQ 2018-01-05T11:28:55,278 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=19, user='djaiswal', attemptId='attempt_1515180518813_0001_19_00_000000_0_10114'} via loader 2018-01-05T11:28:55,278 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=19, user='djaiswal', attemptId='attempt_1515180518813_0001_19_00_000000_0_10114'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:28:55,278 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_19_00_000000_0_10114 not found 2018-01-05T11:28:55,278 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_19_00_000000_0_10114,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/19/output/attempt_1515180518813_0001_19_00_000000_0_10114/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/19/output/attempt_1515180518813_0001_19_00_000000_0_10114/file.out.index 2018-01-05T11:28:55,278 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Content Length in shuffle : 54962 2018-01-05T11:28:55,278 INFO [Fetcher_B {Map_1} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=19&reduce=0&map=attempt_1515180518813_0001_19_00_000000_0_10114 sent hash and receievd reply 0 ms 2018-01-05T11:28:55,278 INFO [Fetcher_B {Map_1} #0] ShuffleManager.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_19_00_000000_0_10114} to MEMORY, csize=54907, dsize=54903, EndTime=1515180535278, TimeTaken=0, Rate=0.00 MB/s 2018-01-05T11:28:55,278 INFO [Fetcher_B {Map_1} #0] impl.ShuffleManager: All inputs fetched for input vertex : Map 1 2018-01-05T11:28:55,278 INFO [Fetcher_B {Map_1} #0] impl.ShuffleManager: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.05 MB/s) 2018-01-05T11:28:55,278 INFO [TezTR-518813_1_19_1_0_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:28:55,278 INFO [ShuffleRunner {Map_1}] impl.ShuffleManager: Map_1: Shutting down FetchScheduler, Was Interrupted: false 2018-01-05T11:28:55,278 INFO [ShuffleRunner {Map_1}] impl.ShuffleManager: Map_1: Scheduler thread completed 2018-01-05T11:28:55,278 INFO [TezTR-518813_1_19_1_0_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:55,278 DEBUG [TezTR-518813_1_19_1_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:28:55,278 DEBUG [TezTR-518813_1_19_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:55,278 DEBUG [TezTR-518813_1_19_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:55,279 DEBUG [TezTR-518813_1_19_1_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11] columnTypes=[struct, struct, struct, struct, struct, struct, struct, struct, struct, struct, struct, struct] 2018-01-05T11:28:55,279 DEBUG [TezTR-518813_1_19_1_0_0] log.PerfLogger: 2018-01-05T11:28:55,279 INFO [TezTR-518813_1_19_1_0_0] exec.GroupByOperator: Initializing operator GBY[4] 2018-01-05T11:28:55,279 DEBUG [TezTR-518813_1_19_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:55,279 DEBUG [TezTR-518813_1_19_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:55,279 DEBUG [TezTR-518813_1_19_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:55,279 DEBUG [TezTR-518813_1_19_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:55,279 DEBUG [TezTR-518813_1_19_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:55,279 DEBUG [TezTR-518813_1_19_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:55,279 DEBUG [TezTR-518813_1_19_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:55,279 DEBUG [TezTR-518813_1_19_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:55,279 DEBUG [TezTR-518813_1_19_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:55,279 DEBUG [TezTR-518813_1_19_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:55,279 DEBUG [TezTR-518813_1_19_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:55,279 DEBUG [TezTR-518813_1_19_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:55,279 INFO [TezTR-518813_1_19_1_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:55,279 DEBUG [TezTR-518813_1_19_1_0_0] exec.GroupByOperator: Initialization Done 4 GBY 2018-01-05T11:28:55,279 DEBUG [TezTR-518813_1_19_1_0_0] exec.GroupByOperator: Operator 4 GBY initialized 2018-01-05T11:28:55,279 DEBUG [TezTR-518813_1_19_1_0_0] exec.GroupByOperator: Initializing children of 4 GBY 2018-01-05T11:28:55,279 DEBUG [TezTR-518813_1_19_1_0_0] exec.FileSinkOperator: Initializing child 6 FS 2018-01-05T11:28:55,279 INFO [TezTR-518813_1_19_1_0_0] exec.FileSinkOperator: Initializing operator FS[6] 2018-01-05T11:28:55,279 DEBUG [TezTR-518813_1_19_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:55,280 INFO [TezTR-518813_1_19_1_0_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@7eea3466]:[_col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11]:[struct, struct, struct, struct, struct, struct, struct, struct, struct, struct, struct, struct]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@18d84e08 2018-01-05T11:28:55,280 DEBUG [TezTR-518813_1_19_1_0_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:28:55,280 DEBUG [TezTR-518813_1_19_1_0_0] exec.FileSinkOperator: Initialization Done 6 FS 2018-01-05T11:28:55,280 DEBUG [TezTR-518813_1_19_1_0_0] exec.FileSinkOperator: Operator 6 FS initialized 2018-01-05T11:28:55,280 DEBUG [TezTR-518813_1_19_1_0_0] exec.FileSinkOperator: Initialization Done 6 FS done is reset. 2018-01-05T11:28:55,280 DEBUG [TezTR-518813_1_19_1_0_0] exec.GroupByOperator: Initialization Done 4 GBY done is reset. 2018-01-05T11:28:55,280 INFO [TezTR-518813_1_19_1_0_0] tez.ReduceRecordProcessor: Id =4 Id =6 <\Children> Id = 4 null<\Parent> <\FS> <\Children> <\Parent> <\GBY> 2018-01-05T11:28:55,280 DEBUG [TezTR-518813_1_19_1_0_0] log.PerfLogger: 2018-01-05T11:28:55,280 INFO [TezTR-518813_1_19_1_0_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 2 2018-01-05T11:28:55,280 DEBUG [TezTR-518813_1_19_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:28:55,280 DEBUG [TezTR-518813_1_19_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:28:55,280 DEBUG [TezTR-518813_1_19_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:28:55,280 DEBUG [TezTR-518813_1_19_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:28:55,280 DEBUG [TezTR-518813_1_19_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:28:55,280 DEBUG [TezTR-518813_1_19_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:28:55,283 INFO [TezTR-518813_1_19_1_0_0] readers.UnorderedKVReader: Num Records read: 1 2018-01-05T11:28:55,283 DEBUG [TezTR-518813_1_19_1_0_0] log.PerfLogger: 2018-01-05T11:28:55,283 DEBUG [TezTR-518813_1_19_1_0_0] mr.ObjectCache: djaiswal_20180105112853_18452c72-dc2a-4031-8352-113fe0f24ba2_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:28:55,283 DEBUG [TezTR-518813_1_19_1_0_0] exec.GroupByOperator: close called for operator GBY[4] 2018-01-05T11:28:55,283 INFO [TezTR-518813_1_19_1_0_0] exec.GroupByOperator: Closing operator GBY[4] 2018-01-05T11:28:55,284 INFO [TezTR-518813_1_19_1_0_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-54_051_544946193635317641-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-54_051_544946193635317641-1/_tmp.-ext-10001/000000_0 2018-01-05T11:28:55,284 INFO [TezTR-518813_1_19_1_0_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-54_051_544946193635317641-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-54_051_544946193635317641-1/_task_tmp.-ext-10001/_tmp.000000_0 2018-01-05T11:28:55,284 INFO [TezTR-518813_1_19_1_0_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-54_051_544946193635317641-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-54_051_544946193635317641-1/_tmp.-ext-10001/000000_0 2018-01-05T11:28:55,319 INFO [TezTR-518813_1_19_1_0_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:28:55,320 DEBUG [TezTR-518813_1_19_1_0_0] exec.GroupByOperator: Closing child = FS[6] 2018-01-05T11:28:55,320 DEBUG [TezTR-518813_1_19_1_0_0] exec.FileSinkOperator: close called for operator FS[6] 2018-01-05T11:28:55,320 DEBUG [TezTR-518813_1_19_1_0_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:55,320 INFO [TezTR-518813_1_19_1_0_0] exec.FileSinkOperator: Closing operator FS[6] 2018-01-05T11:28:55,320 INFO [TezTR-518813_1_19_1_0_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:28:55,320 INFO [TezTR-518813_1_19_1_0_0] exec.FileSinkOperator: RECORDS_OUT_0:1, 2018-01-05T11:28:55,320 DEBUG [TezTR-518813_1_19_1_0_0] exec.FileSinkOperator: 6 Close done 2018-01-05T11:28:55,320 DEBUG [TezTR-518813_1_19_1_0_0] exec.GroupByOperator: 4 Close done 2018-01-05T11:28:55,320 INFO [TezTR-518813_1_19_1_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_19_01_000000_0 2018-01-05T11:28:55,320 INFO [TezTR-518813_1_19_1_0_0] impl.ShuffleInputEventHandlerImpl: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:28:55,320 INFO [TezTR-518813_1_19_1_0_0] impl.ShuffleManager: Shutting down pending fetchers on sourceMap_1: 0 2018-01-05T11:28:55,320 INFO [TezTR-518813_1_19_1_0_0] output.MROutput: out_Reducer 2 closed 2018-01-05T11:28:55,320 DEBUG [TezTR-518813_1_19_1_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:55,320 INFO [TezTR-518813_1_19_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:55,321 INFO [TezTR-518813_1_19_1_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_19_01_000000_0, askedToStop=false 2018-01-05T11:28:55,321 INFO [TezTR-518813_1_19_1_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_19_01_000000_0, stopRequested=false 2018-01-05T11:28:55,321 INFO [TezTR-518813_1_19_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_19_01_000000_0: Counters: 14 [[org.apache.tez.common.counters.TaskCounter NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=54907, SHUFFLE_BYTES_DECOMPRESSED=54903, SHUFFLE_BYTES_TO_MEM=54907, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=2, FIRST_EVENT_RECEIVED=0, LAST_EVENT_RECEIVED=0][HIVE CREATED_FILES=1, RECORDS_OUT_0=1]] 2018-01-05T11:28:55,321 DEBUG [TezTR-518813_1_19_1_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_19_1_0_0 thread id: 1319 scheme: file bytesRead: 0 bytesWritten: 73876 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:55,321 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_19_01_000000_0, isTaskDone=true 2018-01-05T11:28:55,321 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000039, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_19_01_000000_0, eventCount=2 } 2018-01-05T11:28:55,322 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_19][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_19_01_000000_0, creationTime=1515180535263, allocationTime=1515180535264, startTime=1515180535265, finishTime=1515180535322, timeTaken=57, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:55,322 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_19_01_000000_0 2018-01-05T11:28:55,322 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_19_01_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:55,322 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_19_01_000000_0; false 2018-01-05T11:28:55,322 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:28:55,322 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_19_01_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:55,322 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_19_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:55,322 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_19_01_000000_0 2018-01-05T11:28:55,322 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_19][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_19_01_000000, startTime=1515180535265, finishTime=1515180535322, timeTaken=57, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_19_01_000000_0, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=73876, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=54907, SHUFFLE_BYTES_DECOMPRESSED=54903, SHUFFLE_BYTES_TO_MEM=54907, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=2, FIRST_EVENT_RECEIVED=0, LAST_EVENT_RECEIVED=0, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1 2018-01-05T11:28:55,322 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:55,323 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:55,323 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_19_01 [Reducer 2], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:55,323 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000039=55 2018-01-05T11:28:55,323 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_19_01 [Reducer 2] 2018-01-05T11:28:55,323 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_19_01_000000_0: true 2018-01-05T11:28:55,323 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_19][Event:VERTEX_FINISHED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_19_01, initRequestedTime=1515180534298, initedTime=1515180534305, startRequestedTime=1515180534321, startedTime=1515180534321, finishTime=1515180535323, timeTaken=1002, status=SUCCEEDED, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=73876, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=54907, SHUFFLE_BYTES_DECOMPRESSED=54903, SHUFFLE_BYTES_TO_MEM=54907, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=2, FIRST_EVENT_RECEIVED=0, LAST_EVENT_RECEIVED=0, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1, vertexStats=firstTaskStartTime=1515180535265, firstTasksToStart=[ task_1515180518813_0001_19_01_000000 ], lastTaskFinishTime=1515180535322, lastTasksToFinish=[ task_1515180518813_0001_19_01_000000 ], minTaskDuration=57, maxTaskDuration=57, avgTaskDuration=57.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_19_01_000000 ], longestDurationTasks=[ task_1515180518813_0001_19_01_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:55,324 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_19_01 [Reducer 2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:55,324 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_19][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000039, stoppedTime=1515180535324, exitStatus=0 2018-01-05T11:28:55,324 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000039 exited with diagnostics set to Container completed. 2018-01-05T11:28:55,324 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_19_01 [Reducer 2] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2 2018-01-05T11:28:55,324 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null 2018-01-05T11:28:55,324 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_19_01 [Reducer 2] 2018-01-05T11:28:55,324 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_19_00 [Map 1] 2018-01-05T11:28:55,325 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_19, queueSize=0 2018-01-05T11:28:55,325 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_19][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_19, startTime=1515180534289, finishTime=1515180535324, timeTaken=1035, status=SUCCEEDED, diagnostics=, counters=Counters: 47, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, AM_CPU_MILLISECONDS=2399, AM_GC_TIME_MILLIS=22, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=728549, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=12289, INPUT_SPLIT_LENGTH_BYTES=376777, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=54893, OUTPUT_BYTES_WITH_OVERHEAD=54903, OUTPUT_BYTES_PHYSICAL=54931, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=54907, SHUFFLE_BYTES_DECOMPRESSED=54903, SHUFFLE_BYTES_TO_MEM=54907, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=2, FIRST_EVENT_RECEIVED=0, LAST_EVENT_RECEIVED=0, HIVE, CREATED_FILES=2, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=12288, RECORDS_OUT_0=1, RECORDS_OUT_1_default.alltypesparquet=12288, RECORDS_OUT_INTERMEDIATE_Map_1=1, org.apache.hadoop.hive.llap.counters.LlapIOCounters, CACHE_HIT_BYTES=283052, CACHE_MISS_BYTES=0, CONSUMER_TIME_NS=613186356, DECODE_TIME_NS=24340296, HDFS_TIME_NS=142615, METADATA_CACHE_HIT=4, NUM_DECODED_BATCHES=3, NUM_VECTOR_BATCHES=13, ROWS_EMITTED=12288, SELECTED_ROWGROUPS=3, TOTAL_IO_TIME_NS=36810377 2018-01-05T11:28:55,325 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_19 finished with state: SUCCEEDED 2018-01-05T11:28:55,325 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_19 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:28:55,325 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000039 2018-01-05T11:28:55,325 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_19, dagState=SUCCEEDED 2018-01-05T11:28:55,326 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_19_post 2018-01-05T11:28:55,326 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:55,326 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:28:55,326 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:28:55,326 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 19 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=2, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=2,NumRequestsWithoutLocation=2, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=2} 2018-01-05T11:28:55,326 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:28:55,326 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=INSERT OVERWRITE TABLE alltype...alltypesorc (Stage-1), with id=dag_1515180518813_0001_19 2018-01-05T11:28:55,326 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 19, to 10.22.8.182:49818 2018-01-05T11:28:55,326 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=2, Remaining=0 2018-01-05T11:28:55,326 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=INSERT OVERWRITE TABLE alltype...alltypesorc (Stage-1), with id=dag_1515180518813_0001_19 2018-01-05T11:28:55,326 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=19} 2018-01-05T11:28:55,326 INFO [IPC Server handler 3 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=19}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:28:55,326 INFO [IPC Server handler 3 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/19 after 300 seconds 2018-01-05T11:28:55,326 ERROR [IPC Server handler 3 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112853_18452c72-dc2a-4031-8352-113fe0f24ba2, dag_1515180518813_0001_19 2018-01-05T11:28:55,326 INFO [IPC Server handler 3 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112853_18452c72-dc2a-4031-8352-113fe0f24ba2 2018-01-05T11:28:55,326 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=19} = 1 2018-01-05T11:28:55,326 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Issuing killFragment for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=19} attempt_1515180518813_0001_19_01_000000_0 2018-01-05T11:28:55,326 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Removing attempt_1515180518813_0001_19_01_000000_0 from preemptionQueue 2018-01-05T11:28:55,326 INFO [IPC Server handler 3 on 49818] impl.TaskRunnerCallable: Ignoring kill request for task attempt_1515180518813_0001_19_01_000000_0 since it's already complete 2018-01-05T11:28:55,326 DEBUG [IPC Server handler 3 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=19} 2018-01-05T11:28:55,326 DEBUG [IPC Server handler 3 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:28:55,326 DEBUG [IPC Server handler 3 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=19} 2018-01-05T11:28:55,327 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,327 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,327 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,327 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 Reducer 2: 1/1 2018-01-05T11:28:55,327 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,329 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,329 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: TaskId for 000000_0 = 000000 2018-01-05T11:28:55,329 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,329 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,329 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,330 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,330 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: TaskId for 000000_0 = 000000 2018-01-05T11:28:55,330 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,330 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,330 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,332 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-2:DEPENDENCY_COLLECTION] in serial mode 2018-01-05T11:28:55,332 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MOVE] in serial mode 2018-01-05T11:28:55,332 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@6332f548, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:55,332 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:55,332 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,332 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:55,332 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:55,333 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:55,333 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:55,333 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,333 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:55,333 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Loading data to table default.alltypesparquet from file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet/.hive-staging_hive_2018-01-05_11-28-53_925_2323731700822862447-1/-ext-10000 2018-01-05T11:28:55,333 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,333 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:55,334 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:55,334 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:55,334 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:55,334 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:55,334 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:55,334 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:55,335 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:55,335 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:55,335 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:55,335 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:55,335 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,335 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:55,335 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,335 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesparquet 2018-01-05T11:28:55,335 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesparquet 2018-01-05T11:28:55,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:55,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,339 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesparquet 2018-01-05T11:28:55,339 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesparquet 2018-01-05T11:28:55,341 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:55,341 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,341 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] FileOperations: moving file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet/.hive-staging_hive_2018-01-05_11-28-53_925_2323731700822862447-1/-ext-10000 to file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet (replace = REPLACE_ALL) 2018-01-05T11:28:55,342 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] FileOperations: Deleting old paths for replace in file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet and old path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet 2018-01-05T11:28:55,342 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet/ and the destination path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet/ 2018-01-05T11:28:55,342 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet 2018-01-05T11:28:55,342 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet/.hive-staging_hive_2018-01-05_11-28-53_925_2323731700822862447-1/-ext-10000/ and the destination path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet/ 2018-01-05T11:28:55,342 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet/ and the destination path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet/.hive-staging_hive_2018-01-05_11-28-53_925_2323731700822862447-1/-ext-10000/ 2018-01-05T11:28:55,342 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:55,342 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:55,358 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,358 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=alltypesparquet newtbl=alltypesparquet 2018-01-05T11:28:55,358 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=alltypesparquet newtbl=alltypesparquet 2018-01-05T11:28:55,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:55,362 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000038, QueryId=djaiswal_20180105112853_18452c72-dc2a-4031-8352-113fe0f24ba2-19, DagId=19, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180534340, EndTime=1515180535362 2018-01-05T11:28:55,363 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_19_01_000000_0 2018-01-05T11:28:55,363 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_19_01_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:55,363 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_19_01_000000_0 2018-01-05T11:28:55,363 INFO [ExecutionCompletionThread #0] impl.QueryTracker: Ignoring fragmentComplete message for unknown query: QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=19} 2018-01-05T11:28:55,369 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:55,369 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,370 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-3:STATS] in serial mode 2018-01-05T11:28:55,370 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@1cc1ddad, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:55,370 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:55,370 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,370 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:55,370 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:55,370 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:55,370 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:55,370 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,370 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:55,370 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:55,371 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_925_2323731700822862447-1/-mr-10001 2018-01-05T11:28:55,371 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_925_2323731700822862447-1/-mr-10001 2018-01-05T11:28:55,371 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,372 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:55,372 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:55,372 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:55,372 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:55,372 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:55,372 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:55,372 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:55,373 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:55,373 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:55,373 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:55,373 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:55,373 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,373 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:55,373 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,373 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesparquet 2018-01-05T11:28:55,373 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesparquet 2018-01-05T11:28:55,375 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:55,376 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,376 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.alltypesparquet/ numRows 2018-01-05T11:28:55,376 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.alltypesparquet/ numRows 12288 2018-01-05T11:28:55,377 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.alltypesparquet/ rawDataSize 2018-01-05T11:28:55,377 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.alltypesparquet/ rawDataSize 147456 2018-01-05T11:28:55,377 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,377 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=alltypesparquet newtbl=alltypesparquet 2018-01-05T11:28:55,377 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=alltypesparquet newtbl=alltypesparquet 2018-01-05T11:28:55,379 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:55,384 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:55,384 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,384 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.alltypesparquet stats: [numFiles=1, numRows=12288, totalSize=594976, rawDataSize=147456] 2018-01-05T11:28:55,384 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: About to delete stats tmp dir :file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_925_2323731700822862447-1/-mr-10001 2018-01-05T11:28:55,399 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Time taken to get FileStatuses: 13 2018-01-05T11:28:55,399 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:28:55,399 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 13 2018-01-05T11:28:55,399 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FetchOperator: Creating fetchTask with deserializer typeinfo: struct<_col0:struct,_col1:struct,_col2:struct,_col3:struct,_col4:struct,_col5:struct,_col6:struct,_col7:struct,_col8:struct,_col9:struct,_col10:struct,_col11:struct> 2018-01-05T11:28:55,399 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FetchOperator: deserializer properties: table properties: {columns=_col0,_col1,_col2,_col3,_col4,_col5,_col6,_col7,_col8,_col9,_col10,_col11, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct:struct:struct:struct:struct:struct:struct:struct:struct:struct:struct:struct, escape.delim=\} partition properties: {columns=_col0,_col1,_col2,_col3,_col4,_col5,_col6,_col7,_col8,_col9,_col10,_col11, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct:struct:struct:struct:struct:struct:struct:struct:struct:struct:struct:struct, escape.delim=\} 2018-01-05T11:28:55,402 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,402 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: write_column_statistics: db=default table=alltypesparquet 2018-01-05T11:28:55,402 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=write_column_statistics: db=default table=alltypesparquet 2018-01-05T11:28:55,404 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=alltypesparquet tableName=default colName=ctinyint 2018-01-05T11:28:55,405 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=alltypesparquet tableName=default colName=csmallint 2018-01-05T11:28:55,405 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=alltypesparquet tableName=default colName=cint 2018-01-05T11:28:55,406 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:55,406 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=alltypesparquet tableName=default colName=cbigint 2018-01-05T11:28:55,407 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=alltypesparquet tableName=default colName=cfloat 2018-01-05T11:28:55,408 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=alltypesparquet tableName=default colName=cdouble 2018-01-05T11:28:55,408 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=alltypesparquet tableName=default colName=cstring1 2018-01-05T11:28:55,409 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=alltypesparquet tableName=default colName=cstring2 2018-01-05T11:28:55,409 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:55,410 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=alltypesparquet tableName=default colName=ctimestamp1 2018-01-05T11:28:55,410 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=alltypesparquet tableName=default colName=ctimestamp2 2018-01-05T11:28:55,411 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=alltypesparquet tableName=default colName=cboolean1 2018-01-05T11:28:55,411 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=alltypesparquet tableName=default colName=cboolean2 2018-01-05T11:28:55,413 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:55,414 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,414 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,414 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,414 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: INSERT OVERWRITE TABLE alltypesparquet SELECT * FROM alltypesorc 2018-01-05T11:28:55,414 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:28:55,414 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@alltypesorc 2018-01-05T11:28:55,414 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@alltypesparquet 2018-01-05T11:28:55,414 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Lineage: alltypesparquet.cbigint SIMPLE [(alltypesorc)alltypesorc.FieldSchema(name:cbigint, type:bigint, comment:null), ] 2018-01-05T11:28:55,414 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Lineage: alltypesparquet.cboolean1 SIMPLE [(alltypesorc)alltypesorc.FieldSchema(name:cboolean1, type:boolean, comment:null), ] 2018-01-05T11:28:55,414 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Lineage: alltypesparquet.cboolean2 SIMPLE [(alltypesorc)alltypesorc.FieldSchema(name:cboolean2, type:boolean, comment:null), ] 2018-01-05T11:28:55,414 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Lineage: alltypesparquet.cdouble SIMPLE [(alltypesorc)alltypesorc.FieldSchema(name:cdouble, type:double, comment:null), ] 2018-01-05T11:28:55,414 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Lineage: alltypesparquet.cfloat SIMPLE [(alltypesorc)alltypesorc.FieldSchema(name:cfloat, type:float, comment:null), ] 2018-01-05T11:28:55,414 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Lineage: alltypesparquet.cint SIMPLE [(alltypesorc)alltypesorc.FieldSchema(name:cint, type:int, comment:null), ] 2018-01-05T11:28:55,414 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Lineage: alltypesparquet.csmallint SIMPLE [(alltypesorc)alltypesorc.FieldSchema(name:csmallint, type:smallint, comment:null), ] 2018-01-05T11:28:55,414 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Lineage: alltypesparquet.cstring1 SIMPLE [(alltypesorc)alltypesorc.FieldSchema(name:cstring1, type:string, comment:null), ] 2018-01-05T11:28:55,414 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Lineage: alltypesparquet.cstring2 SIMPLE [(alltypesorc)alltypesorc.FieldSchema(name:cstring2, type:string, comment:null), ] 2018-01-05T11:28:55,414 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Lineage: alltypesparquet.ctimestamp1 SIMPLE [(alltypesorc)alltypesorc.FieldSchema(name:ctimestamp1, type:timestamp, comment:null), ] 2018-01-05T11:28:55,414 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Lineage: alltypesparquet.ctimestamp2 SIMPLE [(alltypesorc)alltypesorc.FieldSchema(name:ctimestamp2, type:timestamp, comment:null), ] 2018-01-05T11:28:55,414 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Lineage: alltypesparquet.ctinyint SIMPLE [(alltypesorc)alltypesorc.FieldSchema(name:ctinyint, type:tinyint, comment:null), ] 2018-01-05T11:28:55,414 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,414 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,414 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:55,414 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=7, getTable_(String, String, )=3, setPartitionColumnStatistics_(SetPartitionsStatsRequest, )=12} 2018-01-05T11:28:55,414 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112853_18452c72-dc2a-4031-8352-113fe0f24ba2); Time taken: 1.286 seconds 2018-01-05T11:28:55,414 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:55,414 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,414 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:55,414 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,414 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query INSERT OVERWRITE TABLE alltypesparquet SELECT * FROM alltypesorc 2018-01-05T11:28:55,414 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet/.hive-staging_hive_2018-01-05_11-28-53_925_2323731700822862447-1 2018-01-05T11:28:55,415 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-53_925_2323731700822862447-1 2018-01-05T11:28:55,415 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 1.489 seconds 2018-01-05T11:28:55,415 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:55,415 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:55,415 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:55,415 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:55,415 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,415 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:55,415 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,415 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,415 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,415 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: ANALYZE TABLE alltypesparquet COMPUTE STATISTICS 2018-01-05T11:28:55,415 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112855_6cd1ed81-d48d-42b1-ae0c-1baf3e08b1ec): ANALYZE TABLE alltypesparquet COMPUTE STATISTICS 2018-01-05T11:28:55,416 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,416 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: ANALYZE TABLE alltypesparquet COMPUTE STATISTICS 2018-01-05T11:28:55,416 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:55,416 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,416 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,416 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,416 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,416 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:55,416 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Invoking analyze on original query 2018-01-05T11:28:55,416 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Starting Semantic Analysis 2018-01-05T11:28:55,416 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed phase 1 of Semantic Analysis 2018-01-05T11:28:55,416 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for source tables 2018-01-05T11:28:55,416 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,416 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesparquet 2018-01-05T11:28:55,416 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesparquet 2018-01-05T11:28:55,418 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:55,419 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,419 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,419 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesparquet 2018-01-05T11:28:55,419 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesparquet 2018-01-05T11:28:55,421 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:55,422 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,422 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for subqueries 2018-01-05T11:28:55,422 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for destination tables 2018-01-05T11:28:55,422 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed getting MetaData in Semantic Analysis 2018-01-05T11:28:55,434 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-55_416_8910412052478619212-1/-mr-10000 2018-01-05T11:28:55,434 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Table Plan for alltypesparquet TS[0] 2018-01-05T11:28:55,434 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Body Plan for Query Block null 2018-01-05T11:28:55,434 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Plan for Query Block null 2018-01-05T11:28:55,434 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Before logical optimization TS[0] 2018-01-05T11:28:55,434 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,434 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,434 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] lineage.Generator: Time taken for lineage transform=0 2018-01-05T11:28:55,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(ctinyint: tinyint|{alltypesparquet}ctinyint,csmallint: smallint|{alltypesparquet}csmallint,cint: int|{alltypesparquet}cint,cbigint: bigint|{alltypesparquet}cbigint,cfloat: float|{alltypesparquet}cfloat,cdouble: double|{alltypesparquet}cdouble,cstring1: string|{alltypesparquet}cstring1,cstring2: string|{alltypesparquet}cstring2,ctimestamp1: timestamp|{alltypesparquet}ctimestamp1,ctimestamp2: timestamp|{alltypesparquet}ctimestamp2,cboolean1: boolean|{alltypesparquet}cboolean1,cboolean2: boolean|{alltypesparquet}cboolean2,BLOCK__OFFSET__INSIDE__FILE: bigint|{alltypesparquet}block__offset__inside__file,INPUT__FILE__NAME: string|{alltypesparquet}input__file__name,ROW__ID: struct|{alltypesparquet}row__id,RAW__DATA__SIZE: bigint|{alltypesparquet}raw__data__size) 2018-01-05T11:28:55,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,435 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:28:55,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.PredicatePushDown: After PPD: TS[0] 2018-01-05T11:28:55,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(ctinyint: tinyint|{alltypesparquet}ctinyint,csmallint: smallint|{alltypesparquet}csmallint,cint: int|{alltypesparquet}cint,cbigint: bigint|{alltypesparquet}cbigint,cfloat: float|{alltypesparquet}cfloat,cdouble: double|{alltypesparquet}cdouble,cstring1: string|{alltypesparquet}cstring1,cstring2: string|{alltypesparquet}cstring2,ctimestamp1: timestamp|{alltypesparquet}ctimestamp1,ctimestamp2: timestamp|{alltypesparquet}ctimestamp2,cboolean1: boolean|{alltypesparquet}cboolean1,cboolean2: boolean|{alltypesparquet}cboolean2,BLOCK__OFFSET__INSIDE__FILE: bigint|{alltypesparquet}block__offset__inside__file,INPUT__FILE__NAME: string|{alltypesparquet}input__file__name,ROW__ID: struct|{alltypesparquet}row__id,RAW__DATA__SIZE: bigint|{alltypesparquet}raw__data__size) 2018-01-05T11:28:55,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(ctinyint: tinyint|{alltypesparquet}ctinyint,csmallint: smallint|{alltypesparquet}csmallint,cint: int|{alltypesparquet}cint,cbigint: bigint|{alltypesparquet}cbigint,cfloat: float|{alltypesparquet}cfloat,cdouble: double|{alltypesparquet}cdouble,cstring1: string|{alltypesparquet}cstring1,cstring2: string|{alltypesparquet}cstring2,ctimestamp1: timestamp|{alltypesparquet}ctimestamp1,ctimestamp2: timestamp|{alltypesparquet}ctimestamp2,cboolean1: boolean|{alltypesparquet}cboolean1,cboolean2: boolean|{alltypesparquet}cboolean2,BLOCK__OFFSET__INSIDE__FILE: bigint|{alltypesparquet}block__offset__inside__file,INPUT__FILE__NAME: string|{alltypesparquet}input__file__name,ROW__ID: struct|{alltypesparquet}row__id,RAW__DATA__SIZE: bigint|{alltypesparquet}raw__data__size) 2018-01-05T11:28:55,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,435 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,436 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,436 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,436 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,436 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,436 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,436 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,436 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,436 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,436 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,436 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,436 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,436 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,436 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,436 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,436 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,436 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,436 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,436 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,436 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: After logical optimization TS[0] 2018-01-05T11:28:55,436 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,436 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,436 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,436 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,436 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,436 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,436 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=alltypesparquet 2018-01-05T11:28:55,436 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=alltypesparquet 2018-01-05T11:28:55,439 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:55,441 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting stats (Num rows: 12288 Data size: 147456 Basic stats: COMPLETE Column stats: COMPLETE) on TS[0] 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (alltypesparquet): numRows: 12288 dataSize: 147456 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on TS[0] 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:28:55,442 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Cycle free: true 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(ctinyint: tinyint|{alltypesparquet}ctinyint,csmallint: smallint|{alltypesparquet}csmallint,cint: int|{alltypesparquet}cint,cbigint: bigint|{alltypesparquet}cbigint,cfloat: float|{alltypesparquet}cfloat,cdouble: double|{alltypesparquet}cdouble,cstring1: string|{alltypesparquet}cstring1,cstring2: string|{alltypesparquet}cstring2,ctimestamp1: timestamp|{alltypesparquet}ctimestamp1,ctimestamp2: timestamp|{alltypesparquet}ctimestamp2,cboolean1: boolean|{alltypesparquet}cboolean1,cboolean2: boolean|{alltypesparquet}cboolean2,BLOCK__OFFSET__INSIDE__FILE: bigint|{alltypesparquet}block__offset__inside__file,INPUT__FILE__NAME: string|{alltypesparquet}input__file__name,ROW__ID: struct|{alltypesparquet}row__id,RAW__DATA__SIZE: bigint|{alltypesparquet}raw__data__size) 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping vectorization 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:28:55,442 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: llap mode: only 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.QueryPlanPostProcessor: Found org.apache.hadoop.hive.ql.plan.StatsWork - no FileSinkOperation can be present. executionId=hive_2018-01-05_11-28-55_416_8910412052478619212 2018-01-05T11:28:55,442 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed plan generation 2018-01-05T11:28:55,442 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validation start 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Not a partition. 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,442 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:alltypesparquet.ctinyint, type:tinyint, comment:null), FieldSchema(name:alltypesparquet.csmallint, type:smallint, comment:null), FieldSchema(name:alltypesparquet.cint, type:int, comment:null), FieldSchema(name:alltypesparquet.cbigint, type:bigint, comment:null), FieldSchema(name:alltypesparquet.cfloat, type:float, comment:null), FieldSchema(name:alltypesparquet.cdouble, type:double, comment:null), FieldSchema(name:alltypesparquet.cstring1, type:string, comment:null), FieldSchema(name:alltypesparquet.cstring2, type:string, comment:null), FieldSchema(name:alltypesparquet.ctimestamp1, type:timestamp, comment:null), FieldSchema(name:alltypesparquet.ctimestamp2, type:timestamp, comment:null), FieldSchema(name:alltypesparquet.cboolean1, type:boolean, comment:null), FieldSchema(name:alltypesparquet.cboolean2, type:boolean, comment:null)], properties:null) 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,442 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:55,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=6, flushCache_()=0, getTableColumnStatistics_(String, String, List, )=6} 2018-01-05T11:28:55,442 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112855_6cd1ed81-d48d-42b1-ae0c-1baf3e08b1ec); Time taken: 0.027 seconds 2018-01-05T11:28:55,443 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:55,443 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,443 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112855_6cd1ed81-d48d-42b1-ae0c-1baf3e08b1ec): ANALYZE TABLE alltypesparquet COMPUTE STATISTICS 2018-01-05T11:28:55,444 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,444 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: ANALYZE TABLE alltypesparquet COMPUTE STATISTICS 2018-01-05T11:28:55,444 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:28:55,444 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@alltypesparquet 2018-01-05T11:28:55,444 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@alltypesparquet 2018-01-05T11:28:55,444 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,444 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,444 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,444 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,444 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-2:STATS] in serial mode 2018-01-05T11:28:55,444 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@38cd4663, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:55,444 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:55,444 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,444 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:55,444 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:55,444 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:55,444 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:55,444 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,444 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:55,444 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsNoJobTask: Executing stats (no job) task 2018-01-05T11:28:55,444 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.StatsTask: Initialized threadpool for stats computation with 10 threads 2018-01-05T11:28:55,445 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsNoJobTask: Stats collection waiting for threadpool to shutdown.. 2018-01-05T11:28:55,445 DEBUG [StatsNoJobTask-Thread-0] stats.BasicStatsNoJobTask: Aggregating stats for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet 2018-01-05T11:28:55,445 DEBUG [StatsNoJobTask-Thread-0] stats.BasicStatsNoJobTask: Computing stats for DeprecatedRawLocalFileStatus{path=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet/000000_0; isDirectory=false; length=594976; replication=1; blocksize=33554432; modification_time=1515180535000; access_time=1515180534000; owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false} 2018-01-05T11:28:55,445 DEBUG [StatsNoJobTask-Thread-0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:55,450 DEBUG [StatsNoJobTask-Thread-0] parquet.MapredParquetInputFormat: Using row-mode record reader 2018-01-05T11:28:55,467 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000039, QueryId=djaiswal_20180105112853_18452c72-dc2a-4031-8352-113fe0f24ba2-19, DagId=19, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180535267, EndTime=1515180535467 2018-01-05T11:28:55,477 WARN [StatsNoJobTask-Thread-0] parquet.ParquetRecordReaderBase: Skipping split, could not find row group in: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet/000000_0:0+0 2018-01-05T11:28:55,477 DEBUG [StatsNoJobTask-Thread-0] stats.BasicStatsNoJobTask: Table default.alltypesparquet stats: [numFiles=1, numRows=12288, totalSize=594976, rawDataSize=593707] 2018-01-05T11:28:55,477 INFO [StatsNoJobTask-Thread-0] stats.BasicStatsNoJobTask: Table default.alltypesparquet stats: [numFiles=1, numRows=12288, totalSize=594976, rawDataSize=593707] 2018-01-05T11:28:55,477 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsNoJobTask: Stats collection threadpool shutdown successful. 2018-01-05T11:28:55,477 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsNoJobTask: Collectors.size(): [default@alltypesparquet#PTable] 2018-01-05T11:28:55,477 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsNoJobTask: Updating stats for: default.alltypesparquet 2018-01-05T11:28:55,478 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,478 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:55,478 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:55,478 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:55,478 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:55,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:55,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:55,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:55,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:55,480 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:55,480 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:55,480 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:55,480 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,480 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:55,480 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,480 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=alltypesparquet newtbl=alltypesparquet 2018-01-05T11:28:55,480 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=alltypesparquet newtbl=alltypesparquet 2018-01-05T11:28:55,482 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:55,489 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:55,489 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,489 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsNoJobTask: Updated stats for default.alltypesparquet. 2018-01-05T11:28:55,489 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsNoJobTask: Updated stats for: default.alltypesparquet 2018-01-05T11:28:55,489 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,489 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,489 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: ANALYZE TABLE alltypesparquet COMPUTE STATISTICS 2018-01-05T11:28:55,489 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:28:55,489 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@alltypesparquet 2018-01-05T11:28:55,489 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@alltypesparquet 2018-01-05T11:28:55,489 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,489 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,489 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:55,489 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=9} 2018-01-05T11:28:55,489 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112855_6cd1ed81-d48d-42b1-ae0c-1baf3e08b1ec); Time taken: 0.046 seconds 2018-01-05T11:28:55,489 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:55,489 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,489 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:55,489 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,489 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query ANALYZE TABLE alltypesparquet COMPUTE STATISTICS 2018-01-05T11:28:55,490 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-55_416_8910412052478619212-1 2018-01-05T11:28:55,490 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.074 seconds 2018-01-05T11:28:55,490 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:55,490 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:55,490 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:55,490 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:55,490 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,490 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:55,490 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,490 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,490 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,490 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: ANALYZE TABLE alltypesparquet COMPUTE STATISTICS FOR COLUMNS ctinyint,csmallint,cint,cbigint,cfloat,cdouble,cstring1,cstring2,ctimestamp1,ctimestamp2,cboolean1,cboolean2 2018-01-05T11:28:55,490 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112855_cf8184a1-b651-405c-b849-0ee626114470): ANALYZE TABLE alltypesparquet COMPUTE STATISTICS FOR COLUMNS ctinyint,csmallint,cint,cbigint,cfloat,cdouble,cstring1,cstring2,ctimestamp1,ctimestamp2,cboolean1,cboolean2 2018-01-05T11:28:55,490 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,490 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: ANALYZE TABLE alltypesparquet COMPUTE STATISTICS FOR COLUMNS ctinyint,csmallint,cint,cbigint,cfloat,cdouble,cstring1,cstring2,ctimestamp1,ctimestamp2,cboolean1,cboolean2 2018-01-05T11:28:55,490 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:55,490 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,490 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,491 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,491 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,491 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:55,491 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,491 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesparquet 2018-01-05T11:28:55,491 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesparquet 2018-01-05T11:28:55,494 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:55,495 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,495 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: select compute_stats(`ctinyint`, 'hll') , compute_stats(`csmallint`, 'hll') , compute_stats(`cint`, 'hll') , compute_stats(`cbigint`, 'hll') , compute_stats(`cfloat`, 'hll') , compute_stats(`cdouble`, 'hll') , compute_stats(`cstring1`, 'hll') , compute_stats(`cstring2`, 'hll') , compute_stats(`ctimestamp1`, 'hll') , compute_stats(`ctimestamp2`, 'hll') , compute_stats(`cboolean1`, 'hll') , compute_stats(`cboolean2`, 'hll') from `default`.`alltypesparquet` 2018-01-05T11:28:55,495 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: select compute_stats(`ctinyint`, 'hll') , compute_stats(`csmallint`, 'hll') , compute_stats(`cint`, 'hll') , compute_stats(`cbigint`, 'hll') , compute_stats(`cfloat`, 'hll') , compute_stats(`cdouble`, 'hll') , compute_stats(`cstring1`, 'hll') , compute_stats(`cstring2`, 'hll') , compute_stats(`ctimestamp1`, 'hll') , compute_stats(`ctimestamp2`, 'hll') , compute_stats(`cboolean1`, 'hll') , compute_stats(`cboolean2`, 'hll') from `default`.`alltypesparquet` 2018-01-05T11:28:55,497 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:55,497 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Invoking analyze on rewritten query 2018-01-05T11:28:55,497 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Starting Semantic Analysis 2018-01-05T11:28:55,497 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:55,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:55,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:55,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:55,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:55,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:55,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:55,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:55,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:55,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:55,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:55,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:55,498 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed phase 1 of Semantic Analysis 2018-01-05T11:28:55,498 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for source tables 2018-01-05T11:28:55,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,498 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesparquet 2018-01-05T11:28:55,498 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesparquet 2018-01-05T11:28:55,501 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:55,501 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,502 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for subqueries 2018-01-05T11:28:55,502 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for destination tables 2018-01-05T11:28:55,514 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed getting MetaData in Semantic Analysis 2018-01-05T11:28:55,514 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-55_495_9015283846029904157-1/-mr-10001 2018-01-05T11:28:55,514 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Table Plan for alltypesparquet TS[0] 2018-01-05T11:28:55,514 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:55,514 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:55,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:55,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:55,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:55,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:55,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:55,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:55,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:55,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:55,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:55,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:55,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: RR before GB alltypesparquet{(ctinyint,ctinyint: tinyint)(csmallint,csmallint: smallint)(cint,cint: int)(cbigint,cbigint: bigint)(cfloat,cfloat: float)(cdouble,cdouble: double)(cstring1,cstring1: string)(cstring2,cstring2: string)(ctimestamp1,ctimestamp1: timestamp)(ctimestamp2,ctimestamp2: timestamp)(cboolean1,cboolean1: boolean)(cboolean2,cboolean2: boolean)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)(RAW__DATA__SIZE,RAW__DATA__SIZE: bigint)} after GB {((tok_function compute_stats (tok_table_or_col ctinyint) 'hll'),_col0: struct)((tok_function compute_stats (tok_table_or_col csmallint) 'hll'),_col1: struct)((tok_function compute_stats (tok_table_or_col cint) 'hll'),_col2: struct)((tok_function compute_stats (tok_table_or_col cbigint) 'hll'),_col3: struct)((tok_function compute_stats (tok_table_or_col cfloat) 'hll'),_col4: struct)((tok_function compute_stats (tok_table_or_col cdouble) 'hll'),_col5: struct)((tok_function compute_stats (tok_table_or_col cstring1) 'hll'),_col6: struct)((tok_function compute_stats (tok_table_or_col cstring2) 'hll'),_col7: struct)((tok_function compute_stats (tok_table_or_col ctimestamp1) 'hll'),_col8: struct)((tok_function compute_stats (tok_table_or_col ctimestamp2) 'hll'),_col9: struct)((tok_function compute_stats (tok_table_or_col cboolean1) 'hll'),_col10: struct)((tok_function compute_stats (tok_table_or_col cboolean2) 'hll'),_col11: struct)} 2018-01-05T11:28:55,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: tree: (tok_select (tok_selexpr (tok_function compute_stats (tok_table_or_col ctinyint) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col csmallint) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col cint) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col cbigint) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col cfloat) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col cdouble) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col cstring1) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col cstring2) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col ctimestamp1) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col ctimestamp2) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col cboolean1) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col cboolean2) 'hll'))) 2018-01-05T11:28:55,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: genSelectPlan: input = {((tok_function compute_stats (tok_table_or_col ctinyint) 'hll'),_col0: struct)((tok_function compute_stats (tok_table_or_col csmallint) 'hll'),_col1: struct)((tok_function compute_stats (tok_table_or_col cint) 'hll'),_col2: struct)((tok_function compute_stats (tok_table_or_col cbigint) 'hll'),_col3: struct)((tok_function compute_stats (tok_table_or_col cfloat) 'hll'),_col4: struct)((tok_function compute_stats (tok_table_or_col cdouble) 'hll'),_col5: struct)((tok_function compute_stats (tok_table_or_col cstring1) 'hll'),_col6: struct)((tok_function compute_stats (tok_table_or_col cstring2) 'hll'),_col7: struct)((tok_function compute_stats (tok_table_or_col ctimestamp1) 'hll'),_col8: struct)((tok_function compute_stats (tok_table_or_col ctimestamp2) 'hll'),_col9: struct)((tok_function compute_stats (tok_table_or_col cboolean1) 'hll'),_col10: struct)((tok_function compute_stats (tok_table_or_col cboolean2) 'hll'),_col11: struct)} starRr = alltypesparquet{(ctinyint,ctinyint: tinyint)(csmallint,csmallint: smallint)(cint,cint: int)(cbigint,cbigint: bigint)(cfloat,cfloat: float)(cdouble,cdouble: double)(cstring1,cstring1: string)(cstring2,cstring2: string)(ctimestamp1,ctimestamp1: timestamp)(ctimestamp2,ctimestamp2: timestamp)(cboolean1,cboolean1: boolean)(cboolean2,cboolean2: boolean)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)(RAW__DATA__SIZE,RAW__DATA__SIZE: bigint)} 2018-01-05T11:28:55,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Select Plan row schema: null{(_c0,_col0: struct)(_c1,_col1: struct)(_c2,_col2: struct)(_c3,_col3: struct)(_c4,_col4: struct)(_c5,_col5: struct)(_c6,_col6: struct)(_c7,_col7: struct)(_c8,_col8: struct)(_c9,_col9: struct)(_c10,_col10: struct)(_c11,_col11: struct)} 2018-01-05T11:28:55,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Select Plan for clause: insclause-0 2018-01-05T11:28:55,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-55_495_9015283846029904157-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-55_495_9015283846029904157-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-55_495_9015283846029904157-1/-mr-10000 2018-01-05T11:28:55,516 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-55_495_9015283846029904157-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-55_495_9015283846029904157-1 2018-01-05T11:28:55,537 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-55_495_9015283846029904157-1/-mr-10003 2018-01-05T11:28:55,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-55_495_9015283846029904157-1/-mr-10000 row schema: null{(_c0,_col0: struct)(_c1,_col1: struct)(_c2,_col2: struct)(_c3,_col3: struct)(_c4,_col4: struct)(_c5,_col5: struct)(_c6,_col6: struct)(_c7,_col7: struct)(_c8,_col8: struct)(_c9,_col9: struct)(_c10,_col10: struct)(_c11,_col11: struct)} 2018-01-05T11:28:55,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Body Plan for Query Block null 2018-01-05T11:28:55,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Plan for Query Block null 2018-01-05T11:28:55,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Before logical optimization TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-SEL[5]-FS[6] 2018-01-05T11:28:55,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,538 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,539 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] lineage.Generator: Time taken for lineage transform=1 2018-01-05T11:28:55,539 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,539 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,539 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,539 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,539 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,539 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(ctinyint: tinyint|{alltypesparquet}ctinyint,csmallint: smallint|{alltypesparquet}csmallint,cint: int|{alltypesparquet}cint,cbigint: bigint|{alltypesparquet}cbigint,cfloat: float|{alltypesparquet}cfloat,cdouble: double|{alltypesparquet}cdouble,cstring1: string|{alltypesparquet}cstring1,cstring2: string|{alltypesparquet}cstring2,ctimestamp1: timestamp|{alltypesparquet}ctimestamp1,ctimestamp2: timestamp|{alltypesparquet}ctimestamp2,cboolean1: boolean|{alltypesparquet}cboolean1,cboolean2: boolean|{alltypesparquet}cboolean2,BLOCK__OFFSET__INSIDE__FILE: bigint|{alltypesparquet}block__offset__inside__file,INPUT__FILE__NAME: string|{alltypesparquet}input__file__name,ROW__ID: struct|{alltypesparquet}row__id,RAW__DATA__SIZE: bigint|{alltypesparquet}raw__data__size) 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(ctinyint: tinyint|{alltypesparquet}ctinyint,csmallint: smallint|{alltypesparquet}csmallint,cint: int|{alltypesparquet}cint,cbigint: bigint|{alltypesparquet}cbigint,cfloat: float|{alltypesparquet}cfloat,cdouble: double|{alltypesparquet}cdouble,cstring1: string|{alltypesparquet}cstring1,cstring2: string|{alltypesparquet}cstring2,ctimestamp1: timestamp|{alltypesparquet}ctimestamp1,ctimestamp2: timestamp|{alltypesparquet}ctimestamp2,cboolean1: boolean|{alltypesparquet}cboolean1,cboolean2: boolean|{alltypesparquet}cboolean2,BLOCK__OFFSET__INSIDE__FILE: bigint|{alltypesparquet}block__offset__inside__file,INPUT__FILE__NAME: string|{alltypesparquet}input__file__name,ROW__ID: struct|{alltypesparquet}row__id,RAW__DATA__SIZE: bigint|{alltypesparquet}raw__data__size) 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[ctinyint] Column[csmallint] Column[cint] Column[cbigint] Column[cfloat] Column[cdouble] Column[cstring1] Column[cstring2] Column[ctimestamp1] Column[ctimestamp2] Column[cboolean1] Column[cboolean2] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col ctinyint) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col csmallint) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col cint) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col cbigint) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col cfloat) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col cdouble) 'hll'),_col6: struct|{}(tok_function compute_stats (tok_table_or_col cstring1) 'hll'),_col7: struct|{}(tok_function compute_stats (tok_table_or_col cstring2) 'hll'),_col8: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp1) 'hll'),_col9: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp2) 'hll'),_col10: struct|{}(tok_function compute_stats (tok_table_or_col cboolean1) 'hll'),_col11: struct|{}(tok_function compute_stats (tok_table_or_col cboolean2) 'hll')) 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col ctinyint) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col csmallint) 'hll'),VALUE._col2: struct|{}(tok_function compute_stats (tok_table_or_col cint) 'hll'),VALUE._col3: struct|{}(tok_function compute_stats (tok_table_or_col cbigint) 'hll'),VALUE._col4: struct|{}(tok_function compute_stats (tok_table_or_col cfloat) 'hll'),VALUE._col5: struct|{}(tok_function compute_stats (tok_table_or_col cdouble) 'hll'),VALUE._col6: struct|{}(tok_function compute_stats (tok_table_or_col cstring1) 'hll'),VALUE._col7: struct|{}(tok_function compute_stats (tok_table_or_col cstring2) 'hll'),VALUE._col8: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp1) 'hll'),VALUE._col9: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp2) 'hll'),VALUE._col10: struct|{}(tok_function compute_stats (tok_table_or_col cboolean1) 'hll'),VALUE._col11: struct|{}(tok_function compute_stats (tok_table_or_col cboolean2) 'hll')) 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col ctinyint) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col csmallint) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col cint) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col cbigint) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col cfloat) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col cdouble) 'hll'),_col6: struct|{}(tok_function compute_stats (tok_table_or_col cstring1) 'hll'),_col7: struct|{}(tok_function compute_stats (tok_table_or_col cstring2) 'hll'),_col8: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp1) 'hll'),_col9: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp2) 'hll'),_col10: struct|{}(tok_function compute_stats (tok_table_or_col cboolean1) 'hll'),_col11: struct|{}(tok_function compute_stats (tok_table_or_col cboolean2) 'hll')) 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1,_col2: struct|{null}_c2,_col3: struct|{null}_c3,_col4: struct|{null}_c4,_col5: struct|{null}_c5,_col6: struct|{null}_c6,_col7: struct|{null}_c7,_col8: struct|{null}_c8,_col9: struct|{null}_c9,_col10: struct|{null}_c10,_col11: struct|{null}_c11) 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0] Column[_col1] Column[_col2] Column[_col3] Column[_col4] Column[_col5] Column[_col6] Column[_col7] Column[_col8] Column[_col9] Column[_col10] Column[_col11]) 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{},_col2: struct|{},_col3: struct|{},_col4: struct|{},_col5: struct|{},_col6: struct|{},_col7: struct|{},_col8: struct|{},_col9: struct|{},_col10: struct|{},_col11: struct|{}) 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,540 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for FS(6) 2018-01-05T11:28:55,540 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(5) 2018-01-05T11:28:55,540 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for GBY(4) 2018-01-05T11:28:55,540 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for RS(3) 2018-01-05T11:28:55,540 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for GBY(2) 2018-01-05T11:28:55,540 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(1) 2018-01-05T11:28:55,540 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.PredicatePushDown: After PPD: TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-SEL[5]-FS[6] 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(ctinyint: tinyint|{alltypesparquet}ctinyint,csmallint: smallint|{alltypesparquet}csmallint,cint: int|{alltypesparquet}cint,cbigint: bigint|{alltypesparquet}cbigint,cfloat: float|{alltypesparquet}cfloat,cdouble: double|{alltypesparquet}cdouble,cstring1: string|{alltypesparquet}cstring1,cstring2: string|{alltypesparquet}cstring2,ctimestamp1: timestamp|{alltypesparquet}ctimestamp1,ctimestamp2: timestamp|{alltypesparquet}ctimestamp2,cboolean1: boolean|{alltypesparquet}cboolean1,cboolean2: boolean|{alltypesparquet}cboolean2,BLOCK__OFFSET__INSIDE__FILE: bigint|{alltypesparquet}block__offset__inside__file,INPUT__FILE__NAME: string|{alltypesparquet}input__file__name,ROW__ID: struct|{alltypesparquet}row__id,RAW__DATA__SIZE: bigint|{alltypesparquet}raw__data__size) 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(ctinyint: tinyint|{alltypesparquet}ctinyint,csmallint: smallint|{alltypesparquet}csmallint,cint: int|{alltypesparquet}cint,cbigint: bigint|{alltypesparquet}cbigint,cfloat: float|{alltypesparquet}cfloat,cdouble: double|{alltypesparquet}cdouble,cstring1: string|{alltypesparquet}cstring1,cstring2: string|{alltypesparquet}cstring2,ctimestamp1: timestamp|{alltypesparquet}ctimestamp1,ctimestamp2: timestamp|{alltypesparquet}ctimestamp2,cboolean1: boolean|{alltypesparquet}cboolean1,cboolean2: boolean|{alltypesparquet}cboolean2,BLOCK__OFFSET__INSIDE__FILE: bigint|{alltypesparquet}block__offset__inside__file,INPUT__FILE__NAME: string|{alltypesparquet}input__file__name,ROW__ID: struct|{alltypesparquet}row__id,RAW__DATA__SIZE: bigint|{alltypesparquet}raw__data__size) 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[ctinyint] Column[csmallint] Column[cint] Column[cbigint] Column[cfloat] Column[cdouble] Column[cstring1] Column[cstring2] Column[ctimestamp1] Column[ctimestamp2] Column[cboolean1] Column[cboolean2] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col ctinyint) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col csmallint) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col cint) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col cbigint) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col cfloat) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col cdouble) 'hll'),_col6: struct|{}(tok_function compute_stats (tok_table_or_col cstring1) 'hll'),_col7: struct|{}(tok_function compute_stats (tok_table_or_col cstring2) 'hll'),_col8: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp1) 'hll'),_col9: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp2) 'hll'),_col10: struct|{}(tok_function compute_stats (tok_table_or_col cboolean1) 'hll'),_col11: struct|{}(tok_function compute_stats (tok_table_or_col cboolean2) 'hll')) 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col ctinyint) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col csmallint) 'hll'),VALUE._col2: struct|{}(tok_function compute_stats (tok_table_or_col cint) 'hll'),VALUE._col3: struct|{}(tok_function compute_stats (tok_table_or_col cbigint) 'hll'),VALUE._col4: struct|{}(tok_function compute_stats (tok_table_or_col cfloat) 'hll'),VALUE._col5: struct|{}(tok_function compute_stats (tok_table_or_col cdouble) 'hll'),VALUE._col6: struct|{}(tok_function compute_stats (tok_table_or_col cstring1) 'hll'),VALUE._col7: struct|{}(tok_function compute_stats (tok_table_or_col cstring2) 'hll'),VALUE._col8: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp1) 'hll'),VALUE._col9: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp2) 'hll'),VALUE._col10: struct|{}(tok_function compute_stats (tok_table_or_col cboolean1) 'hll'),VALUE._col11: struct|{}(tok_function compute_stats (tok_table_or_col cboolean2) 'hll')) 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col ctinyint) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col csmallint) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col cint) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col cbigint) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col cfloat) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col cdouble) 'hll'),_col6: struct|{}(tok_function compute_stats (tok_table_or_col cstring1) 'hll'),_col7: struct|{}(tok_function compute_stats (tok_table_or_col cstring2) 'hll'),_col8: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp1) 'hll'),_col9: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp2) 'hll'),_col10: struct|{}(tok_function compute_stats (tok_table_or_col cboolean1) 'hll'),_col11: struct|{}(tok_function compute_stats (tok_table_or_col cboolean2) 'hll')) 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1,_col2: struct|{null}_c2,_col3: struct|{null}_c3,_col4: struct|{null}_c4,_col5: struct|{null}_c5,_col6: struct|{null}_c6,_col7: struct|{null}_c7,_col8: struct|{null}_c8,_col9: struct|{null}_c9,_col10: struct|{null}_c10,_col11: struct|{null}_c11) 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0] Column[_col1] Column[_col2] Column[_col3] Column[_col4] Column[_col5] Column[_col6] Column[_col7] Column[_col8] Column[_col9] Column[_col10] Column[_col11]) 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{},_col2: struct|{},_col3: struct|{},_col4: struct|{},_col5: struct|{},_col6: struct|{},_col7: struct|{},_col8: struct|{},_col9: struct|{},_col10: struct|{},_col11: struct|{}) 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,541 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,541 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,541 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,541 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,541 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,541 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,541 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(ctinyint: tinyint|{alltypesparquet}ctinyint,csmallint: smallint|{alltypesparquet}csmallint,cint: int|{alltypesparquet}cint,cbigint: bigint|{alltypesparquet}cbigint,cfloat: float|{alltypesparquet}cfloat,cdouble: double|{alltypesparquet}cdouble,cstring1: string|{alltypesparquet}cstring1,cstring2: string|{alltypesparquet}cstring2,ctimestamp1: timestamp|{alltypesparquet}ctimestamp1,ctimestamp2: timestamp|{alltypesparquet}ctimestamp2,cboolean1: boolean|{alltypesparquet}cboolean1,cboolean2: boolean|{alltypesparquet}cboolean2,BLOCK__OFFSET__INSIDE__FILE: bigint|{alltypesparquet}block__offset__inside__file,INPUT__FILE__NAME: string|{alltypesparquet}input__file__name,ROW__ID: struct|{alltypesparquet}row__id,RAW__DATA__SIZE: bigint|{alltypesparquet}raw__data__size) 2018-01-05T11:28:55,541 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(ctinyint: tinyint|{alltypesparquet}ctinyint,csmallint: smallint|{alltypesparquet}csmallint,cint: int|{alltypesparquet}cint,cbigint: bigint|{alltypesparquet}cbigint,cfloat: float|{alltypesparquet}cfloat,cdouble: double|{alltypesparquet}cdouble,cstring1: string|{alltypesparquet}cstring1,cstring2: string|{alltypesparquet}cstring2,ctimestamp1: timestamp|{alltypesparquet}ctimestamp1,ctimestamp2: timestamp|{alltypesparquet}ctimestamp2,cboolean1: boolean|{alltypesparquet}cboolean1,cboolean2: boolean|{alltypesparquet}cboolean2,BLOCK__OFFSET__INSIDE__FILE: bigint|{alltypesparquet}block__offset__inside__file,INPUT__FILE__NAME: string|{alltypesparquet}input__file__name,ROW__ID: struct|{alltypesparquet}row__id,RAW__DATA__SIZE: bigint|{alltypesparquet}raw__data__size) 2018-01-05T11:28:55,541 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:55,541 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:55,541 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[ctinyint] Column[csmallint] Column[cint] Column[cbigint] Column[cfloat] Column[cdouble] Column[cstring1] Column[cstring2] Column[ctimestamp1] Column[ctimestamp2] Column[cboolean1] Column[cboolean2] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:28:55,541 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col ctinyint) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col csmallint) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col cint) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col cbigint) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col cfloat) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col cdouble) 'hll'),_col6: struct|{}(tok_function compute_stats (tok_table_or_col cstring1) 'hll'),_col7: struct|{}(tok_function compute_stats (tok_table_or_col cstring2) 'hll'),_col8: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp1) 'hll'),_col9: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp2) 'hll'),_col10: struct|{}(tok_function compute_stats (tok_table_or_col cboolean1) 'hll'),_col11: struct|{}(tok_function compute_stats (tok_table_or_col cboolean2) 'hll')) 2018-01-05T11:28:55,541 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:55,541 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:55,541 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col ctinyint) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col csmallint) 'hll'),VALUE._col2: struct|{}(tok_function compute_stats (tok_table_or_col cint) 'hll'),VALUE._col3: struct|{}(tok_function compute_stats (tok_table_or_col cbigint) 'hll'),VALUE._col4: struct|{}(tok_function compute_stats (tok_table_or_col cfloat) 'hll'),VALUE._col5: struct|{}(tok_function compute_stats (tok_table_or_col cdouble) 'hll'),VALUE._col6: struct|{}(tok_function compute_stats (tok_table_or_col cstring1) 'hll'),VALUE._col7: struct|{}(tok_function compute_stats (tok_table_or_col cstring2) 'hll'),VALUE._col8: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp1) 'hll'),VALUE._col9: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp2) 'hll'),VALUE._col10: struct|{}(tok_function compute_stats (tok_table_or_col cboolean1) 'hll'),VALUE._col11: struct|{}(tok_function compute_stats (tok_table_or_col cboolean2) 'hll')) 2018-01-05T11:28:55,541 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:55,541 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:55,541 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col ctinyint) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col csmallint) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col cint) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col cbigint) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col cfloat) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col cdouble) 'hll'),_col6: struct|{}(tok_function compute_stats (tok_table_or_col cstring1) 'hll'),_col7: struct|{}(tok_function compute_stats (tok_table_or_col cstring2) 'hll'),_col8: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp1) 'hll'),_col9: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp2) 'hll'),_col10: struct|{}(tok_function compute_stats (tok_table_or_col cboolean1) 'hll'),_col11: struct|{}(tok_function compute_stats (tok_table_or_col cboolean2) 'hll')) 2018-01-05T11:28:55,541 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:55,541 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:55,541 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1,_col2: struct|{null}_c2,_col3: struct|{null}_c3,_col4: struct|{null}_c4,_col5: struct|{null}_c5,_col6: struct|{null}_c6,_col7: struct|{null}_c7,_col8: struct|{null}_c8,_col9: struct|{null}_c9,_col10: struct|{null}_c10,_col11: struct|{null}_c11) 2018-01-05T11:28:55,541 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:55,541 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:55,541 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0] Column[_col1] Column[_col2] Column[_col3] Column[_col4] Column[_col5] Column[_col6] Column[_col7] Column[_col8] Column[_col9] Column[_col10] Column[_col11]) 2018-01-05T11:28:55,541 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{},_col2: struct|{},_col3: struct|{},_col4: struct|{},_col5: struct|{},_col6: struct|{},_col7: struct|{},_col8: struct|{},_col9: struct|{},_col10: struct|{},_col11: struct|{}) 2018-01-05T11:28:55,541 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:55,541 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:55,541 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,541 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,541 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,541 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,541 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 3 key:[] 2018-01-05T11:28:55,541 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 3 oldColExprMap: {VALUE._col2=Column[_col2], VALUE._col3=Column[_col3], VALUE._col4=Column[_col4], VALUE._col5=Column[_col5], VALUE._col10=Column[_col10], VALUE._col0=Column[_col0], VALUE._col11=Column[_col11], VALUE._col1=Column[_col1], VALUE._col6=Column[_col6], VALUE._col7=Column[_col7], VALUE._col8=Column[_col8], VALUE._col9=Column[_col9]} 2018-01-05T11:28:55,541 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 3 newColExprMap: {VALUE._col2=Column[_col2], VALUE._col3=Column[_col3], VALUE._col4=Column[_col4], VALUE._col5=Column[_col5], VALUE._col10=Column[_col10], VALUE._col0=Column[_col0], VALUE._col11=Column[_col11], VALUE._col1=Column[_col1], VALUE._col6=Column[_col6], VALUE._col7=Column[_col7], VALUE._col8=Column[_col8], VALUE._col9=Column[_col9]} 2018-01-05T11:28:55,541 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,541 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.IdentityProjectRemover: Identity project remover optimization removed : SEL[5] 2018-01-05T11:28:55,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: After logical optimization TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-FS[6] 2018-01-05T11:28:55,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,542 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,542 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=alltypesparquet 2018-01-05T11:28:55,542 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=alltypesparquet 2018-01-05T11:28:55,544 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.207441ms + 0.007153ms, the query is [select "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "TAB_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (...)] 2018-01-05T11:28:55,546 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:55,551 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:55,553 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:55,553 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting stats (Num rows: 12288 Data size: 3093170 Basic stats: COMPLETE Column stats: COMPLETE) on TS[0] 2018-01-05T11:28:55,553 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (alltypesparquet): numRows: 12288 dataSize: 3093170 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {cint= colName: cint colType: int countDistincts: 6029 numNulls: 3115 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -1073279343 max: 1073680599 ] isPrimaryKey: false isEstimated: false, ctimestamp2= colName: ctimestamp2 colType: timestamp countDistincts: 0 numNulls: 0 avgColLen: 40.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, ctimestamp1= colName: ctimestamp1 colType: timestamp countDistincts: 0 numNulls: 0 avgColLen: 40.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, csmallint= colName: csmallint colType: smallint countDistincts: 5594 numNulls: 3114 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -16379 max: 16376 ] isPrimaryKey: false isEstimated: false, cdouble= colName: cdouble colType: double countDistincts: 5569 numNulls: 3114 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: -16379.0 max: 9763215.5639 ] isPrimaryKey: false isEstimated: false, cstring2= colName: cstring2 colType: string countDistincts: 6040 numNulls: 3115 avgColLen: 12.136555989583334 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, cboolean2= colName: cboolean2 colType: boolean countDistincts: 2 numNulls: 3115 avgColLen: 4.0 numTrues: 3983 numFalses: 5190 isPrimaryKey: false isEstimated: false, cboolean1= colName: cboolean1 colType: boolean countDistincts: 2 numNulls: 3114 avgColLen: 4.0 numTrues: 6138 numFalses: 3036 isPrimaryKey: false isEstimated: false, ctinyint= colName: ctinyint colType: tinyint countDistincts: 127 numNulls: 3115 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -64 max: 62 ] isPrimaryKey: false isEstimated: false, cfloat= colName: cfloat colType: float countDistincts: 131 numNulls: 3115 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -64.0 max: 79.5530014038086 ] isPrimaryKey: false isEstimated: false, cstring1= colName: cstring1 colType: string countDistincts: 6058 numNulls: 3114 avgColLen: 10.406982421875 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, cbigint= colName: cbigint colType: bigint countDistincts: 6064 numNulls: 3115 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: -2147311592 max: 2145498388 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:55,553 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting stats (Num rows: 12288 Data size: 3093170 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[1] 2018-01-05T11:28:55,553 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-SEL[1]: numRows: 12288 dataSize: 3093170 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {cint= colName: cint colType: int countDistincts: 6029 numNulls: 3115 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -1073279343 max: 1073680599 ] isPrimaryKey: false isEstimated: false, csmallint= colName: csmallint colType: smallint countDistincts: 5594 numNulls: 3114 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -16379 max: 16376 ] isPrimaryKey: false isEstimated: false, cboolean2= colName: cboolean2 colType: boolean countDistincts: 2 numNulls: 3115 avgColLen: 4.0 numTrues: 3983 numFalses: 5190 isPrimaryKey: false isEstimated: false, cboolean1= colName: cboolean1 colType: boolean countDistincts: 2 numNulls: 3114 avgColLen: 4.0 numTrues: 6138 numFalses: 3036 isPrimaryKey: false isEstimated: false, cfloat= colName: cfloat colType: float countDistincts: 131 numNulls: 3115 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -64.0 max: 79.5530014038086 ] isPrimaryKey: false isEstimated: false, cbigint= colName: cbigint colType: bigint countDistincts: 6064 numNulls: 3115 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: -2147311592 max: 2145498388 ] isPrimaryKey: false isEstimated: false, INPUT__FILE__NAME= colName: INPUT__FILE__NAME colType: string countDistincts: 12288 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, ROW__ID= colName: ROW__ID colType: struct countDistincts: 12288 numNulls: 0 avgColLen: 76.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, RAW__DATA__SIZE= colName: RAW__DATA__SIZE colType: bigint countDistincts: 12288 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, ctimestamp2= colName: ctimestamp2 colType: timestamp countDistincts: 0 numNulls: 0 avgColLen: 40.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, ctimestamp1= colName: ctimestamp1 colType: timestamp countDistincts: 0 numNulls: 0 avgColLen: 40.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, cdouble= colName: cdouble colType: double countDistincts: 5569 numNulls: 3114 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: -16379.0 max: 9763215.5639 ] isPrimaryKey: false isEstimated: false, cstring2= colName: cstring2 colType: string countDistincts: 6040 numNulls: 3115 avgColLen: 12.136555989583334 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, ctinyint= colName: ctinyint colType: tinyint countDistincts: 127 numNulls: 3115 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -64 max: 62 ] isPrimaryKey: false isEstimated: false, cstring1= colName: cstring1 colType: string countDistincts: 6058 numNulls: 3114 avgColLen: 10.406982421875 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, BLOCK__OFFSET__INSIDE__FILE= colName: BLOCK__OFFSET__INSIDE__FILE colType: bigint countDistincts: 12288 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:55,553 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[2]: inputSize: 3093170 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[2] hashAgg: true 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [Case 3] STATS-GBY[2]: cardinality: 1 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 4816 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[2] 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-GBY[2]: numRows: 1 dataSize: 4816 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col9= colName: _col9 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col11= colName: _col11 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 272.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col10= colName: _col10 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 272.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col4= colName: _col4 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col6= colName: _col6 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col5= colName: _col5 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col8= colName: _col8 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col7= colName: _col7 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 4816 Basic stats: COMPLETE Column stats: COMPLETE) on RS[3] 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-RS[3]: numRows: 1 dataSize: 4816 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {VALUE._col2= colName: VALUE._col2 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col3= colName: VALUE._col3 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col4= colName: VALUE._col4 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col5= colName: VALUE._col5 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col10= colName: VALUE._col10 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 272.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col0= colName: VALUE._col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col11= colName: VALUE._col11 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 272.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col1= colName: VALUE._col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col6= colName: VALUE._col6 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col7= colName: VALUE._col7 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col8= colName: VALUE._col8 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col9= colName: VALUE._col9 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[4]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[4] hashAgg: false 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [Case 9] STATS-GBY[4]: cardinality: 1 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 4944 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[4] 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-GBY[4]: numRows: 1 dataSize: 4944 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col9= colName: _col9 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col11= colName: _col11 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 272.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col10= colName: _col10 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 272.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col4= colName: _col4 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col6= colName: _col6 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col5= colName: _col5 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col8= colName: _col8 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col7= colName: _col7 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-FS[6]: numRows: 1 dataSize: 4944 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col9= colName: _col9 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col11= colName: _col11 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 272.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col10= colName: _col10 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 272.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col4= colName: _col4 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col6= colName: _col6 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col5= colName: _col5 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col8= colName: _col8 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col7= colName: _col7 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on TS[0] 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on SEL[1] 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[2] 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[3] 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[4] 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FileSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on FS[6] 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,554 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,554 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: FS, 6 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: GBY, 4 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: RS, 3 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: GBY, 2 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: SEL, 1 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:28:55,555 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Cycle free: true 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(ctinyint: tinyint|{alltypesparquet}ctinyint,csmallint: smallint|{alltypesparquet}csmallint,cint: int|{alltypesparquet}cint,cbigint: bigint|{alltypesparquet}cbigint,cfloat: float|{alltypesparquet}cfloat,cdouble: double|{alltypesparquet}cdouble,cstring1: string|{alltypesparquet}cstring1,cstring2: string|{alltypesparquet}cstring2,ctimestamp1: timestamp|{alltypesparquet}ctimestamp1,ctimestamp2: timestamp|{alltypesparquet}ctimestamp2,cboolean1: boolean|{alltypesparquet}cboolean1,cboolean2: boolean|{alltypesparquet}cboolean2,BLOCK__OFFSET__INSIDE__FILE: bigint|{alltypesparquet}block__offset__inside__file,INPUT__FILE__NAME: string|{alltypesparquet}input__file__name,ROW__ID: struct|{alltypesparquet}row__id,RAW__DATA__SIZE: bigint|{alltypesparquet}raw__data__size) 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(ctinyint: tinyint|{alltypesparquet}ctinyint,csmallint: smallint|{alltypesparquet}csmallint,cint: int|{alltypesparquet}cint,cbigint: bigint|{alltypesparquet}cbigint,cfloat: float|{alltypesparquet}cfloat,cdouble: double|{alltypesparquet}cdouble,cstring1: string|{alltypesparquet}cstring1,cstring2: string|{alltypesparquet}cstring2,ctimestamp1: timestamp|{alltypesparquet}ctimestamp1,ctimestamp2: timestamp|{alltypesparquet}ctimestamp2,cboolean1: boolean|{alltypesparquet}cboolean1,cboolean2: boolean|{alltypesparquet}cboolean2) 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[ctinyint] Column[csmallint] Column[cint] Column[cbigint] Column[cfloat] Column[cdouble] Column[cstring1] Column[cstring2] Column[ctimestamp1] Column[ctimestamp2] Column[cboolean1] Column[cboolean2]) 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col ctinyint) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col csmallint) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col cint) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col cbigint) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col cfloat) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col cdouble) 'hll'),_col6: struct|{}(tok_function compute_stats (tok_table_or_col cstring1) 'hll'),_col7: struct|{}(tok_function compute_stats (tok_table_or_col cstring2) 'hll'),_col8: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp1) 'hll'),_col9: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp2) 'hll'),_col10: struct|{}(tok_function compute_stats (tok_table_or_col cboolean1) 'hll'),_col11: struct|{}(tok_function compute_stats (tok_table_or_col cboolean2) 'hll')) 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col ctinyint) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col csmallint) 'hll'),VALUE._col2: struct|{}(tok_function compute_stats (tok_table_or_col cint) 'hll'),VALUE._col3: struct|{}(tok_function compute_stats (tok_table_or_col cbigint) 'hll'),VALUE._col4: struct|{}(tok_function compute_stats (tok_table_or_col cfloat) 'hll'),VALUE._col5: struct|{}(tok_function compute_stats (tok_table_or_col cdouble) 'hll'),VALUE._col6: struct|{}(tok_function compute_stats (tok_table_or_col cstring1) 'hll'),VALUE._col7: struct|{}(tok_function compute_stats (tok_table_or_col cstring2) 'hll'),VALUE._col8: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp1) 'hll'),VALUE._col9: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp2) 'hll'),VALUE._col10: struct|{}(tok_function compute_stats (tok_table_or_col cboolean1) 'hll'),VALUE._col11: struct|{}(tok_function compute_stats (tok_table_or_col cboolean2) 'hll')) 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col ctinyint) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col csmallint) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col cint) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col cbigint) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col cfloat) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col cdouble) 'hll'),_col6: struct|{}(tok_function compute_stats (tok_table_or_col cstring1) 'hll'),_col7: struct|{}(tok_function compute_stats (tok_table_or_col cstring2) 'hll'),_col8: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp1) 'hll'),_col9: struct|{}(tok_function compute_stats (tok_table_or_col ctimestamp2) 'hll'),_col10: struct|{}(tok_function compute_stats (tok_table_or_col cboolean1) 'hll'),_col11: struct|{}(tok_function compute_stats (tok_table_or_col cboolean2) 'hll')) 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{},_col2: struct|{},_col3: struct|{},_col4: struct|{},_col5: struct|{},_col6: struct|{},_col7: struct|{},_col8: struct|{},_col9: struct|{},_col10: struct|{},_col11: struct|{}) 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: TS[0] 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: RS[3] 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet of table alltypesparquet 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: RS[3] 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: GBY[4] 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: FS[6] 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding reduce work (Reducer 2) for GBY[4] 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Setting up reduce sink: RS[3] with following reduce work: Reducer 2 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Removing RS[3] as parent from GBY[4] 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: FS[6] 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping vectorization 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:28:55,556 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: llap mode: only 2018-01-05T11:28:55,556 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(ctinyint, 'hll')' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'ctinyint' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(csmallint, 'hll')' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'csmallint' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(cint, 'hll')' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cint' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(cbigint, 'hll')' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cbigint' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(cfloat, 'hll')' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cfloat' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(cdouble, 'hll')' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cdouble' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(cstring1, 'hll')' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cstring1' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(cstring2, 'hll')' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cstring2' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(ctimestamp1, 'hll')' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'ctimestamp1' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(ctimestamp2, 'hll')' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'ctimestamp2' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(cboolean1, 'hll')' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cboolean1' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(cboolean2, 'hll')' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cboolean2' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'ctinyint' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'csmallint' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cint' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cbigint' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cfloat' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cdouble' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cstring1' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cstring2' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'ctimestamp1' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'ctimestamp2' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cboolean1' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'cboolean2' 2018-01-05T11:28:55,556 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@d1ef2b2 can be done in LLAP 2018-01-05T11:28:55,556 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Reducer 2 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col0)' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col1)' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col1' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col2)' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col2' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col3)' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col3' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col4)' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col4' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col5)' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col5' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col6)' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col6' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col7)' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col7' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col8)' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col8' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col9)' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col9' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col10)' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col10' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col11)' 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col11' 2018-01-05T11:28:55,556 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:55,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@19c22d23 can be done in LLAP 2018-01-05T11:28:55,557 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,557 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed plan generation 2018-01-05T11:28:55,557 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:55,557 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validation start 2018-01-05T11:28:55,557 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Not a partition. 2018-01-05T11:28:55,557 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:28:55,557 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,557 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0, type:struct, comment:null), FieldSchema(name:_c1, type:struct, comment:null), FieldSchema(name:_c2, type:struct, comment:null), FieldSchema(name:_c3, type:struct, comment:null), FieldSchema(name:_c4, type:struct, comment:null), FieldSchema(name:_c5, type:struct, comment:null), FieldSchema(name:_c6, type:struct, comment:null), FieldSchema(name:_c7, type:struct, comment:null), FieldSchema(name:_c8, type:struct, comment:null), FieldSchema(name:_c9, type:struct, comment:null), FieldSchema(name:_c10, type:struct, comment:null), FieldSchema(name:_c11, type:struct, comment:null)], properties:null) 2018-01-05T11:28:55,557 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,557 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:55,557 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=7, flushCache_()=0, getTableColumnStatistics_(String, String, List, )=11} 2018-01-05T11:28:55,557 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112855_cf8184a1-b651-405c-b849-0ee626114470); Time taken: 0.067 seconds 2018-01-05T11:28:55,557 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:55,557 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,557 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112855_cf8184a1-b651-405c-b849-0ee626114470): ANALYZE TABLE alltypesparquet COMPUTE STATISTICS FOR COLUMNS ctinyint,csmallint,cint,cbigint,cfloat,cdouble,cstring1,cstring2,ctimestamp1,ctimestamp2,cboolean1,cboolean2 2018-01-05T11:28:55,558 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,558 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: ANALYZE TABLE alltypesparquet COMPUTE STATISTICS FOR COLUMNS ctinyint,csmallint,cint,cbigint,cfloat,cdouble,cstring1,cstring2,ctimestamp1,ctimestamp2,cboolean1,cboolean2 2018-01-05T11:28:55,558 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:28:55,558 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@alltypesparquet 2018-01-05T11:28:55,558 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@alltypesparquet 2018-01-05T11:28:55,558 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-55_495_9015283846029904157-1/-mr-10000 2018-01-05T11:28:55,558 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,558 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,558 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,558 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Query ID = djaiswal_20180105112855_cf8184a1-b651-405c-b849-0ee626114470 2018-01-05T11:28:55,558 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Total jobs = 1 2018-01-05T11:28:55,558 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,558 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:28:55,558 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MAPRED] in serial mode 2018-01-05T11:28:55,580 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-55_490_8843674450829636198-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:28:55,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:28:55,600 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:28:55,600 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:28:55,600 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112855_cf8184a1-b651-405c-b849-0ee626114470 2018-01-05T11:28:55,600 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Session is already open 2018-01-05T11:28:55,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:28:55,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,600 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Dag name: ANALYZE TABLE alltype...,cboolean1,cboolean2 (Stage-0) 2018-01-05T11:28:55,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: DagInfo: {"context":"Hive","description":"\n\nANALYZE TABLE alltypesparquet COMPUTE STATISTICS FOR COLUMNS ctinyint,csmallint,cint,cbigint,cfloat,cdouble,cstring1,cstring2,ctimestamp1,ctimestamp2,cboolean1,cboolean2"} 2018-01-05T11:28:55,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112855_cf8184a1-b651-405c-b849-0ee626114470 with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:28:55,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,612 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,613 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:28:55,614 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,614 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 2 size: 2.92KB 2018-01-05T11:28:55,631 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,631 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,645 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:28:55,645 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,645 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:28:55,646 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,646 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 5.92KB 2018-01-05T11:28:55,652 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-55_495_9015283846029904157-1/-mr-10001 2018-01-05T11:28:55,664 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-55_495_9015283846029904157-1/-mr-10001 2018-01-05T11:28:55,664 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet 2018-01-05T11:28:55,664 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,666 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=ANALYZE TABLE alltype...,cboolean1,cboolean2 (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112855_cf8184a1-b651-405c-b849-0ee626114470 } 2018-01-05T11:28:55,666 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:55,666 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:55,668 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: ANALYZE TABLE alltype...,cboolean1,cboolean2 (Stage-0) 2018-01-05T11:28:55,668 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:28:55,668 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_20, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_20.dot 2018-01-05T11:28:55,669 INFO [IPC Server handler 0 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_20 2018-01-05T11:28:55,669 WARN [IPC Server handler 0 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:55,669 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Running DAG: ANALYZE TABLE alltype...,cboolean1,cboolean2 (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112855_cf8184a1-b651-405c-b849-0ee626114470 } 2018-01-05T11:28:55,691 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_20][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_20, submitTime=1515180535668, queueName=null 2018-01-05T11:28:55,692 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_20_00 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_20_00 [Map 1], Default Resources= 2018-01-05T11:28:55,692 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_20_00 [Map 1] 2018-01-05T11:28:55,692 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:28:55,692 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_20_00 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:55,692 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_20_01 [Reducer 2]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_20_01 [Reducer 2], Default Resources= 2018-01-05T11:28:55,692 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_20_01 [Reducer 2] 2018-01-05T11:28:55,692 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 2 2018-01-05T11:28:55,692 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_20_01 [Reducer 2] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:55,692 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:28:55,692 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_20][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_20, initTime=1515180535692 2018-01-05T11:28:55,692 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: dag_1515180518813_0001_20 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:28:55,693 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:28:55,693 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_20][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_20, startTime=1515180535693 2018-01-05T11:28:55,693 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_20 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:28:55,693 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {alltypesparquet={InputName=alltypesparquet}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:28:55,693 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: alltypesparquet, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:28:55,693 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_20_00 [Map 1] 2018-01-05T11:28:55,693 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_20, dagName=ANALYZE TABLE alltype...,cboolean1,cboolean2 (Stage-0) 2018-01-05T11:28:55,693 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,693 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,693 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,697 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_20_00 [Map 1] 2018-01-05T11:28:55,697 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_20_00 [Map 1] 2018-01-05T11:28:55,697 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:55,697 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_20_00 [Map 1] 2018-01-05T11:28:55,699 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:28:55,699 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@54dec95d 2018-01-05T11:28:55,699 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-55_490_8843674450829636198-1/djaiswal/_tez_scratch_dir/31467b89-717c-4df1-b91f-0298b497e70e/map.xml 2018-01-05T11:28:55,699 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-55_490_8843674450829636198-1/djaiswal/_tez_scratch_dir/31467b89-717c-4df1-b91f-0298b497e70e/map.xml 2018-01-05T11:28:55,699 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-55_490_8843674450829636198-1/djaiswal/_tez_scratch_dir/31467b89-717c-4df1-b91f-0298b497e70e/map.xml 2018-01-05T11:28:55,699 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:55,699 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:55,699 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:55,700 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:55,700 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 5.92KB 2018-01-05T11:28:55,700 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_20_00 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:28:55,700 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: alltypesparquet on vertex vertex_1515180518813_0001_20_00 [Map 1] 2018-01-05T11:28:55,701 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to ShuffleVertexManager for vertex_1515180518813_0001_20_01 [Reducer 2] 2018-01-05T11:28:55,701 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:28:55,701 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:55,701 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:28:55,701 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:55,701 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-55_490_8843674450829636198-1/djaiswal/_tez_scratch_dir/31467b89-717c-4df1-b91f-0298b497e70e/map.xml 2018-01-05T11:28:55,701 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:55,701 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias alltypesparquet 2018-01-05T11:28:55,701 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet 2018-01-05T11:28:55,701 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet 2018-01-05T11:28:55,701 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 0,1,2,3,4,5,6,7,8,9,10,11 2018-01-05T11:28:55,701 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = ctinyint,csmallint,cint,cbigint,cfloat,cdouble,cstring1,cstring2,ctimestamp1,ctimestamp2,cboolean1,cboolean2 2018-01-05T11:28:55,701 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet 2018-01-05T11:28:55,702 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:55,702 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:55,702 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:55,702 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:55,706 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.05 maxFrac: 0.75 auto: false desiredTaskIput: 104857600 2018-01-05T11:28:55,706 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:28:55,706 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 1 tasks for vertex: vertex_1515180518813_0001_20_01 [Reducer 2] 2018-01-05T11:28:55,706 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_20_01 [Reducer 2] 2018-01-05T11:28:55,706 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_20][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_20_01, reconfigureDoneTime=1515180535706, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:28:55,706 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_20_01 [Reducer 2], numAdditionalOutputs=1 2018-01-05T11:28:55,706 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_20][Event:VERTEX_INITIALIZED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_20_01, initRequestedTime=1515180535700, initedTime=1515180535706, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:55,706 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_20_01 [Reducer 2] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:28:55,715 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Time taken to get FileStatuses: 13 2018-01-05T11:28:55,715 INFO [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:28:55,715 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 13 2018-01-05T11:28:55,715 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 1 2018-01-05T11:28:55,715 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:55,715 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 1. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:28:55,715 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-55_490_8843674450829636198-1/djaiswal/_tez_scratch_dir/31467b89-717c-4df1-b91f-0298b497e70e/map.xml 2018-01-05T11:28:55,715 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-55_490_8843674450829636198-1/djaiswal/_tez_scratch_dir/31467b89-717c-4df1-b91f-0298b497e70e/map.xml 2018-01-05T11:28:55,715 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-55_490_8843674450829636198-1/djaiswal/_tez_scratch_dir/31467b89-717c-4df1-b91f-0298b497e70e/map.xml 2018-01-05T11:28:55,715 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:55,715 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:55,715 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:55,716 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:55,716 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 5.92KB 2018-01-05T11:28:55,716 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet/000000_0 to src new group? true 2018-01-05T11:28:55,716 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:28:55,716 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:28:55,716 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:28:55,716 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 11018 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 594976 Original splits: 1 2018-01-05T11:28:55,716 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Using original number of splits: 1 desired splits: 1 2018-01-05T11:28:55,716 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 1 grouped split count is 1, for bucket: 1 2018-01-05T11:28:55,716 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:28:55,716 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: alltypesparquet on vertex vertex_1515180518813_0001_20_00 [Map 1] 2018-01-05T11:28:55,717 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_20_00 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:55,717 INFO [App Shared Pool - #0] impl.VertexImpl: Got updated RootInputsSpecs: {alltypesparquet=forAllWorkUnits=true, update=[1]} 2018-01-05T11:28:55,718 INFO [App Shared Pool - #0] impl.VertexImpl: Vertex vertex_1515180518813_0001_20_00 [Map 1] parallelism set to 1 2018-01-05T11:28:55,718 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_20_00 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:55,718 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_20][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_20_00, initRequestedTime=1515180535693, initedTime=1515180535718, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:55,718 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_20_00 [Map 1] with distanceFromRoot: 0 2018-01-05T11:28:55,718 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_20_00 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:28:55,718 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_20][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_20_00, startRequestedTime=1515180535700, startedTime=1515180535718 2018-01-05T11:28:55,718 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_20][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_20_00, reconfigureDoneTime=1515180535718, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:28:55,718 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_20_00 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:55,718 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_20_00 for vertex: vertex_1515180518813_0001_20_01 [Reducer 2] numStartedSources: 1 numSources: 1 2018-01-05T11:28:55,718 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_20_01 [Reducer 2] with distanceFromRoot: 1 2018-01-05T11:28:55,718 INFO [App Shared Pool - #0] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:28:55,718 INFO [App Shared Pool - #0] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:28:55,718 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_20][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_20_01, startRequestedTime=1515180535718, startedTime=1515180535718 2018-01-05T11:28:55,719 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_20_00 [Map 1] 2018-01-05T11:28:55,719 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_20_01 [Reducer 2] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:55,719 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_20][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_20_00_000000, scheduledTime=1515180535719, launchTime=1515180535719 2018-01-05T11:28:55,719 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: OnVertexStarted vertex: Reducer 2 with 0 source tasks and 1 pending tasks 2018-01-05T11:28:55,719 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_20_00 [Map 1] 2018-01-05T11:28:55,719 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Map 1 in vertex: Reducer 2 numBipartiteSourceTasks: 1 2018-01-05T11:28:55,719 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_20_00_000000_0, priority=5, capability=, hosts=null 2018-01-05T11:28:55,719 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=5,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:55,720 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=5,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:55,720 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:55,720 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_20_00_000000_0 on hosts=null 2018-01-05T11:28:55,720 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_20_00_000000_0 on hosts=null 2018-01-05T11:28:55,720 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_20_00_000000_0 2018-01-05T11:28:55,720 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:55,720 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #40, task=TaskInfo{task=attempt_1515180518813_0001_20_00_000000_0, priority=5, startTime=0, containerId=null, uniqueId=39, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000040 2018-01-05T11:28:55,720 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_20_00_000000_0; false 2018-01-05T11:28:55,720 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_20_00_000000_0, priority=5, startTime=311556229, containerId=container_111101111_0001_00_000040assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=39, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:55,720 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000040 for node: 10.22.8.182:49818 2018-01-05T11:28:55,722 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 20, name=ANALYZE TABLE alltype...,cboolean1,cboolean2 (Stage-0), queryId=djaiswal_20180105112855_cf8184a1-b651-405c-b849-0ee626114470 2018-01-05T11:28:55,722 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000040, attempt_1515180518813_0001_20_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:55,722 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000040 for node: 10.22.8.182:49818 2018-01-05T11:28:55,722 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000040 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:55,722 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000040, launchTime=1515180535722 2018-01-05T11:28:55,723 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_20_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112855_cf8184a1-b651-405c-b849-0ee626114470, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000040, dagName=ANALYZE TABLE alltype...,cboolean1,cboolean2 (Stage-0), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{alltypesparquet,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{Reducer 2,org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput,1}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180535693, firstAttemptStartTime=1515180535719, currentAttemptStartTime=1515180535722} 2018-01-05T11:28:55,723 INFO [IPC Server handler 1 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000040, QueryId=djaiswal_20180105112855_cf8184a1-b651-405c-b849-0ee626114470-20, DagId=20, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180535723 2018-01-05T11:28:55,723 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_20_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000040] on NM: [10.22.8.182:49818] 2018-01-05T11:28:55,723 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:55,723 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:55,723 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_20][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_20_00_000000_0, startTime=1515180535722, containerId=container_111101111_0001_00_000040, nodeId=10.22.8.182:49818 2018-01-05T11:28:55,723 DEBUG [IPC Server handler 1 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=20} with the ShuffleHandler 2018-01-05T11:28:55,733 INFO [IPC Server handler 1 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:28:55,733 DEBUG [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/20] 2018-01-05T11:28:55,733 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:55,733 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_20_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:55,733 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_20_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180535719, dagStartTime=1515180535693, withinDagPriority=5, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:28:55,733 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_20_00_000000_0 : ACCEPTED 2018-01-05T11:28:55,734 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_20_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:55,734 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_20_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180535719, dagStartTime=1515180535693, withinDagPriority=5, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:28:55,734 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_20_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:55,734 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_20_00_000000_0 to pre-emption queue 2018-01-05T11:28:55,734 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_20_00_000000_0: true 2018-01-05T11:28:55,734 INFO [TaskCommunicator # 1] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_20_00_000000_0 2018-01-05T11:28:55,734 INFO [TaskCommunicator # 1] GuaranteedTasks: Received response for attempt_1515180518813_0001_20_00_000000_0, true 2018-01-05T11:28:55,734 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:1931954929 2018-01-05T11:28:55,735 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : ANALYZE TABLE alltype...,cboolean1,cboolean2 (Stage-0), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_20_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=alltypesparquet, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=Reducer 2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput }}, ] 2018-01-05T11:28:55,735 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1039138816, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:55,735 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_20_00_000000_0 vertexName: Map 1 2018-01-05T11:28:55,736 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000040, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_20_00_000000_0, eventCount=1 } 2018-01-05T11:28:55,736 DEBUG [TezTR-518813_1_20_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_20 queryId: djaiswal_20180105112855_cf8184a1-b651-405c-b849-0ee626114470 instanceType: TaskRunner2Callable 2018-01-05T11:28:55,736 INFO [TezTR-518813_1_20_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_20_00_000000_0 2018-01-05T11:28:55,736 DEBUG [TezTR-518813_1_20_0_0_0] log.PerfLogger: 2018-01-05T11:28:55,736 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:55,737 INFO [I/O Setup 1 Initialize: {Reducer 2}] writers.UnorderedPartitionedKVWriter: Requested BufferSize (tez.runtime.unordered.output.buffer.size-mb) : 100 2018-01-05T11:28:55,737 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:55,738 INFO [IPC Server handler 25 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_20_00_000000_0 sent events: (0-1). 2018-01-05T11:28:55,738 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:28:55,738 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_20_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:28:55,739 DEBUG [TezTR-518813_1_20_0_0_0] log.PerfLogger: 2018-01-05T11:28:55,739 INFO [TezTR-518813_1_20_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:55,739 INFO [TezTR-518813_1_20_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:55,739 INFO [TezTR-518813_1_20_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:55,739 INFO [I/O Setup 0 Initialize: {alltypesparquet}] input.MRInput: alltypesparquet using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:28:55,739 INFO [I/O Setup 0 Initialize: {alltypesparquet}] input.MRInputLegacy: alltypesparquet MRInputLegacy deferring initialization 2018-01-05T11:28:55,739 INFO [TezTR-518813_1_20_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:55,739 INFO [TezTR-518813_1_20_0_0_0] resources.MemoryDistributor: InitialRequests=[alltypesparquet:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [Reducer 2:OUTPUT:104857600:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput] 2018-01-05T11:28:55,739 INFO [TezTR-518813_1_20_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:55,739 INFO [TezTR-518813_1_20_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:55,739 INFO [TezTR-518813_1_20_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 104857600, TotalRequestedScaled: 5.24288E7, TotalJVMHeap: 1039138816, TotalAvailable: 696223006, TotalRequested/TotalJVMHeap:0.10 2018-01-05T11:28:55,739 INFO [TezTR-518813_1_20_0_0_0] resources.MemoryDistributor: Allocations=[alltypesparquet:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput:OUTPUT:104857600:104857600] 2018-01-05T11:28:55,739 INFO [TezTR-518813_1_20_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:55,739 INFO [TezTR-518813_1_20_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: alltypesparquet being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:28:55,739 INFO [TezTR-518813_1_20_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:28:55,739 INFO [TezTR-518813_1_20_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:28:55,739 INFO [I/O Setup 1 Start: {alltypesparquet}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: alltypesparquet 2018-01-05T11:28:55,739 INFO [TezTR-518813_1_20_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:55,739 INFO [TezTR-518813_1_20_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_20_00_000000_0 2018-01-05T11:28:55,739 DEBUG [TezTR-518813_1_20_0_0_0] log.PerfLogger: 2018-01-05T11:28:55,739 DEBUG [TezTR-518813_1_20_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_20_00_000000_0_10118 2018-01-05T11:28:55,739 DEBUG [TezTR-518813_1_20_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:55,739 DEBUG [TezTR-518813_1_20_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_20_00_000000_0 2018-01-05T11:28:55,739 INFO [TezTR-518813_1_20_0_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112855_cf8184a1-b651-405c-b849-0ee626114470 2018-01-05T11:28:55,739 DEBUG [TezTR-518813_1_20_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:55,739 DEBUG [TezTR-518813_1_20_0_0_0] log.PerfLogger: 2018-01-05T11:28:55,739 DEBUG [TezTR-518813_1_20_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:55,739 DEBUG [TezTR-518813_1_20_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:55,739 DEBUG [TezTR-518813_1_20_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112855_cf8184a1-b651-405c-b849-0ee626114470_Map 1__MAP_PLAN__ 2018-01-05T11:28:55,739 INFO [TezTR-518813_1_20_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-55_490_8843674450829636198-1/djaiswal/_tez_scratch_dir/31467b89-717c-4df1-b91f-0298b497e70e/map.xml 2018-01-05T11:28:55,740 DEBUG [TezTR-518813_1_20_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-55_490_8843674450829636198-1/djaiswal/_tez_scratch_dir/31467b89-717c-4df1-b91f-0298b497e70e/map.xml 2018-01-05T11:28:55,740 DEBUG [TezTR-518813_1_20_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-55_490_8843674450829636198-1/djaiswal/_tez_scratch_dir/31467b89-717c-4df1-b91f-0298b497e70e/map.xml 2018-01-05T11:28:55,740 DEBUG [TezTR-518813_1_20_0_0_0] log.PerfLogger: 2018-01-05T11:28:55,740 INFO [TezTR-518813_1_20_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:55,740 DEBUG [TezTR-518813_1_20_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:55,741 DEBUG [TezTR-518813_1_20_0_0_0] log.PerfLogger: 2018-01-05T11:28:55,741 INFO [TezTR-518813_1_20_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 5.92KB 2018-01-05T11:28:55,742 DEBUG [TezTR-518813_1_20_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:55,742 INFO [TezTR-518813_1_20_0_0_0] tez.MapRecordProcessor: The input names are: [alltypesparquet] 2018-01-05T11:28:55,742 INFO [TezTR-518813_1_20_0_0_0] input.MRInput: Initialized MRInput: alltypesparquet 2018-01-05T11:28:55,742 DEBUG [TezTR-518813_1_20_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 1 2018-01-05T11:28:55,742 INFO [TezTR-518813_1_20_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-55_490_8843674450829636198-1/djaiswal/_tez_scratch_dir/31467b89-717c-4df1-b91f-0298b497e70e/map.xml 2018-01-05T11:28:55,742 DEBUG [TezTR-518813_1_20_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:55,742 DEBUG [TezTR-518813_1_20_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet/000000_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat, properties={columns.types=tinyint:smallint:int:bigint:float:double:string:string:timestamp:timestamp:boolean:boolean, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet, columns=ctinyint,csmallint,cint,cbigint,cfloat,cdouble,cstring1,cstring2,ctimestamp1,ctimestamp2,cboolean1,cboolean2, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"cbigint":"true","cboolean1":"true","cboolean2":"true","cdouble":"true","cfloat":"true","cint":"true","csmallint":"true","cstring1":"true","cstring2":"true","ctimestamp1":"true","ctimestamp2":"true","ctinyint":"true"}}, serialization.format=1, numRows=12288, numFiles=1, serialization.ddl=struct alltypesparquet { byte ctinyint, i16 csmallint, i32 cint, i64 cbigint, float cfloat, double cdouble, string cstring1, string cstring2, timestamp ctimestamp1, timestamp ctimestamp2, bool cboolean1, bool cboolean2}, transient_lastDdlTime=1515180535, rawDataSize=593707, columns.comments=, totalSize=594976, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat, serialization.lib=org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe, file.inputformat=org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat, column.name.delimiter=,, name=default.alltypesparquet}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat, properties={columns.types=tinyint:smallint:int:bigint:float:double:string:string:timestamp:timestamp:boolean:boolean, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet, columns=ctinyint,csmallint,cint,cbigint,cfloat,cdouble,cstring1,cstring2,ctimestamp1,ctimestamp2,cboolean1,cboolean2, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"cbigint":"true","cboolean1":"true","cboolean2":"true","cdouble":"true","cfloat":"true","cint":"true","csmallint":"true","cstring1":"true","cstring2":"true","ctimestamp1":"true","ctimestamp2":"true","ctinyint":"true"}}, serialization.format=1, numRows=12288, numFiles=1, serialization.ddl=struct alltypesparquet { byte ctinyint, i16 csmallint, i32 cint, i64 cbigint, float cfloat, double cdouble, string cstring1, string cstring2, timestamp ctimestamp1, timestamp ctimestamp2, bool cboolean1, bool cboolean2}, transient_lastDdlTime=1515180535, rawDataSize=593707, columns.comments=, totalSize=594976, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat, serialization.lib=org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe, file.inputformat=org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat, column.name.delimiter=,, name=default.alltypesparquet}, baseFileName=alltypesparquet, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat, properties={columns.types=tinyint:smallint:int:bigint:float:double:string:string:timestamp:timestamp:boolean:boolean, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet, columns=ctinyint,csmallint,cint,cbigint,cfloat,cdouble,cstring1,cstring2,ctimestamp1,ctimestamp2,cboolean1,cboolean2, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"cbigint":"true","cboolean1":"true","cboolean2":"true","cdouble":"true","cfloat":"true","cint":"true","csmallint":"true","cstring1":"true","cstring2":"true","ctimestamp1":"true","ctimestamp2":"true","ctinyint":"true"}}, serialization.format=1, numRows=12288, numFiles=1, serialization.ddl=struct alltypesparquet { byte ctinyint, i16 csmallint, i32 cint, i64 cbigint, float cfloat, double cdouble, string cstring1, string cstring2, timestamp ctimestamp1, timestamp ctimestamp2, bool cboolean1, bool cboolean2}, transient_lastDdlTime=1515180535, rawDataSize=593707, columns.comments=, totalSize=594976, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat, serialization.lib=org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe, file.inputformat=org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat, column.name.delimiter=,, name=default.alltypesparquet}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat, properties={columns.types=tinyint:smallint:int:bigint:float:double:string:string:timestamp:timestamp:boolean:boolean, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet, columns=ctinyint,csmallint,cint,cbigint,cfloat,cdouble,cstring1,cstring2,ctimestamp1,ctimestamp2,cboolean1,cboolean2, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"cbigint":"true","cboolean1":"true","cboolean2":"true","cdouble":"true","cfloat":"true","cint":"true","csmallint":"true","cstring1":"true","cstring2":"true","ctimestamp1":"true","ctimestamp2":"true","ctinyint":"true"}}, serialization.format=1, numRows=12288, numFiles=1, serialization.ddl=struct alltypesparquet { byte ctinyint, i16 csmallint, i32 cint, i64 cbigint, float cfloat, double cdouble, string cstring1, string cstring2, timestamp ctimestamp1, timestamp ctimestamp2, bool cboolean1, bool cboolean2}, transient_lastDdlTime=1515180535, rawDataSize=593707, columns.comments=, totalSize=594976, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat, serialization.lib=org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe, file.inputformat=org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat, column.name.delimiter=,, name=default.alltypesparquet}, baseFileName=alltypesparquet, vectorPartitionDesc=null]} 2018-01-05T11:28:55,742 DEBUG [TezTR-518813_1_20_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:55,742 DEBUG [TezTR-518813_1_20_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat 2018-01-05T11:28:55,742 INFO [TezTR-518813_1_20_0_0_0] io.HiveInputFormat: Injecting LLAP caches into org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat 2018-01-05T11:28:55,743 DEBUG [TezTR-518813_1_20_0_0_0] parquet.MapredParquetInputFormat: Using row-mode record reader 2018-01-05T11:28:55,743 INFO [TezTR-518813_1_20_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-55_490_8843674450829636198-1/djaiswal/_tez_scratch_dir/31467b89-717c-4df1-b91f-0298b497e70e/map.xml 2018-01-05T11:28:55,743 DEBUG [TezTR-518813_1_20_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:55,744 DEBUG [TezTR-518813_1_20_0_0_0] parquet.ProjectionPusher: Not pushing filters because FilterExpr is null 2018-01-05T11:28:55,744 DEBUG [TezTR-518813_1_20_0_0_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:55,744 DEBUG [TezTR-518813_1_20_0_0_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:55,744 DEBUG [TezTR-518813_1_20_0_0_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:55,744 DEBUG [TezTR-518813_1_20_0_0_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:55,757 ERROR [TezTR-518813_1_20_0_0_0] hadoop.ParquetRecordReader: Can not initialize counter due to context is not a instance of TaskInputOutputContext, but is org.apache.hadoop.mapreduce.task.TaskAttemptContextImpl 2018-01-05T11:28:55,771 INFO [TezTR-518813_1_20_0_0_0] hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 12288 records. 2018-01-05T11:28:55,771 INFO [TezTR-518813_1_20_0_0_0] hadoop.InternalParquetRecordReader: at row 0. reading next block 2018-01-05T11:28:55,778 INFO [TezTR-518813_1_20_0_0_0] hadoop.InternalParquetRecordReader: block read in memory in 7 ms. row count = 12288 2018-01-05T11:28:55,830 DEBUG [TezTR-518813_1_20_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet/000000_0 2018-01-05T11:28:55,830 INFO [TezTR-518813_1_20_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet/000000_0:0+594976], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=594976} 2018-01-05T11:28:55,830 INFO [TezTR-518813_1_20_0_0_0] input.MRInput: alltypesparquet initialized RecordReader from event 2018-01-05T11:28:55,830 DEBUG [TezTR-518813_1_20_0_0_0] tez.MapRecordProcessor: Starting Output: Reducer 2 2018-01-05T11:28:55,831 INFO [TezTR-518813_1_20_0_0_0] writers.BaseUnorderedPartitionedKVWriter: Instantiating Partitioner: [org.apache.tez.mapreduce.partition.MRPartitioner] 2018-01-05T11:28:55,838 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000040, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_20_00_000000_0, eventCount=1 } 2018-01-05T11:28:55,839 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:55,851 INFO [TezTR-518813_1_20_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: numBuffers=2, sizePerBuffer=52428800, skipBuffers=true, numPartitions=1, availableMemory=104857600, maxSingleBufferSizeBytes=2147483647, pipelinedShuffle=false, isFinalMergeEnabled=true, numPartitions=1, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:28:55,852 INFO [TezTR-518813_1_20_0_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:28:55,852 INFO [TezTR-518813_1_20_0_0_0] exec.MapOperator: Initializing operator MAP[0] 2018-01-05T11:28:55,852 DEBUG [TezTR-518813_1_20_0_0_0] exec.MapOperator: Initialization Done 0 MAP 2018-01-05T11:28:55,852 DEBUG [TezTR-518813_1_20_0_0_0] exec.MapOperator: Operator 0 MAP initialized 2018-01-05T11:28:55,852 DEBUG [TezTR-518813_1_20_0_0_0] exec.MapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:28:55,854 DEBUG [TezTR-518813_1_20_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:55,854 DEBUG [TezTR-518813_1_20_0_0_0] exec.MapOperator: Adding alias alltypesparquet to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet 2018-01-05T11:28:55,854 DEBUG [TezTR-518813_1_20_0_0_0] exec.MapOperator: dump TS[0] struct 2018-01-05T11:28:55,854 INFO [TezTR-518813_1_20_0_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:28:55,854 INFO [TezTR-518813_1_20_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:55,854 DEBUG [TezTR-518813_1_20_0_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:28:55,854 INFO [TezTR-518813_1_20_0_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:28:55,854 DEBUG [TezTR-518813_1_20_0_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:28:55,854 DEBUG [TezTR-518813_1_20_0_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:28:55,854 DEBUG [TezTR-518813_1_20_0_0_0] exec.TableScanOperator: Initializing children of 0 TS 2018-01-05T11:28:55,854 DEBUG [TezTR-518813_1_20_0_0_0] exec.SelectOperator: Initializing child 1 SEL 2018-01-05T11:28:55,854 INFO [TezTR-518813_1_20_0_0_0] exec.SelectOperator: Initializing operator SEL[1] 2018-01-05T11:28:55,854 INFO [TezTR-518813_1_20_0_0_0] exec.SelectOperator: SELECT struct 2018-01-05T11:28:55,855 DEBUG [TezTR-518813_1_20_0_0_0] exec.SelectOperator: Initialization Done 1 SEL 2018-01-05T11:28:55,855 DEBUG [TezTR-518813_1_20_0_0_0] exec.SelectOperator: Operator 1 SEL initialized 2018-01-05T11:28:55,855 DEBUG [TezTR-518813_1_20_0_0_0] exec.SelectOperator: Initializing children of 1 SEL 2018-01-05T11:28:55,855 DEBUG [TezTR-518813_1_20_0_0_0] exec.GroupByOperator: Initializing child 2 GBY 2018-01-05T11:28:55,855 INFO [TezTR-518813_1_20_0_0_0] exec.GroupByOperator: Initializing operator GBY[2] 2018-01-05T11:28:55,855 DEBUG [TezTR-518813_1_20_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:55,855 DEBUG [TezTR-518813_1_20_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:55,855 DEBUG [TezTR-518813_1_20_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:55,855 DEBUG [TezTR-518813_1_20_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:55,855 DEBUG [TezTR-518813_1_20_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:55,855 DEBUG [TezTR-518813_1_20_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:55,855 DEBUG [TezTR-518813_1_20_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:55,855 DEBUG [TezTR-518813_1_20_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:55,855 DEBUG [TezTR-518813_1_20_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:55,855 DEBUG [TezTR-518813_1_20_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:55,855 DEBUG [TezTR-518813_1_20_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:55,855 DEBUG [TezTR-518813_1_20_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:55,855 INFO [TezTR-518813_1_20_0_0_0] exec.GroupByOperator: Max hash table memory: 93860656 bytes 2018-01-05T11:28:55,855 INFO [TezTR-518813_1_20_0_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:55,855 DEBUG [TezTR-518813_1_20_0_0_0] exec.GroupByOperator: Initialization Done 2 GBY 2018-01-05T11:28:55,855 DEBUG [TezTR-518813_1_20_0_0_0] exec.GroupByOperator: Operator 2 GBY initialized 2018-01-05T11:28:55,855 DEBUG [TezTR-518813_1_20_0_0_0] exec.GroupByOperator: Initializing children of 2 GBY 2018-01-05T11:28:55,855 DEBUG [TezTR-518813_1_20_0_0_0] exec.ReduceSinkOperator: Initializing child 3 RS 2018-01-05T11:28:55,855 INFO [TezTR-518813_1_20_0_0_0] exec.ReduceSinkOperator: Initializing operator RS[3] 2018-01-05T11:28:55,855 DEBUG [TezTR-518813_1_20_0_0_0] exec.ReduceSinkOperator: keys size is 0 2018-01-05T11:28:55,855 INFO [TezTR-518813_1_20_0_0_0] exec.ReduceSinkOperator: Using tag = -1 2018-01-05T11:28:55,855 DEBUG [TezTR-518813_1_20_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:55,855 DEBUG [TezTR-518813_1_20_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:55,856 DEBUG [TezTR-518813_1_20_0_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11] columnTypes=[struct, struct, struct, struct, struct, struct, struct, struct, struct, struct, struct, struct] 2018-01-05T11:28:55,856 DEBUG [TezTR-518813_1_20_0_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS 2018-01-05T11:28:55,856 DEBUG [TezTR-518813_1_20_0_0_0] exec.ReduceSinkOperator: Operator 3 RS initialized 2018-01-05T11:28:55,856 DEBUG [TezTR-518813_1_20_0_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS done is reset. 2018-01-05T11:28:55,856 DEBUG [TezTR-518813_1_20_0_0_0] exec.GroupByOperator: Initialization Done 2 GBY done is reset. 2018-01-05T11:28:55,856 DEBUG [TezTR-518813_1_20_0_0_0] exec.SelectOperator: Initialization Done 1 SEL done is reset. 2018-01-05T11:28:55,856 DEBUG [TezTR-518813_1_20_0_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:28:55,856 INFO [TezTR-518813_1_20_0_0_0] exec.OperatorUtils: Setting output collector: RS[3] --> Reducer 2 2018-01-05T11:28:55,856 DEBUG [TezTR-518813_1_20_0_0_0] log.PerfLogger: 2018-01-05T11:28:55,856 DEBUG [TezTR-518813_1_20_0_0_0] exec.MapOperator: Processing alias(es) alltypesparquet for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet/000000_0 2018-01-05T11:28:55,856 INFO [TezTR-518813_1_20_0_0_0] exec.MapOperator: MAP[0]: records read - 1 2018-01-05T11:28:55,857 INFO [TezTR-518813_1_20_0_0_0] exec.MapOperator: MAP[0]: records read - 10 2018-01-05T11:28:55,861 INFO [TezTR-518813_1_20_0_0_0] exec.MapOperator: MAP[0]: records read - 100 2018-01-05T11:28:55,884 INFO [TezTR-518813_1_20_0_0_0] exec.MapOperator: MAP[0]: records read - 1000 2018-01-05T11:28:55,884 DEBUG [TezTR-518813_1_20_0_0_0] tez.InterruptibleProcessing: Adjusting abort check rows to 100000 from 1000 2018-01-05T11:28:55,908 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,908 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: Status: Running (Executing on YARN cluster with App id application_1515180518813_0001) 2018-01-05T11:28:55,909 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:55,909 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 0(+1)/1 Reducer 2: 0/1 2018-01-05T11:28:55,944 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000040, requestId=3, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_20_00_000000_0, eventCount=1 } 2018-01-05T11:28:55,945 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=3, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:55,955 INFO [TezTR-518813_1_20_0_0_0] exec.MapOperator: MAP[0]: records read - 10000 2018-01-05T11:28:55,967 DEBUG [TezTR-518813_1_20_0_0_0] log.PerfLogger: 2018-01-05T11:28:55,968 DEBUG [TezTR-518813_1_20_0_0_0] mr.ObjectCache: djaiswal_20180105112855_cf8184a1-b651-405c-b849-0ee626114470_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:28:55,968 DEBUG [TezTR-518813_1_20_0_0_0] exec.MapOperator: close called for operator MAP[0] 2018-01-05T11:28:55,968 INFO [TezTR-518813_1_20_0_0_0] exec.MapOperator: Closing operator MAP[0] 2018-01-05T11:28:55,968 INFO [TezTR-518813_1_20_0_0_0] exec.MapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:12288, 2018-01-05T11:28:55,968 DEBUG [TezTR-518813_1_20_0_0_0] exec.MapOperator: Closing child = TS[0] 2018-01-05T11:28:55,968 DEBUG [TezTR-518813_1_20_0_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:28:55,968 DEBUG [TezTR-518813_1_20_0_0_0] exec.TableScanOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:55,968 INFO [TezTR-518813_1_20_0_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:28:55,968 DEBUG [TezTR-518813_1_20_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:55,968 DEBUG [TezTR-518813_1_20_0_0_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-55_495_9015283846029904157-1/-mr-10001 2018-01-05T11:28:55,968 INFO [TezTR-518813_1_20_0_0_0] exec.TableScanOperator: publishing : default.alltypesparquet/ : {numRows=12288, rawDataSize=147456} 2018-01-05T11:28:55,992 DEBUG [TezTR-518813_1_20_0_0_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-55_495_9015283846029904157-1/-mr-10001/tmpstats-0_0 2018-01-05T11:28:55,992 DEBUG [TezTR-518813_1_20_0_0_0] fs.FSStatsPublisher: Writing stats in it : {default.alltypesparquet/={numRows=12288, rawDataSize=147456}} 2018-01-05T11:28:55,992 DEBUG [TezTR-518813_1_20_0_0_0] exec.TableScanOperator: Closing child = SEL[1] 2018-01-05T11:28:55,992 DEBUG [TezTR-518813_1_20_0_0_0] exec.SelectOperator: close called for operator SEL[1] 2018-01-05T11:28:55,992 DEBUG [TezTR-518813_1_20_0_0_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:55,992 INFO [TezTR-518813_1_20_0_0_0] exec.SelectOperator: Closing operator SEL[1] 2018-01-05T11:28:55,992 DEBUG [TezTR-518813_1_20_0_0_0] exec.SelectOperator: Closing child = GBY[2] 2018-01-05T11:28:55,992 DEBUG [TezTR-518813_1_20_0_0_0] exec.GroupByOperator: close called for operator GBY[2] 2018-01-05T11:28:55,992 DEBUG [TezTR-518813_1_20_0_0_0] exec.GroupByOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:55,992 INFO [TezTR-518813_1_20_0_0_0] exec.GroupByOperator: Closing operator GBY[2] 2018-01-05T11:28:55,992 INFO [TezTR-518813_1_20_0_0_0] exec.GroupByOperator: Begin Hash Table flush: size = 1 2018-01-05T11:28:55,993 INFO [TezTR-518813_1_20_0_0_0] exec.ReduceSinkOperator: keys are [] num distributions: 0 2018-01-05T11:28:55,993 DEBUG [TezTR-518813_1_20_0_0_0] exec.GroupByOperator: Closing child = RS[3] 2018-01-05T11:28:55,993 DEBUG [TezTR-518813_1_20_0_0_0] exec.ReduceSinkOperator: close called for operator RS[3] 2018-01-05T11:28:55,993 DEBUG [TezTR-518813_1_20_0_0_0] exec.ReduceSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:55,993 INFO [TezTR-518813_1_20_0_0_0] exec.ReduceSinkOperator: Closing operator RS[3] 2018-01-05T11:28:55,993 INFO [TezTR-518813_1_20_0_0_0] exec.ReduceSinkOperator: RECORDS_OUT_INTERMEDIATE_Map_1:1, 2018-01-05T11:28:55,993 DEBUG [TezTR-518813_1_20_0_0_0] exec.ReduceSinkOperator: 3 Close done 2018-01-05T11:28:55,993 DEBUG [TezTR-518813_1_20_0_0_0] exec.GroupByOperator: 2 Close done 2018-01-05T11:28:55,993 DEBUG [TezTR-518813_1_20_0_0_0] exec.SelectOperator: 1 Close done 2018-01-05T11:28:55,993 DEBUG [TezTR-518813_1_20_0_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:28:55,993 DEBUG [TezTR-518813_1_20_0_0_0] exec.MapOperator: 0 Close done 2018-01-05T11:28:55,993 INFO [TezTR-518813_1_20_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_20_00_000000_0 2018-01-05T11:28:55,994 INFO [TezTR-518813_1_20_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: Waiting for all spills to complete : Pending : 0 2018-01-05T11:28:55,994 INFO [TezTR-518813_1_20_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: All spills complete 2018-01-05T11:28:56,006 DEBUG [TezTR-518813_1_20_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:56,006 INFO [TezTR-518813_1_20_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:56,007 INFO [TezTR-518813_1_20_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_20_00_000000_0, askedToStop=false 2018-01-05T11:28:56,007 INFO [TezTR-518813_1_20_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_20_00_000000_0, stopRequested=false 2018-01-05T11:28:56,007 INFO [TezTR-518813_1_20_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_20_00_000000_0: Counters: 14 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=12288, INPUT_SPLIT_LENGTH_BYTES=594976, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=54893, OUTPUT_BYTES_WITH_OVERHEAD=54903, OUTPUT_BYTES_PHYSICAL=54931, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=12288, RECORDS_OUT_INTERMEDIATE_Map_1=1]] 2018-01-05T11:28:56,007 DEBUG [TezTR-518813_1_20_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_20_0_0_0 thread id: 1360 scheme: file bytesRead: 610516 bytesWritten: 55037 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:56,007 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_20_00_000000_0, isTaskDone=true 2018-01-05T11:28:56,008 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000040, requestId=4, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_20_00_000000_0, eventCount=4 } 2018-01-05T11:28:56,009 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_20][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_20_00_000000_0, creationTime=1515180535719, allocationTime=1515180535720, startTime=1515180535722, finishTime=1515180536008, timeTaken=286, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:56,009 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=4, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:56,009 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_20_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:56,009 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_20_00_000000_0 2018-01-05T11:28:56,009 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_20_00_000000_0 2018-01-05T11:28:56,009 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_20_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:56,009 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_20_00_000000_0; false 2018-01-05T11:28:56,009 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000040=274 2018-01-05T11:28:56,009 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_20_00_000000_0: true 2018-01-05T11:28:56,009 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_20_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:56,010 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:56,010 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_20_00_000000_0 2018-01-05T11:28:56,011 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:56,010 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_20][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_20_00_000000, startTime=1515180535722, finishTime=1515180536009, timeTaken=287, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_20_00_000000_0, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=610516, FILE_BYTES_WRITTEN=55037, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=12288, INPUT_SPLIT_LENGTH_BYTES=594976, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=54893, OUTPUT_BYTES_WITH_OVERHEAD=54903, OUTPUT_BYTES_PHYSICAL=54931, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=12288, RECORDS_OUT_INTERMEDIATE_Map_1=1 2018-01-05T11:28:56,011 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000040, QueryId=djaiswal_20180105112855_cf8184a1-b651-405c-b849-0ee626114470-20, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:28:56,011 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_20_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:56,011 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_20_00_000000_0 2018-01-05T11:28:56,011 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_20_00 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:56,011 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_20_00 [Map 1] 2018-01-05T11:28:56,011 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_20][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_20_00, initRequestedTime=1515180535693, initedTime=1515180535718, startRequestedTime=1515180535700, startedTime=1515180535718, finishTime=1515180536011, timeTaken=293, status=SUCCEEDED, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=610516, FILE_BYTES_WRITTEN=55037, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=12288, INPUT_SPLIT_LENGTH_BYTES=594976, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=54893, OUTPUT_BYTES_WITH_OVERHEAD=54903, OUTPUT_BYTES_PHYSICAL=54931, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=12288, RECORDS_OUT_INTERMEDIATE_Map_1=1, vertexStats=firstTaskStartTime=1515180535722, firstTasksToStart=[ task_1515180518813_0001_20_00_000000 ], lastTaskFinishTime=1515180536008, lastTasksToFinish=[ task_1515180518813_0001_20_00_000000 ], minTaskDuration=286, maxTaskDuration=286, avgTaskDuration=286.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_20_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_20_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:56,011 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_20_00 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:56,011 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_20][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000040, stoppedTime=1515180536011, exitStatus=0 2018-01-05T11:28:56,011 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000040 exited with diagnostics set to Container completed. 2018-01-05T11:28:56,011 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_20_01 [Reducer 2] attempt: attempt_1515180518813_0001_20_00_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:28:56,011 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_20_00 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2 2018-01-05T11:28:56,011 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null 2018-01-05T11:28:56,011 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000040 2018-01-05T11:28:56,011 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: All source tasks completed. Ramping up 1 remaining tasks for vertex: Reducer 2 2018-01-05T11:28:56,011 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Scheduling 1 tasks for vertex: Reducer 2 with totalTasks: 1. 1 source tasks completed out of 1. MinSourceTaskCompletedFraction: 1.0 min: 0.05 max: 0.75 2018-01-05T11:28:56,011 INFO [App Shared Pool - #0] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Map 1, destinationVertexName=Reducer 2. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.UnorderedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:28:56,011 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_20_01 [Reducer 2] 2018-01-05T11:28:56,011 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_20][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_20_01_000000, scheduledTime=1515180536011, launchTime=1515180536011 2018-01-05T11:28:56,011 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_20_01 [Reducer 2] 2018-01-05T11:28:56,012 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_20_01_000000_0, priority=14, capability=, hosts=null 2018-01-05T11:28:56,012 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:56,012 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:56,012 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:56,012 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_20_01_000000_0 on hosts=null 2018-01-05T11:28:56,012 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_20_01_000000_0 on hosts=null 2018-01-05T11:28:56,012 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_20_01_000000_0 2018-01-05T11:28:56,012 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:56,012 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #41, task=TaskInfo{task=attempt_1515180518813_0001_20_01_000000_0, priority=14, startTime=0, containerId=null, uniqueId=40, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000041 2018-01-05T11:28:56,012 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_20_01_000000_0; false 2018-01-05T11:28:56,012 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_20_01_000000_0, priority=14, startTime=311556521, containerId=container_111101111_0001_00_000041assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=40, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:56,012 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000041 for node: 10.22.8.182:49818 2018-01-05T11:28:56,012 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000041, attempt_1515180518813_0001_20_01_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:56,012 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000041 for node: 10.22.8.182:49818 2018-01-05T11:28:56,012 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000041 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:56,012 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000041, launchTime=1515180536012 2018-01-05T11:28:56,012 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=20}, sourceName=Map 1, state=S_SUCCEEDED 2018-01-05T11:28:56,013 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_20_01_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000041] on NM: [10.22.8.182:49818] 2018-01-05T11:28:56,013 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:56,013 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:56,013 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_20][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_20_01_000000_0, startTime=1515180536012, containerId=container_111101111_0001_00_000041, nodeId=10.22.8.182:49818 2018-01-05T11:28:56,013 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_20_01_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112855_cf8184a1-b651-405c-b849-0ee626114470, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000041, dagName=ANALYZE TABLE alltype...,cboolean1,cboolean2 (Stage-0), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.UnorderedKVInput,1}}, Outputs={{out_Reducer 2,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=2, completedTaskCount=1, dagStartTime=1515180535693, firstAttemptStartTime=1515180536012, currentAttemptStartTime=1515180536012} 2018-01-05T11:28:56,013 INFO [IPC Server handler 4 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000041, QueryId=djaiswal_20180105112855_cf8184a1-b651-405c-b849-0ee626114470-20, DagId=20, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, SubmitTime=1515180536013 2018-01-05T11:28:56,013 DEBUG [IPC Server handler 4 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=20} with the ShuffleHandler 2018-01-05T11:28:56,014 DEBUG [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/20] 2018-01-05T11:28:56,014 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:56,014 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_20_01_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:56,014 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_20_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180536012, dagStartTime=1515180535693, withinDagPriority=14, vertexParallelism= 1, selfAndUpstreamParallelism= 2, selfAndUpstreamComplete= 1}] 2018-01-05T11:28:56,014 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_20_01_000000_0 : ACCEPTED 2018-01-05T11:28:56,014 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_20_01_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:56,014 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_20_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180536012, dagStartTime=1515180535693, withinDagPriority=14, vertexParallelism= 1, selfAndUpstreamParallelism= 2, selfAndUpstreamComplete= 1} 2018-01-05T11:28:56,014 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_20_01_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:56,014 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_20_01_000000_0 to pre-emption queue 2018-01-05T11:28:56,014 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_20_01_000000_0: true 2018-01-05T11:28:56,014 INFO [TaskCommunicator # 2] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_20_01_000000_0 2018-01-05T11:28:56,014 INFO [TaskCommunicator # 2] GuaranteedTasks: Received response for attempt_1515180518813_0001_20_01_000000_0, true 2018-01-05T11:28:56,014 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:1931954929 2018-01-05T11:28:56,015 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : ANALYZE TABLE alltype...,cboolean1,cboolean2 (Stage-0), VertexName: Reducer 2, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_20_01_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:28:56,015 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1039138816, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:56,015 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_20_01_000000_0 vertexName: Reducer 2 2018-01-05T11:28:56,015 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000041, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_20_01_000000_0, eventCount=1 } 2018-01-05T11:28:56,015 DEBUG [TezTR-518813_1_20_1_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_20 queryId: djaiswal_20180105112855_cf8184a1-b651-405c-b849-0ee626114470 instanceType: TaskRunner2Callable 2018-01-05T11:28:56,015 INFO [TezTR-518813_1_20_1_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_20_01_000000_0 2018-01-05T11:28:56,015 DEBUG [TezTR-518813_1_20_1_0_0] log.PerfLogger: 2018-01-05T11:28:56,016 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:56,017 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:56,017 INFO [IPC Server handler 16 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_20_01_000000_0 1 events [0,1) total 1 vertex_1515180518813_0001_20_01 [Reducer 2] 2018-01-05T11:28:56,017 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:28:56,017 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_20_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:28:56,018 DEBUG [TezTR-518813_1_20_1_0_0] log.PerfLogger: 2018-01-05T11:28:56,018 INFO [TezTR-518813_1_20_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:56,018 INFO [TezTR-518813_1_20_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:56,018 INFO [TezTR-518813_1_20_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:56,018 INFO [I/O Setup 1 Initialize: {out_Reducer 2}] output.MROutput: out_Reducer 2: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:28:56,018 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:56,018 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:56,018 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:56,018 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:56,018 INFO [TezTR-518813_1_20_1_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:56,018 INFO [TezTR-518813_1_20_1_0_0] resources.MemoryDistributor: InitialRequests=[out_Reducer 2:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.UnorderedKVInput] 2018-01-05T11:28:56,018 INFO [TezTR-518813_1_20_1_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:56,018 INFO [TezTR-518813_1_20_1_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:56,018 INFO [TezTR-518813_1_20_1_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 131404920, TotalRequestedScaled: 6.570246E7, TotalJVMHeap: 1039138816, TotalAvailable: 696223006, TotalRequested/TotalJVMHeap:0.13 2018-01-05T11:28:56,018 INFO [TezTR-518813_1_20_1_0_0] resources.MemoryDistributor: Allocations=[out_Reducer 2:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [Map 1:org.apache.tez.runtime.library.input.UnorderedKVInput:INPUT:131404920:131404920] 2018-01-05T11:28:56,018 INFO [TezTR-518813_1_20_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:56,018 INFO [TezTR-518813_1_20_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 0 2018-01-05T11:28:56,018 INFO [TezTR-518813_1_20_1_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:56,019 INFO [TezTR-518813_1_20_1_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_20_01_000000_0 2018-01-05T11:28:56,019 DEBUG [TezTR-518813_1_20_1_0_0] log.PerfLogger: 2018-01-05T11:28:56,019 DEBUG [TezTR-518813_1_20_1_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_20_01_000000_0_10121 2018-01-05T11:28:56,019 DEBUG [TezTR-518813_1_20_1_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:56,019 DEBUG [TezTR-518813_1_20_1_0_0] mr.ObjectCache: Creating djaiswal_20180105112855_cf8184a1-b651-405c-b849-0ee626114470_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:28:56,019 INFO [TezTR-518813_1_20_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-55_490_8843674450829636198-1/djaiswal/_tez_scratch_dir/977fabe5-46a1-48e5-b7ad-7983015553a5/reduce.xml 2018-01-05T11:28:56,019 DEBUG [TezTR-518813_1_20_1_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-55_490_8843674450829636198-1/djaiswal/_tez_scratch_dir/977fabe5-46a1-48e5-b7ad-7983015553a5/reduce.xml 2018-01-05T11:28:56,019 DEBUG [TezTR-518813_1_20_1_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-55_490_8843674450829636198-1/djaiswal/_tez_scratch_dir/977fabe5-46a1-48e5-b7ad-7983015553a5/reduce.xml 2018-01-05T11:28:56,019 DEBUG [TezTR-518813_1_20_1_0_0] log.PerfLogger: 2018-01-05T11:28:56,019 INFO [TezTR-518813_1_20_1_0_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:28:56,019 DEBUG [TezTR-518813_1_20_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:56,020 DEBUG [TezTR-518813_1_20_1_0_0] log.PerfLogger: 2018-01-05T11:28:56,020 INFO [TezTR-518813_1_20_1_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 2.92KB 2018-01-05T11:28:56,020 DEBUG [TezTR-518813_1_20_1_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:56,020 DEBUG [TezTR-518813_1_20_1_0_0] log.PerfLogger: 2018-01-05T11:28:56,020 DEBUG [TezTR-518813_1_20_1_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:56,020 DEBUG [TezTR-518813_1_20_1_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:56,020 DEBUG [TezTR-518813_1_20_1_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:56,020 INFO [TezTR-518813_1_20_1_0_0] impl.SimpleFetchedInputAllocator: Map_1: RequestedMemory=131404920, AssignedMemory=131404920, maxSingleShuffleLimit=32851230 2018-01-05T11:28:56,020 INFO [TezTR-518813_1_20_1_0_0] impl.ShuffleManager: Map_1: numInputs=1, compressionCodec=NoCompressionCodec, numFetchers=1, ifileBufferSize=4096, ifileReadAheadEnabled=true, ifileReadAheadLength=4194304, localDiskFetchEnabled=false, sharedFetchEnabled=false, keepAlive=false, keepAliveMaxConnections=20, connectionTimeout=20000, readTimeout=180000, bufferSize=8192, bufferSize=8192, maxTaskOutputAtOnce=20 2018-01-05T11:28:56,021 INFO [TezTR-518813_1_20_1_0_0] impl.ShuffleInputEventHandlerImpl: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:28:56,021 INFO [TezTR-518813_1_20_1_0_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:28:56,022 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=20&reduce=0&map=attempt_1515180518813_0001_20_00_000000_0_10120 mapId: [attempt_1515180518813_0001_20_00_000000_0_10120] reduceId: [0] jobId: [job_1515180518813_0001] dagId: [20] keepAlive: false 2018-01-05T11:28:56,022 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=20&reduce=0&map=attempt_1515180518813_0001_20_00_000000_0_10120; hash=...ycBeQxGFk8r44 2018-01-05T11:28:56,022 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=20&reduce=0&map=attempt_1515180518813_0001_20_00_000000_0_10120;reply=dT0MeaCgmAh2U 2018-01-05T11:28:56,022 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=20, user='djaiswal', attemptId='attempt_1515180518813_0001_20_00_000000_0_10120'} via loader 2018-01-05T11:28:56,022 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=20, user='djaiswal', attemptId='attempt_1515180518813_0001_20_00_000000_0_10120'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:28:56,022 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_20_00_000000_0_10120 not found 2018-01-05T11:28:56,022 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_20_00_000000_0_10120,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/20/output/attempt_1515180518813_0001_20_00_000000_0_10120/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/20/output/attempt_1515180518813_0001_20_00_000000_0_10120/file.out.index 2018-01-05T11:28:56,022 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Content Length in shuffle : 54962 2018-01-05T11:28:56,022 INFO [Fetcher_B {Map_1} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=20&reduce=0&map=attempt_1515180518813_0001_20_00_000000_0_10120 sent hash and receievd reply 0 ms 2018-01-05T11:28:56,022 INFO [Fetcher_B {Map_1} #0] ShuffleManager.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_20_00_000000_0_10120} to MEMORY, csize=54907, dsize=54903, EndTime=1515180536022, TimeTaken=0, Rate=0.00 MB/s 2018-01-05T11:28:56,022 INFO [Fetcher_B {Map_1} #0] impl.ShuffleManager: All inputs fetched for input vertex : Map 1 2018-01-05T11:28:56,023 INFO [Fetcher_B {Map_1} #0] impl.ShuffleManager: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.05 MB/s) 2018-01-05T11:28:56,023 INFO [TezTR-518813_1_20_1_0_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:28:56,023 INFO [ShuffleRunner {Map_1}] impl.ShuffleManager: Map_1: Shutting down FetchScheduler, Was Interrupted: false 2018-01-05T11:28:56,023 INFO [TezTR-518813_1_20_1_0_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:56,023 DEBUG [TezTR-518813_1_20_1_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:28:56,023 INFO [ShuffleRunner {Map_1}] impl.ShuffleManager: Map_1: Scheduler thread completed 2018-01-05T11:28:56,023 DEBUG [TezTR-518813_1_20_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:56,023 DEBUG [TezTR-518813_1_20_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:56,023 DEBUG [TezTR-518813_1_20_1_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11] columnTypes=[struct, struct, struct, struct, struct, struct, struct, struct, struct, struct, struct, struct] 2018-01-05T11:28:56,023 DEBUG [TezTR-518813_1_20_1_0_0] log.PerfLogger: 2018-01-05T11:28:56,023 INFO [TezTR-518813_1_20_1_0_0] exec.GroupByOperator: Initializing operator GBY[4] 2018-01-05T11:28:56,023 DEBUG [TezTR-518813_1_20_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:56,023 DEBUG [TezTR-518813_1_20_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:56,023 DEBUG [TezTR-518813_1_20_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:56,023 DEBUG [TezTR-518813_1_20_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:56,023 DEBUG [TezTR-518813_1_20_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:56,023 DEBUG [TezTR-518813_1_20_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:56,023 DEBUG [TezTR-518813_1_20_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:56,023 DEBUG [TezTR-518813_1_20_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:56,023 DEBUG [TezTR-518813_1_20_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:56,023 DEBUG [TezTR-518813_1_20_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:56,023 DEBUG [TezTR-518813_1_20_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:56,023 DEBUG [TezTR-518813_1_20_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:56,023 INFO [TezTR-518813_1_20_1_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:56,023 DEBUG [TezTR-518813_1_20_1_0_0] exec.GroupByOperator: Initialization Done 4 GBY 2018-01-05T11:28:56,023 DEBUG [TezTR-518813_1_20_1_0_0] exec.GroupByOperator: Operator 4 GBY initialized 2018-01-05T11:28:56,023 DEBUG [TezTR-518813_1_20_1_0_0] exec.GroupByOperator: Initializing children of 4 GBY 2018-01-05T11:28:56,023 DEBUG [TezTR-518813_1_20_1_0_0] exec.FileSinkOperator: Initializing child 6 FS 2018-01-05T11:28:56,023 INFO [TezTR-518813_1_20_1_0_0] exec.FileSinkOperator: Initializing operator FS[6] 2018-01-05T11:28:56,023 DEBUG [TezTR-518813_1_20_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:56,024 INFO [TezTR-518813_1_20_1_0_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@42c60ce9]:[_col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11]:[struct, struct, struct, struct, struct, struct, struct, struct, struct, struct, struct, struct]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@35efcc01 2018-01-05T11:28:56,024 DEBUG [TezTR-518813_1_20_1_0_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:28:56,024 DEBUG [TezTR-518813_1_20_1_0_0] exec.FileSinkOperator: Initialization Done 6 FS 2018-01-05T11:28:56,024 DEBUG [TezTR-518813_1_20_1_0_0] exec.FileSinkOperator: Operator 6 FS initialized 2018-01-05T11:28:56,024 DEBUG [TezTR-518813_1_20_1_0_0] exec.FileSinkOperator: Initialization Done 6 FS done is reset. 2018-01-05T11:28:56,024 DEBUG [TezTR-518813_1_20_1_0_0] exec.GroupByOperator: Initialization Done 4 GBY done is reset. 2018-01-05T11:28:56,024 INFO [TezTR-518813_1_20_1_0_0] tez.ReduceRecordProcessor: Id =4 Id =6 <\Children> Id = 4 null<\Parent> <\FS> <\Children> <\Parent> <\GBY> 2018-01-05T11:28:56,024 DEBUG [TezTR-518813_1_20_1_0_0] log.PerfLogger: 2018-01-05T11:28:56,024 INFO [TezTR-518813_1_20_1_0_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 2 2018-01-05T11:28:56,024 DEBUG [TezTR-518813_1_20_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:28:56,024 DEBUG [TezTR-518813_1_20_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:28:56,024 DEBUG [TezTR-518813_1_20_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:28:56,024 DEBUG [TezTR-518813_1_20_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:28:56,024 DEBUG [TezTR-518813_1_20_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:28:56,024 DEBUG [TezTR-518813_1_20_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:28:56,026 INFO [TezTR-518813_1_20_1_0_0] readers.UnorderedKVReader: Num Records read: 1 2018-01-05T11:28:56,026 DEBUG [TezTR-518813_1_20_1_0_0] log.PerfLogger: 2018-01-05T11:28:56,026 DEBUG [TezTR-518813_1_20_1_0_0] mr.ObjectCache: djaiswal_20180105112855_cf8184a1-b651-405c-b849-0ee626114470_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:28:56,026 DEBUG [TezTR-518813_1_20_1_0_0] exec.GroupByOperator: close called for operator GBY[4] 2018-01-05T11:28:56,026 INFO [TezTR-518813_1_20_1_0_0] exec.GroupByOperator: Closing operator GBY[4] 2018-01-05T11:28:56,028 INFO [TezTR-518813_1_20_1_0_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-55_495_9015283846029904157-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-55_495_9015283846029904157-1/_tmp.-ext-10002/000000_0 2018-01-05T11:28:56,028 INFO [TezTR-518813_1_20_1_0_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-55_495_9015283846029904157-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-55_495_9015283846029904157-1/_task_tmp.-ext-10002/_tmp.000000_0 2018-01-05T11:28:56,028 INFO [TezTR-518813_1_20_1_0_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-55_495_9015283846029904157-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-55_495_9015283846029904157-1/_tmp.-ext-10002/000000_0 2018-01-05T11:28:56,062 INFO [TezTR-518813_1_20_1_0_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:28:56,062 DEBUG [TezTR-518813_1_20_1_0_0] exec.GroupByOperator: Closing child = FS[6] 2018-01-05T11:28:56,062 DEBUG [TezTR-518813_1_20_1_0_0] exec.FileSinkOperator: close called for operator FS[6] 2018-01-05T11:28:56,062 DEBUG [TezTR-518813_1_20_1_0_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:56,062 INFO [TezTR-518813_1_20_1_0_0] exec.FileSinkOperator: Closing operator FS[6] 2018-01-05T11:28:56,062 INFO [TezTR-518813_1_20_1_0_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:28:56,062 INFO [TezTR-518813_1_20_1_0_0] exec.FileSinkOperator: RECORDS_OUT_0:1, 2018-01-05T11:28:56,062 DEBUG [TezTR-518813_1_20_1_0_0] exec.FileSinkOperator: 6 Close done 2018-01-05T11:28:56,062 DEBUG [TezTR-518813_1_20_1_0_0] exec.GroupByOperator: 4 Close done 2018-01-05T11:28:56,063 INFO [TezTR-518813_1_20_1_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_20_01_000000_0 2018-01-05T11:28:56,063 INFO [TezTR-518813_1_20_1_0_0] impl.ShuffleInputEventHandlerImpl: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:28:56,063 INFO [TezTR-518813_1_20_1_0_0] impl.ShuffleManager: Shutting down pending fetchers on sourceMap_1: 0 2018-01-05T11:28:56,063 INFO [TezTR-518813_1_20_1_0_0] output.MROutput: out_Reducer 2 closed 2018-01-05T11:28:56,063 DEBUG [TezTR-518813_1_20_1_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:56,063 INFO [TezTR-518813_1_20_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:56,063 INFO [TezTR-518813_1_20_1_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_20_01_000000_0, askedToStop=false 2018-01-05T11:28:56,063 INFO [TezTR-518813_1_20_1_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_20_01_000000_0, stopRequested=false 2018-01-05T11:28:56,063 INFO [TezTR-518813_1_20_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_20_01_000000_0: Counters: 14 [[org.apache.tez.common.counters.TaskCounter NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=54907, SHUFFLE_BYTES_DECOMPRESSED=54903, SHUFFLE_BYTES_TO_MEM=54907, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=3, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1][HIVE CREATED_FILES=1, RECORDS_OUT_0=1]] 2018-01-05T11:28:56,063 DEBUG [TezTR-518813_1_20_1_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_20_1_0_0 thread id: 1372 scheme: file bytesRead: 0 bytesWritten: 73876 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:56,063 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_20_01_000000_0, isTaskDone=true 2018-01-05T11:28:56,063 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000041, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_20_01_000000_0, eventCount=2 } 2018-01-05T11:28:56,064 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_20][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_20_01_000000_0, creationTime=1515180536011, allocationTime=1515180536012, startTime=1515180536012, finishTime=1515180536064, timeTaken=52, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:56,064 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_20_01_000000_0 2018-01-05T11:28:56,064 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:28:56,064 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_20_01_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:56,064 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_20_01_000000_0; false 2018-01-05T11:28:56,064 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_20_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:56,064 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_20_01_000000_0 2018-01-05T11:28:56,064 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_20_01_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:56,064 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_20][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_20_01_000000, startTime=1515180536012, finishTime=1515180536064, timeTaken=52, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_20_01_000000_0, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=73876, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=54907, SHUFFLE_BYTES_DECOMPRESSED=54903, SHUFFLE_BYTES_TO_MEM=54907, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=3, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1 2018-01-05T11:28:56,065 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:56,065 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000041=51 2018-01-05T11:28:56,065 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_20_01_000000_0: true 2018-01-05T11:28:56,065 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:56,066 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_20_01 [Reducer 2], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:56,066 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_20_01 [Reducer 2] 2018-01-05T11:28:56,066 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_20][Event:VERTEX_FINISHED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_20_01, initRequestedTime=1515180535700, initedTime=1515180535706, startRequestedTime=1515180535718, startedTime=1515180535718, finishTime=1515180536066, timeTaken=348, status=SUCCEEDED, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=73876, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=54907, SHUFFLE_BYTES_DECOMPRESSED=54903, SHUFFLE_BYTES_TO_MEM=54907, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=3, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1, vertexStats=firstTaskStartTime=1515180536012, firstTasksToStart=[ task_1515180518813_0001_20_01_000000 ], lastTaskFinishTime=1515180536064, lastTasksToFinish=[ task_1515180518813_0001_20_01_000000 ], minTaskDuration=52, maxTaskDuration=52, avgTaskDuration=52.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_20_01_000000 ], longestDurationTasks=[ task_1515180518813_0001_20_01_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:56,066 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_20_01 [Reducer 2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:56,066 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_20][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000041, stoppedTime=1515180536066, exitStatus=0 2018-01-05T11:28:56,066 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000041 exited with diagnostics set to Container completed. 2018-01-05T11:28:56,066 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_20_01 [Reducer 2] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2 2018-01-05T11:28:56,066 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null 2018-01-05T11:28:56,066 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_20_00 [Map 1] 2018-01-05T11:28:56,066 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_20_01 [Reducer 2] 2018-01-05T11:28:56,067 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_20, queueSize=0 2018-01-05T11:28:56,067 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_20][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_20, startTime=1515180535693, finishTime=1515180536066, timeTaken=373, status=SUCCEEDED, diagnostics=, counters=Counters: 35, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, AM_CPU_MILLISECONDS=1127, AM_GC_TIME_MILLIS=0, File System Counters, FILE_BYTES_READ=610516, FILE_BYTES_WRITTEN=128913, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=12289, INPUT_SPLIT_LENGTH_BYTES=594976, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=54893, OUTPUT_BYTES_WITH_OVERHEAD=54903, OUTPUT_BYTES_PHYSICAL=54931, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=54907, SHUFFLE_BYTES_DECOMPRESSED=54903, SHUFFLE_BYTES_TO_MEM=54907, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=3, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1, HIVE, CREATED_FILES=1, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=12288, RECORDS_OUT_0=1, RECORDS_OUT_INTERMEDIATE_Map_1=1 2018-01-05T11:28:56,067 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_20 finished with state: SUCCEEDED 2018-01-05T11:28:56,067 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_20 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:28:56,067 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000041 2018-01-05T11:28:56,068 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_20, dagState=SUCCEEDED 2018-01-05T11:28:56,068 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_20_post 2018-01-05T11:28:56,068 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:56,068 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:28:56,068 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:28:56,068 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 20 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=2, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=2,NumRequestsWithoutLocation=2, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=2} 2018-01-05T11:28:56,068 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:28:56,068 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=ANALYZE TABLE alltype...,cboolean1,cboolean2 (Stage-0), with id=dag_1515180518813_0001_20 2018-01-05T11:28:56,068 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 20, to 10.22.8.182:49818 2018-01-05T11:28:56,068 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=2, Remaining=0 2018-01-05T11:28:56,068 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=ANALYZE TABLE alltype...,cboolean1,cboolean2 (Stage-0), with id=dag_1515180518813_0001_20 2018-01-05T11:28:56,068 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=20} 2018-01-05T11:28:56,068 INFO [IPC Server handler 3 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=20}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:28:56,068 INFO [IPC Server handler 3 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/20 after 300 seconds 2018-01-05T11:28:56,068 ERROR [IPC Server handler 3 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112855_cf8184a1-b651-405c-b849-0ee626114470, dag_1515180518813_0001_20 2018-01-05T11:28:56,068 INFO [IPC Server handler 3 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112855_cf8184a1-b651-405c-b849-0ee626114470 2018-01-05T11:28:56,068 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=20} = 1 2018-01-05T11:28:56,068 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Issuing killFragment for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=20} attempt_1515180518813_0001_20_01_000000_0 2018-01-05T11:28:56,068 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Removing attempt_1515180518813_0001_20_01_000000_0 from preemptionQueue 2018-01-05T11:28:56,068 INFO [IPC Server handler 3 on 49818] impl.TaskRunnerCallable: Ignoring kill request for task attempt_1515180518813_0001_20_01_000000_0 since it's already complete 2018-01-05T11:28:56,068 DEBUG [IPC Server handler 3 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=20} 2018-01-05T11:28:56,068 DEBUG [IPC Server handler 3 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:28:56,068 DEBUG [IPC Server handler 3 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=20} 2018-01-05T11:28:56,069 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,069 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,069 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,069 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 Reducer 2: 1/1 2018-01-05T11:28:56,069 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,070 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,070 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: TaskId for 000000_0 = 000000 2018-01-05T11:28:56,070 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,070 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,070 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,071 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-2:STATS] in serial mode 2018-01-05T11:28:56,071 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@289e504f, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:56,071 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:56,071 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,071 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:56,071 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:56,072 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:56,072 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:56,072 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,072 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:56,072 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:56,072 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-55_495_9015283846029904157-1/-mr-10001 2018-01-05T11:28:56,072 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-55_495_9015283846029904157-1/-mr-10001 2018-01-05T11:28:56,073 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.alltypesparquet/ numRows 2018-01-05T11:28:56,073 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.alltypesparquet/ numRows 12288 2018-01-05T11:28:56,073 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.alltypesparquet/ rawDataSize 2018-01-05T11:28:56,073 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.alltypesparquet/ rawDataSize 147456 2018-01-05T11:28:56,073 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,074 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:56,074 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:56,074 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:56,074 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:56,074 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,074 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,074 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,075 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:56,075 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:56,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,075 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:56,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,075 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=alltypesparquet newtbl=alltypesparquet 2018-01-05T11:28:56,075 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=alltypesparquet newtbl=alltypesparquet 2018-01-05T11:28:56,078 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,082 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,082 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,082 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.alltypesparquet stats: [numFiles=1, numRows=12288, totalSize=594976, rawDataSize=147456] 2018-01-05T11:28:56,082 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: About to delete stats tmp dir :file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-55_495_9015283846029904157-1/-mr-10001 2018-01-05T11:28:56,098 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Time taken to get FileStatuses: 14 2018-01-05T11:28:56,098 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:28:56,098 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 14 2018-01-05T11:28:56,098 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FetchOperator: Creating fetchTask with deserializer typeinfo: struct<_col0:struct,_col1:struct,_col2:struct,_col3:struct,_col4:struct,_col5:struct,_col6:struct,_col7:struct,_col8:struct,_col9:struct,_col10:struct,_col11:struct> 2018-01-05T11:28:56,098 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FetchOperator: deserializer properties: table properties: {columns=_col0,_col1,_col2,_col3,_col4,_col5,_col6,_col7,_col8,_col9,_col10,_col11, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct:struct:struct:struct:struct:struct:struct:struct:struct:struct:struct:struct, escape.delim=\} partition properties: {columns=_col0,_col1,_col2,_col3,_col4,_col5,_col6,_col7,_col8,_col9,_col10,_col11, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct:struct:struct:struct:struct:struct:struct:struct:struct:struct:struct:struct, escape.delim=\} 2018-01-05T11:28:56,100 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,100 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: write_column_statistics: db=default table=alltypesparquet 2018-01-05T11:28:56,101 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=write_column_statistics: db=default table=alltypesparquet 2018-01-05T11:28:56,104 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=alltypesparquet tableName=default colName=ctinyint 2018-01-05T11:28:56,105 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=alltypesparquet tableName=default colName=csmallint 2018-01-05T11:28:56,105 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=alltypesparquet tableName=default colName=cint 2018-01-05T11:28:56,105 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=alltypesparquet tableName=default colName=cbigint 2018-01-05T11:28:56,106 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=alltypesparquet tableName=default colName=cfloat 2018-01-05T11:28:56,106 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=alltypesparquet tableName=default colName=cdouble 2018-01-05T11:28:56,107 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=alltypesparquet tableName=default colName=cstring1 2018-01-05T11:28:56,107 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=alltypesparquet tableName=default colName=cstring2 2018-01-05T11:28:56,108 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=alltypesparquet tableName=default colName=ctimestamp1 2018-01-05T11:28:56,108 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=alltypesparquet tableName=default colName=ctimestamp2 2018-01-05T11:28:56,108 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=alltypesparquet tableName=default colName=cboolean1 2018-01-05T11:28:56,111 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=alltypesparquet tableName=default colName=cboolean2 2018-01-05T11:28:56,114 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000040, QueryId=djaiswal_20180105112855_cf8184a1-b651-405c-b849-0ee626114470-20, DagId=20, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180535734, EndTime=1515180536114 2018-01-05T11:28:56,114 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_20_01_000000_0 2018-01-05T11:28:56,114 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_20_01_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:56,114 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_20_01_000000_0 2018-01-05T11:28:56,114 INFO [ExecutionCompletionThread #0] impl.QueryTracker: Ignoring fragmentComplete message for unknown query: QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=20} 2018-01-05T11:28:56,115 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,115 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,115 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,115 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,115 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: ANALYZE TABLE alltypesparquet COMPUTE STATISTICS FOR COLUMNS ctinyint,csmallint,cint,cbigint,cfloat,cdouble,cstring1,cstring2,ctimestamp1,ctimestamp2,cboolean1,cboolean2 2018-01-05T11:28:56,116 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:28:56,116 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@alltypesparquet 2018-01-05T11:28:56,116 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@alltypesparquet 2018-01-05T11:28:56,116 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-55_495_9015283846029904157-1/-mr-10000 2018-01-05T11:28:56,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,116 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:56,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=7, setPartitionColumnStatistics_(SetPartitionsStatsRequest, )=15} 2018-01-05T11:28:56,116 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112855_cf8184a1-b651-405c-b849-0ee626114470); Time taken: 0.559 seconds 2018-01-05T11:28:56,116 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:56,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,116 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query ANALYZE TABLE alltypesparquet COMPUTE STATISTICS FOR COLUMNS ctinyint,csmallint,cint,cbigint,cfloat,cdouble,cstring1,cstring2,ctimestamp1,ctimestamp2,cboolean1,cboolean2 2018-01-05T11:28:56,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-55_490_8843674450829636198-1 2018-01-05T11:28:56,116 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.626 seconds 2018-01-05T11:28:56,116 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,116 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:56,116 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,116 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:56,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:56,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: -- -- Table primitives -- DROP TABLE IF EXISTS primitives 2018-01-05T11:28:56,117 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112856_acd4945d-3ad9-4c59-887d-77b6736e4293): -- -- Table primitives -- DROP TABLE IF EXISTS primitives 2018-01-05T11:28:56,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: -- -- Table primitives -- DROP TABLE IF EXISTS primitives 2018-01-05T11:28:56,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:56,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,117 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,117 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=primitives 2018-01-05T11:28:56,117 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=primitives 2018-01-05T11:28:56,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,118 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:56,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,118 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:56,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,118 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:56,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:28:56,118 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112856_acd4945d-3ad9-4c59-887d-77b6736e4293); Time taken: 0.001 seconds 2018-01-05T11:28:56,118 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,118 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112856_acd4945d-3ad9-4c59-887d-77b6736e4293): -- -- Table primitives -- DROP TABLE IF EXISTS primitives 2018-01-05T11:28:56,119 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,119 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: -- -- Table primitives -- DROP TABLE IF EXISTS primitives 2018-01-05T11:28:56,119 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:56,119 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,119 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,119 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,119 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,119 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:56,119 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,119 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=primitives 2018-01-05T11:28:56,119 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=primitives 2018-01-05T11:28:56,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,120 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table primitives not found: default.primitives table not found 2018-01-05T11:28:56,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,120 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=primitives 2018-01-05T11:28:56,120 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=primitives 2018-01-05T11:28:56,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,120 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: -- -- Table primitives -- DROP TABLE IF EXISTS primitives 2018-01-05T11:28:56,120 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:56,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,120 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:56,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=0} 2018-01-05T11:28:56,120 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112856_acd4945d-3ad9-4c59-887d-77b6736e4293); Time taken: 0.002 seconds 2018-01-05T11:28:56,120 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:56,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,120 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query -- -- Table primitives -- DROP TABLE IF EXISTS primitives 2018-01-05T11:28:56,120 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.003 seconds 2018-01-05T11:28:56,120 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,120 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:56,121 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,121 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:56,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:56,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: CREATE TABLE primitives ( id INT COMMENT 'default', bool_col BOOLEAN COMMENT 'default', tinyint_col TINYINT COMMENT 'default', smallint_col SMALLINT COMMENT 'default', int_col INT COMMENT 'default', bigint_col BIGINT COMMENT 'default', float_col FLOAT COMMENT 'default', double_col DOUBLE COMMENT 'default', date_string_col STRING COMMENT 'default', string_col STRING COMMENT 'default', timestamp_col TIMESTAMP COMMENT 'default') PARTITIONED BY (year INT COMMENT 'default', month INT COMMENT 'default') ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' ESCAPED BY '\\' STORED AS TEXTFILE 2018-01-05T11:28:56,121 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112856_5805bccb-93ad-43c8-af52-378a83c1a0e7): CREATE TABLE primitives ( id INT COMMENT 'default', bool_col BOOLEAN COMMENT 'default', tinyint_col TINYINT COMMENT 'default', smallint_col SMALLINT COMMENT 'default', int_col INT COMMENT 'default', bigint_col BIGINT COMMENT 'default', float_col FLOAT COMMENT 'default', double_col DOUBLE COMMENT 'default', date_string_col STRING COMMENT 'default', string_col STRING COMMENT 'default', timestamp_col TIMESTAMP COMMENT 'default') PARTITIONED BY (year INT COMMENT 'default', month INT COMMENT 'default') ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' ESCAPED BY '\\' STORED AS TEXTFILE 2018-01-05T11:28:56,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: CREATE TABLE primitives ( id INT COMMENT 'default', bool_col BOOLEAN COMMENT 'default', tinyint_col TINYINT COMMENT 'default', smallint_col SMALLINT COMMENT 'default', int_col INT COMMENT 'default', bigint_col BIGINT COMMENT 'default', float_col FLOAT COMMENT 'default', double_col DOUBLE COMMENT 'default', date_string_col STRING COMMENT 'default', string_col STRING COMMENT 'default', timestamp_col TIMESTAMP COMMENT 'default') PARTITIONED BY (year INT COMMENT 'default', month INT COMMENT 'default') ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' ESCAPED BY '\\' STORED AS TEXTFILE 2018-01-05T11:28:56,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:56,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,122 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,122 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,122 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:28:56,122 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Creating table default.primitives position=13 2018-01-05T11:28:56,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,123 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_database: default 2018-01-05T11:28:56,123 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_database: default 2018-01-05T11:28:56,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:56,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,123 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:56,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: validation start 2018-01-05T11:28:56,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:28:56,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Not a partition. 2018-01-05T11:28:56,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,123 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:56,124 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,124 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:56,124 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getDatabase_(String, )=0, flushCache_()=0} 2018-01-05T11:28:56,124 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112856_5805bccb-93ad-43c8-af52-378a83c1a0e7); Time taken: 0.003 seconds 2018-01-05T11:28:56,124 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,124 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,124 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112856_5805bccb-93ad-43c8-af52-378a83c1a0e7): CREATE TABLE primitives ( id INT COMMENT 'default', bool_col BOOLEAN COMMENT 'default', tinyint_col TINYINT COMMENT 'default', smallint_col SMALLINT COMMENT 'default', int_col INT COMMENT 'default', bigint_col BIGINT COMMENT 'default', float_col FLOAT COMMENT 'default', double_col DOUBLE COMMENT 'default', date_string_col STRING COMMENT 'default', string_col STRING COMMENT 'default', timestamp_col TIMESTAMP COMMENT 'default') PARTITIONED BY (year INT COMMENT 'default', month INT COMMENT 'default') ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' ESCAPED BY '\\' STORED AS TEXTFILE 2018-01-05T11:28:56,125 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,125 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: CREATE TABLE primitives ( id INT COMMENT 'default', bool_col BOOLEAN COMMENT 'default', tinyint_col TINYINT COMMENT 'default', smallint_col SMALLINT COMMENT 'default', int_col INT COMMENT 'default', bigint_col BIGINT COMMENT 'default', float_col FLOAT COMMENT 'default', double_col DOUBLE COMMENT 'default', date_string_col STRING COMMENT 'default', string_col STRING COMMENT 'default', timestamp_col TIMESTAMP COMMENT 'default') PARTITIONED BY (year INT COMMENT 'default', month INT COMMENT 'default') ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' ESCAPED BY '\\' STORED AS TEXTFILE 2018-01-05T11:28:56,125 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: CREATETABLE 2018-01-05T11:28:56,125 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: database:default 2018-01-05T11:28:56,125 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@primitives 2018-01-05T11:28:56,125 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,125 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,125 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,125 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,125 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:56,125 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Found class for org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe 2018-01-05T11:28:56,125 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: creating table default.primitives on file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/primitives 2018-01-05T11:28:56,126 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,126 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: create_table: Table(tableName:primitives, dbName:default, owner:hive_test_user, createTime:1515180536, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:id, type:int, comment:default), FieldSchema(name:bool_col, type:boolean, comment:default), FieldSchema(name:tinyint_col, type:tinyint, comment:default), FieldSchema(name:smallint_col, type:smallint, comment:default), FieldSchema(name:int_col, type:int, comment:default), FieldSchema(name:bigint_col, type:bigint, comment:default), FieldSchema(name:float_col, type:float, comment:default), FieldSchema(name:double_col, type:double, comment:default), FieldSchema(name:date_string_col, type:string, comment:default), FieldSchema(name:string_col, type:string, comment:default), FieldSchema(name:timestamp_col, type:timestamp, comment:default)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/primitives, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{escape.delim=\, field.delim=,, serialization.format=,}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[FieldSchema(name:year, type:int, comment:default), FieldSchema(name:month, type:int, comment:default)], parameters:{}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:56,126 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=create_table: Table(tableName:primitives, dbName:default, owner:hive_test_user, createTime:1515180536, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:id, type:int, comment:default), FieldSchema(name:bool_col, type:boolean, comment:default), FieldSchema(name:tinyint_col, type:tinyint, comment:default), FieldSchema(name:smallint_col, type:smallint, comment:default), FieldSchema(name:int_col, type:int, comment:default), FieldSchema(name:bigint_col, type:bigint, comment:default), FieldSchema(name:float_col, type:float, comment:default), FieldSchema(name:double_col, type:double, comment:default), FieldSchema(name:date_string_col, type:string, comment:default), FieldSchema(name:string_col, type:string, comment:default), FieldSchema(name:timestamp_col, type:timestamp, comment:default)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/primitives, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{escape.delim=\, field.delim=,, serialization.format=,}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[FieldSchema(name:year, type:int, comment:default), FieldSchema(name:month, type:int, comment:default)], parameters:{}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:56,126 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hive-site.xml 2018-01-05T11:28:56,127 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hivemetastore-site.xml 2018-01-05T11:28:56,127 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Unable to find config file metastore-site.xml 2018-01-05T11:28:56,127 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file null 2018-01-05T11:28:56,135 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:56,135 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: Location: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/primitives specified for non-external table:primitives 2018-01-05T11:28:56,135 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/primitives 2018-01-05T11:28:56,152 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,154 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@primitives) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@primitives) Type=TABLE WriteType=DDL_NO_LOCK is present 2018-01-05T11:28:56,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,155 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: CREATE TABLE primitives ( id INT COMMENT 'default', bool_col BOOLEAN COMMENT 'default', tinyint_col TINYINT COMMENT 'default', smallint_col SMALLINT COMMENT 'default', int_col INT COMMENT 'default', bigint_col BIGINT COMMENT 'default', float_col FLOAT COMMENT 'default', double_col DOUBLE COMMENT 'default', date_string_col STRING COMMENT 'default', string_col STRING COMMENT 'default', timestamp_col TIMESTAMP COMMENT 'default') PARTITIONED BY (year INT COMMENT 'default', month INT COMMENT 'default') ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' ESCAPED BY '\\' STORED AS TEXTFILE 2018-01-05T11:28:56,155 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: CREATETABLE 2018-01-05T11:28:56,155 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: database:default 2018-01-05T11:28:56,155 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@primitives 2018-01-05T11:28:56,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,155 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:56,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {createTable_(Table, )=29, isCompatibleWith_(Configuration, )=0} 2018-01-05T11:28:56,155 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112856_5805bccb-93ad-43c8-af52-378a83c1a0e7); Time taken: 0.031 seconds 2018-01-05T11:28:56,155 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:56,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,155 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query CREATE TABLE primitives ( id INT COMMENT 'default', bool_col BOOLEAN COMMENT 'default', tinyint_col TINYINT COMMENT 'default', smallint_col SMALLINT COMMENT 'default', int_col INT COMMENT 'default', bigint_col BIGINT COMMENT 'default', float_col FLOAT COMMENT 'default', double_col DOUBLE COMMENT 'default', date_string_col STRING COMMENT 'default', string_col STRING COMMENT 'default', timestamp_col TIMESTAMP COMMENT 'default') PARTITIONED BY (year INT COMMENT 'default', month INT COMMENT 'default') ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' ESCAPED BY '\\' STORED AS TEXTFILE 2018-01-05T11:28:56,155 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.034 seconds 2018-01-05T11:28:56,155 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,155 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:56,155 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,155 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:56,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:56,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,155 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/types/primitives/090101.txt" OVERWRITE INTO TABLE primitives PARTITION(year=2009, month=1) 2018-01-05T11:28:56,155 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112856_647eb6b7-38a1-41f7-8b32-d0cd56339984): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/types/primitives/090101.txt" OVERWRITE INTO TABLE primitives PARTITION(year=2009, month=1) 2018-01-05T11:28:56,156 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,156 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/types/primitives/090101.txt" OVERWRITE INTO TABLE primitives PARTITION(year=2009, month=1) 2018-01-05T11:28:56,156 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:56,156 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,156 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,156 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,156 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,156 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,156 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.LoadSemanticAnalyzer: file@null@/Users/djaiswal/hive/hive/data/files/types/primitives/090101.txt 2018-01-05T11:28:56,156 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,156 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=primitives 2018-01-05T11:28:56,156 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=primitives 2018-01-05T11:28:56,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,167 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=primitives[2009,1] 2018-01-05T11:28:56,167 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=primitives[2009,1] 2018-01-05T11:28:56,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,171 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,171 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,172 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,172 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=primitives[2009,1] 2018-01-05T11:28:56,172 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=primitives[2009,1] 2018-01-05T11:28:56,175 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,176 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,176 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,177 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:56,177 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,177 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:56,177 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,177 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:56,177 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=3, flushCache_()=0} 2018-01-05T11:28:56,177 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112856_647eb6b7-38a1-41f7-8b32-d0cd56339984); Time taken: 0.022 seconds 2018-01-05T11:28:56,177 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,177 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,177 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112856_647eb6b7-38a1-41f7-8b32-d0cd56339984): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/types/primitives/090101.txt" OVERWRITE INTO TABLE primitives PARTITION(year=2009, month=1) 2018-01-05T11:28:56,178 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,178 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/types/primitives/090101.txt" OVERWRITE INTO TABLE primitives PARTITION(year=2009, month=1) 2018-01-05T11:28:56,178 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: LOAD 2018-01-05T11:28:56,178 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/types/primitives/090101.txt 2018-01-05T11:28:56,178 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@primitives 2018-01-05T11:28:56,178 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,178 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,178 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,178 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,178 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MOVE] in serial mode 2018-01-05T11:28:56,178 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@107947a2, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:56,178 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:56,178 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,178 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:56,178 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:56,178 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:56,178 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:56,178 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,178 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,178 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:56,178 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:56,178 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,178 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:56,178 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,178 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:56,178 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:56,178 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,178 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Loading data to table default.primitives partition (year=2009, month=1) from file:/Users/djaiswal/hive/hive/data/files/types/primitives/090101.txt 2018-01-05T11:28:56,179 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,179 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:56,179 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:56,179 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:56,179 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:56,179 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,179 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,179 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,180 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,180 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,180 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:56,180 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:56,180 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,180 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:56,181 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,181 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=primitives 2018-01-05T11:28:56,181 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=primitives 2018-01-05T11:28:56,187 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,188 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,188 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,188 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=primitives[2009,1] 2018-01-05T11:28:56,188 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=primitives[2009,1] 2018-01-05T11:28:56,191 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,192 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,192 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,193 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.MoveTask: Partition is: {year=2009, month=1} 2018-01-05T11:28:56,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,193 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: partition_name_has_valid_characters 2018-01-05T11:28:56,193 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=partition_name_has_valid_characters 2018-01-05T11:28:56,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,193 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=primitives 2018-01-05T11:28:56,193 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=primitives 2018-01-05T11:28:56,195 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,195 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,195 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,195 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=primitives[2009,1] 2018-01-05T11:28:56,195 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=primitives[2009,1] 2018-01-05T11:28:56,198 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,199 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,199 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,199 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,199 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/primitives/year=2009/month=1 2018-01-05T11:28:56,218 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000041, QueryId=djaiswal_20180105112855_cf8184a1-b651-405c-b849-0ee626114470-20, DagId=20, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180536014, EndTime=1515180536218 2018-01-05T11:28:56,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/data/files/types/primitives/090101.txt/ and the destination path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/primitives/year=2009/month=1/090101.txt/ 2018-01-05T11:28:56,221 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/primitives/year=2009/month=1/090101.txt/ and the destination path is /Users/djaiswal/hive/hive/data/files/types/primitives/090101.txt/ 2018-01-05T11:28:56,243 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,243 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: altering partition for table primitives with partition spec : {year=2009, month=1} 2018-01-05T11:28:56,243 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: No new files were created, and is not a replace, or we're inserting into a partition that does not exist yet. Skipping generating INSERT event. 2018-01-05T11:28:56,244 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Adding new partition {year=2009, month=1} 2018-01-05T11:28:56,244 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,244 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: add_partition : db=default tbl=primitives 2018-01-05T11:28:56,244 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=add_partition : db=default tbl=primitives 2018-01-05T11:28:56,251 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,252 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,252 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,252 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=primitives[2009,1] 2018-01-05T11:28:56,252 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=primitives[2009,1] 2018-01-05T11:28:56,255 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,258 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,258 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,259 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-1:STATS] in serial mode 2018-01-05T11:28:56,259 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@2ecbb9fa, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:56,259 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:56,259 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,259 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:56,259 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:56,259 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:56,259 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:56,259 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,259 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,259 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:56,259 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:56,259 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,259 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:56,259 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,259 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:56,259 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:56,259 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,259 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:56,259 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,260 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:56,260 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:56,260 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:56,260 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:56,260 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,260 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,260 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,261 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,261 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,261 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:56,261 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:56,261 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,261 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:56,261 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,261 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=primitives 2018-01-05T11:28:56,261 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=primitives 2018-01-05T11:28:56,263 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,264 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,264 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,264 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=primitives[2009,1] 2018-01-05T11:28:56,264 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=primitives[2009,1] 2018-01-05T11:28:56,266 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,270 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,270 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,270 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Getting file stats of all partitions. threadpool size:15 2018-01-05T11:28:56,271 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Finished getting file stats of all partitions! 2018-01-05T11:28:56,271 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Partition {year=2009, month=1} stats: [numFiles=1, numRows=0, totalSize=1552, rawDataSize=0] 2018-01-05T11:28:56,271 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,271 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_partitions : db=default tbl=primitives 2018-01-05T11:28:56,271 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_partitions : db=default tbl=primitives 2018-01-05T11:28:56,271 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: New partition values:[2009, 1] 2018-01-05T11:28:56,273 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,282 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,285 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,285 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,285 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,285 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,285 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/types/primitives/090101.txt" OVERWRITE INTO TABLE primitives PARTITION(year=2009, month=1) 2018-01-05T11:28:56,285 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: LOAD 2018-01-05T11:28:56,285 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/types/primitives/090101.txt 2018-01-05T11:28:56,285 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@primitives 2018-01-05T11:28:56,285 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@primitives@year=2009/month=1 2018-01-05T11:28:56,285 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,286 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,286 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:56,286 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_partitions_(String, String, List, EnvironmentContext, )=14, getTable_(String, String, )=3, getPartitionWithAuthInfo_(String, String, List, String, List, )=6} 2018-01-05T11:28:56,286 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112856_647eb6b7-38a1-41f7-8b32-d0cd56339984); Time taken: 0.109 seconds 2018-01-05T11:28:56,286 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:56,286 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,286 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,286 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,286 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/types/primitives/090101.txt" OVERWRITE INTO TABLE primitives PARTITION(year=2009, month=1) 2018-01-05T11:28:56,286 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.131 seconds 2018-01-05T11:28:56,286 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,286 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:56,286 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,286 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:56,286 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,286 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:56,286 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,286 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,286 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,286 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/types/primitives/090201.txt" OVERWRITE INTO TABLE primitives PARTITION(year=2009, month=2) 2018-01-05T11:28:56,286 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112856_af1241c7-1ef0-405c-8adc-e0d132a133a4): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/types/primitives/090201.txt" OVERWRITE INTO TABLE primitives PARTITION(year=2009, month=2) 2018-01-05T11:28:56,286 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,286 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/types/primitives/090201.txt" OVERWRITE INTO TABLE primitives PARTITION(year=2009, month=2) 2018-01-05T11:28:56,286 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:56,286 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,286 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,286 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,286 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,286 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,286 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.LoadSemanticAnalyzer: file@null@/Users/djaiswal/hive/hive/data/files/types/primitives/090201.txt 2018-01-05T11:28:56,286 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,286 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=primitives 2018-01-05T11:28:56,287 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=primitives 2018-01-05T11:28:56,289 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,289 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,289 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,290 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=primitives[2009,2] 2018-01-05T11:28:56,290 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=primitives[2009,2] 2018-01-05T11:28:56,292 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,293 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,293 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,295 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,295 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=primitives[2009,2] 2018-01-05T11:28:56,295 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=primitives[2009,2] 2018-01-05T11:28:56,298 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,299 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,299 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,299 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:56,299 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,299 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:56,299 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,299 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:56,299 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=3, flushCache_()=0} 2018-01-05T11:28:56,299 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112856_af1241c7-1ef0-405c-8adc-e0d132a133a4); Time taken: 0.013 seconds 2018-01-05T11:28:56,299 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,299 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,299 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112856_af1241c7-1ef0-405c-8adc-e0d132a133a4): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/types/primitives/090201.txt" OVERWRITE INTO TABLE primitives PARTITION(year=2009, month=2) 2018-01-05T11:28:56,300 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,300 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/types/primitives/090201.txt" OVERWRITE INTO TABLE primitives PARTITION(year=2009, month=2) 2018-01-05T11:28:56,300 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: LOAD 2018-01-05T11:28:56,300 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/types/primitives/090201.txt 2018-01-05T11:28:56,300 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@primitives 2018-01-05T11:28:56,300 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,300 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,300 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,300 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,301 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MOVE] in serial mode 2018-01-05T11:28:56,301 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@14d2b5bf, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:56,301 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:56,301 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,301 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:56,301 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:56,301 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:56,301 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:56,301 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,301 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,301 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:56,301 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:56,301 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,301 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:56,301 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,301 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:56,301 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:56,301 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,301 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Loading data to table default.primitives partition (year=2009, month=2) from file:/Users/djaiswal/hive/hive/data/files/types/primitives/090201.txt 2018-01-05T11:28:56,301 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,301 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:56,301 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:56,302 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:56,302 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:56,302 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,302 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,302 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,302 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,302 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,302 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:56,302 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:56,303 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,303 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:56,303 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,303 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=primitives 2018-01-05T11:28:56,303 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=primitives 2018-01-05T11:28:56,305 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,306 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,306 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,306 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=primitives[2009,2] 2018-01-05T11:28:56,306 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=primitives[2009,2] 2018-01-05T11:28:56,308 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,309 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,309 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,310 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.MoveTask: Partition is: {year=2009, month=2} 2018-01-05T11:28:56,310 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,310 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: partition_name_has_valid_characters 2018-01-05T11:28:56,310 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=partition_name_has_valid_characters 2018-01-05T11:28:56,310 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,310 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,310 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=primitives 2018-01-05T11:28:56,310 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=primitives 2018-01-05T11:28:56,313 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,313 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,313 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,313 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=primitives[2009,2] 2018-01-05T11:28:56,313 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=primitives[2009,2] 2018-01-05T11:28:56,316 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,320 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,320 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,320 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,320 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/primitives/year=2009/month=2 2018-01-05T11:28:56,331 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/data/files/types/primitives/090201.txt/ and the destination path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/primitives/year=2009/month=2/090201.txt/ 2018-01-05T11:28:56,331 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/primitives/year=2009/month=2/090201.txt/ and the destination path is /Users/djaiswal/hive/hive/data/files/types/primitives/090201.txt/ 2018-01-05T11:28:56,354 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,354 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: altering partition for table primitives with partition spec : {year=2009, month=2} 2018-01-05T11:28:56,354 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: No new files were created, and is not a replace, or we're inserting into a partition that does not exist yet. Skipping generating INSERT event. 2018-01-05T11:28:56,354 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Adding new partition {year=2009, month=2} 2018-01-05T11:28:56,354 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,354 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: add_partition : db=default tbl=primitives 2018-01-05T11:28:56,354 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=add_partition : db=default tbl=primitives 2018-01-05T11:28:56,358 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,362 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=primitives[2009,2] 2018-01-05T11:28:56,362 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=primitives[2009,2] 2018-01-05T11:28:56,364 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,367 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,368 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,368 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-1:STATS] in serial mode 2018-01-05T11:28:56,368 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@24d4a409, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:56,368 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:56,368 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,368 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:56,368 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:56,368 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:56,368 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:56,368 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,368 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,368 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:56,368 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:56,368 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,368 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:56,368 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,368 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:56,368 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:56,368 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,368 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:56,369 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,369 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:56,369 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:56,369 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:56,369 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:56,370 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,370 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,370 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,370 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,370 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,370 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:56,370 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:56,371 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,371 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:56,371 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,371 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=primitives 2018-01-05T11:28:56,371 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=primitives 2018-01-05T11:28:56,374 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,374 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,374 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,374 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=primitives[2009,2] 2018-01-05T11:28:56,375 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=primitives[2009,2] 2018-01-05T11:28:56,377 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,380 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,381 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,381 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Getting file stats of all partitions. threadpool size:15 2018-01-05T11:28:56,382 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Finished getting file stats of all partitions! 2018-01-05T11:28:56,382 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Partition {year=2009, month=2} stats: [numFiles=1, numRows=0, totalSize=1539, rawDataSize=0] 2018-01-05T11:28:56,382 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,382 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_partitions : db=default tbl=primitives 2018-01-05T11:28:56,382 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_partitions : db=default tbl=primitives 2018-01-05T11:28:56,382 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: New partition values:[2009, 2] 2018-01-05T11:28:56,385 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,389 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,392 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,392 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,392 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,392 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,392 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/types/primitives/090201.txt" OVERWRITE INTO TABLE primitives PARTITION(year=2009, month=2) 2018-01-05T11:28:56,392 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: LOAD 2018-01-05T11:28:56,392 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/types/primitives/090201.txt 2018-01-05T11:28:56,392 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@primitives 2018-01-05T11:28:56,392 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@primitives@year=2009/month=2 2018-01-05T11:28:56,392 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,392 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,392 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:56,392 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_partitions_(String, String, List, EnvironmentContext, )=10, getTable_(String, String, )=3, getPartitionWithAuthInfo_(String, String, List, String, List, )=7} 2018-01-05T11:28:56,392 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112856_af1241c7-1ef0-405c-8adc-e0d132a133a4); Time taken: 0.093 seconds 2018-01-05T11:28:56,392 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:56,392 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,392 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,392 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,392 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/types/primitives/090201.txt" OVERWRITE INTO TABLE primitives PARTITION(year=2009, month=2) 2018-01-05T11:28:56,392 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.106 seconds 2018-01-05T11:28:56,392 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,392 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:56,392 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,392 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:56,393 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,393 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:56,393 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,393 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,393 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,393 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/types/primitives/090301.txt" OVERWRITE INTO TABLE primitives PARTITION(year=2009, month=3) 2018-01-05T11:28:56,393 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112856_6b5e1045-aab4-423e-a6df-365496e69722): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/types/primitives/090301.txt" OVERWRITE INTO TABLE primitives PARTITION(year=2009, month=3) 2018-01-05T11:28:56,393 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,393 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/types/primitives/090301.txt" OVERWRITE INTO TABLE primitives PARTITION(year=2009, month=3) 2018-01-05T11:28:56,393 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:56,393 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,393 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,393 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,393 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,393 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,393 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.LoadSemanticAnalyzer: file@null@/Users/djaiswal/hive/hive/data/files/types/primitives/090301.txt 2018-01-05T11:28:56,393 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,393 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=primitives 2018-01-05T11:28:56,393 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=primitives 2018-01-05T11:28:56,396 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,396 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,396 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,396 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=primitives[2009,3] 2018-01-05T11:28:56,396 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=primitives[2009,3] 2018-01-05T11:28:56,411 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,412 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,413 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,414 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,414 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=primitives[2009,3] 2018-01-05T11:28:56,414 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=primitives[2009,3] 2018-01-05T11:28:56,416 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,418 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,418 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,418 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:56,418 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,418 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:56,418 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,418 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:56,418 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=3, flushCache_()=0} 2018-01-05T11:28:56,418 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112856_6b5e1045-aab4-423e-a6df-365496e69722); Time taken: 0.025 seconds 2018-01-05T11:28:56,418 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,418 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,418 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112856_6b5e1045-aab4-423e-a6df-365496e69722): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/types/primitives/090301.txt" OVERWRITE INTO TABLE primitives PARTITION(year=2009, month=3) 2018-01-05T11:28:56,419 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,420 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/types/primitives/090301.txt" OVERWRITE INTO TABLE primitives PARTITION(year=2009, month=3) 2018-01-05T11:28:56,420 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: LOAD 2018-01-05T11:28:56,420 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/types/primitives/090301.txt 2018-01-05T11:28:56,420 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@primitives 2018-01-05T11:28:56,420 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,420 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,420 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,420 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,420 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MOVE] in serial mode 2018-01-05T11:28:56,420 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@7510139b, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:56,420 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:56,420 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,420 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:56,420 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:56,420 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:56,420 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:56,420 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,420 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,420 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:56,420 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:56,420 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,420 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:56,420 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,420 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:56,420 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:56,420 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,420 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Loading data to table default.primitives partition (year=2009, month=3) from file:/Users/djaiswal/hive/hive/data/files/types/primitives/090301.txt 2018-01-05T11:28:56,420 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,421 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:56,421 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:56,421 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:56,421 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:56,421 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,421 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,421 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,422 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,422 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,422 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:56,422 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:56,422 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,422 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:56,422 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,422 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=primitives 2018-01-05T11:28:56,422 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=primitives 2018-01-05T11:28:56,425 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,425 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,426 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,426 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=primitives[2009,3] 2018-01-05T11:28:56,426 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=primitives[2009,3] 2018-01-05T11:28:56,428 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,429 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,429 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,430 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.MoveTask: Partition is: {year=2009, month=3} 2018-01-05T11:28:56,430 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,430 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: partition_name_has_valid_characters 2018-01-05T11:28:56,430 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=partition_name_has_valid_characters 2018-01-05T11:28:56,430 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,430 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,430 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=primitives 2018-01-05T11:28:56,430 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=primitives 2018-01-05T11:28:56,433 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,434 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,434 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,434 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=primitives[2009,3] 2018-01-05T11:28:56,434 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=primitives[2009,3] 2018-01-05T11:28:56,436 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,437 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,438 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,438 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,438 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/primitives/year=2009/month=3 2018-01-05T11:28:56,450 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/data/files/types/primitives/090301.txt/ and the destination path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/primitives/year=2009/month=3/090301.txt/ 2018-01-05T11:28:56,450 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/primitives/year=2009/month=3/090301.txt/ and the destination path is /Users/djaiswal/hive/hive/data/files/types/primitives/090301.txt/ 2018-01-05T11:28:56,472 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,472 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: altering partition for table primitives with partition spec : {year=2009, month=3} 2018-01-05T11:28:56,472 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: No new files were created, and is not a replace, or we're inserting into a partition that does not exist yet. Skipping generating INSERT event. 2018-01-05T11:28:56,473 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Adding new partition {year=2009, month=3} 2018-01-05T11:28:56,473 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,473 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: add_partition : db=default tbl=primitives 2018-01-05T11:28:56,473 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=add_partition : db=default tbl=primitives 2018-01-05T11:28:56,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,480 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,480 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,480 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=primitives[2009,3] 2018-01-05T11:28:56,480 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=primitives[2009,3] 2018-01-05T11:28:56,483 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,486 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,486 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,487 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-1:STATS] in serial mode 2018-01-05T11:28:56,487 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@3f4515a9, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:56,487 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:56,487 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,487 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:56,487 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:56,487 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:56,487 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:56,487 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,487 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,487 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:56,487 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:56,487 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,487 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:56,487 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,487 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:56,487 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:56,487 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,487 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:56,487 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,488 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:56,488 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:56,488 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:56,488 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:56,488 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,488 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,488 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,489 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,489 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,489 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:56,489 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:56,489 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,489 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:56,489 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,489 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=primitives 2018-01-05T11:28:56,489 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=primitives 2018-01-05T11:28:56,491 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,492 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,492 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,492 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=primitives[2009,3] 2018-01-05T11:28:56,492 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=primitives[2009,3] 2018-01-05T11:28:56,494 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,497 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Getting file stats of all partitions. threadpool size:15 2018-01-05T11:28:56,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Finished getting file stats of all partitions! 2018-01-05T11:28:56,499 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Partition {year=2009, month=3} stats: [numFiles=1, numRows=0, totalSize=1509, rawDataSize=0] 2018-01-05T11:28:56,499 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,499 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_partitions : db=default tbl=primitives 2018-01-05T11:28:56,499 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_partitions : db=default tbl=primitives 2018-01-05T11:28:56,499 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: New partition values:[2009, 3] 2018-01-05T11:28:56,501 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,506 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,508 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,509 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/types/primitives/090301.txt" OVERWRITE INTO TABLE primitives PARTITION(year=2009, month=3) 2018-01-05T11:28:56,509 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: LOAD 2018-01-05T11:28:56,509 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/types/primitives/090301.txt 2018-01-05T11:28:56,509 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@primitives 2018-01-05T11:28:56,509 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@primitives@year=2009/month=3 2018-01-05T11:28:56,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,509 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:56,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_partitions_(String, String, List, EnvironmentContext, )=10, getTable_(String, String, )=3, getPartitionWithAuthInfo_(String, String, List, String, List, )=6} 2018-01-05T11:28:56,509 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112856_6b5e1045-aab4-423e-a6df-365496e69722); Time taken: 0.091 seconds 2018-01-05T11:28:56,509 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:56,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,509 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/types/primitives/090301.txt" OVERWRITE INTO TABLE primitives PARTITION(year=2009, month=3) 2018-01-05T11:28:56,509 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.116 seconds 2018-01-05T11:28:56,509 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,509 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:56,509 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,509 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:56,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:56,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/types/primitives/090401.txt" OVERWRITE INTO TABLE primitives PARTITION(year=2009, month=4) 2018-01-05T11:28:56,509 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112856_c5cc633c-920e-4c9d-a121-c797ea155329): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/types/primitives/090401.txt" OVERWRITE INTO TABLE primitives PARTITION(year=2009, month=4) 2018-01-05T11:28:56,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/types/primitives/090401.txt" OVERWRITE INTO TABLE primitives PARTITION(year=2009, month=4) 2018-01-05T11:28:56,510 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:56,510 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,510 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,510 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,510 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,510 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,510 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.LoadSemanticAnalyzer: file@null@/Users/djaiswal/hive/hive/data/files/types/primitives/090401.txt 2018-01-05T11:28:56,510 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,510 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=primitives 2018-01-05T11:28:56,510 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=primitives 2018-01-05T11:28:56,512 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,513 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,513 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,513 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=primitives[2009,4] 2018-01-05T11:28:56,513 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=primitives[2009,4] 2018-01-05T11:28:56,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,516 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,518 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,518 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=primitives[2009,4] 2018-01-05T11:28:56,518 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=primitives[2009,4] 2018-01-05T11:28:56,521 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,522 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,523 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,523 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:56,523 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,523 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:56,523 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,523 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:56,523 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=3, flushCache_()=0} 2018-01-05T11:28:56,523 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112856_c5cc633c-920e-4c9d-a121-c797ea155329); Time taken: 0.014 seconds 2018-01-05T11:28:56,523 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,523 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,523 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112856_c5cc633c-920e-4c9d-a121-c797ea155329): LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/types/primitives/090401.txt" OVERWRITE INTO TABLE primitives PARTITION(year=2009, month=4) 2018-01-05T11:28:56,524 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,524 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/types/primitives/090401.txt" OVERWRITE INTO TABLE primitives PARTITION(year=2009, month=4) 2018-01-05T11:28:56,524 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: LOAD 2018-01-05T11:28:56,524 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/types/primitives/090401.txt 2018-01-05T11:28:56,524 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@primitives 2018-01-05T11:28:56,524 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,524 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,524 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,524 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,524 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MOVE] in serial mode 2018-01-05T11:28:56,524 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@bcb4c2e, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:56,524 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:56,524 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,524 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:56,524 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:56,524 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:56,524 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:56,524 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,524 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,524 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:56,524 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:56,524 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,524 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:56,524 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,524 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:56,524 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:56,524 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,524 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Loading data to table default.primitives partition (year=2009, month=4) from file:/Users/djaiswal/hive/hive/data/files/types/primitives/090401.txt 2018-01-05T11:28:56,525 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,525 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:56,525 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:56,525 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:56,525 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:56,525 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,525 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,525 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,526 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,526 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,526 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:56,526 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:56,526 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,526 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:56,526 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,526 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=primitives 2018-01-05T11:28:56,526 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=primitives 2018-01-05T11:28:56,528 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,529 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,529 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,529 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=primitives[2009,4] 2018-01-05T11:28:56,529 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=primitives[2009,4] 2018-01-05T11:28:56,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,532 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,533 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,533 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.MoveTask: Partition is: {year=2009, month=4} 2018-01-05T11:28:56,533 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,533 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: partition_name_has_valid_characters 2018-01-05T11:28:56,533 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=partition_name_has_valid_characters 2018-01-05T11:28:56,533 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,533 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,533 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=primitives 2018-01-05T11:28:56,533 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=primitives 2018-01-05T11:28:56,536 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,536 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,536 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,536 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=primitives[2009,4] 2018-01-05T11:28:56,536 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=primitives[2009,4] 2018-01-05T11:28:56,539 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,540 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/primitives/year=2009/month=4 2018-01-05T11:28:56,551 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/data/files/types/primitives/090401.txt/ and the destination path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/primitives/year=2009/month=4/090401.txt/ 2018-01-05T11:28:56,551 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/primitives/year=2009/month=4/090401.txt/ and the destination path is /Users/djaiswal/hive/hive/data/files/types/primitives/090401.txt/ 2018-01-05T11:28:56,575 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,575 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: altering partition for table primitives with partition spec : {year=2009, month=4} 2018-01-05T11:28:56,575 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: No new files were created, and is not a replace, or we're inserting into a partition that does not exist yet. Skipping generating INSERT event. 2018-01-05T11:28:56,576 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Adding new partition {year=2009, month=4} 2018-01-05T11:28:56,576 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,576 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: add_partition : db=default tbl=primitives 2018-01-05T11:28:56,576 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=add_partition : db=default tbl=primitives 2018-01-05T11:28:56,580 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,581 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,585 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,586 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,586 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,586 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=primitives[2009,4] 2018-01-05T11:28:56,586 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=primitives[2009,4] 2018-01-05T11:28:56,589 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,598 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,598 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-1:STATS] in serial mode 2018-01-05T11:28:56,598 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@17675c2f, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:56,598 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:56,598 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,598 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:56,598 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:56,598 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:56,598 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:56,598 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,598 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,598 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:56,598 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:56,598 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,598 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:56,598 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,598 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:56,598 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:56,598 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,598 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:56,599 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,599 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:56,599 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:56,599 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:56,599 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:56,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,601 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,601 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,601 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:56,601 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:56,601 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,601 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:56,601 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,601 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=primitives 2018-01-05T11:28:56,601 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=primitives 2018-01-05T11:28:56,604 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,605 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,605 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,605 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=primitives[2009,4] 2018-01-05T11:28:56,605 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=primitives[2009,4] 2018-01-05T11:28:56,607 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,611 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,611 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,611 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Getting file stats of all partitions. threadpool size:15 2018-01-05T11:28:56,612 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Finished getting file stats of all partitions! 2018-01-05T11:28:56,612 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Partition {year=2009, month=4} stats: [numFiles=1, numRows=0, totalSize=1562, rawDataSize=0] 2018-01-05T11:28:56,612 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,612 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_partitions : db=default tbl=primitives 2018-01-05T11:28:56,612 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_partitions : db=default tbl=primitives 2018-01-05T11:28:56,612 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: New partition values:[2009, 4] 2018-01-05T11:28:56,615 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,620 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,623 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,624 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,624 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,624 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,624 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/types/primitives/090401.txt" OVERWRITE INTO TABLE primitives PARTITION(year=2009, month=4) 2018-01-05T11:28:56,624 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: LOAD 2018-01-05T11:28:56,624 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/types/primitives/090401.txt 2018-01-05T11:28:56,624 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@primitives 2018-01-05T11:28:56,624 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@primitives@year=2009/month=4 2018-01-05T11:28:56,624 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,624 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,624 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:56,624 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_partitions_(String, String, List, EnvironmentContext, )=12, getTable_(String, String, )=4, getPartitionWithAuthInfo_(String, String, List, String, List, )=6} 2018-01-05T11:28:56,624 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112856_c5cc633c-920e-4c9d-a121-c797ea155329); Time taken: 0.101 seconds 2018-01-05T11:28:56,624 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:56,624 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,624 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,624 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,624 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query LOAD DATA LOCAL INPATH "/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/types/primitives/090401.txt" OVERWRITE INTO TABLE primitives PARTITION(year=2009, month=4) 2018-01-05T11:28:56,624 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.115 seconds 2018-01-05T11:28:56,624 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,624 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:56,624 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,624 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:56,624 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,624 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:56,624 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,624 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,624 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,624 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: -- -- Function qtest_get_java_boolean -- DROP FUNCTION IF EXISTS qtest_get_java_boolean 2018-01-05T11:28:56,624 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112856_35e4dba7-db02-4299-a308-1ad9e18b1dec): -- -- Function qtest_get_java_boolean -- DROP FUNCTION IF EXISTS qtest_get_java_boolean 2018-01-05T11:28:56,625 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,625 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: -- -- Function qtest_get_java_boolean -- DROP FUNCTION IF EXISTS qtest_get_java_boolean 2018-01-05T11:28:56,625 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:56,625 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,625 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,626 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,626 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,626 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,626 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.FunctionSemanticAnalyzer: analyze done 2018-01-05T11:28:56,626 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:56,626 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,626 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:56,627 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,627 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:56,627 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:28:56,627 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112856_35e4dba7-db02-4299-a308-1ad9e18b1dec); Time taken: 0.003 seconds 2018-01-05T11:28:56,627 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,627 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,627 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112856_35e4dba7-db02-4299-a308-1ad9e18b1dec): -- -- Function qtest_get_java_boolean -- DROP FUNCTION IF EXISTS qtest_get_java_boolean 2018-01-05T11:28:56,628 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,628 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: -- -- Function qtest_get_java_boolean -- DROP FUNCTION IF EXISTS qtest_get_java_boolean 2018-01-05T11:28:56,628 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPFUNCTION 2018-01-05T11:28:56,628 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,628 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,628 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,628 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,628 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,628 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,628 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: -- -- Function qtest_get_java_boolean -- DROP FUNCTION IF EXISTS qtest_get_java_boolean 2018-01-05T11:28:56,628 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPFUNCTION 2018-01-05T11:28:56,628 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,628 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,628 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:56,628 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {} 2018-01-05T11:28:56,628 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112856_35e4dba7-db02-4299-a308-1ad9e18b1dec); Time taken: 0.001 seconds 2018-01-05T11:28:56,628 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:56,628 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,628 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,628 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,628 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query -- -- Function qtest_get_java_boolean -- DROP FUNCTION IF EXISTS qtest_get_java_boolean 2018-01-05T11:28:56,628 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.004 seconds 2018-01-05T11:28:56,628 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,628 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:56,628 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,628 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:56,628 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,628 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:56,628 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,628 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,628 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,628 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: CREATE FUNCTION qtest_get_java_boolean AS 'org.apache.hadoop.hive.ql.udf.generic.GenericUDFTestGetJavaBoolean' 2018-01-05T11:28:56,628 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112856_3f6e37be-aa7b-495c-ad67-84faf7426672): CREATE FUNCTION qtest_get_java_boolean AS 'org.apache.hadoop.hive.ql.udf.generic.GenericUDFTestGetJavaBoolean' 2018-01-05T11:28:56,628 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,628 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: CREATE FUNCTION qtest_get_java_boolean AS 'org.apache.hadoop.hive.ql.udf.generic.GenericUDFTestGetJavaBoolean' 2018-01-05T11:28:56,629 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:56,629 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,629 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,629 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,629 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,629 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,629 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: permanent functions created without USING clause will not be replicated. 2018-01-05T11:28:56,631 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,631 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_database: default 2018-01-05T11:28:56,631 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_database: default 2018-01-05T11:28:56,631 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:56,631 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,631 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,631 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.FunctionSemanticAnalyzer: analyze done 2018-01-05T11:28:56,631 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:56,631 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,631 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:56,631 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,631 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:56,631 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getDatabase_(String, )=0, flushCache_()=0} 2018-01-05T11:28:56,631 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112856_3f6e37be-aa7b-495c-ad67-84faf7426672); Time taken: 0.003 seconds 2018-01-05T11:28:56,631 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,631 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,631 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112856_3f6e37be-aa7b-495c-ad67-84faf7426672): CREATE FUNCTION qtest_get_java_boolean AS 'org.apache.hadoop.hive.ql.udf.generic.GenericUDFTestGetJavaBoolean' 2018-01-05T11:28:56,632 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,632 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: CREATE FUNCTION qtest_get_java_boolean AS 'org.apache.hadoop.hive.ql.udf.generic.GenericUDFTestGetJavaBoolean' 2018-01-05T11:28:56,632 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: CREATEFUNCTION 2018-01-05T11:28:56,632 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: database:default 2018-01-05T11:28:56,632 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default.qtest_get_java_boolean 2018-01-05T11:28:56,632 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,632 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,633 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,633 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,633 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:FUNC] in serial mode 2018-01-05T11:28:56,634 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,635 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:56,642 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,647 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,647 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,647 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,647 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,647 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: CREATE FUNCTION qtest_get_java_boolean AS 'org.apache.hadoop.hive.ql.udf.generic.GenericUDFTestGetJavaBoolean' 2018-01-05T11:28:56,647 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: CREATEFUNCTION 2018-01-05T11:28:56,647 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: database:default 2018-01-05T11:28:56,647 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default.qtest_get_java_boolean 2018-01-05T11:28:56,647 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,647 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,647 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:56,647 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, createFunction_(Function, )=13} 2018-01-05T11:28:56,647 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112856_3f6e37be-aa7b-495c-ad67-84faf7426672); Time taken: 0.016 seconds 2018-01-05T11:28:56,647 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:56,647 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,647 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,647 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,647 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query CREATE FUNCTION qtest_get_java_boolean AS 'org.apache.hadoop.hive.ql.udf.generic.GenericUDFTestGetJavaBoolean' 2018-01-05T11:28:56,647 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.019 seconds 2018-01-05T11:28:56,647 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,647 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:56,648 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,648 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:56,648 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,648 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:56,648 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,648 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,648 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,648 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: -- -- Table dest1 -- DROP TABLE IF EXISTS dest1 2018-01-05T11:28:56,648 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112856_de5e24a0-4d71-4483-a085-15ac365e0af4): -- -- Table dest1 -- DROP TABLE IF EXISTS dest1 2018-01-05T11:28:56,648 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,648 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: -- -- Table dest1 -- DROP TABLE IF EXISTS dest1 2018-01-05T11:28:56,648 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:56,648 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,648 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,648 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,648 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,648 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,648 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,648 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest1 2018-01-05T11:28:56,648 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest1 2018-01-05T11:28:56,648 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,648 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,648 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:56,648 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,648 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:56,649 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,649 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:56,649 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:28:56,649 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112856_de5e24a0-4d71-4483-a085-15ac365e0af4); Time taken: 0.001 seconds 2018-01-05T11:28:56,649 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,649 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,649 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112856_de5e24a0-4d71-4483-a085-15ac365e0af4): -- -- Table dest1 -- DROP TABLE IF EXISTS dest1 2018-01-05T11:28:56,649 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,649 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: -- -- Table dest1 -- DROP TABLE IF EXISTS dest1 2018-01-05T11:28:56,649 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:56,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,650 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:56,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,650 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest1 2018-01-05T11:28:56,650 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest1 2018-01-05T11:28:56,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,650 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table dest1 not found: default.dest1 table not found 2018-01-05T11:28:56,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,650 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest1 2018-01-05T11:28:56,650 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest1 2018-01-05T11:28:56,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,650 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: -- -- Table dest1 -- DROP TABLE IF EXISTS dest1 2018-01-05T11:28:56,650 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:56,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,650 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:56,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=0} 2018-01-05T11:28:56,650 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112856_de5e24a0-4d71-4483-a085-15ac365e0af4); Time taken: 0.001 seconds 2018-01-05T11:28:56,650 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:56,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,650 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query -- -- Table dest1 -- DROP TABLE IF EXISTS dest1 2018-01-05T11:28:56,650 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.002 seconds 2018-01-05T11:28:56,651 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,651 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:56,651 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,651 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:56,651 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,651 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:56,651 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,651 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,651 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,651 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: CREATE TABLE dest1 (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' 2018-01-05T11:28:56,651 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112856_63bb871c-79ce-4cf9-97bc-78317b08120a): CREATE TABLE dest1 (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' 2018-01-05T11:28:56,651 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,651 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: CREATE TABLE dest1 (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' 2018-01-05T11:28:56,651 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:56,651 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,651 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,652 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,652 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,652 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,652 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:28:56,652 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Creating table default.dest1 position=13 2018-01-05T11:28:56,652 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,652 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_database: default 2018-01-05T11:28:56,652 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_database: default 2018-01-05T11:28:56,652 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:56,652 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,652 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,652 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:56,652 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: validation start 2018-01-05T11:28:56,652 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:28:56,652 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Not a partition. 2018-01-05T11:28:56,652 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,652 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:56,652 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,652 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:56,652 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getDatabase_(String, )=0, flushCache_()=0} 2018-01-05T11:28:56,652 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112856_63bb871c-79ce-4cf9-97bc-78317b08120a); Time taken: 0.001 seconds 2018-01-05T11:28:56,652 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,652 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,652 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112856_63bb871c-79ce-4cf9-97bc-78317b08120a): CREATE TABLE dest1 (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' 2018-01-05T11:28:56,653 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,653 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: CREATE TABLE dest1 (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' 2018-01-05T11:28:56,653 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: CREATETABLE 2018-01-05T11:28:56,653 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: database:default 2018-01-05T11:28:56,653 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@dest1 2018-01-05T11:28:56,653 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,653 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,653 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,653 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,654 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:56,654 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] plan.CreateTableDesc: Default to org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe for table dest1 2018-01-05T11:28:56,654 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: creating table default.dest1 on file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/dest1 2018-01-05T11:28:56,654 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,654 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: create_table: Table(tableName:dest1, dbName:default, owner:hive_test_user, createTime:1515180536, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:string, comment:default), FieldSchema(name:value, type:string, comment:default)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/dest1, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, numFiles=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:56,654 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=create_table: Table(tableName:dest1, dbName:default, owner:hive_test_user, createTime:1515180536, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:string, comment:default), FieldSchema(name:value, type:string, comment:default)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/dest1, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, numFiles=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:56,655 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hive-site.xml 2018-01-05T11:28:56,655 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hivemetastore-site.xml 2018-01-05T11:28:56,656 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Unable to find config file metastore-site.xml 2018-01-05T11:28:56,656 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file null 2018-01-05T11:28:56,665 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:56,665 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: Location: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/dest1 specified for non-external table:dest1 2018-01-05T11:28:56,665 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/dest1 2018-01-05T11:28:56,679 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,680 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,681 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,681 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@dest1) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@dest1) Type=TABLE WriteType=DDL_NO_LOCK is present 2018-01-05T11:28:56,681 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,681 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,681 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: CREATE TABLE dest1 (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' 2018-01-05T11:28:56,681 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: CREATETABLE 2018-01-05T11:28:56,681 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: database:default 2018-01-05T11:28:56,681 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@dest1 2018-01-05T11:28:56,681 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,681 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,681 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:56,681 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {createTable_(Table, )=27, isCompatibleWith_(Configuration, )=0} 2018-01-05T11:28:56,681 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112856_63bb871c-79ce-4cf9-97bc-78317b08120a); Time taken: 0.029 seconds 2018-01-05T11:28:56,681 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:56,681 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,681 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,681 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,681 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query CREATE TABLE dest1 (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' 2018-01-05T11:28:56,681 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.03 seconds 2018-01-05T11:28:56,681 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,681 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:56,681 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,681 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:56,681 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,681 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:56,681 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,681 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,681 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,681 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: -- -- Table dest2 -- DROP TABLE IF EXISTS dest2 2018-01-05T11:28:56,681 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112856_0751b728-8719-462b-9954-e55e4b61a794): -- -- Table dest2 -- DROP TABLE IF EXISTS dest2 2018-01-05T11:28:56,681 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,681 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: -- -- Table dest2 -- DROP TABLE IF EXISTS dest2 2018-01-05T11:28:56,682 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:56,682 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,682 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,682 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,682 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,682 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,682 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,682 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest2 2018-01-05T11:28:56,682 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest2 2018-01-05T11:28:56,682 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,682 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,682 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:56,682 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,682 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:56,682 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,682 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:56,682 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:28:56,682 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112856_0751b728-8719-462b-9954-e55e4b61a794); Time taken: 0.001 seconds 2018-01-05T11:28:56,682 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,682 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,682 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112856_0751b728-8719-462b-9954-e55e4b61a794): -- -- Table dest2 -- DROP TABLE IF EXISTS dest2 2018-01-05T11:28:56,683 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,683 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: -- -- Table dest2 -- DROP TABLE IF EXISTS dest2 2018-01-05T11:28:56,683 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:56,683 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,683 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,683 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,683 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,683 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:56,683 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,683 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest2 2018-01-05T11:28:56,683 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest2 2018-01-05T11:28:56,684 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,684 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,684 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table dest2 not found: default.dest2 table not found 2018-01-05T11:28:56,684 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,684 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest2 2018-01-05T11:28:56,684 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest2 2018-01-05T11:28:56,684 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,684 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,684 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,684 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,684 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: -- -- Table dest2 -- DROP TABLE IF EXISTS dest2 2018-01-05T11:28:56,684 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:56,684 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,684 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,684 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:56,684 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=0} 2018-01-05T11:28:56,684 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112856_0751b728-8719-462b-9954-e55e4b61a794); Time taken: 0.002 seconds 2018-01-05T11:28:56,684 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:56,684 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,684 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,684 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,684 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query -- -- Table dest2 -- DROP TABLE IF EXISTS dest2 2018-01-05T11:28:56,684 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.003 seconds 2018-01-05T11:28:56,684 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,684 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:56,684 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,684 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:56,684 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,684 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:56,684 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,684 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,684 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,684 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: CREATE TABLE dest2 (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' 2018-01-05T11:28:56,684 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112856_647b956e-be2c-4011-ae60-04503b4c2975): CREATE TABLE dest2 (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' 2018-01-05T11:28:56,684 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,685 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: CREATE TABLE dest2 (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' 2018-01-05T11:28:56,685 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:56,685 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,685 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,685 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,685 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,685 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,685 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:28:56,685 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Creating table default.dest2 position=13 2018-01-05T11:28:56,685 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,685 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_database: default 2018-01-05T11:28:56,685 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_database: default 2018-01-05T11:28:56,686 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:56,686 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,686 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,686 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:56,686 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: validation start 2018-01-05T11:28:56,686 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:28:56,686 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Not a partition. 2018-01-05T11:28:56,686 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,686 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:56,686 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,686 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:56,686 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getDatabase_(String, )=1, flushCache_()=0} 2018-01-05T11:28:56,686 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112856_647b956e-be2c-4011-ae60-04503b4c2975); Time taken: 0.002 seconds 2018-01-05T11:28:56,686 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,686 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,686 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112856_647b956e-be2c-4011-ae60-04503b4c2975): CREATE TABLE dest2 (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' 2018-01-05T11:28:56,687 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,687 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: CREATE TABLE dest2 (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' 2018-01-05T11:28:56,687 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: CREATETABLE 2018-01-05T11:28:56,687 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: database:default 2018-01-05T11:28:56,687 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@dest2 2018-01-05T11:28:56,687 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,687 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,687 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,687 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,687 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:56,687 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] plan.CreateTableDesc: Default to org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe for table dest2 2018-01-05T11:28:56,687 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: creating table default.dest2 on file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/dest2 2018-01-05T11:28:56,688 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,688 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: create_table: Table(tableName:dest2, dbName:default, owner:hive_test_user, createTime:1515180536, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:string, comment:default), FieldSchema(name:value, type:string, comment:default)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/dest2, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, numFiles=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:56,688 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=create_table: Table(tableName:dest2, dbName:default, owner:hive_test_user, createTime:1515180536, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:string, comment:default), FieldSchema(name:value, type:string, comment:default)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/dest2, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, numFiles=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:56,688 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hive-site.xml 2018-01-05T11:28:56,688 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hivemetastore-site.xml 2018-01-05T11:28:56,689 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Unable to find config file metastore-site.xml 2018-01-05T11:28:56,689 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file null 2018-01-05T11:28:56,698 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:56,698 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: Location: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/dest2 specified for non-external table:dest2 2018-01-05T11:28:56,698 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/dest2 2018-01-05T11:28:56,713 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,713 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,713 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@dest2) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@dest2) Type=TABLE WriteType=DDL_NO_LOCK is present 2018-01-05T11:28:56,713 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,713 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,713 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: CREATE TABLE dest2 (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' 2018-01-05T11:28:56,713 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: CREATETABLE 2018-01-05T11:28:56,713 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: database:default 2018-01-05T11:28:56,713 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@dest2 2018-01-05T11:28:56,713 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,713 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,713 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:56,713 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {createTable_(Table, )=26, isCompatibleWith_(Configuration, )=0} 2018-01-05T11:28:56,713 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112856_647b956e-be2c-4011-ae60-04503b4c2975); Time taken: 0.027 seconds 2018-01-05T11:28:56,713 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:56,713 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,713 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,713 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,713 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query CREATE TABLE dest2 (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' 2018-01-05T11:28:56,713 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.029 seconds 2018-01-05T11:28:56,713 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,713 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:56,713 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,713 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:56,714 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,714 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:56,714 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,714 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,714 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,714 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: -- -- Table dest3 -- DROP TABLE IF EXISTS dest3 2018-01-05T11:28:56,714 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112856_01a2f72e-125a-4a0b-bb4c-9013be6498db): -- -- Table dest3 -- DROP TABLE IF EXISTS dest3 2018-01-05T11:28:56,714 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,714 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: -- -- Table dest3 -- DROP TABLE IF EXISTS dest3 2018-01-05T11:28:56,714 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:56,714 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,714 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,714 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,714 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,714 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,714 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,714 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest3 2018-01-05T11:28:56,714 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest3 2018-01-05T11:28:56,714 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,714 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,714 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:56,714 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,714 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:56,714 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,714 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:56,714 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:28:56,714 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112856_01a2f72e-125a-4a0b-bb4c-9013be6498db); Time taken: 0.0 seconds 2018-01-05T11:28:56,714 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,714 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,714 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112856_01a2f72e-125a-4a0b-bb4c-9013be6498db): -- -- Table dest3 -- DROP TABLE IF EXISTS dest3 2018-01-05T11:28:56,715 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,715 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: -- -- Table dest3 -- DROP TABLE IF EXISTS dest3 2018-01-05T11:28:56,715 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:56,715 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,715 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,715 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,715 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,715 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:56,715 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,715 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest3 2018-01-05T11:28:56,716 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest3 2018-01-05T11:28:56,716 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,716 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,716 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table dest3 not found: default.dest3 table not found 2018-01-05T11:28:56,716 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,716 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest3 2018-01-05T11:28:56,716 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest3 2018-01-05T11:28:56,716 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,716 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,716 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,716 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,716 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: -- -- Table dest3 -- DROP TABLE IF EXISTS dest3 2018-01-05T11:28:56,716 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:56,716 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,716 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,716 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:56,716 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=0} 2018-01-05T11:28:56,716 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112856_01a2f72e-125a-4a0b-bb4c-9013be6498db); Time taken: 0.002 seconds 2018-01-05T11:28:56,716 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:56,716 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,716 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,716 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,716 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query -- -- Table dest3 -- DROP TABLE IF EXISTS dest3 2018-01-05T11:28:56,716 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.002 seconds 2018-01-05T11:28:56,716 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,717 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:56,717 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,717 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:56,717 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,717 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:56,717 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,717 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,717 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,717 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: CREATE TABLE dest3 (key STRING COMMENT 'default', value STRING COMMENT 'default') PARTITIONED BY (ds STRING, hr STRING) STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' 2018-01-05T11:28:56,717 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112856_cf045a3e-f035-4548-bd53-ff45930246c2): CREATE TABLE dest3 (key STRING COMMENT 'default', value STRING COMMENT 'default') PARTITIONED BY (ds STRING, hr STRING) STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' 2018-01-05T11:28:56,717 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,717 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: CREATE TABLE dest3 (key STRING COMMENT 'default', value STRING COMMENT 'default') PARTITIONED BY (ds STRING, hr STRING) STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' 2018-01-05T11:28:56,717 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:56,717 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,717 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,717 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,717 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,717 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,717 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:28:56,717 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Creating table default.dest3 position=13 2018-01-05T11:28:56,718 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,718 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_database: default 2018-01-05T11:28:56,718 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_database: default 2018-01-05T11:28:56,718 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:56,718 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,718 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,718 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:56,718 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: validation start 2018-01-05T11:28:56,718 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:28:56,718 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Not a partition. 2018-01-05T11:28:56,718 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,718 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:56,718 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,718 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:56,718 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getDatabase_(String, )=0, flushCache_()=0} 2018-01-05T11:28:56,718 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112856_cf045a3e-f035-4548-bd53-ff45930246c2); Time taken: 0.001 seconds 2018-01-05T11:28:56,718 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,718 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,718 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112856_cf045a3e-f035-4548-bd53-ff45930246c2): CREATE TABLE dest3 (key STRING COMMENT 'default', value STRING COMMENT 'default') PARTITIONED BY (ds STRING, hr STRING) STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' 2018-01-05T11:28:56,719 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,719 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: CREATE TABLE dest3 (key STRING COMMENT 'default', value STRING COMMENT 'default') PARTITIONED BY (ds STRING, hr STRING) STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' 2018-01-05T11:28:56,719 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: CREATETABLE 2018-01-05T11:28:56,719 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: database:default 2018-01-05T11:28:56,719 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@dest3 2018-01-05T11:28:56,719 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,719 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,719 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,719 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,719 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:56,719 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] plan.CreateTableDesc: Default to org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe for table dest3 2018-01-05T11:28:56,720 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: creating table default.dest3 on file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/dest3 2018-01-05T11:28:56,720 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,720 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: create_table: Table(tableName:dest3, dbName:default, owner:hive_test_user, createTime:1515180536, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:string, comment:default), FieldSchema(name:value, type:string, comment:default)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/dest3, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[FieldSchema(name:ds, type:string, comment:null), FieldSchema(name:hr, type:string, comment:null)], parameters:{}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:56,720 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=create_table: Table(tableName:dest3, dbName:default, owner:hive_test_user, createTime:1515180536, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:string, comment:default), FieldSchema(name:value, type:string, comment:default)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/dest3, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[FieldSchema(name:ds, type:string, comment:null), FieldSchema(name:hr, type:string, comment:null)], parameters:{}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:56,721 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hive-site.xml 2018-01-05T11:28:56,721 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hivemetastore-site.xml 2018-01-05T11:28:56,721 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Unable to find config file metastore-site.xml 2018-01-05T11:28:56,721 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file null 2018-01-05T11:28:56,728 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:56,729 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: Location: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/dest3 specified for non-external table:dest3 2018-01-05T11:28:56,729 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/dest3 2018-01-05T11:28:56,740 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,741 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,742 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,744 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,745 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,745 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@dest3) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@dest3) Type=TABLE WriteType=DDL_NO_LOCK is present 2018-01-05T11:28:56,745 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,745 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,745 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: CREATE TABLE dest3 (key STRING COMMENT 'default', value STRING COMMENT 'default') PARTITIONED BY (ds STRING, hr STRING) STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' 2018-01-05T11:28:56,745 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: CREATETABLE 2018-01-05T11:28:56,745 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: database:default 2018-01-05T11:28:56,745 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@dest3 2018-01-05T11:28:56,745 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,745 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,745 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:56,745 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {createTable_(Table, )=25, isCompatibleWith_(Configuration, )=0} 2018-01-05T11:28:56,745 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112856_cf045a3e-f035-4548-bd53-ff45930246c2); Time taken: 0.027 seconds 2018-01-05T11:28:56,745 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:56,745 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,745 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,745 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,745 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query CREATE TABLE dest3 (key STRING COMMENT 'default', value STRING COMMENT 'default') PARTITIONED BY (ds STRING, hr STRING) STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' 2018-01-05T11:28:56,745 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.028 seconds 2018-01-05T11:28:56,745 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,745 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:56,745 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,745 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:56,745 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,745 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:56,745 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,745 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,745 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,745 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: ALTER TABLE dest3 ADD PARTITION (ds='2008-04-08',hr='12') 2018-01-05T11:28:56,745 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112856_1aa93730-79b0-4acd-95fe-6b4d9d900425): ALTER TABLE dest3 ADD PARTITION (ds='2008-04-08',hr='12') 2018-01-05T11:28:56,745 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,745 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: ALTER TABLE dest3 ADD PARTITION (ds='2008-04-08',hr='12') 2018-01-05T11:28:56,746 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:56,746 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,746 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,746 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,746 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,746 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,746 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,746 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest3 2018-01-05T11:28:56,746 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest3 2018-01-05T11:28:56,749 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,749 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,751 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:56,751 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,751 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:56,751 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,751 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:56,751 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=3, flushCache_()=0} 2018-01-05T11:28:56,751 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112856_1aa93730-79b0-4acd-95fe-6b4d9d900425); Time taken: 0.006 seconds 2018-01-05T11:28:56,751 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,751 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,751 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112856_1aa93730-79b0-4acd-95fe-6b4d9d900425): ALTER TABLE dest3 ADD PARTITION (ds='2008-04-08',hr='12') 2018-01-05T11:28:56,752 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,752 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: ALTER TABLE dest3 ADD PARTITION (ds='2008-04-08',hr='12') 2018-01-05T11:28:56,752 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: ALTERTABLE_ADDPARTS 2018-01-05T11:28:56,752 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@dest3 2018-01-05T11:28:56,752 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,752 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,752 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,752 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,752 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:56,752 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,752 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest3 2018-01-05T11:28:56,752 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest3 2018-01-05T11:28:56,755 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,755 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,755 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,757 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: add_partitions 2018-01-05T11:28:56,757 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=add_partitions 2018-01-05T11:28:56,763 INFO [HMSHandler #0] utils.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/dest3/ds=2008-04-08/hr=12 2018-01-05T11:28:56,788 DEBUG [AMReporterQueueDrainer] impl.AMReporter: Removing am 10.22.8.182:49824 with last associated dag QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=10} from heartbeat with taskCount=0, amFailed=false, isDone=true 2018-01-05T11:28:56,790 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,791 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,791 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,791 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,791 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: ALTER TABLE dest3 ADD PARTITION (ds='2008-04-08',hr='12') 2018-01-05T11:28:56,791 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: ALTERTABLE_ADDPARTS 2018-01-05T11:28:56,791 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@dest3 2018-01-05T11:28:56,791 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@dest3@ds=2008-04-08/hr=12 2018-01-05T11:28:56,791 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,791 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,791 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:56,791 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=3, add_partitions_(List, boolean, boolean, )=36} 2018-01-05T11:28:56,791 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112856_1aa93730-79b0-4acd-95fe-6b4d9d900425); Time taken: 0.04 seconds 2018-01-05T11:28:56,791 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:56,791 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,791 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,791 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,791 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query ALTER TABLE dest3 ADD PARTITION (ds='2008-04-08',hr='12') 2018-01-05T11:28:56,791 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.046 seconds 2018-01-05T11:28:56,791 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,791 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:56,791 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,791 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:56,791 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,791 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:56,791 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,791 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,791 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,791 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: -- -- Table dest4 -- DROP TABLE IF EXISTS dest4 2018-01-05T11:28:56,791 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112856_814a4461-5575-48d9-9d5b-01063bfc698e): -- -- Table dest4 -- DROP TABLE IF EXISTS dest4 2018-01-05T11:28:56,791 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,791 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: -- -- Table dest4 -- DROP TABLE IF EXISTS dest4 2018-01-05T11:28:56,792 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:56,792 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,792 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,792 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,792 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,792 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,792 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,792 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest4 2018-01-05T11:28:56,792 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest4 2018-01-05T11:28:56,792 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,792 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,792 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:56,792 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,792 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:56,792 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,792 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:56,792 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:28:56,792 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112856_814a4461-5575-48d9-9d5b-01063bfc698e); Time taken: 0.001 seconds 2018-01-05T11:28:56,792 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,792 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,792 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112856_814a4461-5575-48d9-9d5b-01063bfc698e): -- -- Table dest4 -- DROP TABLE IF EXISTS dest4 2018-01-05T11:28:56,793 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,793 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: -- -- Table dest4 -- DROP TABLE IF EXISTS dest4 2018-01-05T11:28:56,793 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:56,793 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,793 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,793 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,793 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,793 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:56,794 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,794 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest4 2018-01-05T11:28:56,794 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest4 2018-01-05T11:28:56,794 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,794 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,794 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table dest4 not found: default.dest4 table not found 2018-01-05T11:28:56,794 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,794 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest4 2018-01-05T11:28:56,794 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest4 2018-01-05T11:28:56,794 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,794 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,794 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,794 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,794 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: -- -- Table dest4 -- DROP TABLE IF EXISTS dest4 2018-01-05T11:28:56,794 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:56,794 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,794 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,794 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:56,794 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=0} 2018-01-05T11:28:56,794 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112856_814a4461-5575-48d9-9d5b-01063bfc698e); Time taken: 0.002 seconds 2018-01-05T11:28:56,794 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:56,794 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,794 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,794 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,794 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query -- -- Table dest4 -- DROP TABLE IF EXISTS dest4 2018-01-05T11:28:56,795 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.003 seconds 2018-01-05T11:28:56,795 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,795 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:56,795 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,795 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:56,795 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,795 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:56,795 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,795 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,795 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,795 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: CREATE TABLE dest4 (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' 2018-01-05T11:28:56,795 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112856_79ca7f1f-e048-4228-8d82-ab91a521480e): CREATE TABLE dest4 (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' 2018-01-05T11:28:56,795 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,795 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: CREATE TABLE dest4 (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' 2018-01-05T11:28:56,795 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:56,795 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,795 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,795 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,795 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,795 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,795 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:28:56,795 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Creating table default.dest4 position=13 2018-01-05T11:28:56,795 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,795 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_database: default 2018-01-05T11:28:56,795 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_database: default 2018-01-05T11:28:56,796 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:56,796 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,796 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,796 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:56,796 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: validation start 2018-01-05T11:28:56,796 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:28:56,796 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Not a partition. 2018-01-05T11:28:56,796 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,796 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:56,796 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,796 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:56,796 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getDatabase_(String, )=1, flushCache_()=0} 2018-01-05T11:28:56,796 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112856_79ca7f1f-e048-4228-8d82-ab91a521480e); Time taken: 0.001 seconds 2018-01-05T11:28:56,796 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,796 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,796 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112856_79ca7f1f-e048-4228-8d82-ab91a521480e): CREATE TABLE dest4 (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' 2018-01-05T11:28:56,797 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,797 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: CREATE TABLE dest4 (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' 2018-01-05T11:28:56,797 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: CREATETABLE 2018-01-05T11:28:56,797 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: database:default 2018-01-05T11:28:56,797 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@dest4 2018-01-05T11:28:56,797 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,797 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,797 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,797 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,797 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:56,797 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] plan.CreateTableDesc: Default to org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe for table dest4 2018-01-05T11:28:56,798 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: creating table default.dest4 on file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/dest4 2018-01-05T11:28:56,798 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,798 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: create_table: Table(tableName:dest4, dbName:default, owner:hive_test_user, createTime:1515180536, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:string, comment:default), FieldSchema(name:value, type:string, comment:default)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/dest4, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, numFiles=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:56,798 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=create_table: Table(tableName:dest4, dbName:default, owner:hive_test_user, createTime:1515180536, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:string, comment:default), FieldSchema(name:value, type:string, comment:default)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/dest4, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, numFiles=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:56,799 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hive-site.xml 2018-01-05T11:28:56,799 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hivemetastore-site.xml 2018-01-05T11:28:56,800 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Unable to find config file metastore-site.xml 2018-01-05T11:28:56,800 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file null 2018-01-05T11:28:56,810 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:56,810 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: Location: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/dest4 specified for non-external table:dest4 2018-01-05T11:28:56,810 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/dest4 2018-01-05T11:28:56,822 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,823 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,825 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,827 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,827 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,827 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@dest4) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@dest4) Type=TABLE WriteType=DDL_NO_LOCK is present 2018-01-05T11:28:56,827 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,827 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,827 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: CREATE TABLE dest4 (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' 2018-01-05T11:28:56,827 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: CREATETABLE 2018-01-05T11:28:56,827 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: database:default 2018-01-05T11:28:56,827 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@dest4 2018-01-05T11:28:56,827 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,827 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,827 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:56,827 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {createTable_(Table, )=29, isCompatibleWith_(Configuration, )=0} 2018-01-05T11:28:56,828 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112856_79ca7f1f-e048-4228-8d82-ab91a521480e); Time taken: 0.031 seconds 2018-01-05T11:28:56,828 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:56,828 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,828 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,828 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,828 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query CREATE TABLE dest4 (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' 2018-01-05T11:28:56,828 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.033 seconds 2018-01-05T11:28:56,828 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,828 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:56,828 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,828 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:56,828 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,828 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:56,828 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,828 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,828 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,828 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: -- -- Table dest4_sequencefile -- DROP TABLE IF EXISTS dest4_sequencefile 2018-01-05T11:28:56,828 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112856_ada167c9-d9a1-4994-986d-0b37b0038cd2): -- -- Table dest4_sequencefile -- DROP TABLE IF EXISTS dest4_sequencefile 2018-01-05T11:28:56,828 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,828 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: -- -- Table dest4_sequencefile -- DROP TABLE IF EXISTS dest4_sequencefile 2018-01-05T11:28:56,828 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:56,828 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,828 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,829 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,829 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,829 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,829 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,829 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest4_sequencefile 2018-01-05T11:28:56,829 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest4_sequencefile 2018-01-05T11:28:56,829 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,829 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,829 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:56,829 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,829 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:56,829 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,829 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:56,829 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=1} 2018-01-05T11:28:56,829 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112856_ada167c9-d9a1-4994-986d-0b37b0038cd2); Time taken: 0.001 seconds 2018-01-05T11:28:56,829 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,829 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,829 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112856_ada167c9-d9a1-4994-986d-0b37b0038cd2): -- -- Table dest4_sequencefile -- DROP TABLE IF EXISTS dest4_sequencefile 2018-01-05T11:28:56,830 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,830 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: -- -- Table dest4_sequencefile -- DROP TABLE IF EXISTS dest4_sequencefile 2018-01-05T11:28:56,830 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:56,830 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,830 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,830 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,830 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,830 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:56,830 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,830 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest4_sequencefile 2018-01-05T11:28:56,830 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest4_sequencefile 2018-01-05T11:28:56,831 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,831 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,831 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table dest4_sequencefile not found: default.dest4_sequencefile table not found 2018-01-05T11:28:56,831 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,831 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest4_sequencefile 2018-01-05T11:28:56,831 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest4_sequencefile 2018-01-05T11:28:56,831 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,831 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,831 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,831 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,831 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: -- -- Table dest4_sequencefile -- DROP TABLE IF EXISTS dest4_sequencefile 2018-01-05T11:28:56,831 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:56,831 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,831 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,831 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:56,831 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=0} 2018-01-05T11:28:56,831 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112856_ada167c9-d9a1-4994-986d-0b37b0038cd2); Time taken: 0.002 seconds 2018-01-05T11:28:56,831 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:56,831 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,831 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,831 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,831 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query -- -- Table dest4_sequencefile -- DROP TABLE IF EXISTS dest4_sequencefile 2018-01-05T11:28:56,831 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.003 seconds 2018-01-05T11:28:56,831 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,831 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:56,831 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,831 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:56,831 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,831 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:56,831 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,831 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,831 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,831 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: CREATE TABLE dest4_sequencefile (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.SequenceFileInputFormat' OUTPUTFORMAT 'org.apache.hadoop.mapred.SequenceFileOutputFormat' 2018-01-05T11:28:56,831 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112856_32b02c23-8003-49c2-9860-48943fe56145): CREATE TABLE dest4_sequencefile (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.SequenceFileInputFormat' OUTPUTFORMAT 'org.apache.hadoop.mapred.SequenceFileOutputFormat' 2018-01-05T11:28:56,831 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,831 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: CREATE TABLE dest4_sequencefile (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.SequenceFileInputFormat' OUTPUTFORMAT 'org.apache.hadoop.mapred.SequenceFileOutputFormat' 2018-01-05T11:28:56,832 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:56,832 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,832 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,832 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,832 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,832 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,832 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:28:56,832 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Creating table default.dest4_sequencefile position=13 2018-01-05T11:28:56,832 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,832 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_database: default 2018-01-05T11:28:56,832 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_database: default 2018-01-05T11:28:56,832 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:56,832 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,832 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,833 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:56,833 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: validation start 2018-01-05T11:28:56,833 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:28:56,833 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Not a partition. 2018-01-05T11:28:56,833 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,833 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:56,833 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,833 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:56,833 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getDatabase_(String, )=0, flushCache_()=0} 2018-01-05T11:28:56,833 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112856_32b02c23-8003-49c2-9860-48943fe56145); Time taken: 0.002 seconds 2018-01-05T11:28:56,833 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,833 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,833 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112856_32b02c23-8003-49c2-9860-48943fe56145): CREATE TABLE dest4_sequencefile (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.SequenceFileInputFormat' OUTPUTFORMAT 'org.apache.hadoop.mapred.SequenceFileOutputFormat' 2018-01-05T11:28:56,834 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,834 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: CREATE TABLE dest4_sequencefile (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.SequenceFileInputFormat' OUTPUTFORMAT 'org.apache.hadoop.mapred.SequenceFileOutputFormat' 2018-01-05T11:28:56,834 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: CREATETABLE 2018-01-05T11:28:56,834 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: database:default 2018-01-05T11:28:56,834 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@dest4_sequencefile 2018-01-05T11:28:56,834 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,834 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,834 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,834 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,834 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:56,834 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] plan.CreateTableDesc: Default to org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe for table dest4_sequencefile 2018-01-05T11:28:56,834 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: creating table default.dest4_sequencefile on file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/dest4_sequencefile 2018-01-05T11:28:56,834 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,834 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: create_table: Table(tableName:dest4_sequencefile, dbName:default, owner:hive_test_user, createTime:1515180536, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:string, comment:default), FieldSchema(name:value, type:string, comment:default)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/dest4_sequencefile, inputFormat:org.apache.hadoop.mapred.SequenceFileInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, numFiles=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:56,835 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=create_table: Table(tableName:dest4_sequencefile, dbName:default, owner:hive_test_user, createTime:1515180536, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:string, comment:default), FieldSchema(name:value, type:string, comment:default)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/dest4_sequencefile, inputFormat:org.apache.hadoop.mapred.SequenceFileInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, numFiles=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:56,835 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hive-site.xml 2018-01-05T11:28:56,835 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hivemetastore-site.xml 2018-01-05T11:28:56,836 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Unable to find config file metastore-site.xml 2018-01-05T11:28:56,836 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file null 2018-01-05T11:28:56,843 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:56,843 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: Location: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/dest4_sequencefile specified for non-external table:dest4_sequencefile 2018-01-05T11:28:56,843 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/dest4_sequencefile 2018-01-05T11:28:56,857 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,858 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,859 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,859 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@dest4_sequencefile) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@dest4_sequencefile) Type=TABLE WriteType=DDL_NO_LOCK is present 2018-01-05T11:28:56,859 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,859 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,859 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: CREATE TABLE dest4_sequencefile (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.SequenceFileInputFormat' OUTPUTFORMAT 'org.apache.hadoop.mapred.SequenceFileOutputFormat' 2018-01-05T11:28:56,859 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: CREATETABLE 2018-01-05T11:28:56,859 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: database:default 2018-01-05T11:28:56,859 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@dest4_sequencefile 2018-01-05T11:28:56,859 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,859 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,859 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:56,859 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {createTable_(Table, )=25, isCompatibleWith_(Configuration, )=0} 2018-01-05T11:28:56,859 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112856_32b02c23-8003-49c2-9860-48943fe56145); Time taken: 0.026 seconds 2018-01-05T11:28:56,859 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:56,859 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,859 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,859 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,859 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query CREATE TABLE dest4_sequencefile (key STRING COMMENT 'default', value STRING COMMENT 'default') STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.SequenceFileInputFormat' OUTPUTFORMAT 'org.apache.hadoop.mapred.SequenceFileOutputFormat' 2018-01-05T11:28:56,859 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.028 seconds 2018-01-05T11:28:56,859 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,859 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:56,859 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,859 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:56,859 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,859 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:56,859 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,859 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,859 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,859 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: -- -- CBO tables -- drop table if exists cbo_t1 2018-01-05T11:28:56,859 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112856_ce2483ef-64c2-4098-9dda-b21ffd9a5180): -- -- CBO tables -- drop table if exists cbo_t1 2018-01-05T11:28:56,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: -- -- CBO tables -- drop table if exists cbo_t1 2018-01-05T11:28:56,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:56,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,860 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,860 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t1 2018-01-05T11:28:56,860 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t1 2018-01-05T11:28:56,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,860 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:56,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,860 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:56,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,860 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:56,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:28:56,861 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112856_ce2483ef-64c2-4098-9dda-b21ffd9a5180); Time taken: 0.001 seconds 2018-01-05T11:28:56,861 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,861 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,861 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112856_ce2483ef-64c2-4098-9dda-b21ffd9a5180): -- -- CBO tables -- drop table if exists cbo_t1 2018-01-05T11:28:56,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,862 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: -- -- CBO tables -- drop table if exists cbo_t1 2018-01-05T11:28:56,862 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:56,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,862 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:56,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,862 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t1 2018-01-05T11:28:56,862 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t1 2018-01-05T11:28:56,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,863 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table cbo_t1 not found: default.cbo_t1 table not found 2018-01-05T11:28:56,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,863 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t1 2018-01-05T11:28:56,863 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t1 2018-01-05T11:28:56,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,863 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: -- -- CBO tables -- drop table if exists cbo_t1 2018-01-05T11:28:56,863 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:56,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,863 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:56,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=0} 2018-01-05T11:28:56,863 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112856_ce2483ef-64c2-4098-9dda-b21ffd9a5180); Time taken: 0.002 seconds 2018-01-05T11:28:56,863 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:56,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,863 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query -- -- CBO tables -- drop table if exists cbo_t1 2018-01-05T11:28:56,863 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.004 seconds 2018-01-05T11:28:56,863 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,863 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:56,864 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,864 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:56,864 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,864 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:56,864 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,864 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,864 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,864 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: drop table if exists cbo_t2 2018-01-05T11:28:56,864 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112856_7e3fe7a9-ab63-400e-a892-8dd8eb0ebb6d): drop table if exists cbo_t2 2018-01-05T11:28:56,864 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,864 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: drop table if exists cbo_t2 2018-01-05T11:28:56,864 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:56,864 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,864 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,864 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,864 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,864 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,864 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,864 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t2 2018-01-05T11:28:56,864 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t2 2018-01-05T11:28:56,865 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,865 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,865 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:56,865 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,865 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:56,865 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,865 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:56,865 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:28:56,865 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112856_7e3fe7a9-ab63-400e-a892-8dd8eb0ebb6d); Time taken: 0.001 seconds 2018-01-05T11:28:56,865 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,865 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,865 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112856_7e3fe7a9-ab63-400e-a892-8dd8eb0ebb6d): drop table if exists cbo_t2 2018-01-05T11:28:56,866 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,866 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: drop table if exists cbo_t2 2018-01-05T11:28:56,866 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:56,866 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,866 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,866 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,866 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,866 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:56,866 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,866 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t2 2018-01-05T11:28:56,866 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t2 2018-01-05T11:28:56,866 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,866 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,866 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table cbo_t2 not found: default.cbo_t2 table not found 2018-01-05T11:28:56,866 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,866 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t2 2018-01-05T11:28:56,866 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t2 2018-01-05T11:28:56,867 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,867 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,867 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,867 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,867 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: drop table if exists cbo_t2 2018-01-05T11:28:56,867 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:56,867 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,867 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,867 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:56,867 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=1} 2018-01-05T11:28:56,867 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112856_7e3fe7a9-ab63-400e-a892-8dd8eb0ebb6d); Time taken: 0.002 seconds 2018-01-05T11:28:56,867 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:56,867 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,867 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,867 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,867 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query drop table if exists cbo_t2 2018-01-05T11:28:56,867 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.003 seconds 2018-01-05T11:28:56,867 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,867 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:56,867 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,867 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:56,867 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,867 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:56,867 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,867 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,867 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,867 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: drop table if exists cbo_t3 2018-01-05T11:28:56,867 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112856_aa7aa43e-fa60-47f7-8b11-24909204c156): drop table if exists cbo_t3 2018-01-05T11:28:56,867 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,867 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: drop table if exists cbo_t3 2018-01-05T11:28:56,867 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:56,867 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,867 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,867 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,867 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,867 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,867 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,867 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t3 2018-01-05T11:28:56,867 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t3 2018-01-05T11:28:56,868 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,868 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,868 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:56,868 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,868 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:56,868 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,868 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:56,868 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:28:56,868 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112856_aa7aa43e-fa60-47f7-8b11-24909204c156); Time taken: 0.001 seconds 2018-01-05T11:28:56,868 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,868 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,868 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112856_aa7aa43e-fa60-47f7-8b11-24909204c156): drop table if exists cbo_t3 2018-01-05T11:28:56,868 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,868 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: drop table if exists cbo_t3 2018-01-05T11:28:56,868 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:56,868 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,868 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,868 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,869 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:56,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,869 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t3 2018-01-05T11:28:56,869 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t3 2018-01-05T11:28:56,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,869 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table cbo_t3 not found: default.cbo_t3 table not found 2018-01-05T11:28:56,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,869 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t3 2018-01-05T11:28:56,869 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t3 2018-01-05T11:28:56,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,869 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: drop table if exists cbo_t3 2018-01-05T11:28:56,869 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:56,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,869 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:56,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=0} 2018-01-05T11:28:56,869 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112856_aa7aa43e-fa60-47f7-8b11-24909204c156); Time taken: 0.001 seconds 2018-01-05T11:28:56,869 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:56,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,869 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,869 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query drop table if exists cbo_t3 2018-01-05T11:28:56,870 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.003 seconds 2018-01-05T11:28:56,870 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,870 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:56,870 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,870 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:56,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:56,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: drop table if exists src_cbo 2018-01-05T11:28:56,870 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112856_230b112c-e30b-44df-b4ad-79f04ded0d92): drop table if exists src_cbo 2018-01-05T11:28:56,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: drop table if exists src_cbo 2018-01-05T11:28:56,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:56,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,870 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,870 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_cbo 2018-01-05T11:28:56,870 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_cbo 2018-01-05T11:28:56,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,870 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:56,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,871 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:56,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,871 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:56,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:28:56,871 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112856_230b112c-e30b-44df-b4ad-79f04ded0d92); Time taken: 0.001 seconds 2018-01-05T11:28:56,871 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,871 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112856_230b112c-e30b-44df-b4ad-79f04ded0d92): drop table if exists src_cbo 2018-01-05T11:28:56,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,871 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: drop table if exists src_cbo 2018-01-05T11:28:56,871 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:56,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,872 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:56,872 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,872 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_cbo 2018-01-05T11:28:56,872 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_cbo 2018-01-05T11:28:56,872 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,872 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,872 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table src_cbo not found: default.src_cbo table not found 2018-01-05T11:28:56,872 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,872 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_cbo 2018-01-05T11:28:56,872 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_cbo 2018-01-05T11:28:56,872 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,872 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,872 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,872 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,872 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: drop table if exists src_cbo 2018-01-05T11:28:56,872 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:56,872 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,872 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,872 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:56,872 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=0} 2018-01-05T11:28:56,872 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112856_230b112c-e30b-44df-b4ad-79f04ded0d92); Time taken: 0.001 seconds 2018-01-05T11:28:56,872 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:56,872 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,872 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,872 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,872 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query drop table if exists src_cbo 2018-01-05T11:28:56,872 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.002 seconds 2018-01-05T11:28:56,872 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,872 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:56,873 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,873 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:56,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:56,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: drop table if exists part 2018-01-05T11:28:56,873 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112856_73aae8c2-4124-47a4-80c2-74fe21307b07): drop table if exists part 2018-01-05T11:28:56,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: drop table if exists part 2018-01-05T11:28:56,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:56,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,873 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,873 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=part 2018-01-05T11:28:56,873 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=part 2018-01-05T11:28:56,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,873 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:56,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,873 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:56,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,873 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:56,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:28:56,873 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112856_73aae8c2-4124-47a4-80c2-74fe21307b07); Time taken: 0.0 seconds 2018-01-05T11:28:56,873 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,874 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112856_73aae8c2-4124-47a4-80c2-74fe21307b07): drop table if exists part 2018-01-05T11:28:56,874 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,874 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: drop table if exists part 2018-01-05T11:28:56,874 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:56,874 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,874 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,874 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,875 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,875 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:56,875 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,875 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=part 2018-01-05T11:28:56,875 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=part 2018-01-05T11:28:56,875 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,875 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,875 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table part not found: default.part table not found 2018-01-05T11:28:56,875 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,875 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=part 2018-01-05T11:28:56,875 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=part 2018-01-05T11:28:56,875 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,875 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,875 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,875 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,875 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: drop table if exists part 2018-01-05T11:28:56,875 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:56,875 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,875 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,875 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:56,876 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=0} 2018-01-05T11:28:56,876 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112856_73aae8c2-4124-47a4-80c2-74fe21307b07); Time taken: 0.002 seconds 2018-01-05T11:28:56,876 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:56,876 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,876 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,876 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,876 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query drop table if exists part 2018-01-05T11:28:56,876 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.003 seconds 2018-01-05T11:28:56,876 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,876 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:56,876 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,876 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:56,876 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,876 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:56,876 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,876 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,876 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,876 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: drop table if exists lineitem 2018-01-05T11:28:56,876 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112856_9a474a23-cc22-4eac-a958-a4cf40cd604b): drop table if exists lineitem 2018-01-05T11:28:56,876 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,876 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: drop table if exists lineitem 2018-01-05T11:28:56,876 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:56,876 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,876 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,876 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,876 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,876 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,876 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,876 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=lineitem 2018-01-05T11:28:56,876 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=lineitem 2018-01-05T11:28:56,876 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,876 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,876 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:56,876 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,876 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:56,876 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,877 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:56,877 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:28:56,877 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112856_9a474a23-cc22-4eac-a958-a4cf40cd604b); Time taken: 0.0 seconds 2018-01-05T11:28:56,877 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,877 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,877 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112856_9a474a23-cc22-4eac-a958-a4cf40cd604b): drop table if exists lineitem 2018-01-05T11:28:56,877 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,877 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: drop table if exists lineitem 2018-01-05T11:28:56,877 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:28:56,877 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,877 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,877 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,877 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,877 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:56,878 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,878 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=lineitem 2018-01-05T11:28:56,878 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=lineitem 2018-01-05T11:28:56,878 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,878 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,878 ERROR [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Table lineitem not found: default.lineitem table not found 2018-01-05T11:28:56,878 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,878 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=lineitem 2018-01-05T11:28:56,878 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=lineitem 2018-01-05T11:28:56,878 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,878 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,878 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,878 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,878 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: drop table if exists lineitem 2018-01-05T11:28:56,878 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:28:56,878 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,878 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,878 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:56,878 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=1, dropTable_(String, String, boolean, boolean, boolean, )=0} 2018-01-05T11:28:56,878 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112856_9a474a23-cc22-4eac-a958-a4cf40cd604b); Time taken: 0.001 seconds 2018-01-05T11:28:56,878 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:56,878 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,878 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,878 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,878 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query drop table if exists lineitem 2018-01-05T11:28:56,878 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.002 seconds 2018-01-05T11:28:56,878 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,878 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:56,878 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,878 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:56,879 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: true 2018-01-05T11:28:56,879 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,879 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:56,879 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,879 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:56,879 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,879 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:56,879 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,879 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,879 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,879 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: create table cbo_t1(key string, value string, c_int int, c_float float, c_boolean boolean) partitioned by (dt string) row format delimited fields terminated by ',' STORED AS TEXTFILE 2018-01-05T11:28:56,879 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112856_a66de5e3-811a-41dd-957c-55501d5ac7e0): create table cbo_t1(key string, value string, c_int int, c_float float, c_boolean boolean) partitioned by (dt string) row format delimited fields terminated by ',' STORED AS TEXTFILE 2018-01-05T11:28:56,879 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,879 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: create table cbo_t1(key string, value string, c_int int, c_float float, c_boolean boolean) partitioned by (dt string) row format delimited fields terminated by ',' STORED AS TEXTFILE 2018-01-05T11:28:56,879 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:56,879 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,879 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,879 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,879 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,879 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,879 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:28:56,879 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Creating table default.cbo_t1 position=13 2018-01-05T11:28:56,879 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,879 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_database: default 2018-01-05T11:28:56,879 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_database: default 2018-01-05T11:28:56,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:56,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,880 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:56,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: validation start 2018-01-05T11:28:56,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:28:56,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Not a partition. 2018-01-05T11:28:56,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,880 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:56,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,880 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:56,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getDatabase_(String, )=1, flushCache_()=0} 2018-01-05T11:28:56,880 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112856_a66de5e3-811a-41dd-957c-55501d5ac7e0); Time taken: 0.001 seconds 2018-01-05T11:28:56,880 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,880 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,880 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112856_a66de5e3-811a-41dd-957c-55501d5ac7e0): create table cbo_t1(key string, value string, c_int int, c_float float, c_boolean boolean) partitioned by (dt string) row format delimited fields terminated by ',' STORED AS TEXTFILE 2018-01-05T11:28:56,881 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,881 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: create table cbo_t1(key string, value string, c_int int, c_float float, c_boolean boolean) partitioned by (dt string) row format delimited fields terminated by ',' STORED AS TEXTFILE 2018-01-05T11:28:56,881 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: CREATETABLE 2018-01-05T11:28:56,881 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: database:default 2018-01-05T11:28:56,881 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@cbo_t1 2018-01-05T11:28:56,881 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,881 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,881 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,881 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,881 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:56,881 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Found class for org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe 2018-01-05T11:28:56,881 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: creating table default.cbo_t1 on file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1 2018-01-05T11:28:56,882 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,882 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: create_table: Table(tableName:cbo_t1, dbName:default, owner:hive_test_user, createTime:1515180536, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:string, comment:null), FieldSchema(name:value, type:string, comment:null), FieldSchema(name:c_int, type:int, comment:null), FieldSchema(name:c_float, type:float, comment:null), FieldSchema(name:c_boolean, type:boolean, comment:null)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{field.delim=,, serialization.format=,}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[FieldSchema(name:dt, type:string, comment:null)], parameters:{}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:56,882 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=create_table: Table(tableName:cbo_t1, dbName:default, owner:hive_test_user, createTime:1515180536, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:string, comment:null), FieldSchema(name:value, type:string, comment:null), FieldSchema(name:c_int, type:int, comment:null), FieldSchema(name:c_float, type:float, comment:null), FieldSchema(name:c_boolean, type:boolean, comment:null)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{field.delim=,, serialization.format=,}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[FieldSchema(name:dt, type:string, comment:null)], parameters:{}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:56,882 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hive-site.xml 2018-01-05T11:28:56,883 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hivemetastore-site.xml 2018-01-05T11:28:56,883 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Unable to find config file metastore-site.xml 2018-01-05T11:28:56,883 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file null 2018-01-05T11:28:56,891 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:56,891 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: Location: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1 specified for non-external table:cbo_t1 2018-01-05T11:28:56,891 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1 2018-01-05T11:28:56,909 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,911 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,911 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,911 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@cbo_t1) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@cbo_t1) Type=TABLE WriteType=DDL_NO_LOCK is present 2018-01-05T11:28:56,911 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,911 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,911 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: create table cbo_t1(key string, value string, c_int int, c_float float, c_boolean boolean) partitioned by (dt string) row format delimited fields terminated by ',' STORED AS TEXTFILE 2018-01-05T11:28:56,911 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: CREATETABLE 2018-01-05T11:28:56,911 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: database:default 2018-01-05T11:28:56,911 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@cbo_t1 2018-01-05T11:28:56,911 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,911 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,911 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:56,911 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {createTable_(Table, )=29, isCompatibleWith_(Configuration, )=0} 2018-01-05T11:28:56,911 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112856_a66de5e3-811a-41dd-957c-55501d5ac7e0); Time taken: 0.031 seconds 2018-01-05T11:28:56,911 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:56,911 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,911 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,911 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,911 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query create table cbo_t1(key string, value string, c_int int, c_float float, c_boolean boolean) partitioned by (dt string) row format delimited fields terminated by ',' STORED AS TEXTFILE 2018-01-05T11:28:56,911 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.032 seconds 2018-01-05T11:28:56,911 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,911 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:56,911 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,911 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:56,912 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,912 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:56,912 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,912 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,912 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,912 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: create table cbo_t2(key string, value string, c_int int, c_float float, c_boolean boolean) partitioned by (dt string) row format delimited fields terminated by ',' STORED AS TEXTFILE 2018-01-05T11:28:56,912 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112856_75b25e5d-abb9-43cb-9509-249006deab40): create table cbo_t2(key string, value string, c_int int, c_float float, c_boolean boolean) partitioned by (dt string) row format delimited fields terminated by ',' STORED AS TEXTFILE 2018-01-05T11:28:56,912 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,912 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: create table cbo_t2(key string, value string, c_int int, c_float float, c_boolean boolean) partitioned by (dt string) row format delimited fields terminated by ',' STORED AS TEXTFILE 2018-01-05T11:28:56,912 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:56,912 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,912 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,912 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,912 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,912 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,912 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:28:56,912 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Creating table default.cbo_t2 position=13 2018-01-05T11:28:56,912 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,912 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_database: default 2018-01-05T11:28:56,913 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_database: default 2018-01-05T11:28:56,913 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:56,913 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,913 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,913 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:56,913 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: validation start 2018-01-05T11:28:56,913 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:28:56,913 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Not a partition. 2018-01-05T11:28:56,913 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,913 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:56,913 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,913 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:56,913 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getDatabase_(String, )=1, flushCache_()=0} 2018-01-05T11:28:56,913 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112856_75b25e5d-abb9-43cb-9509-249006deab40); Time taken: 0.001 seconds 2018-01-05T11:28:56,913 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,913 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,913 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112856_75b25e5d-abb9-43cb-9509-249006deab40): create table cbo_t2(key string, value string, c_int int, c_float float, c_boolean boolean) partitioned by (dt string) row format delimited fields terminated by ',' STORED AS TEXTFILE 2018-01-05T11:28:56,914 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,914 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: create table cbo_t2(key string, value string, c_int int, c_float float, c_boolean boolean) partitioned by (dt string) row format delimited fields terminated by ',' STORED AS TEXTFILE 2018-01-05T11:28:56,914 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: CREATETABLE 2018-01-05T11:28:56,914 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: database:default 2018-01-05T11:28:56,914 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@cbo_t2 2018-01-05T11:28:56,914 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,914 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,914 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,914 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,914 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:56,915 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Found class for org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe 2018-01-05T11:28:56,915 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: creating table default.cbo_t2 on file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2 2018-01-05T11:28:56,915 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,915 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: create_table: Table(tableName:cbo_t2, dbName:default, owner:hive_test_user, createTime:1515180536, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:string, comment:null), FieldSchema(name:value, type:string, comment:null), FieldSchema(name:c_int, type:int, comment:null), FieldSchema(name:c_float, type:float, comment:null), FieldSchema(name:c_boolean, type:boolean, comment:null)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{field.delim=,, serialization.format=,}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[FieldSchema(name:dt, type:string, comment:null)], parameters:{}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:56,915 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=create_table: Table(tableName:cbo_t2, dbName:default, owner:hive_test_user, createTime:1515180536, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:string, comment:null), FieldSchema(name:value, type:string, comment:null), FieldSchema(name:c_int, type:int, comment:null), FieldSchema(name:c_float, type:float, comment:null), FieldSchema(name:c_boolean, type:boolean, comment:null)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{field.delim=,, serialization.format=,}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[FieldSchema(name:dt, type:string, comment:null)], parameters:{}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:56,916 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hive-site.xml 2018-01-05T11:28:56,916 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hivemetastore-site.xml 2018-01-05T11:28:56,917 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Unable to find config file metastore-site.xml 2018-01-05T11:28:56,917 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file null 2018-01-05T11:28:56,925 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:56,925 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: Location: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2 specified for non-external table:cbo_t2 2018-01-05T11:28:56,925 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2 2018-01-05T11:28:56,944 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,945 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,945 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@cbo_t2) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@cbo_t2) Type=TABLE WriteType=DDL_NO_LOCK is present 2018-01-05T11:28:56,945 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,945 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,945 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: create table cbo_t2(key string, value string, c_int int, c_float float, c_boolean boolean) partitioned by (dt string) row format delimited fields terminated by ',' STORED AS TEXTFILE 2018-01-05T11:28:56,945 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: CREATETABLE 2018-01-05T11:28:56,945 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: database:default 2018-01-05T11:28:56,945 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@cbo_t2 2018-01-05T11:28:56,945 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,945 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,945 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:56,945 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {createTable_(Table, )=30, isCompatibleWith_(Configuration, )=1} 2018-01-05T11:28:56,945 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112856_75b25e5d-abb9-43cb-9509-249006deab40); Time taken: 0.032 seconds 2018-01-05T11:28:56,945 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:56,945 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,945 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,945 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,945 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query create table cbo_t2(key string, value string, c_int int, c_float float, c_boolean boolean) partitioned by (dt string) row format delimited fields terminated by ',' STORED AS TEXTFILE 2018-01-05T11:28:56,945 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.033 seconds 2018-01-05T11:28:56,945 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,945 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:56,945 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,945 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:56,945 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,945 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:56,945 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,945 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,946 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,946 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: create table cbo_t3(key string, value string, c_int int, c_float float, c_boolean boolean) row format delimited fields terminated by ',' STORED AS TEXTFILE 2018-01-05T11:28:56,946 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112856_e26a54d2-0580-49f5-929d-5624e1af89e8): create table cbo_t3(key string, value string, c_int int, c_float float, c_boolean boolean) row format delimited fields terminated by ',' STORED AS TEXTFILE 2018-01-05T11:28:56,946 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,946 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: create table cbo_t3(key string, value string, c_int int, c_float float, c_boolean boolean) row format delimited fields terminated by ',' STORED AS TEXTFILE 2018-01-05T11:28:56,946 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:56,946 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,946 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,946 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,946 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,946 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,946 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:28:56,946 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Creating table default.cbo_t3 position=13 2018-01-05T11:28:56,946 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,946 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_database: default 2018-01-05T11:28:56,946 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_database: default 2018-01-05T11:28:56,947 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:56,947 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,947 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,947 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:56,947 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: validation start 2018-01-05T11:28:56,947 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:28:56,947 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Not a partition. 2018-01-05T11:28:56,947 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,947 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:56,947 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,947 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:56,947 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getDatabase_(String, )=1, flushCache_()=0} 2018-01-05T11:28:56,947 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112856_e26a54d2-0580-49f5-929d-5624e1af89e8); Time taken: 0.002 seconds 2018-01-05T11:28:56,947 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,947 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,947 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112856_e26a54d2-0580-49f5-929d-5624e1af89e8): create table cbo_t3(key string, value string, c_int int, c_float float, c_boolean boolean) row format delimited fields terminated by ',' STORED AS TEXTFILE 2018-01-05T11:28:56,948 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,948 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: create table cbo_t3(key string, value string, c_int int, c_float float, c_boolean boolean) row format delimited fields terminated by ',' STORED AS TEXTFILE 2018-01-05T11:28:56,948 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: CREATETABLE 2018-01-05T11:28:56,948 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: database:default 2018-01-05T11:28:56,948 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@cbo_t3 2018-01-05T11:28:56,948 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,948 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,948 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,948 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,948 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:56,948 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Found class for org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe 2018-01-05T11:28:56,949 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: creating table default.cbo_t3 on file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3 2018-01-05T11:28:56,949 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,949 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: create_table: Table(tableName:cbo_t3, dbName:default, owner:hive_test_user, createTime:1515180536, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:string, comment:null), FieldSchema(name:value, type:string, comment:null), FieldSchema(name:c_int, type:int, comment:null), FieldSchema(name:c_float, type:float, comment:null), FieldSchema(name:c_boolean, type:boolean, comment:null)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{field.delim=,, serialization.format=,}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"c_boolean":"true","c_float":"true","c_int":"true","key":"true","value":"true"}}, numFiles=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:56,949 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=create_table: Table(tableName:cbo_t3, dbName:default, owner:hive_test_user, createTime:1515180536, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:string, comment:null), FieldSchema(name:value, type:string, comment:null), FieldSchema(name:c_int, type:int, comment:null), FieldSchema(name:c_float, type:float, comment:null), FieldSchema(name:c_boolean, type:boolean, comment:null)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{field.delim=,, serialization.format=,}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"c_boolean":"true","c_float":"true","c_int":"true","key":"true","value":"true"}}, numFiles=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:56,950 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hive-site.xml 2018-01-05T11:28:56,950 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hivemetastore-site.xml 2018-01-05T11:28:56,951 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Unable to find config file metastore-site.xml 2018-01-05T11:28:56,951 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file null 2018-01-05T11:28:56,959 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:56,959 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: Location: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3 specified for non-external table:cbo_t3 2018-01-05T11:28:56,959 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3 2018-01-05T11:28:56,971 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,973 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,975 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,977 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,977 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,977 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@cbo_t3) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@cbo_t3) Type=TABLE WriteType=DDL_NO_LOCK is present 2018-01-05T11:28:56,977 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,977 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,977 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: create table cbo_t3(key string, value string, c_int int, c_float float, c_boolean boolean) row format delimited fields terminated by ',' STORED AS TEXTFILE 2018-01-05T11:28:56,977 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: CREATETABLE 2018-01-05T11:28:56,977 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: database:default 2018-01-05T11:28:56,977 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@cbo_t3 2018-01-05T11:28:56,977 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,977 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,977 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:56,977 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {createTable_(Table, )=28, isCompatibleWith_(Configuration, )=0} 2018-01-05T11:28:56,977 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112856_e26a54d2-0580-49f5-929d-5624e1af89e8); Time taken: 0.03 seconds 2018-01-05T11:28:56,977 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:56,978 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,978 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,978 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,978 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query create table cbo_t3(key string, value string, c_int int, c_float float, c_boolean boolean) row format delimited fields terminated by ',' STORED AS TEXTFILE 2018-01-05T11:28:56,978 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.033 seconds 2018-01-05T11:28:56,978 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,978 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:56,978 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:56,978 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:56,978 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,978 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:56,978 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,978 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,978 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,978 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: load data local inpath '${hiveconf:test.data.dir}/cbo_t1.txt' into table cbo_t1 partition (dt='2014') 2018-01-05T11:28:56,978 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112856_ebff9843-4cd5-47dc-8a79-cd768e4fc521): load data local inpath '/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/cbo_t1.txt' into table cbo_t1 partition (dt='2014') 2018-01-05T11:28:56,978 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,978 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: load data local inpath '/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/cbo_t1.txt' into table cbo_t1 partition (dt='2014') 2018-01-05T11:28:56,978 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:56,978 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,978 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,978 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,978 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,978 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,978 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.LoadSemanticAnalyzer: file@null@/Users/djaiswal/hive/hive/data/files/cbo_t1.txt 2018-01-05T11:28:56,978 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,978 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t1 2018-01-05T11:28:56,979 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t1 2018-01-05T11:28:56,982 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,982 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,982 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,982 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=cbo_t1[2014] 2018-01-05T11:28:56,982 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=cbo_t1[2014] 2018-01-05T11:28:56,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,986 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,988 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,988 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=cbo_t1[2014] 2018-01-05T11:28:56,988 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=cbo_t1[2014] 2018-01-05T11:28:56,990 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,991 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,992 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,992 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:56,992 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,992 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:56,992 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,992 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:56,992 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=4, flushCache_()=0} 2018-01-05T11:28:56,992 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112856_ebff9843-4cd5-47dc-8a79-cd768e4fc521); Time taken: 0.014 seconds 2018-01-05T11:28:56,992 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:56,992 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,992 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112856_ebff9843-4cd5-47dc-8a79-cd768e4fc521): load data local inpath '/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/cbo_t1.txt' into table cbo_t1 partition (dt='2014') 2018-01-05T11:28:56,993 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,993 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: load data local inpath '/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/cbo_t1.txt' into table cbo_t1 partition (dt='2014') 2018-01-05T11:28:56,993 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: LOAD 2018-01-05T11:28:56,993 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/cbo_t1.txt 2018-01-05T11:28:56,993 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@cbo_t1 2018-01-05T11:28:56,993 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,993 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,993 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,993 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:56,993 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MOVE] in serial mode 2018-01-05T11:28:56,993 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@3a00cdcb, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:56,993 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:56,993 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,993 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:56,993 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:56,993 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:56,993 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:56,993 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,993 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,993 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:56,993 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:56,993 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,993 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:56,993 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,993 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:56,993 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:56,993 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,993 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Loading data to table default.cbo_t1 partition (dt=2014) from file:/Users/djaiswal/hive/hive/data/files/cbo_t1.txt 2018-01-05T11:28:56,994 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,994 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:56,994 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:56,994 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:56,994 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:56,994 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,994 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,995 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,995 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,995 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,995 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:56,995 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:56,995 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,996 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:56,996 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,996 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t1 2018-01-05T11:28:56,996 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t1 2018-01-05T11:28:56,998 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:56,998 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,998 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:56,998 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=cbo_t1[2014] 2018-01-05T11:28:56,998 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=cbo_t1[2014] 2018-01-05T11:28:57,001 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,006 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,006 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,008 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.MoveTask: Partition is: {dt=2014} 2018-01-05T11:28:57,008 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,008 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: partition_name_has_valid_characters 2018-01-05T11:28:57,008 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=partition_name_has_valid_characters 2018-01-05T11:28:57,008 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,008 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,008 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t1 2018-01-05T11:28:57,008 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t1 2018-01-05T11:28:57,010 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,010 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,010 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,010 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=cbo_t1[2014] 2018-01-05T11:28:57,010 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=cbo_t1[2014] 2018-01-05T11:28:57,012 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,013 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,014 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,014 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,014 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1/dt=2014 2018-01-05T11:28:57,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/data/files/cbo_t1.txt/ and the destination path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1/dt=2014/cbo_t1.txt/ 2018-01-05T11:28:57,025 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1/dt=2014/cbo_t1.txt/ and the destination path is /Users/djaiswal/hive/hive/data/files/cbo_t1.txt/ 2018-01-05T11:28:57,048 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,048 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: altering partition for table cbo_t1 with partition spec : {dt=2014} 2018-01-05T11:28:57,048 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: No new files were created, and is not a replace, or we're inserting into a partition that does not exist yet. Skipping generating INSERT event. 2018-01-05T11:28:57,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Adding new partition {dt=2014} 2018-01-05T11:28:57,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,049 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: add_partition : db=default tbl=cbo_t1 2018-01-05T11:28:57,049 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=add_partition : db=default tbl=cbo_t1 2018-01-05T11:28:57,052 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,052 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,055 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,056 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,056 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,056 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=cbo_t1[2014] 2018-01-05T11:28:57,056 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=cbo_t1[2014] 2018-01-05T11:28:57,058 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,064 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,064 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,064 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-1:STATS] in serial mode 2018-01-05T11:28:57,064 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@5a7409e2, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:57,064 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:57,064 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,064 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:57,064 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:57,064 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:57,064 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:57,064 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,064 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,064 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:57,064 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:57,064 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,064 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:57,064 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,064 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:57,064 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:57,064 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,064 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:57,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,065 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:57,065 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:57,065 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:57,065 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:57,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,066 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,066 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,066 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,066 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,066 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:57,066 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:57,066 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,066 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:57,067 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,067 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t1 2018-01-05T11:28:57,067 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t1 2018-01-05T11:28:57,069 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,070 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,070 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,070 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=cbo_t1[2014] 2018-01-05T11:28:57,070 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=cbo_t1[2014] 2018-01-05T11:28:57,072 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,076 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Getting file stats of all partitions. threadpool size:15 2018-01-05T11:28:57,076 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Finished getting file stats of all partitions! 2018-01-05T11:28:57,076 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Partition {dt=2014} stats: [numFiles=1, numRows=0, totalSize=282, rawDataSize=0] 2018-01-05T11:28:57,076 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,076 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_partitions : db=default tbl=cbo_t1 2018-01-05T11:28:57,076 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_partitions : db=default tbl=cbo_t1 2018-01-05T11:28:57,076 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: New partition values:[2014] 2018-01-05T11:28:57,078 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,083 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,086 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,086 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,086 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,086 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,086 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: load data local inpath '/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/cbo_t1.txt' into table cbo_t1 partition (dt='2014') 2018-01-05T11:28:57,086 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: LOAD 2018-01-05T11:28:57,086 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/cbo_t1.txt 2018-01-05T11:28:57,086 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@cbo_t1 2018-01-05T11:28:57,086 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@cbo_t1@dt=2014 2018-01-05T11:28:57,086 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,086 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,086 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:57,086 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_partitions_(String, String, List, EnvironmentContext, )=10, getTable_(String, String, )=4, getPartitionWithAuthInfo_(String, String, List, String, List, )=5} 2018-01-05T11:28:57,086 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112856_ebff9843-4cd5-47dc-8a79-cd768e4fc521); Time taken: 0.094 seconds 2018-01-05T11:28:57,086 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:57,086 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,086 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:57,086 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,086 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query load data local inpath '/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/cbo_t1.txt' into table cbo_t1 partition (dt='2014') 2018-01-05T11:28:57,086 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.108 seconds 2018-01-05T11:28:57,086 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:57,086 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:57,086 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:57,087 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:57,087 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,087 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:57,087 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,087 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,087 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,087 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: load data local inpath '${hiveconf:test.data.dir}/cbo_t2.txt' into table cbo_t2 partition (dt='2014') 2018-01-05T11:28:57,087 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112857_2d02aa99-3b53-4abf-a36d-ccd3e02f992a): load data local inpath '/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/cbo_t2.txt' into table cbo_t2 partition (dt='2014') 2018-01-05T11:28:57,087 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,087 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: load data local inpath '/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/cbo_t2.txt' into table cbo_t2 partition (dt='2014') 2018-01-05T11:28:57,087 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:57,087 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,087 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,087 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,087 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,087 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:57,087 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.LoadSemanticAnalyzer: file@null@/Users/djaiswal/hive/hive/data/files/cbo_t2.txt 2018-01-05T11:28:57,087 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,087 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t2 2018-01-05T11:28:57,087 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t2 2018-01-05T11:28:57,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,091 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=cbo_t2[2014] 2018-01-05T11:28:57,091 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=cbo_t2[2014] 2018-01-05T11:28:57,093 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,098 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,098 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,100 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,100 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=cbo_t2[2014] 2018-01-05T11:28:57,100 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=cbo_t2[2014] 2018-01-05T11:28:57,102 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,104 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:57,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,104 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:57,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,104 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:57,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=3, flushCache_()=0} 2018-01-05T11:28:57,104 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112857_2d02aa99-3b53-4abf-a36d-ccd3e02f992a); Time taken: 0.017 seconds 2018-01-05T11:28:57,104 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:57,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,105 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112857_2d02aa99-3b53-4abf-a36d-ccd3e02f992a): load data local inpath '/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/cbo_t2.txt' into table cbo_t2 partition (dt='2014') 2018-01-05T11:28:57,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,106 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: load data local inpath '/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/cbo_t2.txt' into table cbo_t2 partition (dt='2014') 2018-01-05T11:28:57,106 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: LOAD 2018-01-05T11:28:57,106 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/cbo_t2.txt 2018-01-05T11:28:57,106 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@cbo_t2 2018-01-05T11:28:57,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,106 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MOVE] in serial mode 2018-01-05T11:28:57,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@5fcdda47, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:57,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:57,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,106 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:57,106 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:57,106 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:57,106 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:57,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,106 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:57,106 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:57,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:57,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,106 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:57,106 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:57,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,106 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Loading data to table default.cbo_t2 partition (dt=2014) from file:/Users/djaiswal/hive/hive/data/files/cbo_t2.txt 2018-01-05T11:28:57,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,107 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:57,107 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:57,107 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:57,107 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:57,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,108 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:57,108 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:57,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,108 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:57,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,108 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t2 2018-01-05T11:28:57,108 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t2 2018-01-05T11:28:57,110 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,111 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,111 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,111 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=cbo_t2[2014] 2018-01-05T11:28:57,111 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=cbo_t2[2014] 2018-01-05T11:28:57,114 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,115 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,116 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.MoveTask: Partition is: {dt=2014} 2018-01-05T11:28:57,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,116 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: partition_name_has_valid_characters 2018-01-05T11:28:57,116 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=partition_name_has_valid_characters 2018-01-05T11:28:57,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,116 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t2 2018-01-05T11:28:57,117 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t2 2018-01-05T11:28:57,119 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,119 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,119 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,119 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=cbo_t2[2014] 2018-01-05T11:28:57,119 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=cbo_t2[2014] 2018-01-05T11:28:57,121 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,123 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,123 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2/dt=2014 2018-01-05T11:28:57,134 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/data/files/cbo_t2.txt/ and the destination path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2/dt=2014/cbo_t2.txt/ 2018-01-05T11:28:57,134 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2/dt=2014/cbo_t2.txt/ and the destination path is /Users/djaiswal/hive/hive/data/files/cbo_t2.txt/ 2018-01-05T11:28:57,157 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,157 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: altering partition for table cbo_t2 with partition spec : {dt=2014} 2018-01-05T11:28:57,157 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: No new files were created, and is not a replace, or we're inserting into a partition that does not exist yet. Skipping generating INSERT event. 2018-01-05T11:28:57,158 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Adding new partition {dt=2014} 2018-01-05T11:28:57,158 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,158 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: add_partition : db=default tbl=cbo_t2 2018-01-05T11:28:57,158 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=add_partition : db=default tbl=cbo_t2 2018-01-05T11:28:57,162 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,165 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,166 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=cbo_t2[2014] 2018-01-05T11:28:57,166 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=cbo_t2[2014] 2018-01-05T11:28:57,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,174 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,174 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,175 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-1:STATS] in serial mode 2018-01-05T11:28:57,175 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@7291d887, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:57,175 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:57,175 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,175 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:57,175 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:57,175 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:57,175 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:57,175 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,175 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,175 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:57,175 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:57,175 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,175 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:57,175 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,175 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:57,175 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:57,175 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,175 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:57,175 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,176 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:57,176 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:57,176 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:57,176 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:57,176 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,176 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,176 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,177 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,177 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,177 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:57,177 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:57,177 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,177 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:57,177 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,177 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t2 2018-01-05T11:28:57,177 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t2 2018-01-05T11:28:57,181 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,181 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,181 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,181 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=cbo_t2[2014] 2018-01-05T11:28:57,181 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=cbo_t2[2014] 2018-01-05T11:28:57,184 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,189 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,189 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,189 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Getting file stats of all partitions. threadpool size:15 2018-01-05T11:28:57,190 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Finished getting file stats of all partitions! 2018-01-05T11:28:57,190 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Partition {dt=2014} stats: [numFiles=1, numRows=0, totalSize=282, rawDataSize=0] 2018-01-05T11:28:57,190 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,190 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_partitions : db=default tbl=cbo_t2 2018-01-05T11:28:57,190 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_partitions : db=default tbl=cbo_t2 2018-01-05T11:28:57,190 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: New partition values:[2014] 2018-01-05T11:28:57,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,199 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,203 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: load data local inpath '/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/cbo_t2.txt' into table cbo_t2 partition (dt='2014') 2018-01-05T11:28:57,203 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: LOAD 2018-01-05T11:28:57,203 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/cbo_t2.txt 2018-01-05T11:28:57,203 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@cbo_t2 2018-01-05T11:28:57,203 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@cbo_t2@dt=2014 2018-01-05T11:28:57,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,203 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:57,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_partitions_(String, String, List, EnvironmentContext, )=13, getTable_(String, String, )=4, getPartitionWithAuthInfo_(String, String, List, String, List, )=8} 2018-01-05T11:28:57,203 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112857_2d02aa99-3b53-4abf-a36d-ccd3e02f992a); Time taken: 0.099 seconds 2018-01-05T11:28:57,203 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:57,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,203 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:57,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query load data local inpath '/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/cbo_t2.txt' into table cbo_t2 partition (dt='2014') 2018-01-05T11:28:57,203 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.116 seconds 2018-01-05T11:28:57,203 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:57,203 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:57,203 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:57,203 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:57,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:57,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: load data local inpath '${hiveconf:test.data.dir}/cbo_t3.txt' into table cbo_t3 2018-01-05T11:28:57,204 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112857_ef1da066-c26f-4aa5-b47d-9de5d7694c87): load data local inpath '/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/cbo_t3.txt' into table cbo_t3 2018-01-05T11:28:57,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: load data local inpath '/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/cbo_t3.txt' into table cbo_t3 2018-01-05T11:28:57,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:57,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,204 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:57,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.LoadSemanticAnalyzer: file@null@/Users/djaiswal/hive/hive/data/files/cbo_t3.txt 2018-01-05T11:28:57,204 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,204 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t3 2018-01-05T11:28:57,204 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t3 2018-01-05T11:28:57,207 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,208 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,210 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:57,210 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,210 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:57,210 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,210 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:57,210 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=4, flushCache_()=0} 2018-01-05T11:28:57,210 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112857_ef1da066-c26f-4aa5-b47d-9de5d7694c87); Time taken: 0.006 seconds 2018-01-05T11:28:57,210 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:57,210 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,210 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112857_ef1da066-c26f-4aa5-b47d-9de5d7694c87): load data local inpath '/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/cbo_t3.txt' into table cbo_t3 2018-01-05T11:28:57,211 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,211 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: load data local inpath '/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/cbo_t3.txt' into table cbo_t3 2018-01-05T11:28:57,211 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: LOAD 2018-01-05T11:28:57,211 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/cbo_t3.txt 2018-01-05T11:28:57,211 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@cbo_t3 2018-01-05T11:28:57,211 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,212 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,212 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,212 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,212 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MOVE] in serial mode 2018-01-05T11:28:57,212 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@a67e46, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:57,212 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:57,212 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,212 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:57,212 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:57,212 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:57,212 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:57,212 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,212 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,212 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:57,212 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:57,212 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,212 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:57,212 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,212 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:57,212 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:57,212 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,212 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Loading data to table default.cbo_t3 from file:/Users/djaiswal/hive/hive/data/files/cbo_t3.txt 2018-01-05T11:28:57,212 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,213 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:57,213 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:57,213 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:57,213 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:57,213 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,213 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,213 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,214 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,214 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,214 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:57,214 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:57,214 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,214 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:57,214 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,214 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t3 2018-01-05T11:28:57,214 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t3 2018-01-05T11:28:57,218 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,218 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,219 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,219 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t3 2018-01-05T11:28:57,219 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t3 2018-01-05T11:28:57,222 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] FileOperations: moving file:/Users/djaiswal/hive/hive/data/files/cbo_t3.txt to file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3 (replace = KEEP_EXISTING) 2018-01-05T11:28:57,225 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:57,225 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:57,259 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Moved src: file:/Users/djaiswal/hive/hive/data/files/cbo_t3.txt, to dest: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3/cbo_t3.txt 2018-01-05T11:28:57,259 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,259 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=cbo_t3 newtbl=cbo_t3 2018-01-05T11:28:57,259 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=cbo_t3 newtbl=cbo_t3 2018-01-05T11:28:57,261 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,272 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,273 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,273 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@cbo_t3) Type=TABLE WriteType=INSERT because WriteEntity(default@cbo_t3) Type=TABLE WriteType=INSERT is present 2018-01-05T11:28:57,273 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-1:STATS] in serial mode 2018-01-05T11:28:57,273 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@460e17b8, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:57,273 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:57,273 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,273 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:57,273 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:57,273 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:57,273 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:57,273 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,273 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:57,273 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:57,274 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,274 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:57,274 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:57,274 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:57,274 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:57,274 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,274 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,274 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,275 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,275 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,275 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:57,275 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:57,275 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,276 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:57,276 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,276 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t3 2018-01-05T11:28:57,276 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t3 2018-01-05T11:28:57,279 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,280 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,281 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,281 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=cbo_t3 newtbl=cbo_t3 2018-01-05T11:28:57,281 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=cbo_t3 newtbl=cbo_t3 2018-01-05T11:28:57,284 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,293 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,294 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,294 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.cbo_t3 stats: [numFiles=1, numRows=0, totalSize=282, rawDataSize=0] 2018-01-05T11:28:57,294 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,294 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,294 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: load data local inpath '/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/cbo_t3.txt' into table cbo_t3 2018-01-05T11:28:57,294 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: LOAD 2018-01-05T11:28:57,294 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/cbo_t3.txt 2018-01-05T11:28:57,294 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@cbo_t3 2018-01-05T11:28:57,294 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,294 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,294 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:57,294 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=13, getTable_(String, String, )=4} 2018-01-05T11:28:57,294 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112857_ef1da066-c26f-4aa5-b47d-9de5d7694c87); Time taken: 0.084 seconds 2018-01-05T11:28:57,294 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:57,294 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,294 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:57,294 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,294 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query load data local inpath '/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/cbo_t3.txt' into table cbo_t3 2018-01-05T11:28:57,294 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.091 seconds 2018-01-05T11:28:57,294 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:57,294 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:57,294 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:57,294 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:57,295 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,295 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:57,295 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,295 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,295 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,295 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: CREATE TABLE part( p_partkey INT, p_name STRING, p_mfgr STRING, p_brand STRING, p_type STRING, p_size INT, p_container STRING, p_retailprice DOUBLE, p_comment STRING ) 2018-01-05T11:28:57,295 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112857_b7bf4278-2cca-4dc6-a65a-b26cf9d26b4f): CREATE TABLE part( p_partkey INT, p_name STRING, p_mfgr STRING, p_brand STRING, p_type STRING, p_size INT, p_container STRING, p_retailprice DOUBLE, p_comment STRING ) 2018-01-05T11:28:57,295 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,295 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: CREATE TABLE part( p_partkey INT, p_name STRING, p_mfgr STRING, p_brand STRING, p_type STRING, p_size INT, p_container STRING, p_retailprice DOUBLE, p_comment STRING ) 2018-01-05T11:28:57,296 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:57,296 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,296 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,296 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,296 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,296 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:57,296 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:28:57,296 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Creating table default.part position=13 2018-01-05T11:28:57,296 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,296 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_database: default 2018-01-05T11:28:57,296 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_database: default 2018-01-05T11:28:57,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:57,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,297 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:57,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: validation start 2018-01-05T11:28:57,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:28:57,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Not a partition. 2018-01-05T11:28:57,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,297 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:57,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,297 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:57,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getDatabase_(String, )=1, flushCache_()=0} 2018-01-05T11:28:57,297 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112857_b7bf4278-2cca-4dc6-a65a-b26cf9d26b4f); Time taken: 0.002 seconds 2018-01-05T11:28:57,297 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:57,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,297 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112857_b7bf4278-2cca-4dc6-a65a-b26cf9d26b4f): CREATE TABLE part( p_partkey INT, p_name STRING, p_mfgr STRING, p_brand STRING, p_type STRING, p_size INT, p_container STRING, p_retailprice DOUBLE, p_comment STRING ) 2018-01-05T11:28:57,299 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,299 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: CREATE TABLE part( p_partkey INT, p_name STRING, p_mfgr STRING, p_brand STRING, p_type STRING, p_size INT, p_container STRING, p_retailprice DOUBLE, p_comment STRING ) 2018-01-05T11:28:57,299 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: CREATETABLE 2018-01-05T11:28:57,299 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: database:default 2018-01-05T11:28:57,299 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@part 2018-01-05T11:28:57,299 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,299 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,299 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,299 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,299 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:57,299 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Found class for org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe 2018-01-05T11:28:57,300 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: creating table default.part on file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part 2018-01-05T11:28:57,300 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,300 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: create_table: Table(tableName:part, dbName:default, owner:hive_test_user, createTime:1515180537, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:p_partkey, type:int, comment:null), FieldSchema(name:p_name, type:string, comment:null), FieldSchema(name:p_mfgr, type:string, comment:null), FieldSchema(name:p_brand, type:string, comment:null), FieldSchema(name:p_type, type:string, comment:null), FieldSchema(name:p_size, type:int, comment:null), FieldSchema(name:p_container, type:string, comment:null), FieldSchema(name:p_retailprice, type:double, comment:null), FieldSchema(name:p_comment, type:string, comment:null)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"p_brand":"true","p_comment":"true","p_container":"true","p_mfgr":"true","p_name":"true","p_partkey":"true","p_retailprice":"true","p_size":"true","p_type":"true"}}, numFiles=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:57,300 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=create_table: Table(tableName:part, dbName:default, owner:hive_test_user, createTime:1515180537, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:p_partkey, type:int, comment:null), FieldSchema(name:p_name, type:string, comment:null), FieldSchema(name:p_mfgr, type:string, comment:null), FieldSchema(name:p_brand, type:string, comment:null), FieldSchema(name:p_type, type:string, comment:null), FieldSchema(name:p_size, type:int, comment:null), FieldSchema(name:p_container, type:string, comment:null), FieldSchema(name:p_retailprice, type:double, comment:null), FieldSchema(name:p_comment, type:string, comment:null)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"p_brand":"true","p_comment":"true","p_container":"true","p_mfgr":"true","p_name":"true","p_partkey":"true","p_retailprice":"true","p_size":"true","p_type":"true"}}, numFiles=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:57,301 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hive-site.xml 2018-01-05T11:28:57,302 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hivemetastore-site.xml 2018-01-05T11:28:57,302 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Unable to find config file metastore-site.xml 2018-01-05T11:28:57,302 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file null 2018-01-05T11:28:57,314 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:57,314 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: Location: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part specified for non-external table:part 2018-01-05T11:28:57,315 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part 2018-01-05T11:28:57,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,343 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,344 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,344 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@part) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@part) Type=TABLE WriteType=DDL_NO_LOCK is present 2018-01-05T11:28:57,344 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,344 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,344 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: CREATE TABLE part( p_partkey INT, p_name STRING, p_mfgr STRING, p_brand STRING, p_type STRING, p_size INT, p_container STRING, p_retailprice DOUBLE, p_comment STRING ) 2018-01-05T11:28:57,344 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: CREATETABLE 2018-01-05T11:28:57,344 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: database:default 2018-01-05T11:28:57,344 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@part 2018-01-05T11:28:57,344 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,344 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,345 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:57,345 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {createTable_(Table, )=44, isCompatibleWith_(Configuration, )=0} 2018-01-05T11:28:57,345 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112857_b7bf4278-2cca-4dc6-a65a-b26cf9d26b4f); Time taken: 0.047 seconds 2018-01-05T11:28:57,345 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:57,345 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,345 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:57,345 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,345 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query CREATE TABLE part( p_partkey INT, p_name STRING, p_mfgr STRING, p_brand STRING, p_type STRING, p_size INT, p_container STRING, p_retailprice DOUBLE, p_comment STRING ) 2018-01-05T11:28:57,345 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.05 seconds 2018-01-05T11:28:57,345 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:57,345 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:57,345 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:57,345 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:57,345 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,345 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:57,345 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,345 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,345 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,345 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: LOAD DATA LOCAL INPATH '${hiveconf:test.data.dir}/part_tiny.txt' overwrite into table part 2018-01-05T11:28:57,345 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112857_a50bf9f9-5264-4547-af33-f617a6bc6794): LOAD DATA LOCAL INPATH '/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/part_tiny.txt' overwrite into table part 2018-01-05T11:28:57,345 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,345 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: LOAD DATA LOCAL INPATH '/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/part_tiny.txt' overwrite into table part 2018-01-05T11:28:57,346 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:57,346 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,346 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,346 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,346 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,346 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:57,346 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.LoadSemanticAnalyzer: file@null@/Users/djaiswal/hive/hive/data/files/part_tiny.txt 2018-01-05T11:28:57,346 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,346 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=part 2018-01-05T11:28:57,346 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=part 2018-01-05T11:28:57,350 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,350 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,352 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:57,352 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,352 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:57,353 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,353 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:57,353 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=4, flushCache_()=0} 2018-01-05T11:28:57,353 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112857_a50bf9f9-5264-4547-af33-f617a6bc6794); Time taken: 0.008 seconds 2018-01-05T11:28:57,353 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:57,353 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,353 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112857_a50bf9f9-5264-4547-af33-f617a6bc6794): LOAD DATA LOCAL INPATH '/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/part_tiny.txt' overwrite into table part 2018-01-05T11:28:57,355 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,355 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: LOAD DATA LOCAL INPATH '/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/part_tiny.txt' overwrite into table part 2018-01-05T11:28:57,355 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: LOAD 2018-01-05T11:28:57,355 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/part_tiny.txt 2018-01-05T11:28:57,355 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@part 2018-01-05T11:28:57,355 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,355 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,355 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,355 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,355 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MOVE] in serial mode 2018-01-05T11:28:57,355 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@18663aa5, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:57,355 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:57,355 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,355 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:57,355 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:57,355 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:57,355 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:57,355 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,355 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:57,355 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Loading data to table default.part from file:/Users/djaiswal/hive/hive/data/files/part_tiny.txt 2018-01-05T11:28:57,356 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,356 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:57,356 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:57,356 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:57,356 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:57,357 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,357 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,357 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,357 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,357 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,358 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:57,358 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:57,358 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,358 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:57,358 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,358 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=part 2018-01-05T11:28:57,358 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=part 2018-01-05T11:28:57,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,363 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=part 2018-01-05T11:28:57,363 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=part 2018-01-05T11:28:57,365 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,365 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,365 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] FileOperations: moving file:/Users/djaiswal/hive/hive/data/files/part_tiny.txt to file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part (replace = REPLACE_ALL) 2018-01-05T11:28:57,366 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] FileOperations: Deleting old paths for replace in file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part and old path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part 2018-01-05T11:28:57,366 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part/ and the destination path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part/ 2018-01-05T11:28:57,366 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part 2018-01-05T11:28:57,366 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/data/files/part_tiny.txt/ and the destination path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part/part_tiny.txt/ 2018-01-05T11:28:57,366 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part/part_tiny.txt/ and the destination path is /Users/djaiswal/hive/hive/data/files/part_tiny.txt/ 2018-01-05T11:28:57,395 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,395 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=part newtbl=part 2018-01-05T11:28:57,395 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=part newtbl=part 2018-01-05T11:28:57,398 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,402 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,403 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,403 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@part) Type=TABLE WriteType=INSERT_OVERWRITE because WriteEntity(default@part) Type=TABLE WriteType=INSERT_OVERWRITE is present 2018-01-05T11:28:57,403 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-1:STATS] in serial mode 2018-01-05T11:28:57,403 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@3b68430d, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:57,403 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:57,403 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,403 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:57,403 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:57,403 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:57,403 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:57,403 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,403 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:57,403 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:57,404 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,404 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:57,404 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:57,404 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:57,404 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:57,405 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,405 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,405 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,406 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,406 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,406 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:57,406 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:57,406 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,406 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:57,406 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,406 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=part 2018-01-05T11:28:57,406 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=part 2018-01-05T11:28:57,408 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,409 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,410 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,410 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=part newtbl=part 2018-01-05T11:28:57,410 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=part newtbl=part 2018-01-05T11:28:57,412 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,422 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,423 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,423 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.part stats: [numFiles=1, numRows=0, totalSize=3173, rawDataSize=0] 2018-01-05T11:28:57,423 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,423 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,423 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: LOAD DATA LOCAL INPATH '/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/part_tiny.txt' overwrite into table part 2018-01-05T11:28:57,423 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: LOAD 2018-01-05T11:28:57,423 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/part_tiny.txt 2018-01-05T11:28:57,423 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@part 2018-01-05T11:28:57,423 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,423 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,423 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:57,423 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=13, getTable_(String, String, )=3} 2018-01-05T11:28:57,423 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112857_a50bf9f9-5264-4547-af33-f617a6bc6794); Time taken: 0.07 seconds 2018-01-05T11:28:57,423 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:57,423 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,423 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:57,423 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,423 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query LOAD DATA LOCAL INPATH '/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/part_tiny.txt' overwrite into table part 2018-01-05T11:28:57,423 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.078 seconds 2018-01-05T11:28:57,423 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:57,423 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:57,423 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:57,423 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:57,423 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,423 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:57,423 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,423 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,423 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,423 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: CREATE TABLE lineitem (L_ORDERKEY INT, L_PARTKEY INT, L_SUPPKEY INT, L_LINENUMBER INT, L_QUANTITY DOUBLE, L_EXTENDEDPRICE DOUBLE, L_DISCOUNT DOUBLE, L_TAX DOUBLE, L_RETURNFLAG STRING, L_LINESTATUS STRING, l_shipdate STRING, L_COMMITDATE STRING, L_RECEIPTDATE STRING, L_SHIPINSTRUCT STRING, L_SHIPMODE STRING, L_COMMENT STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' 2018-01-05T11:28:57,423 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112857_e412b231-275c-4290-a24e-7144d20b79bf): CREATE TABLE lineitem (L_ORDERKEY INT, L_PARTKEY INT, L_SUPPKEY INT, L_LINENUMBER INT, L_QUANTITY DOUBLE, L_EXTENDEDPRICE DOUBLE, L_DISCOUNT DOUBLE, L_TAX DOUBLE, L_RETURNFLAG STRING, L_LINESTATUS STRING, l_shipdate STRING, L_COMMITDATE STRING, L_RECEIPTDATE STRING, L_SHIPINSTRUCT STRING, L_SHIPMODE STRING, L_COMMENT STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' 2018-01-05T11:28:57,424 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,424 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: CREATE TABLE lineitem (L_ORDERKEY INT, L_PARTKEY INT, L_SUPPKEY INT, L_LINENUMBER INT, L_QUANTITY DOUBLE, L_EXTENDEDPRICE DOUBLE, L_DISCOUNT DOUBLE, L_TAX DOUBLE, L_RETURNFLAG STRING, L_LINESTATUS STRING, l_shipdate STRING, L_COMMITDATE STRING, L_RECEIPTDATE STRING, L_SHIPINSTRUCT STRING, L_SHIPMODE STRING, L_COMMENT STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' 2018-01-05T11:28:57,424 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:57,424 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,424 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,424 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,424 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,424 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:57,424 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:28:57,424 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Creating table default.lineitem position=13 2018-01-05T11:28:57,424 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,425 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_database: default 2018-01-05T11:28:57,425 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_database: default 2018-01-05T11:28:57,425 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:57,425 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,425 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,425 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:57,425 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: validation start 2018-01-05T11:28:57,425 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:28:57,425 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Not a partition. 2018-01-05T11:28:57,425 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,425 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:57,425 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,425 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:57,425 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getDatabase_(String, )=1, flushCache_()=0} 2018-01-05T11:28:57,425 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112857_e412b231-275c-4290-a24e-7144d20b79bf); Time taken: 0.002 seconds 2018-01-05T11:28:57,425 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:57,425 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,425 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112857_e412b231-275c-4290-a24e-7144d20b79bf): CREATE TABLE lineitem (L_ORDERKEY INT, L_PARTKEY INT, L_SUPPKEY INT, L_LINENUMBER INT, L_QUANTITY DOUBLE, L_EXTENDEDPRICE DOUBLE, L_DISCOUNT DOUBLE, L_TAX DOUBLE, L_RETURNFLAG STRING, L_LINESTATUS STRING, l_shipdate STRING, L_COMMITDATE STRING, L_RECEIPTDATE STRING, L_SHIPINSTRUCT STRING, L_SHIPMODE STRING, L_COMMENT STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' 2018-01-05T11:28:57,426 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,427 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: CREATE TABLE lineitem (L_ORDERKEY INT, L_PARTKEY INT, L_SUPPKEY INT, L_LINENUMBER INT, L_QUANTITY DOUBLE, L_EXTENDEDPRICE DOUBLE, L_DISCOUNT DOUBLE, L_TAX DOUBLE, L_RETURNFLAG STRING, L_LINESTATUS STRING, l_shipdate STRING, L_COMMITDATE STRING, L_RECEIPTDATE STRING, L_SHIPINSTRUCT STRING, L_SHIPMODE STRING, L_COMMENT STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' 2018-01-05T11:28:57,427 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: CREATETABLE 2018-01-05T11:28:57,427 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: database:default 2018-01-05T11:28:57,427 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@lineitem 2018-01-05T11:28:57,427 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,427 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,427 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,427 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,427 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:28:57,427 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Found class for org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe 2018-01-05T11:28:57,427 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: creating table default.lineitem on file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem 2018-01-05T11:28:57,427 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,427 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: create_table: Table(tableName:lineitem, dbName:default, owner:hive_test_user, createTime:1515180537, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:l_orderkey, type:int, comment:null), FieldSchema(name:l_partkey, type:int, comment:null), FieldSchema(name:l_suppkey, type:int, comment:null), FieldSchema(name:l_linenumber, type:int, comment:null), FieldSchema(name:l_quantity, type:double, comment:null), FieldSchema(name:l_extendedprice, type:double, comment:null), FieldSchema(name:l_discount, type:double, comment:null), FieldSchema(name:l_tax, type:double, comment:null), FieldSchema(name:l_returnflag, type:string, comment:null), FieldSchema(name:l_linestatus, type:string, comment:null), FieldSchema(name:l_shipdate, type:string, comment:null), FieldSchema(name:l_commitdate, type:string, comment:null), FieldSchema(name:l_receiptdate, type:string, comment:null), FieldSchema(name:l_shipinstruct, type:string, comment:null), FieldSchema(name:l_shipmode, type:string, comment:null), FieldSchema(name:l_comment, type:string, comment:null)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{field.delim=|, serialization.format=|}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"l_comment":"true","l_commitdate":"true","l_discount":"true","l_extendedprice":"true","l_linenumber":"true","l_linestatus":"true","l_orderkey":"true","l_partkey":"true","l_quantity":"true","l_receiptdate":"true","l_returnflag":"true","l_shipdate":"true","l_shipinstruct":"true","l_shipmode":"true","l_suppkey":"true","l_tax":"true"}}, numFiles=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:57,428 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=create_table: Table(tableName:lineitem, dbName:default, owner:hive_test_user, createTime:1515180537, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:l_orderkey, type:int, comment:null), FieldSchema(name:l_partkey, type:int, comment:null), FieldSchema(name:l_suppkey, type:int, comment:null), FieldSchema(name:l_linenumber, type:int, comment:null), FieldSchema(name:l_quantity, type:double, comment:null), FieldSchema(name:l_extendedprice, type:double, comment:null), FieldSchema(name:l_discount, type:double, comment:null), FieldSchema(name:l_tax, type:double, comment:null), FieldSchema(name:l_returnflag, type:string, comment:null), FieldSchema(name:l_linestatus, type:string, comment:null), FieldSchema(name:l_shipdate, type:string, comment:null), FieldSchema(name:l_commitdate, type:string, comment:null), FieldSchema(name:l_receiptdate, type:string, comment:null), FieldSchema(name:l_shipinstruct, type:string, comment:null), FieldSchema(name:l_shipmode, type:string, comment:null), FieldSchema(name:l_comment, type:string, comment:null)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{field.delim=|, serialization.format=|}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"l_comment":"true","l_commitdate":"true","l_discount":"true","l_extendedprice":"true","l_linenumber":"true","l_linestatus":"true","l_orderkey":"true","l_partkey":"true","l_quantity":"true","l_receiptdate":"true","l_returnflag":"true","l_shipdate":"true","l_shipinstruct":"true","l_shipmode":"true","l_suppkey":"true","l_tax":"true"}}, numFiles=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:57,428 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hive-site.xml 2018-01-05T11:28:57,429 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hivemetastore-site.xml 2018-01-05T11:28:57,429 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Unable to find config file metastore-site.xml 2018-01-05T11:28:57,429 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file null 2018-01-05T11:28:57,438 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:57,438 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: Location: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem specified for non-external table:lineitem 2018-01-05T11:28:57,438 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem 2018-01-05T11:28:57,456 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,458 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,458 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,458 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@lineitem) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@lineitem) Type=TABLE WriteType=DDL_NO_LOCK is present 2018-01-05T11:28:57,458 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,458 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,459 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: CREATE TABLE lineitem (L_ORDERKEY INT, L_PARTKEY INT, L_SUPPKEY INT, L_LINENUMBER INT, L_QUANTITY DOUBLE, L_EXTENDEDPRICE DOUBLE, L_DISCOUNT DOUBLE, L_TAX DOUBLE, L_RETURNFLAG STRING, L_LINESTATUS STRING, l_shipdate STRING, L_COMMITDATE STRING, L_RECEIPTDATE STRING, L_SHIPINSTRUCT STRING, L_SHIPMODE STRING, L_COMMENT STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' 2018-01-05T11:28:57,459 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: CREATETABLE 2018-01-05T11:28:57,459 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: database:default 2018-01-05T11:28:57,459 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@lineitem 2018-01-05T11:28:57,459 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,459 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,459 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:57,459 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {createTable_(Table, )=31, isCompatibleWith_(Configuration, )=0} 2018-01-05T11:28:57,459 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112857_e412b231-275c-4290-a24e-7144d20b79bf); Time taken: 0.034 seconds 2018-01-05T11:28:57,459 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:57,459 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,459 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:57,459 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,459 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query CREATE TABLE lineitem (L_ORDERKEY INT, L_PARTKEY INT, L_SUPPKEY INT, L_LINENUMBER INT, L_QUANTITY DOUBLE, L_EXTENDEDPRICE DOUBLE, L_DISCOUNT DOUBLE, L_TAX DOUBLE, L_RETURNFLAG STRING, L_LINESTATUS STRING, l_shipdate STRING, L_COMMITDATE STRING, L_RECEIPTDATE STRING, L_SHIPINSTRUCT STRING, L_SHIPMODE STRING, L_COMMENT STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' 2018-01-05T11:28:57,459 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.036 seconds 2018-01-05T11:28:57,459 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:57,459 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:57,459 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:57,459 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:57,459 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,459 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:57,459 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,459 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,459 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,459 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: LOAD DATA LOCAL INPATH '${hiveconf:test.data.dir}/lineitem.txt' OVERWRITE INTO TABLE lineitem 2018-01-05T11:28:57,459 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112857_bda51ad1-c49b-4691-b510-99f69d6aee83): LOAD DATA LOCAL INPATH '/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/lineitem.txt' OVERWRITE INTO TABLE lineitem 2018-01-05T11:28:57,459 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,459 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: LOAD DATA LOCAL INPATH '/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/lineitem.txt' OVERWRITE INTO TABLE lineitem 2018-01-05T11:28:57,459 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:57,459 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,459 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,460 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,460 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,460 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:57,460 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.LoadSemanticAnalyzer: file@null@/Users/djaiswal/hive/hive/data/files/lineitem.txt 2018-01-05T11:28:57,460 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,460 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=lineitem 2018-01-05T11:28:57,460 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=lineitem 2018-01-05T11:28:57,462 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,463 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,466 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:57,466 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,466 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:28:57,466 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,466 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:57,466 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=3, flushCache_()=0} 2018-01-05T11:28:57,466 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112857_bda51ad1-c49b-4691-b510-99f69d6aee83); Time taken: 0.007 seconds 2018-01-05T11:28:57,466 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:57,466 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,466 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112857_bda51ad1-c49b-4691-b510-99f69d6aee83): LOAD DATA LOCAL INPATH '/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/lineitem.txt' OVERWRITE INTO TABLE lineitem 2018-01-05T11:28:57,467 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,467 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: LOAD DATA LOCAL INPATH '/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/lineitem.txt' OVERWRITE INTO TABLE lineitem 2018-01-05T11:28:57,467 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: LOAD 2018-01-05T11:28:57,467 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/lineitem.txt 2018-01-05T11:28:57,467 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@lineitem 2018-01-05T11:28:57,467 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,467 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,467 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,467 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,467 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MOVE] in serial mode 2018-01-05T11:28:57,467 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@4761643c, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:57,467 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:57,467 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,467 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:57,467 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:57,467 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:57,467 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:57,467 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,467 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:57,468 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Loading data to table default.lineitem from file:/Users/djaiswal/hive/hive/data/files/lineitem.txt 2018-01-05T11:28:57,468 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,468 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:57,468 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:57,468 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:57,468 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:57,468 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,468 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,469 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,469 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,469 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,469 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:57,469 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:57,469 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,469 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:57,469 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,469 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=lineitem 2018-01-05T11:28:57,469 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=lineitem 2018-01-05T11:28:57,472 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,472 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,474 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,474 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=lineitem 2018-01-05T11:28:57,474 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=lineitem 2018-01-05T11:28:57,477 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,480 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] FileOperations: moving file:/Users/djaiswal/hive/hive/data/files/lineitem.txt to file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem (replace = REPLACE_ALL) 2018-01-05T11:28:57,480 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] FileOperations: Deleting old paths for replace in file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem and old path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem 2018-01-05T11:28:57,480 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem/ and the destination path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem/ 2018-01-05T11:28:57,480 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem 2018-01-05T11:28:57,480 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/data/files/lineitem.txt/ and the destination path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem/lineitem.txt/ 2018-01-05T11:28:57,480 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem/lineitem.txt/ and the destination path is /Users/djaiswal/hive/hive/data/files/lineitem.txt/ 2018-01-05T11:28:57,509 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,509 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=lineitem newtbl=lineitem 2018-01-05T11:28:57,509 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=lineitem newtbl=lineitem 2018-01-05T11:28:57,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,521 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,522 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,522 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@lineitem) Type=TABLE WriteType=INSERT_OVERWRITE because WriteEntity(default@lineitem) Type=TABLE WriteType=INSERT_OVERWRITE is present 2018-01-05T11:28:57,522 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-1:STATS] in serial mode 2018-01-05T11:28:57,523 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@6f6bc1e0, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:57,523 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:57,523 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,523 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:57,523 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:57,523 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:57,523 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:57,523 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,523 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:57,523 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:57,524 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,525 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:57,525 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:57,525 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:57,525 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:57,525 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,525 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,525 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,526 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,526 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,526 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:57,526 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:57,526 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,526 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:57,526 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,527 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=lineitem 2018-01-05T11:28:57,527 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=lineitem 2018-01-05T11:28:57,530 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,532 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,532 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=lineitem newtbl=lineitem 2018-01-05T11:28:57,532 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=lineitem newtbl=lineitem 2018-01-05T11:28:57,536 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,545 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,546 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,546 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.lineitem stats: [numFiles=1, numRows=0, totalSize=12099, rawDataSize=0] 2018-01-05T11:28:57,546 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,546 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,546 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: LOAD DATA LOCAL INPATH '/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/lineitem.txt' OVERWRITE INTO TABLE lineitem 2018-01-05T11:28:57,546 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: LOAD 2018-01-05T11:28:57,546 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/lineitem.txt 2018-01-05T11:28:57,546 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@lineitem 2018-01-05T11:28:57,546 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,546 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,546 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:57,546 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=14, getTable_(String, String, )=5} 2018-01-05T11:28:57,546 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112857_bda51ad1-c49b-4691-b510-99f69d6aee83); Time taken: 0.08 seconds 2018-01-05T11:28:57,546 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:57,546 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,546 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:57,546 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,546 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query LOAD DATA LOCAL INPATH '/Users/djaiswal/hive/hive/itests/qtest/../..//data/files/lineitem.txt' OVERWRITE INTO TABLE lineitem 2018-01-05T11:28:57,546 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.087 seconds 2018-01-05T11:28:57,546 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:57,546 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:57,546 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:57,546 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:57,546 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,546 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:57,547 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,547 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,547 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,547 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: create table src_cbo as select * from src 2018-01-05T11:28:57,547 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112857_126fd009-65af-46a3-94f4-76213447cc87): create table src_cbo as select * from src 2018-01-05T11:28:57,547 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,547 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: create table src_cbo as select * from src 2018-01-05T11:28:57,547 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:57,547 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,547 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,547 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,547 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,547 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:57,547 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:28:57,548 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Creating table default.src_cbo position=13 2018-01-05T11:28:57,548 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,548 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_database: default 2018-01-05T11:28:57,548 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_database: default 2018-01-05T11:28:57,548 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:57,548 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,548 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,548 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,548 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_cbo 2018-01-05T11:28:57,548 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_cbo 2018-01-05T11:28:57,549 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,549 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,549 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Completed phase 1 of Semantic Analysis 2018-01-05T11:28:57,549 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:28:57,549 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,549 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src 2018-01-05T11:28:57,549 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src 2018-01-05T11:28:57,551 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,552 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,552 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:28:57,552 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:28:57,552 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,552 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_database: default 2018-01-05T11:28:57,552 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_database: default 2018-01-05T11:28:57,553 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:57,553 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,553 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,553 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/.hive-staging_hive_2018-01-05_11-28-57_547_735783970339321393-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse 2018-01-05T11:28:57,553 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/.hive-staging_hive_2018-01-05_11-28-57_547_735783970339321393-1 2018-01-05T11:28:57,566 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Completed getting MetaData in Semantic Analysis 2018-01-05T11:28:57,568 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,569 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,569 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_not_null_constraints : db=default tbl=src 2018-01-05T11:28:57,569 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_not_null_constraints : db=default tbl=src 2018-01-05T11:28:57,569 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,569 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,569 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,569 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=src 2018-01-05T11:28:57,569 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=src 2018-01-05T11:28:57,570 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,570 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,570 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,570 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=src 2018-01-05T11:28:57,570 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=src 2018-01-05T11:28:57,570 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,570 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,570 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,570 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_unique_constraints : db=default tbl=src 2018-01-05T11:28:57,570 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_unique_constraints : db=default tbl=src 2018-01-05T11:28:57,571 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,571 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,571 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,571 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=src 2018-01-05T11:28:57,571 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=src 2018-01-05T11:28:57,571 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,571 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,572 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:28:57,572 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,602 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] calcite.sql2rel: Plan after trimming unused fields HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.src]], table:alias=[src]) 2018-01-05T11:28:57,602 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Plan before removing subquery: HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.src]], table:alias=[src]) 2018-01-05T11:28:57,603 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Plan just after removing subquery: HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.src]], table:alias=[src]) 2018-01-05T11:28:57,603 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Plan after decorrelation: HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.src]], table:alias=[src]) 2018-01-05T11:28:57,603 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,603 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,603 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,603 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,603 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,604 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,604 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,604 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,604 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,604 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,604 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,618 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,618 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,618 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,618 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,619 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] calcite.sql2rel: Plan after trimming unused fields HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.src]], table:alias=[src]) 2018-01-05T11:28:57,619 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,619 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,620 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,620 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,620 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,620 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,621 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,621 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,621 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,621 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,624 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,624 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,624 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,624 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,625 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,625 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] translator.PlanModifierForASTConv: Original plan for PlanModifier HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.src]], table:alias=[src]) 2018-01-05T11:28:57,625 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] translator.PlanModifierForASTConv: Plan after nested convertOpTree HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.src]], table:alias=[src]) 2018-01-05T11:28:57,625 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] translator.PlanModifierForASTConv: Plan after propagating order HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.src]], table:alias=[src]) 2018-01-05T11:28:57,625 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] translator.PlanModifierForASTConv: Plan after fixTopOBSchema HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.src]], table:alias=[src]) 2018-01-05T11:28:57,625 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] translator.PlanModifierForASTConv: Final plan after modifier HiveProject(src.key=[$0], src.value=[$1]) HiveTableScan(table=[[default.src]], table:alias=[src]) 2018-01-05T11:28:57,625 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Creating table default.src_cbo position=13 2018-01-05T11:28:57,625 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,625 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_database: default 2018-01-05T11:28:57,626 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_database: default 2018-01-05T11:28:57,626 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:57,626 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,626 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,627 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,627 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_cbo 2018-01-05T11:28:57,627 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_cbo 2018-01-05T11:28:57,627 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,627 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,627 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:28:57,627 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,627 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src 2018-01-05T11:28:57,628 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src 2018-01-05T11:28:57,631 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,632 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,632 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:28:57,632 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:28:57,632 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,632 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_database: default 2018-01-05T11:28:57,632 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_database: default 2018-01-05T11:28:57,633 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:57,633 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,633 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,634 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Table Plan for src TS[0] 2018-01-05T11:28:57,634 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: RR before GB src{(key,key: string)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB src{(key,key: string)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:28:57,634 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (. (tok_table_or_col src) key) src.key) (tok_selexpr (. (tok_table_or_col src) value) src.value)) 2018-01-05T11:28:57,634 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: genSelectPlan: input = src{(key,key: string)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} starRr = null 2018-01-05T11:28:57,634 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Select Plan row schema: null{(src.key,_col0: string)(src.value,_col1: string)} 2018-01-05T11:28:57,634 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:28:57,634 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.BaseSemanticAnalyzer: Replacing src.key (produced by CBO) by key 2018-01-05T11:28:57,634 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.BaseSemanticAnalyzer: Replacing src.value (produced by CBO) by value 2018-01-05T11:28:57,648 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_547_735783970339321393-1/-mr-10003 2018-01-05T11:28:57,649 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,649 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_database: default 2018-01-05T11:28:57,649 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_database: default 2018-01-05T11:28:57,649 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:57,649 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,649 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,649 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/.hive-staging_hive_2018-01-05_11-28-57_547_735783970339321393-1/-ext-10001 row schema: null{(src.key,_col0: string)(src.value,_col1: string)} 2018-01-05T11:28:57,649 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Body Plan for Query Block null 2018-01-05T11:28:57,649 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:28:57,649 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: CBO Succeeded; optimized logical plan. 2018-01-05T11:28:57,649 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Before logical optimization TS[0]-SEL[1]-FS[2] 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] lineage.Generator: Time taken for lineage transform=0 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,650 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for FS(2) 2018-01-05T11:28:57,650 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(1) 2018-01-05T11:28:57,650 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.SimplePredicatePushDown: After PPD: TS[0]-SEL[1]-FS[2] 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,650 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,651 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,651 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,651 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,651 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,651 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,651 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,651 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,651 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: After logical optimization TS[0]-SEL[1]-FS[2] 2018-01-05T11:28:57,651 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,651 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,651 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,651 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,651 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,651 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,651 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=src 2018-01-05T11:28:57,651 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=src 2018-01-05T11:28:57,656 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 3.001803ms + 0.009981ms, the query is [select "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "TAB_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (...)] 2018-01-05T11:28:57,658 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,665 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting stats (Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE) on TS[0] 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (src): numRows: 500 dataSize: 89000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {value= colName: value colType: string countDistincts: 309 numNulls: 0 avgColLen: 6.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, key= colName: key colType: string countDistincts: 309 numNulls: 0 avgColLen: 2.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting stats (Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[1] 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-SEL[1]: numRows: 500 dataSize: 89000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: string countDistincts: 309 numNulls: 0 avgColLen: 2.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: string countDistincts: 309 numNulls: 0 avgColLen: 6.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-FS[2]: numRows: 500 dataSize: 89000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: string countDistincts: 309 numNulls: 0 avgColLen: 2.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: string countDistincts: 309 numNulls: 0 avgColLen: 6.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on TS[0] 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on SEL[1] 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FileSinkOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on FS[2] 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: FS, 2 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: SEL, 1 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:28:57,666 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Cycle free: true 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{src}key,value: string|{src}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src}block__offset__inside__file,INPUT__FILE__NAME: string|{src}input__file__name,ROW__ID: struct|{src}row__id) 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(_col0: string|{null}src.key,_col1: string|{null}src.value) 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value]) 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[2] with rs:(key: string|{},value: string|{}) 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:57,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[2] 2018-01-05T11:28:57,667 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,667 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_database: default 2018-01-05T11:28:57,667 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_database: default 2018-01-05T11:28:57,667 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:57,667 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,667 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,667 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,667 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_database: default 2018-01-05T11:28:57,667 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_database: default 2018-01-05T11:28:57,667 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:57,667 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,667 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,668 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,668 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: TS[0] 2018-01-05T11:28:57,668 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: FS[2] 2018-01-05T11:28:57,668 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:28:57,668 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src of table src 2018-01-05T11:28:57,668 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src 2018-01-05T11:28:57,668 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: FS[2] 2018-01-05T11:28:57,668 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Found class for org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe 2018-01-05T11:28:57,668 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:28:57,668 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,668 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,668 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:57,668 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:57,669 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:57,669 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:57,669 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:57,669 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:57,669 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:28:57,669 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping vectorization 2018-01-05T11:28:57,669 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:28:57,669 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: llap mode: only 2018-01-05T11:28:57,669 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:28:57,669 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'key' 2018-01-05T11:28:57,669 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'value' 2018-01-05T11:28:57,669 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:57,669 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@26a9a09a can be done in LLAP 2018-01-05T11:28:57,669 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,670 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Completed plan generation 2018-01-05T11:28:57,670 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:57,670 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: validation start 2018-01-05T11:28:57,670 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:28:57,670 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.CalcitePlanner: Not a partition. 2018-01-05T11:28:57,670 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,670 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:src.key, type:string, comment:null), FieldSchema(name:src.value, type:string, comment:null)], properties:null) 2018-01-05T11:28:57,670 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,670 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:57,670 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {getNotNullConstraints_(NotNullConstraintsRequest, )=0, isCompatibleWith_(Configuration, )=0, getDatabase_(String, )=3, getTable_(String, String, )=8, flushCache_()=0, getUniqueConstraints_(UniqueConstraintsRequest, )=1, getPrimaryKeys_(PrimaryKeysRequest, )=1, getTableColumnStatistics_(String, String, List, )=15, getForeignKeys_(ForeignKeysRequest, )=0} 2018-01-05T11:28:57,670 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112857_126fd009-65af-46a3-94f4-76213447cc87); Time taken: 0.123 seconds 2018-01-05T11:28:57,670 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:57,670 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,670 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112857_126fd009-65af-46a3-94f4-76213447cc87): create table src_cbo as select * from src 2018-01-05T11:28:57,671 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,671 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: create table src_cbo as select * from src 2018-01-05T11:28:57,671 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: CREATETABLE_AS_SELECT 2018-01-05T11:28:57,671 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@src 2018-01-05T11:28:57,671 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: database:default 2018-01-05T11:28:57,671 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@src_cbo 2018-01-05T11:28:57,671 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,671 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,671 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,671 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Query ID = djaiswal_20180105112857_126fd009-65af-46a3-94f4-76213447cc87 2018-01-05T11:28:57,672 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Total jobs = 1 2018-01-05T11:28:57,672 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,672 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:28:57,672 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-1:MAPRED] in serial mode 2018-01-05T11:28:57,686 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_547_735783970339321393-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:28:57,712 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:28:57,712 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:28:57,712 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:28:57,712 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112857_126fd009-65af-46a3-94f4-76213447cc87 2018-01-05T11:28:57,712 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Session is already open 2018-01-05T11:28:57,712 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:28:57,712 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,712 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Dag name: create table src_cbo as select * from src (Stage-1) 2018-01-05T11:28:57,712 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: DagInfo: {"context":"Hive","description":"\n\ncreate table src_cbo as select * from src"} 2018-01-05T11:28:57,712 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112857_126fd009-65af-46a3-94f4-76213447cc87 with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:28:57,712 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,735 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:28:57,735 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,735 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:28:57,736 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,736 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 2.30KB 2018-01-05T11:28:57,742 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_547_735783970339321393-1/-mr-10003 2018-01-05T11:28:57,753 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_547_735783970339321393-1/-mr-10003 2018-01-05T11:28:57,755 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src 2018-01-05T11:28:57,755 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,755 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,755 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,755 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=create table src_cbo as select * from src (Stage-1), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112857_126fd009-65af-46a3-94f4-76213447cc87 } 2018-01-05T11:28:57,756 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:57,756 DEBUG [AMReporterQueueDrainer] impl.AMReporter: Removing am 10.22.8.182:49824 with last associated dag QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=11} from heartbeat with taskCount=0, amFailed=false, isDone=true 2018-01-05T11:28:57,757 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: create table src_cbo as select * from src (Stage-1) 2018-01-05T11:28:57,757 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:28:57,758 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_21, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_21.dot 2018-01-05T11:28:57,758 INFO [IPC Server handler 1 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_21 2018-01-05T11:28:57,758 WARN [IPC Server handler 1 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:57,758 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Running DAG: create table src_cbo as select * from src (Stage-1), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112857_126fd009-65af-46a3-94f4-76213447cc87 } 2018-01-05T11:28:57,781 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_21][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_21, submitTime=1515180537757, queueName=null 2018-01-05T11:28:57,782 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_21_00 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_21_00 [Map 1], Default Resources= 2018-01-05T11:28:57,782 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_21_00 [Map 1] 2018-01-05T11:28:57,782 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_21_00 [Map 1] 2018-01-05T11:28:57,782 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:28:57,782 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_21_00 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:57,782 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:28:57,782 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_21][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_21, initTime=1515180537782 2018-01-05T11:28:57,782 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: dag_1515180518813_0001_21 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:28:57,782 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:28:57,782 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_21][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_21, startTime=1515180537782 2018-01-05T11:28:57,782 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_21 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:28:57,782 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {src={InputName=src}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:28:57,782 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: src, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:28:57,782 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_21_00 [Map 1] 2018-01-05T11:28:57,782 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_21, dagName=create table src_cbo as select * from src (Stage-1) 2018-01-05T11:28:57,782 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,782 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,782 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,786 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_21_00 [Map 1] 2018-01-05T11:28:57,786 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_21_00 [Map 1] 2018-01-05T11:28:57,786 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:57,786 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_21_00 [Map 1] 2018-01-05T11:28:57,789 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:28:57,789 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@5bd4fd5f 2018-01-05T11:28:57,789 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_547_735783970339321393-1/djaiswal/_tez_scratch_dir/b6fc96a2-b323-4a47-b68f-337335ce5b67/map.xml 2018-01-05T11:28:57,789 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_547_735783970339321393-1/djaiswal/_tez_scratch_dir/b6fc96a2-b323-4a47-b68f-337335ce5b67/map.xml 2018-01-05T11:28:57,789 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_547_735783970339321393-1/djaiswal/_tez_scratch_dir/b6fc96a2-b323-4a47-b68f-337335ce5b67/map.xml 2018-01-05T11:28:57,789 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:57,789 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:57,789 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:57,789 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:57,789 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 2.30KB 2018-01-05T11:28:57,790 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_21_00 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:28:57,790 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: src on vertex vertex_1515180518813_0001_21_00 [Map 1] 2018-01-05T11:28:57,790 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:28:57,790 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:57,790 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:28:57,790 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:57,790 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_547_735783970339321393-1/djaiswal/_tez_scratch_dir/b6fc96a2-b323-4a47-b68f-337335ce5b67/map.xml 2018-01-05T11:28:57,790 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:57,790 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias src 2018-01-05T11:28:57,790 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src 2018-01-05T11:28:57,790 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src 2018-01-05T11:28:57,790 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 0,1 2018-01-05T11:28:57,790 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = key,value 2018-01-05T11:28:57,790 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src 2018-01-05T11:28:57,790 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:57,790 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:57,790 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:57,790 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:57,804 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Time taken to get FileStatuses: 13 2018-01-05T11:28:57,804 INFO [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:28:57,804 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 13 2018-01-05T11:28:57,804 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 1 2018-01-05T11:28:57,804 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:57,804 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 1. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:28:57,804 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_547_735783970339321393-1/djaiswal/_tez_scratch_dir/b6fc96a2-b323-4a47-b68f-337335ce5b67/map.xml 2018-01-05T11:28:57,804 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_547_735783970339321393-1/djaiswal/_tez_scratch_dir/b6fc96a2-b323-4a47-b68f-337335ce5b67/map.xml 2018-01-05T11:28:57,804 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_547_735783970339321393-1/djaiswal/_tez_scratch_dir/b6fc96a2-b323-4a47-b68f-337335ce5b67/map.xml 2018-01-05T11:28:57,804 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:57,804 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:57,804 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:57,805 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:57,805 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 2.30KB 2018-01-05T11:28:57,805 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src/kv1.txt to src new group? true 2018-01-05T11:28:57,805 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:28:57,805 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:28:57,805 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:28:57,805 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 107 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 5812 Original splits: 1 2018-01-05T11:28:57,805 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Using original number of splits: 1 desired splits: 1 2018-01-05T11:28:57,805 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 1 grouped split count is 1, for bucket: 1 2018-01-05T11:28:57,805 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:28:57,805 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: src on vertex vertex_1515180518813_0001_21_00 [Map 1] 2018-01-05T11:28:57,806 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_21_00 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:57,806 INFO [App Shared Pool - #0] impl.VertexImpl: Got updated RootInputsSpecs: {src=forAllWorkUnits=true, update=[1]} 2018-01-05T11:28:57,806 INFO [App Shared Pool - #0] impl.VertexImpl: Vertex vertex_1515180518813_0001_21_00 [Map 1] parallelism set to 1 2018-01-05T11:28:57,806 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_21_00 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:57,806 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_21_00 [Map 1], numAdditionalOutputs=1 2018-01-05T11:28:57,806 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_21][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_21_00, initRequestedTime=1515180537782, initedTime=1515180537806, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:57,806 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_21_00 [Map 1] with distanceFromRoot: 0 2018-01-05T11:28:57,806 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_21_00 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:28:57,806 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_21][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_21_00, startRequestedTime=1515180537790, startedTime=1515180537806 2018-01-05T11:28:57,806 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_21][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_21_00, reconfigureDoneTime=1515180537806, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:28:57,806 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_21_00 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:57,806 INFO [App Shared Pool - #0] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:28:57,806 INFO [App Shared Pool - #0] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:28:57,807 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_21_00 [Map 1] 2018-01-05T11:28:57,807 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_21][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_21_00_000000, scheduledTime=1515180537807, launchTime=1515180537807 2018-01-05T11:28:57,807 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_21_00 [Map 1] 2018-01-05T11:28:57,807 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_21_00_000000_0, priority=2, capability=, hosts=null 2018-01-05T11:28:57,807 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=2,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:57,807 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=2,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:57,807 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:57,807 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_21_00_000000_0 on hosts=null 2018-01-05T11:28:57,807 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_21_00_000000_0 on hosts=null 2018-01-05T11:28:57,807 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_21_00_000000_0 2018-01-05T11:28:57,807 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:57,807 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #42, task=TaskInfo{task=attempt_1515180518813_0001_21_00_000000_0, priority=2, startTime=0, containerId=null, uniqueId=41, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000042 2018-01-05T11:28:57,807 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_21_00_000000_0; false 2018-01-05T11:28:57,807 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_21_00_000000_0, priority=2, startTime=311558316, containerId=container_111101111_0001_00_000042assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=41, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:57,807 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000042 for node: 10.22.8.182:49818 2018-01-05T11:28:57,809 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 21, name=create table src_cbo as select * from src (Stage-1), queryId=djaiswal_20180105112857_126fd009-65af-46a3-94f4-76213447cc87 2018-01-05T11:28:57,809 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000042, attempt_1515180518813_0001_21_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:57,809 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000042 for node: 10.22.8.182:49818 2018-01-05T11:28:57,810 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000042 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:57,810 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000042, launchTime=1515180537810 2018-01-05T11:28:57,810 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_21_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112857_126fd009-65af-46a3-94f4-76213447cc87, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000042, dagName=create table src_cbo as select * from src (Stage-1), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{src,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{out_Map 1,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180537782, firstAttemptStartTime=1515180537807, currentAttemptStartTime=1515180537809} 2018-01-05T11:28:57,810 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_21_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000042] on NM: [10.22.8.182:49818] 2018-01-05T11:28:57,810 INFO [IPC Server handler 0 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000042, QueryId=djaiswal_20180105112857_126fd009-65af-46a3-94f4-76213447cc87-21, DagId=21, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180537810 2018-01-05T11:28:57,810 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:57,810 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:57,810 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_21][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_21_00_000000_0, startTime=1515180537810, containerId=container_111101111_0001_00_000042, nodeId=10.22.8.182:49818 2018-01-05T11:28:57,810 DEBUG [IPC Server handler 0 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=21} with the ShuffleHandler 2018-01-05T11:28:57,822 INFO [IPC Server handler 0 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:28:57,822 DEBUG [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/21] 2018-01-05T11:28:57,822 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:57,822 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_21_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:57,822 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_21_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180537807, dagStartTime=1515180537782, withinDagPriority=2, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:28:57,822 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_21_00_000000_0 : ACCEPTED 2018-01-05T11:28:57,822 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_21_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:57,822 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_21_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180537807, dagStartTime=1515180537782, withinDagPriority=2, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:28:57,822 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_21_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:57,822 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_21_00_000000_0 to pre-emption queue 2018-01-05T11:28:57,822 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_21_00_000000_0: true 2018-01-05T11:28:57,823 INFO [TaskCommunicator # 5] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_21_00_000000_0 2018-01-05T11:28:57,823 INFO [TaskCommunicator # 5] GuaranteedTasks: Received response for attempt_1515180518813_0001_21_00_000000_0, true 2018-01-05T11:28:57,823 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:2002406159 2018-01-05T11:28:57,824 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : create table src_cbo as select * from src (Stage-1), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_21_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=src, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=out_Map 1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:28:57,824 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1033895936, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:57,824 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_21_00_000000_0 vertexName: Map 1 2018-01-05T11:28:57,824 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000042, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_21_00_000000_0, eventCount=1 } 2018-01-05T11:28:57,824 DEBUG [TezTR-518813_1_21_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_21 queryId: djaiswal_20180105112857_126fd009-65af-46a3-94f4-76213447cc87 instanceType: TaskRunner2Callable 2018-01-05T11:28:57,824 INFO [TezTR-518813_1_21_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_21_00_000000_0 2018-01-05T11:28:57,825 DEBUG [TezTR-518813_1_21_0_0_0] log.PerfLogger: 2018-01-05T11:28:57,825 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:57,826 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:57,827 INFO [IPC Server handler 22 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_21_00_000000_0 sent events: (0-1). 2018-01-05T11:28:57,827 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:28:57,827 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_21_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:28:57,828 DEBUG [TezTR-518813_1_21_0_0_0] log.PerfLogger: 2018-01-05T11:28:57,828 INFO [TezTR-518813_1_21_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:57,828 INFO [TezTR-518813_1_21_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:57,828 INFO [I/O Setup 0 Initialize: {src}] input.MRInput: src using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:28:57,828 INFO [I/O Setup 0 Initialize: {src}] input.MRInputLegacy: src MRInputLegacy deferring initialization 2018-01-05T11:28:57,828 INFO [TezTR-518813_1_21_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:57,828 INFO [I/O Setup 1 Initialize: {out_Map 1}] output.MROutput: out_Map 1: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:28:57,828 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:57,828 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:57,828 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:57,828 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:57,828 INFO [TezTR-518813_1_21_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:57,828 INFO [TezTR-518813_1_21_0_0_0] resources.MemoryDistributor: InitialRequests=[src:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [out_Map 1:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput] 2018-01-05T11:28:57,828 INFO [TezTR-518813_1_21_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:57,828 INFO [TezTR-518813_1_21_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:57,828 INFO [TezTR-518813_1_21_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 1033895936, TotalAvailable: 692710277, TotalRequested/TotalJVMHeap:0.00 2018-01-05T11:28:57,828 INFO [TezTR-518813_1_21_0_0_0] resources.MemoryDistributor: Allocations=[src:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [out_Map 1:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0] 2018-01-05T11:28:57,828 INFO [TezTR-518813_1_21_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:57,828 INFO [TezTR-518813_1_21_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: src being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:28:57,828 INFO [TezTR-518813_1_21_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:28:57,828 INFO [TezTR-518813_1_21_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:28:57,828 INFO [I/O Setup 0 Start: {src}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: src 2018-01-05T11:28:57,828 INFO [TezTR-518813_1_21_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:57,828 INFO [TezTR-518813_1_21_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_21_00_000000_0 2018-01-05T11:28:57,828 DEBUG [TezTR-518813_1_21_0_0_0] log.PerfLogger: 2018-01-05T11:28:57,828 DEBUG [TezTR-518813_1_21_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_21_00_000000_0_10124 2018-01-05T11:28:57,828 DEBUG [TezTR-518813_1_21_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:57,828 DEBUG [TezTR-518813_1_21_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_21_00_000000_0 2018-01-05T11:28:57,829 INFO [TezTR-518813_1_21_0_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112857_126fd009-65af-46a3-94f4-76213447cc87 2018-01-05T11:28:57,829 DEBUG [TezTR-518813_1_21_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:57,829 DEBUG [TezTR-518813_1_21_0_0_0] log.PerfLogger: 2018-01-05T11:28:57,829 DEBUG [TezTR-518813_1_21_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:57,829 DEBUG [TezTR-518813_1_21_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:57,829 DEBUG [TezTR-518813_1_21_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112857_126fd009-65af-46a3-94f4-76213447cc87_Map 1__MAP_PLAN__ 2018-01-05T11:28:57,829 INFO [TezTR-518813_1_21_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_547_735783970339321393-1/djaiswal/_tez_scratch_dir/b6fc96a2-b323-4a47-b68f-337335ce5b67/map.xml 2018-01-05T11:28:57,829 DEBUG [TezTR-518813_1_21_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_547_735783970339321393-1/djaiswal/_tez_scratch_dir/b6fc96a2-b323-4a47-b68f-337335ce5b67/map.xml 2018-01-05T11:28:57,829 DEBUG [TezTR-518813_1_21_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_547_735783970339321393-1/djaiswal/_tez_scratch_dir/b6fc96a2-b323-4a47-b68f-337335ce5b67/map.xml 2018-01-05T11:28:57,829 DEBUG [TezTR-518813_1_21_0_0_0] log.PerfLogger: 2018-01-05T11:28:57,829 INFO [TezTR-518813_1_21_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:57,829 DEBUG [TezTR-518813_1_21_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:57,829 DEBUG [TezTR-518813_1_21_0_0_0] log.PerfLogger: 2018-01-05T11:28:57,829 INFO [TezTR-518813_1_21_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 2.30KB 2018-01-05T11:28:57,830 DEBUG [TezTR-518813_1_21_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:57,830 INFO [TezTR-518813_1_21_0_0_0] tez.MapRecordProcessor: The input names are: [src] 2018-01-05T11:28:57,830 INFO [TezTR-518813_1_21_0_0_0] input.MRInput: Initialized MRInput: src 2018-01-05T11:28:57,830 DEBUG [TezTR-518813_1_21_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 1 2018-01-05T11:28:57,830 INFO [TezTR-518813_1_21_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_547_735783970339321393-1/djaiswal/_tez_scratch_dir/b6fc96a2-b323-4a47-b68f-337335ce5b67/map.xml 2018-01-05T11:28:57,830 DEBUG [TezTR-518813_1_21_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:57,830 DEBUG [TezTR-518813_1_21_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src/kv1.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct src { string key, string value}, transient_lastDdlTime=1515180521, rawDataSize=5312, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.src}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct src { string key, string value}, transient_lastDdlTime=1515180521, rawDataSize=5312, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.src}, baseFileName=src, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct src { string key, string value}, transient_lastDdlTime=1515180521, rawDataSize=5312, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.src}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct src { string key, string value}, transient_lastDdlTime=1515180521, rawDataSize=5312, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.src}, baseFileName=src, vectorPartitionDesc=null]} 2018-01-05T11:28:57,830 DEBUG [TezTR-518813_1_21_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:57,830 INFO [TezTR-518813_1_21_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:57,830 DEBUG [TezTR-518813_1_21_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:57,831 DEBUG [TezTR-518813_1_21_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:57,831 INFO [TezTR-518813_1_21_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_547_735783970339321393-1/djaiswal/_tez_scratch_dir/b6fc96a2-b323-4a47-b68f-337335ce5b67/map.xml 2018-01-05T11:28:57,831 DEBUG [TezTR-518813_1_21_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:57,831 INFO [TezTR-518813_1_21_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_21_00_000000_0 2018-01-05T11:28:57,831 INFO [TezTR-518813_1_21_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [string, string] (isAcidRead false) 2018-01-05T11:28:57,832 WARN [TezTR-518813_1_21_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src/kv1.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:28:57,832 DEBUG [TezTR-518813_1_21_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:28:57,832 WARN [TezTR-518813_1_21_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src/kv1.txt:0+5812 2018-01-05T11:28:57,834 DEBUG [TezTR-518813_1_21_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src/kv1.txt 2018-01-05T11:28:57,834 INFO [TezTR-518813_1_21_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src/kv1.txt:0+5812], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=5812} 2018-01-05T11:28:57,834 INFO [TezTR-518813_1_21_0_0_0] input.MRInput: src initialized RecordReader from event 2018-01-05T11:28:57,834 DEBUG [TezTR-518813_1_21_0_0_0] tez.MapRecordProcessor: Starting Output: out_Map 1 2018-01-05T11:28:57,834 INFO [TezTR-518813_1_21_0_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:28:57,834 INFO [TezTR-518813_1_21_0_0_0] exec.MapOperator: Initializing operator MAP[0] 2018-01-05T11:28:57,834 DEBUG [TezTR-518813_1_21_0_0_0] exec.MapOperator: Initialization Done 0 MAP 2018-01-05T11:28:57,834 DEBUG [TezTR-518813_1_21_0_0_0] exec.MapOperator: Operator 0 MAP initialized 2018-01-05T11:28:57,834 DEBUG [TezTR-518813_1_21_0_0_0] exec.MapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:28:57,835 DEBUG [TezTR-518813_1_21_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:57,835 DEBUG [TezTR-518813_1_21_0_0_0] exec.MapOperator: Adding alias src to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src 2018-01-05T11:28:57,835 DEBUG [TezTR-518813_1_21_0_0_0] exec.MapOperator: dump TS[0] struct 2018-01-05T11:28:57,835 INFO [TezTR-518813_1_21_0_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:28:57,835 INFO [TezTR-518813_1_21_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:57,835 DEBUG [TezTR-518813_1_21_0_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:28:57,835 INFO [TezTR-518813_1_21_0_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:28:57,835 DEBUG [TezTR-518813_1_21_0_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:28:57,835 DEBUG [TezTR-518813_1_21_0_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:28:57,835 DEBUG [TezTR-518813_1_21_0_0_0] exec.TableScanOperator: Initializing children of 0 TS 2018-01-05T11:28:57,835 DEBUG [TezTR-518813_1_21_0_0_0] exec.SelectOperator: Initializing child 1 SEL 2018-01-05T11:28:57,835 INFO [TezTR-518813_1_21_0_0_0] exec.SelectOperator: Initializing operator SEL[1] 2018-01-05T11:28:57,835 INFO [TezTR-518813_1_21_0_0_0] exec.SelectOperator: SELECT struct 2018-01-05T11:28:57,835 DEBUG [TezTR-518813_1_21_0_0_0] exec.SelectOperator: Initialization Done 1 SEL 2018-01-05T11:28:57,835 DEBUG [TezTR-518813_1_21_0_0_0] exec.SelectOperator: Operator 1 SEL initialized 2018-01-05T11:28:57,835 DEBUG [TezTR-518813_1_21_0_0_0] exec.SelectOperator: Initializing children of 1 SEL 2018-01-05T11:28:57,835 DEBUG [TezTR-518813_1_21_0_0_0] exec.FileSinkOperator: Initializing child 2 FS 2018-01-05T11:28:57,835 INFO [TezTR-518813_1_21_0_0_0] exec.FileSinkOperator: Initializing operator FS[2] 2018-01-05T11:28:57,836 DEBUG [TezTR-518813_1_21_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:57,836 INFO [TezTR-518813_1_21_0_0_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@434669b8]:[key, value]:[string, string]] and formatter : org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat@6ff2e02f 2018-01-05T11:28:57,836 DEBUG [TezTR-518813_1_21_0_0_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:28:57,836 DEBUG [TezTR-518813_1_21_0_0_0] exec.FileSinkOperator: Initialization Done 2 FS 2018-01-05T11:28:57,836 DEBUG [TezTR-518813_1_21_0_0_0] exec.FileSinkOperator: Operator 2 FS initialized 2018-01-05T11:28:57,836 DEBUG [TezTR-518813_1_21_0_0_0] exec.FileSinkOperator: Initialization Done 2 FS done is reset. 2018-01-05T11:28:57,836 DEBUG [TezTR-518813_1_21_0_0_0] exec.SelectOperator: Initialization Done 1 SEL done is reset. 2018-01-05T11:28:57,836 DEBUG [TezTR-518813_1_21_0_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:28:57,836 DEBUG [TezTR-518813_1_21_0_0_0] log.PerfLogger: 2018-01-05T11:28:57,836 INFO [TezTR-518813_1_21_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_547_735783970339321393-1/djaiswal/_tez_scratch_dir/b6fc96a2-b323-4a47-b68f-337335ce5b67/map.xml 2018-01-05T11:28:57,836 DEBUG [TezTR-518813_1_21_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:57,836 DEBUG [TezTR-518813_1_21_0_0_0] exec.MapOperator: Processing alias(es) src for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src/kv1.txt 2018-01-05T11:28:57,836 INFO [TezTR-518813_1_21_0_0_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/.hive-staging_hive_2018-01-05_11-28-57_547_735783970339321393-1/_tmp.-ext-10002/000000_0 2018-01-05T11:28:57,836 INFO [TezTR-518813_1_21_0_0_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/.hive-staging_hive_2018-01-05_11-28-57_547_735783970339321393-1/_task_tmp.-ext-10002/_tmp.000000_0 2018-01-05T11:28:57,836 INFO [TezTR-518813_1_21_0_0_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/.hive-staging_hive_2018-01-05_11-28-57_547_735783970339321393-1/_tmp.-ext-10002/000000_0 2018-01-05T11:28:57,870 INFO [TezTR-518813_1_21_0_0_0] exec.FileSinkOperator: FS[2]: records written - 1 2018-01-05T11:28:57,870 INFO [TezTR-518813_1_21_0_0_0] exec.MapOperator: MAP[0]: records read - 1 2018-01-05T11:28:57,870 INFO [TezTR-518813_1_21_0_0_0] exec.FileSinkOperator: FS[2]: records written - 10 2018-01-05T11:28:57,870 INFO [TezTR-518813_1_21_0_0_0] exec.MapOperator: MAP[0]: records read - 10 2018-01-05T11:28:57,870 INFO [TezTR-518813_1_21_0_0_0] exec.FileSinkOperator: FS[2]: records written - 100 2018-01-05T11:28:57,870 INFO [TezTR-518813_1_21_0_0_0] exec.MapOperator: MAP[0]: records read - 100 2018-01-05T11:28:57,871 DEBUG [TezTR-518813_1_21_0_0_0] log.PerfLogger: 2018-01-05T11:28:57,871 DEBUG [TezTR-518813_1_21_0_0_0] mr.ObjectCache: djaiswal_20180105112857_126fd009-65af-46a3-94f4-76213447cc87_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:28:57,871 DEBUG [TezTR-518813_1_21_0_0_0] exec.MapOperator: close called for operator MAP[0] 2018-01-05T11:28:57,871 INFO [TezTR-518813_1_21_0_0_0] exec.MapOperator: Closing operator MAP[0] 2018-01-05T11:28:57,871 INFO [TezTR-518813_1_21_0_0_0] exec.MapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:500, 2018-01-05T11:28:57,871 DEBUG [TezTR-518813_1_21_0_0_0] exec.MapOperator: Closing child = TS[0] 2018-01-05T11:28:57,871 DEBUG [TezTR-518813_1_21_0_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:28:57,871 DEBUG [TezTR-518813_1_21_0_0_0] exec.TableScanOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:57,871 INFO [TezTR-518813_1_21_0_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:28:57,871 DEBUG [TezTR-518813_1_21_0_0_0] exec.TableScanOperator: Closing child = SEL[1] 2018-01-05T11:28:57,871 DEBUG [TezTR-518813_1_21_0_0_0] exec.SelectOperator: close called for operator SEL[1] 2018-01-05T11:28:57,871 DEBUG [TezTR-518813_1_21_0_0_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:57,871 INFO [TezTR-518813_1_21_0_0_0] exec.SelectOperator: Closing operator SEL[1] 2018-01-05T11:28:57,871 DEBUG [TezTR-518813_1_21_0_0_0] exec.SelectOperator: Closing child = FS[2] 2018-01-05T11:28:57,871 DEBUG [TezTR-518813_1_21_0_0_0] exec.FileSinkOperator: close called for operator FS[2] 2018-01-05T11:28:57,871 DEBUG [TezTR-518813_1_21_0_0_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:57,871 INFO [TezTR-518813_1_21_0_0_0] exec.FileSinkOperator: Closing operator FS[2] 2018-01-05T11:28:57,871 INFO [TezTR-518813_1_21_0_0_0] exec.FileSinkOperator: FS[2]: records written - 500 2018-01-05T11:28:57,872 DEBUG [TezTR-518813_1_21_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:57,872 DEBUG [TezTR-518813_1_21_0_0_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_547_735783970339321393-1/-mr-10003 2018-01-05T11:28:57,895 DEBUG [TezTR-518813_1_21_0_0_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_547_735783970339321393-1/-mr-10003/tmpstats-0 2018-01-05T11:28:57,895 DEBUG [TezTR-518813_1_21_0_0_0] fs.FSStatsPublisher: Writing stats in it : {default.src_cbo/={rawDataSize=5312, numRows=500}} 2018-01-05T11:28:57,895 INFO [TezTR-518813_1_21_0_0_0] exec.FileSinkOperator: RECORDS_OUT_1_default.src_cbo:500, 2018-01-05T11:28:57,895 DEBUG [TezTR-518813_1_21_0_0_0] exec.FileSinkOperator: 2 Close done 2018-01-05T11:28:57,895 DEBUG [TezTR-518813_1_21_0_0_0] exec.SelectOperator: 1 Close done 2018-01-05T11:28:57,895 DEBUG [TezTR-518813_1_21_0_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:28:57,895 DEBUG [TezTR-518813_1_21_0_0_0] exec.MapOperator: 0 Close done 2018-01-05T11:28:57,895 INFO [TezTR-518813_1_21_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_21_00_000000_0 2018-01-05T11:28:57,895 INFO [TezTR-518813_1_21_0_0_0] output.MROutput: out_Map 1 closed 2018-01-05T11:28:57,895 DEBUG [TezTR-518813_1_21_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:57,895 INFO [TezTR-518813_1_21_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:57,895 INFO [TezTR-518813_1_21_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_21_00_000000_0, askedToStop=false 2018-01-05T11:28:57,895 INFO [TezTR-518813_1_21_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_21_00_000000_0, stopRequested=false 2018-01-05T11:28:57,895 INFO [TezTR-518813_1_21_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_21_00_000000_0: Counters: 7 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=500, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=0][HIVE CREATED_FILES=1, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500, RECORDS_OUT_1_default.src_cbo=500]] 2018-01-05T11:28:57,895 DEBUG [TezTR-518813_1_21_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_21_0_0_0 thread id: 1492 scheme: file bytesRead: 5872 bytesWritten: 5954 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:57,895 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_21_00_000000_0, isTaskDone=true 2018-01-05T11:28:57,895 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000042, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_21_00_000000_0, eventCount=2 } 2018-01-05T11:28:57,896 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_21][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_21_00_000000_0, creationTime=1515180537807, allocationTime=1515180537807, startTime=1515180537810, finishTime=1515180537896, timeTaken=86, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:57,896 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:57,896 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_21_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:57,896 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_21_00_000000_0 2018-01-05T11:28:57,896 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_21_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:57,896 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_21_00_000000_0 2018-01-05T11:28:57,896 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_21_00_000000_0; false 2018-01-05T11:28:57,896 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_21_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:57,897 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:57,897 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_21][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_21_00_000000, startTime=1515180537810, finishTime=1515180537896, timeTaken=86, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_21_00_000000_0, diagnostics=, counters=Counters: 13, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=5872, FILE_BYTES_WRITTEN=5954, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=500, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=0, HIVE, CREATED_FILES=1, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500, RECORDS_OUT_1_default.src_cbo=500 2018-01-05T11:28:57,897 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:57,897 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000042=74 2018-01-05T11:28:57,897 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_21_00_000000_0: true 2018-01-05T11:28:57,897 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_21_00_000000_0 2018-01-05T11:28:57,897 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000042, QueryId=djaiswal_20180105112857_126fd009-65af-46a3-94f4-76213447cc87-21, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:28:57,897 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_21_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:57,897 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_21_00_000000_0 2018-01-05T11:28:57,897 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_21_00 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:57,897 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_21_00 [Map 1] 2018-01-05T11:28:57,898 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_21][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_21_00, initRequestedTime=1515180537782, initedTime=1515180537806, startRequestedTime=1515180537790, startedTime=1515180537806, finishTime=1515180537897, timeTaken=91, status=SUCCEEDED, diagnostics=, counters=Counters: 13, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=5872, FILE_BYTES_WRITTEN=5954, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=500, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=0, HIVE, CREATED_FILES=1, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500, RECORDS_OUT_1_default.src_cbo=500, vertexStats=firstTaskStartTime=1515180537810, firstTasksToStart=[ task_1515180518813_0001_21_00_000000 ], lastTaskFinishTime=1515180537896, lastTasksToFinish=[ task_1515180518813_0001_21_00_000000 ], minTaskDuration=86, maxTaskDuration=86, avgTaskDuration=86.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_21_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_21_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:57,898 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_21_00 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:57,898 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_21][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000042, stoppedTime=1515180537898, exitStatus=0 2018-01-05T11:28:57,898 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000042 exited with diagnostics set to Container completed. 2018-01-05T11:28:57,898 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_21_00 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1 2018-01-05T11:28:57,898 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null 2018-01-05T11:28:57,898 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_21_00 [Map 1] 2018-01-05T11:28:57,899 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_21, queueSize=0 2018-01-05T11:28:57,899 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_21][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_21, startTime=1515180537782, finishTime=1515180537898, timeTaken=116, status=SUCCEEDED, diagnostics=, counters=Counters: 17, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, OTHER_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=227, AM_GC_TIME_MILLIS=0, File System Counters, FILE_BYTES_READ=5872, FILE_BYTES_WRITTEN=5954, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=500, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=0, HIVE, CREATED_FILES=1, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500, RECORDS_OUT_1_default.src_cbo=500 2018-01-05T11:28:57,899 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_21 finished with state: SUCCEEDED 2018-01-05T11:28:57,899 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_21 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:28:57,899 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000042 2018-01-05T11:28:57,899 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_21, dagState=SUCCEEDED 2018-01-05T11:28:57,899 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_21_post 2018-01-05T11:28:57,899 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:57,899 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:28:57,899 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:28:57,899 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 21 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=1, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=1,NumRequestsWithoutLocation=1, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=1} 2018-01-05T11:28:57,899 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:28:57,899 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=create table src_cbo as select * from src (Stage-1), with id=dag_1515180518813_0001_21 2018-01-05T11:28:57,899 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 21, to 10.22.8.182:49818 2018-01-05T11:28:57,899 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=1, Remaining=0 2018-01-05T11:28:57,899 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=create table src_cbo as select * from src (Stage-1), with id=dag_1515180518813_0001_21 2018-01-05T11:28:57,900 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=21} 2018-01-05T11:28:57,900 INFO [IPC Server handler 2 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=21}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:28:57,900 INFO [IPC Server handler 2 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/21 after 300 seconds 2018-01-05T11:28:57,900 ERROR [IPC Server handler 2 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112857_126fd009-65af-46a3-94f4-76213447cc87, dag_1515180518813_0001_21 2018-01-05T11:28:57,900 INFO [IPC Server handler 2 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112857_126fd009-65af-46a3-94f4-76213447cc87 2018-01-05T11:28:57,900 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=21} = 0 2018-01-05T11:28:57,900 DEBUG [IPC Server handler 2 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=21} 2018-01-05T11:28:57,900 DEBUG [IPC Server handler 2 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:28:57,900 DEBUG [IPC Server handler 2 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=21} 2018-01-05T11:28:57,900 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,900 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,900 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 2018-01-05T11:28:57,900 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,901 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,901 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: TaskId for 000000_0 = 000000 2018-01-05T11:28:57,902 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,902 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,902 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,902 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-2:DEPENDENCY_COLLECTION] in serial mode 2018-01-05T11:28:57,902 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MOVE] in serial mode 2018-01-05T11:28:57,902 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@519c0918, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:57,902 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:57,902 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,902 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:57,902 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:57,902 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:57,902 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:57,902 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,902 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:57,902 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] FileOperations: MoveTask moving file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/.hive-staging_hive_2018-01-05_11-28-57_547_735783970339321393-1/-ext-10002 to file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo 2018-01-05T11:28:57,902 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Moving data to directory file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo from file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/.hive-staging_hive_2018-01-05_11-28-57_547_735783970339321393-1/-ext-10002 2018-01-05T11:28:57,902 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/.hive-staging_hive_2018-01-05_11-28-57_547_735783970339321393-1/-ext-10002/ and the destination path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo/ 2018-01-05T11:28:57,902 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/.hive-staging_hive_2018-01-05_11-28-57_547_735783970339321393-1/-ext-10002/ and the destination path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo/ 2018-01-05T11:28:57,902 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo/ and the destination path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/.hive-staging_hive_2018-01-05_11-28-57_547_735783970339321393-1/-ext-10002/ 2018-01-05T11:28:57,902 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:57,902 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:28:57,904 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-4:DDL] in serial mode 2018-01-05T11:28:57,904 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Found class for org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe 2018-01-05T11:28:57,904 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: creating table default.src_cbo on file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo 2018-01-05T11:28:57,905 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,905 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:57,905 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:57,905 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:57,906 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:57,906 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,906 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,906 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,906 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,907 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:57,907 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:57,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,907 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:57,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,907 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: create_table: Table(tableName:src_cbo, dbName:default, owner:hive_test_user, createTime:1515180537, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:string, comment:null), FieldSchema(name:value, type:string, comment:null)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:57,907 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=create_table: Table(tableName:src_cbo, dbName:default, owner:hive_test_user, createTime:1515180537, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:string, comment:null), FieldSchema(name:value, type:string, comment:null)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:28:57,908 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hive-site.xml 2018-01-05T11:28:57,908 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hivemetastore-site.xml 2018-01-05T11:28:57,909 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Unable to find config file metastore-site.xml 2018-01-05T11:28:57,909 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.MetastoreConf: Found configuration file null 2018-01-05T11:28:57,918 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:28:57,918 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: Location: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo specified for non-external table:src_cbo 2018-01-05T11:28:57,918 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.MetaStoreUtils: Updating table stats fast for src_cbo 2018-01-05T11:28:57,918 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.MetaStoreUtils: Updated size of table src_cbo to 5812 2018-01-05T11:28:57,922 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,924 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,924 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,924 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,924 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_cbo 2018-01-05T11:28:57,924 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_cbo 2018-01-05T11:28:57,927 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,928 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,928 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.DDLTask: Ignoring request to add WriteEntity(default@src_cbo) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@src_cbo) Type=TABLE WriteType=DDL_NO_LOCK is present 2018-01-05T11:28:57,928 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-3:STATS] in serial mode 2018-01-05T11:28:57,928 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@2e5ac612, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:57,928 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:57,928 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,928 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:57,928 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:57,928 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:57,928 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:57,928 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,928 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:57,928 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:57,928 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_547_735783970339321393-1/-mr-10003 2018-01-05T11:28:57,928 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_547_735783970339321393-1/-mr-10003 2018-01-05T11:28:57,929 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.src_cbo/ numRows 2018-01-05T11:28:57,929 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.src_cbo/ numRows 500 2018-01-05T11:28:57,929 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.src_cbo/ rawDataSize 2018-01-05T11:28:57,929 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.src_cbo/ rawDataSize 5312 2018-01-05T11:28:57,932 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,932 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:57,932 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:57,932 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:57,932 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:57,933 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,933 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,933 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,933 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,933 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,933 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:57,933 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:57,934 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,934 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:57,934 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,934 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=src_cbo newtbl=src_cbo 2018-01-05T11:28:57,934 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=src_cbo newtbl=src_cbo 2018-01-05T11:28:57,936 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,941 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,942 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,942 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.src_cbo stats: [numFiles=1, numRows=500, totalSize=5812, rawDataSize=5312] 2018-01-05T11:28:57,942 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: About to delete stats tmp dir :file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_547_735783970339321393-1/-mr-10003 2018-01-05T11:28:57,943 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,943 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,943 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: create table src_cbo as select * from src 2018-01-05T11:28:57,943 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: CREATETABLE_AS_SELECT 2018-01-05T11:28:57,943 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@src 2018-01-05T11:28:57,943 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: database:default 2018-01-05T11:28:57,943 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@src_cbo 2018-01-05T11:28:57,943 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Lineage: src_cbo.key SIMPLE [(src)src.FieldSchema(name:key, type:string, comment:default), ] 2018-01-05T11:28:57,943 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Lineage: src_cbo.value SIMPLE [(src)src.FieldSchema(name:value, type:string, comment:default), ] 2018-01-05T11:28:57,943 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,943 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,943 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:57,943 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=8} 2018-01-05T11:28:57,943 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112857_126fd009-65af-46a3-94f4-76213447cc87); Time taken: 0.273 seconds 2018-01-05T11:28:57,943 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:57,943 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,943 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:57,943 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,943 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query create table src_cbo as select * from src 2018-01-05T11:28:57,943 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/.hive-staging_hive_2018-01-05_11-28-57_547_735783970339321393-1 2018-01-05T11:28:57,944 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_547_735783970339321393-1 2018-01-05T11:28:57,945 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.397 seconds 2018-01-05T11:28:57,945 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:57,945 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:57,945 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:57,945 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:57,945 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,945 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:57,945 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,945 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,945 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,945 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: analyze table cbo_t1 partition (dt) compute statistics 2018-01-05T11:28:57,945 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112857_0ac4bddb-d537-4cc8-9668-b29df90b2cc0): analyze table cbo_t1 partition (dt) compute statistics 2018-01-05T11:28:57,945 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,945 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: analyze table cbo_t1 partition (dt) compute statistics 2018-01-05T11:28:57,946 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:57,946 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,946 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:57,946 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,946 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,946 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:57,946 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Invoking analyze on original query 2018-01-05T11:28:57,946 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Starting Semantic Analysis 2018-01-05T11:28:57,946 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed phase 1 of Semantic Analysis 2018-01-05T11:28:57,946 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for source tables 2018-01-05T11:28:57,946 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,946 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t1 2018-01-05T11:28:57,946 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t1 2018-01-05T11:28:57,948 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,949 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,949 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,949 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t1 2018-01-05T11:28:57,949 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t1 2018-01-05T11:28:57,952 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,952 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,952 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.BaseSemanticAnalyzer: Dynamic partitioning is used; only validating 0 columns 2018-01-05T11:28:57,953 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,953 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t1 2018-01-05T11:28:57,953 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t1 2018-01-05T11:28:57,956 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,957 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,957 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,957 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partitions_names_ps : db=default tbl=cbo_t1[] 2018-01-05T11:28:57,957 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions_names_ps : db=default tbl=cbo_t1[] 2018-01-05T11:28:57,960 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,972 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,972 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,972 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:57,972 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partitions_by_names : db=default tbl=cbo_t1 2018-01-05T11:28:57,972 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions_by_names : db=default tbl=cbo_t1 2018-01-05T11:28:57,975 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:57,979 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 3.489523ms + 0.006118ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? where "PARTITIONS"."PART_NAME" in (?)] 2018-01-05T11:28:57,982 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 2.787551ms + 0.056201ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (10) order by "PART_NAME" asc] 2018-01-05T11:28:57,985 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 2.345308ms + 0.145808ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (10) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:28:57,987 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 1.723719ms + 0.038354ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (10) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:57,988 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 1.781724ms + 0.014635ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (31) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:28:57,990 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 1.780528ms + 0.015971ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (31) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:57,992 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 2.188345ms + 0.015779ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (31) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:57,995 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 2.091711ms + 0.017612ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (31) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:57,998 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 3.404919ms + 0.047246ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (19) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:58,001 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 2.864409ms + 0.057145ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (31) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:28:58,001 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,001 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000042, QueryId=djaiswal_20180105112857_126fd009-65af-46a3-94f4-76213447cc87-21, DagId=21, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180537822, EndTime=1515180538001 2018-01-05T11:28:58,009 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,010 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,010 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for subqueries 2018-01-05T11:28:58,010 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for destination tables 2018-01-05T11:28:58,010 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed getting MetaData in Semantic Analysis 2018-01-05T11:28:58,022 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_945_7709530907020198391-1/-mr-10000 2018-01-05T11:28:58,023 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: XXX: adding part: cbo_t1(dt=2014) 2018-01-05T11:28:58,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Table Plan for cbo_t1 TS[0] 2018-01-05T11:28:58,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Body Plan for Query Block null 2018-01-05T11:28:58,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Plan for Query Block null 2018-01-05T11:28:58,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Before logical optimization TS[0] 2018-01-05T11:28:58,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] lineage.Generator: Time taken for lineage transform=0 2018-01-05T11:28:58,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{cbo_t1}key,value: string|{cbo_t1}value,c_int: int|{cbo_t1}c_int,c_float: float|{cbo_t1}c_float,c_boolean: boolean|{cbo_t1}c_boolean,dt: string|{cbo_t1}dt,BLOCK__OFFSET__INSIDE__FILE: bigint|{cbo_t1}block__offset__inside__file,INPUT__FILE__NAME: string|{cbo_t1}input__file__name,ROW__ID: struct|{cbo_t1}row__id,RAW__DATA__SIZE: bigint|{cbo_t1}raw__data__size) 2018-01-05T11:28:58,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,023 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:28:58,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.PredicatePushDown: After PPD: TS[0] 2018-01-05T11:28:58,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{cbo_t1}key,value: string|{cbo_t1}value,c_int: int|{cbo_t1}c_int,c_float: float|{cbo_t1}c_float,c_boolean: boolean|{cbo_t1}c_boolean,dt: string|{cbo_t1}dt,BLOCK__OFFSET__INSIDE__FILE: bigint|{cbo_t1}block__offset__inside__file,INPUT__FILE__NAME: string|{cbo_t1}input__file__name,ROW__ID: struct|{cbo_t1}row__id,RAW__DATA__SIZE: bigint|{cbo_t1}raw__data__size) 2018-01-05T11:28:58,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,023 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{cbo_t1}key,value: string|{cbo_t1}value,c_int: int|{cbo_t1}c_int,c_float: float|{cbo_t1}c_float,c_boolean: boolean|{cbo_t1}c_boolean,dt: string|{cbo_t1}dt,BLOCK__OFFSET__INSIDE__FILE: bigint|{cbo_t1}block__offset__inside__file,INPUT__FILE__NAME: string|{cbo_t1}input__file__name,ROW__ID: struct|{cbo_t1}row__id,RAW__DATA__SIZE: bigint|{cbo_t1}raw__data__size) 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: After logical optimization TS[0] 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,024 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,024 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partitions : db=default tbl=cbo_t1 2018-01-05T11:28:58,024 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions : db=default tbl=cbo_t1 2018-01-05T11:28:58,034 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,035 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.344418ms + 0.006588ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? ] 2018-01-05T11:28:58,036 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.229411ms + 0.059873ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (10) order by "PART_NAME" asc] 2018-01-05T11:28:58,036 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.136341ms + 0.074992ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (10) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:28:58,036 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.120807ms + 0.02902ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (10) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:58,036 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.106308ms + 0.014218ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (31) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:28:58,036 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.100428ms + 0.01007ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (31) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:58,036 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.095891ms + 0.009652ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (31) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:58,037 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.098211ms + 0.010128ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (31) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:58,040 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 2.895547ms + 0.077029ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (19) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:58,040 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.18415ms + 0.078405ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (31) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:28:58,040 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,047 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,048 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,048 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,048 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting stats (Num rows: 1 Data size: 2828 Basic stats: PARTIAL Column stats: NONE) on TS[0] 2018-01-05T11:28:58,048 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (cbo_t1): numRows: 1 dataSize: 2828 basicStatsState: PARTIAL colStatsState: NONE colStats: {} 2018-01-05T11:28:58,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on TS[0] 2018-01-05T11:28:58,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:58,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:28:58,049 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Cycle free: true 2018-01-05T11:28:58,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{cbo_t1}key,value: string|{cbo_t1}value,c_int: int|{cbo_t1}c_int,c_float: float|{cbo_t1}c_float,c_boolean: boolean|{cbo_t1}c_boolean,dt: string|{cbo_t1}dt,BLOCK__OFFSET__INSIDE__FILE: bigint|{cbo_t1}block__offset__inside__file,INPUT__FILE__NAME: string|{cbo_t1}input__file__name,ROW__ID: struct|{cbo_t1}row__id,RAW__DATA__SIZE: bigint|{cbo_t1}raw__data__size) 2018-01-05T11:28:58,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:28:58,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1/dt=2014 of table cbo_t1 2018-01-05T11:28:58,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1/dt=2014 2018-01-05T11:28:58,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:28:58,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:58,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:58,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:58,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:58,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:58,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:58,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:28:58,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping vectorization 2018-01-05T11:28:58,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:28:58,050 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: llap mode: only 2018-01-05T11:28:58,050 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:28:58,050 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:58,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@2560a086 can be done in LLAP 2018-01-05T11:28:58,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,050 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed plan generation 2018-01-05T11:28:58,050 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:58,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validation start 2018-01-05T11:28:58,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Not a partition. 2018-01-05T11:28:58,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validated dt=2014 2018-01-05T11:28:58,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: cbo_t1 2018-01-05T11:28:58,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,050 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partitions_ps_with_auth : db=default tbl=cbo_t1[2014] 2018-01-05T11:28:58,050 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions_ps_with_auth : db=default tbl=cbo_t1[2014] 2018-01-05T11:28:58,053 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,064 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,065 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=cbo_t1[2014] 2018-01-05T11:28:58,065 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=cbo_t1[2014] 2018-01-05T11:28:58,067 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,078 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,079 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,079 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,079 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:cbo_t1.key, type:string, comment:null), FieldSchema(name:cbo_t1.value, type:string, comment:null), FieldSchema(name:cbo_t1.c_int, type:int, comment:null), FieldSchema(name:cbo_t1.c_float, type:float, comment:null), FieldSchema(name:cbo_t1.c_boolean, type:boolean, comment:null), FieldSchema(name:cbo_t1.dt, type:string, comment:null)], properties:null) 2018-01-05T11:28:58,079 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,079 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:58,079 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, listPartitionNames_(String, String, List, short, )=15, getTable_(String, String, )=10, flushCache_()=0, getPartitionsByNames_(String, String, List, )=38, getPartitionWithAuthInfo_(String, String, List, String, List, )=14, listPartitions_(String, String, short, )=24, listPartitionsWithAuthInfo_(String, String, List, short, String, List, )=15} 2018-01-05T11:28:58,079 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112857_0ac4bddb-d537-4cc8-9668-b29df90b2cc0); Time taken: 0.134 seconds 2018-01-05T11:28:58,079 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:58,079 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,079 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112857_0ac4bddb-d537-4cc8-9668-b29df90b2cc0): analyze table cbo_t1 partition (dt) compute statistics 2018-01-05T11:28:58,081 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,081 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: analyze table cbo_t1 partition (dt) compute statistics 2018-01-05T11:28:58,081 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:28:58,081 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@cbo_t1 2018-01-05T11:28:58,081 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@cbo_t1@dt=2014 2018-01-05T11:28:58,081 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@cbo_t1 2018-01-05T11:28:58,081 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@cbo_t1@dt=2014 2018-01-05T11:28:58,081 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,081 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,081 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,081 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Query ID = djaiswal_20180105112857_0ac4bddb-d537-4cc8-9668-b29df90b2cc0 2018-01-05T11:28:58,081 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Total jobs = 1 2018-01-05T11:28:58,081 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,081 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:28:58,082 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MAPRED] in serial mode 2018-01-05T11:28:58,093 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_945_7709530907020198391-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:28:58,116 DEBUG [AMReporterQueueDrainer] impl.AMReporter: Removing am 10.22.8.182:49824 with last associated dag QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=12} from heartbeat with taskCount=0, amFailed=false, isDone=true 2018-01-05T11:28:58,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:28:58,120 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:28:58,120 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:28:58,120 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112857_0ac4bddb-d537-4cc8-9668-b29df90b2cc0 2018-01-05T11:28:58,120 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Session is already open 2018-01-05T11:28:58,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:28:58,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,120 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Dag name: analyze table cbo_t1 partition ...statistics (Stage-0) 2018-01-05T11:28:58,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: DagInfo: {"context":"Hive","description":"\n\n\nanalyze table cbo_t1 partition (dt) compute statistics"} 2018-01-05T11:28:58,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112857_0ac4bddb-d537-4cc8-9668-b29df90b2cc0 with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:28:58,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,132 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:28:58,132 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,132 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:28:58,133 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,133 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 2.00KB 2018-01-05T11:28:58,139 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_945_7709530907020198391-1/-mr-10000 2018-01-05T11:28:58,149 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_945_7709530907020198391-1/-mr-10000 2018-01-05T11:28:58,152 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1/dt=2014 2018-01-05T11:28:58,153 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,153 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,153 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,153 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=analyze table cbo_t1 partition ...statistics (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112857_0ac4bddb-d537-4cc8-9668-b29df90b2cc0 } 2018-01-05T11:28:58,153 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:58,155 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: analyze table cbo_t1 partition ...statistics (Stage-0) 2018-01-05T11:28:58,155 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:28:58,155 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_22, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_22.dot 2018-01-05T11:28:58,156 INFO [IPC Server handler 0 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_22 2018-01-05T11:28:58,156 WARN [IPC Server handler 0 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:58,156 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Running DAG: analyze table cbo_t1 partition ...statistics (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112857_0ac4bddb-d537-4cc8-9668-b29df90b2cc0 } 2018-01-05T11:28:58,179 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_22][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_22, submitTime=1515180538155, queueName=null 2018-01-05T11:28:58,180 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_22_00 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_22_00 [Map 1], Default Resources= 2018-01-05T11:28:58,180 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_22_00 [Map 1] 2018-01-05T11:28:58,180 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_22_00 [Map 1] 2018-01-05T11:28:58,180 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:28:58,180 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_22_00 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:58,180 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:28:58,180 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_22][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_22, initTime=1515180538180 2018-01-05T11:28:58,180 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: dag_1515180518813_0001_22 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:28:58,181 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:28:58,181 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_22][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_22, startTime=1515180538181 2018-01-05T11:28:58,181 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_22 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:28:58,181 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {cbo_t1={InputName=cbo_t1}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:28:58,181 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: cbo_t1, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:28:58,181 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_22_00 [Map 1] 2018-01-05T11:28:58,181 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_22, dagName=analyze table cbo_t1 partition ...statistics (Stage-0) 2018-01-05T11:28:58,181 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,181 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,181 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,185 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_22_00 [Map 1] 2018-01-05T11:28:58,185 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_22_00 [Map 1] 2018-01-05T11:28:58,185 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:58,185 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_22_00 [Map 1] 2018-01-05T11:28:58,187 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:28:58,187 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@283acd80 2018-01-05T11:28:58,187 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_945_7709530907020198391-1/djaiswal/_tez_scratch_dir/97d740b9-22c3-4b02-84a1-7b3a720c5f35/map.xml 2018-01-05T11:28:58,187 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_945_7709530907020198391-1/djaiswal/_tez_scratch_dir/97d740b9-22c3-4b02-84a1-7b3a720c5f35/map.xml 2018-01-05T11:28:58,187 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_945_7709530907020198391-1/djaiswal/_tez_scratch_dir/97d740b9-22c3-4b02-84a1-7b3a720c5f35/map.xml 2018-01-05T11:28:58,187 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:58,187 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:58,187 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,188 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:58,188 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 2.00KB 2018-01-05T11:28:58,188 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_22_00 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:28:58,188 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: cbo_t1 on vertex vertex_1515180518813_0001_22_00 [Map 1] 2018-01-05T11:28:58,188 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:28:58,188 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:58,188 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:28:58,188 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:58,188 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_945_7709530907020198391-1/djaiswal/_tez_scratch_dir/97d740b9-22c3-4b02-84a1-7b3a720c5f35/map.xml 2018-01-05T11:28:58,188 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:58,188 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias cbo_t1 2018-01-05T11:28:58,188 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1/dt=2014 2018-01-05T11:28:58,188 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1/dt=2014 2018-01-05T11:28:58,188 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 2018-01-05T11:28:58,188 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = 2018-01-05T11:28:58,188 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1/dt=2014 2018-01-05T11:28:58,189 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:58,189 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:58,189 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:58,189 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:58,207 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Time taken to get FileStatuses: 18 2018-01-05T11:28:58,207 INFO [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:28:58,207 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 18 2018-01-05T11:28:58,207 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 1 2018-01-05T11:28:58,207 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:58,207 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 1. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:28:58,207 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_945_7709530907020198391-1/djaiswal/_tez_scratch_dir/97d740b9-22c3-4b02-84a1-7b3a720c5f35/map.xml 2018-01-05T11:28:58,207 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_945_7709530907020198391-1/djaiswal/_tez_scratch_dir/97d740b9-22c3-4b02-84a1-7b3a720c5f35/map.xml 2018-01-05T11:28:58,207 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_945_7709530907020198391-1/djaiswal/_tez_scratch_dir/97d740b9-22c3-4b02-84a1-7b3a720c5f35/map.xml 2018-01-05T11:28:58,207 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:58,207 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:58,207 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,208 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:58,208 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 2.00KB 2018-01-05T11:28:58,208 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1/dt=2014/cbo_t1.txt to src new group? true 2018-01-05T11:28:58,208 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:28:58,208 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:28:58,208 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:28:58,208 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 5 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 282 Original splits: 1 2018-01-05T11:28:58,208 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Using original number of splits: 1 desired splits: 1 2018-01-05T11:28:58,208 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 1 grouped split count is 1, for bucket: 1 2018-01-05T11:28:58,208 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:28:58,209 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: cbo_t1 on vertex vertex_1515180518813_0001_22_00 [Map 1] 2018-01-05T11:28:58,210 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_22_00 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:58,210 INFO [App Shared Pool - #0] impl.VertexImpl: Got updated RootInputsSpecs: {cbo_t1=forAllWorkUnits=true, update=[1]} 2018-01-05T11:28:58,210 INFO [App Shared Pool - #0] impl.VertexImpl: Vertex vertex_1515180518813_0001_22_00 [Map 1] parallelism set to 1 2018-01-05T11:28:58,210 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_22_00 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:58,210 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_22_00 [Map 1], numAdditionalOutputs=1 2018-01-05T11:28:58,210 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_22][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_22_00, initRequestedTime=1515180538181, initedTime=1515180538210, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:58,210 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_22_00 [Map 1] with distanceFromRoot: 0 2018-01-05T11:28:58,210 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_22_00 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:28:58,210 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_22][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_22_00, startRequestedTime=1515180538188, startedTime=1515180538210 2018-01-05T11:28:58,210 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_22][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_22_00, reconfigureDoneTime=1515180538210, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:28:58,210 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_22_00 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:58,210 INFO [App Shared Pool - #0] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:28:58,211 INFO [App Shared Pool - #0] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:28:58,211 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_22_00 [Map 1] 2018-01-05T11:28:58,211 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_22][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_22_00_000000, scheduledTime=1515180538211, launchTime=1515180538211 2018-01-05T11:28:58,211 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_22_00 [Map 1] 2018-01-05T11:28:58,211 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_22_00_000000_0, priority=2, capability=, hosts=null 2018-01-05T11:28:58,211 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=2,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:58,211 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=2,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:58,211 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:58,211 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_22_00_000000_0 on hosts=null 2018-01-05T11:28:58,211 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_22_00_000000_0 on hosts=null 2018-01-05T11:28:58,211 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_22_00_000000_0 2018-01-05T11:28:58,211 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:58,211 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #43, task=TaskInfo{task=attempt_1515180518813_0001_22_00_000000_0, priority=2, startTime=0, containerId=null, uniqueId=42, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000043 2018-01-05T11:28:58,211 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_22_00_000000_0; false 2018-01-05T11:28:58,211 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_22_00_000000_0, priority=2, startTime=311558720, containerId=container_111101111_0001_00_000043assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=42, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:58,211 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000043 for node: 10.22.8.182:49818 2018-01-05T11:28:58,213 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 22, name=analyze table cbo_t1 partition ...statistics (Stage-0), queryId=djaiswal_20180105112857_0ac4bddb-d537-4cc8-9668-b29df90b2cc0 2018-01-05T11:28:58,213 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000043, attempt_1515180518813_0001_22_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:58,213 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000043 for node: 10.22.8.182:49818 2018-01-05T11:28:58,214 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000043 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:58,214 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000043, launchTime=1515180538214 2018-01-05T11:28:58,215 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_22_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112857_0ac4bddb-d537-4cc8-9668-b29df90b2cc0, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000043, dagName=analyze table cbo_t1 partition ...statistics (Stage-0), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{cbo_t1,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{out_Map 1,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180538181, firstAttemptStartTime=1515180538211, currentAttemptStartTime=1515180538214} 2018-01-05T11:28:58,215 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_22_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000043] on NM: [10.22.8.182:49818] 2018-01-05T11:28:58,215 INFO [IPC Server handler 1 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000043, QueryId=djaiswal_20180105112857_0ac4bddb-d537-4cc8-9668-b29df90b2cc0-22, DagId=22, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180538215 2018-01-05T11:28:58,215 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:58,215 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:58,215 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_22][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_22_00_000000_0, startTime=1515180538214, containerId=container_111101111_0001_00_000043, nodeId=10.22.8.182:49818 2018-01-05T11:28:58,215 DEBUG [IPC Server handler 1 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=22} with the ShuffleHandler 2018-01-05T11:28:58,226 INFO [IPC Server handler 1 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:28:58,226 DEBUG [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/22] 2018-01-05T11:28:58,226 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:58,226 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_22_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:58,226 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_22_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180538211, dagStartTime=1515180538181, withinDagPriority=2, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:28:58,226 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_22_00_000000_0 : ACCEPTED 2018-01-05T11:28:58,226 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_22_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:58,226 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_22_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180538211, dagStartTime=1515180538181, withinDagPriority=2, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:28:58,226 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_22_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:58,226 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_22_00_000000_0 to pre-emption queue 2018-01-05T11:28:58,226 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_22_00_000000_0: true 2018-01-05T11:28:58,227 INFO [TaskCommunicator # 7] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_22_00_000000_0 2018-01-05T11:28:58,227 INFO [TaskCommunicator # 7] GuaranteedTasks: Received response for attempt_1515180518813_0001_22_00_000000_0, true 2018-01-05T11:28:58,227 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:1251809167 2018-01-05T11:28:58,227 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : analyze table cbo_t1 partition ...statistics (Stage-0), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_22_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=cbo_t1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=out_Map 1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:28:58,227 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1033895936, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:58,227 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_22_00_000000_0 vertexName: Map 1 2018-01-05T11:28:58,228 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000043, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_22_00_000000_0, eventCount=1 } 2018-01-05T11:28:58,228 DEBUG [TezTR-518813_1_22_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_22 queryId: djaiswal_20180105112857_0ac4bddb-d537-4cc8-9668-b29df90b2cc0 instanceType: TaskRunner2Callable 2018-01-05T11:28:58,228 INFO [TezTR-518813_1_22_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_22_00_000000_0 2018-01-05T11:28:58,228 DEBUG [TezTR-518813_1_22_0_0_0] log.PerfLogger: 2018-01-05T11:28:58,229 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:58,230 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:58,232 DEBUG [TezTR-518813_1_22_0_0_0] log.PerfLogger: 2018-01-05T11:28:58,232 INFO [TezTR-518813_1_22_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:58,232 INFO [TezTR-518813_1_22_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:58,232 INFO [IPC Server handler 21 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_22_00_000000_0 sent events: (0-1). 2018-01-05T11:28:58,232 INFO [I/O Setup 0 Initialize: {cbo_t1}] input.MRInput: cbo_t1 using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:28:58,232 INFO [I/O Setup 0 Initialize: {cbo_t1}] input.MRInputLegacy: cbo_t1 MRInputLegacy deferring initialization 2018-01-05T11:28:58,232 INFO [I/O Setup 1 Initialize: {out_Map 1}] output.MROutput: out_Map 1: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:28:58,232 INFO [TezTR-518813_1_22_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:58,232 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:58,232 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:58,232 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:58,232 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:58,232 INFO [TezTR-518813_1_22_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:58,232 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:28:58,232 INFO [TezTR-518813_1_22_0_0_0] resources.MemoryDistributor: InitialRequests=[cbo_t1:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [out_Map 1:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput] 2018-01-05T11:28:58,232 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_22_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:28:58,232 INFO [TezTR-518813_1_22_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:58,232 INFO [TezTR-518813_1_22_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:58,232 INFO [TezTR-518813_1_22_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 1033895936, TotalAvailable: 692710277, TotalRequested/TotalJVMHeap:0.00 2018-01-05T11:28:58,232 INFO [TezTR-518813_1_22_0_0_0] resources.MemoryDistributor: Allocations=[cbo_t1:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [out_Map 1:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0] 2018-01-05T11:28:58,232 INFO [TezTR-518813_1_22_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:58,232 INFO [TezTR-518813_1_22_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: cbo_t1 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:28:58,232 INFO [TezTR-518813_1_22_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:28:58,232 INFO [TezTR-518813_1_22_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:28:58,232 INFO [I/O Setup 0 Start: {cbo_t1}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: cbo_t1 2018-01-05T11:28:58,232 INFO [TezTR-518813_1_22_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:58,232 INFO [TezTR-518813_1_22_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_22_00_000000_0 2018-01-05T11:28:58,233 DEBUG [TezTR-518813_1_22_0_0_0] log.PerfLogger: 2018-01-05T11:28:58,233 DEBUG [TezTR-518813_1_22_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_22_00_000000_0_10127 2018-01-05T11:28:58,233 DEBUG [TezTR-518813_1_22_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:58,233 DEBUG [TezTR-518813_1_22_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_22_00_000000_0 2018-01-05T11:28:58,233 INFO [TezTR-518813_1_22_0_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112857_0ac4bddb-d537-4cc8-9668-b29df90b2cc0 2018-01-05T11:28:58,233 DEBUG [TezTR-518813_1_22_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:58,233 DEBUG [TezTR-518813_1_22_0_0_0] log.PerfLogger: 2018-01-05T11:28:58,233 DEBUG [TezTR-518813_1_22_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:58,233 DEBUG [TezTR-518813_1_22_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:58,233 DEBUG [TezTR-518813_1_22_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112857_0ac4bddb-d537-4cc8-9668-b29df90b2cc0_Map 1__MAP_PLAN__ 2018-01-05T11:28:58,233 INFO [TezTR-518813_1_22_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_945_7709530907020198391-1/djaiswal/_tez_scratch_dir/97d740b9-22c3-4b02-84a1-7b3a720c5f35/map.xml 2018-01-05T11:28:58,233 DEBUG [TezTR-518813_1_22_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_945_7709530907020198391-1/djaiswal/_tez_scratch_dir/97d740b9-22c3-4b02-84a1-7b3a720c5f35/map.xml 2018-01-05T11:28:58,233 DEBUG [TezTR-518813_1_22_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_945_7709530907020198391-1/djaiswal/_tez_scratch_dir/97d740b9-22c3-4b02-84a1-7b3a720c5f35/map.xml 2018-01-05T11:28:58,233 DEBUG [TezTR-518813_1_22_0_0_0] log.PerfLogger: 2018-01-05T11:28:58,233 INFO [TezTR-518813_1_22_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:58,233 DEBUG [TezTR-518813_1_22_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,233 DEBUG [TezTR-518813_1_22_0_0_0] log.PerfLogger: 2018-01-05T11:28:58,234 INFO [TezTR-518813_1_22_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 2.00KB 2018-01-05T11:28:58,234 DEBUG [TezTR-518813_1_22_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:58,234 INFO [TezTR-518813_1_22_0_0_0] tez.MapRecordProcessor: The input names are: [cbo_t1] 2018-01-05T11:28:58,234 INFO [TezTR-518813_1_22_0_0_0] input.MRInput: Initialized MRInput: cbo_t1 2018-01-05T11:28:58,234 DEBUG [TezTR-518813_1_22_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 1 2018-01-05T11:28:58,234 INFO [TezTR-518813_1_22_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_945_7709530907020198391-1/djaiswal/_tez_scratch_dir/97d740b9-22c3-4b02-84a1-7b3a720c5f35/map.xml 2018-01-05T11:28:58,234 DEBUG [TezTR-518813_1_22_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:58,234 DEBUG [TezTR-518813_1_22_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1/dt=2014/cbo_t1.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.cbo_t1, field.delim=,, columns.types=string:string:int:float:boolean, serialization.ddl=struct cbo_t1 { string key, string value, i32 c_int, float c_float, bool c_boolean}, serialization.format=,, columns=key,value,c_int,c_float,c_boolean, columns.comments=, partition_columns.types=string, partition_columns=dt, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180536}, jobProperties=null], partSpec={dt=2014}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string:int:float:boolean, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1/dt=2014, columns=key,value,c_int,c_float,c_boolean, partition_columns.types=string, serialization.format=,, numRows=0, numFiles=1, serialization.ddl=struct cbo_t1 { string key, string value, i32 c_int, float c_float, bool c_boolean}, transient_lastDdlTime=1515180537, rawDataSize=0, field.delim=,, columns.comments=, totalSize=282, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=dt, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.cbo_t1}, baseFileName=dt=2014, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1/dt=2014=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.cbo_t1, field.delim=,, columns.types=string:string:int:float:boolean, serialization.ddl=struct cbo_t1 { string key, string value, i32 c_int, float c_float, bool c_boolean}, serialization.format=,, columns=key,value,c_int,c_float,c_boolean, columns.comments=, partition_columns.types=string, partition_columns=dt, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180536}, jobProperties=null], partSpec={dt=2014}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string:int:float:boolean, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1/dt=2014, columns=key,value,c_int,c_float,c_boolean, partition_columns.types=string, serialization.format=,, numRows=0, numFiles=1, serialization.ddl=struct cbo_t1 { string key, string value, i32 c_int, float c_float, bool c_boolean}, transient_lastDdlTime=1515180537, rawDataSize=0, field.delim=,, columns.comments=, totalSize=282, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=dt, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.cbo_t1}, baseFileName=dt=2014, vectorPartitionDesc=null]} 2018-01-05T11:28:58,234 DEBUG [TezTR-518813_1_22_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:58,234 INFO [TezTR-518813_1_22_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:58,234 DEBUG [TezTR-518813_1_22_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:58,238 DEBUG [TezTR-518813_1_22_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:58,238 INFO [TezTR-518813_1_22_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_945_7709530907020198391-1/djaiswal/_tez_scratch_dir/97d740b9-22c3-4b02-84a1-7b3a720c5f35/map.xml 2018-01-05T11:28:58,238 DEBUG [TezTR-518813_1_22_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:58,238 INFO [TezTR-518813_1_22_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_22_00_000000_0 2018-01-05T11:28:58,238 INFO [TezTR-518813_1_22_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value, c_int, c_float, c_boolean] / schema.evolution.columns.types [string, string, int, float, boolean] (isAcidRead false) 2018-01-05T11:28:58,239 WARN [TezTR-518813_1_22_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1/dt=2014/cbo_t1.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:28:58,239 DEBUG [TezTR-518813_1_22_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:28:58,239 WARN [TezTR-518813_1_22_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1/dt=2014/cbo_t1.txt:0+282 2018-01-05T11:28:58,239 DEBUG [TezTR-518813_1_22_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1/dt=2014/cbo_t1.txt 2018-01-05T11:28:58,239 INFO [TezTR-518813_1_22_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1/dt=2014/cbo_t1.txt:0+282], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=282} 2018-01-05T11:28:58,239 INFO [TezTR-518813_1_22_0_0_0] input.MRInput: cbo_t1 initialized RecordReader from event 2018-01-05T11:28:58,239 DEBUG [TezTR-518813_1_22_0_0_0] tez.MapRecordProcessor: Starting Output: out_Map 1 2018-01-05T11:28:58,239 INFO [TezTR-518813_1_22_0_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:28:58,239 INFO [TezTR-518813_1_22_0_0_0] exec.MapOperator: Initializing operator MAP[0] 2018-01-05T11:28:58,239 DEBUG [TezTR-518813_1_22_0_0_0] exec.MapOperator: Initialization Done 0 MAP 2018-01-05T11:28:58,239 DEBUG [TezTR-518813_1_22_0_0_0] exec.MapOperator: Operator 0 MAP initialized 2018-01-05T11:28:58,239 DEBUG [TezTR-518813_1_22_0_0_0] exec.MapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:28:58,239 DEBUG [TezTR-518813_1_22_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,239 DEBUG [TezTR-518813_1_22_0_0_0] exec.MapOperator: Adding alias cbo_t1 to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1/dt=2014 2018-01-05T11:28:58,239 DEBUG [TezTR-518813_1_22_0_0_0] exec.MapOperator: dump TS[0] struct 2018-01-05T11:28:58,239 INFO [TezTR-518813_1_22_0_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:28:58,240 INFO [TezTR-518813_1_22_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:58,240 DEBUG [TezTR-518813_1_22_0_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:28:58,240 INFO [TezTR-518813_1_22_0_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:28:58,240 DEBUG [TezTR-518813_1_22_0_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:28:58,240 DEBUG [TezTR-518813_1_22_0_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:28:58,240 DEBUG [TezTR-518813_1_22_0_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:28:58,240 DEBUG [TezTR-518813_1_22_0_0_0] log.PerfLogger: 2018-01-05T11:28:58,240 INFO [TezTR-518813_1_22_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_945_7709530907020198391-1/djaiswal/_tez_scratch_dir/97d740b9-22c3-4b02-84a1-7b3a720c5f35/map.xml 2018-01-05T11:28:58,240 DEBUG [TezTR-518813_1_22_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:58,240 DEBUG [TezTR-518813_1_22_0_0_0] exec.MapOperator: Processing alias(es) cbo_t1 for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1/dt=2014/cbo_t1.txt 2018-01-05T11:28:58,240 INFO [TezTR-518813_1_22_0_0_0] exec.TableScanOperator: Stats Gathering found a new partition spec = dt=2014 2018-01-05T11:28:58,240 INFO [TezTR-518813_1_22_0_0_0] exec.MapOperator: MAP[0]: records read - 1 2018-01-05T11:28:58,240 INFO [TezTR-518813_1_22_0_0_0] exec.MapOperator: MAP[0]: records read - 10 2018-01-05T11:28:58,240 DEBUG [TezTR-518813_1_22_0_0_0] log.PerfLogger: 2018-01-05T11:28:58,240 DEBUG [TezTR-518813_1_22_0_0_0] mr.ObjectCache: djaiswal_20180105112857_0ac4bddb-d537-4cc8-9668-b29df90b2cc0_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:28:58,240 DEBUG [TezTR-518813_1_22_0_0_0] exec.MapOperator: close called for operator MAP[0] 2018-01-05T11:28:58,240 INFO [TezTR-518813_1_22_0_0_0] exec.MapOperator: Closing operator MAP[0] 2018-01-05T11:28:58,240 INFO [TezTR-518813_1_22_0_0_0] exec.MapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:20, 2018-01-05T11:28:58,240 DEBUG [TezTR-518813_1_22_0_0_0] exec.MapOperator: Closing child = TS[0] 2018-01-05T11:28:58,240 DEBUG [TezTR-518813_1_22_0_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:28:58,240 DEBUG [TezTR-518813_1_22_0_0_0] exec.TableScanOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:58,240 INFO [TezTR-518813_1_22_0_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:28:58,240 DEBUG [TezTR-518813_1_22_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,240 DEBUG [TezTR-518813_1_22_0_0_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_945_7709530907020198391-1/-mr-10000 2018-01-05T11:28:58,240 INFO [TezTR-518813_1_22_0_0_0] exec.TableScanOperator: publishing : default.cbo_t1/dt=2014/ : {numRows=20, rawDataSize=262} 2018-01-05T11:28:58,261 DEBUG [TezTR-518813_1_22_0_0_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_945_7709530907020198391-1/-mr-10000/tmpstats-0_0 2018-01-05T11:28:58,261 DEBUG [TezTR-518813_1_22_0_0_0] fs.FSStatsPublisher: Writing stats in it : {default.cbo_t1/dt=2014/={numRows=20, rawDataSize=262}} 2018-01-05T11:28:58,261 DEBUG [TezTR-518813_1_22_0_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:28:58,261 DEBUG [TezTR-518813_1_22_0_0_0] exec.MapOperator: 0 Close done 2018-01-05T11:28:58,262 INFO [TezTR-518813_1_22_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_22_00_000000_0 2018-01-05T11:28:58,262 INFO [TezTR-518813_1_22_0_0_0] output.MROutput: out_Map 1 closed 2018-01-05T11:28:58,262 DEBUG [TezTR-518813_1_22_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:58,262 INFO [TezTR-518813_1_22_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:58,262 INFO [TezTR-518813_1_22_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_22_00_000000_0, askedToStop=false 2018-01-05T11:28:58,262 INFO [TezTR-518813_1_22_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_22_00_000000_0, stopRequested=false 2018-01-05T11:28:58,262 INFO [TezTR-518813_1_22_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_22_00_000000_0: Counters: 5 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=20, INPUT_SPLIT_LENGTH_BYTES=282, OUTPUT_RECORDS=0][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=20]] 2018-01-05T11:28:58,262 DEBUG [TezTR-518813_1_22_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_22_0_0_0 thread id: 1524 scheme: file bytesRead: 298 bytesWritten: 91 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:58,262 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_22_00_000000_0, isTaskDone=true 2018-01-05T11:28:58,262 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000043, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_22_00_000000_0, eventCount=2 } 2018-01-05T11:28:58,263 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_22][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_22_00_000000_0, creationTime=1515180538211, allocationTime=1515180538212, startTime=1515180538214, finishTime=1515180538263, timeTaken=49, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:58,263 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:58,263 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_22_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:58,263 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_22_00_000000_0 2018-01-05T11:28:58,263 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000043=36 2018-01-05T11:28:58,263 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_22_00_000000_0: true 2018-01-05T11:28:58,263 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_22_00_000000_0 2018-01-05T11:28:58,263 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_22_00_000000_0 2018-01-05T11:28:58,263 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_22_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:58,263 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000043, QueryId=djaiswal_20180105112857_0ac4bddb-d537-4cc8-9668-b29df90b2cc0-22, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:28:58,263 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_22_00_000000_0; false 2018-01-05T11:28:58,263 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_22_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:58,263 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_22_00_000000_0 2018-01-05T11:28:58,263 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_22_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:58,263 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_22][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_22_00_000000, startTime=1515180538214, finishTime=1515180538263, timeTaken=49, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_22_00_000000_0, diagnostics=, counters=Counters: 11, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=298, FILE_BYTES_WRITTEN=91, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=20, INPUT_SPLIT_LENGTH_BYTES=282, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=20 2018-01-05T11:28:58,263 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:58,263 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:58,264 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_22_00 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:58,264 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_22_00 [Map 1] 2018-01-05T11:28:58,264 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_22][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_22_00, initRequestedTime=1515180538181, initedTime=1515180538210, startRequestedTime=1515180538188, startedTime=1515180538210, finishTime=1515180538264, timeTaken=54, status=SUCCEEDED, diagnostics=, counters=Counters: 11, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=298, FILE_BYTES_WRITTEN=91, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=20, INPUT_SPLIT_LENGTH_BYTES=282, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=20, vertexStats=firstTaskStartTime=1515180538214, firstTasksToStart=[ task_1515180518813_0001_22_00_000000 ], lastTaskFinishTime=1515180538263, lastTasksToFinish=[ task_1515180518813_0001_22_00_000000 ], minTaskDuration=49, maxTaskDuration=49, avgTaskDuration=49.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_22_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_22_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:58,264 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_22_00 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:58,264 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_22][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000043, stoppedTime=1515180538264, exitStatus=0 2018-01-05T11:28:58,264 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000043 exited with diagnostics set to Container completed. 2018-01-05T11:28:58,264 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_22_00 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1 2018-01-05T11:28:58,264 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null 2018-01-05T11:28:58,264 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_22_00 [Map 1] 2018-01-05T11:28:58,265 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_22, queueSize=0 2018-01-05T11:28:58,265 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_22][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_22, startTime=1515180538181, finishTime=1515180538264, timeTaken=83, status=SUCCEEDED, diagnostics=, counters=Counters: 15, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, OTHER_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=160, AM_GC_TIME_MILLIS=0, File System Counters, FILE_BYTES_READ=298, FILE_BYTES_WRITTEN=91, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=20, INPUT_SPLIT_LENGTH_BYTES=282, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=20 2018-01-05T11:28:58,265 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_22 finished with state: SUCCEEDED 2018-01-05T11:28:58,265 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_22 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:28:58,265 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000043 2018-01-05T11:28:58,266 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_22, dagState=SUCCEEDED 2018-01-05T11:28:58,266 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_22_post 2018-01-05T11:28:58,266 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:58,266 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:28:58,266 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:28:58,266 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 22 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=1, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=1,NumRequestsWithoutLocation=1, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=1} 2018-01-05T11:28:58,266 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:28:58,266 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=analyze table cbo_t1 partition ...statistics (Stage-0), with id=dag_1515180518813_0001_22 2018-01-05T11:28:58,266 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 22, to 10.22.8.182:49818 2018-01-05T11:28:58,266 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=1, Remaining=0 2018-01-05T11:28:58,266 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=analyze table cbo_t1 partition ...statistics (Stage-0), with id=dag_1515180518813_0001_22 2018-01-05T11:28:58,266 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=22} 2018-01-05T11:28:58,266 INFO [IPC Server handler 0 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=22}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:28:58,266 INFO [IPC Server handler 0 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/22 after 300 seconds 2018-01-05T11:28:58,266 ERROR [IPC Server handler 0 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112857_0ac4bddb-d537-4cc8-9668-b29df90b2cc0, dag_1515180518813_0001_22 2018-01-05T11:28:58,266 INFO [IPC Server handler 0 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112857_0ac4bddb-d537-4cc8-9668-b29df90b2cc0 2018-01-05T11:28:58,266 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=22} = 0 2018-01-05T11:28:58,266 DEBUG [IPC Server handler 0 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=22} 2018-01-05T11:28:58,266 DEBUG [IPC Server handler 0 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:28:58,266 DEBUG [IPC Server handler 0 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=22} 2018-01-05T11:28:58,266 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,266 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,267 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 2018-01-05T11:28:58,267 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,267 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-2:STATS] in serial mode 2018-01-05T11:28:58,267 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@1e5ba462, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:58,267 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:58,267 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,267 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:58,267 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:58,268 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:58,268 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:58,268 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,268 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,268 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:58,268 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:58,268 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,268 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:58,268 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,268 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:58,268 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:58,268 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,268 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:58,268 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_945_7709530907020198391-1/-mr-10000 2018-01-05T11:28:58,268 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_945_7709530907020198391-1/-mr-10000 2018-01-05T11:28:58,270 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Getting file stats of all partitions. threadpool size:15 2018-01-05T11:28:58,271 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Finished getting file stats of all partitions! 2018-01-05T11:28:58,271 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.cbo_t1/dt=2014/ numRows 2018-01-05T11:28:58,271 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.cbo_t1/dt=2014/ numRows 20 2018-01-05T11:28:58,271 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.cbo_t1/dt=2014/ rawDataSize 2018-01-05T11:28:58,271 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.cbo_t1/dt=2014/ rawDataSize 262 2018-01-05T11:28:58,271 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Partition {dt=2014} stats: [numFiles=1, numRows=20, totalSize=282, rawDataSize=262] 2018-01-05T11:28:58,272 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,272 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:58,272 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:58,272 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:58,272 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:58,272 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,273 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,273 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,273 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,274 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,274 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:58,274 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:58,274 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,274 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:58,274 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,274 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_partitions : db=default tbl=cbo_t1 2018-01-05T11:28:58,274 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_partitions : db=default tbl=cbo_t1 2018-01-05T11:28:58,274 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: New partition values:[2014] 2018-01-05T11:28:58,277 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,283 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,286 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,286 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,286 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: About to delete stats tmp dir :file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_945_7709530907020198391-1/-mr-10000 2018-01-05T11:28:58,287 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,287 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,287 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: analyze table cbo_t1 partition (dt) compute statistics 2018-01-05T11:28:58,287 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:28:58,287 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@cbo_t1 2018-01-05T11:28:58,287 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@cbo_t1@dt=2014 2018-01-05T11:28:58,287 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@cbo_t1 2018-01-05T11:28:58,287 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@cbo_t1@dt=2014 2018-01-05T11:28:58,287 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,287 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,287 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:58,287 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_partitions_(String, String, List, EnvironmentContext, )=12} 2018-01-05T11:28:58,287 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112857_0ac4bddb-d537-4cc8-9668-b29df90b2cc0); Time taken: 0.208 seconds 2018-01-05T11:28:58,287 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:58,287 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,287 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:58,287 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,287 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query analyze table cbo_t1 partition (dt) compute statistics 2018-01-05T11:28:58,287 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-57_945_7709530907020198391-1 2018-01-05T11:28:58,288 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.342 seconds 2018-01-05T11:28:58,288 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:58,288 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:58,288 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:58,288 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:58,288 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,288 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:58,288 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,288 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,288 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,288 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: analyze table cbo_t1 compute statistics for columns key, value, c_int, c_float, c_boolean 2018-01-05T11:28:58,288 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89): analyze table cbo_t1 compute statistics for columns key, value, c_int, c_float, c_boolean 2018-01-05T11:28:58,288 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,288 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: analyze table cbo_t1 compute statistics for columns key, value, c_int, c_float, c_boolean 2018-01-05T11:28:58,288 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:58,289 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,289 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,289 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,289 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,289 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:58,289 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,289 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t1 2018-01-05T11:28:58,289 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t1 2018-01-05T11:28:58,291 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,292 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,292 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: select compute_stats(`key`, 'hll') , compute_stats(`value`, 'hll') , compute_stats(`c_int`, 'hll') , compute_stats(`c_float`, 'hll') , compute_stats(`c_boolean`, 'hll') , `dt` from `default`.`cbo_t1` group by `dt` 2018-01-05T11:28:58,292 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: select compute_stats(`key`, 'hll') , compute_stats(`value`, 'hll') , compute_stats(`c_int`, 'hll') , compute_stats(`c_float`, 'hll') , compute_stats(`c_boolean`, 'hll') , `dt` from `default`.`cbo_t1` group by `dt` 2018-01-05T11:28:58,293 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:58,293 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Invoking analyze on rewritten query 2018-01-05T11:28:58,293 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Starting Semantic Analysis 2018-01-05T11:28:58,293 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:58,293 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:58,293 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:58,294 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:58,294 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:58,294 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed phase 1 of Semantic Analysis 2018-01-05T11:28:58,294 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for source tables 2018-01-05T11:28:58,294 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,294 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t1 2018-01-05T11:28:58,294 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t1 2018-01-05T11:28:58,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,297 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for subqueries 2018-01-05T11:28:58,297 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for destination tables 2018-01-05T11:28:58,310 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed getting MetaData in Semantic Analysis 2018-01-05T11:28:58,310 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_292_1911154602830553799-1/-mr-10001 2018-01-05T11:28:58,310 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Table Plan for cbo_t1 TS[0] 2018-01-05T11:28:58,311 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:58,311 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:58,311 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:58,311 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:58,311 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:58,312 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: RR before GB cbo_t1{(key,key: string)(value,value: string)(c_int,c_int: int)(c_float,c_float: float)(c_boolean,c_boolean: boolean)(dt,dt: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)(RAW__DATA__SIZE,RAW__DATA__SIZE: bigint)} after GB {((tok_table_or_col dt),_col0: string)((tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct)((tok_function compute_stats (tok_table_or_col value) 'hll'),_col2: struct)((tok_function compute_stats (tok_table_or_col c_int) 'hll'),_col3: struct)((tok_function compute_stats (tok_table_or_col c_float) 'hll'),_col4: struct)((tok_function compute_stats (tok_table_or_col c_boolean) 'hll'),_col5: struct)} cbo_t1{(dt,_col0: string)} 2018-01-05T11:28:58,312 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: tree: (tok_select (tok_selexpr (tok_function compute_stats (tok_table_or_col key) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col value) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col c_int) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col c_float) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col c_boolean) 'hll')) (tok_selexpr (tok_table_or_col dt))) 2018-01-05T11:28:58,312 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: genSelectPlan: input = {((tok_table_or_col dt),_col0: string)((tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct)((tok_function compute_stats (tok_table_or_col value) 'hll'),_col2: struct)((tok_function compute_stats (tok_table_or_col c_int) 'hll'),_col3: struct)((tok_function compute_stats (tok_table_or_col c_float) 'hll'),_col4: struct)((tok_function compute_stats (tok_table_or_col c_boolean) 'hll'),_col5: struct)} cbo_t1{(dt,_col0: string)} starRr = cbo_t1{(key,key: string)(value,value: string)(c_int,c_int: int)(c_float,c_float: float)(c_boolean,c_boolean: boolean)(dt,dt: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)(RAW__DATA__SIZE,RAW__DATA__SIZE: bigint)} 2018-01-05T11:28:58,312 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Select Plan row schema: null{(_c0,_col0: struct)(_c1,_col1: struct)(_c2,_col2: struct)(_c3,_col3: struct)(_c4,_col4: struct)(dt,_col5: string)} cbo_t1{(dt,_col5: string)} 2018-01-05T11:28:58,312 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Select Plan for clause: insclause-0 2018-01-05T11:28:58,312 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_292_1911154602830553799-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-58_292_1911154602830553799-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_292_1911154602830553799-1/-mr-10000 2018-01-05T11:28:58,312 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_292_1911154602830553799-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-58_292_1911154602830553799-1 2018-01-05T11:28:58,337 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_292_1911154602830553799-1/-mr-10003 2018-01-05T11:28:58,337 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_292_1911154602830553799-1/-mr-10000 row schema: null{(_c0,_col0: struct)(_c1,_col1: struct)(_c2,_col2: struct)(_c3,_col3: struct)(_c4,_col4: struct)(dt,_col5: string)} cbo_t1{(dt,_col5: string)} 2018-01-05T11:28:58,337 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Body Plan for Query Block null 2018-01-05T11:28:58,337 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Plan for Query Block null 2018-01-05T11:28:58,337 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Before logical optimization TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-SEL[5]-FS[6] 2018-01-05T11:28:58,337 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,337 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,337 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] lineage.Generator: Time taken for lineage transform=1 2018-01-05T11:28:58,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{cbo_t1}key,value: string|{cbo_t1}value,c_int: int|{cbo_t1}c_int,c_float: float|{cbo_t1}c_float,c_boolean: boolean|{cbo_t1}c_boolean,dt: string|{cbo_t1}dt,BLOCK__OFFSET__INSIDE__FILE: bigint|{cbo_t1}block__offset__inside__file,INPUT__FILE__NAME: string|{cbo_t1}input__file__name,ROW__ID: struct|{cbo_t1}row__id,RAW__DATA__SIZE: bigint|{cbo_t1}raw__data__size) 2018-01-05T11:28:58,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: string|{cbo_t1}key,value: string|{cbo_t1}value,c_int: int|{cbo_t1}c_int,c_float: float|{cbo_t1}c_float,c_boolean: boolean|{cbo_t1}c_boolean,dt: string|{cbo_t1}dt,BLOCK__OFFSET__INSIDE__FILE: bigint|{cbo_t1}block__offset__inside__file,INPUT__FILE__NAME: string|{cbo_t1}input__file__name,ROW__ID: struct|{cbo_t1}row__id,RAW__DATA__SIZE: bigint|{cbo_t1}raw__data__size) 2018-01-05T11:28:58,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:58,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:58,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value] Column[c_int] Column[c_float] Column[c_boolean] Column[dt] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:28:58,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: string|{}(tok_table_or_col dt),_col1: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col c_int) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col c_float) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col c_boolean) 'hll')) 2018-01-05T11:28:58,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:58,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:58,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(KEY._col0: string|{}(tok_table_or_col dt),VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll'),VALUE._col2: struct|{}(tok_function compute_stats (tok_table_or_col c_int) 'hll'),VALUE._col3: struct|{}(tok_function compute_stats (tok_table_or_col c_float) 'hll'),VALUE._col4: struct|{}(tok_function compute_stats (tok_table_or_col c_boolean) 'hll')) 2018-01-05T11:28:58,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:58,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:58,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: string|{}(tok_table_or_col dt),_col1: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col c_int) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col c_float) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col c_boolean) 'hll')) 2018-01-05T11:28:58,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:58,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:58,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1,_col2: struct|{null}_c2,_col3: struct|{null}_c3,_col4: struct|{null}_c4,_col5: string|{null}dt) 2018-01-05T11:28:58,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:58,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:58,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col1] Column[_col2] Column[_col3] Column[_col4] Column[_col5] Column[_col0]) 2018-01-05T11:28:58,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{},_col2: struct|{},_col3: struct|{},_col4: struct|{},_col5: string|{}) 2018-01-05T11:28:58,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:58,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:58,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,338 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for FS(6) 2018-01-05T11:28:58,338 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(5) 2018-01-05T11:28:58,338 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for GBY(4) 2018-01-05T11:28:58,338 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for RS(3) 2018-01-05T11:28:58,338 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for GBY(2) 2018-01-05T11:28:58,338 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(1) 2018-01-05T11:28:58,338 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:28:58,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.PredicatePushDown: After PPD: TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-SEL[5]-FS[6] 2018-01-05T11:28:58,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,338 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{cbo_t1}key,value: string|{cbo_t1}value,c_int: int|{cbo_t1}c_int,c_float: float|{cbo_t1}c_float,c_boolean: boolean|{cbo_t1}c_boolean,dt: string|{cbo_t1}dt,BLOCK__OFFSET__INSIDE__FILE: bigint|{cbo_t1}block__offset__inside__file,INPUT__FILE__NAME: string|{cbo_t1}input__file__name,ROW__ID: struct|{cbo_t1}row__id,RAW__DATA__SIZE: bigint|{cbo_t1}raw__data__size) 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: string|{cbo_t1}key,value: string|{cbo_t1}value,c_int: int|{cbo_t1}c_int,c_float: float|{cbo_t1}c_float,c_boolean: boolean|{cbo_t1}c_boolean,dt: string|{cbo_t1}dt,BLOCK__OFFSET__INSIDE__FILE: bigint|{cbo_t1}block__offset__inside__file,INPUT__FILE__NAME: string|{cbo_t1}input__file__name,ROW__ID: struct|{cbo_t1}row__id,RAW__DATA__SIZE: bigint|{cbo_t1}raw__data__size) 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value] Column[c_int] Column[c_float] Column[c_boolean] Column[dt] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: string|{}(tok_table_or_col dt),_col1: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col c_int) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col c_float) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col c_boolean) 'hll')) 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(KEY._col0: string|{}(tok_table_or_col dt),VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll'),VALUE._col2: struct|{}(tok_function compute_stats (tok_table_or_col c_int) 'hll'),VALUE._col3: struct|{}(tok_function compute_stats (tok_table_or_col c_float) 'hll'),VALUE._col4: struct|{}(tok_function compute_stats (tok_table_or_col c_boolean) 'hll')) 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: string|{}(tok_table_or_col dt),_col1: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col c_int) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col c_float) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col c_boolean) 'hll')) 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1,_col2: struct|{null}_c2,_col3: struct|{null}_c3,_col4: struct|{null}_c4,_col5: string|{null}dt) 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col1] Column[_col2] Column[_col3] Column[_col4] Column[_col5] Column[_col0]) 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{},_col2: struct|{},_col3: struct|{},_col4: struct|{},_col5: string|{}) 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{cbo_t1}key,value: string|{cbo_t1}value,c_int: int|{cbo_t1}c_int,c_float: float|{cbo_t1}c_float,c_boolean: boolean|{cbo_t1}c_boolean,dt: string|{cbo_t1}dt,BLOCK__OFFSET__INSIDE__FILE: bigint|{cbo_t1}block__offset__inside__file,INPUT__FILE__NAME: string|{cbo_t1}input__file__name,ROW__ID: struct|{cbo_t1}row__id,RAW__DATA__SIZE: bigint|{cbo_t1}raw__data__size) 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: string|{cbo_t1}key,value: string|{cbo_t1}value,c_int: int|{cbo_t1}c_int,c_float: float|{cbo_t1}c_float,c_boolean: boolean|{cbo_t1}c_boolean,dt: string|{cbo_t1}dt,BLOCK__OFFSET__INSIDE__FILE: bigint|{cbo_t1}block__offset__inside__file,INPUT__FILE__NAME: string|{cbo_t1}input__file__name,ROW__ID: struct|{cbo_t1}row__id,RAW__DATA__SIZE: bigint|{cbo_t1}raw__data__size) 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value] Column[c_int] Column[c_float] Column[c_boolean] Column[dt] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: string|{}(tok_table_or_col dt),_col1: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col c_int) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col c_float) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col c_boolean) 'hll')) 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(KEY._col0: string|{}(tok_table_or_col dt),VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll'),VALUE._col2: struct|{}(tok_function compute_stats (tok_table_or_col c_int) 'hll'),VALUE._col3: struct|{}(tok_function compute_stats (tok_table_or_col c_float) 'hll'),VALUE._col4: struct|{}(tok_function compute_stats (tok_table_or_col c_boolean) 'hll')) 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: string|{}(tok_table_or_col dt),_col1: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col c_int) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col c_float) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col c_boolean) 'hll')) 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1,_col2: struct|{null}_c2,_col3: struct|{null}_c3,_col4: struct|{null}_c4,_col5: string|{null}dt) 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col1] Column[_col2] Column[_col3] Column[_col4] Column[_col5] Column[_col0]) 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{},_col2: struct|{},_col3: struct|{},_col4: struct|{},_col5: string|{}) 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,339 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,340 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,340 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,340 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partitions : db=default tbl=cbo_t1 2018-01-05T11:28:58,340 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions : db=default tbl=cbo_t1 2018-01-05T11:28:58,342 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,343 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.351402ms + 0.006527ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? ] 2018-01-05T11:28:58,344 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.222918ms + 0.050379ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (10) order by "PART_NAME" asc] 2018-01-05T11:28:58,344 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.114129ms + 0.084079ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (10) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:28:58,344 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.127737ms + 0.025407ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (10) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:58,344 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.08125ms + 0.00758ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (31) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:28:58,344 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.072902ms + 0.007296ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (31) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:58,344 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.129924ms + 0.013906ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (31) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:58,345 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.120156ms + 0.008147ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (31) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:58,345 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.127781ms + 0.037281ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (19) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:58,345 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.120955ms + 0.049972ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (31) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:28:58,345 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,348 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,349 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,349 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,349 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,349 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,349 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 3 key:[Column[_col0]] 2018-01-05T11:28:58,349 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 3 oldColExprMap: {KEY._col0=Column[_col0], VALUE._col2=Column[_col3], VALUE._col3=Column[_col4], VALUE._col4=Column[_col5], VALUE._col0=Column[_col1], VALUE._col1=Column[_col2]} 2018-01-05T11:28:58,349 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 3 newColExprMap: {KEY._col0=Column[_col0], VALUE._col2=Column[_col3], VALUE._col3=Column[_col4], VALUE._col4=Column[_col5], VALUE._col0=Column[_col1], VALUE._col1=Column[_col2]} 2018-01-05T11:28:58,349 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,349 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,350 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,350 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,350 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,350 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,350 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,350 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,350 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,350 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,350 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,350 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,350 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,350 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,350 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,350 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,350 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,350 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,350 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,350 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,350 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,350 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,350 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,350 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,350 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,350 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: After logical optimization TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-SEL[5]-FS[6] 2018-01-05T11:28:58,350 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,350 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,350 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,350 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.StatsUtils: Estimated average row size: 212 2018-01-05T11:28:58,350 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,350 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_aggr_stats_for: db=default table=cbo_t1 2018-01-05T11:28:58,350 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_aggr_stats_for: db=default table=cbo_t1 2018-01-05T11:28:58,353 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.AggregateStatsCache: No aggregate stats cached for database:default, table:cbo_t1, column:key 2018-01-05T11:28:58,356 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 3.20531ms + 0.007157ms, the query is [select count("COLUMN_NAME") from "PART_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (?,?,?,?,?) and "PARTITION_NAME" in (?) group by "PARTITION_NAME"] 2018-01-05T11:28:58,358 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 1.817729ms + 0.00533ms, the query is [select "PARTITION_NAME", "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "PART_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (%1$s) AND "PARTITION_NAME" in (%2$s) order by "PARTITION_NAME"] 2018-01-05T11:28:58,358 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.MetaStoreUtils: No stats data found for: dbName=default tblName=cbo_t1 partNames= [dt=2014] colNames=[key] 2018-01-05T11:28:58,358 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.AggregateStatsCache: No aggregate stats cached for database:default, table:cbo_t1, column:value 2018-01-05T11:28:58,358 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.120416ms + 0.002864ms, the query is [select "PARTITION_NAME", "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "PART_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (%1$s) AND "PARTITION_NAME" in (%2$s) order by "PARTITION_NAME"] 2018-01-05T11:28:58,358 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.MetaStoreUtils: No stats data found for: dbName=default tblName=cbo_t1 partNames= [dt=2014] colNames=[value] 2018-01-05T11:28:58,358 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.AggregateStatsCache: No aggregate stats cached for database:default, table:cbo_t1, column:c_int 2018-01-05T11:28:58,358 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.096522ms + 0.002827ms, the query is [select "PARTITION_NAME", "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "PART_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (%1$s) AND "PARTITION_NAME" in (%2$s) order by "PARTITION_NAME"] 2018-01-05T11:28:58,358 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.MetaStoreUtils: No stats data found for: dbName=default tblName=cbo_t1 partNames= [dt=2014] colNames=[c_int] 2018-01-05T11:28:58,358 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.AggregateStatsCache: No aggregate stats cached for database:default, table:cbo_t1, column:c_float 2018-01-05T11:28:58,358 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.101709ms + 0.002667ms, the query is [select "PARTITION_NAME", "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "PART_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (%1$s) AND "PARTITION_NAME" in (%2$s) order by "PARTITION_NAME"] 2018-01-05T11:28:58,358 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.MetaStoreUtils: No stats data found for: dbName=default tblName=cbo_t1 partNames= [dt=2014] colNames=[c_float] 2018-01-05T11:28:58,358 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.AggregateStatsCache: No aggregate stats cached for database:default, table:cbo_t1, column:c_boolean 2018-01-05T11:28:58,359 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.100703ms + 0.002659ms, the query is [select "PARTITION_NAME", "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "PART_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (%1$s) AND "PARTITION_NAME" in (%2$s) order by "PARTITION_NAME"] 2018-01-05T11:28:58,359 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.MetaStoreUtils: No stats data found for: dbName=default tblName=cbo_t1 partNames= [dt=2014] colNames=[c_boolean] 2018-01-05T11:28:58,359 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: useDensityFunctionForNDVEstimation = false partsFound = 0 ColumnStatisticsObj = [] 2018-01-05T11:28:58,359 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,360 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,360 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting stats (Num rows: 20 Data size: 11542 Basic stats: COMPLETE Column stats: PARTIAL) on TS[0] 2018-01-05T11:28:58,360 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (cbo_t1): numRows: 20 dataSize: 11542 basicStatsState: COMPLETE colStatsState: PARTIAL colStats: {dt= colName: dt colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, c_float= colName: c_float colType: float countDistincts: 4 numNulls: 1 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 1.4E-45 max: 3.4028235E38 ] isPrimaryKey: false isEstimated: true, c_int= colName: c_int colType: int countDistincts: 4 numNulls: 1 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, value= colName: value colType: string countDistincts: 4 numNulls: 1 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: string countDistincts: 4 numNulls: 1 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, c_boolean= colName: c_boolean colType: boolean countDistincts: 2 numNulls: 1 avgColLen: 4.0 numTrues: 10 numFalses: 10 isPrimaryKey: false isEstimated: true} 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting stats (Num rows: 20 Data size: 11542 Basic stats: COMPLETE Column stats: PARTIAL) on SEL[1] 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-SEL[1]: numRows: 20 dataSize: 11542 basicStatsState: COMPLETE colStatsState: PARTIAL colStats: {dt= colName: dt colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, ROW__ID= colName: ROW__ID colType: struct countDistincts: 20 numNulls: 0 avgColLen: 76.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, RAW__DATA__SIZE= colName: RAW__DATA__SIZE colType: bigint countDistincts: 20 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, c_float= colName: c_float colType: float countDistincts: 4 numNulls: 1 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 1.4E-45 max: 3.4028235E38 ] isPrimaryKey: false isEstimated: true, BLOCK__OFFSET__INSIDE__FILE= colName: BLOCK__OFFSET__INSIDE__FILE colType: bigint countDistincts: 20 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, INPUT__FILE__NAME= colName: INPUT__FILE__NAME colType: string countDistincts: 20 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, c_int= colName: c_int colType: int countDistincts: 4 numNulls: 1 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, value= colName: value colType: string countDistincts: 4 numNulls: 1 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: string countDistincts: 4 numNulls: 1 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, c_boolean= colName: c_boolean colType: boolean countDistincts: 2 numNulls: 1 avgColLen: 4.0 numTrues: 10 numFalses: 10 isPrimaryKey: false isEstimated: true} 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[2]: inputSize: 11542 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[2] hashAgg: true 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [Case 3] STATS-GBY[2]: cardinality: 1 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 2184 Basic stats: COMPLETE Column stats: PARTIAL) on GBY[2] 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-GBY[2]: numRows: 1 dataSize: 2184 basicStatsState: COMPLETE colStatsState: PARTIAL colStats: {_col0= colName: _col0 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col4= colName: _col4 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col5= colName: _col5 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 272.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 2184 Basic stats: COMPLETE Column stats: PARTIAL) on RS[3] 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-RS[3]: numRows: 1 dataSize: 2184 basicStatsState: COMPLETE colStatsState: PARTIAL colStats: {KEY._col0= colName: KEY._col0 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col2= colName: VALUE._col2 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col3= colName: VALUE._col3 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col4= colName: VALUE._col4 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 272.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col0= colName: VALUE._col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col1= colName: VALUE._col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[4]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[4] hashAgg: false 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [Case 9] STATS-GBY[4]: cardinality: 1 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 2216 Basic stats: COMPLETE Column stats: PARTIAL) on GBY[4] 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-GBY[4]: numRows: 1 dataSize: 2216 basicStatsState: COMPLETE colStatsState: PARTIAL colStats: {_col0= colName: _col0 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col4= colName: _col4 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col5= colName: _col5 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 272.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting stats (Num rows: 1 Data size: 2216 Basic stats: COMPLETE Column stats: PARTIAL) on SEL[5] 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-SEL[5]: numRows: 1 dataSize: 2216 basicStatsState: COMPLETE colStatsState: PARTIAL colStats: {_col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col4= colName: _col4 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 272.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col5= colName: _col5 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-FS[6]: numRows: 1 dataSize: 2216 basicStatsState: COMPLETE colStatsState: PARTIAL colStats: {_col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col4= colName: _col4 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 272.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col5= colName: _col5 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on TS[0] 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on SEL[1] 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: -1 }) on GBY[2] 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[KEY._col0]]; sort column names: [[KEY._col0]]; bucket count: -1 }) on RS[3] 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: -1 }) on GBY[4] 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col5]]; sort column names: [[_col5]]; bucket count: -1 }) on SEL[5] 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FileSinkOperator: Setting traits ({ bucket column names: [[_col5]]; sort column names: [[_col5]]; bucket count: -1 }) on FS[6] 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,361 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.SetReducerParallelism: Set parallelism for reduce sink RS[3] to: 1 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,361 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: FS, 6 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: SEL, 5 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: GBY, 4 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: RS, 3 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: GBY, 2 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: SEL, 1 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:28:58,362 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Cycle free: true 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{cbo_t1}key,value: string|{cbo_t1}value,c_int: int|{cbo_t1}c_int,c_float: float|{cbo_t1}c_float,c_boolean: boolean|{cbo_t1}c_boolean,dt: string|{cbo_t1}dt,BLOCK__OFFSET__INSIDE__FILE: bigint|{cbo_t1}block__offset__inside__file,INPUT__FILE__NAME: string|{cbo_t1}input__file__name,ROW__ID: struct|{cbo_t1}row__id,RAW__DATA__SIZE: bigint|{cbo_t1}raw__data__size) 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: string|{cbo_t1}key,value: string|{cbo_t1}value,c_int: int|{cbo_t1}c_int,c_float: float|{cbo_t1}c_float,c_boolean: boolean|{cbo_t1}c_boolean,dt: string|{cbo_t1}dt) 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value] Column[c_int] Column[c_float] Column[c_boolean] Column[dt]) 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: string|{}(tok_table_or_col dt),_col1: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col c_int) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col c_float) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col c_boolean) 'hll')) 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(KEY._col0: string|{}(tok_table_or_col dt),VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll'),VALUE._col2: struct|{}(tok_function compute_stats (tok_table_or_col c_int) 'hll'),VALUE._col3: struct|{}(tok_function compute_stats (tok_table_or_col c_float) 'hll'),VALUE._col4: struct|{}(tok_function compute_stats (tok_table_or_col c_boolean) 'hll')) 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: string|{}(tok_table_or_col dt),_col1: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col c_int) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col c_float) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col c_boolean) 'hll')) 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1,_col2: struct|{null}_c2,_col3: struct|{null}_c3,_col4: struct|{null}_c4,_col5: string|{null}dt) 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col1] Column[_col2] Column[_col3] Column[_col4] Column[_col5] Column[_col0]) 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{},_col2: struct|{},_col3: struct|{},_col4: struct|{},_col5: string|{}) 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: TS[0] 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: RS[3] 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1/dt=2014 of table cbo_t1 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1/dt=2014 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: RS[3] 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: GBY[4] 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: FS[6] 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding reduce work (Reducer 2) for GBY[4] 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Setting up reduce sink: RS[3] with following reduce work: Reducer 2 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Removing RS[3] as parent from GBY[4] 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: FS[6] 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:58,362 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping vectorization 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:28:58,363 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: llap mode: only 2018-01-05T11:28:58,363 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(key, 'hll')' 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'key' 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(value, 'hll')' 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'value' 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(c_int, 'hll')' 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'c_int' 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(c_float, 'hll')' 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'c_float' 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(c_boolean, 'hll')' 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'c_boolean' 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'key' 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'value' 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'c_int' 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'c_float' 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'c_boolean' 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'dt' 2018-01-05T11:28:58,363 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@d8156ac can be done in LLAP 2018-01-05T11:28:58,363 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Reducer 2 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking '_col1' 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking '_col2' 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking '_col3' 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking '_col4' 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking '_col5' 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking '_col0' 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col0)' 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col1)' 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col1' 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col2)' 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col2' 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col3)' 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col3' 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col4)' 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col4' 2018-01-05T11:28:58,363 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@3d8c8ca2 can be done in LLAP 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,363 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed plan generation 2018-01-05T11:28:58,363 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validation start 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Not a partition. 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validated dt=2014 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: cbo_t1 2018-01-05T11:28:58,363 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,363 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partitions_ps_with_auth : db=default tbl=cbo_t1[2014] 2018-01-05T11:28:58,363 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions_ps_with_auth : db=default tbl=cbo_t1[2014] 2018-01-05T11:28:58,366 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,368 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000043, QueryId=djaiswal_20180105112857_0ac4bddb-d537-4cc8-9668-b29df90b2cc0-22, DagId=22, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180538226, EndTime=1515180538368 2018-01-05T11:28:58,373 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,373 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,374 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,374 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=cbo_t1[2014] 2018-01-05T11:28:58,374 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=cbo_t1[2014] 2018-01-05T11:28:58,378 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,382 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,382 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,382 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,382 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0, type:struct, comment:null), FieldSchema(name:_c1, type:struct, comment:null), FieldSchema(name:_c2, type:struct, comment:null), FieldSchema(name:_c3, type:struct, comment:null), FieldSchema(name:_c4, type:struct, comment:null), FieldSchema(name:dt, type:string, comment:null)], properties:null) 2018-01-05T11:28:58,382 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,382 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:58,382 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getAggrColStatsFor_(String, String, List, List, )=10, getTable_(String, String, )=6, flushCache_()=0, getPartitionWithAuthInfo_(String, String, List, String, List, )=8, listPartitions_(String, String, short, )=9, listPartitionsWithAuthInfo_(String, String, List, short, String, List, )=10} 2018-01-05T11:28:58,382 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89); Time taken: 0.094 seconds 2018-01-05T11:28:58,382 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:58,383 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,383 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89): analyze table cbo_t1 compute statistics for columns key, value, c_int, c_float, c_boolean 2018-01-05T11:28:58,384 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,384 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: analyze table cbo_t1 compute statistics for columns key, value, c_int, c_float, c_boolean 2018-01-05T11:28:58,384 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:28:58,384 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@cbo_t1 2018-01-05T11:28:58,384 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@cbo_t1@dt=2014 2018-01-05T11:28:58,384 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@cbo_t1 2018-01-05T11:28:58,384 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@cbo_t1@dt=2014 2018-01-05T11:28:58,384 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_292_1911154602830553799-1/-mr-10000 2018-01-05T11:28:58,384 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,384 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,384 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,384 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Query ID = djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89 2018-01-05T11:28:58,384 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Total jobs = 1 2018-01-05T11:28:58,384 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,384 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:28:58,384 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MAPRED] in serial mode 2018-01-05T11:28:58,407 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_288_8305321905597716466-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:28:58,428 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:28:58,429 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:28:58,429 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:28:58,429 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89 2018-01-05T11:28:58,429 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Session is already open 2018-01-05T11:28:58,429 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:28:58,429 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,429 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Dag name: analyze table cbo_t1 compute sta...c_boolean (Stage-0) 2018-01-05T11:28:58,429 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: DagInfo: {"context":"Hive","description":"\nanalyze table cbo_t1 compute statistics for columns key, value, c_int, c_float, c_boolean"} 2018-01-05T11:28:58,429 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89 with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:28:58,429 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,441 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,441 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:28:58,442 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,442 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 2 size: 2.93KB 2018-01-05T11:28:58,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,468 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:28:58,468 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,468 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:28:58,469 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,469 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 4.41KB 2018-01-05T11:28:58,474 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_292_1911154602830553799-1/-mr-10001 2018-01-05T11:28:58,485 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_292_1911154602830553799-1/-mr-10001 2018-01-05T11:28:58,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1/dt=2014 2018-01-05T11:28:58,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,487 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,487 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,487 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=analyze table cbo_t1 compute sta...c_boolean (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89 } 2018-01-05T11:28:58,487 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:58,487 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:58,489 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: analyze table cbo_t1 compute sta...c_boolean (Stage-0) 2018-01-05T11:28:58,489 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:28:58,489 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_23, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_23.dot 2018-01-05T11:28:58,490 INFO [IPC Server handler 1 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_23 2018-01-05T11:28:58,490 WARN [IPC Server handler 1 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:58,490 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Running DAG: analyze table cbo_t1 compute sta...c_boolean (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89 } 2018-01-05T11:28:58,513 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_23][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_23, submitTime=1515180538489, queueName=null 2018-01-05T11:28:58,514 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_23_00 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_23_00 [Map 1], Default Resources= 2018-01-05T11:28:58,514 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_23_00 [Map 1] 2018-01-05T11:28:58,514 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:28:58,514 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_23_00 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:58,514 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_23_01 [Reducer 2]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_23_01 [Reducer 2], Default Resources= 2018-01-05T11:28:58,514 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_23_01 [Reducer 2] 2018-01-05T11:28:58,514 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 2 2018-01-05T11:28:58,514 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_23_01 [Reducer 2] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:58,514 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:28:58,514 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_23][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_23, initTime=1515180538514 2018-01-05T11:28:58,514 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: dag_1515180518813_0001_23 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:28:58,515 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:28:58,515 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_23][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_23, startTime=1515180538515 2018-01-05T11:28:58,515 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_23 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:28:58,515 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {cbo_t1={InputName=cbo_t1}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:28:58,515 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: cbo_t1, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:28:58,515 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_23_00 [Map 1] 2018-01-05T11:28:58,515 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_23, dagName=analyze table cbo_t1 compute sta...c_boolean (Stage-0) 2018-01-05T11:28:58,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,515 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,519 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_23_00 [Map 1] 2018-01-05T11:28:58,519 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_23_00 [Map 1] 2018-01-05T11:28:58,519 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:58,519 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_23_00 [Map 1] 2018-01-05T11:28:58,521 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:28:58,522 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@a8162ae 2018-01-05T11:28:58,522 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_288_8305321905597716466-1/djaiswal/_tez_scratch_dir/1085f680-5471-4610-9431-9ae847deae9c/map.xml 2018-01-05T11:28:58,522 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_288_8305321905597716466-1/djaiswal/_tez_scratch_dir/1085f680-5471-4610-9431-9ae847deae9c/map.xml 2018-01-05T11:28:58,522 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_288_8305321905597716466-1/djaiswal/_tez_scratch_dir/1085f680-5471-4610-9431-9ae847deae9c/map.xml 2018-01-05T11:28:58,522 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:58,522 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:58,522 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,523 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:58,523 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 4.41KB 2018-01-05T11:28:58,523 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_23_00 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:28:58,523 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: cbo_t1 on vertex vertex_1515180518813_0001_23_00 [Map 1] 2018-01-05T11:28:58,523 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to ShuffleVertexManager for vertex_1515180518813_0001_23_01 [Reducer 2] 2018-01-05T11:28:58,523 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:28:58,523 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:58,523 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:28:58,523 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:58,523 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_288_8305321905597716466-1/djaiswal/_tez_scratch_dir/1085f680-5471-4610-9431-9ae847deae9c/map.xml 2018-01-05T11:28:58,523 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:58,524 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias cbo_t1 2018-01-05T11:28:58,524 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1/dt=2014 2018-01-05T11:28:58,524 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1/dt=2014 2018-01-05T11:28:58,524 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 0,1,2,3,4 2018-01-05T11:28:58,524 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = key,value,c_int,c_float,c_boolean 2018-01-05T11:28:58,524 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1/dt=2014 2018-01-05T11:28:58,524 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:58,524 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:58,524 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:58,524 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:58,529 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.05 maxFrac: 0.75 auto: false desiredTaskIput: 104857600 2018-01-05T11:28:58,529 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:28:58,529 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 4 tasks for vertex: vertex_1515180518813_0001_23_01 [Reducer 2] 2018-01-05T11:28:58,529 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_23_01 [Reducer 2] 2018-01-05T11:28:58,529 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_23][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_23_01, reconfigureDoneTime=1515180538529, numTasks=4, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:28:58,529 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_23_01 [Reducer 2], numAdditionalOutputs=1 2018-01-05T11:28:58,529 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_23][Event:VERTEX_INITIALIZED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_23_01, initRequestedTime=1515180538523, initedTime=1515180538529, numTasks=4, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:58,529 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_23_01 [Reducer 2] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:28:58,544 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Time taken to get FileStatuses: 19 2018-01-05T11:28:58,544 INFO [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:28:58,544 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 19 2018-01-05T11:28:58,544 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 1 2018-01-05T11:28:58,544 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:58,544 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 1. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:28:58,544 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_288_8305321905597716466-1/djaiswal/_tez_scratch_dir/1085f680-5471-4610-9431-9ae847deae9c/map.xml 2018-01-05T11:28:58,544 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_288_8305321905597716466-1/djaiswal/_tez_scratch_dir/1085f680-5471-4610-9431-9ae847deae9c/map.xml 2018-01-05T11:28:58,544 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_288_8305321905597716466-1/djaiswal/_tez_scratch_dir/1085f680-5471-4610-9431-9ae847deae9c/map.xml 2018-01-05T11:28:58,544 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:58,544 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:58,544 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,545 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:58,545 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 4.41KB 2018-01-05T11:28:58,545 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1/dt=2014/cbo_t1.txt to src new group? true 2018-01-05T11:28:58,545 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:28:58,545 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:28:58,545 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:28:58,546 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 5 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 282 Original splits: 1 2018-01-05T11:28:58,546 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Using original number of splits: 1 desired splits: 1 2018-01-05T11:28:58,546 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 1 grouped split count is 1, for bucket: 1 2018-01-05T11:28:58,546 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:28:58,546 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: cbo_t1 on vertex vertex_1515180518813_0001_23_00 [Map 1] 2018-01-05T11:28:58,547 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_23_00 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:58,547 INFO [App Shared Pool - #0] impl.VertexImpl: Got updated RootInputsSpecs: {cbo_t1=forAllWorkUnits=true, update=[1]} 2018-01-05T11:28:58,547 INFO [App Shared Pool - #0] impl.VertexImpl: Vertex vertex_1515180518813_0001_23_00 [Map 1] parallelism set to 1 2018-01-05T11:28:58,547 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_23_00 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:58,547 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_23][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_23_00, initRequestedTime=1515180538515, initedTime=1515180538547, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:58,547 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_23_00 [Map 1] with distanceFromRoot: 0 2018-01-05T11:28:58,547 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_23_00 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:28:58,547 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_23][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_23_00, startRequestedTime=1515180538523, startedTime=1515180538547 2018-01-05T11:28:58,547 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_23][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_23_00, reconfigureDoneTime=1515180538547, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:28:58,547 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_23_00 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:58,547 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_23_00 for vertex: vertex_1515180518813_0001_23_01 [Reducer 2] numStartedSources: 1 numSources: 1 2018-01-05T11:28:58,547 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_23_01 [Reducer 2] with distanceFromRoot: 1 2018-01-05T11:28:58,547 INFO [App Shared Pool - #0] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:28:58,547 INFO [App Shared Pool - #0] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:28:58,547 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_23][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_23_01, startRequestedTime=1515180538547, startedTime=1515180538547 2018-01-05T11:28:58,547 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_23_00 [Map 1] 2018-01-05T11:28:58,547 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_23_01 [Reducer 2] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:58,548 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_23][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_23_00_000000, scheduledTime=1515180538548, launchTime=1515180538548 2018-01-05T11:28:58,548 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: OnVertexStarted vertex: Reducer 2 with 0 source tasks and 4 pending tasks 2018-01-05T11:28:58,548 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_23_00 [Map 1] 2018-01-05T11:28:58,549 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Map 1 in vertex: Reducer 2 numBipartiteSourceTasks: 1 2018-01-05T11:28:58,549 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_23_00_000000_0, priority=5, capability=, hosts=null 2018-01-05T11:28:58,549 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=5,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:58,549 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=5,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:58,549 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:58,549 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_23_00_000000_0 on hosts=null 2018-01-05T11:28:58,549 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_23_00_000000_0 on hosts=null 2018-01-05T11:28:58,549 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_23_00_000000_0 2018-01-05T11:28:58,549 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:58,549 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #44, task=TaskInfo{task=attempt_1515180518813_0001_23_00_000000_0, priority=5, startTime=0, containerId=null, uniqueId=43, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000044 2018-01-05T11:28:58,549 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_23_00_000000_0; false 2018-01-05T11:28:58,549 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_23_00_000000_0, priority=5, startTime=311559058, containerId=container_111101111_0001_00_000044assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=43, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:58,549 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000044 for node: 10.22.8.182:49818 2018-01-05T11:28:58,551 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 23, name=analyze table cbo_t1 compute sta...c_boolean (Stage-0), queryId=djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89 2018-01-05T11:28:58,551 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000044, attempt_1515180518813_0001_23_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:58,551 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000044 for node: 10.22.8.182:49818 2018-01-05T11:28:58,552 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000044 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:58,552 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000044, launchTime=1515180538552 2018-01-05T11:28:58,552 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_23_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000044, dagName=analyze table cbo_t1 compute sta...c_boolean (Stage-0), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{cbo_t1,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{Reducer 2,org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput,4}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180538515, firstAttemptStartTime=1515180538549, currentAttemptStartTime=1515180538551} 2018-01-05T11:28:58,552 INFO [IPC Server handler 2 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000044, QueryId=djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89-23, DagId=23, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180538552 2018-01-05T11:28:58,552 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_23_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000044] on NM: [10.22.8.182:49818] 2018-01-05T11:28:58,552 DEBUG [IPC Server handler 2 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=23} with the ShuffleHandler 2018-01-05T11:28:58,552 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:58,553 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:58,553 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_23][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_23_00_000000_0, startTime=1515180538552, containerId=container_111101111_0001_00_000044, nodeId=10.22.8.182:49818 2018-01-05T11:28:58,565 INFO [IPC Server handler 2 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:28:58,565 DEBUG [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/23] 2018-01-05T11:28:58,566 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:58,566 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_23_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:58,566 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_23_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180538549, dagStartTime=1515180538515, withinDagPriority=5, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:28:58,566 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_23_00_000000_0 : ACCEPTED 2018-01-05T11:28:58,566 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_23_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:58,566 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_23_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180538549, dagStartTime=1515180538515, withinDagPriority=5, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:28:58,566 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_23_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:58,566 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_23_00_000000_0 to pre-emption queue 2018-01-05T11:28:58,566 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_23_00_000000_0: true 2018-01-05T11:28:58,566 INFO [TaskCommunicator # 9] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_23_00_000000_0 2018-01-05T11:28:58,566 INFO [TaskCommunicator # 9] GuaranteedTasks: Received response for attempt_1515180518813_0001_23_00_000000_0, true 2018-01-05T11:28:58,566 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:419057395 2018-01-05T11:28:58,567 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : analyze table cbo_t1 compute sta...c_boolean (Stage-0), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_23_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=cbo_t1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=Reducer 2, physicalEdgeCount=4, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ] 2018-01-05T11:28:58,567 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1033895936, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:58,567 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_23_00_000000_0 vertexName: Map 1 2018-01-05T11:28:58,567 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000044, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_23_00_000000_0, eventCount=1 } 2018-01-05T11:28:58,567 DEBUG [TezTR-518813_1_23_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_23 queryId: djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89 instanceType: TaskRunner2Callable 2018-01-05T11:28:58,568 INFO [TezTR-518813_1_23_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_23_00_000000_0 2018-01-05T11:28:58,568 DEBUG [TezTR-518813_1_23_0_0_0] log.PerfLogger: 2018-01-05T11:28:58,568 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:58,569 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:58,570 INFO [IPC Server handler 10 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_23_00_000000_0 sent events: (0-1). 2018-01-05T11:28:58,570 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:28:58,570 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_23_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:28:58,571 DEBUG [TezTR-518813_1_23_0_0_0] log.PerfLogger: 2018-01-05T11:28:58,571 INFO [TezTR-518813_1_23_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:58,571 INFO [TezTR-518813_1_23_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:58,571 INFO [TezTR-518813_1_23_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:58,571 INFO [I/O Setup 0 Initialize: {cbo_t1}] input.MRInput: cbo_t1 using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:28:58,571 INFO [I/O Setup 0 Initialize: {cbo_t1}] input.MRInputLegacy: cbo_t1 MRInputLegacy deferring initialization 2018-01-05T11:28:58,571 INFO [TezTR-518813_1_23_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:58,571 INFO [TezTR-518813_1_23_0_0_0] resources.MemoryDistributor: InitialRequests=[cbo_t1:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [Reducer 2:OUTPUT:10485760:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput] 2018-01-05T11:28:58,571 INFO [TezTR-518813_1_23_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:58,571 INFO [TezTR-518813_1_23_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:58,571 INFO [TezTR-518813_1_23_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 10485760, TotalRequestedScaled: 9679163.076923078, TotalJVMHeap: 1033895936, TotalAvailable: 692710277, TotalRequested/TotalJVMHeap:0.01 2018-01-05T11:28:58,571 INFO [TezTR-518813_1_23_0_0_0] resources.MemoryDistributor: Allocations=[cbo_t1:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:10485760:10485760] 2018-01-05T11:28:58,571 INFO [TezTR-518813_1_23_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:58,571 INFO [TezTR-518813_1_23_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: cbo_t1 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:28:58,571 INFO [TezTR-518813_1_23_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:28:58,571 INFO [TezTR-518813_1_23_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:28:58,571 INFO [I/O Setup 1 Start: {cbo_t1}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: cbo_t1 2018-01-05T11:28:58,571 INFO [TezTR-518813_1_23_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:58,571 INFO [TezTR-518813_1_23_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_23_00_000000_0 2018-01-05T11:28:58,571 DEBUG [TezTR-518813_1_23_0_0_0] log.PerfLogger: 2018-01-05T11:28:58,571 DEBUG [TezTR-518813_1_23_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_23_00_000000_0_10130 2018-01-05T11:28:58,571 DEBUG [TezTR-518813_1_23_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:58,571 DEBUG [TezTR-518813_1_23_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_23_00_000000_0 2018-01-05T11:28:58,571 INFO [TezTR-518813_1_23_0_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89 2018-01-05T11:28:58,571 DEBUG [TezTR-518813_1_23_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:58,571 DEBUG [TezTR-518813_1_23_0_0_0] log.PerfLogger: 2018-01-05T11:28:58,572 DEBUG [TezTR-518813_1_23_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:58,572 DEBUG [TezTR-518813_1_23_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:58,572 DEBUG [TezTR-518813_1_23_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89_Map 1__MAP_PLAN__ 2018-01-05T11:28:58,572 INFO [TezTR-518813_1_23_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_288_8305321905597716466-1/djaiswal/_tez_scratch_dir/1085f680-5471-4610-9431-9ae847deae9c/map.xml 2018-01-05T11:28:58,572 DEBUG [TezTR-518813_1_23_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_288_8305321905597716466-1/djaiswal/_tez_scratch_dir/1085f680-5471-4610-9431-9ae847deae9c/map.xml 2018-01-05T11:28:58,572 DEBUG [TezTR-518813_1_23_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_288_8305321905597716466-1/djaiswal/_tez_scratch_dir/1085f680-5471-4610-9431-9ae847deae9c/map.xml 2018-01-05T11:28:58,572 DEBUG [TezTR-518813_1_23_0_0_0] log.PerfLogger: 2018-01-05T11:28:58,572 INFO [TezTR-518813_1_23_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:58,572 DEBUG [TezTR-518813_1_23_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,573 DEBUG [TezTR-518813_1_23_0_0_0] log.PerfLogger: 2018-01-05T11:28:58,573 INFO [TezTR-518813_1_23_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 4.41KB 2018-01-05T11:28:58,573 DEBUG [TezTR-518813_1_23_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:58,573 INFO [TezTR-518813_1_23_0_0_0] tez.MapRecordProcessor: The input names are: [cbo_t1] 2018-01-05T11:28:58,573 INFO [TezTR-518813_1_23_0_0_0] input.MRInput: Initialized MRInput: cbo_t1 2018-01-05T11:28:58,573 DEBUG [TezTR-518813_1_23_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 1 2018-01-05T11:28:58,573 INFO [TezTR-518813_1_23_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_288_8305321905597716466-1/djaiswal/_tez_scratch_dir/1085f680-5471-4610-9431-9ae847deae9c/map.xml 2018-01-05T11:28:58,573 DEBUG [TezTR-518813_1_23_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:58,573 DEBUG [TezTR-518813_1_23_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1/dt=2014/cbo_t1.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.cbo_t1, field.delim=,, columns.types=string:string:int:float:boolean, serialization.ddl=struct cbo_t1 { string key, string value, i32 c_int, float c_float, bool c_boolean}, serialization.format=,, columns=key,value,c_int,c_float,c_boolean, columns.comments=, partition_columns.types=string, partition_columns=dt, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180536}, jobProperties=null], partSpec={dt=2014}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string:int:float:boolean, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1/dt=2014, columns=key,value,c_int,c_float,c_boolean, partition_columns.types=string, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=,, numRows=20, numFiles=1, serialization.ddl=struct cbo_t1 { string key, string value, i32 c_int, float c_float, bool c_boolean}, transient_lastDdlTime=1515180538, rawDataSize=262, field.delim=,, columns.comments=, totalSize=282, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=dt, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.cbo_t1}, baseFileName=dt=2014, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1/dt=2014=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.cbo_t1, field.delim=,, columns.types=string:string:int:float:boolean, serialization.ddl=struct cbo_t1 { string key, string value, i32 c_int, float c_float, bool c_boolean}, serialization.format=,, columns=key,value,c_int,c_float,c_boolean, columns.comments=, partition_columns.types=string, partition_columns=dt, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180536}, jobProperties=null], partSpec={dt=2014}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string:int:float:boolean, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1/dt=2014, columns=key,value,c_int,c_float,c_boolean, partition_columns.types=string, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=,, numRows=20, numFiles=1, serialization.ddl=struct cbo_t1 { string key, string value, i32 c_int, float c_float, bool c_boolean}, transient_lastDdlTime=1515180538, rawDataSize=262, field.delim=,, columns.comments=, totalSize=282, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=dt, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.cbo_t1}, baseFileName=dt=2014, vectorPartitionDesc=null]} 2018-01-05T11:28:58,574 DEBUG [TezTR-518813_1_23_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:58,574 INFO [TezTR-518813_1_23_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:58,574 DEBUG [TezTR-518813_1_23_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:58,575 DEBUG [TezTR-518813_1_23_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:58,575 INFO [TezTR-518813_1_23_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_288_8305321905597716466-1/djaiswal/_tez_scratch_dir/1085f680-5471-4610-9431-9ae847deae9c/map.xml 2018-01-05T11:28:58,575 DEBUG [TezTR-518813_1_23_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:58,575 INFO [TezTR-518813_1_23_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_23_00_000000_0 2018-01-05T11:28:58,575 INFO [TezTR-518813_1_23_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value, c_int, c_float, c_boolean] / schema.evolution.columns.types [string, string, int, float, boolean] (isAcidRead false) 2018-01-05T11:28:58,575 DEBUG [TezTR-518813_1_23_0_0_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:58,575 DEBUG [TezTR-518813_1_23_0_0_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:58,575 DEBUG [TezTR-518813_1_23_0_0_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:58,575 DEBUG [TezTR-518813_1_23_0_0_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:58,575 WARN [TezTR-518813_1_23_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1/dt=2014/cbo_t1.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:28:58,576 DEBUG [TezTR-518813_1_23_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:28:58,576 WARN [TezTR-518813_1_23_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1/dt=2014/cbo_t1.txt:0+282 2018-01-05T11:28:58,576 DEBUG [TezTR-518813_1_23_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1/dt=2014/cbo_t1.txt 2018-01-05T11:28:58,576 INFO [TezTR-518813_1_23_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1/dt=2014/cbo_t1.txt:0+282], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=282} 2018-01-05T11:28:58,576 INFO [TezTR-518813_1_23_0_0_0] input.MRInput: cbo_t1 initialized RecordReader from event 2018-01-05T11:28:58,576 DEBUG [TezTR-518813_1_23_0_0_0] tez.MapRecordProcessor: Starting Output: Reducer 2 2018-01-05T11:28:58,577 INFO [TezTR-518813_1_23_0_0_0] impl.ExternalSorter: Reducer 2 using: memoryMb=10, keySerializerClass=class org.apache.hadoop.hive.ql.io.HiveKey, valueSerializerClass=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization$TezBytesWritableSerializer@3943603c, comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator@6bcd1395, partitioner=org.apache.tez.mapreduce.partition.MRPartitioner, serialization=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.hadoop.io.serializer.WritableSerialization, org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization, org.apache.hadoop.io.serializer.avro.AvroReflectSerialization, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:28:58,577 INFO [TezTR-518813_1_23_0_0_0] partition.MRPartitioner: Using oldApi, MRpartitionerClass=org.apache.hadoop.hive.ql.io.DefaultHivePartitioner 2018-01-05T11:28:58,577 INFO [TezTR-518813_1_23_0_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 2: , UsingHashComparator=true 2018-01-05T11:28:58,578 INFO [TezTR-518813_1_23_0_0_0] impl.PipelinedSorter: Newly allocated block size=10485760, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=10485760, total=10485760 2018-01-05T11:28:58,578 INFO [TezTR-518813_1_23_0_0_0] impl.PipelinedSorter: Pre allocating rest of memory buffers upfront 2018-01-05T11:28:58,578 INFO [TezTR-518813_1_23_0_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 2: , UsingHashComparator=true#blocks=1, maxMemUsage=10485760, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=10485760, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=10 2018-01-05T11:28:58,578 INFO [TezTR-518813_1_23_0_0_0] impl.PipelinedSorter: Reducer 2: reserved.remaining()=10485760, reserved.metasize=5242880 2018-01-05T11:28:58,578 INFO [TezTR-518813_1_23_0_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:28:58,578 INFO [TezTR-518813_1_23_0_0_0] exec.MapOperator: Initializing operator MAP[0] 2018-01-05T11:28:58,578 DEBUG [TezTR-518813_1_23_0_0_0] exec.MapOperator: Initialization Done 0 MAP 2018-01-05T11:28:58,578 DEBUG [TezTR-518813_1_23_0_0_0] exec.MapOperator: Operator 0 MAP initialized 2018-01-05T11:28:58,578 DEBUG [TezTR-518813_1_23_0_0_0] exec.MapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:28:58,579 DEBUG [TezTR-518813_1_23_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,579 DEBUG [TezTR-518813_1_23_0_0_0] exec.MapOperator: Adding alias cbo_t1 to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1/dt=2014 2018-01-05T11:28:58,579 DEBUG [TezTR-518813_1_23_0_0_0] exec.MapOperator: dump TS[0] struct 2018-01-05T11:28:58,579 INFO [TezTR-518813_1_23_0_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:28:58,579 INFO [TezTR-518813_1_23_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:58,579 DEBUG [TezTR-518813_1_23_0_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:28:58,579 INFO [TezTR-518813_1_23_0_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:28:58,579 DEBUG [TezTR-518813_1_23_0_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:28:58,579 DEBUG [TezTR-518813_1_23_0_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:28:58,579 DEBUG [TezTR-518813_1_23_0_0_0] exec.TableScanOperator: Initializing children of 0 TS 2018-01-05T11:28:58,579 DEBUG [TezTR-518813_1_23_0_0_0] exec.SelectOperator: Initializing child 1 SEL 2018-01-05T11:28:58,579 INFO [TezTR-518813_1_23_0_0_0] exec.SelectOperator: Initializing operator SEL[1] 2018-01-05T11:28:58,579 INFO [TezTR-518813_1_23_0_0_0] exec.SelectOperator: SELECT struct 2018-01-05T11:28:58,579 DEBUG [TezTR-518813_1_23_0_0_0] exec.SelectOperator: Initialization Done 1 SEL 2018-01-05T11:28:58,579 DEBUG [TezTR-518813_1_23_0_0_0] exec.SelectOperator: Operator 1 SEL initialized 2018-01-05T11:28:58,579 DEBUG [TezTR-518813_1_23_0_0_0] exec.SelectOperator: Initializing children of 1 SEL 2018-01-05T11:28:58,579 DEBUG [TezTR-518813_1_23_0_0_0] exec.GroupByOperator: Initializing child 2 GBY 2018-01-05T11:28:58,579 INFO [TezTR-518813_1_23_0_0_0] exec.GroupByOperator: Initializing operator GBY[2] 2018-01-05T11:28:58,579 DEBUG [TezTR-518813_1_23_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,580 DEBUG [TezTR-518813_1_23_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,580 DEBUG [TezTR-518813_1_23_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,580 DEBUG [TezTR-518813_1_23_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,580 DEBUG [TezTR-518813_1_23_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,580 INFO [TezTR-518813_1_23_0_0_0] exec.GroupByOperator: Max hash table memory: 93860656 bytes 2018-01-05T11:28:58,580 INFO [TezTR-518813_1_23_0_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:58,580 DEBUG [TezTR-518813_1_23_0_0_0] exec.GroupByOperator: Initialization Done 2 GBY 2018-01-05T11:28:58,580 DEBUG [TezTR-518813_1_23_0_0_0] exec.GroupByOperator: Operator 2 GBY initialized 2018-01-05T11:28:58,580 DEBUG [TezTR-518813_1_23_0_0_0] exec.GroupByOperator: Initializing children of 2 GBY 2018-01-05T11:28:58,580 DEBUG [TezTR-518813_1_23_0_0_0] exec.ReduceSinkOperator: Initializing child 3 RS 2018-01-05T11:28:58,580 INFO [TezTR-518813_1_23_0_0_0] exec.ReduceSinkOperator: Initializing operator RS[3] 2018-01-05T11:28:58,580 DEBUG [TezTR-518813_1_23_0_0_0] exec.ReduceSinkOperator: keys size is 1 2018-01-05T11:28:58,580 DEBUG [TezTR-518813_1_23_0_0_0] exec.ReduceSinkOperator: Key exprNodeDesc _col0 2018-01-05T11:28:58,580 INFO [TezTR-518813_1_23_0_0_0] exec.ReduceSinkOperator: Using tag = -1 2018-01-05T11:28:58,580 DEBUG [TezTR-518813_1_23_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,580 DEBUG [TezTR-518813_1_23_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,580 DEBUG [TezTR-518813_1_23_0_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1, _col2, _col3, _col4] columnTypes=[struct, struct, struct, struct, struct] 2018-01-05T11:28:58,580 DEBUG [TezTR-518813_1_23_0_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS 2018-01-05T11:28:58,580 DEBUG [TezTR-518813_1_23_0_0_0] exec.ReduceSinkOperator: Operator 3 RS initialized 2018-01-05T11:28:58,580 DEBUG [TezTR-518813_1_23_0_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS done is reset. 2018-01-05T11:28:58,580 DEBUG [TezTR-518813_1_23_0_0_0] exec.GroupByOperator: Initialization Done 2 GBY done is reset. 2018-01-05T11:28:58,580 DEBUG [TezTR-518813_1_23_0_0_0] exec.SelectOperator: Initialization Done 1 SEL done is reset. 2018-01-05T11:28:58,580 DEBUG [TezTR-518813_1_23_0_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:28:58,580 INFO [TezTR-518813_1_23_0_0_0] exec.OperatorUtils: Setting output collector: RS[3] --> Reducer 2 2018-01-05T11:28:58,580 DEBUG [TezTR-518813_1_23_0_0_0] log.PerfLogger: 2018-01-05T11:28:58,580 INFO [TezTR-518813_1_23_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_288_8305321905597716466-1/djaiswal/_tez_scratch_dir/1085f680-5471-4610-9431-9ae847deae9c/map.xml 2018-01-05T11:28:58,580 DEBUG [TezTR-518813_1_23_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:58,580 DEBUG [TezTR-518813_1_23_0_0_0] exec.MapOperator: Processing alias(es) cbo_t1 for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1/dt=2014/cbo_t1.txt 2018-01-05T11:28:58,580 INFO [TezTR-518813_1_23_0_0_0] exec.TableScanOperator: Stats Gathering found a new partition spec = dt=2014 2018-01-05T11:28:58,582 INFO [TezTR-518813_1_23_0_0_0] exec.MapOperator: MAP[0]: records read - 1 2018-01-05T11:28:58,582 INFO [TezTR-518813_1_23_0_0_0] exec.MapOperator: MAP[0]: records read - 10 2018-01-05T11:28:58,582 DEBUG [TezTR-518813_1_23_0_0_0] log.PerfLogger: 2018-01-05T11:28:58,582 DEBUG [TezTR-518813_1_23_0_0_0] mr.ObjectCache: djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:28:58,582 DEBUG [TezTR-518813_1_23_0_0_0] exec.MapOperator: close called for operator MAP[0] 2018-01-05T11:28:58,582 INFO [TezTR-518813_1_23_0_0_0] exec.MapOperator: Closing operator MAP[0] 2018-01-05T11:28:58,582 INFO [TezTR-518813_1_23_0_0_0] exec.MapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:20, 2018-01-05T11:28:58,582 DEBUG [TezTR-518813_1_23_0_0_0] exec.MapOperator: Closing child = TS[0] 2018-01-05T11:28:58,582 DEBUG [TezTR-518813_1_23_0_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:28:58,582 DEBUG [TezTR-518813_1_23_0_0_0] exec.TableScanOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:58,582 INFO [TezTR-518813_1_23_0_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:28:58,582 DEBUG [TezTR-518813_1_23_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,582 DEBUG [TezTR-518813_1_23_0_0_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_292_1911154602830553799-1/-mr-10001 2018-01-05T11:28:58,583 INFO [TezTR-518813_1_23_0_0_0] exec.TableScanOperator: publishing : default.cbo_t1/dt=2014/ : {numRows=20, rawDataSize=262} 2018-01-05T11:28:58,606 DEBUG [TezTR-518813_1_23_0_0_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_292_1911154602830553799-1/-mr-10001/tmpstats-0_0 2018-01-05T11:28:58,606 DEBUG [TezTR-518813_1_23_0_0_0] fs.FSStatsPublisher: Writing stats in it : {default.cbo_t1/dt=2014/={numRows=20, rawDataSize=262}} 2018-01-05T11:28:58,606 DEBUG [TezTR-518813_1_23_0_0_0] exec.TableScanOperator: Closing child = SEL[1] 2018-01-05T11:28:58,606 DEBUG [TezTR-518813_1_23_0_0_0] exec.SelectOperator: close called for operator SEL[1] 2018-01-05T11:28:58,606 DEBUG [TezTR-518813_1_23_0_0_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:58,606 INFO [TezTR-518813_1_23_0_0_0] exec.SelectOperator: Closing operator SEL[1] 2018-01-05T11:28:58,606 DEBUG [TezTR-518813_1_23_0_0_0] exec.SelectOperator: Closing child = GBY[2] 2018-01-05T11:28:58,606 DEBUG [TezTR-518813_1_23_0_0_0] exec.GroupByOperator: close called for operator GBY[2] 2018-01-05T11:28:58,606 DEBUG [TezTR-518813_1_23_0_0_0] exec.GroupByOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:58,606 INFO [TezTR-518813_1_23_0_0_0] exec.GroupByOperator: Closing operator GBY[2] 2018-01-05T11:28:58,606 INFO [TezTR-518813_1_23_0_0_0] exec.GroupByOperator: Begin Hash Table flush: size = 1 2018-01-05T11:28:58,606 INFO [TezTR-518813_1_23_0_0_0] exec.ReduceSinkOperator: keys are [_col0] num distributions: 1 2018-01-05T11:28:58,606 DEBUG [TezTR-518813_1_23_0_0_0] exec.GroupByOperator: Closing child = RS[3] 2018-01-05T11:28:58,606 DEBUG [TezTR-518813_1_23_0_0_0] exec.ReduceSinkOperator: close called for operator RS[3] 2018-01-05T11:28:58,606 DEBUG [TezTR-518813_1_23_0_0_0] exec.ReduceSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:58,606 INFO [TezTR-518813_1_23_0_0_0] exec.ReduceSinkOperator: Closing operator RS[3] 2018-01-05T11:28:58,606 INFO [TezTR-518813_1_23_0_0_0] exec.ReduceSinkOperator: RECORDS_OUT_INTERMEDIATE_Map_1:1, 2018-01-05T11:28:58,606 DEBUG [TezTR-518813_1_23_0_0_0] exec.ReduceSinkOperator: 3 Close done 2018-01-05T11:28:58,606 DEBUG [TezTR-518813_1_23_0_0_0] exec.GroupByOperator: 2 Close done 2018-01-05T11:28:58,606 DEBUG [TezTR-518813_1_23_0_0_0] exec.SelectOperator: 1 Close done 2018-01-05T11:28:58,606 DEBUG [TezTR-518813_1_23_0_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:28:58,606 DEBUG [TezTR-518813_1_23_0_0_0] exec.MapOperator: 0 Close done 2018-01-05T11:28:58,606 INFO [TezTR-518813_1_23_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_23_00_000000_0 2018-01-05T11:28:58,606 INFO [TezTR-518813_1_23_0_0_0] impl.PipelinedSorter: Reducer 2: Starting flush of map output 2018-01-05T11:28:58,606 INFO [TezTR-518813_1_23_0_0_0] impl.PipelinedSorter: Reducer 2: Span0.length = 1, perItem = 173 2018-01-05T11:28:58,606 INFO [TezTR-518813_1_23_0_0_0] impl.PipelinedSorter: Reducer 2: done sorting span=0, length=1, time=0 2018-01-05T11:28:58,606 DEBUG [TezTR-518813_1_23_0_0_0] util.Progress: Illegal progress value found, progress is Float.NaN. Progress will be changed to 0 2018-01-05T11:28:58,607 INFO [TezTR-518813_1_23_0_0_0] impl.PipelinedSorter: Reducer 2: Heap = SpanIterator<0:0> (span=Span[5242880,173]), 2018-01-05T11:28:58,627 INFO [TezTR-518813_1_23_0_0_0] impl.PipelinedSorter: Reducer 2: Spilling to /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/23/output/attempt_1515180518813_0001_23_00_000000_0_10132_0/file.out 2018-01-05T11:28:58,640 INFO [TezTR-518813_1_23_0_0_0] shuffle.ShuffleUtils: EmptyPartition bitsetSize=3, numOutputs=4, emptyPartitions=3, compressedSize=3 2018-01-05T11:28:58,640 DEBUG [TezTR-518813_1_23_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:58,640 INFO [TezTR-518813_1_23_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:58,640 INFO [TezTR-518813_1_23_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_23_00_000000_0, askedToStop=false 2018-01-05T11:28:58,641 INFO [TezTR-518813_1_23_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_23_00_000000_0, stopRequested=false 2018-01-05T11:28:58,641 INFO [TezTR-518813_1_23_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_23_00_000000_0: Counters: 15 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=1, INPUT_RECORDS_PROCESSED=20, INPUT_SPLIT_LENGTH_BYTES=282, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=173, OUTPUT_BYTES_WITH_OVERHEAD=182, OUTPUT_BYTES_PHYSICAL=186, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=20, RECORDS_OUT_INTERMEDIATE_Map_1=1]] 2018-01-05T11:28:58,641 DEBUG [TezTR-518813_1_23_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_23_0_0_0 thread id: 1555 scheme: file bytesRead: 506 bytesWritten: 381 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:58,641 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_23_00_000000_0, isTaskDone=true 2018-01-05T11:28:58,641 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000044, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_23_00_000000_0, eventCount=4 } 2018-01-05T11:28:58,642 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_23][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_23_00_000000_0, creationTime=1515180538548, allocationTime=1515180538549, startTime=1515180538552, finishTime=1515180538641, timeTaken=89, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:58,642 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:58,642 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_23_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:58,642 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_23_00_000000_0 2018-01-05T11:28:58,642 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_23_00_000000_0 2018-01-05T11:28:58,642 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_23_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:58,642 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_23_00_000000_0; false 2018-01-05T11:28:58,642 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000044=75 2018-01-05T11:28:58,642 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_23_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:58,642 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_23_00_000000_0: true 2018-01-05T11:28:58,642 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_23][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_23_00_000000, startTime=1515180538552, finishTime=1515180538642, timeTaken=90, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_23_00_000000_0, diagnostics=, counters=Counters: 21, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=506, FILE_BYTES_WRITTEN=381, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=1, INPUT_RECORDS_PROCESSED=20, INPUT_SPLIT_LENGTH_BYTES=282, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=173, OUTPUT_BYTES_WITH_OVERHEAD=182, OUTPUT_BYTES_PHYSICAL=186, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=20, RECORDS_OUT_INTERMEDIATE_Map_1=1 2018-01-05T11:28:58,642 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:58,642 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_23_00_000000_0 2018-01-05T11:28:58,642 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:58,642 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000044, QueryId=djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89-23, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:28:58,642 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_23_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:58,642 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_23_00_000000_0 2018-01-05T11:28:58,642 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_23_00 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:58,642 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_23_00 [Map 1] 2018-01-05T11:28:58,642 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_23][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_23_00, initRequestedTime=1515180538515, initedTime=1515180538547, startRequestedTime=1515180538523, startedTime=1515180538547, finishTime=1515180538642, timeTaken=95, status=SUCCEEDED, diagnostics=, counters=Counters: 21, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=506, FILE_BYTES_WRITTEN=381, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=1, INPUT_RECORDS_PROCESSED=20, INPUT_SPLIT_LENGTH_BYTES=282, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=173, OUTPUT_BYTES_WITH_OVERHEAD=182, OUTPUT_BYTES_PHYSICAL=186, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=20, RECORDS_OUT_INTERMEDIATE_Map_1=1, vertexStats=firstTaskStartTime=1515180538552, firstTasksToStart=[ task_1515180518813_0001_23_00_000000 ], lastTaskFinishTime=1515180538641, lastTasksToFinish=[ task_1515180518813_0001_23_00_000000 ], minTaskDuration=89, maxTaskDuration=89, avgTaskDuration=89.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_23_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_23_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:58,642 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_23_00 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:58,642 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_23][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000044, stoppedTime=1515180538642, exitStatus=0 2018-01-05T11:28:58,642 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000044 exited with diagnostics set to Container completed. 2018-01-05T11:28:58,642 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_23_01 [Reducer 2] attempt: attempt_1515180518813_0001_23_00_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:28:58,643 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_23_00 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2 2018-01-05T11:28:58,643 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null 2018-01-05T11:28:58,643 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000044 2018-01-05T11:28:58,643 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: All source tasks completed. Ramping up 4 remaining tasks for vertex: Reducer 2 2018-01-05T11:28:58,643 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Scheduling 4 tasks for vertex: Reducer 2 with totalTasks: 4. 1 source tasks completed out of 1. MinSourceTaskCompletedFraction: 1.0 min: 0.05 max: 0.75 2018-01-05T11:28:58,643 INFO [App Shared Pool - #0] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Map 1, destinationVertexName=Reducer 2. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:28:58,643 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_23_01 [Reducer 2] 2018-01-05T11:28:58,643 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_23][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_23_01_000003, scheduledTime=1515180538643, launchTime=1515180538643 2018-01-05T11:28:58,643 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_23][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_23_01_000000, scheduledTime=1515180538643, launchTime=1515180538643 2018-01-05T11:28:58,643 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_23][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_23_01_000001, scheduledTime=1515180538643, launchTime=1515180538643 2018-01-05T11:28:58,643 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_23][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_23_01_000002, scheduledTime=1515180538643, launchTime=1515180538643 2018-01-05T11:28:58,643 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_23_01 [Reducer 2] 2018-01-05T11:28:58,643 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_23_01_000003_0, priority=14, capability=, hosts=null 2018-01-05T11:28:58,643 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:58,643 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_23_01_000000_0, priority=14, capability=, hosts=null 2018-01-05T11:28:58,643 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:58,643 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:58,643 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_23_01_000003_0 on hosts=null 2018-01-05T11:28:58,643 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_23_01_000003_0 on hosts=null 2018-01-05T11:28:58,643 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_23_01_000003_0 2018-01-05T11:28:58,643 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:58,643 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #45, task=TaskInfo{task=attempt_1515180518813_0001_23_01_000003_0, priority=14, startTime=0, containerId=null, uniqueId=44, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000045 2018-01-05T11:28:58,643 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_23_01_000003_0; false 2018-01-05T11:28:58,643 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_23_01_000003_0, priority=14, startTime=311559153, containerId=container_111101111_0001_00_000045assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=44, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:58,643 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:58,643 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:58,644 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:58,644 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_23_01_000000_0 on hosts=null 2018-01-05T11:28:58,644 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_23_01_000000_0 on hosts=null 2018-01-05T11:28:58,644 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_23_01_000001_0, priority=14, capability=, hosts=null 2018-01-05T11:28:58,644 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_23_01_000000_0 2018-01-05T11:28:58,644 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:58,644 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #46, task=TaskInfo{task=attempt_1515180518813_0001_23_01_000000_0, priority=14, startTime=0, containerId=null, uniqueId=45, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, to container=container_111101111_0001_00_000046 2018-01-05T11:28:58,644 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000045 for node: 10.22.8.182:49818 2018-01-05T11:28:58,644 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_23_01_000000_0; false 2018-01-05T11:28:58,644 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_23_01_000000_0, priority=14, startTime=311559153, containerId=container_111101111_0001_00_000046assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=2, ac=6, commF=false, disabled=false}, uniqueId=45, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:58,644 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000045, attempt_1515180518813_0001_23_01_000003_0 for node: 10.22.8.182:49818 2018-01-05T11:28:58,644 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000045 for node: 10.22.8.182:49818 2018-01-05T11:28:58,644 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:58,644 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:58,644 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_23_01_000002_0, priority=14, capability=, hosts=null 2018-01-05T11:28:58,644 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:58,644 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_23_01_000001_0 on hosts=null 2018-01-05T11:28:58,644 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_23_01_000001_0 on hosts=null 2018-01-05T11:28:58,644 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_23_01_000001_0 2018-01-05T11:28:58,644 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=2, ac=6, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:58,644 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #47, task=TaskInfo{task=attempt_1515180518813_0001_23_01_000001_0, priority=14, startTime=0, containerId=null, uniqueId=46, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=2, ac=6, commF=false, disabled=false}, to container=container_111101111_0001_00_000047 2018-01-05T11:28:58,644 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_23_01_000001_0; false 2018-01-05T11:28:58,644 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_23_01_000001_0, priority=14, startTime=311559153, containerId=container_111101111_0001_00_000047assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=3, ac=5, commF=false, disabled=false}, uniqueId=46, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:58,644 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:58,644 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:58,644 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:58,644 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000045 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:58,644 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_23_01_000002_0 on hosts=null 2018-01-05T11:28:58,644 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000045, launchTime=1515180538644 2018-01-05T11:28:58,644 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_23_01_000002_0 on hosts=null 2018-01-05T11:28:58,644 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=23}, sourceName=Map 1, state=S_SUCCEEDED 2018-01-05T11:28:58,644 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_23_01_000002_0 2018-01-05T11:28:58,644 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=3, ac=5, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:58,644 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #48, task=TaskInfo{task=attempt_1515180518813_0001_23_01_000002_0, priority=14, startTime=0, containerId=null, uniqueId=47, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=3, ac=5, commF=false, disabled=false}, to container=container_111101111_0001_00_000048 2018-01-05T11:28:58,644 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_23_01_000002_0; false 2018-01-05T11:28:58,644 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_23_01_000002_0, priority=14, startTime=311559153, containerId=container_111101111_0001_00_000048assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=4, ac=4, commF=false, disabled=false}, uniqueId=47, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:58,645 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_23_01_000003_0] submitted. Is using containerId: [container_111101111_0001_00_000045] on NM: [10.22.8.182:49818] 2018-01-05T11:28:58,645 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:58,645 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:58,645 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_23_01_000003_0, am_details=10.22.8.182:49824, taskInfo= fragment 3 attempt 0, user=djaiswal, queryId=djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000045, dagName=analyze table cbo_t1 compute sta...c_boolean (Stage-0), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.OrderedGroupedKVInput,1}}, Outputs={{out_Reducer 2,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=5, completedTaskCount=1, dagStartTime=1515180538515, firstAttemptStartTime=1515180538643, currentAttemptStartTime=1515180538644} 2018-01-05T11:28:58,645 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_23][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_23_01_000003_0, startTime=1515180538644, containerId=container_111101111_0001_00_000045, nodeId=10.22.8.182:49818 2018-01-05T11:28:58,645 INFO [IPC Server handler 3 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000045, QueryId=djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89-23, DagId=23, VertexName=Reducer 2, TaskId=3, TaskAttemptId=0, SubmitTime=1515180538645 2018-01-05T11:28:58,645 DEBUG [IPC Server handler 3 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=23} with the ShuffleHandler 2018-01-05T11:28:58,645 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000046 for node: 10.22.8.182:49818 2018-01-05T11:28:58,645 DEBUG [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/23] 2018-01-05T11:28:58,645 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000046, attempt_1515180518813_0001_23_01_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:58,645 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000046 for node: 10.22.8.182:49818 2018-01-05T11:28:58,645 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000047 for node: 10.22.8.182:49818 2018-01-05T11:28:58,645 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000047, attempt_1515180518813_0001_23_01_000001_0 for node: 10.22.8.182:49818 2018-01-05T11:28:58,645 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000047 for node: 10.22.8.182:49818 2018-01-05T11:28:58,645 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:58,645 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_23_01_000003_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:58,646 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_23_01_000003_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180538643, dagStartTime=1515180538515, withinDagPriority=14, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1}] 2018-01-05T11:28:58,646 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_23_01_000003_0 : ACCEPTED 2018-01-05T11:28:58,646 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_23_01_000003_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:58,646 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_23_01_000003_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180538643, dagStartTime=1515180538515, withinDagPriority=14, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1} 2018-01-05T11:28:58,646 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000048 for node: 10.22.8.182:49818 2018-01-05T11:28:58,646 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_23_01_000003_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:58,646 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_23_01_000003_0 to pre-emption queue 2018-01-05T11:28:58,646 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_23_01_000003_0: true 2018-01-05T11:28:58,646 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000048, attempt_1515180518813_0001_23_01_000002_0 for node: 10.22.8.182:49818 2018-01-05T11:28:58,646 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000048 for node: 10.22.8.182:49818 2018-01-05T11:28:58,646 INFO [TaskCommunicator # 1] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_23_01_000003_0 2018-01-05T11:28:58,646 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:419057395 2018-01-05T11:28:58,646 INFO [TaskCommunicator # 1] GuaranteedTasks: Received response for attempt_1515180518813_0001_23_01_000003_0, true 2018-01-05T11:28:58,646 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000046 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:58,646 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000046, launchTime=1515180538646 2018-01-05T11:28:58,646 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : analyze table cbo_t1 compute sta...c_boolean (Stage-0), VertexName: Reducer 2, VertexParallelism: 4, TaskAttemptID:attempt_1515180518813_0001_23_01_000003_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:28:58,646 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1033895936, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:58,646 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_23_01_000003_0 vertexName: Reducer 2 2018-01-05T11:28:58,647 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_23_01_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000046, dagName=analyze table cbo_t1 compute sta...c_boolean (Stage-0), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.OrderedGroupedKVInput,1}}, Outputs={{out_Reducer 2,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=5, completedTaskCount=1, dagStartTime=1515180538515, firstAttemptStartTime=1515180538643, currentAttemptStartTime=1515180538645} 2018-01-05T11:28:58,647 INFO [IPC Server handler 1 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000046, QueryId=djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89-23, DagId=23, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, SubmitTime=1515180538647 2018-01-05T11:28:58,647 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_23_01_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000046] on NM: [10.22.8.182:49818] 2018-01-05T11:28:58,647 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:58,647 DEBUG [IPC Server handler 1 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=23} with the ShuffleHandler 2018-01-05T11:28:58,647 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:58,647 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000045, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_23_01_000003_0, eventCount=1 } 2018-01-05T11:28:58,647 DEBUG [TezTR-518813_1_23_1_3_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_23 queryId: djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89 instanceType: TaskRunner2Callable 2018-01-05T11:28:58,647 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_23][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_23_01_000000_0, startTime=1515180538646, containerId=container_111101111_0001_00_000046, nodeId=10.22.8.182:49818 2018-01-05T11:28:58,647 DEBUG [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/23] 2018-01-05T11:28:58,647 INFO [TezTR-518813_1_23_1_3_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_23_01_000003_0 2018-01-05T11:28:58,647 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000047 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:58,647 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000047, launchTime=1515180538647 2018-01-05T11:28:58,647 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=3, runningFragmentCount=1 2018-01-05T11:28:58,647 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_23_01_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:58,647 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_23_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180538643, dagStartTime=1515180538515, withinDagPriority=14, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1}] 2018-01-05T11:28:58,647 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_23_01_000000_0 : ACCEPTED 2018-01-05T11:28:58,647 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_23_01_000000_0, canFinish=true. Current state: preemptionQueueSize=1, numSlotsAvailable=3, waitQueueSize=1 2018-01-05T11:28:58,647 DEBUG [TezTR-518813_1_23_1_3_0] log.PerfLogger: 2018-01-05T11:28:58,647 INFO [TaskCommunicator # 3] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_23_01_000000_0 2018-01-05T11:28:58,648 INFO [TaskCommunicator # 3] GuaranteedTasks: Received response for attempt_1515180518813_0001_23_01_000000_0, true 2018-01-05T11:28:58,648 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_23_01_000001_0] submitted. Is using containerId: [container_111101111_0001_00_000047] on NM: [10.22.8.182:49818] 2018-01-05T11:28:58,648 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:58,648 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:58,647 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_23_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180538643, dagStartTime=1515180538515, withinDagPriority=14, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1} 2018-01-05T11:28:58,648 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_23][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_23_01_000001_0, startTime=1515180538647, containerId=container_111101111_0001_00_000047, nodeId=10.22.8.182:49818 2018-01-05T11:28:58,648 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000048 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:58,648 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000048, launchTime=1515180538648 2018-01-05T11:28:58,648 INFO [I/O Setup 0 Initialize: {Map 1}] orderedgrouped.MergeManager: Initial Memory required for SHUFFLE_BUFFER=131404920 based on INPUT_BUFFER_FACTOR=0.7, for final merged output=0, using factor: 0.0 2018-01-05T11:28:58,648 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:58,648 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_23_01_000000_0: true 2018-01-05T11:28:58,648 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: taskOwner hashCode:1906564448 2018-01-05T11:28:58,648 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_23_01_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:58,648 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_23_01_000000_0 to pre-emption queue 2018-01-05T11:28:58,649 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_23_01_000002_0] submitted. Is using containerId: [container_111101111_0001_00_000048] on NM: [10.22.8.182:49818] 2018-01-05T11:28:58,649 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:58,649 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:58,649 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_23][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_23_01_000002_0, startTime=1515180538648, containerId=container_111101111_0001_00_000048, nodeId=10.22.8.182:49818 2018-01-05T11:28:58,649 INFO [Task-Executor-3] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : analyze table cbo_t1 compute sta...c_boolean (Stage-0), VertexName: Reducer 2, VertexParallelism: 4, TaskAttemptID:attempt_1515180518813_0001_23_01_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:28:58,649 INFO [Task-Executor-3] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1033895936, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:58,649 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_23_01_000001_0, am_details=10.22.8.182:49824, taskInfo= fragment 1 attempt 0, user=djaiswal, queryId=djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000047, dagName=analyze table cbo_t1 compute sta...c_boolean (Stage-0), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.OrderedGroupedKVInput,1}}, Outputs={{out_Reducer 2,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=5, completedTaskCount=1, dagStartTime=1515180538515, firstAttemptStartTime=1515180538643, currentAttemptStartTime=1515180538645} 2018-01-05T11:28:58,649 INFO [IPC Server handler 0 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000047, QueryId=djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89-23, DagId=23, VertexName=Reducer 2, TaskId=1, TaskAttemptId=0, SubmitTime=1515180538649 2018-01-05T11:28:58,649 DEBUG [IPC Server handler 0 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=23} with the ShuffleHandler 2018-01-05T11:28:58,649 DEBUG [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/23] 2018-01-05T11:28:58,649 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:58,649 INFO [Task-Executor-3] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_23_01_000000_0 vertexName: Reducer 2 2018-01-05T11:28:58,650 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=2, runningFragmentCount=2 2018-01-05T11:28:58,650 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_23_01_000001_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:58,650 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_23_01_000001_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180538643, dagStartTime=1515180538515, withinDagPriority=14, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1}] 2018-01-05T11:28:58,650 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_23_01_000001_0 : ACCEPTED 2018-01-05T11:28:58,650 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_23_01_000001_0, canFinish=true. Current state: preemptionQueueSize=2, numSlotsAvailable=2, waitQueueSize=1 2018-01-05T11:28:58,650 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_23_01_000001_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180538643, dagStartTime=1515180538515, withinDagPriority=14, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1} 2018-01-05T11:28:58,650 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_23_01_000001_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:58,650 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_23_01_000001_0 to pre-emption queue 2018-01-05T11:28:58,650 INFO [TaskCommunicator # 2] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_23_01_000001_0 2018-01-05T11:28:58,650 INFO [TaskCommunicator # 2] GuaranteedTasks: Received response for attempt_1515180518813_0001_23_01_000001_0, true 2018-01-05T11:28:58,650 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_23_01_000001_0: true 2018-01-05T11:28:58,650 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000046, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_23_01_000000_0, eventCount=1 } 2018-01-05T11:28:58,650 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: taskOwner hashCode:132573017 2018-01-05T11:28:58,650 DEBUG [TezTR-518813_1_23_1_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_23 queryId: djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89 instanceType: TaskRunner2Callable 2018-01-05T11:28:58,650 INFO [TezTR-518813_1_23_1_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_23_01_000000_0 2018-01-05T11:28:58,651 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_23_01_000002_0, am_details=10.22.8.182:49824, taskInfo= fragment 2 attempt 0, user=djaiswal, queryId=djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000048, dagName=analyze table cbo_t1 compute sta...c_boolean (Stage-0), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.OrderedGroupedKVInput,1}}, Outputs={{out_Reducer 2,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=5, completedTaskCount=1, dagStartTime=1515180538515, firstAttemptStartTime=1515180538643, currentAttemptStartTime=1515180538646} 2018-01-05T11:28:58,651 INFO [IPC Server handler 2 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000048, QueryId=djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89-23, DagId=23, VertexName=Reducer 2, TaskId=2, TaskAttemptId=0, SubmitTime=1515180538651 2018-01-05T11:28:58,651 INFO [IPC Server handler 26 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_23_01_000003_0 1 events [0,1) total 1 vertex_1515180518813_0001_23_01 [Reducer 2] 2018-01-05T11:28:58,651 DEBUG [IPC Server handler 2 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=23} with the ShuffleHandler 2018-01-05T11:28:58,651 DEBUG [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/23] 2018-01-05T11:28:58,651 DEBUG [TezTR-518813_1_23_1_0_0] log.PerfLogger: 2018-01-05T11:28:58,651 INFO [I/O Setup 0 Initialize: {Map 1}] orderedgrouped.MergeManager: Initial Memory required for SHUFFLE_BUFFER=131404920 based on INPUT_BUFFER_FACTOR=0.7, for final merged output=0, using factor: 0.0 2018-01-05T11:28:58,651 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=1, runningFragmentCount=3 2018-01-05T11:28:58,651 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_23_01_000002_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:58,651 DEBUG [TezTR-518813_1_23_1_3_0] log.PerfLogger: 2018-01-05T11:28:58,651 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_23_01_000002_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180538643, dagStartTime=1515180538515, withinDagPriority=14, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1}] 2018-01-05T11:28:58,651 INFO [TezTR-518813_1_23_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:58,652 INFO [TezTR-518813_1_23_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:58,652 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_23_01_000002_0, canFinish=true. Current state: preemptionQueueSize=3, numSlotsAvailable=1, waitQueueSize=1 2018-01-05T11:28:58,652 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:28:58,652 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_23_01_000002_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180538643, dagStartTime=1515180538515, withinDagPriority=14, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1} 2018-01-05T11:28:58,652 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_23_01_000002_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:58,652 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_23_01_000002_0 to pre-emption queue 2018-01-05T11:28:58,652 INFO [TezTR-518813_1_23_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:58,653 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_23_01_000002_0: true 2018-01-05T11:28:58,653 INFO [I/O Setup 1 Initialize: {out_Reducer 2}] output.MROutput: out_Reducer 2: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:28:58,653 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: taskOwner hashCode:1267016377 2018-01-05T11:28:58,652 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_23_01_000002_0 : ACCEPTED 2018-01-05T11:28:58,652 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_23_01_000003_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:28:58,653 INFO [Task-Executor-1] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : analyze table cbo_t1 compute sta...c_boolean (Stage-0), VertexName: Reducer 2, VertexParallelism: 4, TaskAttemptID:attempt_1515180518813_0001_23_01_000001_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:28:58,653 INFO [Task-Executor-1] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1033895936, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:58,653 INFO [Task-Executor-1] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_23_01_000001_0 vertexName: Reducer 2 2018-01-05T11:28:58,653 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:58,653 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:58,653 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:58,653 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:58,653 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:58,653 INFO [TaskCommunicator # 4] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_23_01_000002_0 2018-01-05T11:28:58,653 INFO [TaskCommunicator # 4] GuaranteedTasks: Received response for attempt_1515180518813_0001_23_01_000002_0, true 2018-01-05T11:28:58,653 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000047, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_23_01_000001_0, eventCount=1 } 2018-01-05T11:28:58,653 DEBUG [TezTR-518813_1_23_1_1_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_23 queryId: djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89 instanceType: TaskRunner2Callable 2018-01-05T11:28:58,653 INFO [TezTR-518813_1_23_1_3_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:58,653 INFO [TezTR-518813_1_23_1_1_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_23_01_000001_0 2018-01-05T11:28:58,653 INFO [TezTR-518813_1_23_1_3_0] resources.MemoryDistributor: InitialRequests=[out_Reducer 2:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.OrderedGroupedKVInput] 2018-01-05T11:28:58,654 INFO [TezTR-518813_1_23_1_3_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:58,654 INFO [TezTR-518813_1_23_1_3_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:58,654 INFO [TezTR-518813_1_23_1_3_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 131404920, TotalRequestedScaled: 1.2129684923076923E8, TotalJVMHeap: 1033895936, TotalAvailable: 692710277, TotalRequested/TotalJVMHeap:0.13 2018-01-05T11:28:58,654 INFO [TezTR-518813_1_23_1_3_0] resources.MemoryDistributor: Allocations=[out_Reducer 2:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [Map 1:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:131404920:131404920] 2018-01-05T11:28:58,654 INFO [TezTR-518813_1_23_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:58,654 INFO [Task-Executor-2] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : analyze table cbo_t1 compute sta...c_boolean (Stage-0), VertexName: Reducer 2, VertexParallelism: 4, TaskAttemptID:attempt_1515180518813_0001_23_01_000002_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:28:58,654 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:58,654 INFO [Task-Executor-2] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1033895936, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:58,654 INFO [Task-Executor-2] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_23_01_000002_0 vertexName: Reducer 2 2018-01-05T11:28:58,654 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000048, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_23_01_000002_0, eventCount=1 } 2018-01-05T11:28:58,655 INFO [I/O Setup 0 Initialize: {Map 1}] orderedgrouped.MergeManager: Initial Memory required for SHUFFLE_BUFFER=131404920 based on INPUT_BUFFER_FACTOR=0.7, for final merged output=0, using factor: 0.0 2018-01-05T11:28:58,655 DEBUG [TezTR-518813_1_23_1_2_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_23 queryId: djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89 instanceType: TaskRunner2Callable 2018-01-05T11:28:58,655 INFO [TezTR-518813_1_23_1_2_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_23_01_000002_0 2018-01-05T11:28:58,656 DEBUG [TezTR-518813_1_23_1_0_0] log.PerfLogger: 2018-01-05T11:28:58,656 INFO [TezTR-518813_1_23_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:58,656 INFO [TezTR-518813_1_23_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:58,656 INFO [TezTR-518813_1_23_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:58,656 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:58,656 DEBUG [TezTR-518813_1_23_1_1_0] log.PerfLogger: 2018-01-05T11:28:58,656 INFO [TezTR-518813_1_23_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Input: Map 1 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:28:58,656 INFO [TezTR-518813_1_23_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:28:58,656 INFO [TezTR-518813_1_23_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:28:58,656 INFO [I/O Setup 0 Initialize: {Map 1}] orderedgrouped.MergeManager: Initial Memory required for SHUFFLE_BUFFER=131404920 based on INPUT_BUFFER_FACTOR=0.7, for final merged output=0, using factor: 0.0 2018-01-05T11:28:58,656 INFO [I/O Setup 1 Initialize: {out_Reducer 2}] output.MROutput: out_Reducer 2: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:28:58,656 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:58,656 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:58,656 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:58,656 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:58,656 DEBUG [TezTR-518813_1_23_1_2_0] log.PerfLogger: 2018-01-05T11:28:58,656 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:58,657 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.Shuffle: Map_1: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true 2018-01-05T11:28:58,657 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.MergeManager: Map 1: MergerManager: memoryLimit=131404920, maxSingleShuffleLimit=32851230, mergeThreshold=86727248, ioSortFactor=10, postMergeMem=0, memToMemMergeOutputsThreshold=10 2018-01-05T11:28:58,658 INFO [IPC Server handler 27 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_23_01_000000_0 1 events [0,1) total 1 vertex_1515180518813_0001_23_01 [Reducer 2] 2018-01-05T11:28:58,658 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:28:58,658 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_23_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:28:58,658 INFO [IPC Server handler 25 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_23_01_000001_0 1 events [0,1) total 1 vertex_1515180518813_0001_23_01 [Reducer 2] 2018-01-05T11:28:58,659 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:58,660 DEBUG [TezTR-518813_1_23_1_1_0] log.PerfLogger: 2018-01-05T11:28:58,660 INFO [TezTR-518813_1_23_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:58,660 INFO [TezTR-518813_1_23_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:58,660 INFO [TezTR-518813_1_23_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:58,660 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:58,660 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:28:58,660 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_23_01_000001_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:28:58,661 INFO [I/O Setup 1 Initialize: {out_Reducer 2}] output.MROutput: out_Reducer 2: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:28:58,661 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:58,661 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:58,661 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:58,661 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:58,661 INFO [TezTR-518813_1_23_1_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:58,661 INFO [TezTR-518813_1_23_1_0_0] resources.MemoryDistributor: InitialRequests=[Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [out_Reducer 2:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput] 2018-01-05T11:28:58,661 INFO [TezTR-518813_1_23_1_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:58,661 INFO [TezTR-518813_1_23_1_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:58,661 INFO [TezTR-518813_1_23_1_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 131404920, TotalRequestedScaled: 1.2129684923076923E8, TotalJVMHeap: 1033895936, TotalAvailable: 692710277, TotalRequested/TotalJVMHeap:0.13 2018-01-05T11:28:58,661 INFO [TezTR-518813_1_23_1_0_0] resources.MemoryDistributor: Allocations=[Map 1:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:131404920:131404920], [out_Reducer 2:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0] 2018-01-05T11:28:58,661 INFO [TezTR-518813_1_23_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:58,661 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.ShuffleScheduler: ShuffleScheduler running for sourceVertex: Map 1 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true 2018-01-05T11:28:58,661 DEBUG [TezTR-518813_1_23_1_2_0] log.PerfLogger: 2018-01-05T11:28:58,661 INFO [TezTR-518813_1_23_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:58,661 INFO [TezTR-518813_1_23_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:58,661 INFO [TezTR-518813_1_23_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:58,661 INFO [TezTR-518813_1_23_1_2_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:58,661 INFO [TezTR-518813_1_23_1_2_0] resources.MemoryDistributor: InitialRequests=[Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [out_Reducer 2:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput] 2018-01-05T11:28:58,661 INFO [TezTR-518813_1_23_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: Map 1 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:28:58,661 INFO [TezTR-518813_1_23_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:28:58,661 INFO [TezTR-518813_1_23_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:28:58,661 INFO [TezTR-518813_1_23_1_2_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:58,661 INFO [TezTR-518813_1_23_1_2_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:58,661 INFO [I/O Setup 1 Initialize: {out_Reducer 2}] output.MROutput: out_Reducer 2: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:28:58,661 INFO [TezTR-518813_1_23_1_2_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 131404920, TotalRequestedScaled: 1.2129684923076923E8, TotalJVMHeap: 1033895936, TotalAvailable: 692710277, TotalRequested/TotalJVMHeap:0.13 2018-01-05T11:28:58,661 INFO [TezTR-518813_1_23_1_2_0] resources.MemoryDistributor: Allocations=[Map 1:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:131404920:131404920], [out_Reducer 2:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0] 2018-01-05T11:28:58,661 INFO [TezTR-518813_1_23_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:58,661 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:58,661 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:58,662 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:58,662 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:58,662 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.Shuffle: Map_1: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true 2018-01-05T11:28:58,662 INFO [I/O Setup 0 Start: {Map 1}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: Map 1 2018-01-05T11:28:58,662 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.MergeManager: Map 1: MergerManager: memoryLimit=131404920, maxSingleShuffleLimit=32851230, mergeThreshold=86727248, ioSortFactor=10, postMergeMem=0, memToMemMergeOutputsThreshold=10 2018-01-05T11:28:58,662 INFO [TezTR-518813_1_23_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Input: Map 1 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:28:58,662 INFO [TezTR-518813_1_23_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:28:58,662 INFO [TezTR-518813_1_23_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:28:58,662 INFO [TezTR-518813_1_23_1_3_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:58,662 INFO [TezTR-518813_1_23_1_1_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:58,662 INFO [TezTR-518813_1_23_1_1_0] resources.MemoryDistributor: InitialRequests=[Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [out_Reducer 2:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput] 2018-01-05T11:28:58,662 INFO [IPC Server handler 11 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_23_01_000002_0 1 events [0,1) total 1 vertex_1515180518813_0001_23_01 [Reducer 2] 2018-01-05T11:28:58,662 INFO [TezTR-518813_1_23_1_1_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:58,663 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.Shuffle: Map_1: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true 2018-01-05T11:28:58,663 INFO [TezTR-518813_1_23_1_1_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:58,663 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.MergeManager: Map 1: MergerManager: memoryLimit=131404920, maxSingleShuffleLimit=32851230, mergeThreshold=86727248, ioSortFactor=10, postMergeMem=0, memToMemMergeOutputsThreshold=10 2018-01-05T11:28:58,663 INFO [TezTR-518813_1_23_1_1_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 131404920, TotalRequestedScaled: 1.2129684923076923E8, TotalJVMHeap: 1033895936, TotalAvailable: 692710277, TotalRequested/TotalJVMHeap:0.13 2018-01-05T11:28:58,663 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.ShuffleScheduler: ShuffleScheduler running for sourceVertex: Map 1 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true 2018-01-05T11:28:58,662 INFO [TezTR-518813_1_23_1_3_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_23_01_000003_0 2018-01-05T11:28:58,663 DEBUG [TezTR-518813_1_23_1_3_0] log.PerfLogger: 2018-01-05T11:28:58,663 DEBUG [TezTR-518813_1_23_1_3_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_23_01_000003_0_10133 2018-01-05T11:28:58,663 DEBUG [TezTR-518813_1_23_1_3_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:58,663 INFO [TezTR-518813_1_23_1_1_0] resources.MemoryDistributor: Allocations=[Map 1:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:131404920:131404920], [out_Reducer 2:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0] 2018-01-05T11:28:58,663 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:28:58,663 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_23_01_000002_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:28:58,663 DEBUG [TezTR-518813_1_23_1_3_0] mr.ObjectCache: Creating djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:28:58,663 INFO [TezTR-518813_1_23_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:58,663 INFO [TezTR-518813_1_23_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Input: Map 1 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:28:58,663 INFO [TezTaskEventRouter{attempt_1515180518813_0001_23_01_000003_0}] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:28:58,663 INFO [TezTR-518813_1_23_1_3_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_288_8305321905597716466-1/djaiswal/_tez_scratch_dir/8903eac4-2f04-45af-83ec-188e6a72521c/reduce.xml 2018-01-05T11:28:58,663 DEBUG [TezTR-518813_1_23_1_3_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_288_8305321905597716466-1/djaiswal/_tez_scratch_dir/8903eac4-2f04-45af-83ec-188e6a72521c/reduce.xml 2018-01-05T11:28:58,663 DEBUG [TezTR-518813_1_23_1_3_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_288_8305321905597716466-1/djaiswal/_tez_scratch_dir/8903eac4-2f04-45af-83ec-188e6a72521c/reduce.xml 2018-01-05T11:28:58,663 INFO [I/O Setup 0 Start: {Map 1}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: Map 1 2018-01-05T11:28:58,663 INFO [TezTR-518813_1_23_1_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:58,663 DEBUG [TezTR-518813_1_23_1_3_0] log.PerfLogger: 2018-01-05T11:28:58,664 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.Shuffle: Map_1: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true 2018-01-05T11:28:58,664 INFO [TezTR-518813_1_23_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:28:58,664 INFO [TezTR-518813_1_23_1_3_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:28:58,664 INFO [TezTR-518813_1_23_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:28:58,664 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.MergeManager: Map 1: MergerManager: memoryLimit=131404920, maxSingleShuffleLimit=32851230, mergeThreshold=86727248, ioSortFactor=10, postMergeMem=0, memToMemMergeOutputsThreshold=10 2018-01-05T11:28:58,664 DEBUG [TezTR-518813_1_23_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,664 INFO [TezTR-518813_1_23_1_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_23_01_000000_0 2018-01-05T11:28:58,664 DEBUG [TezTR-518813_1_23_1_0_0] log.PerfLogger: 2018-01-05T11:28:58,664 DEBUG [TezTR-518813_1_23_1_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_23_01_000000_0_10136 2018-01-05T11:28:58,664 INFO [TezTaskEventRouter{attempt_1515180518813_0001_23_01_000000_0}] orderedgrouped.ShuffleScheduler: All inputs fetched for input vertex : Map 1 2018-01-05T11:28:58,664 INFO [TezTaskEventRouter{attempt_1515180518813_0001_23_01_000000_0}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:28:58,664 INFO [TezTaskEventRouter{attempt_1515180518813_0001_23_01_000000_0}] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=1, numObsoletionEventsSeen=0 2018-01-05T11:28:58,664 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.ShuffleScheduler: ShuffleScheduler running for sourceVertex: Map 1 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true 2018-01-05T11:28:58,664 DEBUG [TezTR-518813_1_23_1_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:58,664 DEBUG [TezTR-518813_1_23_1_0_0] mr.ObjectCache: Creating djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:28:58,665 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down FetchScheduler for input: Map_1, wasInterrupted=false 2018-01-05T11:28:58,665 INFO [TezTR-518813_1_23_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_288_8305321905597716466-1/djaiswal/_tez_scratch_dir/8903eac4-2f04-45af-83ec-188e6a72521c/reduce.xml 2018-01-05T11:28:58,665 DEBUG [TezTR-518813_1_23_1_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_288_8305321905597716466-1/djaiswal/_tez_scratch_dir/8903eac4-2f04-45af-83ec-188e6a72521c/reduce.xml 2018-01-05T11:28:58,665 DEBUG [TezTR-518813_1_23_1_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_288_8305321905597716466-1/djaiswal/_tez_scratch_dir/8903eac4-2f04-45af-83ec-188e6a72521c/reduce.xml 2018-01-05T11:28:58,665 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.ShuffleScheduler: ShuffleScheduler running for sourceVertex: Map 1 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true 2018-01-05T11:28:58,665 DEBUG [TezTR-518813_1_23_1_0_0] log.PerfLogger: 2018-01-05T11:28:58,665 INFO [TezTR-518813_1_23_1_0_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:28:58,665 DEBUG [TezTR-518813_1_23_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,665 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:28:58,665 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=23&reduce=3&map=attempt_1515180518813_0001_23_00_000000_0_10132 mapId: [attempt_1515180518813_0001_23_00_000000_0_10132] reduceId: [3] jobId: [job_1515180518813_0001] dagId: [23] keepAlive: false 2018-01-05T11:28:58,665 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=23&reduce=3&map=attempt_1515180518813_0001_23_00_000000_0_10132; hash=...CdmIV5fOplT2w 2018-01-05T11:28:58,665 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=23&reduce=3&map=attempt_1515180518813_0001_23_00_000000_0_10132;reply=YFyRxePNjcmJY 2018-01-05T11:28:58,666 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down fetchers for input: Map_1, shutdown timetaken: 0 ms, hasFetcherExecutorStopped: true 2018-01-05T11:28:58,666 INFO [I/O Setup 0 Start: {Map 1}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: Map 1 2018-01-05T11:28:58,666 INFO [TezTR-518813_1_23_1_2_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:58,666 INFO [I/O Setup 0 Start: {Map 1}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: Map 1 2018-01-05T11:28:58,666 INFO [TezTR-518813_1_23_1_1_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:58,666 INFO [TezTR-518813_1_23_1_2_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_23_01_000002_0 2018-01-05T11:28:58,666 DEBUG [TezTR-518813_1_23_1_2_0] log.PerfLogger: 2018-01-05T11:28:58,666 DEBUG [TezTR-518813_1_23_1_2_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_23_01_000002_0_10141 2018-01-05T11:28:58,666 DEBUG [TezTR-518813_1_23_1_2_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:58,666 DEBUG [TezTR-518813_1_23_1_2_0] mr.ObjectCache: Creating djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:28:58,666 DEBUG [TezTR-518813_1_23_1_3_0] log.PerfLogger: 2018-01-05T11:28:58,666 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=23, user='djaiswal', attemptId='attempt_1515180518813_0001_23_00_000000_0_10132'} via loader 2018-01-05T11:28:58,667 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: finalMerge with #inMemoryOutputs=0, size=0 and #onDiskOutputs=0, size=0 2018-01-05T11:28:58,667 INFO [TezTR-518813_1_23_1_3_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 2.93KB 2018-01-05T11:28:58,667 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=23, user='djaiswal', attemptId='attempt_1515180518813_0001_23_00_000000_0_10132'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:28:58,667 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_23_00_000000_0_10132 not found 2018-01-05T11:28:58,666 INFO [TezTR-518813_1_23_1_2_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_288_8305321905597716466-1/djaiswal/_tez_scratch_dir/8903eac4-2f04-45af-83ec-188e6a72521c/reduce.xml 2018-01-05T11:28:58,667 DEBUG [TezTR-518813_1_23_1_2_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_288_8305321905597716466-1/djaiswal/_tez_scratch_dir/8903eac4-2f04-45af-83ec-188e6a72521c/reduce.xml 2018-01-05T11:28:58,667 DEBUG [TezTR-518813_1_23_1_2_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_288_8305321905597716466-1/djaiswal/_tez_scratch_dir/8903eac4-2f04-45af-83ec-188e6a72521c/reduce.xml 2018-01-05T11:28:58,667 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: . DiskSeg: 0, 0. MemSeg: 0, 0 2018-01-05T11:28:58,667 INFO [ShuffleAndMergeRunner {Map_1}] impl.TezMerger: Nothing to merge. Returning an empty iterator 2018-01-05T11:28:58,667 DEBUG [TezTR-518813_1_23_1_2_0] log.PerfLogger: 2018-01-05T11:28:58,667 INFO [TezTR-518813_1_23_1_2_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:28:58,667 DEBUG [TezTR-518813_1_23_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,667 DEBUG [TezTR-518813_1_23_1_0_0] log.PerfLogger: 2018-01-05T11:28:58,667 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: merge complete for input vertex : Map_1 2018-01-05T11:28:58,667 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: Map_1: Shuffle Runner thread complete 2018-01-05T11:28:58,667 INFO [TezTR-518813_1_23_1_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 2.93KB 2018-01-05T11:28:58,667 INFO [TezTR-518813_1_23_1_1_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_23_01_000001_0 2018-01-05T11:28:58,667 DEBUG [TezTR-518813_1_23_1_1_0] log.PerfLogger: 2018-01-05T11:28:58,667 DEBUG [TezTR-518813_1_23_1_1_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_23_01_000001_0_10139 2018-01-05T11:28:58,667 DEBUG [TezTR-518813_1_23_1_1_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:58,667 DEBUG [TezTR-518813_1_23_1_1_0] mr.ObjectCache: Creating djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:28:58,667 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_23_00_000000_0_10132,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/23/output/attempt_1515180518813_0001_23_00_000000_0_10132/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/23/output/attempt_1515180518813_0001_23_00_000000_0_10132/file.out.index 2018-01-05T11:28:58,668 DEBUG [TezTR-518813_1_23_1_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:58,668 DEBUG [TezTR-518813_1_23_1_0_0] log.PerfLogger: 2018-01-05T11:28:58,668 INFO [TezTaskEventRouter{attempt_1515180518813_0001_23_01_000001_0}] orderedgrouped.ShuffleScheduler: All inputs fetched for input vertex : Map 1 2018-01-05T11:28:58,667 DEBUG [TezTR-518813_1_23_1_3_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:58,668 DEBUG [TezTR-518813_1_23_1_3_0] log.PerfLogger: 2018-01-05T11:28:58,667 INFO [TezTaskEventRouter{attempt_1515180518813_0001_23_01_000002_0}] orderedgrouped.ShuffleScheduler: All inputs fetched for input vertex : Map 1 2018-01-05T11:28:58,668 DEBUG [TezTR-518813_1_23_1_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:58,668 INFO [TezTaskEventRouter{attempt_1515180518813_0001_23_01_000002_0}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:28:58,668 INFO [TezTaskEventRouter{attempt_1515180518813_0001_23_01_000002_0}] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=1, numObsoletionEventsSeen=0 2018-01-05T11:28:58,668 DEBUG [TezTR-518813_1_23_1_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:58,668 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Content Length in shuffle : 239 2018-01-05T11:28:58,668 DEBUG [TezTR-518813_1_23_1_3_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:58,668 DEBUG [TezTR-518813_1_23_1_3_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:58,668 DEBUG [New I/O worker #1] shufflehandler.FadvisedFileRegion: shuffleBufferSize: 131072, path: /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/23/output/attempt_1515180518813_0001_23_00_000000_0_10132/file.out 2018-01-05T11:28:58,668 INFO [TezTaskEventRouter{attempt_1515180518813_0001_23_01_000001_0}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:28:58,669 DEBUG [TezTR-518813_1_23_1_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:58,669 INFO [TezTR-518813_1_23_1_0_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:28:58,669 DEBUG [TezTR-518813_1_23_1_3_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:58,669 INFO [TezTR-518813_1_23_1_0_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:28:58,669 DEBUG [TezTR-518813_1_23_1_2_0] log.PerfLogger: 2018-01-05T11:28:58,669 INFO [TezTR-518813_1_23_1_0_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:58,669 INFO [TezTR-518813_1_23_1_2_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 2.93KB 2018-01-05T11:28:58,668 INFO [TezTR-518813_1_23_1_1_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_288_8305321905597716466-1/djaiswal/_tez_scratch_dir/8903eac4-2f04-45af-83ec-188e6a72521c/reduce.xml 2018-01-05T11:28:58,669 DEBUG [TezTR-518813_1_23_1_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:28:58,669 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down FetchScheduler for input: Map_1, wasInterrupted=false 2018-01-05T11:28:58,668 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down FetchScheduler for input: Map_1, wasInterrupted=false 2018-01-05T11:28:58,669 INFO [Fetcher_O {Map_1} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=23&reduce=3&map=attempt_1515180518813_0001_23_00_000000_0_10132 sent hash and receievd reply 0 ms 2018-01-05T11:28:58,669 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:28:58,669 INFO [TezTaskEventRouter{attempt_1515180518813_0001_23_01_000001_0}] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=1, numObsoletionEventsSeen=0 2018-01-05T11:28:58,669 INFO [TezTR-518813_1_23_1_3_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:28:58,669 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:28:58,669 DEBUG [TezTR-518813_1_23_1_1_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_288_8305321905597716466-1/djaiswal/_tez_scratch_dir/8903eac4-2f04-45af-83ec-188e6a72521c/reduce.xml 2018-01-05T11:28:58,670 DEBUG [TezTR-518813_1_23_1_1_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_288_8305321905597716466-1/djaiswal/_tez_scratch_dir/8903eac4-2f04-45af-83ec-188e6a72521c/reduce.xml 2018-01-05T11:28:58,669 DEBUG [TezTR-518813_1_23_1_2_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:58,670 DEBUG [TezTR-518813_1_23_1_1_0] log.PerfLogger: 2018-01-05T11:28:58,670 INFO [TezTR-518813_1_23_1_0_0] input.OrderedGroupedKVInput: Map 1: creating ValuesIterator with comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator, keyClass=org.apache.hadoop.hive.ql.io.HiveKey, valClass=org.apache.hadoop.io.BytesWritable 2018-01-05T11:28:58,670 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down fetchers for input: Map_1, shutdown timetaken: 0 ms, hasFetcherExecutorStopped: true 2018-01-05T11:28:58,670 DEBUG [TezTR-518813_1_23_1_2_0] log.PerfLogger: 2018-01-05T11:28:58,670 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: finalMerge with #inMemoryOutputs=0, size=0 and #onDiskOutputs=0, size=0 2018-01-05T11:28:58,670 DEBUG [TezTR-518813_1_23_1_2_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:58,670 DEBUG [TezTR-518813_1_23_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,671 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: . DiskSeg: 0, 0. MemSeg: 0, 0 2018-01-05T11:28:58,670 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down fetchers for input: Map_1, shutdown timetaken: 0 ms, hasFetcherExecutorStopped: true 2018-01-05T11:28:58,671 INFO [ShuffleAndMergeRunner {Map_1}] impl.TezMerger: Nothing to merge. Returning an empty iterator 2018-01-05T11:28:58,670 INFO [TezTR-518813_1_23_1_1_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:28:58,671 DEBUG [TezTR-518813_1_23_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,670 DEBUG [TezTR-518813_1_23_1_2_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:58,671 DEBUG [TezTR-518813_1_23_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,671 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: finalMerge with #inMemoryOutputs=0, size=0 and #onDiskOutputs=0, size=0 2018-01-05T11:28:58,670 INFO [Fetcher_O {Map_1} #0] orderedgrouped.MergeManager: CloseInMemoryFile. Current state: inMemoryMapOutputs.size=1, commitMemory=0, usedMemory=182. Since last log: count=1, min=182, max=182, total=182, avg=182.0 2018-01-05T11:28:58,671 INFO [Fetcher_O {Map_1} #0] ShuffleScheduler.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_23_00_000000_0_10132} to MEMORY, csize=186, dsize=182, EndTime=1515180538671, TimeTaken=1, Rate=0.17 MB/s 2018-01-05T11:28:58,671 INFO [Fetcher_O {Map_1} #0] orderedgrouped.ShuffleScheduler: All inputs fetched for input vertex : Map 1 2018-01-05T11:28:58,671 INFO [Fetcher_O {Map_1} #0] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:28:58,671 DEBUG [TezTR-518813_1_23_1_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1, _col2, _col3, _col4] columnTypes=[struct, struct, struct, struct, struct] 2018-01-05T11:28:58,671 DEBUG [TezTR-518813_1_23_1_0_0] log.PerfLogger: 2018-01-05T11:28:58,671 INFO [TezTR-518813_1_23_1_0_0] exec.GroupByOperator: Initializing operator GBY[4] 2018-01-05T11:28:58,671 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: . DiskSeg: 0, 0. MemSeg: 0, 0 2018-01-05T11:28:58,671 INFO [ShuffleAndMergeRunner {Map_1}] impl.TezMerger: Nothing to merge. Returning an empty iterator 2018-01-05T11:28:58,671 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: merge complete for input vertex : Map_1 2018-01-05T11:28:58,671 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: Map_1: Shuffle Runner thread complete 2018-01-05T11:28:58,671 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: merge complete for input vertex : Map_1 2018-01-05T11:28:58,671 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down FetchScheduler for input: Map_1, wasInterrupted=false 2018-01-05T11:28:58,671 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: Map_1: Shuffle Runner thread complete 2018-01-05T11:28:58,671 DEBUG [TezTR-518813_1_23_1_2_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:58,671 INFO [TezTR-518813_1_23_1_2_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:28:58,671 INFO [TezTR-518813_1_23_1_2_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:28:58,672 INFO [TezTR-518813_1_23_1_2_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:58,671 DEBUG [TezTR-518813_1_23_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,672 DEBUG [TezTR-518813_1_23_1_2_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:28:58,672 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:28:58,672 DEBUG [TezTR-518813_1_23_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,672 DEBUG [TezTR-518813_1_23_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,672 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down fetchers for input: Map_1, shutdown timetaken: 0 ms, hasFetcherExecutorStopped: true 2018-01-05T11:28:58,672 INFO [TezTR-518813_1_23_1_2_0] input.OrderedGroupedKVInput: Map 1: creating ValuesIterator with comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator, keyClass=org.apache.hadoop.hive.ql.io.HiveKey, valClass=org.apache.hadoop.io.BytesWritable 2018-01-05T11:28:58,672 DEBUG [TezTR-518813_1_23_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,672 DEBUG [TezTR-518813_1_23_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,672 DEBUG [TezTR-518813_1_23_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,672 DEBUG [TezTR-518813_1_23_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,672 DEBUG [TezTR-518813_1_23_1_1_0] log.PerfLogger: 2018-01-05T11:28:58,672 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: TotalInMemFetchStats: count=1, totalSize=182, min=182, max=182, avg=1.0 2018-01-05T11:28:58,672 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: finalMerge with #inMemoryOutputs=1, size=182 and #onDiskOutputs=0, size=0 2018-01-05T11:28:58,672 DEBUG [TezTR-518813_1_23_1_2_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1, _col2, _col3, _col4] columnTypes=[struct, struct, struct, struct, struct] 2018-01-05T11:28:58,672 DEBUG [TezTR-518813_1_23_1_2_0] log.PerfLogger: 2018-01-05T11:28:58,672 INFO [TezTR-518813_1_23_1_1_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 2.93KB 2018-01-05T11:28:58,672 INFO [TezTR-518813_1_23_1_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:58,672 INFO [TezTR-518813_1_23_1_2_0] exec.GroupByOperator: Initializing operator GBY[4] 2018-01-05T11:28:58,672 DEBUG [TezTR-518813_1_23_1_0_0] exec.GroupByOperator: Initialization Done 4 GBY 2018-01-05T11:28:58,672 DEBUG [TezTR-518813_1_23_1_0_0] exec.GroupByOperator: Operator 4 GBY initialized 2018-01-05T11:28:58,672 DEBUG [TezTR-518813_1_23_1_0_0] exec.GroupByOperator: Initializing children of 4 GBY 2018-01-05T11:28:58,672 DEBUG [TezTR-518813_1_23_1_0_0] exec.SelectOperator: Initializing child 5 SEL 2018-01-05T11:28:58,672 INFO [TezTR-518813_1_23_1_0_0] exec.SelectOperator: Initializing operator SEL[5] 2018-01-05T11:28:58,672 DEBUG [TezTR-518813_1_23_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,672 DEBUG [TezTR-518813_1_23_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,672 INFO [TezTR-518813_1_23_1_0_0] exec.SelectOperator: SELECT struct<_col0:string,_col1:struct,_col2:struct,_col3:struct,_col4:struct,_col5:struct> 2018-01-05T11:28:58,672 DEBUG [TezTR-518813_1_23_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,673 DEBUG [TezTR-518813_1_23_1_0_0] exec.SelectOperator: Initialization Done 5 SEL 2018-01-05T11:28:58,673 DEBUG [TezTR-518813_1_23_1_0_0] exec.SelectOperator: Operator 5 SEL initialized 2018-01-05T11:28:58,673 DEBUG [TezTR-518813_1_23_1_0_0] exec.SelectOperator: Initializing children of 5 SEL 2018-01-05T11:28:58,673 DEBUG [TezTR-518813_1_23_1_1_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:58,673 DEBUG [TezTR-518813_1_23_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,673 DEBUG [TezTR-518813_1_23_1_1_0] log.PerfLogger: 2018-01-05T11:28:58,673 DEBUG [TezTR-518813_1_23_1_0_0] exec.FileSinkOperator: Initializing child 6 FS 2018-01-05T11:28:58,673 INFO [TezTR-518813_1_23_1_0_0] exec.FileSinkOperator: Initializing operator FS[6] 2018-01-05T11:28:58,673 DEBUG [TezTR-518813_1_23_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,673 DEBUG [TezTR-518813_1_23_1_1_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:58,673 DEBUG [TezTR-518813_1_23_1_1_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:58,673 INFO [TezTR-518813_1_23_1_2_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:58,673 DEBUG [TezTR-518813_1_23_1_2_0] exec.GroupByOperator: Initialization Done 4 GBY 2018-01-05T11:28:58,673 DEBUG [TezTR-518813_1_23_1_2_0] exec.GroupByOperator: Operator 4 GBY initialized 2018-01-05T11:28:58,673 DEBUG [TezTR-518813_1_23_1_2_0] exec.GroupByOperator: Initializing children of 4 GBY 2018-01-05T11:28:58,673 DEBUG [TezTR-518813_1_23_1_2_0] exec.SelectOperator: Initializing child 5 SEL 2018-01-05T11:28:58,673 DEBUG [TezTR-518813_1_23_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,673 INFO [TezTR-518813_1_23_1_2_0] exec.SelectOperator: Initializing operator SEL[5] 2018-01-05T11:28:58,673 INFO [TezTR-518813_1_23_1_2_0] exec.SelectOperator: SELECT struct<_col0:string,_col1:struct,_col2:struct,_col3:struct,_col4:struct,_col5:struct> 2018-01-05T11:28:58,673 DEBUG [TezTR-518813_1_23_1_2_0] exec.SelectOperator: Initialization Done 5 SEL 2018-01-05T11:28:58,673 DEBUG [TezTR-518813_1_23_1_2_0] exec.SelectOperator: Operator 5 SEL initialized 2018-01-05T11:28:58,673 DEBUG [TezTR-518813_1_23_1_2_0] exec.SelectOperator: Initializing children of 5 SEL 2018-01-05T11:28:58,673 DEBUG [TezTR-518813_1_23_1_2_0] exec.FileSinkOperator: Initializing child 6 FS 2018-01-05T11:28:58,673 INFO [TezTR-518813_1_23_1_2_0] exec.FileSinkOperator: Initializing operator FS[6] 2018-01-05T11:28:58,673 DEBUG [TezTR-518813_1_23_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,673 DEBUG [TezTR-518813_1_23_1_1_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:58,673 INFO [TezTR-518813_1_23_1_0_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@7f40d97f]:[_col0, _col1, _col2, _col3, _col4, _col5]:[struct, struct, struct, struct, struct, string]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@7d116c7d 2018-01-05T11:28:58,673 INFO [TezTR-518813_1_23_1_1_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:28:58,673 INFO [TezTR-518813_1_23_1_1_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:28:58,673 DEBUG [TezTR-518813_1_23_1_0_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:28:58,673 INFO [TezTR-518813_1_23_1_1_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:58,673 DEBUG [TezTR-518813_1_23_1_0_0] exec.FileSinkOperator: Initialization Done 6 FS 2018-01-05T11:28:58,673 DEBUG [TezTR-518813_1_23_1_1_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:28:58,673 DEBUG [TezTR-518813_1_23_1_0_0] exec.FileSinkOperator: Operator 6 FS initialized 2018-01-05T11:28:58,673 DEBUG [TezTR-518813_1_23_1_0_0] exec.FileSinkOperator: Initialization Done 6 FS done is reset. 2018-01-05T11:28:58,673 DEBUG [TezTR-518813_1_23_1_0_0] exec.SelectOperator: Initialization Done 5 SEL done is reset. 2018-01-05T11:28:58,673 DEBUG [TezTR-518813_1_23_1_0_0] exec.GroupByOperator: Initialization Done 4 GBY done is reset. 2018-01-05T11:28:58,673 INFO [TezTR-518813_1_23_1_1_0] input.OrderedGroupedKVInput: Map 1: creating ValuesIterator with comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator, keyClass=org.apache.hadoop.hive.ql.io.HiveKey, valClass=org.apache.hadoop.io.BytesWritable 2018-01-05T11:28:58,673 INFO [TezTR-518813_1_23_1_0_0] tez.ReduceRecordProcessor: Id =4 Id =5 Id =6 <\Children> Id = 5 null<\Parent> <\FS> <\Children> Id = 4 null<\Parent> <\SEL> <\Children> <\Parent> <\GBY> 2018-01-05T11:28:58,673 INFO [TezTR-518813_1_23_1_2_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@42a609d2]:[_col0, _col1, _col2, _col3, _col4, _col5]:[struct, struct, struct, struct, struct, string]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@1b0fb95a 2018-01-05T11:28:58,673 DEBUG [TezTR-518813_1_23_1_0_0] log.PerfLogger: 2018-01-05T11:28:58,673 INFO [TezTR-518813_1_23_1_0_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 2 2018-01-05T11:28:58,673 DEBUG [TezTR-518813_1_23_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,673 DEBUG [TezTR-518813_1_23_1_0_0] log.PerfLogger: 2018-01-05T11:28:58,673 DEBUG [TezTR-518813_1_23_1_2_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:28:58,673 DEBUG [TezTR-518813_1_23_1_0_0] mr.ObjectCache: djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:28:58,673 DEBUG [TezTR-518813_1_23_1_2_0] exec.FileSinkOperator: Initialization Done 6 FS 2018-01-05T11:28:58,673 DEBUG [TezTR-518813_1_23_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,673 DEBUG [TezTR-518813_1_23_1_2_0] exec.FileSinkOperator: Operator 6 FS initialized 2018-01-05T11:28:58,673 DEBUG [TezTR-518813_1_23_1_0_0] exec.GroupByOperator: close called for operator GBY[4] 2018-01-05T11:28:58,673 INFO [TezTR-518813_1_23_1_0_0] exec.GroupByOperator: Closing operator GBY[4] 2018-01-05T11:28:58,674 DEBUG [TezTR-518813_1_23_1_0_0] exec.GroupByOperator: Closing child = SEL[5] 2018-01-05T11:28:58,674 DEBUG [TezTR-518813_1_23_1_0_0] exec.SelectOperator: close called for operator SEL[5] 2018-01-05T11:28:58,673 DEBUG [TezTR-518813_1_23_1_2_0] exec.FileSinkOperator: Initialization Done 6 FS done is reset. 2018-01-05T11:28:58,674 DEBUG [TezTR-518813_1_23_1_0_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:58,674 DEBUG [TezTR-518813_1_23_1_1_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1, _col2, _col3, _col4] columnTypes=[struct, struct, struct, struct, struct] 2018-01-05T11:28:58,674 DEBUG [TezTR-518813_1_23_1_2_0] exec.SelectOperator: Initialization Done 5 SEL done is reset. 2018-01-05T11:28:58,674 DEBUG [TezTR-518813_1_23_1_2_0] exec.GroupByOperator: Initialization Done 4 GBY done is reset. 2018-01-05T11:28:58,674 INFO [TezTR-518813_1_23_1_0_0] exec.SelectOperator: Closing operator SEL[5] 2018-01-05T11:28:58,674 DEBUG [TezTR-518813_1_23_1_1_0] log.PerfLogger: 2018-01-05T11:28:58,674 INFO [TezTR-518813_1_23_1_2_0] tez.ReduceRecordProcessor: Id =4 Id =5 Id =6 <\Children> Id = 5 null<\Parent> <\FS> <\Children> Id = 4 null<\Parent> <\SEL> <\Children> <\Parent> <\GBY> 2018-01-05T11:28:58,674 DEBUG [TezTR-518813_1_23_1_0_0] exec.SelectOperator: Closing child = FS[6] 2018-01-05T11:28:58,674 DEBUG [TezTR-518813_1_23_1_2_0] log.PerfLogger: 2018-01-05T11:28:58,674 INFO [TezTR-518813_1_23_1_2_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 2 2018-01-05T11:28:58,674 INFO [TezTR-518813_1_23_1_1_0] exec.GroupByOperator: Initializing operator GBY[4] 2018-01-05T11:28:58,674 DEBUG [TezTR-518813_1_23_1_0_0] exec.FileSinkOperator: close called for operator FS[6] 2018-01-05T11:28:58,674 DEBUG [TezTR-518813_1_23_1_0_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:58,674 DEBUG [TezTR-518813_1_23_1_2_0] log.PerfLogger: 2018-01-05T11:28:58,674 INFO [TezTR-518813_1_23_1_0_0] exec.FileSinkOperator: Closing operator FS[6] 2018-01-05T11:28:58,674 INFO [TezTR-518813_1_23_1_0_0] exec.FileSinkOperator: FS[6]: records written - 0 2018-01-05T11:28:58,674 DEBUG [TezTR-518813_1_23_1_2_0] mr.ObjectCache: djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:28:58,674 DEBUG [TezTR-518813_1_23_1_2_0] exec.GroupByOperator: close called for operator GBY[4] 2018-01-05T11:28:58,674 INFO [TezTR-518813_1_23_1_2_0] exec.GroupByOperator: Closing operator GBY[4] 2018-01-05T11:28:58,674 DEBUG [TezTR-518813_1_23_1_2_0] exec.GroupByOperator: Closing child = SEL[5] 2018-01-05T11:28:58,674 INFO [TezTR-518813_1_23_1_0_0] exec.FileSinkOperator: RECORDS_OUT_0:0, 2018-01-05T11:28:58,674 DEBUG [TezTR-518813_1_23_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,674 DEBUG [TezTR-518813_1_23_1_0_0] exec.FileSinkOperator: 6 Close done 2018-01-05T11:28:58,674 DEBUG [TezTR-518813_1_23_1_0_0] exec.SelectOperator: 5 Close done 2018-01-05T11:28:58,674 DEBUG [TezTR-518813_1_23_1_0_0] exec.GroupByOperator: 4 Close done 2018-01-05T11:28:58,674 DEBUG [TezTR-518813_1_23_1_2_0] exec.SelectOperator: close called for operator SEL[5] 2018-01-05T11:28:58,674 DEBUG [TezTR-518813_1_23_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,674 DEBUG [TezTR-518813_1_23_1_2_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:58,674 INFO [TezTR-518813_1_23_1_2_0] exec.SelectOperator: Closing operator SEL[5] 2018-01-05T11:28:58,674 DEBUG [TezTR-518813_1_23_1_2_0] exec.SelectOperator: Closing child = FS[6] 2018-01-05T11:28:58,674 INFO [TezTR-518813_1_23_1_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_23_01_000000_0 2018-01-05T11:28:58,674 DEBUG [TezTR-518813_1_23_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,674 INFO [TezTR-518813_1_23_1_0_0] orderedgrouped.Shuffle: Shutting down Shuffle for source: Map_1 2018-01-05T11:28:58,674 DEBUG [TezTR-518813_1_23_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,674 DEBUG [TezTR-518813_1_23_1_2_0] exec.FileSinkOperator: close called for operator FS[6] 2018-01-05T11:28:58,674 INFO [TezTR-518813_1_23_1_0_0] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=1, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:28:58,674 DEBUG [TezTR-518813_1_23_1_2_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:58,674 INFO [TezTR-518813_1_23_1_2_0] exec.FileSinkOperator: Closing operator FS[6] 2018-01-05T11:28:58,674 INFO [TezTR-518813_1_23_1_2_0] exec.FileSinkOperator: FS[6]: records written - 0 2018-01-05T11:28:58,674 INFO [TezTR-518813_1_23_1_0_0] output.MROutput: out_Reducer 2 closed 2018-01-05T11:28:58,674 INFO [TezTR-518813_1_23_1_2_0] exec.FileSinkOperator: RECORDS_OUT_0:0, 2018-01-05T11:28:58,674 DEBUG [TezTR-518813_1_23_1_2_0] exec.FileSinkOperator: 6 Close done 2018-01-05T11:28:58,674 DEBUG [TezTR-518813_1_23_1_2_0] exec.SelectOperator: 5 Close done 2018-01-05T11:28:58,674 DEBUG [TezTR-518813_1_23_1_2_0] exec.GroupByOperator: 4 Close done 2018-01-05T11:28:58,674 DEBUG [TezTR-518813_1_23_1_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:58,674 INFO [TezTR-518813_1_23_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:58,674 INFO [TezTR-518813_1_23_1_2_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_23_01_000002_0 2018-01-05T11:28:58,674 INFO [TezTR-518813_1_23_1_2_0] orderedgrouped.Shuffle: Shutting down Shuffle for source: Map_1 2018-01-05T11:28:58,674 INFO [TezTR-518813_1_23_1_2_0] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=1, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:28:58,674 INFO [TezTR-518813_1_23_1_2_0] output.MROutput: out_Reducer 2 closed 2018-01-05T11:28:58,674 DEBUG [TezTR-518813_1_23_1_2_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:58,674 INFO [TezTR-518813_1_23_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:58,674 DEBUG [TezTR-518813_1_23_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,674 INFO [TezTR-518813_1_23_1_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_23_01_000000_0, askedToStop=false 2018-01-05T11:28:58,674 INFO [TezTR-518813_1_23_1_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_23_01_000000_0, stopRequested=false 2018-01-05T11:28:58,675 INFO [TezTR-518813_1_23_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_23_01_000000_0: Counters: 29 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=0, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=0, NUM_SKIPPED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=0, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, SHUFFLE_BYTES=0, SHUFFLE_BYTES_DECOMPRESSED=0, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=5, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2][HIVE RECORDS_OUT_0=0][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0]] 2018-01-05T11:28:58,675 INFO [TezTR-518813_1_23_1_2_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_23_01_000002_0, askedToStop=false 2018-01-05T11:28:58,675 INFO [TezTR-518813_1_23_1_2_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_23_01_000002_0, stopRequested=false 2018-01-05T11:28:58,675 DEBUG [TezTR-518813_1_23_1_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_23_1_0_0 thread id: 1573 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:58,675 INFO [TezTR-518813_1_23_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_23_01_000002_0: Counters: 29 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=0, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=0, NUM_SKIPPED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=0, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, SHUFFLE_BYTES=0, SHUFFLE_BYTES_DECOMPRESSED=0, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][HIVE RECORDS_OUT_0=0][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0]] 2018-01-05T11:28:58,675 INFO [TezTR-518813_1_23_1_1_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:58,675 DEBUG [TezTR-518813_1_23_1_1_0] exec.GroupByOperator: Initialization Done 4 GBY 2018-01-05T11:28:58,675 DEBUG [TezTR-518813_1_23_1_1_0] exec.GroupByOperator: Operator 4 GBY initialized 2018-01-05T11:28:58,675 DEBUG [TezTR-518813_1_23_1_1_0] exec.GroupByOperator: Initializing children of 4 GBY 2018-01-05T11:28:58,675 DEBUG [TezTR-518813_1_23_1_1_0] exec.SelectOperator: Initializing child 5 SEL 2018-01-05T11:28:58,675 INFO [TezTR-518813_1_23_1_1_0] exec.SelectOperator: Initializing operator SEL[5] 2018-01-05T11:28:58,675 DEBUG [TezTR-518813_1_23_1_2_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_23_1_2_0 thread id: 1584 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:58,675 INFO [TezTR-518813_1_23_1_1_0] exec.SelectOperator: SELECT struct<_col0:string,_col1:struct,_col2:struct,_col3:struct,_col4:struct,_col5:struct> 2018-01-05T11:28:58,675 DEBUG [TezTR-518813_1_23_1_1_0] exec.SelectOperator: Initialization Done 5 SEL 2018-01-05T11:28:58,675 DEBUG [TezTR-518813_1_23_1_1_0] exec.SelectOperator: Operator 5 SEL initialized 2018-01-05T11:28:58,675 DEBUG [TezTR-518813_1_23_1_1_0] exec.SelectOperator: Initializing children of 5 SEL 2018-01-05T11:28:58,675 DEBUG [TezTR-518813_1_23_1_1_0] exec.FileSinkOperator: Initializing child 6 FS 2018-01-05T11:28:58,675 INFO [TezTR-518813_1_23_1_1_0] exec.FileSinkOperator: Initializing operator FS[6] 2018-01-05T11:28:58,675 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_23_01_000002_0, isTaskDone=true 2018-01-05T11:28:58,675 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000048, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_23_01_000002_0, eventCount=2 } 2018-01-05T11:28:58,675 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_23_01_000000_0, isTaskDone=true 2018-01-05T11:28:58,675 DEBUG [TezTR-518813_1_23_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,675 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000046, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_23_01_000000_0, eventCount=2 } 2018-01-05T11:28:58,675 INFO [TezTR-518813_1_23_1_1_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@6297b344]:[_col0, _col1, _col2, _col3, _col4, _col5]:[struct, struct, struct, struct, struct, string]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@23e2881d 2018-01-05T11:28:58,675 DEBUG [TezTR-518813_1_23_1_1_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:28:58,675 DEBUG [TezTR-518813_1_23_1_1_0] exec.FileSinkOperator: Initialization Done 6 FS 2018-01-05T11:28:58,675 DEBUG [TezTR-518813_1_23_1_1_0] exec.FileSinkOperator: Operator 6 FS initialized 2018-01-05T11:28:58,675 DEBUG [TezTR-518813_1_23_1_1_0] exec.FileSinkOperator: Initialization Done 6 FS done is reset. 2018-01-05T11:28:58,675 DEBUG [TezTR-518813_1_23_1_1_0] exec.SelectOperator: Initialization Done 5 SEL done is reset. 2018-01-05T11:28:58,675 DEBUG [TezTR-518813_1_23_1_1_0] exec.GroupByOperator: Initialization Done 4 GBY done is reset. 2018-01-05T11:28:58,675 INFO [TezTR-518813_1_23_1_1_0] tez.ReduceRecordProcessor: Id =4 Id =5 Id =6 <\Children> Id = 5 null<\Parent> <\FS> <\Children> Id = 4 null<\Parent> <\SEL> <\Children> <\Parent> <\GBY> 2018-01-05T11:28:58,675 DEBUG [TezTR-518813_1_23_1_1_0] log.PerfLogger: 2018-01-05T11:28:58,675 INFO [TezTR-518813_1_23_1_1_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 2 2018-01-05T11:28:58,676 DEBUG [TezTR-518813_1_23_1_1_0] log.PerfLogger: 2018-01-05T11:28:58,676 DEBUG [TezTR-518813_1_23_1_1_0] mr.ObjectCache: djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:28:58,676 DEBUG [TezTR-518813_1_23_1_1_0] exec.GroupByOperator: close called for operator GBY[4] 2018-01-05T11:28:58,676 INFO [TezTR-518813_1_23_1_1_0] exec.GroupByOperator: Closing operator GBY[4] 2018-01-05T11:28:58,676 DEBUG [TezTR-518813_1_23_1_1_0] exec.GroupByOperator: Closing child = SEL[5] 2018-01-05T11:28:58,676 DEBUG [TezTR-518813_1_23_1_1_0] exec.SelectOperator: close called for operator SEL[5] 2018-01-05T11:28:58,676 DEBUG [TezTR-518813_1_23_1_1_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:58,676 INFO [TezTR-518813_1_23_1_1_0] exec.SelectOperator: Closing operator SEL[5] 2018-01-05T11:28:58,676 DEBUG [TezTR-518813_1_23_1_1_0] exec.SelectOperator: Closing child = FS[6] 2018-01-05T11:28:58,676 DEBUG [TezTR-518813_1_23_1_1_0] exec.FileSinkOperator: close called for operator FS[6] 2018-01-05T11:28:58,676 DEBUG [TezTR-518813_1_23_1_1_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:58,676 INFO [TezTR-518813_1_23_1_1_0] exec.FileSinkOperator: Closing operator FS[6] 2018-01-05T11:28:58,676 INFO [TezTR-518813_1_23_1_1_0] exec.FileSinkOperator: FS[6]: records written - 0 2018-01-05T11:28:58,676 INFO [TezTR-518813_1_23_1_1_0] exec.FileSinkOperator: RECORDS_OUT_0:0, 2018-01-05T11:28:58,676 DEBUG [TezTR-518813_1_23_1_1_0] exec.FileSinkOperator: 6 Close done 2018-01-05T11:28:58,676 DEBUG [TezTR-518813_1_23_1_1_0] exec.SelectOperator: 5 Close done 2018-01-05T11:28:58,676 DEBUG [TezTR-518813_1_23_1_1_0] exec.GroupByOperator: 4 Close done 2018-01-05T11:28:58,676 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_23][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_23_01_000002_0, creationTime=1515180538643, allocationTime=1515180538646, startTime=1515180538648, finishTime=1515180538676, timeTaken=28, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:58,676 INFO [TezTR-518813_1_23_1_1_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_23_01_000001_0 2018-01-05T11:28:58,676 INFO [TezTR-518813_1_23_1_1_0] orderedgrouped.Shuffle: Shutting down Shuffle for source: Map_1 2018-01-05T11:28:58,676 INFO [TezTR-518813_1_23_1_1_0] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=1, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:28:58,676 INFO [TezTR-518813_1_23_1_1_0] output.MROutput: out_Reducer 2 closed 2018-01-05T11:28:58,676 DEBUG [TezTR-518813_1_23_1_1_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:58,676 INFO [TezTR-518813_1_23_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:58,676 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_23][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_23_01_000000_0, creationTime=1515180538643, allocationTime=1515180538645, startTime=1515180538646, finishTime=1515180538676, timeTaken=30, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:58,676 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_23_01_000002_0 2018-01-05T11:28:58,676 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:28:58,676 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_23][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_23_01_000002, startTime=1515180538648, finishTime=1515180538676, timeTaken=28, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_23_01_000002_0, diagnostics=, counters=Counters: 34, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=0, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=0, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=0, NUM_SKIPPED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=0, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, SHUFFLE_BYTES=0, SHUFFLE_BYTES_DECOMPRESSED=0, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, HIVE, RECORDS_OUT_0=0, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0 2018-01-05T11:28:58,676 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_23_01_000002_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:58,676 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_23_01_000002_0; false 2018-01-05T11:28:58,676 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_23_01_000000_0 2018-01-05T11:28:58,676 INFO [Task-Executor-2] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_23_01_000002_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:58,676 INFO [Task-Executor-2] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_23_01_000002_0 2018-01-05T11:28:58,676 INFO [TezTR-518813_1_23_1_1_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_23_01_000001_0, askedToStop=false 2018-01-05T11:28:58,676 INFO [TezTR-518813_1_23_1_1_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_23_01_000001_0, stopRequested=false 2018-01-05T11:28:58,676 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_23_01_000002_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:58,676 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_23_01_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:58,676 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_23_01_000000_0; false 2018-01-05T11:28:58,676 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_23_01_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:58,676 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:28:58,676 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:58,676 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:58,676 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_23][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_23_01_000000, startTime=1515180538646, finishTime=1515180538676, timeTaken=30, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_23_01_000000_0, diagnostics=, counters=Counters: 34, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=0, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=0, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=0, NUM_SKIPPED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=0, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, SHUFFLE_BYTES=0, SHUFFLE_BYTES_DECOMPRESSED=0, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=5, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, HIVE, RECORDS_OUT_0=0, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0 2018-01-05T11:28:58,676 INFO [TezTR-518813_1_23_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_23_01_000001_0: Counters: 29 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=0, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=0, NUM_SKIPPED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=0, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, SHUFFLE_BYTES=0, SHUFFLE_BYTES_DECOMPRESSED=0, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][HIVE RECORDS_OUT_0=0][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0]] 2018-01-05T11:28:58,677 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_23_01 [Reducer 2], tasks=4, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:58,676 INFO [Task-Executor-2] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000048=23 2018-01-05T11:28:58,677 DEBUG [TezTR-518813_1_23_1_1_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_23_1_1_0 thread id: 1579 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:58,676 INFO [Task-Executor-3] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_23_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:58,677 INFO [Task-Executor-3] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_23_01_000000_0 2018-01-05T11:28:58,677 INFO [Task-Executor-3] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000046=28 2018-01-05T11:28:58,677 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_23_01 [Reducer 2], tasks=4, failed=0, killed=0, success=2, completed=2, commits=0, err=null 2018-01-05T11:28:58,677 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_23_01_000000_0: true 2018-01-05T11:28:58,677 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_23_01_000002_0: true 2018-01-05T11:28:58,677 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_23][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000048, stoppedTime=1515180538677, exitStatus=0 2018-01-05T11:28:58,677 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000048 exited with diagnostics set to Container completed. 2018-01-05T11:28:58,677 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_23][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000046, stoppedTime=1515180538677, exitStatus=0 2018-01-05T11:28:58,677 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_23_01_000001_0, isTaskDone=true 2018-01-05T11:28:58,677 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000046 exited with diagnostics set to Container completed. 2018-01-05T11:28:58,677 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000047, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_23_01_000001_0, eventCount=2 } 2018-01-05T11:28:58,677 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000048 2018-01-05T11:28:58,677 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000046 2018-01-05T11:28:58,677 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_23][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_23_01_000001_0, creationTime=1515180538643, allocationTime=1515180538645, startTime=1515180538647, finishTime=1515180538677, timeTaken=30, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:58,678 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_23_01_000001_0 2018-01-05T11:28:58,678 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_23_01_000001_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:58,678 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:28:58,678 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_23][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_23_01_000001, startTime=1515180538647, finishTime=1515180538678, timeTaken=31, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_23_01_000001_0, diagnostics=, counters=Counters: 34, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=0, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=0, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=0, NUM_SKIPPED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=0, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, SHUFFLE_BYTES=0, SHUFFLE_BYTES_DECOMPRESSED=0, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, HIVE, RECORDS_OUT_0=0, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0 2018-01-05T11:28:58,678 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_23_01_000001_0; false 2018-01-05T11:28:58,678 INFO [Task-Executor-1] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_23_01_000001_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:58,678 INFO [Task-Executor-1] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_23_01_000001_0 2018-01-05T11:28:58,678 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_23_01_000001_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:58,678 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_23_01 [Reducer 2], tasks=4, failed=0, killed=0, success=3, completed=3, commits=0, err=null 2018-01-05T11:28:58,678 INFO [Task-Executor-1] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000047=27 2018-01-05T11:28:58,678 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_23_01_000001_0: true 2018-01-05T11:28:58,678 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:58,678 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:58,678 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_23][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000047, stoppedTime=1515180538678, exitStatus=0 2018-01-05T11:28:58,678 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000047 exited with diagnostics set to Container completed. 2018-01-05T11:28:58,678 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000047 2018-01-05T11:28:58,687 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: merge complete for input vertex : Map_1 2018-01-05T11:28:58,687 INFO [TezTR-518813_1_23_1_3_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:28:58,687 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: Map_1: Shuffle Runner thread complete 2018-01-05T11:28:58,688 INFO [TezTR-518813_1_23_1_3_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:58,688 DEBUG [TezTR-518813_1_23_1_3_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:28:58,688 INFO [TezTR-518813_1_23_1_3_0] input.OrderedGroupedKVInput: Map 1: creating ValuesIterator with comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator, keyClass=org.apache.hadoop.hive.ql.io.HiveKey, valClass=org.apache.hadoop.io.BytesWritable 2018-01-05T11:28:58,688 DEBUG [TezTR-518813_1_23_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,688 DEBUG [TezTR-518813_1_23_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,688 DEBUG [TezTR-518813_1_23_1_3_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1, _col2, _col3, _col4] columnTypes=[struct, struct, struct, struct, struct] 2018-01-05T11:28:58,688 DEBUG [TezTR-518813_1_23_1_3_0] log.PerfLogger: 2018-01-05T11:28:58,688 INFO [TezTR-518813_1_23_1_3_0] exec.GroupByOperator: Initializing operator GBY[4] 2018-01-05T11:28:58,688 DEBUG [TezTR-518813_1_23_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,688 DEBUG [TezTR-518813_1_23_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,688 DEBUG [TezTR-518813_1_23_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,688 DEBUG [TezTR-518813_1_23_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,688 DEBUG [TezTR-518813_1_23_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,688 INFO [TezTR-518813_1_23_1_3_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:58,688 DEBUG [TezTR-518813_1_23_1_3_0] exec.GroupByOperator: Initialization Done 4 GBY 2018-01-05T11:28:58,688 DEBUG [TezTR-518813_1_23_1_3_0] exec.GroupByOperator: Operator 4 GBY initialized 2018-01-05T11:28:58,688 DEBUG [TezTR-518813_1_23_1_3_0] exec.GroupByOperator: Initializing children of 4 GBY 2018-01-05T11:28:58,688 DEBUG [TezTR-518813_1_23_1_3_0] exec.SelectOperator: Initializing child 5 SEL 2018-01-05T11:28:58,688 INFO [TezTR-518813_1_23_1_3_0] exec.SelectOperator: Initializing operator SEL[5] 2018-01-05T11:28:58,688 INFO [TezTR-518813_1_23_1_3_0] exec.SelectOperator: SELECT struct<_col0:string,_col1:struct,_col2:struct,_col3:struct,_col4:struct,_col5:struct> 2018-01-05T11:28:58,688 DEBUG [TezTR-518813_1_23_1_3_0] exec.SelectOperator: Initialization Done 5 SEL 2018-01-05T11:28:58,688 DEBUG [TezTR-518813_1_23_1_3_0] exec.SelectOperator: Operator 5 SEL initialized 2018-01-05T11:28:58,688 DEBUG [TezTR-518813_1_23_1_3_0] exec.SelectOperator: Initializing children of 5 SEL 2018-01-05T11:28:58,688 DEBUG [TezTR-518813_1_23_1_3_0] exec.FileSinkOperator: Initializing child 6 FS 2018-01-05T11:28:58,688 INFO [TezTR-518813_1_23_1_3_0] exec.FileSinkOperator: Initializing operator FS[6] 2018-01-05T11:28:58,688 DEBUG [TezTR-518813_1_23_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,688 INFO [TezTR-518813_1_23_1_3_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@c99a788]:[_col0, _col1, _col2, _col3, _col4, _col5]:[struct, struct, struct, struct, struct, string]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@38960822 2018-01-05T11:28:58,688 DEBUG [TezTR-518813_1_23_1_3_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:28:58,688 DEBUG [TezTR-518813_1_23_1_3_0] exec.FileSinkOperator: Initialization Done 6 FS 2018-01-05T11:28:58,688 DEBUG [TezTR-518813_1_23_1_3_0] exec.FileSinkOperator: Operator 6 FS initialized 2018-01-05T11:28:58,688 DEBUG [TezTR-518813_1_23_1_3_0] exec.FileSinkOperator: Initialization Done 6 FS done is reset. 2018-01-05T11:28:58,688 DEBUG [TezTR-518813_1_23_1_3_0] exec.SelectOperator: Initialization Done 5 SEL done is reset. 2018-01-05T11:28:58,688 DEBUG [TezTR-518813_1_23_1_3_0] exec.GroupByOperator: Initialization Done 4 GBY done is reset. 2018-01-05T11:28:58,689 INFO [TezTR-518813_1_23_1_3_0] tez.ReduceRecordProcessor: Id =4 Id =5 Id =6 <\Children> Id = 5 null<\Parent> <\FS> <\Children> Id = 4 null<\Parent> <\SEL> <\Children> <\Parent> <\GBY> 2018-01-05T11:28:58,689 DEBUG [TezTR-518813_1_23_1_3_0] log.PerfLogger: 2018-01-05T11:28:58,689 INFO [TezTR-518813_1_23_1_3_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 2 2018-01-05T11:28:58,689 DEBUG [TezTR-518813_1_23_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:28:58,689 DEBUG [TezTR-518813_1_23_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:28:58,689 DEBUG [TezTR-518813_1_23_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:28:58,689 DEBUG [TezTR-518813_1_23_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:28:58,689 DEBUG [TezTR-518813_1_23_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:28:58,689 DEBUG [TezTR-518813_1_23_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:28:58,689 DEBUG [TezTR-518813_1_23_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:28:58,689 DEBUG [TezTR-518813_1_23_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:28:58,689 DEBUG [TezTR-518813_1_23_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:28:58,690 DEBUG [TezTR-518813_1_23_1_3_0] log.PerfLogger: 2018-01-05T11:28:58,690 DEBUG [TezTR-518813_1_23_1_3_0] mr.ObjectCache: djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:28:58,690 DEBUG [TezTR-518813_1_23_1_3_0] exec.GroupByOperator: close called for operator GBY[4] 2018-01-05T11:28:58,690 INFO [TezTR-518813_1_23_1_3_0] exec.GroupByOperator: Closing operator GBY[4] 2018-01-05T11:28:58,690 INFO [TezTR-518813_1_23_1_3_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_292_1911154602830553799-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-58_292_1911154602830553799-1/_tmp.-ext-10002/000003_0 2018-01-05T11:28:58,690 INFO [TezTR-518813_1_23_1_3_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_292_1911154602830553799-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-58_292_1911154602830553799-1/_task_tmp.-ext-10002/_tmp.000003_0 2018-01-05T11:28:58,690 INFO [TezTR-518813_1_23_1_3_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_292_1911154602830553799-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-58_292_1911154602830553799-1/_tmp.-ext-10002/000003_0 2018-01-05T11:28:58,725 INFO [TezTR-518813_1_23_1_3_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:28:58,725 DEBUG [TezTR-518813_1_23_1_3_0] exec.GroupByOperator: Closing child = SEL[5] 2018-01-05T11:28:58,725 DEBUG [TezTR-518813_1_23_1_3_0] exec.SelectOperator: close called for operator SEL[5] 2018-01-05T11:28:58,725 DEBUG [TezTR-518813_1_23_1_3_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:58,725 INFO [TezTR-518813_1_23_1_3_0] exec.SelectOperator: Closing operator SEL[5] 2018-01-05T11:28:58,725 DEBUG [TezTR-518813_1_23_1_3_0] exec.SelectOperator: Closing child = FS[6] 2018-01-05T11:28:58,725 DEBUG [TezTR-518813_1_23_1_3_0] exec.FileSinkOperator: close called for operator FS[6] 2018-01-05T11:28:58,725 DEBUG [TezTR-518813_1_23_1_3_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:58,725 INFO [TezTR-518813_1_23_1_3_0] exec.FileSinkOperator: Closing operator FS[6] 2018-01-05T11:28:58,725 INFO [TezTR-518813_1_23_1_3_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:28:58,725 INFO [TezTR-518813_1_23_1_3_0] exec.FileSinkOperator: RECORDS_OUT_0:1, 2018-01-05T11:28:58,725 DEBUG [TezTR-518813_1_23_1_3_0] exec.FileSinkOperator: 6 Close done 2018-01-05T11:28:58,725 DEBUG [TezTR-518813_1_23_1_3_0] exec.SelectOperator: 5 Close done 2018-01-05T11:28:58,725 DEBUG [TezTR-518813_1_23_1_3_0] exec.GroupByOperator: 4 Close done 2018-01-05T11:28:58,725 INFO [TezTR-518813_1_23_1_3_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_23_01_000003_0 2018-01-05T11:28:58,725 INFO [TezTR-518813_1_23_1_3_0] orderedgrouped.Shuffle: Shutting down Shuffle for source: Map_1 2018-01-05T11:28:58,725 INFO [TezTR-518813_1_23_1_3_0] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:28:58,725 INFO [TezTR-518813_1_23_1_3_0] output.MROutput: out_Reducer 2 closed 2018-01-05T11:28:58,725 DEBUG [TezTR-518813_1_23_1_3_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:58,725 INFO [TezTR-518813_1_23_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:58,726 INFO [TezTR-518813_1_23_1_3_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_23_01_000003_0, askedToStop=false 2018-01-05T11:28:58,726 INFO [TezTR-518813_1_23_1_3_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_23_01_000003_0, stopRequested=false 2018-01-05T11:28:58,726 INFO [TezTR-518813_1_23_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_23_01_000003_0: Counters: 30 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=1, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=186, ADDITIONAL_SPILLS_BYTES_READ=186, SHUFFLE_BYTES=186, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=186, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=15, MERGE_PHASE_TIME=30, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][HIVE CREATED_FILES=1, RECORDS_OUT_0=1][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0]] 2018-01-05T11:28:58,726 DEBUG [TezTR-518813_1_23_1_3_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_23_1_3_0 thread id: 1567 scheme: file bytesRead: 0 bytesWritten: 306 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:58,726 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_23_01_000003_0, isTaskDone=true 2018-01-05T11:28:58,726 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000045, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_23_01_000003_0, eventCount=2 } 2018-01-05T11:28:58,726 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_23][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_23_01_000003_0, creationTime=1515180538643, allocationTime=1515180538644, startTime=1515180538644, finishTime=1515180538726, timeTaken=82, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:58,727 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:28:58,727 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_23_01_000003_0 2018-01-05T11:28:58,727 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_23_01_000003_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:58,727 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_23_01_000003_0 2018-01-05T11:28:58,727 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_23_01_000003_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:58,727 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_23_01_000003_0; false 2018-01-05T11:28:58,727 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_23_01_000003_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:58,727 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000045=81 2018-01-05T11:28:58,727 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:58,727 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_23_01_000003_0: true 2018-01-05T11:28:58,727 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:58,727 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_23][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_23_01_000003, startTime=1515180538644, finishTime=1515180538727, timeTaken=83, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_23_01_000003_0, diagnostics=, counters=Counters: 35, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=306, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=1, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=186, ADDITIONAL_SPILLS_BYTES_READ=186, SHUFFLE_BYTES=186, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=186, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=15, MERGE_PHASE_TIME=30, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0 2018-01-05T11:28:58,727 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_23_01 [Reducer 2], tasks=4, failed=0, killed=0, success=4, completed=4, commits=0, err=null 2018-01-05T11:28:58,727 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_23_01 [Reducer 2] 2018-01-05T11:28:58,728 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_23][Event:VERTEX_FINISHED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_23_01, initRequestedTime=1515180538523, initedTime=1515180538529, startRequestedTime=1515180538547, startedTime=1515180538547, finishTime=1515180538727, timeTaken=180, status=SUCCEEDED, diagnostics=, counters=Counters: 35, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=306, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=1, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=3, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=186, ADDITIONAL_SPILLS_BYTES_READ=186, SHUFFLE_BYTES=186, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=186, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=33, MERGE_PHASE_TIME=50, FIRST_EVENT_RECEIVED=18, LAST_EVENT_RECEIVED=18, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, vertexStats=firstTaskStartTime=1515180538644, firstTasksToStart=[ task_1515180518813_0001_23_01_000003 ], lastTaskFinishTime=1515180538726, lastTasksToFinish=[ task_1515180518813_0001_23_01_000003 ], minTaskDuration=28, maxTaskDuration=82, avgTaskDuration=42.5, numSuccessfulTasks=4, shortestDurationTasks=[ task_1515180518813_0001_23_01_000002 ], longestDurationTasks=[ task_1515180518813_0001_23_01_000003 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=4, numSucceededTasks=4, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:58,728 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_23_01 [Reducer 2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:58,728 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_23][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000045, stoppedTime=1515180538728, exitStatus=0 2018-01-05T11:28:58,728 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000045 exited with diagnostics set to Container completed. 2018-01-05T11:28:58,728 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_23_01 [Reducer 2] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2 2018-01-05T11:28:58,728 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null 2018-01-05T11:28:58,728 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_23_00 [Map 1] 2018-01-05T11:28:58,728 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_23_01 [Reducer 2] 2018-01-05T11:28:58,729 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_23, queueSize=0 2018-01-05T11:28:58,729 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_23][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_23, startTime=1515180538515, finishTime=1515180538728, timeTaken=213, status=SUCCEEDED, diagnostics=, counters=Counters: 51, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=5, TOTAL_LAUNCHED_TASKS=5, OTHER_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=763, AM_GC_TIME_MILLIS=0, File System Counters, FILE_BYTES_READ=506, FILE_BYTES_WRITTEN=687, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=3, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=20, INPUT_SPLIT_LENGTH_BYTES=282, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=173, OUTPUT_BYTES_WITH_OVERHEAD=182, OUTPUT_BYTES_PHYSICAL=186, ADDITIONAL_SPILLS_BYTES_WRITTEN=186, ADDITIONAL_SPILLS_BYTES_READ=186, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=186, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=186, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=33, MERGE_PHASE_TIME=50, FIRST_EVENT_RECEIVED=18, LAST_EVENT_RECEIVED=18, HIVE, CREATED_FILES=1, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=20, RECORDS_OUT_0=1, RECORDS_OUT_INTERMEDIATE_Map_1=1, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0 2018-01-05T11:28:58,729 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_23 finished with state: SUCCEEDED 2018-01-05T11:28:58,729 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_23 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:28:58,729 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000045 2018-01-05T11:28:58,730 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_23, dagState=SUCCEEDED 2018-01-05T11:28:58,730 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_23_post 2018-01-05T11:28:58,730 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:58,730 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:28:58,730 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:28:58,730 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 23 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=5, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=5,NumRequestsWithoutLocation=5, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=5} 2018-01-05T11:28:58,730 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:28:58,730 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=analyze table cbo_t1 compute sta...c_boolean (Stage-0), with id=dag_1515180518813_0001_23 2018-01-05T11:28:58,730 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 23, to 10.22.8.182:49818 2018-01-05T11:28:58,730 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=5, Remaining=0 2018-01-05T11:28:58,730 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=analyze table cbo_t1 compute sta...c_boolean (Stage-0), with id=dag_1515180518813_0001_23 2018-01-05T11:28:58,730 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=23} 2018-01-05T11:28:58,730 INFO [IPC Server handler 4 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=23}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:28:58,730 INFO [IPC Server handler 4 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/23 after 300 seconds 2018-01-05T11:28:58,730 ERROR [IPC Server handler 4 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89, dag_1515180518813_0001_23 2018-01-05T11:28:58,730 INFO [IPC Server handler 4 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89 2018-01-05T11:28:58,730 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=23} = 4 2018-01-05T11:28:58,730 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Issuing killFragment for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=23} attempt_1515180518813_0001_23_01_000000_0 2018-01-05T11:28:58,730 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Removing attempt_1515180518813_0001_23_01_000000_0 from preemptionQueue 2018-01-05T11:28:58,730 INFO [IPC Server handler 4 on 49818] impl.TaskRunnerCallable: Ignoring kill request for task attempt_1515180518813_0001_23_01_000000_0 since it's already complete 2018-01-05T11:28:58,730 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Issuing killFragment for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=23} attempt_1515180518813_0001_23_01_000001_0 2018-01-05T11:28:58,730 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Removing attempt_1515180518813_0001_23_01_000001_0 from preemptionQueue 2018-01-05T11:28:58,730 INFO [IPC Server handler 4 on 49818] impl.TaskRunnerCallable: Ignoring kill request for task attempt_1515180518813_0001_23_01_000001_0 since it's already complete 2018-01-05T11:28:58,730 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Issuing killFragment for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=23} attempt_1515180518813_0001_23_01_000002_0 2018-01-05T11:28:58,730 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Removing attempt_1515180518813_0001_23_01_000002_0 from preemptionQueue 2018-01-05T11:28:58,730 INFO [IPC Server handler 4 on 49818] impl.TaskRunnerCallable: Ignoring kill request for task attempt_1515180518813_0001_23_01_000002_0 since it's already complete 2018-01-05T11:28:58,730 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Issuing killFragment for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=23} attempt_1515180518813_0001_23_01_000003_0 2018-01-05T11:28:58,730 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Removing attempt_1515180518813_0001_23_01_000003_0 from preemptionQueue 2018-01-05T11:28:58,730 INFO [IPC Server handler 4 on 49818] impl.TaskRunnerCallable: Ignoring kill request for task attempt_1515180518813_0001_23_01_000003_0 since it's already complete 2018-01-05T11:28:58,730 DEBUG [IPC Server handler 4 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=23} 2018-01-05T11:28:58,730 DEBUG [IPC Server handler 4 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:28:58,730 DEBUG [IPC Server handler 4 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=23} 2018-01-05T11:28:58,731 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,731 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,731 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,731 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,731 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 Reducer 2: 4/4 2018-01-05T11:28:58,731 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,733 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,733 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: TaskId for 000003_0 = 000003 2018-01-05T11:28:58,733 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,733 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,733 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,734 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-2:STATS] in serial mode 2018-01-05T11:28:58,734 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@6b8abde9, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:58,734 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:58,734 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,734 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:58,734 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:58,734 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:58,734 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:58,734 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,734 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,734 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:58,734 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:58,734 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,734 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:58,734 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,734 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:58,734 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:58,734 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,734 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:58,734 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_292_1911154602830553799-1/-mr-10001 2018-01-05T11:28:58,734 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_292_1911154602830553799-1/-mr-10001 2018-01-05T11:28:58,735 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Getting file stats of all partitions. threadpool size:15 2018-01-05T11:28:58,735 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Finished getting file stats of all partitions! 2018-01-05T11:28:58,735 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.cbo_t1/dt=2014/ numRows 2018-01-05T11:28:58,735 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.cbo_t1/dt=2014/ numRows 20 2018-01-05T11:28:58,735 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.cbo_t1/dt=2014/ rawDataSize 2018-01-05T11:28:58,735 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.cbo_t1/dt=2014/ rawDataSize 262 2018-01-05T11:28:58,735 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Partition {dt=2014} stats: [numFiles=1, numRows=20, totalSize=282, rawDataSize=262] 2018-01-05T11:28:58,736 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,736 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:58,736 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:58,736 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:58,736 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:58,737 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,737 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,737 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,737 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,738 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,738 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:58,738 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:58,738 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,738 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:58,738 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,738 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_partitions : db=default tbl=cbo_t1 2018-01-05T11:28:58,738 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_partitions : db=default tbl=cbo_t1 2018-01-05T11:28:58,738 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: New partition values:[2014] 2018-01-05T11:28:58,740 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,743 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000044, QueryId=djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89-23, DagId=23, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180538566, EndTime=1515180538743 2018-01-05T11:28:58,743 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_23_01_000000_0 2018-01-05T11:28:58,743 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_23_01_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=1, preemptionQueueSize=0 2018-01-05T11:28:58,743 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_23_01_000000_0 2018-01-05T11:28:58,743 INFO [ExecutionCompletionThread #0] impl.QueryTracker: Ignoring fragmentComplete message for unknown query: QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=23} 2018-01-05T11:28:58,743 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000046, QueryId=djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89-23, DagId=23, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-3, Succeeded=true, StartTime=1515180538648, EndTime=1515180538743 2018-01-05T11:28:58,743 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_23_01_000002_0 2018-01-05T11:28:58,744 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_23_01_000002_0 complete. WaitQueueSize=0, numSlotsAvailable=2, preemptionQueueSize=0 2018-01-05T11:28:58,744 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_23_01_000002_0 2018-01-05T11:28:58,744 INFO [ExecutionCompletionThread #0] impl.QueryTracker: Ignoring fragmentComplete message for unknown query: QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=23} 2018-01-05T11:28:58,744 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000048, QueryId=djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89-23, DagId=23, VertexName=Reducer 2, TaskId=2, TaskAttemptId=0, ThreadName=Task-Executor-2, Succeeded=true, StartTime=1515180538653, EndTime=1515180538744 2018-01-05T11:28:58,744 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_23_01_000001_0 2018-01-05T11:28:58,744 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_23_01_000001_0 complete. WaitQueueSize=0, numSlotsAvailable=3, preemptionQueueSize=0 2018-01-05T11:28:58,744 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_23_01_000001_0 2018-01-05T11:28:58,744 INFO [ExecutionCompletionThread #0] impl.QueryTracker: Ignoring fragmentComplete message for unknown query: QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=23} 2018-01-05T11:28:58,744 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000047, QueryId=djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89-23, DagId=23, VertexName=Reducer 2, TaskId=1, TaskAttemptId=0, ThreadName=Task-Executor-1, Succeeded=true, StartTime=1515180538650, EndTime=1515180538744 2018-01-05T11:28:58,744 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_23_01_000003_0 2018-01-05T11:28:58,744 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_23_01_000003_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:58,744 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_23_01_000003_0 2018-01-05T11:28:58,744 INFO [ExecutionCompletionThread #0] impl.QueryTracker: Ignoring fragmentComplete message for unknown query: QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=23} 2018-01-05T11:28:58,745 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,747 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,748 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,748 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: About to delete stats tmp dir :file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_292_1911154602830553799-1/-mr-10001 2018-01-05T11:28:58,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Time taken to get FileStatuses: 15 2018-01-05T11:28:58,764 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:28:58,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 15 2018-01-05T11:28:58,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FetchOperator: Creating fetchTask with deserializer typeinfo: struct<_col0:struct,_col1:struct,_col2:struct,_col3:struct,_col4:struct,_col5:string> 2018-01-05T11:28:58,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FetchOperator: deserializer properties: table properties: {columns=_col0,_col1,_col2,_col3,_col4,_col5, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct:struct:struct:struct:struct:string, escape.delim=\} partition properties: {columns=_col0,_col1,_col2,_col3,_col4,_col5, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct:struct:struct:struct:struct:string, escape.delim=\} 2018-01-05T11:28:58,765 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,768 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,768 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: write_partition_column_statistics: db=default table=cbo_t1 part=dt=2014 2018-01-05T11:28:58,768 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=write_partition_column_statistics: db=default table=cbo_t1 part=dt=2014 2018-01-05T11:28:58,777 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating partition level column statistics for db=default tableName=cbo_t1 partName=dt=2014 colName=key 2018-01-05T11:28:58,777 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating partition level column statistics for db=default tableName=cbo_t1 partName=dt=2014 colName=value 2018-01-05T11:28:58,780 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating partition level column statistics for db=default tableName=cbo_t1 partName=dt=2014 colName=c_int 2018-01-05T11:28:58,780 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,781 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating partition level column statistics for db=default tableName=cbo_t1 partName=dt=2014 colName=c_float 2018-01-05T11:28:58,781 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating partition level column statistics for db=default tableName=cbo_t1 partName=dt=2014 colName=c_boolean 2018-01-05T11:28:58,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,786 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,786 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,786 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,786 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: analyze table cbo_t1 compute statistics for columns key, value, c_int, c_float, c_boolean 2018-01-05T11:28:58,786 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:28:58,786 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@cbo_t1 2018-01-05T11:28:58,786 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@cbo_t1@dt=2014 2018-01-05T11:28:58,786 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@cbo_t1 2018-01-05T11:28:58,786 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@cbo_t1@dt=2014 2018-01-05T11:28:58,786 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_292_1911154602830553799-1/-mr-10000 2018-01-05T11:28:58,786 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,786 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,786 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:58,786 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_partitions_(String, String, List, EnvironmentContext, )=10, setPartitionColumnStatistics_(SetPartitionsStatsRequest, )=21} 2018-01-05T11:28:58,787 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89); Time taken: 0.403 seconds 2018-01-05T11:28:58,787 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:58,787 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,787 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:58,787 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,787 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query analyze table cbo_t1 compute statistics for columns key, value, c_int, c_float, c_boolean 2018-01-05T11:28:58,787 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_288_8305321905597716466-1 2018-01-05T11:28:58,788 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.499 seconds 2018-01-05T11:28:58,788 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:58,788 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:58,788 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:58,788 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:58,788 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,788 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:58,788 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,788 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,788 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,788 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: analyze table cbo_t2 partition (dt) compute statistics 2018-01-05T11:28:58,788 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112858_b060ea84-873c-4c33-97d6-51b6d7348b47): analyze table cbo_t2 partition (dt) compute statistics 2018-01-05T11:28:58,788 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,788 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: analyze table cbo_t2 partition (dt) compute statistics 2018-01-05T11:28:58,788 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:58,788 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,788 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,788 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,788 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,788 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:58,788 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Invoking analyze on original query 2018-01-05T11:28:58,788 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Starting Semantic Analysis 2018-01-05T11:28:58,788 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed phase 1 of Semantic Analysis 2018-01-05T11:28:58,789 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for source tables 2018-01-05T11:28:58,789 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,789 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t2 2018-01-05T11:28:58,789 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t2 2018-01-05T11:28:58,791 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,791 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,792 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,792 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t2 2018-01-05T11:28:58,792 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t2 2018-01-05T11:28:58,795 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,795 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,795 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.BaseSemanticAnalyzer: Dynamic partitioning is used; only validating 0 columns 2018-01-05T11:28:58,795 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,795 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t2 2018-01-05T11:28:58,795 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t2 2018-01-05T11:28:58,798 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,798 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,798 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,798 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partitions_names_ps : db=default tbl=cbo_t2[] 2018-01-05T11:28:58,798 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions_names_ps : db=default tbl=cbo_t2[] 2018-01-05T11:28:58,802 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,806 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,806 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,806 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,806 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partitions_by_names : db=default tbl=cbo_t2 2018-01-05T11:28:58,806 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions_by_names : db=default tbl=cbo_t2 2018-01-05T11:28:58,809 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,810 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.205015ms + 0.004905ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? where "PARTITIONS"."PART_NAME" in (?)] 2018-01-05T11:28:58,814 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 3.670361ms + 0.079444ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (11) order by "PART_NAME" asc] 2018-01-05T11:28:58,817 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 3.006125ms + 0.072556ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (11) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:28:58,819 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 1.706574ms + 0.034916ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (11) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:58,820 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 1.884313ms + 0.013437ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (32) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:28:58,822 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 1.657182ms + 0.013411ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (32) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:58,824 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 1.795079ms + 0.015285ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (32) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:58,826 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 1.741024ms + 0.014862ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (32) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:58,828 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 1.861684ms + 0.040891ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (20) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:58,830 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 1.883766ms + 0.046429ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (32) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:28:58,830 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,848 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000045, QueryId=djaiswal_20180105112858_b12256da-e599-4890-a470-edae04e7cc89-23, DagId=23, VertexName=Reducer 2, TaskId=3, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180538646, EndTime=1515180538848 2018-01-05T11:28:58,849 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,850 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,850 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for subqueries 2018-01-05T11:28:58,850 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for destination tables 2018-01-05T11:28:58,850 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed getting MetaData in Semantic Analysis 2018-01-05T11:28:58,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_788_1004233992122089697-1/-mr-10000 2018-01-05T11:28:58,862 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: XXX: adding part: cbo_t2(dt=2014) 2018-01-05T11:28:58,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Table Plan for cbo_t2 TS[0] 2018-01-05T11:28:58,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Body Plan for Query Block null 2018-01-05T11:28:58,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Plan for Query Block null 2018-01-05T11:28:58,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Before logical optimization TS[0] 2018-01-05T11:28:58,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] lineage.Generator: Time taken for lineage transform=0 2018-01-05T11:28:58,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{cbo_t2}key,value: string|{cbo_t2}value,c_int: int|{cbo_t2}c_int,c_float: float|{cbo_t2}c_float,c_boolean: boolean|{cbo_t2}c_boolean,dt: string|{cbo_t2}dt,BLOCK__OFFSET__INSIDE__FILE: bigint|{cbo_t2}block__offset__inside__file,INPUT__FILE__NAME: string|{cbo_t2}input__file__name,ROW__ID: struct|{cbo_t2}row__id,RAW__DATA__SIZE: bigint|{cbo_t2}raw__data__size) 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.PredicatePushDown: After PPD: TS[0] 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{cbo_t2}key,value: string|{cbo_t2}value,c_int: int|{cbo_t2}c_int,c_float: float|{cbo_t2}c_float,c_boolean: boolean|{cbo_t2}c_boolean,dt: string|{cbo_t2}dt,BLOCK__OFFSET__INSIDE__FILE: bigint|{cbo_t2}block__offset__inside__file,INPUT__FILE__NAME: string|{cbo_t2}input__file__name,ROW__ID: struct|{cbo_t2}row__id,RAW__DATA__SIZE: bigint|{cbo_t2}raw__data__size) 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{cbo_t2}key,value: string|{cbo_t2}value,c_int: int|{cbo_t2}c_int,c_float: float|{cbo_t2}c_float,c_boolean: boolean|{cbo_t2}c_boolean,dt: string|{cbo_t2}dt,BLOCK__OFFSET__INSIDE__FILE: bigint|{cbo_t2}block__offset__inside__file,INPUT__FILE__NAME: string|{cbo_t2}input__file__name,ROW__ID: struct|{cbo_t2}row__id,RAW__DATA__SIZE: bigint|{cbo_t2}raw__data__size) 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: After logical optimization TS[0] 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,863 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,863 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partitions : db=default tbl=cbo_t2 2018-01-05T11:28:58,863 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions : db=default tbl=cbo_t2 2018-01-05T11:28:58,866 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,867 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.198353ms + 0.0055ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? ] 2018-01-05T11:28:58,867 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.147158ms + 0.046861ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (11) order by "PART_NAME" asc] 2018-01-05T11:28:58,867 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.086874ms + 0.045771ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (11) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:28:58,867 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.072901ms + 0.018149ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (11) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:58,867 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.065786ms + 0.006592ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (32) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:28:58,868 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.060894ms + 0.006082ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (32) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:58,868 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.05871ms + 0.005998ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (32) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:58,868 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.054318ms + 0.00561ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (32) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:58,868 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.090762ms + 0.047506ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (20) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:58,868 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.068623ms + 0.027272ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (32) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:28:58,868 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,870 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting stats (Num rows: 1 Data size: 2828 Basic stats: PARTIAL Column stats: NONE) on TS[0] 2018-01-05T11:28:58,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (cbo_t2): numRows: 1 dataSize: 2828 basicStatsState: PARTIAL colStatsState: NONE colStats: {} 2018-01-05T11:28:58,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on TS[0] 2018-01-05T11:28:58,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:58,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:28:58,871 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Cycle free: true 2018-01-05T11:28:58,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,872 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,872 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{cbo_t2}key,value: string|{cbo_t2}value,c_int: int|{cbo_t2}c_int,c_float: float|{cbo_t2}c_float,c_boolean: boolean|{cbo_t2}c_boolean,dt: string|{cbo_t2}dt,BLOCK__OFFSET__INSIDE__FILE: bigint|{cbo_t2}block__offset__inside__file,INPUT__FILE__NAME: string|{cbo_t2}input__file__name,ROW__ID: struct|{cbo_t2}row__id,RAW__DATA__SIZE: bigint|{cbo_t2}raw__data__size) 2018-01-05T11:28:58,872 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,872 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:28:58,872 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2/dt=2014 of table cbo_t2 2018-01-05T11:28:58,872 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2/dt=2014 2018-01-05T11:28:58,872 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:28:58,872 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,872 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:58,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:58,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:58,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:58,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:58,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:58,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:28:58,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping vectorization 2018-01-05T11:28:58,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:28:58,873 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: llap mode: only 2018-01-05T11:28:58,873 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:28:58,873 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:58,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@3bb4ade3 can be done in LLAP 2018-01-05T11:28:58,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,873 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed plan generation 2018-01-05T11:28:58,873 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:58,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validation start 2018-01-05T11:28:58,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Not a partition. 2018-01-05T11:28:58,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validated dt=2014 2018-01-05T11:28:58,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: cbo_t2 2018-01-05T11:28:58,873 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,873 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partitions_ps_with_auth : db=default tbl=cbo_t2[2014] 2018-01-05T11:28:58,873 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions_ps_with_auth : db=default tbl=cbo_t2[2014] 2018-01-05T11:28:58,876 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,883 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,884 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,884 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,884 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=cbo_t2[2014] 2018-01-05T11:28:58,884 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=cbo_t2[2014] 2018-01-05T11:28:58,888 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,892 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:58,892 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:58,892 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,892 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:cbo_t2.key, type:string, comment:null), FieldSchema(name:cbo_t2.value, type:string, comment:null), FieldSchema(name:cbo_t2.c_int, type:int, comment:null), FieldSchema(name:cbo_t2.c_float, type:float, comment:null), FieldSchema(name:cbo_t2.c_boolean, type:boolean, comment:null), FieldSchema(name:cbo_t2.dt, type:string, comment:null)], properties:null) 2018-01-05T11:28:58,892 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,892 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:58,892 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, listPartitionNames_(String, String, List, short, )=8, getTable_(String, String, )=9, flushCache_()=0, getPartitionsByNames_(String, String, List, )=44, getPartitionWithAuthInfo_(String, String, List, String, List, )=8, listPartitions_(String, String, short, )=8, listPartitionsWithAuthInfo_(String, String, List, short, String, List, )=11} 2018-01-05T11:28:58,892 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112858_b060ea84-873c-4c33-97d6-51b6d7348b47); Time taken: 0.104 seconds 2018-01-05T11:28:58,893 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:58,893 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,893 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112858_b060ea84-873c-4c33-97d6-51b6d7348b47): analyze table cbo_t2 partition (dt) compute statistics 2018-01-05T11:28:58,894 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,894 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: analyze table cbo_t2 partition (dt) compute statistics 2018-01-05T11:28:58,894 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:28:58,894 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@cbo_t2 2018-01-05T11:28:58,894 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@cbo_t2@dt=2014 2018-01-05T11:28:58,894 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@cbo_t2 2018-01-05T11:28:58,894 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@cbo_t2@dt=2014 2018-01-05T11:28:58,894 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,894 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,894 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,894 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Query ID = djaiswal_20180105112858_b060ea84-873c-4c33-97d6-51b6d7348b47 2018-01-05T11:28:58,894 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Total jobs = 1 2018-01-05T11:28:58,894 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,894 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:28:58,894 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MAPRED] in serial mode 2018-01-05T11:28:58,905 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_788_1004233992122089697-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:28:58,927 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:28:58,927 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:28:58,927 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:28:58,927 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112858_b060ea84-873c-4c33-97d6-51b6d7348b47 2018-01-05T11:28:58,927 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Session is already open 2018-01-05T11:28:58,927 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:28:58,927 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,927 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Dag name: analyze table cbo_t2 partition ...statistics (Stage-0) 2018-01-05T11:28:58,927 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: DagInfo: {"context":"Hive","description":"\nanalyze table cbo_t2 partition (dt) compute statistics"} 2018-01-05T11:28:58,927 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112858_b060ea84-873c-4c33-97d6-51b6d7348b47 with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:28:58,927 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,939 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:28:58,939 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,939 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:28:58,939 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,939 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 2.00KB 2018-01-05T11:28:58,944 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_788_1004233992122089697-1/-mr-10000 2018-01-05T11:28:58,956 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_788_1004233992122089697-1/-mr-10000 2018-01-05T11:28:58,958 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2/dt=2014 2018-01-05T11:28:58,958 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,958 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,958 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,958 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=analyze table cbo_t2 partition ...statistics (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112858_b060ea84-873c-4c33-97d6-51b6d7348b47 } 2018-01-05T11:28:58,959 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:58,961 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: analyze table cbo_t2 partition ...statistics (Stage-0) 2018-01-05T11:28:58,961 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:28:58,961 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_24, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_24.dot 2018-01-05T11:28:58,962 INFO [IPC Server handler 1 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_24 2018-01-05T11:28:58,962 WARN [IPC Server handler 1 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:58,962 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Running DAG: analyze table cbo_t2 partition ...statistics (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112858_b060ea84-873c-4c33-97d6-51b6d7348b47 } 2018-01-05T11:28:58,986 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_24][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_24, submitTime=1515180538961, queueName=null 2018-01-05T11:28:58,987 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_24_00 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_24_00 [Map 1], Default Resources= 2018-01-05T11:28:58,987 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_24_00 [Map 1] 2018-01-05T11:28:58,987 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_24_00 [Map 1] 2018-01-05T11:28:58,987 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:28:58,987 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_24_00 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:58,987 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:28:58,987 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_24][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_24, initTime=1515180538987 2018-01-05T11:28:58,987 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: dag_1515180518813_0001_24 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:28:58,988 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:28:58,988 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_24][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_24, startTime=1515180538988 2018-01-05T11:28:58,988 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_24 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:28:58,988 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {cbo_t2={InputName=cbo_t2}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:28:58,988 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: cbo_t2, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:28:58,988 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_24_00 [Map 1] 2018-01-05T11:28:58,988 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_24, dagName=analyze table cbo_t2 partition ...statistics (Stage-0) 2018-01-05T11:28:58,988 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,988 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,988 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:58,995 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_24_00 [Map 1] 2018-01-05T11:28:58,995 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_24_00 [Map 1] 2018-01-05T11:28:58,995 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:58,995 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_24_00 [Map 1] 2018-01-05T11:28:58,997 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:28:58,997 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@242fcf13 2018-01-05T11:28:58,997 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_788_1004233992122089697-1/djaiswal/_tez_scratch_dir/2055c940-e0a2-4579-965a-4af004ae1182/map.xml 2018-01-05T11:28:58,997 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_788_1004233992122089697-1/djaiswal/_tez_scratch_dir/2055c940-e0a2-4579-965a-4af004ae1182/map.xml 2018-01-05T11:28:58,997 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_788_1004233992122089697-1/djaiswal/_tez_scratch_dir/2055c940-e0a2-4579-965a-4af004ae1182/map.xml 2018-01-05T11:28:58,997 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:58,997 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:58,997 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:58,997 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:58,997 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 2.00KB 2018-01-05T11:28:58,998 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_24_00 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:28:58,998 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: cbo_t2 on vertex vertex_1515180518813_0001_24_00 [Map 1] 2018-01-05T11:28:58,998 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:28:58,998 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:58,998 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:28:58,998 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:58,998 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_788_1004233992122089697-1/djaiswal/_tez_scratch_dir/2055c940-e0a2-4579-965a-4af004ae1182/map.xml 2018-01-05T11:28:58,998 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:58,998 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias cbo_t2 2018-01-05T11:28:58,998 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2/dt=2014 2018-01-05T11:28:58,998 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2/dt=2014 2018-01-05T11:28:58,998 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 2018-01-05T11:28:58,998 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = 2018-01-05T11:28:58,998 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2/dt=2014 2018-01-05T11:28:58,998 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:58,998 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:58,998 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:58,998 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:59,013 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Time taken to get FileStatuses: 14 2018-01-05T11:28:59,013 INFO [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:28:59,013 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 14 2018-01-05T11:28:59,013 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 1 2018-01-05T11:28:59,013 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:59,013 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 1. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:28:59,013 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_788_1004233992122089697-1/djaiswal/_tez_scratch_dir/2055c940-e0a2-4579-965a-4af004ae1182/map.xml 2018-01-05T11:28:59,013 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_788_1004233992122089697-1/djaiswal/_tez_scratch_dir/2055c940-e0a2-4579-965a-4af004ae1182/map.xml 2018-01-05T11:28:59,013 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_788_1004233992122089697-1/djaiswal/_tez_scratch_dir/2055c940-e0a2-4579-965a-4af004ae1182/map.xml 2018-01-05T11:28:59,014 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:59,014 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:59,014 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,014 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:59,014 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 2.00KB 2018-01-05T11:28:59,014 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2/dt=2014/cbo_t2.txt to src new group? true 2018-01-05T11:28:59,014 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:28:59,014 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:28:59,014 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:28:59,014 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 5 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 282 Original splits: 1 2018-01-05T11:28:59,014 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Using original number of splits: 1 desired splits: 1 2018-01-05T11:28:59,014 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 1 grouped split count is 1, for bucket: 1 2018-01-05T11:28:59,014 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:28:59,014 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: cbo_t2 on vertex vertex_1515180518813_0001_24_00 [Map 1] 2018-01-05T11:28:59,015 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_24_00 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:59,015 INFO [App Shared Pool - #0] impl.VertexImpl: Got updated RootInputsSpecs: {cbo_t2=forAllWorkUnits=true, update=[1]} 2018-01-05T11:28:59,016 INFO [App Shared Pool - #0] impl.VertexImpl: Vertex vertex_1515180518813_0001_24_00 [Map 1] parallelism set to 1 2018-01-05T11:28:59,016 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_24_00 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:59,016 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_24_00 [Map 1], numAdditionalOutputs=1 2018-01-05T11:28:59,016 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_24][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_24_00, initRequestedTime=1515180538988, initedTime=1515180539016, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:59,016 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_24_00 [Map 1] with distanceFromRoot: 0 2018-01-05T11:28:59,016 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_24_00 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:28:59,016 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_24][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_24_00, startRequestedTime=1515180538998, startedTime=1515180539016 2018-01-05T11:28:59,016 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_24][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_24_00, reconfigureDoneTime=1515180539016, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:28:59,016 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_24_00 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:59,016 INFO [App Shared Pool - #0] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:28:59,016 INFO [App Shared Pool - #0] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:28:59,016 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_24_00 [Map 1] 2018-01-05T11:28:59,016 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_24][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_24_00_000000, scheduledTime=1515180539016, launchTime=1515180539016 2018-01-05T11:28:59,016 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_24_00 [Map 1] 2018-01-05T11:28:59,017 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_24_00_000000_0, priority=2, capability=, hosts=null 2018-01-05T11:28:59,017 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=2,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:59,017 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=2,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:59,017 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:59,017 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_24_00_000000_0 on hosts=null 2018-01-05T11:28:59,017 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_24_00_000000_0 on hosts=null 2018-01-05T11:28:59,017 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_24_00_000000_0 2018-01-05T11:28:59,017 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:59,017 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #49, task=TaskInfo{task=attempt_1515180518813_0001_24_00_000000_0, priority=2, startTime=0, containerId=null, uniqueId=48, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000049 2018-01-05T11:28:59,017 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_24_00_000000_0; false 2018-01-05T11:28:59,017 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_24_00_000000_0, priority=2, startTime=311559526, containerId=container_111101111_0001_00_000049assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=48, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:59,017 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000049 for node: 10.22.8.182:49818 2018-01-05T11:28:59,019 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 24, name=analyze table cbo_t2 partition ...statistics (Stage-0), queryId=djaiswal_20180105112858_b060ea84-873c-4c33-97d6-51b6d7348b47 2018-01-05T11:28:59,019 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000049, attempt_1515180518813_0001_24_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:59,019 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000049 for node: 10.22.8.182:49818 2018-01-05T11:28:59,019 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000049 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:59,019 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000049, launchTime=1515180539019 2018-01-05T11:28:59,020 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_24_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112858_b060ea84-873c-4c33-97d6-51b6d7348b47, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000049, dagName=analyze table cbo_t2 partition ...statistics (Stage-0), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{cbo_t2,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{out_Map 1,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180538988, firstAttemptStartTime=1515180539016, currentAttemptStartTime=1515180539019} 2018-01-05T11:28:59,020 INFO [IPC Server handler 3 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000049, QueryId=djaiswal_20180105112858_b060ea84-873c-4c33-97d6-51b6d7348b47-24, DagId=24, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180539020 2018-01-05T11:28:59,020 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_24_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000049] on NM: [10.22.8.182:49818] 2018-01-05T11:28:59,020 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:59,020 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:59,020 DEBUG [IPC Server handler 3 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=24} with the ShuffleHandler 2018-01-05T11:28:59,020 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_24][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_24_00_000000_0, startTime=1515180539020, containerId=container_111101111_0001_00_000049, nodeId=10.22.8.182:49818 2018-01-05T11:28:59,032 INFO [IPC Server handler 3 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:28:59,032 DEBUG [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/24] 2018-01-05T11:28:59,032 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:59,032 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_24_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:59,032 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_24_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180539016, dagStartTime=1515180538988, withinDagPriority=2, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:28:59,032 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_24_00_000000_0 : ACCEPTED 2018-01-05T11:28:59,032 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_24_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:59,032 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_24_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180539016, dagStartTime=1515180538988, withinDagPriority=2, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:28:59,032 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_24_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:59,032 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_24_00_000000_0 to pre-emption queue 2018-01-05T11:28:59,032 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_24_00_000000_0: true 2018-01-05T11:28:59,033 INFO [TaskCommunicator # 6] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_24_00_000000_0 2018-01-05T11:28:59,033 INFO [TaskCommunicator # 6] GuaranteedTasks: Received response for attempt_1515180518813_0001_24_00_000000_0, true 2018-01-05T11:28:59,033 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:1404499845 2018-01-05T11:28:59,033 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : analyze table cbo_t2 partition ...statistics (Stage-0), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_24_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=cbo_t2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=out_Map 1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:28:59,033 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1037041664, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:59,033 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_24_00_000000_0 vertexName: Map 1 2018-01-05T11:28:59,034 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000049, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_24_00_000000_0, eventCount=1 } 2018-01-05T11:28:59,034 DEBUG [TezTR-518813_1_24_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_24 queryId: djaiswal_20180105112858_b060ea84-873c-4c33-97d6-51b6d7348b47 instanceType: TaskRunner2Callable 2018-01-05T11:28:59,034 INFO [TezTR-518813_1_24_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_24_00_000000_0 2018-01-05T11:28:59,034 DEBUG [TezTR-518813_1_24_0_0_0] log.PerfLogger: 2018-01-05T11:28:59,035 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:59,035 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:59,036 INFO [IPC Server handler 8 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_24_00_000000_0 sent events: (0-1). 2018-01-05T11:28:59,036 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:28:59,036 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_24_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:28:59,036 DEBUG [TezTR-518813_1_24_0_0_0] log.PerfLogger: 2018-01-05T11:28:59,036 INFO [TezTR-518813_1_24_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:59,036 INFO [TezTR-518813_1_24_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:59,037 INFO [I/O Setup 1 Initialize: {out_Map 1}] output.MROutput: out_Map 1: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:28:59,037 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:59,037 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:59,037 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:59,037 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:59,037 INFO [TezTR-518813_1_24_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:59,037 INFO [I/O Setup 0 Initialize: {cbo_t2}] input.MRInput: cbo_t2 using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:28:59,037 INFO [I/O Setup 0 Initialize: {cbo_t2}] input.MRInputLegacy: cbo_t2 MRInputLegacy deferring initialization 2018-01-05T11:28:59,037 INFO [TezTR-518813_1_24_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:59,037 INFO [TezTR-518813_1_24_0_0_0] resources.MemoryDistributor: InitialRequests=[cbo_t2:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [out_Map 1:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput] 2018-01-05T11:28:59,037 INFO [TezTR-518813_1_24_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:59,037 INFO [TezTR-518813_1_24_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:59,037 INFO [TezTR-518813_1_24_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 1037041664, TotalAvailable: 694817914, TotalRequested/TotalJVMHeap:0.00 2018-01-05T11:28:59,037 INFO [TezTR-518813_1_24_0_0_0] resources.MemoryDistributor: Allocations=[cbo_t2:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [out_Map 1:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0] 2018-01-05T11:28:59,037 INFO [TezTR-518813_1_24_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:59,037 INFO [TezTR-518813_1_24_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: cbo_t2 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:28:59,037 INFO [TezTR-518813_1_24_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:28:59,037 INFO [TezTR-518813_1_24_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:28:59,037 INFO [I/O Setup 1 Start: {cbo_t2}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: cbo_t2 2018-01-05T11:28:59,037 INFO [TezTR-518813_1_24_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:59,037 INFO [TezTR-518813_1_24_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_24_00_000000_0 2018-01-05T11:28:59,037 DEBUG [TezTR-518813_1_24_0_0_0] log.PerfLogger: 2018-01-05T11:28:59,037 DEBUG [TezTR-518813_1_24_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_24_00_000000_0_10145 2018-01-05T11:28:59,037 DEBUG [TezTR-518813_1_24_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:59,037 DEBUG [TezTR-518813_1_24_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_24_00_000000_0 2018-01-05T11:28:59,037 INFO [TezTR-518813_1_24_0_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112858_b060ea84-873c-4c33-97d6-51b6d7348b47 2018-01-05T11:28:59,038 DEBUG [TezTR-518813_1_24_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:59,038 DEBUG [TezTR-518813_1_24_0_0_0] log.PerfLogger: 2018-01-05T11:28:59,038 DEBUG [TezTR-518813_1_24_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:59,038 DEBUG [TezTR-518813_1_24_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:59,038 DEBUG [TezTR-518813_1_24_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112858_b060ea84-873c-4c33-97d6-51b6d7348b47_Map 1__MAP_PLAN__ 2018-01-05T11:28:59,038 INFO [TezTR-518813_1_24_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_788_1004233992122089697-1/djaiswal/_tez_scratch_dir/2055c940-e0a2-4579-965a-4af004ae1182/map.xml 2018-01-05T11:28:59,038 DEBUG [TezTR-518813_1_24_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_788_1004233992122089697-1/djaiswal/_tez_scratch_dir/2055c940-e0a2-4579-965a-4af004ae1182/map.xml 2018-01-05T11:28:59,038 DEBUG [TezTR-518813_1_24_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_788_1004233992122089697-1/djaiswal/_tez_scratch_dir/2055c940-e0a2-4579-965a-4af004ae1182/map.xml 2018-01-05T11:28:59,038 DEBUG [TezTR-518813_1_24_0_0_0] log.PerfLogger: 2018-01-05T11:28:59,038 INFO [TezTR-518813_1_24_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:59,038 DEBUG [TezTR-518813_1_24_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,038 DEBUG [TezTR-518813_1_24_0_0_0] log.PerfLogger: 2018-01-05T11:28:59,038 INFO [TezTR-518813_1_24_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 2.00KB 2018-01-05T11:28:59,038 DEBUG [TezTR-518813_1_24_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:59,038 INFO [TezTR-518813_1_24_0_0_0] tez.MapRecordProcessor: The input names are: [cbo_t2] 2018-01-05T11:28:59,038 INFO [TezTR-518813_1_24_0_0_0] input.MRInput: Initialized MRInput: cbo_t2 2018-01-05T11:28:59,039 DEBUG [TezTR-518813_1_24_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 1 2018-01-05T11:28:59,039 INFO [TezTR-518813_1_24_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_788_1004233992122089697-1/djaiswal/_tez_scratch_dir/2055c940-e0a2-4579-965a-4af004ae1182/map.xml 2018-01-05T11:28:59,039 DEBUG [TezTR-518813_1_24_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:59,039 DEBUG [TezTR-518813_1_24_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2/dt=2014/cbo_t2.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.cbo_t2, field.delim=,, columns.types=string:string:int:float:boolean, serialization.ddl=struct cbo_t2 { string key, string value, i32 c_int, float c_float, bool c_boolean}, serialization.format=,, columns=key,value,c_int,c_float,c_boolean, columns.comments=, partition_columns.types=string, partition_columns=dt, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180536}, jobProperties=null], partSpec={dt=2014}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string:int:float:boolean, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2/dt=2014, columns=key,value,c_int,c_float,c_boolean, partition_columns.types=string, serialization.format=,, numRows=0, numFiles=1, serialization.ddl=struct cbo_t2 { string key, string value, i32 c_int, float c_float, bool c_boolean}, transient_lastDdlTime=1515180537, rawDataSize=0, field.delim=,, columns.comments=, totalSize=282, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=dt, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.cbo_t2}, baseFileName=dt=2014, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2/dt=2014=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.cbo_t2, field.delim=,, columns.types=string:string:int:float:boolean, serialization.ddl=struct cbo_t2 { string key, string value, i32 c_int, float c_float, bool c_boolean}, serialization.format=,, columns=key,value,c_int,c_float,c_boolean, columns.comments=, partition_columns.types=string, partition_columns=dt, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180536}, jobProperties=null], partSpec={dt=2014}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string:int:float:boolean, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2/dt=2014, columns=key,value,c_int,c_float,c_boolean, partition_columns.types=string, serialization.format=,, numRows=0, numFiles=1, serialization.ddl=struct cbo_t2 { string key, string value, i32 c_int, float c_float, bool c_boolean}, transient_lastDdlTime=1515180537, rawDataSize=0, field.delim=,, columns.comments=, totalSize=282, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=dt, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.cbo_t2}, baseFileName=dt=2014, vectorPartitionDesc=null]} 2018-01-05T11:28:59,039 DEBUG [TezTR-518813_1_24_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:59,039 INFO [TezTR-518813_1_24_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:59,039 DEBUG [TezTR-518813_1_24_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:59,040 DEBUG [TezTR-518813_1_24_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:59,040 INFO [TezTR-518813_1_24_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_788_1004233992122089697-1/djaiswal/_tez_scratch_dir/2055c940-e0a2-4579-965a-4af004ae1182/map.xml 2018-01-05T11:28:59,040 DEBUG [TezTR-518813_1_24_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:59,040 INFO [TezTR-518813_1_24_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_24_00_000000_0 2018-01-05T11:28:59,040 INFO [TezTR-518813_1_24_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value, c_int, c_float, c_boolean] / schema.evolution.columns.types [string, string, int, float, boolean] (isAcidRead false) 2018-01-05T11:28:59,041 WARN [TezTR-518813_1_24_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2/dt=2014/cbo_t2.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:28:59,041 DEBUG [TezTR-518813_1_24_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:28:59,041 WARN [TezTR-518813_1_24_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2/dt=2014/cbo_t2.txt:0+282 2018-01-05T11:28:59,042 DEBUG [TezTR-518813_1_24_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2/dt=2014/cbo_t2.txt 2018-01-05T11:28:59,042 INFO [TezTR-518813_1_24_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2/dt=2014/cbo_t2.txt:0+282], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=282} 2018-01-05T11:28:59,042 INFO [TezTR-518813_1_24_0_0_0] input.MRInput: cbo_t2 initialized RecordReader from event 2018-01-05T11:28:59,042 DEBUG [TezTR-518813_1_24_0_0_0] tez.MapRecordProcessor: Starting Output: out_Map 1 2018-01-05T11:28:59,042 INFO [TezTR-518813_1_24_0_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:28:59,042 INFO [TezTR-518813_1_24_0_0_0] exec.MapOperator: Initializing operator MAP[0] 2018-01-05T11:28:59,042 DEBUG [TezTR-518813_1_24_0_0_0] exec.MapOperator: Initialization Done 0 MAP 2018-01-05T11:28:59,042 DEBUG [TezTR-518813_1_24_0_0_0] exec.MapOperator: Operator 0 MAP initialized 2018-01-05T11:28:59,042 DEBUG [TezTR-518813_1_24_0_0_0] exec.MapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:28:59,042 DEBUG [TezTR-518813_1_24_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,042 DEBUG [TezTR-518813_1_24_0_0_0] exec.MapOperator: Adding alias cbo_t2 to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2/dt=2014 2018-01-05T11:28:59,042 DEBUG [TezTR-518813_1_24_0_0_0] exec.MapOperator: dump TS[0] struct 2018-01-05T11:28:59,042 INFO [TezTR-518813_1_24_0_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:28:59,042 INFO [TezTR-518813_1_24_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:59,042 DEBUG [TezTR-518813_1_24_0_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:28:59,042 INFO [TezTR-518813_1_24_0_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:28:59,042 DEBUG [TezTR-518813_1_24_0_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:28:59,042 DEBUG [TezTR-518813_1_24_0_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:28:59,042 DEBUG [TezTR-518813_1_24_0_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:28:59,042 DEBUG [TezTR-518813_1_24_0_0_0] log.PerfLogger: 2018-01-05T11:28:59,042 INFO [TezTR-518813_1_24_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_788_1004233992122089697-1/djaiswal/_tez_scratch_dir/2055c940-e0a2-4579-965a-4af004ae1182/map.xml 2018-01-05T11:28:59,042 DEBUG [TezTR-518813_1_24_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:59,042 DEBUG [TezTR-518813_1_24_0_0_0] exec.MapOperator: Processing alias(es) cbo_t2 for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2/dt=2014/cbo_t2.txt 2018-01-05T11:28:59,042 INFO [TezTR-518813_1_24_0_0_0] exec.TableScanOperator: Stats Gathering found a new partition spec = dt=2014 2018-01-05T11:28:59,042 INFO [TezTR-518813_1_24_0_0_0] exec.MapOperator: MAP[0]: records read - 1 2018-01-05T11:28:59,042 INFO [TezTR-518813_1_24_0_0_0] exec.MapOperator: MAP[0]: records read - 10 2018-01-05T11:28:59,043 DEBUG [TezTR-518813_1_24_0_0_0] log.PerfLogger: 2018-01-05T11:28:59,043 DEBUG [TezTR-518813_1_24_0_0_0] mr.ObjectCache: djaiswal_20180105112858_b060ea84-873c-4c33-97d6-51b6d7348b47_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:28:59,043 DEBUG [TezTR-518813_1_24_0_0_0] exec.MapOperator: close called for operator MAP[0] 2018-01-05T11:28:59,043 INFO [TezTR-518813_1_24_0_0_0] exec.MapOperator: Closing operator MAP[0] 2018-01-05T11:28:59,043 INFO [TezTR-518813_1_24_0_0_0] exec.MapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:20, 2018-01-05T11:28:59,043 DEBUG [TezTR-518813_1_24_0_0_0] exec.MapOperator: Closing child = TS[0] 2018-01-05T11:28:59,043 DEBUG [TezTR-518813_1_24_0_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:28:59,043 DEBUG [TezTR-518813_1_24_0_0_0] exec.TableScanOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:59,043 INFO [TezTR-518813_1_24_0_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:28:59,043 DEBUG [TezTR-518813_1_24_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,043 DEBUG [TezTR-518813_1_24_0_0_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_788_1004233992122089697-1/-mr-10000 2018-01-05T11:28:59,043 INFO [TezTR-518813_1_24_0_0_0] exec.TableScanOperator: publishing : default.cbo_t2/dt=2014/ : {numRows=20, rawDataSize=262} 2018-01-05T11:28:59,066 DEBUG [TezTR-518813_1_24_0_0_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_788_1004233992122089697-1/-mr-10000/tmpstats-0_0 2018-01-05T11:28:59,066 DEBUG [TezTR-518813_1_24_0_0_0] fs.FSStatsPublisher: Writing stats in it : {default.cbo_t2/dt=2014/={numRows=20, rawDataSize=262}} 2018-01-05T11:28:59,066 DEBUG [TezTR-518813_1_24_0_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:28:59,067 DEBUG [TezTR-518813_1_24_0_0_0] exec.MapOperator: 0 Close done 2018-01-05T11:28:59,067 INFO [TezTR-518813_1_24_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_24_00_000000_0 2018-01-05T11:28:59,067 INFO [TezTR-518813_1_24_0_0_0] output.MROutput: out_Map 1 closed 2018-01-05T11:28:59,067 DEBUG [TezTR-518813_1_24_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:59,067 INFO [TezTR-518813_1_24_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:59,067 INFO [TezTR-518813_1_24_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_24_00_000000_0, askedToStop=false 2018-01-05T11:28:59,067 INFO [TezTR-518813_1_24_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_24_00_000000_0, stopRequested=false 2018-01-05T11:28:59,067 INFO [TezTR-518813_1_24_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_24_00_000000_0: Counters: 5 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=20, INPUT_SPLIT_LENGTH_BYTES=282, OUTPUT_RECORDS=0][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=20]] 2018-01-05T11:28:59,067 DEBUG [TezTR-518813_1_24_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_24_0_0_0 thread id: 1637 scheme: file bytesRead: 298 bytesWritten: 91 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:59,067 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_24_00_000000_0, isTaskDone=true 2018-01-05T11:28:59,067 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000049, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_24_00_000000_0, eventCount=2 } 2018-01-05T11:28:59,068 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_24][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_24_00_000000_0, creationTime=1515180539016, allocationTime=1515180539017, startTime=1515180539020, finishTime=1515180539068, timeTaken=48, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:59,068 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:59,068 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_24_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:59,068 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_24_00_000000_0 2018-01-05T11:28:59,068 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_24_00_000000_0 2018-01-05T11:28:59,068 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_24_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:59,068 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_24_00_000000_0; false 2018-01-05T11:28:59,068 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000049=35 2018-01-05T11:28:59,068 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_24_00_000000_0: true 2018-01-05T11:28:59,068 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_24_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:59,068 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_24][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_24_00_000000, startTime=1515180539020, finishTime=1515180539068, timeTaken=48, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_24_00_000000_0, diagnostics=, counters=Counters: 11, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=298, FILE_BYTES_WRITTEN=91, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=20, INPUT_SPLIT_LENGTH_BYTES=282, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=20 2018-01-05T11:28:59,068 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:59,068 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_24_00_000000_0 2018-01-05T11:28:59,068 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:59,068 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000049, QueryId=djaiswal_20180105112858_b060ea84-873c-4c33-97d6-51b6d7348b47-24, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:28:59,068 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_24_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:59,068 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_24_00_000000_0 2018-01-05T11:28:59,068 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_24_00 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:59,068 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_24_00 [Map 1] 2018-01-05T11:28:59,068 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_24][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_24_00, initRequestedTime=1515180538988, initedTime=1515180539016, startRequestedTime=1515180538998, startedTime=1515180539016, finishTime=1515180539068, timeTaken=52, status=SUCCEEDED, diagnostics=, counters=Counters: 11, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=298, FILE_BYTES_WRITTEN=91, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=20, INPUT_SPLIT_LENGTH_BYTES=282, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=20, vertexStats=firstTaskStartTime=1515180539020, firstTasksToStart=[ task_1515180518813_0001_24_00_000000 ], lastTaskFinishTime=1515180539068, lastTasksToFinish=[ task_1515180518813_0001_24_00_000000 ], minTaskDuration=48, maxTaskDuration=48, avgTaskDuration=48.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_24_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_24_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:59,068 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_24_00 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:59,068 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_24][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000049, stoppedTime=1515180539068, exitStatus=0 2018-01-05T11:28:59,068 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000049 exited with diagnostics set to Container completed. 2018-01-05T11:28:59,068 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_24_00 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1 2018-01-05T11:28:59,068 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null 2018-01-05T11:28:59,068 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_24_00 [Map 1] 2018-01-05T11:28:59,069 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_24, queueSize=0 2018-01-05T11:28:59,069 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_24][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_24, startTime=1515180538988, finishTime=1515180539068, timeTaken=80, status=SUCCEEDED, diagnostics=, counters=Counters: 15, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, OTHER_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=167, AM_GC_TIME_MILLIS=0, File System Counters, FILE_BYTES_READ=298, FILE_BYTES_WRITTEN=91, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=20, INPUT_SPLIT_LENGTH_BYTES=282, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=20 2018-01-05T11:28:59,069 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_24 finished with state: SUCCEEDED 2018-01-05T11:28:59,069 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_24 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:28:59,069 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000049 2018-01-05T11:28:59,070 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_24, dagState=SUCCEEDED 2018-01-05T11:28:59,070 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_24_post 2018-01-05T11:28:59,070 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:59,070 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:28:59,070 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:28:59,070 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 24 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=1, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=1,NumRequestsWithoutLocation=1, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=1} 2018-01-05T11:28:59,070 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:28:59,070 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=analyze table cbo_t2 partition ...statistics (Stage-0), with id=dag_1515180518813_0001_24 2018-01-05T11:28:59,070 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 24, to 10.22.8.182:49818 2018-01-05T11:28:59,070 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=1, Remaining=0 2018-01-05T11:28:59,070 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=analyze table cbo_t2 partition ...statistics (Stage-0), with id=dag_1515180518813_0001_24 2018-01-05T11:28:59,070 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=24} 2018-01-05T11:28:59,070 INFO [IPC Server handler 1 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=24}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:28:59,070 INFO [IPC Server handler 1 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/24 after 300 seconds 2018-01-05T11:28:59,070 ERROR [IPC Server handler 1 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112858_b060ea84-873c-4c33-97d6-51b6d7348b47, dag_1515180518813_0001_24 2018-01-05T11:28:59,070 INFO [IPC Server handler 1 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112858_b060ea84-873c-4c33-97d6-51b6d7348b47 2018-01-05T11:28:59,070 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=24} = 0 2018-01-05T11:28:59,070 DEBUG [IPC Server handler 1 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=24} 2018-01-05T11:28:59,070 DEBUG [IPC Server handler 1 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:28:59,070 DEBUG [IPC Server handler 1 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=24} 2018-01-05T11:28:59,070 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,070 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,070 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 2018-01-05T11:28:59,070 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,072 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-2:STATS] in serial mode 2018-01-05T11:28:59,072 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@6aaa9cc, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:59,072 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:59,072 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,072 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:59,072 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:59,072 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:59,072 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:59,072 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,072 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,072 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:59,072 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:59,072 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,072 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:59,072 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,072 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:59,072 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:59,072 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,073 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:59,073 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_788_1004233992122089697-1/-mr-10000 2018-01-05T11:28:59,073 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_788_1004233992122089697-1/-mr-10000 2018-01-05T11:28:59,073 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Getting file stats of all partitions. threadpool size:15 2018-01-05T11:28:59,074 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Finished getting file stats of all partitions! 2018-01-05T11:28:59,074 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.cbo_t2/dt=2014/ numRows 2018-01-05T11:28:59,074 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.cbo_t2/dt=2014/ numRows 20 2018-01-05T11:28:59,074 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.cbo_t2/dt=2014/ rawDataSize 2018-01-05T11:28:59,074 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.cbo_t2/dt=2014/ rawDataSize 262 2018-01-05T11:28:59,074 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Partition {dt=2014} stats: [numFiles=1, numRows=20, totalSize=282, rawDataSize=262] 2018-01-05T11:28:59,074 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,075 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:59,075 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:59,075 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:59,075 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:59,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,075 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,076 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,076 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,076 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:59,076 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:59,076 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,076 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:59,076 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,076 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_partitions : db=default tbl=cbo_t2 2018-01-05T11:28:59,076 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_partitions : db=default tbl=cbo_t2 2018-01-05T11:28:59,076 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: New partition values:[2014] 2018-01-05T11:28:59,079 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,084 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,087 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,088 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,088 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: About to delete stats tmp dir :file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_788_1004233992122089697-1/-mr-10000 2018-01-05T11:28:59,088 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,088 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,088 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: analyze table cbo_t2 partition (dt) compute statistics 2018-01-05T11:28:59,088 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:28:59,088 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@cbo_t2 2018-01-05T11:28:59,088 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@cbo_t2@dt=2014 2018-01-05T11:28:59,088 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@cbo_t2 2018-01-05T11:28:59,088 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@cbo_t2@dt=2014 2018-01-05T11:28:59,088 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,089 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,089 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:59,089 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_partitions_(String, String, List, EnvironmentContext, )=12} 2018-01-05T11:28:59,089 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112858_b060ea84-873c-4c33-97d6-51b6d7348b47); Time taken: 0.196 seconds 2018-01-05T11:28:59,089 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:59,089 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,089 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:59,089 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,089 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query analyze table cbo_t2 partition (dt) compute statistics 2018-01-05T11:28:59,089 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-58_788_1004233992122089697-1 2018-01-05T11:28:59,089 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.301 seconds 2018-01-05T11:28:59,089 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:59,089 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:59,089 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:59,089 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:59,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:59,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: analyze table cbo_t2 compute statistics for columns key, value, c_int, c_float, c_boolean 2018-01-05T11:28:59,090 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136): analyze table cbo_t2 compute statistics for columns key, value, c_int, c_float, c_boolean 2018-01-05T11:28:59,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: analyze table cbo_t2 compute statistics for columns key, value, c_int, c_float, c_boolean 2018-01-05T11:28:59,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:59,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,090 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:59,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,090 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t2 2018-01-05T11:28:59,090 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t2 2018-01-05T11:28:59,093 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,093 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,093 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: select compute_stats(`key`, 'hll') , compute_stats(`value`, 'hll') , compute_stats(`c_int`, 'hll') , compute_stats(`c_float`, 'hll') , compute_stats(`c_boolean`, 'hll') , `dt` from `default`.`cbo_t2` group by `dt` 2018-01-05T11:28:59,093 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: select compute_stats(`key`, 'hll') , compute_stats(`value`, 'hll') , compute_stats(`c_int`, 'hll') , compute_stats(`c_float`, 'hll') , compute_stats(`c_boolean`, 'hll') , `dt` from `default`.`cbo_t2` group by `dt` 2018-01-05T11:28:59,099 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:59,099 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Invoking analyze on rewritten query 2018-01-05T11:28:59,099 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Starting Semantic Analysis 2018-01-05T11:28:59,099 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:59,099 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:59,099 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:59,099 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:59,099 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:59,099 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed phase 1 of Semantic Analysis 2018-01-05T11:28:59,099 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for source tables 2018-01-05T11:28:59,099 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,099 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t2 2018-01-05T11:28:59,100 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t2 2018-01-05T11:28:59,103 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,104 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for subqueries 2018-01-05T11:28:59,104 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for destination tables 2018-01-05T11:28:59,117 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed getting MetaData in Semantic Analysis 2018-01-05T11:28:59,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_093_9159397621401875189-1/-mr-10001 2018-01-05T11:28:59,117 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Table Plan for cbo_t2 TS[0] 2018-01-05T11:28:59,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:59,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:59,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:59,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:59,118 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:59,119 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: RR before GB cbo_t2{(key,key: string)(value,value: string)(c_int,c_int: int)(c_float,c_float: float)(c_boolean,c_boolean: boolean)(dt,dt: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)(RAW__DATA__SIZE,RAW__DATA__SIZE: bigint)} after GB {((tok_table_or_col dt),_col0: string)((tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct)((tok_function compute_stats (tok_table_or_col value) 'hll'),_col2: struct)((tok_function compute_stats (tok_table_or_col c_int) 'hll'),_col3: struct)((tok_function compute_stats (tok_table_or_col c_float) 'hll'),_col4: struct)((tok_function compute_stats (tok_table_or_col c_boolean) 'hll'),_col5: struct)} cbo_t2{(dt,_col0: string)} 2018-01-05T11:28:59,119 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: tree: (tok_select (tok_selexpr (tok_function compute_stats (tok_table_or_col key) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col value) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col c_int) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col c_float) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col c_boolean) 'hll')) (tok_selexpr (tok_table_or_col dt))) 2018-01-05T11:28:59,119 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: genSelectPlan: input = {((tok_table_or_col dt),_col0: string)((tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct)((tok_function compute_stats (tok_table_or_col value) 'hll'),_col2: struct)((tok_function compute_stats (tok_table_or_col c_int) 'hll'),_col3: struct)((tok_function compute_stats (tok_table_or_col c_float) 'hll'),_col4: struct)((tok_function compute_stats (tok_table_or_col c_boolean) 'hll'),_col5: struct)} cbo_t2{(dt,_col0: string)} starRr = cbo_t2{(key,key: string)(value,value: string)(c_int,c_int: int)(c_float,c_float: float)(c_boolean,c_boolean: boolean)(dt,dt: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)(RAW__DATA__SIZE,RAW__DATA__SIZE: bigint)} 2018-01-05T11:28:59,119 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Select Plan row schema: null{(_c0,_col0: struct)(_c1,_col1: struct)(_c2,_col2: struct)(_c3,_col3: struct)(_c4,_col4: struct)(dt,_col5: string)} cbo_t2{(dt,_col5: string)} 2018-01-05T11:28:59,119 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Select Plan for clause: insclause-0 2018-01-05T11:28:59,119 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_093_9159397621401875189-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-59_093_9159397621401875189-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_093_9159397621401875189-1/-mr-10000 2018-01-05T11:28:59,119 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_093_9159397621401875189-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-59_093_9159397621401875189-1 2018-01-05T11:28:59,146 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_093_9159397621401875189-1/-mr-10003 2018-01-05T11:28:59,146 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_093_9159397621401875189-1/-mr-10000 row schema: null{(_c0,_col0: struct)(_c1,_col1: struct)(_c2,_col2: struct)(_c3,_col3: struct)(_c4,_col4: struct)(dt,_col5: string)} cbo_t2{(dt,_col5: string)} 2018-01-05T11:28:59,146 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Body Plan for Query Block null 2018-01-05T11:28:59,146 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Plan for Query Block null 2018-01-05T11:28:59,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Before logical optimization TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-SEL[5]-FS[6] 2018-01-05T11:28:59,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] lineage.Generator: Time taken for lineage transform=0 2018-01-05T11:28:59,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{cbo_t2}key,value: string|{cbo_t2}value,c_int: int|{cbo_t2}c_int,c_float: float|{cbo_t2}c_float,c_boolean: boolean|{cbo_t2}c_boolean,dt: string|{cbo_t2}dt,BLOCK__OFFSET__INSIDE__FILE: bigint|{cbo_t2}block__offset__inside__file,INPUT__FILE__NAME: string|{cbo_t2}input__file__name,ROW__ID: struct|{cbo_t2}row__id,RAW__DATA__SIZE: bigint|{cbo_t2}raw__data__size) 2018-01-05T11:28:59,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: string|{cbo_t2}key,value: string|{cbo_t2}value,c_int: int|{cbo_t2}c_int,c_float: float|{cbo_t2}c_float,c_boolean: boolean|{cbo_t2}c_boolean,dt: string|{cbo_t2}dt,BLOCK__OFFSET__INSIDE__FILE: bigint|{cbo_t2}block__offset__inside__file,INPUT__FILE__NAME: string|{cbo_t2}input__file__name,ROW__ID: struct|{cbo_t2}row__id,RAW__DATA__SIZE: bigint|{cbo_t2}raw__data__size) 2018-01-05T11:28:59,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:59,147 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value] Column[c_int] Column[c_float] Column[c_boolean] Column[dt] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: string|{}(tok_table_or_col dt),_col1: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col c_int) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col c_float) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col c_boolean) 'hll')) 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(KEY._col0: string|{}(tok_table_or_col dt),VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll'),VALUE._col2: struct|{}(tok_function compute_stats (tok_table_or_col c_int) 'hll'),VALUE._col3: struct|{}(tok_function compute_stats (tok_table_or_col c_float) 'hll'),VALUE._col4: struct|{}(tok_function compute_stats (tok_table_or_col c_boolean) 'hll')) 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: string|{}(tok_table_or_col dt),_col1: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col c_int) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col c_float) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col c_boolean) 'hll')) 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1,_col2: struct|{null}_c2,_col3: struct|{null}_c3,_col4: struct|{null}_c4,_col5: string|{null}dt) 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col1] Column[_col2] Column[_col3] Column[_col4] Column[_col5] Column[_col0]) 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{},_col2: struct|{},_col3: struct|{},_col4: struct|{},_col5: string|{}) 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,148 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for FS(6) 2018-01-05T11:28:59,148 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(5) 2018-01-05T11:28:59,148 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for GBY(4) 2018-01-05T11:28:59,148 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for RS(3) 2018-01-05T11:28:59,148 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for GBY(2) 2018-01-05T11:28:59,148 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(1) 2018-01-05T11:28:59,148 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.PredicatePushDown: After PPD: TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-SEL[5]-FS[6] 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{cbo_t2}key,value: string|{cbo_t2}value,c_int: int|{cbo_t2}c_int,c_float: float|{cbo_t2}c_float,c_boolean: boolean|{cbo_t2}c_boolean,dt: string|{cbo_t2}dt,BLOCK__OFFSET__INSIDE__FILE: bigint|{cbo_t2}block__offset__inside__file,INPUT__FILE__NAME: string|{cbo_t2}input__file__name,ROW__ID: struct|{cbo_t2}row__id,RAW__DATA__SIZE: bigint|{cbo_t2}raw__data__size) 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: string|{cbo_t2}key,value: string|{cbo_t2}value,c_int: int|{cbo_t2}c_int,c_float: float|{cbo_t2}c_float,c_boolean: boolean|{cbo_t2}c_boolean,dt: string|{cbo_t2}dt,BLOCK__OFFSET__INSIDE__FILE: bigint|{cbo_t2}block__offset__inside__file,INPUT__FILE__NAME: string|{cbo_t2}input__file__name,ROW__ID: struct|{cbo_t2}row__id,RAW__DATA__SIZE: bigint|{cbo_t2}raw__data__size) 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value] Column[c_int] Column[c_float] Column[c_boolean] Column[dt] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: string|{}(tok_table_or_col dt),_col1: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col c_int) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col c_float) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col c_boolean) 'hll')) 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(KEY._col0: string|{}(tok_table_or_col dt),VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll'),VALUE._col2: struct|{}(tok_function compute_stats (tok_table_or_col c_int) 'hll'),VALUE._col3: struct|{}(tok_function compute_stats (tok_table_or_col c_float) 'hll'),VALUE._col4: struct|{}(tok_function compute_stats (tok_table_or_col c_boolean) 'hll')) 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: string|{}(tok_table_or_col dt),_col1: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col c_int) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col c_float) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col c_boolean) 'hll')) 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1,_col2: struct|{null}_c2,_col3: struct|{null}_c3,_col4: struct|{null}_c4,_col5: string|{null}dt) 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col1] Column[_col2] Column[_col3] Column[_col4] Column[_col5] Column[_col0]) 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{},_col2: struct|{},_col3: struct|{},_col4: struct|{},_col5: string|{}) 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{cbo_t2}key,value: string|{cbo_t2}value,c_int: int|{cbo_t2}c_int,c_float: float|{cbo_t2}c_float,c_boolean: boolean|{cbo_t2}c_boolean,dt: string|{cbo_t2}dt,BLOCK__OFFSET__INSIDE__FILE: bigint|{cbo_t2}block__offset__inside__file,INPUT__FILE__NAME: string|{cbo_t2}input__file__name,ROW__ID: struct|{cbo_t2}row__id,RAW__DATA__SIZE: bigint|{cbo_t2}raw__data__size) 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: string|{cbo_t2}key,value: string|{cbo_t2}value,c_int: int|{cbo_t2}c_int,c_float: float|{cbo_t2}c_float,c_boolean: boolean|{cbo_t2}c_boolean,dt: string|{cbo_t2}dt,BLOCK__OFFSET__INSIDE__FILE: bigint|{cbo_t2}block__offset__inside__file,INPUT__FILE__NAME: string|{cbo_t2}input__file__name,ROW__ID: struct|{cbo_t2}row__id,RAW__DATA__SIZE: bigint|{cbo_t2}raw__data__size) 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value] Column[c_int] Column[c_float] Column[c_boolean] Column[dt] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: string|{}(tok_table_or_col dt),_col1: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col c_int) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col c_float) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col c_boolean) 'hll')) 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(KEY._col0: string|{}(tok_table_or_col dt),VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll'),VALUE._col2: struct|{}(tok_function compute_stats (tok_table_or_col c_int) 'hll'),VALUE._col3: struct|{}(tok_function compute_stats (tok_table_or_col c_float) 'hll'),VALUE._col4: struct|{}(tok_function compute_stats (tok_table_or_col c_boolean) 'hll')) 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: string|{}(tok_table_or_col dt),_col1: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col c_int) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col c_float) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col c_boolean) 'hll')) 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1,_col2: struct|{null}_c2,_col3: struct|{null}_c3,_col4: struct|{null}_c4,_col5: string|{null}dt) 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:59,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:59,149 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col1] Column[_col2] Column[_col3] Column[_col4] Column[_col5] Column[_col0]) 2018-01-05T11:28:59,149 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{},_col2: struct|{},_col3: struct|{},_col4: struct|{},_col5: string|{}) 2018-01-05T11:28:59,149 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:59,149 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:59,149 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,149 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,149 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,149 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,149 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partitions : db=default tbl=cbo_t2 2018-01-05T11:28:59,149 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions : db=default tbl=cbo_t2 2018-01-05T11:28:59,152 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,153 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.235397ms + 0.006428ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? ] 2018-01-05T11:28:59,153 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.254529ms + 0.08969ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (11) order by "PART_NAME" asc] 2018-01-05T11:28:59,153 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.140868ms + 0.074677ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (11) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:28:59,153 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.107986ms + 0.023908ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (11) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:59,154 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.127531ms + 0.013488ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (32) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:28:59,154 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.107581ms + 0.011545ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (32) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:59,154 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.095312ms + 0.015255ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (32) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:59,154 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.076992ms + 0.007079ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (32) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:59,154 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.130526ms + 0.034255ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (20) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:28:59,154 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.080165ms + 0.032217ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (32) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:28:59,154 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,158 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,158 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,158 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,158 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,158 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 3 key:[Column[_col0]] 2018-01-05T11:28:59,159 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 3 oldColExprMap: {KEY._col0=Column[_col0], VALUE._col2=Column[_col3], VALUE._col3=Column[_col4], VALUE._col4=Column[_col5], VALUE._col0=Column[_col1], VALUE._col1=Column[_col2]} 2018-01-05T11:28:59,159 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 3 newColExprMap: {KEY._col0=Column[_col0], VALUE._col2=Column[_col3], VALUE._col3=Column[_col4], VALUE._col4=Column[_col5], VALUE._col0=Column[_col1], VALUE._col1=Column[_col2]} 2018-01-05T11:28:59,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: After logical optimization TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-SEL[5]-FS[6] 2018-01-05T11:28:59,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,159 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.StatsUtils: Estimated average row size: 212 2018-01-05T11:28:59,160 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,160 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_aggr_stats_for: db=default table=cbo_t2 2018-01-05T11:28:59,160 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_aggr_stats_for: db=default table=cbo_t2 2018-01-05T11:28:59,162 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.AggregateStatsCache: No aggregate stats cached for database:default, table:cbo_t2, column:key 2018-01-05T11:28:59,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.227105ms + 0.004368ms, the query is [select count("COLUMN_NAME") from "PART_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (?,?,?,?,?) and "PARTITION_NAME" in (?) group by "PARTITION_NAME"] 2018-01-05T11:28:59,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.183583ms + 0.011336ms, the query is [select "PARTITION_NAME", "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "PART_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (%1$s) AND "PARTITION_NAME" in (%2$s) order by "PARTITION_NAME"] 2018-01-05T11:28:59,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.MetaStoreUtils: No stats data found for: dbName=default tblName=cbo_t2 partNames= [dt=2014] colNames=[key] 2018-01-05T11:28:59,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.AggregateStatsCache: No aggregate stats cached for database:default, table:cbo_t2, column:value 2018-01-05T11:28:59,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.127425ms + 0.003267ms, the query is [select "PARTITION_NAME", "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "PART_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (%1$s) AND "PARTITION_NAME" in (%2$s) order by "PARTITION_NAME"] 2018-01-05T11:28:59,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.MetaStoreUtils: No stats data found for: dbName=default tblName=cbo_t2 partNames= [dt=2014] colNames=[value] 2018-01-05T11:28:59,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.AggregateStatsCache: No aggregate stats cached for database:default, table:cbo_t2, column:c_int 2018-01-05T11:28:59,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.157647ms + 0.003272ms, the query is [select "PARTITION_NAME", "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "PART_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (%1$s) AND "PARTITION_NAME" in (%2$s) order by "PARTITION_NAME"] 2018-01-05T11:28:59,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.MetaStoreUtils: No stats data found for: dbName=default tblName=cbo_t2 partNames= [dt=2014] colNames=[c_int] 2018-01-05T11:28:59,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.AggregateStatsCache: No aggregate stats cached for database:default, table:cbo_t2, column:c_float 2018-01-05T11:28:59,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.109519ms + 0.002612ms, the query is [select "PARTITION_NAME", "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "PART_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (%1$s) AND "PARTITION_NAME" in (%2$s) order by "PARTITION_NAME"] 2018-01-05T11:28:59,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.MetaStoreUtils: No stats data found for: dbName=default tblName=cbo_t2 partNames= [dt=2014] colNames=[c_float] 2018-01-05T11:28:59,163 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.AggregateStatsCache: No aggregate stats cached for database:default, table:cbo_t2, column:c_boolean 2018-01-05T11:28:59,164 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 0.155805ms + 0.004069ms, the query is [select "PARTITION_NAME", "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "PART_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (%1$s) AND "PARTITION_NAME" in (%2$s) order by "PARTITION_NAME"] 2018-01-05T11:28:59,164 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] utils.MetaStoreUtils: No stats data found for: dbName=default tblName=cbo_t2 partNames= [dt=2014] colNames=[c_boolean] 2018-01-05T11:28:59,164 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: useDensityFunctionForNDVEstimation = false partsFound = 0 ColumnStatisticsObj = [] 2018-01-05T11:28:59,165 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,165 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,165 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting stats (Num rows: 20 Data size: 11542 Basic stats: COMPLETE Column stats: PARTIAL) on TS[0] 2018-01-05T11:28:59,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (cbo_t2): numRows: 20 dataSize: 11542 basicStatsState: COMPLETE colStatsState: PARTIAL colStats: {dt= colName: dt colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, c_float= colName: c_float colType: float countDistincts: 4 numNulls: 1 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 1.4E-45 max: 3.4028235E38 ] isPrimaryKey: false isEstimated: true, c_int= colName: c_int colType: int countDistincts: 4 numNulls: 1 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, value= colName: value colType: string countDistincts: 4 numNulls: 1 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: string countDistincts: 4 numNulls: 1 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, c_boolean= colName: c_boolean colType: boolean countDistincts: 2 numNulls: 1 avgColLen: 4.0 numTrues: 10 numFalses: 10 isPrimaryKey: false isEstimated: true} 2018-01-05T11:28:59,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting stats (Num rows: 20 Data size: 11542 Basic stats: COMPLETE Column stats: PARTIAL) on SEL[1] 2018-01-05T11:28:59,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-SEL[1]: numRows: 20 dataSize: 11542 basicStatsState: COMPLETE colStatsState: PARTIAL colStats: {dt= colName: dt colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, ROW__ID= colName: ROW__ID colType: struct countDistincts: 20 numNulls: 0 avgColLen: 76.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, RAW__DATA__SIZE= colName: RAW__DATA__SIZE colType: bigint countDistincts: 20 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, c_float= colName: c_float colType: float countDistincts: 4 numNulls: 1 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 1.4E-45 max: 3.4028235E38 ] isPrimaryKey: false isEstimated: true, BLOCK__OFFSET__INSIDE__FILE= colName: BLOCK__OFFSET__INSIDE__FILE colType: bigint countDistincts: 20 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, INPUT__FILE__NAME= colName: INPUT__FILE__NAME colType: string countDistincts: 20 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, c_int= colName: c_int colType: int countDistincts: 4 numNulls: 1 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, value= colName: value colType: string countDistincts: 4 numNulls: 1 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: string countDistincts: 4 numNulls: 1 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, c_boolean= colName: c_boolean colType: boolean countDistincts: 2 numNulls: 1 avgColLen: 4.0 numTrues: 10 numFalses: 10 isPrimaryKey: false isEstimated: true} 2018-01-05T11:28:59,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[2]: inputSize: 11542 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:28:59,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[2] hashAgg: true 2018-01-05T11:28:59,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [Case 3] STATS-GBY[2]: cardinality: 1 2018-01-05T11:28:59,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 2184 Basic stats: COMPLETE Column stats: PARTIAL) on GBY[2] 2018-01-05T11:28:59,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-GBY[2]: numRows: 1 dataSize: 2184 basicStatsState: COMPLETE colStatsState: PARTIAL colStats: {_col0= colName: _col0 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col4= colName: _col4 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col5= colName: _col5 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 272.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:59,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 2184 Basic stats: COMPLETE Column stats: PARTIAL) on RS[3] 2018-01-05T11:28:59,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-RS[3]: numRows: 1 dataSize: 2184 basicStatsState: COMPLETE colStatsState: PARTIAL colStats: {KEY._col0= colName: KEY._col0 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col2= colName: VALUE._col2 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col3= colName: VALUE._col3 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col4= colName: VALUE._col4 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 272.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col0= colName: VALUE._col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col1= colName: VALUE._col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:59,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[4]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:28:59,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[4] hashAgg: false 2018-01-05T11:28:59,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [Case 9] STATS-GBY[4]: cardinality: 1 2018-01-05T11:28:59,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 2216 Basic stats: COMPLETE Column stats: PARTIAL) on GBY[4] 2018-01-05T11:28:59,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-GBY[4]: numRows: 1 dataSize: 2216 basicStatsState: COMPLETE colStatsState: PARTIAL colStats: {_col0= colName: _col0 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col4= colName: _col4 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col5= colName: _col5 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 272.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:59,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting stats (Num rows: 1 Data size: 2216 Basic stats: COMPLETE Column stats: PARTIAL) on SEL[5] 2018-01-05T11:28:59,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-SEL[5]: numRows: 1 dataSize: 2216 basicStatsState: COMPLETE colStatsState: PARTIAL colStats: {_col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col4= colName: _col4 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 272.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col5= colName: _col5 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:59,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-FS[6]: numRows: 1 dataSize: 2216 basicStatsState: COMPLETE colStatsState: PARTIAL colStats: {_col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col4= colName: _col4 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 272.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col5= colName: _col5 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:28:59,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on TS[0] 2018-01-05T11:28:59,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on SEL[1] 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: -1 }) on GBY[2] 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[KEY._col0]]; sort column names: [[KEY._col0]]; bucket count: -1 }) on RS[3] 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: -1 }) on GBY[4] 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col5]]; sort column names: [[_col5]]; bucket count: -1 }) on SEL[5] 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FileSinkOperator: Setting traits ({ bucket column names: [[_col5]]; sort column names: [[_col5]]; bucket count: -1 }) on FS[6] 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,167 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.SetReducerParallelism: Set parallelism for reduce sink RS[3] to: 1 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: FS, 6 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: SEL, 5 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: GBY, 4 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: RS, 3 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: GBY, 2 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: SEL, 1 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:28:59,167 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Cycle free: true 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{cbo_t2}key,value: string|{cbo_t2}value,c_int: int|{cbo_t2}c_int,c_float: float|{cbo_t2}c_float,c_boolean: boolean|{cbo_t2}c_boolean,dt: string|{cbo_t2}dt,BLOCK__OFFSET__INSIDE__FILE: bigint|{cbo_t2}block__offset__inside__file,INPUT__FILE__NAME: string|{cbo_t2}input__file__name,ROW__ID: struct|{cbo_t2}row__id,RAW__DATA__SIZE: bigint|{cbo_t2}raw__data__size) 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: string|{cbo_t2}key,value: string|{cbo_t2}value,c_int: int|{cbo_t2}c_int,c_float: float|{cbo_t2}c_float,c_boolean: boolean|{cbo_t2}c_boolean,dt: string|{cbo_t2}dt) 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value] Column[c_int] Column[c_float] Column[c_boolean] Column[dt]) 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: string|{}(tok_table_or_col dt),_col1: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col c_int) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col c_float) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col c_boolean) 'hll')) 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(KEY._col0: string|{}(tok_table_or_col dt),VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll'),VALUE._col2: struct|{}(tok_function compute_stats (tok_table_or_col c_int) 'hll'),VALUE._col3: struct|{}(tok_function compute_stats (tok_table_or_col c_float) 'hll'),VALUE._col4: struct|{}(tok_function compute_stats (tok_table_or_col c_boolean) 'hll')) 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: string|{}(tok_table_or_col dt),_col1: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col c_int) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col c_float) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col c_boolean) 'hll')) 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:59,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:59,168 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1,_col2: struct|{null}_c2,_col3: struct|{null}_c3,_col4: struct|{null}_c4,_col5: string|{null}dt) 2018-01-05T11:28:59,168 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:59,168 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:59,168 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col1] Column[_col2] Column[_col3] Column[_col4] Column[_col5] Column[_col0]) 2018-01-05T11:28:59,168 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{},_col2: struct|{},_col3: struct|{},_col4: struct|{},_col5: string|{}) 2018-01-05T11:28:59,168 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:59,168 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:59,168 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,168 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: TS[0] 2018-01-05T11:28:59,168 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: RS[3] 2018-01-05T11:28:59,168 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:28:59,168 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2/dt=2014 of table cbo_t2 2018-01-05T11:28:59,168 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2/dt=2014 2018-01-05T11:28:59,168 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: RS[3] 2018-01-05T11:28:59,168 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: GBY[4] 2018-01-05T11:28:59,168 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: FS[6] 2018-01-05T11:28:59,168 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding reduce work (Reducer 2) for GBY[4] 2018-01-05T11:28:59,168 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Setting up reduce sink: RS[3] with following reduce work: Reducer 2 2018-01-05T11:28:59,168 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Removing RS[3] as parent from GBY[4] 2018-01-05T11:28:59,168 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: FS[6] 2018-01-05T11:28:59,168 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:28:59,168 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,168 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,168 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:59,168 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping vectorization 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:28:59,169 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: llap mode: only 2018-01-05T11:28:59,169 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(key, 'hll')' 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'key' 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(value, 'hll')' 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'value' 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(c_int, 'hll')' 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'c_int' 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(c_float, 'hll')' 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'c_float' 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(c_boolean, 'hll')' 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'c_boolean' 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'key' 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'value' 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'c_int' 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'c_float' 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'c_boolean' 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'dt' 2018-01-05T11:28:59,169 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@2612fc5c can be done in LLAP 2018-01-05T11:28:59,169 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Reducer 2 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking '_col1' 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking '_col2' 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking '_col3' 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking '_col4' 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking '_col5' 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking '_col0' 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col0)' 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col1)' 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col1' 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col2)' 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col2' 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col3)' 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col3' 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col4)' 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col4' 2018-01-05T11:28:59,169 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@4636c7ee can be done in LLAP 2018-01-05T11:28:59,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,170 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed plan generation 2018-01-05T11:28:59,170 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:59,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validation start 2018-01-05T11:28:59,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Not a partition. 2018-01-05T11:28:59,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:28:59,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validated dt=2014 2018-01-05T11:28:59,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: cbo_t2 2018-01-05T11:28:59,170 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,170 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partitions_ps_with_auth : db=default tbl=cbo_t2[2014] 2018-01-05T11:28:59,170 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions_ps_with_auth : db=default tbl=cbo_t2[2014] 2018-01-05T11:28:59,172 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000049, QueryId=djaiswal_20180105112858_b060ea84-873c-4c33-97d6-51b6d7348b47-24, DagId=24, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180539032, EndTime=1515180539172 2018-01-05T11:28:59,172 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,178 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,178 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,179 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,179 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=cbo_t2[2014] 2018-01-05T11:28:59,179 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=cbo_t2[2014] 2018-01-05T11:28:59,181 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,184 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,185 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,185 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,185 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0, type:struct, comment:null), FieldSchema(name:_c1, type:struct, comment:null), FieldSchema(name:_c2, type:struct, comment:null), FieldSchema(name:_c3, type:struct, comment:null), FieldSchema(name:_c4, type:struct, comment:null), FieldSchema(name:dt, type:string, comment:null)], properties:null) 2018-01-05T11:28:59,185 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,185 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:59,185 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getAggrColStatsFor_(String, String, List, List, )=5, getTable_(String, String, )=8, flushCache_()=0, getPartitionWithAuthInfo_(String, String, List, String, List, )=6, listPartitions_(String, String, short, )=9, listPartitionsWithAuthInfo_(String, String, List, short, String, List, )=8} 2018-01-05T11:28:59,185 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136); Time taken: 0.095 seconds 2018-01-05T11:28:59,185 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:59,185 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,185 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136): analyze table cbo_t2 compute statistics for columns key, value, c_int, c_float, c_boolean 2018-01-05T11:28:59,186 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,186 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: analyze table cbo_t2 compute statistics for columns key, value, c_int, c_float, c_boolean 2018-01-05T11:28:59,186 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:28:59,186 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@cbo_t2 2018-01-05T11:28:59,186 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@cbo_t2@dt=2014 2018-01-05T11:28:59,186 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@cbo_t2 2018-01-05T11:28:59,186 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@cbo_t2@dt=2014 2018-01-05T11:28:59,186 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_093_9159397621401875189-1/-mr-10000 2018-01-05T11:28:59,186 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,186 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,186 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,187 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Query ID = djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136 2018-01-05T11:28:59,187 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Total jobs = 1 2018-01-05T11:28:59,187 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,187 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:28:59,187 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MAPRED] in serial mode 2018-01-05T11:28:59,209 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_090_4354755825235708462-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:28:59,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:28:59,230 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:28:59,230 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:28:59,230 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136 2018-01-05T11:28:59,230 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Session is already open 2018-01-05T11:28:59,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:28:59,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,230 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Dag name: analyze table cbo_t2 compute sta...c_boolean (Stage-0) 2018-01-05T11:28:59,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: DagInfo: {"context":"Hive","description":"\nanalyze table cbo_t2 compute statistics for columns key, value, c_int, c_float, c_boolean"} 2018-01-05T11:28:59,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136 with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:28:59,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,241 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,241 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:28:59,242 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,242 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 2 size: 2.92KB 2018-01-05T11:28:59,258 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,258 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,269 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:28:59,269 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,269 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:28:59,270 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,270 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 4.41KB 2018-01-05T11:28:59,276 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_093_9159397621401875189-1/-mr-10001 2018-01-05T11:28:59,289 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_093_9159397621401875189-1/-mr-10001 2018-01-05T11:28:59,289 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2/dt=2014 2018-01-05T11:28:59,289 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,291 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,291 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,291 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=analyze table cbo_t2 compute sta...c_boolean (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136 } 2018-01-05T11:28:59,291 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:59,291 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:59,293 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: analyze table cbo_t2 compute sta...c_boolean (Stage-0) 2018-01-05T11:28:59,293 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:28:59,293 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_25, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_25.dot 2018-01-05T11:28:59,294 INFO [IPC Server handler 0 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_25 2018-01-05T11:28:59,294 WARN [IPC Server handler 0 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:59,294 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Running DAG: analyze table cbo_t2 compute sta...c_boolean (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136 } 2018-01-05T11:28:59,316 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_25][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_25, submitTime=1515180539293, queueName=null 2018-01-05T11:28:59,317 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_25_00 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_25_00 [Map 1], Default Resources= 2018-01-05T11:28:59,317 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_25_00 [Map 1] 2018-01-05T11:28:59,317 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:28:59,317 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_25_00 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:59,317 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_25_01 [Reducer 2]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_25_01 [Reducer 2], Default Resources= 2018-01-05T11:28:59,317 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_25_01 [Reducer 2] 2018-01-05T11:28:59,317 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 2 2018-01-05T11:28:59,317 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_25_01 [Reducer 2] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:59,317 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:28:59,317 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_25][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_25, initTime=1515180539317 2018-01-05T11:28:59,317 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: dag_1515180518813_0001_25 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:28:59,318 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:28:59,318 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_25][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_25, startTime=1515180539318 2018-01-05T11:28:59,318 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_25 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:28:59,318 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {cbo_t2={InputName=cbo_t2}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:28:59,318 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: cbo_t2, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:28:59,318 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_25_00 [Map 1] 2018-01-05T11:28:59,318 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_25, dagName=analyze table cbo_t2 compute sta...c_boolean (Stage-0) 2018-01-05T11:28:59,318 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,318 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,318 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,322 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_25_00 [Map 1] 2018-01-05T11:28:59,322 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_25_00 [Map 1] 2018-01-05T11:28:59,322 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:59,322 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_25_00 [Map 1] 2018-01-05T11:28:59,324 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:28:59,324 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@57305871 2018-01-05T11:28:59,324 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_090_4354755825235708462-1/djaiswal/_tez_scratch_dir/20540099-bfea-4ddb-a7a2-f77e49537cff/map.xml 2018-01-05T11:28:59,324 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_090_4354755825235708462-1/djaiswal/_tez_scratch_dir/20540099-bfea-4ddb-a7a2-f77e49537cff/map.xml 2018-01-05T11:28:59,324 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_090_4354755825235708462-1/djaiswal/_tez_scratch_dir/20540099-bfea-4ddb-a7a2-f77e49537cff/map.xml 2018-01-05T11:28:59,324 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:59,324 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:59,324 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,325 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:59,325 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 4.41KB 2018-01-05T11:28:59,325 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_25_00 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:28:59,325 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: cbo_t2 on vertex vertex_1515180518813_0001_25_00 [Map 1] 2018-01-05T11:28:59,326 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to ShuffleVertexManager for vertex_1515180518813_0001_25_01 [Reducer 2] 2018-01-05T11:28:59,326 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:28:59,326 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:59,326 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:28:59,326 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:59,326 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_090_4354755825235708462-1/djaiswal/_tez_scratch_dir/20540099-bfea-4ddb-a7a2-f77e49537cff/map.xml 2018-01-05T11:28:59,326 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:59,326 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias cbo_t2 2018-01-05T11:28:59,326 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2/dt=2014 2018-01-05T11:28:59,326 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2/dt=2014 2018-01-05T11:28:59,326 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 0,1,2,3,4 2018-01-05T11:28:59,326 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = key,value,c_int,c_float,c_boolean 2018-01-05T11:28:59,326 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2/dt=2014 2018-01-05T11:28:59,327 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:59,327 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:59,327 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:59,327 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:59,330 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.05 maxFrac: 0.75 auto: false desiredTaskIput: 104857600 2018-01-05T11:28:59,330 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:28:59,330 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 4 tasks for vertex: vertex_1515180518813_0001_25_01 [Reducer 2] 2018-01-05T11:28:59,330 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_25_01 [Reducer 2] 2018-01-05T11:28:59,330 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_25][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_25_01, reconfigureDoneTime=1515180539330, numTasks=4, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:28:59,330 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_25_01 [Reducer 2], numAdditionalOutputs=1 2018-01-05T11:28:59,330 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_25][Event:VERTEX_INITIALIZED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_25_01, initRequestedTime=1515180539325, initedTime=1515180539330, numTasks=4, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:59,330 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_25_01 [Reducer 2] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:28:59,339 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Time taken to get FileStatuses: 12 2018-01-05T11:28:59,339 INFO [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:28:59,339 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 12 2018-01-05T11:28:59,340 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 1 2018-01-05T11:28:59,340 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:59,340 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 1. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:28:59,340 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_090_4354755825235708462-1/djaiswal/_tez_scratch_dir/20540099-bfea-4ddb-a7a2-f77e49537cff/map.xml 2018-01-05T11:28:59,340 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_090_4354755825235708462-1/djaiswal/_tez_scratch_dir/20540099-bfea-4ddb-a7a2-f77e49537cff/map.xml 2018-01-05T11:28:59,340 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_090_4354755825235708462-1/djaiswal/_tez_scratch_dir/20540099-bfea-4ddb-a7a2-f77e49537cff/map.xml 2018-01-05T11:28:59,340 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:59,340 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:59,340 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,340 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:59,341 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 4.41KB 2018-01-05T11:28:59,341 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2/dt=2014/cbo_t2.txt to src new group? true 2018-01-05T11:28:59,341 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:28:59,341 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:28:59,341 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:28:59,341 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 5 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 282 Original splits: 1 2018-01-05T11:28:59,341 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Using original number of splits: 1 desired splits: 1 2018-01-05T11:28:59,341 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 1 grouped split count is 1, for bucket: 1 2018-01-05T11:28:59,341 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:28:59,341 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: cbo_t2 on vertex vertex_1515180518813_0001_25_00 [Map 1] 2018-01-05T11:28:59,343 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_25_00 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:59,343 INFO [App Shared Pool - #0] impl.VertexImpl: Got updated RootInputsSpecs: {cbo_t2=forAllWorkUnits=true, update=[1]} 2018-01-05T11:28:59,344 INFO [App Shared Pool - #0] impl.VertexImpl: Vertex vertex_1515180518813_0001_25_00 [Map 1] parallelism set to 1 2018-01-05T11:28:59,344 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_25_00 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:59,344 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_25][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_25_00, initRequestedTime=1515180539318, initedTime=1515180539344, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:59,344 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_25_00 [Map 1] with distanceFromRoot: 0 2018-01-05T11:28:59,344 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_25_00 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:28:59,344 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_25][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_25_00, startRequestedTime=1515180539325, startedTime=1515180539344 2018-01-05T11:28:59,344 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_25][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_25_00, reconfigureDoneTime=1515180539344, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:28:59,344 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_25_00 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:59,344 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_25_00 for vertex: vertex_1515180518813_0001_25_01 [Reducer 2] numStartedSources: 1 numSources: 1 2018-01-05T11:28:59,344 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_25_01 [Reducer 2] with distanceFromRoot: 1 2018-01-05T11:28:59,344 INFO [App Shared Pool - #0] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:28:59,344 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_25][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_25_01, startRequestedTime=1515180539344, startedTime=1515180539344 2018-01-05T11:28:59,344 INFO [App Shared Pool - #0] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:28:59,344 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_25_00 [Map 1] 2018-01-05T11:28:59,344 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_25_01 [Reducer 2] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:59,344 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: OnVertexStarted vertex: Reducer 2 with 0 source tasks and 4 pending tasks 2018-01-05T11:28:59,344 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_25][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_25_00_000000, scheduledTime=1515180539344, launchTime=1515180539344 2018-01-05T11:28:59,345 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_25_00 [Map 1] 2018-01-05T11:28:59,345 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Map 1 in vertex: Reducer 2 numBipartiteSourceTasks: 1 2018-01-05T11:28:59,345 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_25_00_000000_0, priority=5, capability=, hosts=null 2018-01-05T11:28:59,345 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=5,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:59,345 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=5,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:59,345 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:59,345 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_25_00_000000_0 on hosts=null 2018-01-05T11:28:59,345 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_25_00_000000_0 on hosts=null 2018-01-05T11:28:59,345 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_25_00_000000_0 2018-01-05T11:28:59,345 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:59,345 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #50, task=TaskInfo{task=attempt_1515180518813_0001_25_00_000000_0, priority=5, startTime=0, containerId=null, uniqueId=49, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000050 2018-01-05T11:28:59,345 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_25_00_000000_0; false 2018-01-05T11:28:59,345 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_25_00_000000_0, priority=5, startTime=311559854, containerId=container_111101111_0001_00_000050assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=49, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:59,345 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000050 for node: 10.22.8.182:49818 2018-01-05T11:28:59,347 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 25, name=analyze table cbo_t2 compute sta...c_boolean (Stage-0), queryId=djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136 2018-01-05T11:28:59,347 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000050, attempt_1515180518813_0001_25_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:59,347 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000050 for node: 10.22.8.182:49818 2018-01-05T11:28:59,347 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000050 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:59,347 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000050, launchTime=1515180539347 2018-01-05T11:28:59,348 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_25_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000050, dagName=analyze table cbo_t2 compute sta...c_boolean (Stage-0), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{cbo_t2,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{Reducer 2,org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput,4}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180539318, firstAttemptStartTime=1515180539345, currentAttemptStartTime=1515180539347} 2018-01-05T11:28:59,348 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_25_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000050] on NM: [10.22.8.182:49818] 2018-01-05T11:28:59,348 INFO [IPC Server handler 0 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000050, QueryId=djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136-25, DagId=25, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180539348 2018-01-05T11:28:59,348 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:59,348 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:59,348 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_25][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_25_00_000000_0, startTime=1515180539347, containerId=container_111101111_0001_00_000050, nodeId=10.22.8.182:49818 2018-01-05T11:28:59,348 DEBUG [IPC Server handler 0 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=25} with the ShuffleHandler 2018-01-05T11:28:59,359 INFO [IPC Server handler 0 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:28:59,359 DEBUG [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/25] 2018-01-05T11:28:59,359 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:59,359 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_25_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:59,359 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_25_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180539345, dagStartTime=1515180539318, withinDagPriority=5, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:28:59,359 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_25_00_000000_0 : ACCEPTED 2018-01-05T11:28:59,359 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_25_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:59,359 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_25_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180539345, dagStartTime=1515180539318, withinDagPriority=5, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:28:59,359 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_25_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:59,359 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_25_00_000000_0 to pre-emption queue 2018-01-05T11:28:59,360 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_25_00_000000_0: true 2018-01-05T11:28:59,360 INFO [TaskCommunicator # 8] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_25_00_000000_0 2018-01-05T11:28:59,360 INFO [TaskCommunicator # 8] GuaranteedTasks: Received response for attempt_1515180518813_0001_25_00_000000_0, true 2018-01-05T11:28:59,360 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:491159562 2018-01-05T11:28:59,361 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : analyze table cbo_t2 compute sta...c_boolean (Stage-0), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_25_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=cbo_t2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=Reducer 2, physicalEdgeCount=4, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ] 2018-01-05T11:28:59,361 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1037041664, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:59,361 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_25_00_000000_0 vertexName: Map 1 2018-01-05T11:28:59,361 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000050, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_25_00_000000_0, eventCount=1 } 2018-01-05T11:28:59,361 DEBUG [TezTR-518813_1_25_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_25 queryId: djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136 instanceType: TaskRunner2Callable 2018-01-05T11:28:59,361 INFO [TezTR-518813_1_25_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_25_00_000000_0 2018-01-05T11:28:59,362 DEBUG [TezTR-518813_1_25_0_0_0] log.PerfLogger: 2018-01-05T11:28:59,362 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:59,363 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:59,363 INFO [IPC Server handler 15 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_25_00_000000_0 sent events: (0-1). 2018-01-05T11:28:59,364 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:28:59,364 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_25_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:28:59,364 DEBUG [TezTR-518813_1_25_0_0_0] log.PerfLogger: 2018-01-05T11:28:59,364 INFO [TezTR-518813_1_25_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:59,364 INFO [TezTR-518813_1_25_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:59,364 INFO [TezTR-518813_1_25_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:59,364 INFO [I/O Setup 0 Initialize: {cbo_t2}] input.MRInput: cbo_t2 using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:28:59,364 INFO [I/O Setup 0 Initialize: {cbo_t2}] input.MRInputLegacy: cbo_t2 MRInputLegacy deferring initialization 2018-01-05T11:28:59,364 INFO [TezTR-518813_1_25_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:59,364 INFO [TezTR-518813_1_25_0_0_0] resources.MemoryDistributor: InitialRequests=[cbo_t2:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [Reducer 2:OUTPUT:10485760:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput] 2018-01-05T11:28:59,364 INFO [TezTR-518813_1_25_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:59,364 INFO [TezTR-518813_1_25_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:59,364 INFO [TezTR-518813_1_25_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 10485760, TotalRequestedScaled: 9679163.076923078, TotalJVMHeap: 1037041664, TotalAvailable: 694817914, TotalRequested/TotalJVMHeap:0.01 2018-01-05T11:28:59,364 INFO [TezTR-518813_1_25_0_0_0] resources.MemoryDistributor: Allocations=[cbo_t2:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:10485760:10485760] 2018-01-05T11:28:59,364 INFO [TezTR-518813_1_25_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:59,364 INFO [TezTR-518813_1_25_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: cbo_t2 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:28:59,364 INFO [TezTR-518813_1_25_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:28:59,364 INFO [TezTR-518813_1_25_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:28:59,364 INFO [I/O Setup 1 Start: {cbo_t2}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: cbo_t2 2018-01-05T11:28:59,364 INFO [TezTR-518813_1_25_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:59,364 INFO [TezTR-518813_1_25_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_25_00_000000_0 2018-01-05T11:28:59,365 DEBUG [TezTR-518813_1_25_0_0_0] log.PerfLogger: 2018-01-05T11:28:59,365 DEBUG [TezTR-518813_1_25_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_25_00_000000_0_10148 2018-01-05T11:28:59,365 DEBUG [TezTR-518813_1_25_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:59,365 DEBUG [TezTR-518813_1_25_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_25_00_000000_0 2018-01-05T11:28:59,365 INFO [TezTR-518813_1_25_0_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136 2018-01-05T11:28:59,365 DEBUG [TezTR-518813_1_25_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:59,365 DEBUG [TezTR-518813_1_25_0_0_0] log.PerfLogger: 2018-01-05T11:28:59,365 DEBUG [TezTR-518813_1_25_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:59,365 DEBUG [TezTR-518813_1_25_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:59,365 DEBUG [TezTR-518813_1_25_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136_Map 1__MAP_PLAN__ 2018-01-05T11:28:59,365 INFO [TezTR-518813_1_25_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_090_4354755825235708462-1/djaiswal/_tez_scratch_dir/20540099-bfea-4ddb-a7a2-f77e49537cff/map.xml 2018-01-05T11:28:59,365 DEBUG [TezTR-518813_1_25_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_090_4354755825235708462-1/djaiswal/_tez_scratch_dir/20540099-bfea-4ddb-a7a2-f77e49537cff/map.xml 2018-01-05T11:28:59,365 DEBUG [TezTR-518813_1_25_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_090_4354755825235708462-1/djaiswal/_tez_scratch_dir/20540099-bfea-4ddb-a7a2-f77e49537cff/map.xml 2018-01-05T11:28:59,365 DEBUG [TezTR-518813_1_25_0_0_0] log.PerfLogger: 2018-01-05T11:28:59,365 INFO [TezTR-518813_1_25_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:59,365 DEBUG [TezTR-518813_1_25_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,366 DEBUG [TezTR-518813_1_25_0_0_0] log.PerfLogger: 2018-01-05T11:28:59,366 INFO [TezTR-518813_1_25_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 4.41KB 2018-01-05T11:28:59,366 DEBUG [TezTR-518813_1_25_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:59,366 INFO [TezTR-518813_1_25_0_0_0] tez.MapRecordProcessor: The input names are: [cbo_t2] 2018-01-05T11:28:59,366 INFO [TezTR-518813_1_25_0_0_0] input.MRInput: Initialized MRInput: cbo_t2 2018-01-05T11:28:59,366 DEBUG [TezTR-518813_1_25_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 1 2018-01-05T11:28:59,366 INFO [TezTR-518813_1_25_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_090_4354755825235708462-1/djaiswal/_tez_scratch_dir/20540099-bfea-4ddb-a7a2-f77e49537cff/map.xml 2018-01-05T11:28:59,366 DEBUG [TezTR-518813_1_25_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:59,366 DEBUG [TezTR-518813_1_25_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2/dt=2014/cbo_t2.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.cbo_t2, field.delim=,, columns.types=string:string:int:float:boolean, serialization.ddl=struct cbo_t2 { string key, string value, i32 c_int, float c_float, bool c_boolean}, serialization.format=,, columns=key,value,c_int,c_float,c_boolean, columns.comments=, partition_columns.types=string, partition_columns=dt, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180536}, jobProperties=null], partSpec={dt=2014}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string:int:float:boolean, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2/dt=2014, columns=key,value,c_int,c_float,c_boolean, partition_columns.types=string, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=,, numRows=20, numFiles=1, serialization.ddl=struct cbo_t2 { string key, string value, i32 c_int, float c_float, bool c_boolean}, transient_lastDdlTime=1515180539, rawDataSize=262, field.delim=,, columns.comments=, totalSize=282, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=dt, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.cbo_t2}, baseFileName=dt=2014, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2/dt=2014=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.cbo_t2, field.delim=,, columns.types=string:string:int:float:boolean, serialization.ddl=struct cbo_t2 { string key, string value, i32 c_int, float c_float, bool c_boolean}, serialization.format=,, columns=key,value,c_int,c_float,c_boolean, columns.comments=, partition_columns.types=string, partition_columns=dt, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180536}, jobProperties=null], partSpec={dt=2014}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string:int:float:boolean, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2/dt=2014, columns=key,value,c_int,c_float,c_boolean, partition_columns.types=string, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=,, numRows=20, numFiles=1, serialization.ddl=struct cbo_t2 { string key, string value, i32 c_int, float c_float, bool c_boolean}, transient_lastDdlTime=1515180539, rawDataSize=262, field.delim=,, columns.comments=, totalSize=282, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=dt, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.cbo_t2}, baseFileName=dt=2014, vectorPartitionDesc=null]} 2018-01-05T11:28:59,367 DEBUG [TezTR-518813_1_25_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:59,367 INFO [TezTR-518813_1_25_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:59,367 DEBUG [TezTR-518813_1_25_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:59,368 DEBUG [TezTR-518813_1_25_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:59,368 INFO [TezTR-518813_1_25_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_090_4354755825235708462-1/djaiswal/_tez_scratch_dir/20540099-bfea-4ddb-a7a2-f77e49537cff/map.xml 2018-01-05T11:28:59,368 DEBUG [TezTR-518813_1_25_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:59,368 INFO [TezTR-518813_1_25_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_25_00_000000_0 2018-01-05T11:28:59,368 INFO [TezTR-518813_1_25_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value, c_int, c_float, c_boolean] / schema.evolution.columns.types [string, string, int, float, boolean] (isAcidRead false) 2018-01-05T11:28:59,368 DEBUG [TezTR-518813_1_25_0_0_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:59,368 DEBUG [TezTR-518813_1_25_0_0_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:59,368 DEBUG [TezTR-518813_1_25_0_0_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:59,368 DEBUG [TezTR-518813_1_25_0_0_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:59,368 WARN [TezTR-518813_1_25_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2/dt=2014/cbo_t2.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:28:59,369 DEBUG [TezTR-518813_1_25_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:28:59,369 WARN [TezTR-518813_1_25_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2/dt=2014/cbo_t2.txt:0+282 2018-01-05T11:28:59,369 DEBUG [TezTR-518813_1_25_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2/dt=2014/cbo_t2.txt 2018-01-05T11:28:59,369 INFO [TezTR-518813_1_25_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2/dt=2014/cbo_t2.txt:0+282], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=282} 2018-01-05T11:28:59,369 INFO [TezTR-518813_1_25_0_0_0] input.MRInput: cbo_t2 initialized RecordReader from event 2018-01-05T11:28:59,369 DEBUG [TezTR-518813_1_25_0_0_0] tez.MapRecordProcessor: Starting Output: Reducer 2 2018-01-05T11:28:59,370 INFO [TezTR-518813_1_25_0_0_0] impl.ExternalSorter: Reducer 2 using: memoryMb=10, keySerializerClass=class org.apache.hadoop.hive.ql.io.HiveKey, valueSerializerClass=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization$TezBytesWritableSerializer@5146ca53, comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator@904d58d, partitioner=org.apache.tez.mapreduce.partition.MRPartitioner, serialization=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.hadoop.io.serializer.WritableSerialization, org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization, org.apache.hadoop.io.serializer.avro.AvroReflectSerialization, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:28:59,370 INFO [TezTR-518813_1_25_0_0_0] partition.MRPartitioner: Using oldApi, MRpartitionerClass=org.apache.hadoop.hive.ql.io.DefaultHivePartitioner 2018-01-05T11:28:59,370 INFO [TezTR-518813_1_25_0_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 2: , UsingHashComparator=true 2018-01-05T11:28:59,371 INFO [TezTR-518813_1_25_0_0_0] impl.PipelinedSorter: Newly allocated block size=10485760, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=10485760, total=10485760 2018-01-05T11:28:59,371 INFO [TezTR-518813_1_25_0_0_0] impl.PipelinedSorter: Pre allocating rest of memory buffers upfront 2018-01-05T11:28:59,371 INFO [TezTR-518813_1_25_0_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 2: , UsingHashComparator=true#blocks=1, maxMemUsage=10485760, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=10485760, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=10 2018-01-05T11:28:59,371 INFO [TezTR-518813_1_25_0_0_0] impl.PipelinedSorter: Reducer 2: reserved.remaining()=10485760, reserved.metasize=5242880 2018-01-05T11:28:59,371 INFO [TezTR-518813_1_25_0_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:28:59,371 INFO [TezTR-518813_1_25_0_0_0] exec.MapOperator: Initializing operator MAP[0] 2018-01-05T11:28:59,371 DEBUG [TezTR-518813_1_25_0_0_0] exec.MapOperator: Initialization Done 0 MAP 2018-01-05T11:28:59,371 DEBUG [TezTR-518813_1_25_0_0_0] exec.MapOperator: Operator 0 MAP initialized 2018-01-05T11:28:59,371 DEBUG [TezTR-518813_1_25_0_0_0] exec.MapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:28:59,371 DEBUG [TezTR-518813_1_25_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,372 DEBUG [TezTR-518813_1_25_0_0_0] exec.MapOperator: Adding alias cbo_t2 to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2/dt=2014 2018-01-05T11:28:59,372 DEBUG [TezTR-518813_1_25_0_0_0] exec.MapOperator: dump TS[0] struct 2018-01-05T11:28:59,372 INFO [TezTR-518813_1_25_0_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:28:59,372 INFO [TezTR-518813_1_25_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:59,372 DEBUG [TezTR-518813_1_25_0_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:28:59,372 INFO [TezTR-518813_1_25_0_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:28:59,372 DEBUG [TezTR-518813_1_25_0_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:28:59,372 DEBUG [TezTR-518813_1_25_0_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:28:59,372 DEBUG [TezTR-518813_1_25_0_0_0] exec.TableScanOperator: Initializing children of 0 TS 2018-01-05T11:28:59,372 DEBUG [TezTR-518813_1_25_0_0_0] exec.SelectOperator: Initializing child 1 SEL 2018-01-05T11:28:59,372 INFO [TezTR-518813_1_25_0_0_0] exec.SelectOperator: Initializing operator SEL[1] 2018-01-05T11:28:59,372 INFO [TezTR-518813_1_25_0_0_0] exec.SelectOperator: SELECT struct 2018-01-05T11:28:59,372 DEBUG [TezTR-518813_1_25_0_0_0] exec.SelectOperator: Initialization Done 1 SEL 2018-01-05T11:28:59,372 DEBUG [TezTR-518813_1_25_0_0_0] exec.SelectOperator: Operator 1 SEL initialized 2018-01-05T11:28:59,372 DEBUG [TezTR-518813_1_25_0_0_0] exec.SelectOperator: Initializing children of 1 SEL 2018-01-05T11:28:59,372 DEBUG [TezTR-518813_1_25_0_0_0] exec.GroupByOperator: Initializing child 2 GBY 2018-01-05T11:28:59,372 INFO [TezTR-518813_1_25_0_0_0] exec.GroupByOperator: Initializing operator GBY[2] 2018-01-05T11:28:59,372 DEBUG [TezTR-518813_1_25_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,372 DEBUG [TezTR-518813_1_25_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,372 DEBUG [TezTR-518813_1_25_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,372 DEBUG [TezTR-518813_1_25_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,372 DEBUG [TezTR-518813_1_25_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,372 INFO [TezTR-518813_1_25_0_0_0] exec.GroupByOperator: Max hash table memory: 93860656 bytes 2018-01-05T11:28:59,372 INFO [TezTR-518813_1_25_0_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:59,372 DEBUG [TezTR-518813_1_25_0_0_0] exec.GroupByOperator: Initialization Done 2 GBY 2018-01-05T11:28:59,372 DEBUG [TezTR-518813_1_25_0_0_0] exec.GroupByOperator: Operator 2 GBY initialized 2018-01-05T11:28:59,372 DEBUG [TezTR-518813_1_25_0_0_0] exec.GroupByOperator: Initializing children of 2 GBY 2018-01-05T11:28:59,372 DEBUG [TezTR-518813_1_25_0_0_0] exec.ReduceSinkOperator: Initializing child 3 RS 2018-01-05T11:28:59,372 INFO [TezTR-518813_1_25_0_0_0] exec.ReduceSinkOperator: Initializing operator RS[3] 2018-01-05T11:28:59,372 DEBUG [TezTR-518813_1_25_0_0_0] exec.ReduceSinkOperator: keys size is 1 2018-01-05T11:28:59,372 DEBUG [TezTR-518813_1_25_0_0_0] exec.ReduceSinkOperator: Key exprNodeDesc _col0 2018-01-05T11:28:59,372 INFO [TezTR-518813_1_25_0_0_0] exec.ReduceSinkOperator: Using tag = -1 2018-01-05T11:28:59,372 DEBUG [TezTR-518813_1_25_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,372 DEBUG [TezTR-518813_1_25_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,372 DEBUG [TezTR-518813_1_25_0_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1, _col2, _col3, _col4] columnTypes=[struct, struct, struct, struct, struct] 2018-01-05T11:28:59,372 DEBUG [TezTR-518813_1_25_0_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS 2018-01-05T11:28:59,372 DEBUG [TezTR-518813_1_25_0_0_0] exec.ReduceSinkOperator: Operator 3 RS initialized 2018-01-05T11:28:59,372 DEBUG [TezTR-518813_1_25_0_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS done is reset. 2018-01-05T11:28:59,372 DEBUG [TezTR-518813_1_25_0_0_0] exec.GroupByOperator: Initialization Done 2 GBY done is reset. 2018-01-05T11:28:59,372 DEBUG [TezTR-518813_1_25_0_0_0] exec.SelectOperator: Initialization Done 1 SEL done is reset. 2018-01-05T11:28:59,372 DEBUG [TezTR-518813_1_25_0_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:28:59,372 INFO [TezTR-518813_1_25_0_0_0] exec.OperatorUtils: Setting output collector: RS[3] --> Reducer 2 2018-01-05T11:28:59,372 DEBUG [TezTR-518813_1_25_0_0_0] log.PerfLogger: 2018-01-05T11:28:59,372 INFO [TezTR-518813_1_25_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_090_4354755825235708462-1/djaiswal/_tez_scratch_dir/20540099-bfea-4ddb-a7a2-f77e49537cff/map.xml 2018-01-05T11:28:59,372 DEBUG [TezTR-518813_1_25_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:59,372 DEBUG [TezTR-518813_1_25_0_0_0] exec.MapOperator: Processing alias(es) cbo_t2 for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2/dt=2014/cbo_t2.txt 2018-01-05T11:28:59,372 INFO [TezTR-518813_1_25_0_0_0] exec.TableScanOperator: Stats Gathering found a new partition spec = dt=2014 2018-01-05T11:28:59,373 INFO [TezTR-518813_1_25_0_0_0] exec.MapOperator: MAP[0]: records read - 1 2018-01-05T11:28:59,373 INFO [TezTR-518813_1_25_0_0_0] exec.MapOperator: MAP[0]: records read - 10 2018-01-05T11:28:59,373 DEBUG [TezTR-518813_1_25_0_0_0] log.PerfLogger: 2018-01-05T11:28:59,373 DEBUG [TezTR-518813_1_25_0_0_0] mr.ObjectCache: djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:28:59,373 DEBUG [TezTR-518813_1_25_0_0_0] exec.MapOperator: close called for operator MAP[0] 2018-01-05T11:28:59,373 INFO [TezTR-518813_1_25_0_0_0] exec.MapOperator: Closing operator MAP[0] 2018-01-05T11:28:59,373 INFO [TezTR-518813_1_25_0_0_0] exec.MapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:20, 2018-01-05T11:28:59,373 DEBUG [TezTR-518813_1_25_0_0_0] exec.MapOperator: Closing child = TS[0] 2018-01-05T11:28:59,373 DEBUG [TezTR-518813_1_25_0_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:28:59,373 DEBUG [TezTR-518813_1_25_0_0_0] exec.TableScanOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:59,373 INFO [TezTR-518813_1_25_0_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:28:59,373 DEBUG [TezTR-518813_1_25_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,373 DEBUG [TezTR-518813_1_25_0_0_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_093_9159397621401875189-1/-mr-10001 2018-01-05T11:28:59,373 INFO [TezTR-518813_1_25_0_0_0] exec.TableScanOperator: publishing : default.cbo_t2/dt=2014/ : {numRows=20, rawDataSize=262} 2018-01-05T11:28:59,382 DEBUG [AMReporterQueueDrainer] impl.AMReporter: Removing am 10.22.8.182:49824 with last associated dag QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=13} from heartbeat with taskCount=0, amFailed=false, isDone=true 2018-01-05T11:28:59,400 DEBUG [TezTR-518813_1_25_0_0_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_093_9159397621401875189-1/-mr-10001/tmpstats-0_0 2018-01-05T11:28:59,400 DEBUG [TezTR-518813_1_25_0_0_0] fs.FSStatsPublisher: Writing stats in it : {default.cbo_t2/dt=2014/={numRows=20, rawDataSize=262}} 2018-01-05T11:28:59,400 DEBUG [TezTR-518813_1_25_0_0_0] exec.TableScanOperator: Closing child = SEL[1] 2018-01-05T11:28:59,400 DEBUG [TezTR-518813_1_25_0_0_0] exec.SelectOperator: close called for operator SEL[1] 2018-01-05T11:28:59,400 DEBUG [TezTR-518813_1_25_0_0_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:59,400 INFO [TezTR-518813_1_25_0_0_0] exec.SelectOperator: Closing operator SEL[1] 2018-01-05T11:28:59,400 DEBUG [TezTR-518813_1_25_0_0_0] exec.SelectOperator: Closing child = GBY[2] 2018-01-05T11:28:59,400 DEBUG [TezTR-518813_1_25_0_0_0] exec.GroupByOperator: close called for operator GBY[2] 2018-01-05T11:28:59,400 DEBUG [TezTR-518813_1_25_0_0_0] exec.GroupByOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:59,400 INFO [TezTR-518813_1_25_0_0_0] exec.GroupByOperator: Closing operator GBY[2] 2018-01-05T11:28:59,400 INFO [TezTR-518813_1_25_0_0_0] exec.GroupByOperator: Begin Hash Table flush: size = 1 2018-01-05T11:28:59,400 INFO [TezTR-518813_1_25_0_0_0] exec.ReduceSinkOperator: keys are [_col0] num distributions: 1 2018-01-05T11:28:59,400 DEBUG [TezTR-518813_1_25_0_0_0] exec.GroupByOperator: Closing child = RS[3] 2018-01-05T11:28:59,400 DEBUG [TezTR-518813_1_25_0_0_0] exec.ReduceSinkOperator: close called for operator RS[3] 2018-01-05T11:28:59,400 DEBUG [TezTR-518813_1_25_0_0_0] exec.ReduceSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:59,400 INFO [TezTR-518813_1_25_0_0_0] exec.ReduceSinkOperator: Closing operator RS[3] 2018-01-05T11:28:59,400 INFO [TezTR-518813_1_25_0_0_0] exec.ReduceSinkOperator: RECORDS_OUT_INTERMEDIATE_Map_1:1, 2018-01-05T11:28:59,400 DEBUG [TezTR-518813_1_25_0_0_0] exec.ReduceSinkOperator: 3 Close done 2018-01-05T11:28:59,400 DEBUG [TezTR-518813_1_25_0_0_0] exec.GroupByOperator: 2 Close done 2018-01-05T11:28:59,400 DEBUG [TezTR-518813_1_25_0_0_0] exec.SelectOperator: 1 Close done 2018-01-05T11:28:59,400 DEBUG [TezTR-518813_1_25_0_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:28:59,400 DEBUG [TezTR-518813_1_25_0_0_0] exec.MapOperator: 0 Close done 2018-01-05T11:28:59,400 INFO [TezTR-518813_1_25_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_25_00_000000_0 2018-01-05T11:28:59,400 INFO [TezTR-518813_1_25_0_0_0] impl.PipelinedSorter: Reducer 2: Starting flush of map output 2018-01-05T11:28:59,400 INFO [TezTR-518813_1_25_0_0_0] impl.PipelinedSorter: Reducer 2: Span0.length = 1, perItem = 191 2018-01-05T11:28:59,400 INFO [TezTR-518813_1_25_0_0_0] impl.PipelinedSorter: Reducer 2: done sorting span=0, length=1, time=0 2018-01-05T11:28:59,400 DEBUG [TezTR-518813_1_25_0_0_0] util.Progress: Illegal progress value found, progress is Float.NaN. Progress will be changed to 0 2018-01-05T11:28:59,400 INFO [TezTR-518813_1_25_0_0_0] impl.PipelinedSorter: Reducer 2: Heap = SpanIterator<0:0> (span=Span[5242880,191]), 2018-01-05T11:28:59,414 INFO [TezTR-518813_1_25_0_0_0] impl.PipelinedSorter: Reducer 2: Spilling to /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/25/output/attempt_1515180518813_0001_25_00_000000_0_10150_0/file.out 2018-01-05T11:28:59,429 INFO [TezTR-518813_1_25_0_0_0] shuffle.ShuffleUtils: EmptyPartition bitsetSize=3, numOutputs=4, emptyPartitions=3, compressedSize=3 2018-01-05T11:28:59,431 DEBUG [TezTR-518813_1_25_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:59,431 INFO [TezTR-518813_1_25_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:59,431 INFO [TezTR-518813_1_25_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_25_00_000000_0, askedToStop=false 2018-01-05T11:28:59,431 INFO [TezTR-518813_1_25_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_25_00_000000_0, stopRequested=false 2018-01-05T11:28:59,431 INFO [TezTR-518813_1_25_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_25_00_000000_0: Counters: 15 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=1, INPUT_RECORDS_PROCESSED=20, INPUT_SPLIT_LENGTH_BYTES=282, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=191, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=20, RECORDS_OUT_INTERMEDIATE_Map_1=1]] 2018-01-05T11:28:59,431 DEBUG [TezTR-518813_1_25_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_25_0_0_0 thread id: 1668 scheme: file bytesRead: 506 bytesWritten: 399 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:59,432 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_25_00_000000_0, isTaskDone=true 2018-01-05T11:28:59,432 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000050, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_25_00_000000_0, eventCount=4 } 2018-01-05T11:28:59,432 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_25][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_25_00_000000_0, creationTime=1515180539344, allocationTime=1515180539345, startTime=1515180539347, finishTime=1515180539432, timeTaken=85, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:59,433 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:59,433 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_25_00_000000_0 2018-01-05T11:28:59,433 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_25_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:59,433 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_25_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:59,433 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_25_00_000000_0; false 2018-01-05T11:28:59,433 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_25_00_000000_0 2018-01-05T11:28:59,433 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_25_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:59,433 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_25][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_25_00_000000, startTime=1515180539347, finishTime=1515180539433, timeTaken=86, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_25_00_000000_0, diagnostics=, counters=Counters: 21, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=506, FILE_BYTES_WRITTEN=399, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=1, INPUT_RECORDS_PROCESSED=20, INPUT_SPLIT_LENGTH_BYTES=282, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=191, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=20, RECORDS_OUT_INTERMEDIATE_Map_1=1 2018-01-05T11:28:59,433 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000050=72 2018-01-05T11:28:59,433 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:59,433 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_25_00_000000_0: true 2018-01-05T11:28:59,433 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:59,433 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_25_00_000000_0 2018-01-05T11:28:59,433 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000050, QueryId=djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136-25, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:28:59,433 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_25_00 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:59,433 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_25_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:59,433 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_25_00 [Map 1] 2018-01-05T11:28:59,433 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_25_00_000000_0 2018-01-05T11:28:59,433 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_25][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_25_00, initRequestedTime=1515180539318, initedTime=1515180539344, startRequestedTime=1515180539325, startedTime=1515180539344, finishTime=1515180539433, timeTaken=89, status=SUCCEEDED, diagnostics=, counters=Counters: 21, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=506, FILE_BYTES_WRITTEN=399, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=1, INPUT_RECORDS_PROCESSED=20, INPUT_SPLIT_LENGTH_BYTES=282, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=191, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=20, RECORDS_OUT_INTERMEDIATE_Map_1=1, vertexStats=firstTaskStartTime=1515180539347, firstTasksToStart=[ task_1515180518813_0001_25_00_000000 ], lastTaskFinishTime=1515180539432, lastTasksToFinish=[ task_1515180518813_0001_25_00_000000 ], minTaskDuration=85, maxTaskDuration=85, avgTaskDuration=85.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_25_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_25_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:59,433 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_25_00 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:59,433 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_25][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000050, stoppedTime=1515180539433, exitStatus=0 2018-01-05T11:28:59,433 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000050 exited with diagnostics set to Container completed. 2018-01-05T11:28:59,433 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_25_01 [Reducer 2] attempt: attempt_1515180518813_0001_25_00_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:28:59,433 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_25_00 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2 2018-01-05T11:28:59,433 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null 2018-01-05T11:28:59,433 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000050 2018-01-05T11:28:59,433 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: All source tasks completed. Ramping up 4 remaining tasks for vertex: Reducer 2 2018-01-05T11:28:59,433 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Scheduling 4 tasks for vertex: Reducer 2 with totalTasks: 4. 1 source tasks completed out of 1. MinSourceTaskCompletedFraction: 1.0 min: 0.05 max: 0.75 2018-01-05T11:28:59,434 INFO [App Shared Pool - #0] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Map 1, destinationVertexName=Reducer 2. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:28:59,434 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_25_01 [Reducer 2] 2018-01-05T11:28:59,434 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_25][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_25_01_000003, scheduledTime=1515180539434, launchTime=1515180539434 2018-01-05T11:28:59,434 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_25][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_25_01_000000, scheduledTime=1515180539434, launchTime=1515180539434 2018-01-05T11:28:59,434 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_25][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_25_01_000001, scheduledTime=1515180539434, launchTime=1515180539434 2018-01-05T11:28:59,434 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_25][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_25_01_000002, scheduledTime=1515180539434, launchTime=1515180539434 2018-01-05T11:28:59,434 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_25_01 [Reducer 2] 2018-01-05T11:28:59,434 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_25_01_000003_0, priority=14, capability=, hosts=null 2018-01-05T11:28:59,434 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:59,434 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_25_01_000000_0, priority=14, capability=, hosts=null 2018-01-05T11:28:59,434 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=14,c=2]. totalPendingTasks=2. delayedTaskQueueSize=0 2018-01-05T11:28:59,434 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_25_01_000001_0, priority=14, capability=, hosts=null 2018-01-05T11:28:59,434 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=14,c=2]. totalPendingTasks=2. delayedTaskQueueSize=0 2018-01-05T11:28:59,434 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:59,434 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_25_01_000003_0 on hosts=null 2018-01-05T11:28:59,434 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_25_01_000003_0 on hosts=null 2018-01-05T11:28:59,434 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_25_01_000003_0 2018-01-05T11:28:59,434 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:59,434 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #51, task=TaskInfo{task=attempt_1515180518813_0001_25_01_000003_0, priority=14, startTime=0, containerId=null, uniqueId=50, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000051 2018-01-05T11:28:59,434 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_25_01_000003_0; false 2018-01-05T11:28:59,434 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_25_01_000003_0, priority=14, startTime=311559943, containerId=container_111101111_0001_00_000051assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=50, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:59,434 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_25_01_000000_0 on hosts=null 2018-01-05T11:28:59,434 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_25_01_000000_0 on hosts=null 2018-01-05T11:28:59,434 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_25_01_000000_0 2018-01-05T11:28:59,434 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:59,434 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #52, task=TaskInfo{task=attempt_1515180518813_0001_25_01_000000_0, priority=14, startTime=0, containerId=null, uniqueId=51, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, to container=container_111101111_0001_00_000052 2018-01-05T11:28:59,434 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_25_01_000000_0; false 2018-01-05T11:28:59,434 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_25_01_000000_0, priority=14, startTime=311559944, containerId=container_111101111_0001_00_000052assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=2, ac=6, commF=false, disabled=false}, uniqueId=51, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:59,435 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:59,435 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:59,435 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:59,435 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:59,435 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_25_01_000002_0, priority=14, capability=, hosts=null 2018-01-05T11:28:59,435 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:59,435 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_25_01_000001_0 on hosts=null 2018-01-05T11:28:59,435 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_25_01_000001_0 on hosts=null 2018-01-05T11:28:59,435 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_25_01_000001_0 2018-01-05T11:28:59,435 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=2, ac=6, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:59,435 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #53, task=TaskInfo{task=attempt_1515180518813_0001_25_01_000001_0, priority=14, startTime=0, containerId=null, uniqueId=52, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=2, ac=6, commF=false, disabled=false}, to container=container_111101111_0001_00_000053 2018-01-05T11:28:59,435 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_25_01_000001_0; false 2018-01-05T11:28:59,435 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_25_01_000001_0, priority=14, startTime=311559944, containerId=container_111101111_0001_00_000053assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=3, ac=5, commF=false, disabled=false}, uniqueId=52, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:59,435 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:59,435 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:59,435 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:59,435 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_25_01_000002_0 on hosts=null 2018-01-05T11:28:59,435 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_25_01_000002_0 on hosts=null 2018-01-05T11:28:59,435 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_25_01_000002_0 2018-01-05T11:28:59,435 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=3, ac=5, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:59,435 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #54, task=TaskInfo{task=attempt_1515180518813_0001_25_01_000002_0, priority=14, startTime=0, containerId=null, uniqueId=53, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=3, ac=5, commF=false, disabled=false}, to container=container_111101111_0001_00_000054 2018-01-05T11:28:59,435 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_25_01_000002_0; false 2018-01-05T11:28:59,435 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000051 for node: 10.22.8.182:49818 2018-01-05T11:28:59,435 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_25_01_000002_0, priority=14, startTime=311559944, containerId=container_111101111_0001_00_000054assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=4, ac=4, commF=false, disabled=false}, uniqueId=53, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:59,435 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000051, attempt_1515180518813_0001_25_01_000003_0 for node: 10.22.8.182:49818 2018-01-05T11:28:59,435 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000051 for node: 10.22.8.182:49818 2018-01-05T11:28:59,435 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000052 for node: 10.22.8.182:49818 2018-01-05T11:28:59,436 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000052, attempt_1515180518813_0001_25_01_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:59,436 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000052 for node: 10.22.8.182:49818 2018-01-05T11:28:59,436 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=25}, sourceName=Map 1, state=S_SUCCEEDED 2018-01-05T11:28:59,436 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000053 for node: 10.22.8.182:49818 2018-01-05T11:28:59,436 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000053, attempt_1515180518813_0001_25_01_000001_0 for node: 10.22.8.182:49818 2018-01-05T11:28:59,436 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000053 for node: 10.22.8.182:49818 2018-01-05T11:28:59,436 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000051 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:59,436 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000051, launchTime=1515180539436 2018-01-05T11:28:59,436 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000054 for node: 10.22.8.182:49818 2018-01-05T11:28:59,436 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000054, attempt_1515180518813_0001_25_01_000002_0 for node: 10.22.8.182:49818 2018-01-05T11:28:59,436 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000054 for node: 10.22.8.182:49818 2018-01-05T11:28:59,436 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_25_01_000003_0, am_details=10.22.8.182:49824, taskInfo= fragment 3 attempt 0, user=djaiswal, queryId=djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000051, dagName=analyze table cbo_t2 compute sta...c_boolean (Stage-0), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.OrderedGroupedKVInput,1}}, Outputs={{out_Reducer 2,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=5, completedTaskCount=1, dagStartTime=1515180539318, firstAttemptStartTime=1515180539434, currentAttemptStartTime=1515180539435} 2018-01-05T11:28:59,436 INFO [IPC Server handler 4 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000051, QueryId=djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136-25, DagId=25, VertexName=Reducer 2, TaskId=3, TaskAttemptId=0, SubmitTime=1515180539436 2018-01-05T11:28:59,436 DEBUG [IPC Server handler 4 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=25} with the ShuffleHandler 2018-01-05T11:28:59,437 DEBUG [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/25] 2018-01-05T11:28:59,437 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:59,437 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_25_01_000003_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:59,437 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_25_01_000003_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180539434, dagStartTime=1515180539318, withinDagPriority=14, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1}] 2018-01-05T11:28:59,437 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_25_01_000003_0 : ACCEPTED 2018-01-05T11:28:59,437 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_25_01_000003_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:59,437 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_25_01_000003_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180539434, dagStartTime=1515180539318, withinDagPriority=14, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1} 2018-01-05T11:28:59,437 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_25_01_000003_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:59,437 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_25_01_000003_0 to pre-emption queue 2018-01-05T11:28:59,437 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_25_01_000003_0: true 2018-01-05T11:28:59,437 INFO [TaskCommunicator # 0] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_25_01_000003_0 2018-01-05T11:28:59,437 INFO [TaskCommunicator # 0] GuaranteedTasks: Received response for attempt_1515180518813_0001_25_01_000003_0, true 2018-01-05T11:28:59,437 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:491159562 2018-01-05T11:28:59,437 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_25_01_000003_0] submitted. Is using containerId: [container_111101111_0001_00_000051] on NM: [10.22.8.182:49818] 2018-01-05T11:28:59,437 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:59,437 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:59,437 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_25][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_25_01_000003_0, startTime=1515180539436, containerId=container_111101111_0001_00_000051, nodeId=10.22.8.182:49818 2018-01-05T11:28:59,437 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000052 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:59,437 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000052, launchTime=1515180539437 2018-01-05T11:28:59,438 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : analyze table cbo_t2 compute sta...c_boolean (Stage-0), VertexName: Reducer 2, VertexParallelism: 4, TaskAttemptID:attempt_1515180518813_0001_25_01_000003_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:28:59,438 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1037041664, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:59,438 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_25_01_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000052, dagName=analyze table cbo_t2 compute sta...c_boolean (Stage-0), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.OrderedGroupedKVInput,1}}, Outputs={{out_Reducer 2,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=5, completedTaskCount=1, dagStartTime=1515180539318, firstAttemptStartTime=1515180539434, currentAttemptStartTime=1515180539436} 2018-01-05T11:28:59,438 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_25_01_000003_0 vertexName: Reducer 2 2018-01-05T11:28:59,438 INFO [IPC Server handler 3 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000052, QueryId=djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136-25, DagId=25, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, SubmitTime=1515180539438 2018-01-05T11:28:59,438 DEBUG [IPC Server handler 3 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=25} with the ShuffleHandler 2018-01-05T11:28:59,438 DEBUG [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/25] 2018-01-05T11:28:59,438 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_25_01_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000052] on NM: [10.22.8.182:49818] 2018-01-05T11:28:59,438 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:59,438 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000051, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_25_01_000003_0, eventCount=1 } 2018-01-05T11:28:59,438 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:59,438 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_25][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_25_01_000000_0, startTime=1515180539437, containerId=container_111101111_0001_00_000052, nodeId=10.22.8.182:49818 2018-01-05T11:28:59,438 DEBUG [TezTR-518813_1_25_1_3_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_25 queryId: djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136 instanceType: TaskRunner2Callable 2018-01-05T11:28:59,438 INFO [TezTR-518813_1_25_1_3_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_25_01_000003_0 2018-01-05T11:28:59,438 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000053 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:59,438 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=3, runningFragmentCount=1 2018-01-05T11:28:59,438 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000053, launchTime=1515180539438 2018-01-05T11:28:59,438 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_25_01_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:59,438 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_25_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180539434, dagStartTime=1515180539318, withinDagPriority=14, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1}] 2018-01-05T11:28:59,438 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_25_01_000000_0 : ACCEPTED 2018-01-05T11:28:59,438 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_25_01_000000_0, canFinish=true. Current state: preemptionQueueSize=1, numSlotsAvailable=3, waitQueueSize=1 2018-01-05T11:28:59,438 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_25_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180539434, dagStartTime=1515180539318, withinDagPriority=14, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1} 2018-01-05T11:28:59,438 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_25_01_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:59,438 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_25_01_000000_0 to pre-emption queue 2018-01-05T11:28:59,438 DEBUG [TezTR-518813_1_25_1_3_0] log.PerfLogger: 2018-01-05T11:28:59,438 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_25_01_000000_0: true 2018-01-05T11:28:59,439 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: taskOwner hashCode:2059137052 2018-01-05T11:28:59,439 INFO [TaskCommunicator # 1] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_25_01_000000_0 2018-01-05T11:28:59,439 INFO [TaskCommunicator # 1] GuaranteedTasks: Received response for attempt_1515180518813_0001_25_01_000000_0, true 2018-01-05T11:28:59,439 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_25_01_000001_0] submitted. Is using containerId: [container_111101111_0001_00_000053] on NM: [10.22.8.182:49818] 2018-01-05T11:28:59,439 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:59,439 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:59,439 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:59,439 INFO [I/O Setup 0 Initialize: {Map 1}] orderedgrouped.MergeManager: Initial Memory required for SHUFFLE_BUFFER=131404920 based on INPUT_BUFFER_FACTOR=0.7, for final merged output=0, using factor: 0.0 2018-01-05T11:28:59,439 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_25][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_25_01_000001_0, startTime=1515180539438, containerId=container_111101111_0001_00_000053, nodeId=10.22.8.182:49818 2018-01-05T11:28:59,439 INFO [Task-Executor-1] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : analyze table cbo_t2 compute sta...c_boolean (Stage-0), VertexName: Reducer 2, VertexParallelism: 4, TaskAttemptID:attempt_1515180518813_0001_25_01_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:28:59,440 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000054 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:59,440 INFO [Task-Executor-1] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1037041664, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:59,440 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000054, launchTime=1515180539440 2018-01-05T11:28:59,440 INFO [Task-Executor-1] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_25_01_000000_0 vertexName: Reducer 2 2018-01-05T11:28:59,440 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_25_01_000001_0, am_details=10.22.8.182:49824, taskInfo= fragment 1 attempt 0, user=djaiswal, queryId=djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000053, dagName=analyze table cbo_t2 compute sta...c_boolean (Stage-0), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.OrderedGroupedKVInput,1}}, Outputs={{out_Reducer 2,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=5, completedTaskCount=1, dagStartTime=1515180539318, firstAttemptStartTime=1515180539434, currentAttemptStartTime=1515180539436} 2018-01-05T11:28:59,440 INFO [IPC Server handler 1 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000053, QueryId=djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136-25, DagId=25, VertexName=Reducer 2, TaskId=1, TaskAttemptId=0, SubmitTime=1515180539440 2018-01-05T11:28:59,440 DEBUG [IPC Server handler 1 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=25} with the ShuffleHandler 2018-01-05T11:28:59,440 DEBUG [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/25] 2018-01-05T11:28:59,440 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000052, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_25_01_000000_0, eventCount=1 } 2018-01-05T11:28:59,440 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_25_01_000002_0] submitted. Is using containerId: [container_111101111_0001_00_000054] on NM: [10.22.8.182:49818] 2018-01-05T11:28:59,440 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=2, runningFragmentCount=2 2018-01-05T11:28:59,440 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:59,440 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_25_01_000001_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:59,440 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_25_01_000001_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180539434, dagStartTime=1515180539318, withinDagPriority=14, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1}] 2018-01-05T11:28:59,440 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_25_01_000001_0 : ACCEPTED 2018-01-05T11:28:59,440 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:59,441 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:59,441 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_25][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_25_01_000002_0, startTime=1515180539440, containerId=container_111101111_0001_00_000054, nodeId=10.22.8.182:49818 2018-01-05T11:28:59,441 DEBUG [TezTR-518813_1_25_1_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_25 queryId: djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136 instanceType: TaskRunner2Callable 2018-01-05T11:28:59,441 INFO [TezTR-518813_1_25_1_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_25_01_000000_0 2018-01-05T11:28:59,441 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_25_01_000001_0, canFinish=true. Current state: preemptionQueueSize=2, numSlotsAvailable=2, waitQueueSize=1 2018-01-05T11:28:59,442 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_25_01_000001_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180539434, dagStartTime=1515180539318, withinDagPriority=14, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1} 2018-01-05T11:28:59,442 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_25_01_000001_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:59,442 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_25_01_000001_0 to pre-emption queue 2018-01-05T11:28:59,442 INFO [I/O Setup 0 Initialize: {Map 1}] orderedgrouped.MergeManager: Initial Memory required for SHUFFLE_BUFFER=131404920 based on INPUT_BUFFER_FACTOR=0.7, for final merged output=0, using factor: 0.0 2018-01-05T11:28:59,441 INFO [TaskCommunicator # 3] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_25_01_000001_0 2018-01-05T11:28:59,442 INFO [TaskCommunicator # 3] GuaranteedTasks: Received response for attempt_1515180518813_0001_25_01_000001_0, true 2018-01-05T11:28:59,442 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_25_01_000001_0: true 2018-01-05T11:28:59,442 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: taskOwner hashCode:491893650 2018-01-05T11:28:59,442 INFO [IPC Server handler 23 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_25_01_000003_0 1 events [0,1) total 1 vertex_1515180518813_0001_25_01 [Reducer 2] 2018-01-05T11:28:59,442 DEBUG [TezTR-518813_1_25_1_0_0] log.PerfLogger: 2018-01-05T11:28:59,442 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:59,444 INFO [I/O Setup 1 Initialize: {out_Reducer 2}] output.MROutput: out_Reducer 2: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:28:59,444 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:59,444 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:59,444 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:59,444 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:59,443 INFO [Task-Executor-2] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : analyze table cbo_t2 compute sta...c_boolean (Stage-0), VertexName: Reducer 2, VertexParallelism: 4, TaskAttemptID:attempt_1515180518813_0001_25_01_000001_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:28:59,444 INFO [Task-Executor-2] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1037041664, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:59,444 INFO [Task-Executor-2] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_25_01_000001_0 vertexName: Reducer 2 2018-01-05T11:28:59,444 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_25_01_000002_0, am_details=10.22.8.182:49824, taskInfo= fragment 2 attempt 0, user=djaiswal, queryId=djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000054, dagName=analyze table cbo_t2 compute sta...c_boolean (Stage-0), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.OrderedGroupedKVInput,1}}, Outputs={{out_Reducer 2,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=5, completedTaskCount=1, dagStartTime=1515180539318, firstAttemptStartTime=1515180539434, currentAttemptStartTime=1515180539436} 2018-01-05T11:28:59,444 INFO [IPC Server handler 0 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000054, QueryId=djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136-25, DagId=25, VertexName=Reducer 2, TaskId=2, TaskAttemptId=0, SubmitTime=1515180539444 2018-01-05T11:28:59,444 DEBUG [IPC Server handler 0 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=25} with the ShuffleHandler 2018-01-05T11:28:59,444 DEBUG [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/25] 2018-01-05T11:28:59,444 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:59,444 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=1, runningFragmentCount=3 2018-01-05T11:28:59,445 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_25_01_000002_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:59,445 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_25_01_000002_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180539434, dagStartTime=1515180539318, withinDagPriority=14, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1}] 2018-01-05T11:28:59,445 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_25_01_000002_0 : ACCEPTED 2018-01-05T11:28:59,445 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000053, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_25_01_000001_0, eventCount=1 } 2018-01-05T11:28:59,445 DEBUG [TezTR-518813_1_25_1_3_0] log.PerfLogger: 2018-01-05T11:28:59,445 INFO [TezTR-518813_1_25_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:59,445 INFO [TezTR-518813_1_25_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:59,445 INFO [TezTR-518813_1_25_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:59,445 INFO [TezTR-518813_1_25_1_3_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:59,445 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_25_01_000002_0, canFinish=true. Current state: preemptionQueueSize=3, numSlotsAvailable=1, waitQueueSize=1 2018-01-05T11:28:59,445 INFO [TezTR-518813_1_25_1_3_0] resources.MemoryDistributor: InitialRequests=[out_Reducer 2:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.OrderedGroupedKVInput] 2018-01-05T11:28:59,445 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_25_01_000002_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180539434, dagStartTime=1515180539318, withinDagPriority=14, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1} 2018-01-05T11:28:59,445 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_25_01_000002_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:59,445 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_25_01_000002_0 to pre-emption queue 2018-01-05T11:28:59,445 INFO [TezTR-518813_1_25_1_3_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:59,445 INFO [TezTR-518813_1_25_1_3_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:59,445 INFO [TezTR-518813_1_25_1_3_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 131404920, TotalRequestedScaled: 1.2129684923076923E8, TotalJVMHeap: 1037041664, TotalAvailable: 694817914, TotalRequested/TotalJVMHeap:0.13 2018-01-05T11:28:59,445 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_25_01_000002_0: true 2018-01-05T11:28:59,445 INFO [TezTR-518813_1_25_1_3_0] resources.MemoryDistributor: Allocations=[out_Reducer 2:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [Map 1:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:131404920:131404920] 2018-01-05T11:28:59,445 INFO [TezTR-518813_1_25_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:59,445 INFO [TezTR-518813_1_25_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Input: Map 1 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:28:59,445 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: taskOwner hashCode:2033039878 2018-01-05T11:28:59,445 INFO [TezTR-518813_1_25_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:28:59,445 INFO [TezTR-518813_1_25_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:28:59,445 DEBUG [TezTR-518813_1_25_1_1_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_25 queryId: djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136 instanceType: TaskRunner2Callable 2018-01-05T11:28:59,445 INFO [TezTR-518813_1_25_1_1_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_25_01_000001_0 2018-01-05T11:28:59,445 INFO [TaskCommunicator # 2] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_25_01_000002_0 2018-01-05T11:28:59,445 INFO [TaskCommunicator # 2] GuaranteedTasks: Received response for attempt_1515180518813_0001_25_01_000002_0, true 2018-01-05T11:28:59,445 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:28:59,446 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_25_01_000003_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:28:59,446 DEBUG [TezTR-518813_1_25_1_1_0] log.PerfLogger: 2018-01-05T11:28:59,446 INFO [I/O Setup 0 Initialize: {Map 1}] orderedgrouped.MergeManager: Initial Memory required for SHUFFLE_BUFFER=131404920 based on INPUT_BUFFER_FACTOR=0.7, for final merged output=0, using factor: 0.0 2018-01-05T11:28:59,446 INFO [IPC Server handler 14 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_25_01_000000_0 1 events [0,1) total 1 vertex_1515180518813_0001_25_01 [Reducer 2] 2018-01-05T11:28:59,446 INFO [Task-Executor-3] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : analyze table cbo_t2 compute sta...c_boolean (Stage-0), VertexName: Reducer 2, VertexParallelism: 4, TaskAttemptID:attempt_1515180518813_0001_25_01_000002_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:28:59,447 INFO [Task-Executor-3] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1037041664, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:59,447 INFO [Task-Executor-3] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_25_01_000002_0 vertexName: Reducer 2 2018-01-05T11:28:59,447 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.Shuffle: Map_1: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true 2018-01-05T11:28:59,447 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.MergeManager: Map 1: MergerManager: memoryLimit=131404920, maxSingleShuffleLimit=32851230, mergeThreshold=86727248, ioSortFactor=10, postMergeMem=0, memToMemMergeOutputsThreshold=10 2018-01-05T11:28:59,447 INFO [I/O Setup 1 Initialize: {out_Reducer 2}] output.MROutput: out_Reducer 2: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:28:59,447 DEBUG [TezTR-518813_1_25_1_0_0] log.PerfLogger: 2018-01-05T11:28:59,447 INFO [TezTR-518813_1_25_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:59,447 INFO [TezTR-518813_1_25_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:59,447 INFO [TezTR-518813_1_25_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:59,447 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:59,447 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:59,447 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:59,447 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:59,447 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000054, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_25_01_000002_0, eventCount=1 } 2018-01-05T11:28:59,448 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:28:59,448 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_25_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:28:59,448 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:59,448 INFO [TezTR-518813_1_25_1_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:59,448 INFO [TezTR-518813_1_25_1_0_0] resources.MemoryDistributor: InitialRequests=[Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [out_Reducer 2:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput] 2018-01-05T11:28:59,448 INFO [TezTR-518813_1_25_1_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:59,448 INFO [TezTR-518813_1_25_1_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:59,448 DEBUG [TezTR-518813_1_25_1_2_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_25 queryId: djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136 instanceType: TaskRunner2Callable 2018-01-05T11:28:59,448 INFO [TezTR-518813_1_25_1_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 131404920, TotalRequestedScaled: 1.2129684923076923E8, TotalJVMHeap: 1037041664, TotalAvailable: 694817914, TotalRequested/TotalJVMHeap:0.13 2018-01-05T11:28:59,448 INFO [TezTR-518813_1_25_1_0_0] resources.MemoryDistributor: Allocations=[Map 1:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:131404920:131404920], [out_Reducer 2:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0] 2018-01-05T11:28:59,448 INFO [TezTR-518813_1_25_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:59,448 INFO [TezTR-518813_1_25_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: Map 1 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:28:59,449 INFO [TezTR-518813_1_25_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:28:59,449 INFO [TezTR-518813_1_25_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:28:59,449 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.Shuffle: Map_1: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true 2018-01-05T11:28:59,449 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.MergeManager: Map 1: MergerManager: memoryLimit=131404920, maxSingleShuffleLimit=32851230, mergeThreshold=86727248, ioSortFactor=10, postMergeMem=0, memToMemMergeOutputsThreshold=10 2018-01-05T11:28:59,449 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:59,448 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.ShuffleScheduler: ShuffleScheduler running for sourceVertex: Map 1 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true 2018-01-05T11:28:59,448 INFO [TezTR-518813_1_25_1_2_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_25_01_000002_0 2018-01-05T11:28:59,450 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.ShuffleScheduler: ShuffleScheduler running for sourceVertex: Map 1 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true 2018-01-05T11:28:59,450 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:59,450 INFO [I/O Setup 0 Start: {Map 1}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: Map 1 2018-01-05T11:28:59,450 INFO [IPC Server handler 13 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_25_01_000001_0 1 events [0,1) total 1 vertex_1515180518813_0001_25_01 [Reducer 2] 2018-01-05T11:28:59,451 INFO [I/O Setup 0 Start: {Map 1}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: Map 1 2018-01-05T11:28:59,451 INFO [I/O Setup 0 Initialize: {Map 1}] orderedgrouped.MergeManager: Initial Memory required for SHUFFLE_BUFFER=131404920 based on INPUT_BUFFER_FACTOR=0.7, for final merged output=0, using factor: 0.0 2018-01-05T11:28:59,451 INFO [TezTR-518813_1_25_1_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:59,451 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:59,451 INFO [TezTR-518813_1_25_1_3_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:59,451 DEBUG [TezTR-518813_1_25_1_2_0] log.PerfLogger: 2018-01-05T11:28:59,451 DEBUG [TezTR-518813_1_25_1_1_0] log.PerfLogger: 2018-01-05T11:28:59,451 INFO [TezTR-518813_1_25_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:59,451 INFO [TezTR-518813_1_25_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:59,452 INFO [TezTR-518813_1_25_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:59,452 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:28:59,452 INFO [TezTR-518813_1_25_1_3_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_25_01_000003_0 2018-01-05T11:28:59,452 DEBUG [TezTR-518813_1_25_1_3_0] log.PerfLogger: 2018-01-05T11:28:59,452 DEBUG [TezTR-518813_1_25_1_3_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_25_01_000003_0_10151 2018-01-05T11:28:59,452 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_25_01_000001_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:28:59,452 INFO [TezTR-518813_1_25_1_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_25_01_000000_0 2018-01-05T11:28:59,452 DEBUG [TezTR-518813_1_25_1_3_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:59,452 INFO [I/O Setup 1 Initialize: {out_Reducer 2}] output.MROutput: out_Reducer 2: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:28:59,452 INFO [TezTaskEventRouter{attempt_1515180518813_0001_25_01_000003_0}] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:28:59,452 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:59,452 DEBUG [TezTR-518813_1_25_1_0_0] log.PerfLogger: 2018-01-05T11:28:59,452 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:59,452 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:59,452 DEBUG [TezTR-518813_1_25_1_3_0] mr.ObjectCache: Creating djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:28:59,452 DEBUG [TezTR-518813_1_25_1_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_25_01_000000_0_10154 2018-01-05T11:28:59,453 INFO [TezTaskEventRouter{attempt_1515180518813_0001_25_01_000000_0}] orderedgrouped.ShuffleScheduler: All inputs fetched for input vertex : Map 1 2018-01-05T11:28:59,452 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:59,453 INFO [TezTaskEventRouter{attempt_1515180518813_0001_25_01_000000_0}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:28:59,453 INFO [TezTaskEventRouter{attempt_1515180518813_0001_25_01_000000_0}] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=1, numObsoletionEventsSeen=0 2018-01-05T11:28:59,453 DEBUG [TezTR-518813_1_25_1_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:59,453 DEBUG [TezTR-518813_1_25_1_0_0] mr.ObjectCache: Creating djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:28:59,453 INFO [TezTR-518813_1_25_1_3_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_090_4354755825235708462-1/djaiswal/_tez_scratch_dir/b596d131-34d9-4d74-b4f1-cc4783ba711f/reduce.xml 2018-01-05T11:28:59,453 INFO [TezTR-518813_1_25_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_090_4354755825235708462-1/djaiswal/_tez_scratch_dir/b596d131-34d9-4d74-b4f1-cc4783ba711f/reduce.xml 2018-01-05T11:28:59,453 DEBUG [TezTR-518813_1_25_1_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_090_4354755825235708462-1/djaiswal/_tez_scratch_dir/b596d131-34d9-4d74-b4f1-cc4783ba711f/reduce.xml 2018-01-05T11:28:59,453 DEBUG [TezTR-518813_1_25_1_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_090_4354755825235708462-1/djaiswal/_tez_scratch_dir/b596d131-34d9-4d74-b4f1-cc4783ba711f/reduce.xml 2018-01-05T11:28:59,453 INFO [IPC Server handler 0 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_25_01_000002_0 1 events [0,1) total 1 vertex_1515180518813_0001_25_01 [Reducer 2] 2018-01-05T11:28:59,453 DEBUG [TezTR-518813_1_25_1_3_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_090_4354755825235708462-1/djaiswal/_tez_scratch_dir/b596d131-34d9-4d74-b4f1-cc4783ba711f/reduce.xml 2018-01-05T11:28:59,453 INFO [TezTR-518813_1_25_1_1_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:59,453 DEBUG [TezTR-518813_1_25_1_0_0] log.PerfLogger: 2018-01-05T11:28:59,453 INFO [TezTR-518813_1_25_1_0_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:28:59,453 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down FetchScheduler for input: Map_1, wasInterrupted=false 2018-01-05T11:28:59,453 INFO [TezTR-518813_1_25_1_1_0] resources.MemoryDistributor: InitialRequests=[out_Reducer 2:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.OrderedGroupedKVInput] 2018-01-05T11:28:59,453 DEBUG [TezTR-518813_1_25_1_3_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_090_4354755825235708462-1/djaiswal/_tez_scratch_dir/b596d131-34d9-4d74-b4f1-cc4783ba711f/reduce.xml 2018-01-05T11:28:59,453 DEBUG [TezTR-518813_1_25_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,454 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:28:59,454 DEBUG [TezTR-518813_1_25_1_3_0] log.PerfLogger: 2018-01-05T11:28:59,454 INFO [TezTR-518813_1_25_1_3_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:28:59,454 DEBUG [TezTR-518813_1_25_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,454 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=25&reduce=3&map=attempt_1515180518813_0001_25_00_000000_0_10150 mapId: [attempt_1515180518813_0001_25_00_000000_0_10150] reduceId: [3] jobId: [job_1515180518813_0001] dagId: [25] keepAlive: false 2018-01-05T11:28:59,454 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:28:59,454 INFO [TezTR-518813_1_25_1_1_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:59,454 INFO [TezTR-518813_1_25_1_1_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:59,454 INFO [TezTR-518813_1_25_1_1_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 131404920, TotalRequestedScaled: 1.2129684923076923E8, TotalJVMHeap: 1037041664, TotalAvailable: 694817914, TotalRequested/TotalJVMHeap:0.13 2018-01-05T11:28:59,454 INFO [TezTR-518813_1_25_1_1_0] resources.MemoryDistributor: Allocations=[out_Reducer 2:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [Map 1:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:131404920:131404920] 2018-01-05T11:28:59,454 INFO [TezTR-518813_1_25_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:59,454 INFO [TezTR-518813_1_25_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Input: Map 1 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:28:59,454 INFO [TezTR-518813_1_25_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:28:59,454 INFO [TezTR-518813_1_25_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:28:59,454 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down fetchers for input: Map_1, shutdown timetaken: 0 ms, hasFetcherExecutorStopped: true 2018-01-05T11:28:59,455 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: finalMerge with #inMemoryOutputs=0, size=0 and #onDiskOutputs=0, size=0 2018-01-05T11:28:59,454 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=25&reduce=3&map=attempt_1515180518813_0001_25_00_000000_0_10150; hash=...wHlHPhLXC9ODo 2018-01-05T11:28:59,454 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_25_01_000002_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:28:59,455 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: . DiskSeg: 0, 0. MemSeg: 0, 0 2018-01-05T11:28:59,455 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.Shuffle: Map_1: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true 2018-01-05T11:28:59,455 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=25&reduce=3&map=attempt_1515180518813_0001_25_00_000000_0_10150;reply=vEBqc6KzhQ1K8 2018-01-05T11:28:59,455 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.MergeManager: Map 1: MergerManager: memoryLimit=131404920, maxSingleShuffleLimit=32851230, mergeThreshold=86727248, ioSortFactor=10, postMergeMem=0, memToMemMergeOutputsThreshold=10 2018-01-05T11:28:59,455 INFO [ShuffleAndMergeRunner {Map_1}] impl.TezMerger: Nothing to merge. Returning an empty iterator 2018-01-05T11:28:59,455 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: merge complete for input vertex : Map_1 2018-01-05T11:28:59,455 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: Map_1: Shuffle Runner thread complete 2018-01-05T11:28:59,455 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=25, user='djaiswal', attemptId='attempt_1515180518813_0001_25_00_000000_0_10150'} via loader 2018-01-05T11:28:59,455 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=25, user='djaiswal', attemptId='attempt_1515180518813_0001_25_00_000000_0_10150'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:28:59,455 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_25_00_000000_0_10150 not found 2018-01-05T11:28:59,456 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.ShuffleScheduler: ShuffleScheduler running for sourceVertex: Map 1 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true 2018-01-05T11:28:59,456 DEBUG [TezTR-518813_1_25_1_3_0] log.PerfLogger: 2018-01-05T11:28:59,456 INFO [TezTR-518813_1_25_1_3_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 2.92KB 2018-01-05T11:28:59,456 DEBUG [TezTR-518813_1_25_1_0_0] log.PerfLogger: 2018-01-05T11:28:59,456 INFO [TezTR-518813_1_25_1_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 2.92KB 2018-01-05T11:28:59,457 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_25_00_000000_0_10150,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/25/output/attempt_1515180518813_0001_25_00_000000_0_10150/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/25/output/attempt_1515180518813_0001_25_00_000000_0_10150/file.out.index 2018-01-05T11:28:59,457 INFO [I/O Setup 0 Start: {Map 1}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: Map 1 2018-01-05T11:28:59,458 DEBUG [TezTR-518813_1_25_1_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:59,458 DEBUG [TezTR-518813_1_25_1_0_0] log.PerfLogger: 2018-01-05T11:28:59,458 INFO [TezTR-518813_1_25_1_1_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:59,458 DEBUG [TezTR-518813_1_25_1_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:59,458 DEBUG [TezTR-518813_1_25_1_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:59,458 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Content Length in shuffle : 257 2018-01-05T11:28:59,458 DEBUG [TezTR-518813_1_25_1_3_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:59,458 DEBUG [TezTR-518813_1_25_1_3_0] log.PerfLogger: 2018-01-05T11:28:59,458 DEBUG [TezTR-518813_1_25_1_3_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:59,458 INFO [TezTR-518813_1_25_1_1_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_25_01_000001_0 2018-01-05T11:28:59,458 DEBUG [TezTR-518813_1_25_1_1_0] log.PerfLogger: 2018-01-05T11:28:59,458 DEBUG [TezTR-518813_1_25_1_1_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_25_01_000001_0_10157 2018-01-05T11:28:59,458 DEBUG [TezTR-518813_1_25_1_1_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:59,458 DEBUG [TezTR-518813_1_25_1_1_0] mr.ObjectCache: Creating djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:28:59,458 INFO [TezTR-518813_1_25_1_1_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_090_4354755825235708462-1/djaiswal/_tez_scratch_dir/b596d131-34d9-4d74-b4f1-cc4783ba711f/reduce.xml 2018-01-05T11:28:59,458 DEBUG [TezTR-518813_1_25_1_1_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_090_4354755825235708462-1/djaiswal/_tez_scratch_dir/b596d131-34d9-4d74-b4f1-cc4783ba711f/reduce.xml 2018-01-05T11:28:59,458 DEBUG [TezTR-518813_1_25_1_1_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_090_4354755825235708462-1/djaiswal/_tez_scratch_dir/b596d131-34d9-4d74-b4f1-cc4783ba711f/reduce.xml 2018-01-05T11:28:59,458 DEBUG [TezTR-518813_1_25_1_1_0] log.PerfLogger: 2018-01-05T11:28:59,458 INFO [TezTR-518813_1_25_1_1_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:28:59,458 DEBUG [TezTR-518813_1_25_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,458 DEBUG [New I/O worker #1] shufflehandler.FadvisedFileRegion: shuffleBufferSize: 131072, path: /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/25/output/attempt_1515180518813_0001_25_00_000000_0_10150/file.out 2018-01-05T11:28:59,458 INFO [Fetcher_O {Map_1} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=25&reduce=3&map=attempt_1515180518813_0001_25_00_000000_0_10150 sent hash and receievd reply 0 ms 2018-01-05T11:28:59,458 DEBUG [TezTR-518813_1_25_1_3_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:59,458 DEBUG [TezTR-518813_1_25_1_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:59,458 INFO [TezTR-518813_1_25_1_0_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:28:59,459 INFO [TezTR-518813_1_25_1_0_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:28:59,459 DEBUG [TezTR-518813_1_25_1_2_0] log.PerfLogger: 2018-01-05T11:28:59,459 INFO [TezTR-518813_1_25_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:59,459 INFO [TezTaskEventRouter{attempt_1515180518813_0001_25_01_000001_0}] orderedgrouped.ShuffleScheduler: All inputs fetched for input vertex : Map 1 2018-01-05T11:28:59,459 INFO [TezTR-518813_1_25_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:59,459 INFO [TezTR-518813_1_25_1_0_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:59,459 INFO [TezTaskEventRouter{attempt_1515180518813_0001_25_01_000001_0}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:28:59,459 INFO [Fetcher_O {Map_1} #0] orderedgrouped.MergeManager: CloseInMemoryFile. Current state: inMemoryMapOutputs.size=1, commitMemory=0, usedMemory=200. Since last log: count=1, min=200, max=200, total=200, avg=200.0 2018-01-05T11:28:59,459 INFO [TezTR-518813_1_25_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:59,459 INFO [Fetcher_O {Map_1} #0] ShuffleScheduler.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_25_00_000000_0_10150} to MEMORY, csize=204, dsize=200, EndTime=1515180539459, TimeTaken=1, Rate=0.19 MB/s 2018-01-05T11:28:59,459 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down FetchScheduler for input: Map_1, wasInterrupted=false 2018-01-05T11:28:59,459 INFO [TezTaskEventRouter{attempt_1515180518813_0001_25_01_000001_0}] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=1, numObsoletionEventsSeen=0 2018-01-05T11:28:59,459 DEBUG [TezTR-518813_1_25_1_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:28:59,459 INFO [Fetcher_O {Map_1} #0] orderedgrouped.ShuffleScheduler: All inputs fetched for input vertex : Map 1 2018-01-05T11:28:59,460 DEBUG [TezTR-518813_1_25_1_3_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:59,460 INFO [TezTR-518813_1_25_1_0_0] input.OrderedGroupedKVInput: Map 1: creating ValuesIterator with comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator, keyClass=org.apache.hadoop.hive.ql.io.HiveKey, valClass=org.apache.hadoop.io.BytesWritable 2018-01-05T11:28:59,460 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:28:59,460 INFO [TezTR-518813_1_25_1_3_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:28:59,460 INFO [Fetcher_O {Map_1} #0] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:28:59,460 DEBUG [TezTR-518813_1_25_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,460 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down fetchers for input: Map_1, shutdown timetaken: 0 ms, hasFetcherExecutorStopped: true 2018-01-05T11:28:59,460 INFO [I/O Setup 1 Initialize: {out_Reducer 2}] output.MROutput: out_Reducer 2: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:28:59,460 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down FetchScheduler for input: Map_1, wasInterrupted=false 2018-01-05T11:28:59,460 DEBUG [TezTR-518813_1_25_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,460 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:59,460 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:59,461 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:28:59,460 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: finalMerge with #inMemoryOutputs=0, size=0 and #onDiskOutputs=0, size=0 2018-01-05T11:28:59,461 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:59,461 DEBUG [TezTR-518813_1_25_1_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1, _col2, _col3, _col4] columnTypes=[struct, struct, struct, struct, struct] 2018-01-05T11:28:59,461 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down fetchers for input: Map_1, shutdown timetaken: 0 ms, hasFetcherExecutorStopped: true 2018-01-05T11:28:59,461 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: . DiskSeg: 0, 0. MemSeg: 0, 0 2018-01-05T11:28:59,461 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:59,461 DEBUG [TezTR-518813_1_25_1_0_0] log.PerfLogger: 2018-01-05T11:28:59,461 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: TotalInMemFetchStats: count=1, totalSize=200, min=200, max=200, avg=1.0 2018-01-05T11:28:59,461 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: finalMerge with #inMemoryOutputs=1, size=200 and #onDiskOutputs=0, size=0 2018-01-05T11:28:59,461 INFO [ShuffleAndMergeRunner {Map_1}] impl.TezMerger: Nothing to merge. Returning an empty iterator 2018-01-05T11:28:59,461 INFO [TezTR-518813_1_25_1_2_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:59,461 INFO [TezTR-518813_1_25_1_0_0] exec.GroupByOperator: Initializing operator GBY[4] 2018-01-05T11:28:59,461 DEBUG [TezTR-518813_1_25_1_1_0] log.PerfLogger: 2018-01-05T11:28:59,461 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: merge complete for input vertex : Map_1 2018-01-05T11:28:59,461 INFO [TezTR-518813_1_25_1_2_0] resources.MemoryDistributor: InitialRequests=[Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [out_Reducer 2:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput] 2018-01-05T11:28:59,461 INFO [TezTR-518813_1_25_1_1_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 2.92KB 2018-01-05T11:28:59,461 DEBUG [TezTR-518813_1_25_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,461 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: Map_1: Shuffle Runner thread complete 2018-01-05T11:28:59,461 DEBUG [TezTR-518813_1_25_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,461 DEBUG [TezTR-518813_1_25_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,461 INFO [TezTR-518813_1_25_1_2_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:59,462 DEBUG [TezTR-518813_1_25_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,462 DEBUG [TezTR-518813_1_25_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,462 INFO [TezTR-518813_1_25_1_2_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:59,462 DEBUG [TezTR-518813_1_25_1_1_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:59,462 INFO [TezTR-518813_1_25_1_2_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 131404920, TotalRequestedScaled: 1.2129684923076923E8, TotalJVMHeap: 1037041664, TotalAvailable: 694817914, TotalRequested/TotalJVMHeap:0.13 2018-01-05T11:28:59,462 DEBUG [TezTR-518813_1_25_1_1_0] log.PerfLogger: 2018-01-05T11:28:59,462 INFO [TezTR-518813_1_25_1_2_0] resources.MemoryDistributor: Allocations=[Map 1:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:131404920:131404920], [out_Reducer 2:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0] 2018-01-05T11:28:59,462 INFO [TezTR-518813_1_25_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:59,462 DEBUG [TezTR-518813_1_25_1_1_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:59,462 DEBUG [TezTR-518813_1_25_1_1_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:59,462 INFO [TezTR-518813_1_25_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Input: Map 1 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:28:59,462 INFO [TezTR-518813_1_25_1_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:59,462 DEBUG [TezTR-518813_1_25_1_0_0] exec.GroupByOperator: Initialization Done 4 GBY 2018-01-05T11:28:59,462 INFO [TezTR-518813_1_25_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:28:59,462 INFO [TezTR-518813_1_25_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:28:59,462 DEBUG [TezTR-518813_1_25_1_0_0] exec.GroupByOperator: Operator 4 GBY initialized 2018-01-05T11:28:59,462 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.Shuffle: Map_1: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true 2018-01-05T11:28:59,462 DEBUG [TezTR-518813_1_25_1_0_0] exec.GroupByOperator: Initializing children of 4 GBY 2018-01-05T11:28:59,462 DEBUG [TezTR-518813_1_25_1_0_0] exec.SelectOperator: Initializing child 5 SEL 2018-01-05T11:28:59,462 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.MergeManager: Map 1: MergerManager: memoryLimit=131404920, maxSingleShuffleLimit=32851230, mergeThreshold=86727248, ioSortFactor=10, postMergeMem=0, memToMemMergeOutputsThreshold=10 2018-01-05T11:28:59,462 INFO [TezTR-518813_1_25_1_0_0] exec.SelectOperator: Initializing operator SEL[5] 2018-01-05T11:28:59,462 INFO [TezTR-518813_1_25_1_0_0] exec.SelectOperator: SELECT struct<_col0:string,_col1:struct,_col2:struct,_col3:struct,_col4:struct,_col5:struct> 2018-01-05T11:28:59,462 DEBUG [TezTR-518813_1_25_1_0_0] exec.SelectOperator: Initialization Done 5 SEL 2018-01-05T11:28:59,462 DEBUG [TezTR-518813_1_25_1_0_0] exec.SelectOperator: Operator 5 SEL initialized 2018-01-05T11:28:59,462 DEBUG [TezTR-518813_1_25_1_0_0] exec.SelectOperator: Initializing children of 5 SEL 2018-01-05T11:28:59,462 DEBUG [TezTR-518813_1_25_1_0_0] exec.FileSinkOperator: Initializing child 6 FS 2018-01-05T11:28:59,462 INFO [TezTR-518813_1_25_1_0_0] exec.FileSinkOperator: Initializing operator FS[6] 2018-01-05T11:28:59,462 DEBUG [TezTR-518813_1_25_1_1_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:59,462 INFO [TezTR-518813_1_25_1_1_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:28:59,462 INFO [TezTR-518813_1_25_1_1_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:28:59,462 DEBUG [TezTR-518813_1_25_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,462 INFO [TezTR-518813_1_25_1_1_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:59,462 DEBUG [TezTR-518813_1_25_1_1_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:28:59,462 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.ShuffleScheduler: ShuffleScheduler running for sourceVertex: Map 1 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true 2018-01-05T11:28:59,462 INFO [TezTR-518813_1_25_1_1_0] input.OrderedGroupedKVInput: Map 1: creating ValuesIterator with comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator, keyClass=org.apache.hadoop.hive.ql.io.HiveKey, valClass=org.apache.hadoop.io.BytesWritable 2018-01-05T11:28:59,463 DEBUG [TezTR-518813_1_25_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,463 DEBUG [TezTR-518813_1_25_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,463 INFO [TezTR-518813_1_25_1_0_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@62f70f69]:[_col0, _col1, _col2, _col3, _col4, _col5]:[struct, struct, struct, struct, struct, string]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@32f81fc7 2018-01-05T11:28:59,463 INFO [I/O Setup 0 Start: {Map 1}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: Map 1 2018-01-05T11:28:59,463 DEBUG [TezTR-518813_1_25_1_0_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:28:59,463 DEBUG [TezTR-518813_1_25_1_1_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1, _col2, _col3, _col4] columnTypes=[struct, struct, struct, struct, struct] 2018-01-05T11:28:59,463 INFO [TezTR-518813_1_25_1_2_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:59,463 DEBUG [TezTR-518813_1_25_1_1_0] log.PerfLogger: 2018-01-05T11:28:59,463 DEBUG [TezTR-518813_1_25_1_0_0] exec.FileSinkOperator: Initialization Done 6 FS 2018-01-05T11:28:59,463 DEBUG [TezTR-518813_1_25_1_0_0] exec.FileSinkOperator: Operator 6 FS initialized 2018-01-05T11:28:59,463 DEBUG [TezTR-518813_1_25_1_0_0] exec.FileSinkOperator: Initialization Done 6 FS done is reset. 2018-01-05T11:28:59,463 DEBUG [TezTR-518813_1_25_1_0_0] exec.SelectOperator: Initialization Done 5 SEL done is reset. 2018-01-05T11:28:59,463 INFO [TezTR-518813_1_25_1_1_0] exec.GroupByOperator: Initializing operator GBY[4] 2018-01-05T11:28:59,463 DEBUG [TezTR-518813_1_25_1_0_0] exec.GroupByOperator: Initialization Done 4 GBY done is reset. 2018-01-05T11:28:59,463 INFO [TezTR-518813_1_25_1_0_0] tez.ReduceRecordProcessor: Id =4 Id =5 Id =6 <\Children> Id = 5 null<\Parent> <\FS> <\Children> Id = 4 null<\Parent> <\SEL> <\Children> <\Parent> <\GBY> 2018-01-05T11:28:59,463 DEBUG [TezTR-518813_1_25_1_0_0] log.PerfLogger: 2018-01-05T11:28:59,463 INFO [TezTR-518813_1_25_1_2_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_25_01_000002_0 2018-01-05T11:28:59,464 DEBUG [TezTR-518813_1_25_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,464 INFO [TezTR-518813_1_25_1_0_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 2 2018-01-05T11:28:59,464 DEBUG [TezTR-518813_1_25_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,464 DEBUG [TezTR-518813_1_25_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,464 DEBUG [TezTR-518813_1_25_1_0_0] log.PerfLogger: 2018-01-05T11:28:59,464 DEBUG [TezTR-518813_1_25_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,464 DEBUG [TezTR-518813_1_25_1_0_0] mr.ObjectCache: djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:28:59,464 DEBUG [TezTR-518813_1_25_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,464 DEBUG [TezTR-518813_1_25_1_0_0] exec.GroupByOperator: close called for operator GBY[4] 2018-01-05T11:28:59,464 INFO [TezTR-518813_1_25_1_0_0] exec.GroupByOperator: Closing operator GBY[4] 2018-01-05T11:28:59,464 DEBUG [TezTR-518813_1_25_1_2_0] log.PerfLogger: 2018-01-05T11:28:59,464 INFO [TezTaskEventRouter{attempt_1515180518813_0001_25_01_000002_0}] orderedgrouped.ShuffleScheduler: All inputs fetched for input vertex : Map 1 2018-01-05T11:28:59,464 DEBUG [TezTR-518813_1_25_1_2_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_25_01_000002_0_10160 2018-01-05T11:28:59,464 DEBUG [TezTR-518813_1_25_1_0_0] exec.GroupByOperator: Closing child = SEL[5] 2018-01-05T11:28:59,464 DEBUG [TezTR-518813_1_25_1_2_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:59,464 DEBUG [TezTR-518813_1_25_1_0_0] exec.SelectOperator: close called for operator SEL[5] 2018-01-05T11:28:59,464 DEBUG [TezTR-518813_1_25_1_0_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:59,464 INFO [TezTR-518813_1_25_1_0_0] exec.SelectOperator: Closing operator SEL[5] 2018-01-05T11:28:59,464 INFO [TezTR-518813_1_25_1_1_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:59,464 DEBUG [TezTR-518813_1_25_1_1_0] exec.GroupByOperator: Initialization Done 4 GBY 2018-01-05T11:28:59,464 DEBUG [TezTR-518813_1_25_1_1_0] exec.GroupByOperator: Operator 4 GBY initialized 2018-01-05T11:28:59,464 DEBUG [TezTR-518813_1_25_1_1_0] exec.GroupByOperator: Initializing children of 4 GBY 2018-01-05T11:28:59,464 DEBUG [TezTR-518813_1_25_1_1_0] exec.SelectOperator: Initializing child 5 SEL 2018-01-05T11:28:59,464 INFO [TezTR-518813_1_25_1_1_0] exec.SelectOperator: Initializing operator SEL[5] 2018-01-05T11:28:59,464 DEBUG [TezTR-518813_1_25_1_2_0] mr.ObjectCache: Creating djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:28:59,464 INFO [TezTaskEventRouter{attempt_1515180518813_0001_25_01_000002_0}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:28:59,464 INFO [TezTaskEventRouter{attempt_1515180518813_0001_25_01_000002_0}] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=1, numObsoletionEventsSeen=0 2018-01-05T11:28:59,464 INFO [TezTR-518813_1_25_1_2_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_090_4354755825235708462-1/djaiswal/_tez_scratch_dir/b596d131-34d9-4d74-b4f1-cc4783ba711f/reduce.xml 2018-01-05T11:28:59,464 DEBUG [TezTR-518813_1_25_1_0_0] exec.SelectOperator: Closing child = FS[6] 2018-01-05T11:28:59,464 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down FetchScheduler for input: Map_1, wasInterrupted=false 2018-01-05T11:28:59,464 DEBUG [TezTR-518813_1_25_1_2_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_090_4354755825235708462-1/djaiswal/_tez_scratch_dir/b596d131-34d9-4d74-b4f1-cc4783ba711f/reduce.xml 2018-01-05T11:28:59,464 INFO [TezTR-518813_1_25_1_1_0] exec.SelectOperator: SELECT struct<_col0:string,_col1:struct,_col2:struct,_col3:struct,_col4:struct,_col5:struct> 2018-01-05T11:28:59,464 DEBUG [TezTR-518813_1_25_1_2_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_090_4354755825235708462-1/djaiswal/_tez_scratch_dir/b596d131-34d9-4d74-b4f1-cc4783ba711f/reduce.xml 2018-01-05T11:28:59,465 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:28:59,465 DEBUG [TezTR-518813_1_25_1_1_0] exec.SelectOperator: Initialization Done 5 SEL 2018-01-05T11:28:59,465 DEBUG [TezTR-518813_1_25_1_1_0] exec.SelectOperator: Operator 5 SEL initialized 2018-01-05T11:28:59,465 DEBUG [TezTR-518813_1_25_1_0_0] exec.FileSinkOperator: close called for operator FS[6] 2018-01-05T11:28:59,465 DEBUG [TezTR-518813_1_25_1_2_0] log.PerfLogger: 2018-01-05T11:28:59,465 DEBUG [TezTR-518813_1_25_1_1_0] exec.SelectOperator: Initializing children of 5 SEL 2018-01-05T11:28:59,465 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down fetchers for input: Map_1, shutdown timetaken: 0 ms, hasFetcherExecutorStopped: true 2018-01-05T11:28:59,465 DEBUG [TezTR-518813_1_25_1_0_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:59,465 INFO [TezTR-518813_1_25_1_0_0] exec.FileSinkOperator: Closing operator FS[6] 2018-01-05T11:28:59,465 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: finalMerge with #inMemoryOutputs=0, size=0 and #onDiskOutputs=0, size=0 2018-01-05T11:28:59,465 DEBUG [TezTR-518813_1_25_1_1_0] exec.FileSinkOperator: Initializing child 6 FS 2018-01-05T11:28:59,465 INFO [TezTR-518813_1_25_1_1_0] exec.FileSinkOperator: Initializing operator FS[6] 2018-01-05T11:28:59,465 INFO [TezTR-518813_1_25_1_2_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:28:59,465 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: . DiskSeg: 0, 0. MemSeg: 0, 0 2018-01-05T11:28:59,465 INFO [TezTR-518813_1_25_1_0_0] exec.FileSinkOperator: FS[6]: records written - 0 2018-01-05T11:28:59,465 DEBUG [TezTR-518813_1_25_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,465 INFO [ShuffleAndMergeRunner {Map_1}] impl.TezMerger: Nothing to merge. Returning an empty iterator 2018-01-05T11:28:59,465 INFO [TezTR-518813_1_25_1_0_0] exec.FileSinkOperator: RECORDS_OUT_0:0, 2018-01-05T11:28:59,465 DEBUG [TezTR-518813_1_25_1_0_0] exec.FileSinkOperator: 6 Close done 2018-01-05T11:28:59,465 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: merge complete for input vertex : Map_1 2018-01-05T11:28:59,465 DEBUG [TezTR-518813_1_25_1_0_0] exec.SelectOperator: 5 Close done 2018-01-05T11:28:59,465 DEBUG [TezTR-518813_1_25_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,465 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: Map_1: Shuffle Runner thread complete 2018-01-05T11:28:59,465 DEBUG [TezTR-518813_1_25_1_0_0] exec.GroupByOperator: 4 Close done 2018-01-05T11:28:59,466 INFO [TezTR-518813_1_25_1_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_25_01_000000_0 2018-01-05T11:28:59,466 INFO [TezTR-518813_1_25_1_0_0] orderedgrouped.Shuffle: Shutting down Shuffle for source: Map_1 2018-01-05T11:28:59,466 INFO [TezTR-518813_1_25_1_0_0] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=1, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:28:59,466 INFO [TezTR-518813_1_25_1_0_0] output.MROutput: out_Reducer 2 closed 2018-01-05T11:28:59,466 DEBUG [TezTR-518813_1_25_1_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:59,466 INFO [TezTR-518813_1_25_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:59,466 INFO [TezTR-518813_1_25_1_1_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@b58090f]:[_col0, _col1, _col2, _col3, _col4, _col5]:[struct, struct, struct, struct, struct, string]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@2fa713fc 2018-01-05T11:28:59,466 DEBUG [TezTR-518813_1_25_1_1_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:28:59,466 DEBUG [TezTR-518813_1_25_1_1_0] exec.FileSinkOperator: Initialization Done 6 FS 2018-01-05T11:28:59,466 DEBUG [TezTR-518813_1_25_1_1_0] exec.FileSinkOperator: Operator 6 FS initialized 2018-01-05T11:28:59,466 INFO [TezTR-518813_1_25_1_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_25_01_000000_0, askedToStop=false 2018-01-05T11:28:59,466 DEBUG [TezTR-518813_1_25_1_1_0] exec.FileSinkOperator: Initialization Done 6 FS done is reset. 2018-01-05T11:28:59,466 DEBUG [TezTR-518813_1_25_1_1_0] exec.SelectOperator: Initialization Done 5 SEL done is reset. 2018-01-05T11:28:59,466 DEBUG [TezTR-518813_1_25_1_1_0] exec.GroupByOperator: Initialization Done 4 GBY done is reset. 2018-01-05T11:28:59,466 INFO [TezTR-518813_1_25_1_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_25_01_000000_0, stopRequested=false 2018-01-05T11:28:59,466 INFO [TezTR-518813_1_25_1_1_0] tez.ReduceRecordProcessor: Id =4 Id =5 Id =6 <\Children> Id = 5 null<\Parent> <\FS> <\Children> Id = 4 null<\Parent> <\SEL> <\Children> <\Parent> <\GBY> 2018-01-05T11:28:59,466 DEBUG [TezTR-518813_1_25_1_1_0] log.PerfLogger: 2018-01-05T11:28:59,466 INFO [TezTR-518813_1_25_1_1_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 2 2018-01-05T11:28:59,466 DEBUG [TezTR-518813_1_25_1_1_0] log.PerfLogger: 2018-01-05T11:28:59,466 DEBUG [TezTR-518813_1_25_1_1_0] mr.ObjectCache: djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:28:59,466 INFO [TezTR-518813_1_25_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_25_01_000000_0: Counters: 29 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=0, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=0, NUM_SKIPPED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=0, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, SHUFFLE_BYTES=0, SHUFFLE_BYTES_DECOMPRESSED=0, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][HIVE RECORDS_OUT_0=0][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0]] 2018-01-05T11:28:59,466 DEBUG [TezTR-518813_1_25_1_1_0] exec.GroupByOperator: close called for operator GBY[4] 2018-01-05T11:28:59,466 INFO [TezTR-518813_1_25_1_1_0] exec.GroupByOperator: Closing operator GBY[4] 2018-01-05T11:28:59,466 DEBUG [TezTR-518813_1_25_1_1_0] exec.GroupByOperator: Closing child = SEL[5] 2018-01-05T11:28:59,466 DEBUG [TezTR-518813_1_25_1_1_0] exec.SelectOperator: close called for operator SEL[5] 2018-01-05T11:28:59,466 DEBUG [TezTR-518813_1_25_1_1_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:59,466 INFO [TezTR-518813_1_25_1_1_0] exec.SelectOperator: Closing operator SEL[5] 2018-01-05T11:28:59,466 DEBUG [TezTR-518813_1_25_1_1_0] exec.SelectOperator: Closing child = FS[6] 2018-01-05T11:28:59,466 DEBUG [TezTR-518813_1_25_1_1_0] exec.FileSinkOperator: close called for operator FS[6] 2018-01-05T11:28:59,466 DEBUG [TezTR-518813_1_25_1_1_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:59,466 INFO [TezTR-518813_1_25_1_1_0] exec.FileSinkOperator: Closing operator FS[6] 2018-01-05T11:28:59,466 INFO [TezTR-518813_1_25_1_1_0] exec.FileSinkOperator: FS[6]: records written - 0 2018-01-05T11:28:59,466 INFO [TezTR-518813_1_25_1_1_0] exec.FileSinkOperator: RECORDS_OUT_0:0, 2018-01-05T11:28:59,466 DEBUG [TezTR-518813_1_25_1_1_0] exec.FileSinkOperator: 6 Close done 2018-01-05T11:28:59,466 DEBUG [TezTR-518813_1_25_1_1_0] exec.SelectOperator: 5 Close done 2018-01-05T11:28:59,466 DEBUG [TezTR-518813_1_25_1_1_0] exec.GroupByOperator: 4 Close done 2018-01-05T11:28:59,466 DEBUG [TezTR-518813_1_25_1_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_25_1_0_0 thread id: 1686 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:59,466 INFO [TezTR-518813_1_25_1_1_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_25_01_000001_0 2018-01-05T11:28:59,466 INFO [TezTR-518813_1_25_1_1_0] orderedgrouped.Shuffle: Shutting down Shuffle for source: Map_1 2018-01-05T11:28:59,466 INFO [TezTR-518813_1_25_1_1_0] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=1, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:28:59,466 INFO [TezTR-518813_1_25_1_1_0] output.MROutput: out_Reducer 2 closed 2018-01-05T11:28:59,466 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_25_01_000000_0, isTaskDone=true 2018-01-05T11:28:59,466 DEBUG [TezTR-518813_1_25_1_1_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:59,466 INFO [TezTR-518813_1_25_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:59,466 DEBUG [TezTR-518813_1_25_1_2_0] log.PerfLogger: 2018-01-05T11:28:59,466 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000052, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_25_01_000000_0, eventCount=2 } 2018-01-05T11:28:59,466 INFO [TezTR-518813_1_25_1_2_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 2.92KB 2018-01-05T11:28:59,466 INFO [TezTR-518813_1_25_1_1_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_25_01_000001_0, askedToStop=false 2018-01-05T11:28:59,466 INFO [TezTR-518813_1_25_1_1_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_25_01_000001_0, stopRequested=false 2018-01-05T11:28:59,467 INFO [TezTR-518813_1_25_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_25_01_000001_0: Counters: 29 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=0, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=0, NUM_SKIPPED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=0, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, SHUFFLE_BYTES=0, SHUFFLE_BYTES_DECOMPRESSED=0, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][HIVE RECORDS_OUT_0=0][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0]] 2018-01-05T11:28:59,467 DEBUG [TezTR-518813_1_25_1_2_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:59,467 DEBUG [TezTR-518813_1_25_1_2_0] log.PerfLogger: 2018-01-05T11:28:59,467 DEBUG [TezTR-518813_1_25_1_1_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_25_1_1_0 thread id: 1692 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:59,467 DEBUG [TezTR-518813_1_25_1_2_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:59,467 DEBUG [TezTR-518813_1_25_1_2_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:59,467 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_25_01_000001_0, isTaskDone=true 2018-01-05T11:28:59,467 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000053, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_25_01_000001_0, eventCount=2 } 2018-01-05T11:28:59,467 DEBUG [TezTR-518813_1_25_1_2_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:59,467 INFO [TezTR-518813_1_25_1_2_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:28:59,467 INFO [TezTR-518813_1_25_1_2_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:28:59,467 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_25][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_25_01_000000_0, creationTime=1515180539434, allocationTime=1515180539436, startTime=1515180539437, finishTime=1515180539467, timeTaken=30, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:59,467 INFO [TezTR-518813_1_25_1_2_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:59,467 DEBUG [TezTR-518813_1_25_1_2_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:28:59,467 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_25][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_25_01_000001_0, creationTime=1515180539434, allocationTime=1515180539436, startTime=1515180539438, finishTime=1515180539467, timeTaken=29, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:59,467 INFO [TezTR-518813_1_25_1_2_0] input.OrderedGroupedKVInput: Map 1: creating ValuesIterator with comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator, keyClass=org.apache.hadoop.hive.ql.io.HiveKey, valClass=org.apache.hadoop.io.BytesWritable 2018-01-05T11:28:59,467 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:28:59,467 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_25_01_000000_0 2018-01-05T11:28:59,467 DEBUG [TezTR-518813_1_25_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,467 INFO [Task-Executor-2] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_25_01_000001_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:59,467 INFO [Task-Executor-2] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_25_01_000001_0 2018-01-05T11:28:59,467 DEBUG [TezTR-518813_1_25_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,467 INFO [Task-Executor-2] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000053=25 2018-01-05T11:28:59,467 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_25_01_000001_0: true 2018-01-05T11:28:59,467 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_25_01_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:59,468 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_25_01_000000_0; false 2018-01-05T11:28:59,468 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:28:59,468 DEBUG [TezTR-518813_1_25_1_2_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1, _col2, _col3, _col4] columnTypes=[struct, struct, struct, struct, struct] 2018-01-05T11:28:59,468 INFO [Task-Executor-1] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_25_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:59,468 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_25_01_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:59,468 DEBUG [TezTR-518813_1_25_1_2_0] log.PerfLogger: 2018-01-05T11:28:59,468 INFO [TezTR-518813_1_25_1_2_0] exec.GroupByOperator: Initializing operator GBY[4] 2018-01-05T11:28:59,468 INFO [Task-Executor-1] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_25_01_000000_0 2018-01-05T11:28:59,468 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:59,468 INFO [Task-Executor-1] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000052=29 2018-01-05T11:28:59,468 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_25_01_000000_0: true 2018-01-05T11:28:59,468 DEBUG [TezTR-518813_1_25_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,468 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_25][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_25_01_000000, startTime=1515180539437, finishTime=1515180539467, timeTaken=30, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_25_01_000000_0, diagnostics=, counters=Counters: 34, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=0, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=0, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=0, NUM_SKIPPED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=0, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, SHUFFLE_BYTES=0, SHUFFLE_BYTES_DECOMPRESSED=0, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, HIVE, RECORDS_OUT_0=0, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0 2018-01-05T11:28:59,468 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:59,468 DEBUG [TezTR-518813_1_25_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,468 DEBUG [TezTR-518813_1_25_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,468 DEBUG [TezTR-518813_1_25_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,468 DEBUG [TezTR-518813_1_25_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,468 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_25_01_000001_0 2018-01-05T11:28:59,468 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_25_01_000001_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:59,468 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_25_01_000001_0; false 2018-01-05T11:28:59,468 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_25_01_000001_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:59,468 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:59,468 INFO [TezTR-518813_1_25_1_2_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:59,468 DEBUG [TezTR-518813_1_25_1_2_0] exec.GroupByOperator: Initialization Done 4 GBY 2018-01-05T11:28:59,468 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_25][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_25_01_000001, startTime=1515180539438, finishTime=1515180539468, timeTaken=30, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_25_01_000001_0, diagnostics=, counters=Counters: 34, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=0, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=0, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=0, NUM_SKIPPED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=0, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, SHUFFLE_BYTES=0, SHUFFLE_BYTES_DECOMPRESSED=0, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, HIVE, RECORDS_OUT_0=0, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0 2018-01-05T11:28:59,468 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:59,468 DEBUG [TezTR-518813_1_25_1_2_0] exec.GroupByOperator: Operator 4 GBY initialized 2018-01-05T11:28:59,468 DEBUG [TezTR-518813_1_25_1_2_0] exec.GroupByOperator: Initializing children of 4 GBY 2018-01-05T11:28:59,468 DEBUG [TezTR-518813_1_25_1_2_0] exec.SelectOperator: Initializing child 5 SEL 2018-01-05T11:28:59,468 INFO [TezTR-518813_1_25_1_2_0] exec.SelectOperator: Initializing operator SEL[5] 2018-01-05T11:28:59,468 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_25_01 [Reducer 2], tasks=4, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:59,468 INFO [TezTR-518813_1_25_1_2_0] exec.SelectOperator: SELECT struct<_col0:string,_col1:struct,_col2:struct,_col3:struct,_col4:struct,_col5:struct> 2018-01-05T11:28:59,468 DEBUG [TezTR-518813_1_25_1_2_0] exec.SelectOperator: Initialization Done 5 SEL 2018-01-05T11:28:59,468 DEBUG [TezTR-518813_1_25_1_2_0] exec.SelectOperator: Operator 5 SEL initialized 2018-01-05T11:28:59,468 DEBUG [TezTR-518813_1_25_1_2_0] exec.SelectOperator: Initializing children of 5 SEL 2018-01-05T11:28:59,468 DEBUG [TezTR-518813_1_25_1_2_0] exec.FileSinkOperator: Initializing child 6 FS 2018-01-05T11:28:59,468 INFO [TezTR-518813_1_25_1_2_0] exec.FileSinkOperator: Initializing operator FS[6] 2018-01-05T11:28:59,468 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_25][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000052, stoppedTime=1515180539468, exitStatus=0 2018-01-05T11:28:59,468 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000052 exited with diagnostics set to Container completed. 2018-01-05T11:28:59,468 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_25_01 [Reducer 2], tasks=4, failed=0, killed=0, success=2, completed=2, commits=0, err=null 2018-01-05T11:28:59,468 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_25][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000053, stoppedTime=1515180539468, exitStatus=0 2018-01-05T11:28:59,468 DEBUG [TezTR-518813_1_25_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,468 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000053 exited with diagnostics set to Container completed. 2018-01-05T11:28:59,468 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000052 2018-01-05T11:28:59,468 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000053 2018-01-05T11:28:59,468 INFO [TezTR-518813_1_25_1_2_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@1cc03eca]:[_col0, _col1, _col2, _col3, _col4, _col5]:[struct, struct, struct, struct, struct, string]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@ca29e88 2018-01-05T11:28:59,468 DEBUG [TezTR-518813_1_25_1_2_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:28:59,469 DEBUG [TezTR-518813_1_25_1_2_0] exec.FileSinkOperator: Initialization Done 6 FS 2018-01-05T11:28:59,469 DEBUG [TezTR-518813_1_25_1_2_0] exec.FileSinkOperator: Operator 6 FS initialized 2018-01-05T11:28:59,469 DEBUG [TezTR-518813_1_25_1_2_0] exec.FileSinkOperator: Initialization Done 6 FS done is reset. 2018-01-05T11:28:59,469 DEBUG [TezTR-518813_1_25_1_2_0] exec.SelectOperator: Initialization Done 5 SEL done is reset. 2018-01-05T11:28:59,469 DEBUG [TezTR-518813_1_25_1_2_0] exec.GroupByOperator: Initialization Done 4 GBY done is reset. 2018-01-05T11:28:59,469 INFO [TezTR-518813_1_25_1_2_0] tez.ReduceRecordProcessor: Id =4 Id =5 Id =6 <\Children> Id = 5 null<\Parent> <\FS> <\Children> Id = 4 null<\Parent> <\SEL> <\Children> <\Parent> <\GBY> 2018-01-05T11:28:59,469 DEBUG [TezTR-518813_1_25_1_2_0] log.PerfLogger: 2018-01-05T11:28:59,469 INFO [TezTR-518813_1_25_1_2_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 2 2018-01-05T11:28:59,469 DEBUG [TezTR-518813_1_25_1_2_0] log.PerfLogger: 2018-01-05T11:28:59,469 DEBUG [TezTR-518813_1_25_1_2_0] mr.ObjectCache: djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:28:59,469 DEBUG [TezTR-518813_1_25_1_2_0] exec.GroupByOperator: close called for operator GBY[4] 2018-01-05T11:28:59,469 INFO [TezTR-518813_1_25_1_2_0] exec.GroupByOperator: Closing operator GBY[4] 2018-01-05T11:28:59,469 DEBUG [TezTR-518813_1_25_1_2_0] exec.GroupByOperator: Closing child = SEL[5] 2018-01-05T11:28:59,469 DEBUG [TezTR-518813_1_25_1_2_0] exec.SelectOperator: close called for operator SEL[5] 2018-01-05T11:28:59,469 DEBUG [TezTR-518813_1_25_1_2_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:59,469 INFO [TezTR-518813_1_25_1_2_0] exec.SelectOperator: Closing operator SEL[5] 2018-01-05T11:28:59,469 DEBUG [TezTR-518813_1_25_1_2_0] exec.SelectOperator: Closing child = FS[6] 2018-01-05T11:28:59,469 DEBUG [TezTR-518813_1_25_1_2_0] exec.FileSinkOperator: close called for operator FS[6] 2018-01-05T11:28:59,469 DEBUG [TezTR-518813_1_25_1_2_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:59,469 INFO [TezTR-518813_1_25_1_2_0] exec.FileSinkOperator: Closing operator FS[6] 2018-01-05T11:28:59,469 INFO [TezTR-518813_1_25_1_2_0] exec.FileSinkOperator: FS[6]: records written - 0 2018-01-05T11:28:59,469 INFO [TezTR-518813_1_25_1_2_0] exec.FileSinkOperator: RECORDS_OUT_0:0, 2018-01-05T11:28:59,469 DEBUG [TezTR-518813_1_25_1_2_0] exec.FileSinkOperator: 6 Close done 2018-01-05T11:28:59,469 DEBUG [TezTR-518813_1_25_1_2_0] exec.SelectOperator: 5 Close done 2018-01-05T11:28:59,469 DEBUG [TezTR-518813_1_25_1_2_0] exec.GroupByOperator: 4 Close done 2018-01-05T11:28:59,469 INFO [TezTR-518813_1_25_1_2_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_25_01_000002_0 2018-01-05T11:28:59,469 INFO [TezTR-518813_1_25_1_2_0] orderedgrouped.Shuffle: Shutting down Shuffle for source: Map_1 2018-01-05T11:28:59,469 INFO [TezTR-518813_1_25_1_2_0] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=1, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:28:59,469 INFO [TezTR-518813_1_25_1_2_0] output.MROutput: out_Reducer 2 closed 2018-01-05T11:28:59,469 DEBUG [TezTR-518813_1_25_1_2_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:59,469 INFO [TezTR-518813_1_25_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:59,469 INFO [TezTR-518813_1_25_1_2_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_25_01_000002_0, askedToStop=false 2018-01-05T11:28:59,469 INFO [TezTR-518813_1_25_1_2_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_25_01_000002_0, stopRequested=false 2018-01-05T11:28:59,469 INFO [TezTR-518813_1_25_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_25_01_000002_0: Counters: 29 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=0, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=0, NUM_SKIPPED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=0, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, SHUFFLE_BYTES=0, SHUFFLE_BYTES_DECOMPRESSED=0, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=3, MERGE_PHASE_TIME=3, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2][HIVE RECORDS_OUT_0=0][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0]] 2018-01-05T11:28:59,469 DEBUG [TezTR-518813_1_25_1_2_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_25_1_2_0 thread id: 1698 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:59,469 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_25_01_000002_0, isTaskDone=true 2018-01-05T11:28:59,469 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000054, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_25_01_000002_0, eventCount=2 } 2018-01-05T11:28:59,470 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_25][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_25_01_000002_0, creationTime=1515180539434, allocationTime=1515180539436, startTime=1515180539440, finishTime=1515180539470, timeTaken=30, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:59,470 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_25_01_000002_0 2018-01-05T11:28:59,470 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_25_01_000002_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:59,470 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:28:59,470 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_25_01_000002_0; false 2018-01-05T11:28:59,470 INFO [Task-Executor-3] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_25_01_000002_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:59,470 INFO [Task-Executor-3] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_25_01_000002_0 2018-01-05T11:28:59,470 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_25_01_000002_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:59,470 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_25][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_25_01_000002, startTime=1515180539440, finishTime=1515180539470, timeTaken=30, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_25_01_000002_0, diagnostics=, counters=Counters: 34, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=0, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=0, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=0, NUM_SKIPPED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=0, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, SHUFFLE_BYTES=0, SHUFFLE_BYTES_DECOMPRESSED=0, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=3, MERGE_PHASE_TIME=3, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, HIVE, RECORDS_OUT_0=0, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0 2018-01-05T11:28:59,470 INFO [Task-Executor-3] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000054=24 2018-01-05T11:28:59,470 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:59,470 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_25_01_000002_0: true 2018-01-05T11:28:59,470 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:59,470 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_25_01 [Reducer 2], tasks=4, failed=0, killed=0, success=3, completed=3, commits=0, err=null 2018-01-05T11:28:59,470 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_25][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000054, stoppedTime=1515180539470, exitStatus=0 2018-01-05T11:28:59,470 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000054 exited with diagnostics set to Container completed. 2018-01-05T11:28:59,470 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000054 2018-01-05T11:28:59,476 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: merge complete for input vertex : Map_1 2018-01-05T11:28:59,476 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: Map_1: Shuffle Runner thread complete 2018-01-05T11:28:59,476 INFO [TezTR-518813_1_25_1_3_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:28:59,476 INFO [TezTR-518813_1_25_1_3_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:59,476 DEBUG [TezTR-518813_1_25_1_3_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:28:59,476 INFO [TezTR-518813_1_25_1_3_0] input.OrderedGroupedKVInput: Map 1: creating ValuesIterator with comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator, keyClass=org.apache.hadoop.hive.ql.io.HiveKey, valClass=org.apache.hadoop.io.BytesWritable 2018-01-05T11:28:59,476 DEBUG [TezTR-518813_1_25_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,476 DEBUG [TezTR-518813_1_25_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,476 DEBUG [TezTR-518813_1_25_1_3_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1, _col2, _col3, _col4] columnTypes=[struct, struct, struct, struct, struct] 2018-01-05T11:28:59,476 DEBUG [TezTR-518813_1_25_1_3_0] log.PerfLogger: 2018-01-05T11:28:59,476 INFO [TezTR-518813_1_25_1_3_0] exec.GroupByOperator: Initializing operator GBY[4] 2018-01-05T11:28:59,476 DEBUG [TezTR-518813_1_25_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,476 DEBUG [TezTR-518813_1_25_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,476 DEBUG [TezTR-518813_1_25_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,476 DEBUG [TezTR-518813_1_25_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,476 DEBUG [TezTR-518813_1_25_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,476 INFO [TezTR-518813_1_25_1_3_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:28:59,476 DEBUG [TezTR-518813_1_25_1_3_0] exec.GroupByOperator: Initialization Done 4 GBY 2018-01-05T11:28:59,476 DEBUG [TezTR-518813_1_25_1_3_0] exec.GroupByOperator: Operator 4 GBY initialized 2018-01-05T11:28:59,476 DEBUG [TezTR-518813_1_25_1_3_0] exec.GroupByOperator: Initializing children of 4 GBY 2018-01-05T11:28:59,476 DEBUG [TezTR-518813_1_25_1_3_0] exec.SelectOperator: Initializing child 5 SEL 2018-01-05T11:28:59,476 INFO [TezTR-518813_1_25_1_3_0] exec.SelectOperator: Initializing operator SEL[5] 2018-01-05T11:28:59,476 INFO [TezTR-518813_1_25_1_3_0] exec.SelectOperator: SELECT struct<_col0:string,_col1:struct,_col2:struct,_col3:struct,_col4:struct,_col5:struct> 2018-01-05T11:28:59,476 DEBUG [TezTR-518813_1_25_1_3_0] exec.SelectOperator: Initialization Done 5 SEL 2018-01-05T11:28:59,476 DEBUG [TezTR-518813_1_25_1_3_0] exec.SelectOperator: Operator 5 SEL initialized 2018-01-05T11:28:59,476 DEBUG [TezTR-518813_1_25_1_3_0] exec.SelectOperator: Initializing children of 5 SEL 2018-01-05T11:28:59,476 DEBUG [TezTR-518813_1_25_1_3_0] exec.FileSinkOperator: Initializing child 6 FS 2018-01-05T11:28:59,476 INFO [TezTR-518813_1_25_1_3_0] exec.FileSinkOperator: Initializing operator FS[6] 2018-01-05T11:28:59,477 DEBUG [TezTR-518813_1_25_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,477 INFO [TezTR-518813_1_25_1_3_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@54be75a3]:[_col0, _col1, _col2, _col3, _col4, _col5]:[struct, struct, struct, struct, struct, string]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@737da83f 2018-01-05T11:28:59,477 DEBUG [TezTR-518813_1_25_1_3_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:28:59,477 DEBUG [TezTR-518813_1_25_1_3_0] exec.FileSinkOperator: Initialization Done 6 FS 2018-01-05T11:28:59,477 DEBUG [TezTR-518813_1_25_1_3_0] exec.FileSinkOperator: Operator 6 FS initialized 2018-01-05T11:28:59,477 DEBUG [TezTR-518813_1_25_1_3_0] exec.FileSinkOperator: Initialization Done 6 FS done is reset. 2018-01-05T11:28:59,477 DEBUG [TezTR-518813_1_25_1_3_0] exec.SelectOperator: Initialization Done 5 SEL done is reset. 2018-01-05T11:28:59,477 DEBUG [TezTR-518813_1_25_1_3_0] exec.GroupByOperator: Initialization Done 4 GBY done is reset. 2018-01-05T11:28:59,477 INFO [TezTR-518813_1_25_1_3_0] tez.ReduceRecordProcessor: Id =4 Id =5 Id =6 <\Children> Id = 5 null<\Parent> <\FS> <\Children> Id = 4 null<\Parent> <\SEL> <\Children> <\Parent> <\GBY> 2018-01-05T11:28:59,477 DEBUG [TezTR-518813_1_25_1_3_0] log.PerfLogger: 2018-01-05T11:28:59,477 INFO [TezTR-518813_1_25_1_3_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 2 2018-01-05T11:28:59,478 DEBUG [TezTR-518813_1_25_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:28:59,478 DEBUG [TezTR-518813_1_25_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:28:59,478 DEBUG [TezTR-518813_1_25_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:28:59,478 DEBUG [TezTR-518813_1_25_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:28:59,478 DEBUG [TezTR-518813_1_25_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:28:59,478 DEBUG [TezTR-518813_1_25_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:28:59,478 DEBUG [TezTR-518813_1_25_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:28:59,478 DEBUG [TezTR-518813_1_25_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:28:59,478 DEBUG [TezTR-518813_1_25_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:28:59,478 DEBUG [TezTR-518813_1_25_1_3_0] log.PerfLogger: 2018-01-05T11:28:59,478 DEBUG [TezTR-518813_1_25_1_3_0] mr.ObjectCache: djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:28:59,478 DEBUG [TezTR-518813_1_25_1_3_0] exec.GroupByOperator: close called for operator GBY[4] 2018-01-05T11:28:59,478 INFO [TezTR-518813_1_25_1_3_0] exec.GroupByOperator: Closing operator GBY[4] 2018-01-05T11:28:59,478 INFO [TezTR-518813_1_25_1_3_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_093_9159397621401875189-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-59_093_9159397621401875189-1/_tmp.-ext-10002/000003_0 2018-01-05T11:28:59,478 INFO [TezTR-518813_1_25_1_3_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_093_9159397621401875189-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-59_093_9159397621401875189-1/_task_tmp.-ext-10002/_tmp.000003_0 2018-01-05T11:28:59,478 INFO [TezTR-518813_1_25_1_3_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_093_9159397621401875189-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-59_093_9159397621401875189-1/_tmp.-ext-10002/000003_0 2018-01-05T11:28:59,511 INFO [TezTR-518813_1_25_1_3_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:28:59,511 DEBUG [TezTR-518813_1_25_1_3_0] exec.GroupByOperator: Closing child = SEL[5] 2018-01-05T11:28:59,511 DEBUG [TezTR-518813_1_25_1_3_0] exec.SelectOperator: close called for operator SEL[5] 2018-01-05T11:28:59,511 DEBUG [TezTR-518813_1_25_1_3_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:59,511 INFO [TezTR-518813_1_25_1_3_0] exec.SelectOperator: Closing operator SEL[5] 2018-01-05T11:28:59,511 DEBUG [TezTR-518813_1_25_1_3_0] exec.SelectOperator: Closing child = FS[6] 2018-01-05T11:28:59,511 DEBUG [TezTR-518813_1_25_1_3_0] exec.FileSinkOperator: close called for operator FS[6] 2018-01-05T11:28:59,511 DEBUG [TezTR-518813_1_25_1_3_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:59,511 INFO [TezTR-518813_1_25_1_3_0] exec.FileSinkOperator: Closing operator FS[6] 2018-01-05T11:28:59,512 INFO [TezTR-518813_1_25_1_3_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:28:59,512 INFO [TezTR-518813_1_25_1_3_0] exec.FileSinkOperator: RECORDS_OUT_0:1, 2018-01-05T11:28:59,512 DEBUG [TezTR-518813_1_25_1_3_0] exec.FileSinkOperator: 6 Close done 2018-01-05T11:28:59,512 DEBUG [TezTR-518813_1_25_1_3_0] exec.SelectOperator: 5 Close done 2018-01-05T11:28:59,512 DEBUG [TezTR-518813_1_25_1_3_0] exec.GroupByOperator: 4 Close done 2018-01-05T11:28:59,512 INFO [TezTR-518813_1_25_1_3_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_25_01_000003_0 2018-01-05T11:28:59,512 INFO [TezTR-518813_1_25_1_3_0] orderedgrouped.Shuffle: Shutting down Shuffle for source: Map_1 2018-01-05T11:28:59,512 INFO [TezTR-518813_1_25_1_3_0] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:28:59,512 INFO [TezTR-518813_1_25_1_3_0] output.MROutput: out_Reducer 2 closed 2018-01-05T11:28:59,512 DEBUG [TezTR-518813_1_25_1_3_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:59,512 INFO [TezTR-518813_1_25_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:59,512 INFO [TezTR-518813_1_25_1_3_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_25_01_000003_0, askedToStop=false 2018-01-05T11:28:59,512 INFO [TezTR-518813_1_25_1_3_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_25_01_000003_0, stopRequested=false 2018-01-05T11:28:59,512 INFO [TezTR-518813_1_25_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_25_01_000003_0: Counters: 30 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=1, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=204, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=204, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=14, MERGE_PHASE_TIME=29, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][HIVE CREATED_FILES=1, RECORDS_OUT_0=1][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0]] 2018-01-05T11:28:59,513 DEBUG [TezTR-518813_1_25_1_3_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_25_1_3_0 thread id: 1680 scheme: file bytesRead: 0 bytesWritten: 326 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:59,513 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_25_01_000003_0, isTaskDone=true 2018-01-05T11:28:59,513 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000051, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_25_01_000003_0, eventCount=2 } 2018-01-05T11:28:59,513 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_25][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_25_01_000003_0, creationTime=1515180539434, allocationTime=1515180539435, startTime=1515180539436, finishTime=1515180539513, timeTaken=77, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:59,513 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_25_01_000003_0 2018-01-05T11:28:59,513 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_25_01_000003_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:59,513 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_25_01_000003_0; false 2018-01-05T11:28:59,513 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:28:59,513 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_25_01_000003_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:59,513 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_25_01_000003_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:59,514 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_25_01_000003_0 2018-01-05T11:28:59,514 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:59,514 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_25][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_25_01_000003, startTime=1515180539436, finishTime=1515180539513, timeTaken=77, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_25_01_000003_0, diagnostics=, counters=Counters: 35, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=326, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=1, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=204, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=204, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=14, MERGE_PHASE_TIME=29, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0 2018-01-05T11:28:59,514 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:59,514 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000051=76 2018-01-05T11:28:59,514 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_25_01_000003_0: true 2018-01-05T11:28:59,514 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_25_01 [Reducer 2], tasks=4, failed=0, killed=0, success=4, completed=4, commits=0, err=null 2018-01-05T11:28:59,514 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_25_01 [Reducer 2] 2018-01-05T11:28:59,514 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_25][Event:VERTEX_FINISHED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_25_01, initRequestedTime=1515180539325, initedTime=1515180539330, startRequestedTime=1515180539344, startedTime=1515180539344, finishTime=1515180539514, timeTaken=170, status=SUCCEEDED, diagnostics=, counters=Counters: 35, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=326, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=1, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=3, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=204, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=204, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=27, MERGE_PHASE_TIME=44, FIRST_EVENT_RECEIVED=15, LAST_EVENT_RECEIVED=15, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, vertexStats=firstTaskStartTime=1515180539436, firstTasksToStart=[ task_1515180518813_0001_25_01_000003 ], lastTaskFinishTime=1515180539513, lastTasksToFinish=[ task_1515180518813_0001_25_01_000003 ], minTaskDuration=29, maxTaskDuration=77, avgTaskDuration=41.5, numSuccessfulTasks=4, shortestDurationTasks=[ task_1515180518813_0001_25_01_000001 ], longestDurationTasks=[ task_1515180518813_0001_25_01_000003 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=4, numSucceededTasks=4, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:59,514 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_25_01 [Reducer 2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:59,514 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_25][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000051, stoppedTime=1515180539514, exitStatus=0 2018-01-05T11:28:59,515 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000051 exited with diagnostics set to Container completed. 2018-01-05T11:28:59,515 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_25_01 [Reducer 2] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2 2018-01-05T11:28:59,515 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null 2018-01-05T11:28:59,515 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_25_00 [Map 1] 2018-01-05T11:28:59,515 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_25_01 [Reducer 2] 2018-01-05T11:28:59,515 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_25, queueSize=0 2018-01-05T11:28:59,516 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_25][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_25, startTime=1515180539318, finishTime=1515180539515, timeTaken=197, status=SUCCEEDED, diagnostics=, counters=Counters: 51, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=5, TOTAL_LAUNCHED_TASKS=5, OTHER_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=361, AM_GC_TIME_MILLIS=0, File System Counters, FILE_BYTES_READ=506, FILE_BYTES_WRITTEN=725, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=3, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, INPUT_RECORDS_PROCESSED=20, INPUT_SPLIT_LENGTH_BYTES=282, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=191, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=204, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=204, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=27, MERGE_PHASE_TIME=44, FIRST_EVENT_RECEIVED=15, LAST_EVENT_RECEIVED=15, HIVE, CREATED_FILES=1, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=20, RECORDS_OUT_0=1, RECORDS_OUT_INTERMEDIATE_Map_1=1, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0 2018-01-05T11:28:59,516 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_25 finished with state: SUCCEEDED 2018-01-05T11:28:59,516 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_25 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:28:59,516 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000051 2018-01-05T11:28:59,516 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_25, dagState=SUCCEEDED 2018-01-05T11:28:59,516 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_25_post 2018-01-05T11:28:59,516 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:59,516 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:28:59,516 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:28:59,516 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 25 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=5, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=5,NumRequestsWithoutLocation=5, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=5} 2018-01-05T11:28:59,516 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:28:59,516 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=analyze table cbo_t2 compute sta...c_boolean (Stage-0), with id=dag_1515180518813_0001_25 2018-01-05T11:28:59,516 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 25, to 10.22.8.182:49818 2018-01-05T11:28:59,516 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=5, Remaining=0 2018-01-05T11:28:59,516 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=analyze table cbo_t2 compute sta...c_boolean (Stage-0), with id=dag_1515180518813_0001_25 2018-01-05T11:28:59,516 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=25} 2018-01-05T11:28:59,516 INFO [IPC Server handler 2 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=25}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:28:59,516 INFO [IPC Server handler 2 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/25 after 300 seconds 2018-01-05T11:28:59,516 ERROR [IPC Server handler 2 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136, dag_1515180518813_0001_25 2018-01-05T11:28:59,516 INFO [IPC Server handler 2 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136 2018-01-05T11:28:59,516 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=25} = 4 2018-01-05T11:28:59,516 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Issuing killFragment for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=25} attempt_1515180518813_0001_25_01_000000_0 2018-01-05T11:28:59,516 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Removing attempt_1515180518813_0001_25_01_000000_0 from preemptionQueue 2018-01-05T11:28:59,516 INFO [IPC Server handler 2 on 49818] impl.TaskRunnerCallable: Ignoring kill request for task attempt_1515180518813_0001_25_01_000000_0 since it's already complete 2018-01-05T11:28:59,516 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Issuing killFragment for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=25} attempt_1515180518813_0001_25_01_000001_0 2018-01-05T11:28:59,516 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Removing attempt_1515180518813_0001_25_01_000001_0 from preemptionQueue 2018-01-05T11:28:59,516 INFO [IPC Server handler 2 on 49818] impl.TaskRunnerCallable: Ignoring kill request for task attempt_1515180518813_0001_25_01_000001_0 since it's already complete 2018-01-05T11:28:59,517 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Issuing killFragment for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=25} attempt_1515180518813_0001_25_01_000002_0 2018-01-05T11:28:59,517 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Removing attempt_1515180518813_0001_25_01_000002_0 from preemptionQueue 2018-01-05T11:28:59,517 INFO [IPC Server handler 2 on 49818] impl.TaskRunnerCallable: Ignoring kill request for task attempt_1515180518813_0001_25_01_000002_0 since it's already complete 2018-01-05T11:28:59,517 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Issuing killFragment for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=25} attempt_1515180518813_0001_25_01_000003_0 2018-01-05T11:28:59,517 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Removing attempt_1515180518813_0001_25_01_000003_0 from preemptionQueue 2018-01-05T11:28:59,517 INFO [IPC Server handler 2 on 49818] impl.TaskRunnerCallable: Ignoring kill request for task attempt_1515180518813_0001_25_01_000003_0 since it's already complete 2018-01-05T11:28:59,517 DEBUG [IPC Server handler 2 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=25} 2018-01-05T11:28:59,517 DEBUG [IPC Server handler 2 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:28:59,517 DEBUG [IPC Server handler 2 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=25} 2018-01-05T11:28:59,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,517 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 Reducer 2: 4/4 2018-01-05T11:28:59,517 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,518 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,518 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: TaskId for 000003_0 = 000003 2018-01-05T11:28:59,518 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,519 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,519 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,519 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-2:STATS] in serial mode 2018-01-05T11:28:59,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@109eb6d3, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:59,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:59,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,520 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:59,520 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:59,520 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:59,520 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:59,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,520 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:59,520 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:59,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:59,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,520 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:59,520 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:59,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,520 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:59,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_093_9159397621401875189-1/-mr-10001 2018-01-05T11:28:59,520 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_093_9159397621401875189-1/-mr-10001 2018-01-05T11:28:59,521 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Getting file stats of all partitions. threadpool size:15 2018-01-05T11:28:59,521 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Finished getting file stats of all partitions! 2018-01-05T11:28:59,521 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.cbo_t2/dt=2014/ numRows 2018-01-05T11:28:59,521 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.cbo_t2/dt=2014/ numRows 20 2018-01-05T11:28:59,521 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.cbo_t2/dt=2014/ rawDataSize 2018-01-05T11:28:59,521 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.cbo_t2/dt=2014/ rawDataSize 262 2018-01-05T11:28:59,521 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Partition {dt=2014} stats: [numFiles=1, numRows=20, totalSize=282, rawDataSize=262] 2018-01-05T11:28:59,522 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,522 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:59,522 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:59,522 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:59,522 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:59,522 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,522 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,523 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,523 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,523 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,523 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:59,523 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:59,523 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,523 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:59,524 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,524 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_partitions : db=default tbl=cbo_t2 2018-01-05T11:28:59,524 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_partitions : db=default tbl=cbo_t2 2018-01-05T11:28:59,524 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: New partition values:[2014] 2018-01-05T11:28:59,526 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,533 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000050, QueryId=djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136-25, DagId=25, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180539360, EndTime=1515180539533 2018-01-05T11:28:59,533 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_25_01_000001_0 2018-01-05T11:28:59,533 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_25_01_000001_0 complete. WaitQueueSize=0, numSlotsAvailable=1, preemptionQueueSize=0 2018-01-05T11:28:59,533 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_25_01_000001_0 2018-01-05T11:28:59,533 INFO [ExecutionCompletionThread #0] impl.QueryTracker: Ignoring fragmentComplete message for unknown query: QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=25} 2018-01-05T11:28:59,533 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000053, QueryId=djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136-25, DagId=25, VertexName=Reducer 2, TaskId=1, TaskAttemptId=0, ThreadName=Task-Executor-2, Succeeded=true, StartTime=1515180539442, EndTime=1515180539533 2018-01-05T11:28:59,534 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_25_01_000000_0 2018-01-05T11:28:59,534 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_25_01_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=2, preemptionQueueSize=0 2018-01-05T11:28:59,534 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_25_01_000000_0 2018-01-05T11:28:59,534 INFO [ExecutionCompletionThread #0] impl.QueryTracker: Ignoring fragmentComplete message for unknown query: QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=25} 2018-01-05T11:28:59,534 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000052, QueryId=djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136-25, DagId=25, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-1, Succeeded=true, StartTime=1515180539438, EndTime=1515180539534 2018-01-05T11:28:59,534 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_25_01_000002_0 2018-01-05T11:28:59,534 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_25_01_000002_0 complete. WaitQueueSize=0, numSlotsAvailable=3, preemptionQueueSize=0 2018-01-05T11:28:59,534 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_25_01_000002_0 2018-01-05T11:28:59,534 INFO [ExecutionCompletionThread #0] impl.QueryTracker: Ignoring fragmentComplete message for unknown query: QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=25} 2018-01-05T11:28:59,534 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000054, QueryId=djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136-25, DagId=25, VertexName=Reducer 2, TaskId=2, TaskAttemptId=0, ThreadName=Task-Executor-3, Succeeded=true, StartTime=1515180539445, EndTime=1515180539534 2018-01-05T11:28:59,534 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_25_01_000003_0 2018-01-05T11:28:59,534 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_25_01_000003_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:59,534 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_25_01_000003_0 2018-01-05T11:28:59,534 INFO [ExecutionCompletionThread #0] impl.QueryTracker: Ignoring fragmentComplete message for unknown query: QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=25} 2018-01-05T11:28:59,534 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,535 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,535 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: About to delete stats tmp dir :file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_093_9159397621401875189-1/-mr-10001 2018-01-05T11:28:59,551 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Time taken to get FileStatuses: 14 2018-01-05T11:28:59,551 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:28:59,551 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 15 2018-01-05T11:28:59,551 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FetchOperator: Creating fetchTask with deserializer typeinfo: struct<_col0:struct,_col1:struct,_col2:struct,_col3:struct,_col4:struct,_col5:string> 2018-01-05T11:28:59,551 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FetchOperator: deserializer properties: table properties: {columns=_col0,_col1,_col2,_col3,_col4,_col5, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct:struct:struct:struct:struct:string, escape.delim=\} partition properties: {columns=_col0,_col1,_col2,_col3,_col4,_col5, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct:struct:struct:struct:struct:string, escape.delim=\} 2018-01-05T11:28:59,552 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,555 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,555 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: write_partition_column_statistics: db=default table=cbo_t2 part=dt=2014 2018-01-05T11:28:59,556 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=write_partition_column_statistics: db=default table=cbo_t2 part=dt=2014 2018-01-05T11:28:59,559 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating partition level column statistics for db=default tableName=cbo_t2 partName=dt=2014 colName=key 2018-01-05T11:28:59,560 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating partition level column statistics for db=default tableName=cbo_t2 partName=dt=2014 colName=value 2018-01-05T11:28:59,560 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating partition level column statistics for db=default tableName=cbo_t2 partName=dt=2014 colName=c_int 2018-01-05T11:28:59,563 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,563 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating partition level column statistics for db=default tableName=cbo_t2 partName=dt=2014 colName=c_float 2018-01-05T11:28:59,563 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating partition level column statistics for db=default tableName=cbo_t2 partName=dt=2014 colName=c_boolean 2018-01-05T11:28:59,568 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,570 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,570 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,570 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,570 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: analyze table cbo_t2 compute statistics for columns key, value, c_int, c_float, c_boolean 2018-01-05T11:28:59,570 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:28:59,570 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@cbo_t2 2018-01-05T11:28:59,570 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@cbo_t2@dt=2014 2018-01-05T11:28:59,570 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@cbo_t2 2018-01-05T11:28:59,570 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@cbo_t2@dt=2014 2018-01-05T11:28:59,570 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_093_9159397621401875189-1/-mr-10000 2018-01-05T11:28:59,570 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,570 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,570 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:59,570 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_partitions_(String, String, List, EnvironmentContext, )=12, setPartitionColumnStatistics_(SetPartitionsStatsRequest, )=18} 2018-01-05T11:28:59,570 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136); Time taken: 0.385 seconds 2018-01-05T11:28:59,570 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:59,570 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,570 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:59,570 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,570 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query analyze table cbo_t2 compute statistics for columns key, value, c_int, c_float, c_boolean 2018-01-05T11:28:59,570 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_090_4354755825235708462-1 2018-01-05T11:28:59,571 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.48 seconds 2018-01-05T11:28:59,571 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:59,571 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:59,571 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:59,571 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:59,571 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,571 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:59,571 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,571 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,571 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,571 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: analyze table cbo_t3 compute statistics 2018-01-05T11:28:59,571 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112859_7fb2c9ac-06b5-4ba2-87ba-2dd685912638): analyze table cbo_t3 compute statistics 2018-01-05T11:28:59,571 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,571 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: analyze table cbo_t3 compute statistics 2018-01-05T11:28:59,571 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:59,571 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,571 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,571 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,571 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,571 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:59,571 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Invoking analyze on original query 2018-01-05T11:28:59,571 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Starting Semantic Analysis 2018-01-05T11:28:59,572 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed phase 1 of Semantic Analysis 2018-01-05T11:28:59,572 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for source tables 2018-01-05T11:28:59,572 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,572 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t3 2018-01-05T11:28:59,572 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t3 2018-01-05T11:28:59,574 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,574 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,574 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,574 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t3 2018-01-05T11:28:59,574 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t3 2018-01-05T11:28:59,577 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,578 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,578 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for subqueries 2018-01-05T11:28:59,578 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for destination tables 2018-01-05T11:28:59,578 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed getting MetaData in Semantic Analysis 2018-01-05T11:28:59,590 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_571_7301919214835618340-1/-mr-10000 2018-01-05T11:28:59,590 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Table Plan for cbo_t3 TS[0] 2018-01-05T11:28:59,590 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Body Plan for Query Block null 2018-01-05T11:28:59,590 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Plan for Query Block null 2018-01-05T11:28:59,590 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Before logical optimization TS[0] 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] lineage.Generator: Time taken for lineage transform=0 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{cbo_t3}key,value: string|{cbo_t3}value,c_int: int|{cbo_t3}c_int,c_float: float|{cbo_t3}c_float,c_boolean: boolean|{cbo_t3}c_boolean,BLOCK__OFFSET__INSIDE__FILE: bigint|{cbo_t3}block__offset__inside__file,INPUT__FILE__NAME: string|{cbo_t3}input__file__name,ROW__ID: struct|{cbo_t3}row__id,RAW__DATA__SIZE: bigint|{cbo_t3}raw__data__size) 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.PredicatePushDown: After PPD: TS[0] 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{cbo_t3}key,value: string|{cbo_t3}value,c_int: int|{cbo_t3}c_int,c_float: float|{cbo_t3}c_float,c_boolean: boolean|{cbo_t3}c_boolean,BLOCK__OFFSET__INSIDE__FILE: bigint|{cbo_t3}block__offset__inside__file,INPUT__FILE__NAME: string|{cbo_t3}input__file__name,ROW__ID: struct|{cbo_t3}row__id,RAW__DATA__SIZE: bigint|{cbo_t3}raw__data__size) 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{cbo_t3}key,value: string|{cbo_t3}value,c_int: int|{cbo_t3}c_int,c_float: float|{cbo_t3}c_float,c_boolean: boolean|{cbo_t3}c_boolean,BLOCK__OFFSET__INSIDE__FILE: bigint|{cbo_t3}block__offset__inside__file,INPUT__FILE__NAME: string|{cbo_t3}input__file__name,ROW__ID: struct|{cbo_t3}row__id,RAW__DATA__SIZE: bigint|{cbo_t3}raw__data__size) 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: After logical optimization TS[0] 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,591 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,592 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,592 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,592 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=cbo_t3 2018-01-05T11:28:59,592 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=cbo_t3 2018-01-05T11:28:59,594 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting stats (Num rows: 1 Data size: 2820 Basic stats: COMPLETE Column stats: COMPLETE) on TS[0] 2018-01-05T11:28:59,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (cbo_t3): numRows: 1 dataSize: 2820 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:28:59,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on TS[0] 2018-01-05T11:28:59,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:59,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:28:59,597 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Cycle free: true 2018-01-05T11:28:59,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,597 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,598 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{cbo_t3}key,value: string|{cbo_t3}value,c_int: int|{cbo_t3}c_int,c_float: float|{cbo_t3}c_float,c_boolean: boolean|{cbo_t3}c_boolean,BLOCK__OFFSET__INSIDE__FILE: bigint|{cbo_t3}block__offset__inside__file,INPUT__FILE__NAME: string|{cbo_t3}input__file__name,ROW__ID: struct|{cbo_t3}row__id,RAW__DATA__SIZE: bigint|{cbo_t3}raw__data__size) 2018-01-05T11:28:59,598 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,598 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:28:59,598 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3 of table cbo_t3 2018-01-05T11:28:59,598 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3 2018-01-05T11:28:59,598 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:28:59,598 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,598 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,598 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:59,598 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:59,598 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:59,598 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:59,598 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:59,598 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:59,598 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:28:59,598 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping vectorization 2018-01-05T11:28:59,598 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:28:59,598 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: llap mode: only 2018-01-05T11:28:59,598 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:28:59,598 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:59,598 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@50a071c5 can be done in LLAP 2018-01-05T11:28:59,598 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,598 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed plan generation 2018-01-05T11:28:59,598 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:59,598 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validation start 2018-01-05T11:28:59,598 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Not a partition. 2018-01-05T11:28:59,598 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,598 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:cbo_t3.key, type:string, comment:null), FieldSchema(name:cbo_t3.value, type:string, comment:null), FieldSchema(name:cbo_t3.c_int, type:int, comment:null), FieldSchema(name:cbo_t3.c_float, type:float, comment:null), FieldSchema(name:cbo_t3.c_boolean, type:boolean, comment:null)], properties:null) 2018-01-05T11:28:59,598 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,598 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:59,598 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=6, flushCache_()=0, getTableColumnStatistics_(String, String, List, )=5} 2018-01-05T11:28:59,598 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112859_7fb2c9ac-06b5-4ba2-87ba-2dd685912638); Time taken: 0.027 seconds 2018-01-05T11:28:59,598 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:59,598 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,598 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112859_7fb2c9ac-06b5-4ba2-87ba-2dd685912638): analyze table cbo_t3 compute statistics 2018-01-05T11:28:59,599 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,600 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: analyze table cbo_t3 compute statistics 2018-01-05T11:28:59,600 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:28:59,600 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@cbo_t3 2018-01-05T11:28:59,600 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@cbo_t3 2018-01-05T11:28:59,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,600 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Query ID = djaiswal_20180105112859_7fb2c9ac-06b5-4ba2-87ba-2dd685912638 2018-01-05T11:28:59,600 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Total jobs = 1 2018-01-05T11:28:59,600 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,600 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:28:59,600 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MAPRED] in serial mode 2018-01-05T11:28:59,610 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_571_7301919214835618340-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:28:59,633 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:28:59,633 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:28:59,633 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:28:59,633 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112859_7fb2c9ac-06b5-4ba2-87ba-2dd685912638 2018-01-05T11:28:59,633 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Session is already open 2018-01-05T11:28:59,633 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:28:59,633 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,633 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Dag name: analyze table cbo_t3 compute statistics (Stage-0) 2018-01-05T11:28:59,633 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: DagInfo: {"context":"Hive","description":"\nanalyze table cbo_t3 compute statistics"} 2018-01-05T11:28:59,633 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112859_7fb2c9ac-06b5-4ba2-87ba-2dd685912638 with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:28:59,633 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,637 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000051, QueryId=djaiswal_20180105112859_191fb44e-0f97-485d-ac05-6c9f8c135136-25, DagId=25, VertexName=Reducer 2, TaskId=3, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180539437, EndTime=1515180539637 2018-01-05T11:28:59,646 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:28:59,646 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,646 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:28:59,646 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,646 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 1.92KB 2018-01-05T11:28:59,652 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_571_7301919214835618340-1/-mr-10000 2018-01-05T11:28:59,664 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_571_7301919214835618340-1/-mr-10000 2018-01-05T11:28:59,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3 2018-01-05T11:28:59,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,666 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,666 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=analyze table cbo_t3 compute statistics (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112859_7fb2c9ac-06b5-4ba2-87ba-2dd685912638 } 2018-01-05T11:28:59,667 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:59,668 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: analyze table cbo_t3 compute statistics (Stage-0) 2018-01-05T11:28:59,668 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:28:59,669 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_26, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_26.dot 2018-01-05T11:28:59,669 INFO [IPC Server handler 0 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_26 2018-01-05T11:28:59,669 WARN [IPC Server handler 0 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:59,669 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Running DAG: analyze table cbo_t3 compute statistics (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112859_7fb2c9ac-06b5-4ba2-87ba-2dd685912638 } 2018-01-05T11:28:59,693 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_26][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_26, submitTime=1515180539668, queueName=null 2018-01-05T11:28:59,694 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_26_00 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_26_00 [Map 1], Default Resources= 2018-01-05T11:28:59,694 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_26_00 [Map 1] 2018-01-05T11:28:59,694 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_26_00 [Map 1] 2018-01-05T11:28:59,694 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:28:59,694 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_26_00 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:59,694 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:28:59,694 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_26][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_26, initTime=1515180539694 2018-01-05T11:28:59,694 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: dag_1515180518813_0001_26 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:28:59,695 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:28:59,695 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_26][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_26, startTime=1515180539695 2018-01-05T11:28:59,695 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_26 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:28:59,695 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {cbo_t3={InputName=cbo_t3}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:28:59,695 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: cbo_t3, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:28:59,695 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_26_00 [Map 1] 2018-01-05T11:28:59,695 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_26, dagName=analyze table cbo_t3 compute statistics (Stage-0) 2018-01-05T11:28:59,695 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,695 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,695 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,699 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_26_00 [Map 1] 2018-01-05T11:28:59,699 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_26_00 [Map 1] 2018-01-05T11:28:59,699 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:59,699 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_26_00 [Map 1] 2018-01-05T11:28:59,701 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:28:59,701 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@917d5e3 2018-01-05T11:28:59,701 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_571_7301919214835618340-1/djaiswal/_tez_scratch_dir/e1c9f955-2c45-4ea3-a78c-0915cecba053/map.xml 2018-01-05T11:28:59,702 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_571_7301919214835618340-1/djaiswal/_tez_scratch_dir/e1c9f955-2c45-4ea3-a78c-0915cecba053/map.xml 2018-01-05T11:28:59,702 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_571_7301919214835618340-1/djaiswal/_tez_scratch_dir/e1c9f955-2c45-4ea3-a78c-0915cecba053/map.xml 2018-01-05T11:28:59,702 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:59,702 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:59,702 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,702 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:59,702 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 1.92KB 2018-01-05T11:28:59,702 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_26_00 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:28:59,702 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: cbo_t3 on vertex vertex_1515180518813_0001_26_00 [Map 1] 2018-01-05T11:28:59,702 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:28:59,702 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:59,702 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:28:59,702 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:59,702 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_571_7301919214835618340-1/djaiswal/_tez_scratch_dir/e1c9f955-2c45-4ea3-a78c-0915cecba053/map.xml 2018-01-05T11:28:59,702 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:59,702 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias cbo_t3 2018-01-05T11:28:59,702 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3 2018-01-05T11:28:59,702 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3 2018-01-05T11:28:59,703 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 2018-01-05T11:28:59,703 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = 2018-01-05T11:28:59,703 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3 2018-01-05T11:28:59,703 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:59,703 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:59,703 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:59,703 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:59,719 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Time taken to get FileStatuses: 16 2018-01-05T11:28:59,719 INFO [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:28:59,719 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 16 2018-01-05T11:28:59,720 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 1 2018-01-05T11:28:59,720 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:59,720 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 1. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:28:59,720 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_571_7301919214835618340-1/djaiswal/_tez_scratch_dir/e1c9f955-2c45-4ea3-a78c-0915cecba053/map.xml 2018-01-05T11:28:59,720 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_571_7301919214835618340-1/djaiswal/_tez_scratch_dir/e1c9f955-2c45-4ea3-a78c-0915cecba053/map.xml 2018-01-05T11:28:59,720 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_571_7301919214835618340-1/djaiswal/_tez_scratch_dir/e1c9f955-2c45-4ea3-a78c-0915cecba053/map.xml 2018-01-05T11:28:59,720 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:59,720 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:59,720 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,720 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:28:59,720 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 1.92KB 2018-01-05T11:28:59,720 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3/cbo_t3.txt to src new group? true 2018-01-05T11:28:59,720 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:28:59,720 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:28:59,720 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:28:59,720 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 5 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 282 Original splits: 1 2018-01-05T11:28:59,720 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Using original number of splits: 1 desired splits: 1 2018-01-05T11:28:59,720 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 1 grouped split count is 1, for bucket: 1 2018-01-05T11:28:59,720 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:28:59,721 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: cbo_t3 on vertex vertex_1515180518813_0001_26_00 [Map 1] 2018-01-05T11:28:59,721 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_26_00 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:59,721 INFO [App Shared Pool - #0] impl.VertexImpl: Got updated RootInputsSpecs: {cbo_t3=forAllWorkUnits=true, update=[1]} 2018-01-05T11:28:59,721 INFO [App Shared Pool - #0] impl.VertexImpl: Vertex vertex_1515180518813_0001_26_00 [Map 1] parallelism set to 1 2018-01-05T11:28:59,721 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_26_00 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:28:59,721 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_26_00 [Map 1], numAdditionalOutputs=1 2018-01-05T11:28:59,721 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_26][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_26_00, initRequestedTime=1515180539695, initedTime=1515180539721, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:59,721 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_26_00 [Map 1] with distanceFromRoot: 0 2018-01-05T11:28:59,721 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_26_00 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:28:59,721 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_26][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_26_00, startRequestedTime=1515180539702, startedTime=1515180539721 2018-01-05T11:28:59,721 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_26][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_26_00, reconfigureDoneTime=1515180539721, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:28:59,721 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_26_00 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:28:59,721 INFO [App Shared Pool - #0] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:28:59,721 INFO [App Shared Pool - #0] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:28:59,721 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_26_00 [Map 1] 2018-01-05T11:28:59,722 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_26][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_26_00_000000, scheduledTime=1515180539722, launchTime=1515180539722 2018-01-05T11:28:59,722 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_26_00 [Map 1] 2018-01-05T11:28:59,722 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_26_00_000000_0, priority=2, capability=, hosts=null 2018-01-05T11:28:59,722 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=2,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:59,722 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=2,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:28:59,722 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:59,722 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_26_00_000000_0 on hosts=null 2018-01-05T11:28:59,722 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_26_00_000000_0 on hosts=null 2018-01-05T11:28:59,722 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_26_00_000000_0 2018-01-05T11:28:59,722 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:28:59,722 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #55, task=TaskInfo{task=attempt_1515180518813_0001_26_00_000000_0, priority=2, startTime=0, containerId=null, uniqueId=54, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000055 2018-01-05T11:28:59,722 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_26_00_000000_0; false 2018-01-05T11:28:59,722 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_26_00_000000_0, priority=2, startTime=311560231, containerId=container_111101111_0001_00_000055assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=54, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:28:59,722 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000055 for node: 10.22.8.182:49818 2018-01-05T11:28:59,724 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 26, name=analyze table cbo_t3 compute statistics (Stage-0), queryId=djaiswal_20180105112859_7fb2c9ac-06b5-4ba2-87ba-2dd685912638 2018-01-05T11:28:59,724 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000055, attempt_1515180518813_0001_26_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:28:59,724 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000055 for node: 10.22.8.182:49818 2018-01-05T11:28:59,724 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000055 succeeded on host: 10.22.8.182:49818 2018-01-05T11:28:59,724 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000055, launchTime=1515180539724 2018-01-05T11:28:59,725 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_26_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112859_7fb2c9ac-06b5-4ba2-87ba-2dd685912638, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000055, dagName=analyze table cbo_t3 compute statistics (Stage-0), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{cbo_t3,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{out_Map 1,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180539695, firstAttemptStartTime=1515180539722, currentAttemptStartTime=1515180539724} 2018-01-05T11:28:59,725 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_26_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000055] on NM: [10.22.8.182:49818] 2018-01-05T11:28:59,725 INFO [IPC Server handler 4 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000055, QueryId=djaiswal_20180105112859_7fb2c9ac-06b5-4ba2-87ba-2dd685912638-26, DagId=26, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180539725 2018-01-05T11:28:59,725 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:59,725 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:28:59,725 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_26][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_26_00_000000_0, startTime=1515180539724, containerId=container_111101111_0001_00_000055, nodeId=10.22.8.182:49818 2018-01-05T11:28:59,725 DEBUG [IPC Server handler 4 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=26} with the ShuffleHandler 2018-01-05T11:28:59,736 INFO [IPC Server handler 4 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:28:59,736 DEBUG [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/26] 2018-01-05T11:28:59,737 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:28:59,737 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_26_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:28:59,737 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_26_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180539722, dagStartTime=1515180539695, withinDagPriority=2, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:28:59,737 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_26_00_000000_0 : ACCEPTED 2018-01-05T11:28:59,737 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_26_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:28:59,737 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_26_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180539722, dagStartTime=1515180539695, withinDagPriority=2, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:28:59,737 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_26_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:28:59,737 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_26_00_000000_0 to pre-emption queue 2018-01-05T11:28:59,737 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_26_00_000000_0: true 2018-01-05T11:28:59,737 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:1395289183 2018-01-05T11:28:59,737 INFO [TaskCommunicator # 5] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_26_00_000000_0 2018-01-05T11:28:59,737 INFO [TaskCommunicator # 5] GuaranteedTasks: Received response for attempt_1515180518813_0001_26_00_000000_0, true 2018-01-05T11:28:59,738 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : analyze table cbo_t3 compute statistics (Stage-0), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_26_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=cbo_t3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=out_Map 1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:28:59,738 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1037041664, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:28:59,738 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_26_00_000000_0 vertexName: Map 1 2018-01-05T11:28:59,738 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000055, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_26_00_000000_0, eventCount=1 } 2018-01-05T11:28:59,738 DEBUG [TezTR-518813_1_26_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_26 queryId: djaiswal_20180105112859_7fb2c9ac-06b5-4ba2-87ba-2dd685912638 instanceType: TaskRunner2Callable 2018-01-05T11:28:59,739 INFO [TezTR-518813_1_26_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_26_00_000000_0 2018-01-05T11:28:59,739 DEBUG [TezTR-518813_1_26_0_0_0] log.PerfLogger: 2018-01-05T11:28:59,740 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:28:59,740 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:28:59,741 INFO [IPC Server handler 29 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_26_00_000000_0 sent events: (0-1). 2018-01-05T11:28:59,741 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:28:59,741 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_26_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:28:59,742 DEBUG [TezTR-518813_1_26_0_0_0] log.PerfLogger: 2018-01-05T11:28:59,742 INFO [TezTR-518813_1_26_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:28:59,742 INFO [TezTR-518813_1_26_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:28:59,742 INFO [I/O Setup 0 Initialize: {cbo_t3}] input.MRInput: cbo_t3 using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:28:59,742 INFO [I/O Setup 0 Initialize: {cbo_t3}] input.MRInputLegacy: cbo_t3 MRInputLegacy deferring initialization 2018-01-05T11:28:59,742 INFO [TezTR-518813_1_26_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:28:59,742 INFO [I/O Setup 1 Initialize: {out_Map 1}] output.MROutput: out_Map 1: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:28:59,742 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:28:59,742 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:28:59,742 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:28:59,742 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:28:59,742 INFO [TezTR-518813_1_26_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:28:59,742 INFO [TezTR-518813_1_26_0_0_0] resources.MemoryDistributor: InitialRequests=[cbo_t3:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [out_Map 1:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput] 2018-01-05T11:28:59,742 INFO [TezTR-518813_1_26_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:28:59,742 INFO [TezTR-518813_1_26_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:28:59,742 INFO [TezTR-518813_1_26_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 1037041664, TotalAvailable: 694817914, TotalRequested/TotalJVMHeap:0.00 2018-01-05T11:28:59,742 INFO [TezTR-518813_1_26_0_0_0] resources.MemoryDistributor: Allocations=[cbo_t3:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [out_Map 1:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0] 2018-01-05T11:28:59,742 INFO [TezTR-518813_1_26_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:28:59,742 INFO [TezTR-518813_1_26_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: cbo_t3 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:28:59,742 INFO [TezTR-518813_1_26_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:28:59,742 INFO [TezTR-518813_1_26_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:28:59,742 INFO [I/O Setup 0 Start: {cbo_t3}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: cbo_t3 2018-01-05T11:28:59,742 INFO [TezTR-518813_1_26_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:28:59,742 INFO [TezTR-518813_1_26_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_26_00_000000_0 2018-01-05T11:28:59,742 DEBUG [TezTR-518813_1_26_0_0_0] log.PerfLogger: 2018-01-05T11:28:59,742 DEBUG [TezTR-518813_1_26_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_26_00_000000_0_10163 2018-01-05T11:28:59,742 DEBUG [TezTR-518813_1_26_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:28:59,742 DEBUG [TezTR-518813_1_26_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_26_00_000000_0 2018-01-05T11:28:59,743 INFO [TezTR-518813_1_26_0_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112859_7fb2c9ac-06b5-4ba2-87ba-2dd685912638 2018-01-05T11:28:59,743 DEBUG [TezTR-518813_1_26_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:28:59,743 DEBUG [TezTR-518813_1_26_0_0_0] log.PerfLogger: 2018-01-05T11:28:59,743 DEBUG [TezTR-518813_1_26_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:59,743 DEBUG [TezTR-518813_1_26_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:28:59,743 DEBUG [TezTR-518813_1_26_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112859_7fb2c9ac-06b5-4ba2-87ba-2dd685912638_Map 1__MAP_PLAN__ 2018-01-05T11:28:59,743 INFO [TezTR-518813_1_26_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_571_7301919214835618340-1/djaiswal/_tez_scratch_dir/e1c9f955-2c45-4ea3-a78c-0915cecba053/map.xml 2018-01-05T11:28:59,743 DEBUG [TezTR-518813_1_26_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_571_7301919214835618340-1/djaiswal/_tez_scratch_dir/e1c9f955-2c45-4ea3-a78c-0915cecba053/map.xml 2018-01-05T11:28:59,743 DEBUG [TezTR-518813_1_26_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_571_7301919214835618340-1/djaiswal/_tez_scratch_dir/e1c9f955-2c45-4ea3-a78c-0915cecba053/map.xml 2018-01-05T11:28:59,743 DEBUG [TezTR-518813_1_26_0_0_0] log.PerfLogger: 2018-01-05T11:28:59,743 INFO [TezTR-518813_1_26_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:59,743 DEBUG [TezTR-518813_1_26_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,743 DEBUG [TezTR-518813_1_26_0_0_0] log.PerfLogger: 2018-01-05T11:28:59,743 INFO [TezTR-518813_1_26_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 1.92KB 2018-01-05T11:28:59,744 DEBUG [TezTR-518813_1_26_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:28:59,744 INFO [TezTR-518813_1_26_0_0_0] tez.MapRecordProcessor: The input names are: [cbo_t3] 2018-01-05T11:28:59,744 INFO [TezTR-518813_1_26_0_0_0] input.MRInput: Initialized MRInput: cbo_t3 2018-01-05T11:28:59,744 DEBUG [TezTR-518813_1_26_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 1 2018-01-05T11:28:59,744 INFO [TezTR-518813_1_26_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_571_7301919214835618340-1/djaiswal/_tez_scratch_dir/e1c9f955-2c45-4ea3-a78c-0915cecba053/map.xml 2018-01-05T11:28:59,744 DEBUG [TezTR-518813_1_26_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:59,744 DEBUG [TezTR-518813_1_26_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3/cbo_t3.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string:int:float:boolean, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3, columns=key,value,c_int,c_float,c_boolean, serialization.format=,, numRows=0, numFiles=1, serialization.ddl=struct cbo_t3 { string key, string value, i32 c_int, float c_float, bool c_boolean}, transient_lastDdlTime=1515180537, rawDataSize=0, field.delim=,, columns.comments=, totalSize=282, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.cbo_t3}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string:int:float:boolean, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3, columns=key,value,c_int,c_float,c_boolean, serialization.format=,, numRows=0, numFiles=1, serialization.ddl=struct cbo_t3 { string key, string value, i32 c_int, float c_float, bool c_boolean}, transient_lastDdlTime=1515180537, rawDataSize=0, field.delim=,, columns.comments=, totalSize=282, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.cbo_t3}, baseFileName=cbo_t3, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string:int:float:boolean, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3, columns=key,value,c_int,c_float,c_boolean, serialization.format=,, numRows=0, numFiles=1, serialization.ddl=struct cbo_t3 { string key, string value, i32 c_int, float c_float, bool c_boolean}, transient_lastDdlTime=1515180537, rawDataSize=0, field.delim=,, columns.comments=, totalSize=282, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.cbo_t3}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string:int:float:boolean, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3, columns=key,value,c_int,c_float,c_boolean, serialization.format=,, numRows=0, numFiles=1, serialization.ddl=struct cbo_t3 { string key, string value, i32 c_int, float c_float, bool c_boolean}, transient_lastDdlTime=1515180537, rawDataSize=0, field.delim=,, columns.comments=, totalSize=282, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.cbo_t3}, baseFileName=cbo_t3, vectorPartitionDesc=null]} 2018-01-05T11:28:59,744 DEBUG [TezTR-518813_1_26_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:28:59,744 INFO [TezTR-518813_1_26_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:59,744 DEBUG [TezTR-518813_1_26_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:28:59,746 DEBUG [TezTR-518813_1_26_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:28:59,746 INFO [TezTR-518813_1_26_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_571_7301919214835618340-1/djaiswal/_tez_scratch_dir/e1c9f955-2c45-4ea3-a78c-0915cecba053/map.xml 2018-01-05T11:28:59,746 DEBUG [TezTR-518813_1_26_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:59,746 INFO [TezTR-518813_1_26_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_26_00_000000_0 2018-01-05T11:28:59,746 INFO [TezTR-518813_1_26_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value, c_int, c_float, c_boolean] / schema.evolution.columns.types [string, string, int, float, boolean] (isAcidRead false) 2018-01-05T11:28:59,746 WARN [TezTR-518813_1_26_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3/cbo_t3.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:28:59,746 DEBUG [TezTR-518813_1_26_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:28:59,746 WARN [TezTR-518813_1_26_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3/cbo_t3.txt:0+282 2018-01-05T11:28:59,746 DEBUG [TezTR-518813_1_26_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3/cbo_t3.txt 2018-01-05T11:28:59,746 INFO [TezTR-518813_1_26_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3/cbo_t3.txt:0+282], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=282} 2018-01-05T11:28:59,746 INFO [TezTR-518813_1_26_0_0_0] input.MRInput: cbo_t3 initialized RecordReader from event 2018-01-05T11:28:59,747 DEBUG [TezTR-518813_1_26_0_0_0] tez.MapRecordProcessor: Starting Output: out_Map 1 2018-01-05T11:28:59,747 INFO [TezTR-518813_1_26_0_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:28:59,747 INFO [TezTR-518813_1_26_0_0_0] exec.MapOperator: Initializing operator MAP[0] 2018-01-05T11:28:59,747 DEBUG [TezTR-518813_1_26_0_0_0] exec.MapOperator: Initialization Done 0 MAP 2018-01-05T11:28:59,747 DEBUG [TezTR-518813_1_26_0_0_0] exec.MapOperator: Operator 0 MAP initialized 2018-01-05T11:28:59,747 DEBUG [TezTR-518813_1_26_0_0_0] exec.MapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:28:59,747 DEBUG [TezTR-518813_1_26_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,747 DEBUG [TezTR-518813_1_26_0_0_0] exec.MapOperator: Adding alias cbo_t3 to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3 2018-01-05T11:28:59,747 DEBUG [TezTR-518813_1_26_0_0_0] exec.MapOperator: dump TS[0] struct 2018-01-05T11:28:59,747 INFO [TezTR-518813_1_26_0_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:28:59,747 INFO [TezTR-518813_1_26_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:28:59,747 DEBUG [TezTR-518813_1_26_0_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:28:59,747 INFO [TezTR-518813_1_26_0_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:28:59,747 DEBUG [TezTR-518813_1_26_0_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:28:59,747 DEBUG [TezTR-518813_1_26_0_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:28:59,747 DEBUG [TezTR-518813_1_26_0_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:28:59,747 DEBUG [TezTR-518813_1_26_0_0_0] log.PerfLogger: 2018-01-05T11:28:59,747 INFO [TezTR-518813_1_26_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_571_7301919214835618340-1/djaiswal/_tez_scratch_dir/e1c9f955-2c45-4ea3-a78c-0915cecba053/map.xml 2018-01-05T11:28:59,747 DEBUG [TezTR-518813_1_26_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:28:59,747 DEBUG [TezTR-518813_1_26_0_0_0] exec.MapOperator: Processing alias(es) cbo_t3 for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3/cbo_t3.txt 2018-01-05T11:28:59,747 INFO [TezTR-518813_1_26_0_0_0] exec.MapOperator: MAP[0]: records read - 1 2018-01-05T11:28:59,747 INFO [TezTR-518813_1_26_0_0_0] exec.MapOperator: MAP[0]: records read - 10 2018-01-05T11:28:59,747 DEBUG [TezTR-518813_1_26_0_0_0] log.PerfLogger: 2018-01-05T11:28:59,747 DEBUG [TezTR-518813_1_26_0_0_0] mr.ObjectCache: djaiswal_20180105112859_7fb2c9ac-06b5-4ba2-87ba-2dd685912638_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:28:59,747 DEBUG [TezTR-518813_1_26_0_0_0] exec.MapOperator: close called for operator MAP[0] 2018-01-05T11:28:59,747 INFO [TezTR-518813_1_26_0_0_0] exec.MapOperator: Closing operator MAP[0] 2018-01-05T11:28:59,747 INFO [TezTR-518813_1_26_0_0_0] exec.MapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:20, 2018-01-05T11:28:59,747 DEBUG [TezTR-518813_1_26_0_0_0] exec.MapOperator: Closing child = TS[0] 2018-01-05T11:28:59,747 DEBUG [TezTR-518813_1_26_0_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:28:59,747 DEBUG [TezTR-518813_1_26_0_0_0] exec.TableScanOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:28:59,747 INFO [TezTR-518813_1_26_0_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:28:59,747 DEBUG [TezTR-518813_1_26_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,747 DEBUG [TezTR-518813_1_26_0_0_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_571_7301919214835618340-1/-mr-10000 2018-01-05T11:28:59,748 INFO [TezTR-518813_1_26_0_0_0] exec.TableScanOperator: publishing : default.cbo_t3/ : {numRows=20, rawDataSize=262} 2018-01-05T11:28:59,769 DEBUG [TezTR-518813_1_26_0_0_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_571_7301919214835618340-1/-mr-10000/tmpstats-0_0 2018-01-05T11:28:59,769 DEBUG [TezTR-518813_1_26_0_0_0] fs.FSStatsPublisher: Writing stats in it : {default.cbo_t3/={numRows=20, rawDataSize=262}} 2018-01-05T11:28:59,770 DEBUG [TezTR-518813_1_26_0_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:28:59,770 DEBUG [TezTR-518813_1_26_0_0_0] exec.MapOperator: 0 Close done 2018-01-05T11:28:59,770 INFO [TezTR-518813_1_26_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_26_00_000000_0 2018-01-05T11:28:59,770 INFO [TezTR-518813_1_26_0_0_0] output.MROutput: out_Map 1 closed 2018-01-05T11:28:59,770 DEBUG [TezTR-518813_1_26_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:28:59,770 INFO [TezTR-518813_1_26_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:28:59,770 INFO [TezTR-518813_1_26_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_26_00_000000_0, askedToStop=false 2018-01-05T11:28:59,770 INFO [TezTR-518813_1_26_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_26_00_000000_0, stopRequested=false 2018-01-05T11:28:59,770 INFO [TezTR-518813_1_26_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_26_00_000000_0: Counters: 5 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=20, INPUT_SPLIT_LENGTH_BYTES=282, OUTPUT_RECORDS=0][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=20]] 2018-01-05T11:28:59,770 DEBUG [TezTR-518813_1_26_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_26_0_0_0 thread id: 1750 scheme: file bytesRead: 298 bytesWritten: 83 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:28:59,770 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_26_00_000000_0, isTaskDone=true 2018-01-05T11:28:59,770 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000055, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_26_00_000000_0, eventCount=2 } 2018-01-05T11:28:59,771 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_26][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_26_00_000000_0, creationTime=1515180539722, allocationTime=1515180539722, startTime=1515180539724, finishTime=1515180539771, timeTaken=47, status=SUCCEEDED, diagnostics= 2018-01-05T11:28:59,771 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_26_00_000000_0 2018-01-05T11:28:59,771 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_26_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:28:59,771 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_26_00_000000_0; false 2018-01-05T11:28:59,771 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_26_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:28:59,771 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:28:59,771 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:28:59,771 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_26_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:28:59,771 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_26_00_000000_0 2018-01-05T11:28:59,771 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:59,772 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000055=34 2018-01-05T11:28:59,772 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_26_00_000000_0: true 2018-01-05T11:28:59,772 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_26][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_26_00_000000, startTime=1515180539724, finishTime=1515180539771, timeTaken=47, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_26_00_000000_0, diagnostics=, counters=Counters: 11, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=298, FILE_BYTES_WRITTEN=83, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=20, INPUT_SPLIT_LENGTH_BYTES=282, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=20 2018-01-05T11:28:59,772 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_26_00_000000_0 2018-01-05T11:28:59,772 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000055, QueryId=djaiswal_20180105112859_7fb2c9ac-06b5-4ba2-87ba-2dd685912638-26, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:28:59,772 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_26_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:28:59,772 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_26_00_000000_0 2018-01-05T11:28:59,772 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_26_00 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:28:59,772 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_26_00 [Map 1] 2018-01-05T11:28:59,772 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_26][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_26_00, initRequestedTime=1515180539695, initedTime=1515180539721, startRequestedTime=1515180539702, startedTime=1515180539721, finishTime=1515180539772, timeTaken=51, status=SUCCEEDED, diagnostics=, counters=Counters: 11, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=298, FILE_BYTES_WRITTEN=83, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=20, INPUT_SPLIT_LENGTH_BYTES=282, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=20, vertexStats=firstTaskStartTime=1515180539724, firstTasksToStart=[ task_1515180518813_0001_26_00_000000 ], lastTaskFinishTime=1515180539771, lastTasksToFinish=[ task_1515180518813_0001_26_00_000000 ], minTaskDuration=47, maxTaskDuration=47, avgTaskDuration=47.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_26_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_26_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:28:59,772 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_26_00 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:28:59,772 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_26][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000055, stoppedTime=1515180539772, exitStatus=0 2018-01-05T11:28:59,772 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000055 exited with diagnostics set to Container completed. 2018-01-05T11:28:59,772 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_26_00 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1 2018-01-05T11:28:59,772 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null 2018-01-05T11:28:59,772 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_26_00 [Map 1] 2018-01-05T11:28:59,773 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_26, queueSize=0 2018-01-05T11:28:59,773 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_26][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_26, startTime=1515180539695, finishTime=1515180539772, timeTaken=77, status=SUCCEEDED, diagnostics=, counters=Counters: 15, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, OTHER_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=60, AM_GC_TIME_MILLIS=0, File System Counters, FILE_BYTES_READ=298, FILE_BYTES_WRITTEN=83, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=20, INPUT_SPLIT_LENGTH_BYTES=282, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=20 2018-01-05T11:28:59,773 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_26 finished with state: SUCCEEDED 2018-01-05T11:28:59,773 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_26 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:28:59,773 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000055 2018-01-05T11:28:59,773 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_26, dagState=SUCCEEDED 2018-01-05T11:28:59,773 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_26_post 2018-01-05T11:28:59,773 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:59,773 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:28:59,773 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:28:59,773 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 26 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=1, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=1,NumRequestsWithoutLocation=1, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=1} 2018-01-05T11:28:59,773 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:28:59,773 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=analyze table cbo_t3 compute statistics (Stage-0), with id=dag_1515180518813_0001_26 2018-01-05T11:28:59,773 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 26, to 10.22.8.182:49818 2018-01-05T11:28:59,773 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=1, Remaining=0 2018-01-05T11:28:59,773 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=analyze table cbo_t3 compute statistics (Stage-0), with id=dag_1515180518813_0001_26 2018-01-05T11:28:59,774 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=26} 2018-01-05T11:28:59,774 INFO [IPC Server handler 3 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=26}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:28:59,774 INFO [IPC Server handler 3 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/26 after 300 seconds 2018-01-05T11:28:59,774 ERROR [IPC Server handler 3 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112859_7fb2c9ac-06b5-4ba2-87ba-2dd685912638, dag_1515180518813_0001_26 2018-01-05T11:28:59,774 INFO [IPC Server handler 3 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112859_7fb2c9ac-06b5-4ba2-87ba-2dd685912638 2018-01-05T11:28:59,774 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=26} = 0 2018-01-05T11:28:59,774 DEBUG [IPC Server handler 3 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=26} 2018-01-05T11:28:59,774 DEBUG [IPC Server handler 3 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:28:59,774 DEBUG [IPC Server handler 3 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=26} 2018-01-05T11:28:59,774 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,774 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,774 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 2018-01-05T11:28:59,774 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,774 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-2:STATS] in serial mode 2018-01-05T11:28:59,774 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@4095c586, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:28:59,775 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:59,775 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,775 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:28:59,775 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:28:59,775 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:28:59,775 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:28:59,775 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,775 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:28:59,775 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:28:59,775 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_571_7301919214835618340-1/-mr-10000 2018-01-05T11:28:59,775 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_571_7301919214835618340-1/-mr-10000 2018-01-05T11:28:59,776 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.cbo_t3/ numRows 2018-01-05T11:28:59,776 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.cbo_t3/ numRows 20 2018-01-05T11:28:59,776 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.cbo_t3/ rawDataSize 2018-01-05T11:28:59,776 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.cbo_t3/ rawDataSize 262 2018-01-05T11:28:59,776 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,777 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:28:59,777 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:28:59,777 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:28:59,777 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:28:59,777 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,777 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,777 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,778 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,778 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,778 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:28:59,778 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:28:59,778 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,778 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:28:59,778 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,778 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=cbo_t3 newtbl=cbo_t3 2018-01-05T11:28:59,778 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=cbo_t3 newtbl=cbo_t3 2018-01-05T11:28:59,780 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,786 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.cbo_t3 stats: [numFiles=1, numRows=20, totalSize=282, rawDataSize=262] 2018-01-05T11:28:59,786 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: About to delete stats tmp dir :file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_571_7301919214835618340-1/-mr-10000 2018-01-05T11:28:59,786 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,786 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,786 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: analyze table cbo_t3 compute statistics 2018-01-05T11:28:59,786 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:28:59,786 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@cbo_t3 2018-01-05T11:28:59,786 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@cbo_t3 2018-01-05T11:28:59,786 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,786 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,786 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:28:59,786 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=8} 2018-01-05T11:28:59,786 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112859_7fb2c9ac-06b5-4ba2-87ba-2dd685912638); Time taken: 0.188 seconds 2018-01-05T11:28:59,786 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:28:59,786 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,786 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:59,786 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,786 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query analyze table cbo_t3 compute statistics 2018-01-05T11:28:59,786 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_571_7301919214835618340-1 2018-01-05T11:28:59,787 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.215 seconds 2018-01-05T11:28:59,787 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:59,787 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:28:59,787 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:28:59,787 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:28:59,787 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,787 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:28:59,787 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,787 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,787 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,787 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: analyze table cbo_t3 compute statistics for columns key, value, c_int, c_float, c_boolean 2018-01-05T11:28:59,787 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112859_5f019fd9-7a31-4694-b9e6-c1ae3adfa206): analyze table cbo_t3 compute statistics for columns key, value, c_int, c_float, c_boolean 2018-01-05T11:28:59,787 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,787 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: analyze table cbo_t3 compute statistics for columns key, value, c_int, c_float, c_boolean 2018-01-05T11:28:59,787 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:59,787 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,787 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,787 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,788 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,788 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:59,788 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,788 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t3 2018-01-05T11:28:59,788 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t3 2018-01-05T11:28:59,790 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,790 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,790 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: select compute_stats(`key`, 'hll') , compute_stats(`value`, 'hll') , compute_stats(`c_int`, 'hll') , compute_stats(`c_float`, 'hll') , compute_stats(`c_boolean`, 'hll') from `default`.`cbo_t3` 2018-01-05T11:28:59,790 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: select compute_stats(`key`, 'hll') , compute_stats(`value`, 'hll') , compute_stats(`c_int`, 'hll') , compute_stats(`c_float`, 'hll') , compute_stats(`c_boolean`, 'hll') from `default`.`cbo_t3` 2018-01-05T11:28:59,791 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:28:59,791 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Invoking analyze on rewritten query 2018-01-05T11:28:59,791 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Starting Semantic Analysis 2018-01-05T11:28:59,791 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:59,791 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:59,791 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:59,791 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:59,791 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:59,791 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed phase 1 of Semantic Analysis 2018-01-05T11:28:59,791 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for source tables 2018-01-05T11:28:59,791 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,791 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t3 2018-01-05T11:28:59,791 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t3 2018-01-05T11:28:59,794 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,794 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,794 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for subqueries 2018-01-05T11:28:59,794 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for destination tables 2018-01-05T11:28:59,807 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed getting MetaData in Semantic Analysis 2018-01-05T11:28:59,807 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_790_2754539752752503788-1/-mr-10001 2018-01-05T11:28:59,808 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Table Plan for cbo_t3 TS[0] 2018-01-05T11:28:59,808 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:59,808 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:59,808 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:59,808 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:59,808 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:28:59,809 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: RR before GB cbo_t3{(key,key: string)(value,value: string)(c_int,c_int: int)(c_float,c_float: float)(c_boolean,c_boolean: boolean)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)(RAW__DATA__SIZE,RAW__DATA__SIZE: bigint)} after GB {((tok_function compute_stats (tok_table_or_col key) 'hll'),_col0: struct)((tok_function compute_stats (tok_table_or_col value) 'hll'),_col1: struct)((tok_function compute_stats (tok_table_or_col c_int) 'hll'),_col2: struct)((tok_function compute_stats (tok_table_or_col c_float) 'hll'),_col3: struct)((tok_function compute_stats (tok_table_or_col c_boolean) 'hll'),_col4: struct)} 2018-01-05T11:28:59,809 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: tree: (tok_select (tok_selexpr (tok_function compute_stats (tok_table_or_col key) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col value) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col c_int) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col c_float) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col c_boolean) 'hll'))) 2018-01-05T11:28:59,809 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: genSelectPlan: input = {((tok_function compute_stats (tok_table_or_col key) 'hll'),_col0: struct)((tok_function compute_stats (tok_table_or_col value) 'hll'),_col1: struct)((tok_function compute_stats (tok_table_or_col c_int) 'hll'),_col2: struct)((tok_function compute_stats (tok_table_or_col c_float) 'hll'),_col3: struct)((tok_function compute_stats (tok_table_or_col c_boolean) 'hll'),_col4: struct)} starRr = cbo_t3{(key,key: string)(value,value: string)(c_int,c_int: int)(c_float,c_float: float)(c_boolean,c_boolean: boolean)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)(RAW__DATA__SIZE,RAW__DATA__SIZE: bigint)} 2018-01-05T11:28:59,809 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Select Plan row schema: null{(_c0,_col0: struct)(_c1,_col1: struct)(_c2,_col2: struct)(_c3,_col3: struct)(_c4,_col4: struct)} 2018-01-05T11:28:59,809 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Select Plan for clause: insclause-0 2018-01-05T11:28:59,809 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_790_2754539752752503788-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-59_790_2754539752752503788-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_790_2754539752752503788-1/-mr-10000 2018-01-05T11:28:59,809 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_790_2754539752752503788-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-59_790_2754539752752503788-1 2018-01-05T11:28:59,834 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_790_2754539752752503788-1/-mr-10003 2018-01-05T11:28:59,834 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_790_2754539752752503788-1/-mr-10000 row schema: null{(_c0,_col0: struct)(_c1,_col1: struct)(_c2,_col2: struct)(_c3,_col3: struct)(_c4,_col4: struct)} 2018-01-05T11:28:59,834 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Body Plan for Query Block null 2018-01-05T11:28:59,834 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Plan for Query Block null 2018-01-05T11:28:59,834 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Before logical optimization TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-SEL[5]-FS[6] 2018-01-05T11:28:59,834 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,834 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,834 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] lineage.Generator: Time taken for lineage transform=1 2018-01-05T11:28:59,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{cbo_t3}key,value: string|{cbo_t3}value,c_int: int|{cbo_t3}c_int,c_float: float|{cbo_t3}c_float,c_boolean: boolean|{cbo_t3}c_boolean,BLOCK__OFFSET__INSIDE__FILE: bigint|{cbo_t3}block__offset__inside__file,INPUT__FILE__NAME: string|{cbo_t3}input__file__name,ROW__ID: struct|{cbo_t3}row__id,RAW__DATA__SIZE: bigint|{cbo_t3}raw__data__size) 2018-01-05T11:28:59,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: string|{cbo_t3}key,value: string|{cbo_t3}value,c_int: int|{cbo_t3}c_int,c_float: float|{cbo_t3}c_float,c_boolean: boolean|{cbo_t3}c_boolean,BLOCK__OFFSET__INSIDE__FILE: bigint|{cbo_t3}block__offset__inside__file,INPUT__FILE__NAME: string|{cbo_t3}input__file__name,ROW__ID: struct|{cbo_t3}row__id,RAW__DATA__SIZE: bigint|{cbo_t3}raw__data__size) 2018-01-05T11:28:59,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:59,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:59,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value] Column[c_int] Column[c_float] Column[c_boolean] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:28:59,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col c_int) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col c_float) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col c_boolean) 'hll')) 2018-01-05T11:28:59,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:59,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:59,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll'),VALUE._col2: struct|{}(tok_function compute_stats (tok_table_or_col c_int) 'hll'),VALUE._col3: struct|{}(tok_function compute_stats (tok_table_or_col c_float) 'hll'),VALUE._col4: struct|{}(tok_function compute_stats (tok_table_or_col c_boolean) 'hll')) 2018-01-05T11:28:59,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:59,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:59,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col c_int) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col c_float) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col c_boolean) 'hll')) 2018-01-05T11:28:59,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:59,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:59,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1,_col2: struct|{null}_c2,_col3: struct|{null}_c3,_col4: struct|{null}_c4) 2018-01-05T11:28:59,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:59,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:59,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0] Column[_col1] Column[_col2] Column[_col3] Column[_col4]) 2018-01-05T11:28:59,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{},_col2: struct|{},_col3: struct|{},_col4: struct|{}) 2018-01-05T11:28:59,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:59,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:59,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,835 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for FS(6) 2018-01-05T11:28:59,835 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(5) 2018-01-05T11:28:59,835 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for GBY(4) 2018-01-05T11:28:59,835 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for RS(3) 2018-01-05T11:28:59,835 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for GBY(2) 2018-01-05T11:28:59,835 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(1) 2018-01-05T11:28:59,835 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.PredicatePushDown: After PPD: TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-SEL[5]-FS[6] 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{cbo_t3}key,value: string|{cbo_t3}value,c_int: int|{cbo_t3}c_int,c_float: float|{cbo_t3}c_float,c_boolean: boolean|{cbo_t3}c_boolean,BLOCK__OFFSET__INSIDE__FILE: bigint|{cbo_t3}block__offset__inside__file,INPUT__FILE__NAME: string|{cbo_t3}input__file__name,ROW__ID: struct|{cbo_t3}row__id,RAW__DATA__SIZE: bigint|{cbo_t3}raw__data__size) 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: string|{cbo_t3}key,value: string|{cbo_t3}value,c_int: int|{cbo_t3}c_int,c_float: float|{cbo_t3}c_float,c_boolean: boolean|{cbo_t3}c_boolean,BLOCK__OFFSET__INSIDE__FILE: bigint|{cbo_t3}block__offset__inside__file,INPUT__FILE__NAME: string|{cbo_t3}input__file__name,ROW__ID: struct|{cbo_t3}row__id,RAW__DATA__SIZE: bigint|{cbo_t3}raw__data__size) 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value] Column[c_int] Column[c_float] Column[c_boolean] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col c_int) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col c_float) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col c_boolean) 'hll')) 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll'),VALUE._col2: struct|{}(tok_function compute_stats (tok_table_or_col c_int) 'hll'),VALUE._col3: struct|{}(tok_function compute_stats (tok_table_or_col c_float) 'hll'),VALUE._col4: struct|{}(tok_function compute_stats (tok_table_or_col c_boolean) 'hll')) 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col c_int) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col c_float) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col c_boolean) 'hll')) 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1,_col2: struct|{null}_c2,_col3: struct|{null}_c3,_col4: struct|{null}_c4) 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0] Column[_col1] Column[_col2] Column[_col3] Column[_col4]) 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{},_col2: struct|{},_col3: struct|{},_col4: struct|{}) 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{cbo_t3}key,value: string|{cbo_t3}value,c_int: int|{cbo_t3}c_int,c_float: float|{cbo_t3}c_float,c_boolean: boolean|{cbo_t3}c_boolean,BLOCK__OFFSET__INSIDE__FILE: bigint|{cbo_t3}block__offset__inside__file,INPUT__FILE__NAME: string|{cbo_t3}input__file__name,ROW__ID: struct|{cbo_t3}row__id,RAW__DATA__SIZE: bigint|{cbo_t3}raw__data__size) 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: string|{cbo_t3}key,value: string|{cbo_t3}value,c_int: int|{cbo_t3}c_int,c_float: float|{cbo_t3}c_float,c_boolean: boolean|{cbo_t3}c_boolean,BLOCK__OFFSET__INSIDE__FILE: bigint|{cbo_t3}block__offset__inside__file,INPUT__FILE__NAME: string|{cbo_t3}input__file__name,ROW__ID: struct|{cbo_t3}row__id,RAW__DATA__SIZE: bigint|{cbo_t3}raw__data__size) 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value] Column[c_int] Column[c_float] Column[c_boolean] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col c_int) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col c_float) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col c_boolean) 'hll')) 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll'),VALUE._col2: struct|{}(tok_function compute_stats (tok_table_or_col c_int) 'hll'),VALUE._col3: struct|{}(tok_function compute_stats (tok_table_or_col c_float) 'hll'),VALUE._col4: struct|{}(tok_function compute_stats (tok_table_or_col c_boolean) 'hll')) 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col c_int) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col c_float) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col c_boolean) 'hll')) 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1,_col2: struct|{null}_c2,_col3: struct|{null}_c3,_col4: struct|{null}_c4) 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0] Column[_col1] Column[_col2] Column[_col3] Column[_col4]) 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{},_col2: struct|{},_col3: struct|{},_col4: struct|{}) 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 3 key:[] 2018-01-05T11:28:59,836 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 3 oldColExprMap: {VALUE._col2=Column[_col2], VALUE._col3=Column[_col3], VALUE._col4=Column[_col4], VALUE._col0=Column[_col0], VALUE._col1=Column[_col1]} 2018-01-05T11:28:59,836 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 3 newColExprMap: {VALUE._col2=Column[_col2], VALUE._col3=Column[_col3], VALUE._col4=Column[_col4], VALUE._col0=Column[_col0], VALUE._col1=Column[_col1]} 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,837 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,837 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,837 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,837 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,837 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,837 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,837 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,837 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,837 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,837 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,837 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.IdentityProjectRemover: Identity project remover optimization removed : SEL[5] 2018-01-05T11:28:59,837 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,837 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,837 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,837 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,837 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,837 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,837 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,837 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: After logical optimization TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-FS[6] 2018-01-05T11:28:59,837 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,837 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,837 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,837 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,837 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,837 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,837 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=cbo_t3 2018-01-05T11:28:59,837 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=cbo_t3 2018-01-05T11:28:59,843 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 3.589572ms + 0.007511ms, the query is [select "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "TAB_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (...)] 2018-01-05T11:28:59,844 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,850 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:28:59,851 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:28:59,851 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting stats (Num rows: 20 Data size: 7600 Basic stats: COMPLETE Column stats: NONE) on TS[0] 2018-01-05T11:28:59,851 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (cbo_t3): numRows: 20 dataSize: 7600 basicStatsState: COMPLETE colStatsState: NONE colStats: {c_float= colName: c_float colType: float countDistincts: 4 numNulls: 1 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 1.4E-45 max: 3.4028235E38 ] isPrimaryKey: false isEstimated: true, c_int= colName: c_int colType: int countDistincts: 4 numNulls: 1 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, value= colName: value colType: string countDistincts: 4 numNulls: 1 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: string countDistincts: 4 numNulls: 1 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, c_boolean= colName: c_boolean colType: boolean countDistincts: 2 numNulls: 1 avgColLen: 4.0 numTrues: 10 numFalses: 10 isPrimaryKey: false isEstimated: true} 2018-01-05T11:28:59,851 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting stats (Num rows: 20 Data size: 7600 Basic stats: COMPLETE Column stats: NONE) on SEL[1] 2018-01-05T11:28:59,851 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [1] STATS-SEL[1]: numRows: 20 dataSize: 7600 basicStatsState: COMPLETE colStatsState: NONE colStats: {c_float= colName: c_float colType: float countDistincts: 4 numNulls: 1 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 1.4E-45 max: 3.4028235E38 ] isPrimaryKey: false isEstimated: true, c_int= colName: c_int colType: int countDistincts: 4 numNulls: 1 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, value= colName: value colType: string countDistincts: 4 numNulls: 1 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: string countDistincts: 4 numNulls: 1 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, c_boolean= colName: c_boolean colType: boolean countDistincts: 2 numNulls: 1 avgColLen: 4.0 numTrues: 10 numFalses: 10 isPrimaryKey: false isEstimated: true} 2018-01-05T11:28:59,851 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[2]: inputSize: 7600 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:28:59,851 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [Case 1] STATS-GBY[2]: cardinality: 20 2018-01-05T11:28:59,851 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 2380 Basic stats: COMPLETE Column stats: NONE) on GBY[2] 2018-01-05T11:28:59,851 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-GBY[2]: numRows: 1 dataSize: 2380 basicStatsState: COMPLETE colStatsState: NONE colStats: {c_float= colName: c_float colType: float countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 1.4E-45 max: 3.4028235E38 ] isPrimaryKey: false isEstimated: true, _col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col4= colName: _col4 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 272.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, c_int= colName: c_int colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, c_boolean= colName: c_boolean colType: boolean countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 10 numFalses: 10 isPrimaryKey: false isEstimated: true} 2018-01-05T11:28:59,851 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 2380 Basic stats: COMPLETE Column stats: NONE) on RS[3] 2018-01-05T11:28:59,851 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-RS[3]: numRows: 1 dataSize: 2380 basicStatsState: COMPLETE colStatsState: NONE colStats: {c_float= colName: c_float colType: float countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 1.4E-45 max: 3.4028235E38 ] isPrimaryKey: false isEstimated: true, _col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col4= colName: _col4 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 272.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, c_int= colName: c_int colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, c_boolean= colName: c_boolean colType: boolean countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 10 numFalses: 10 isPrimaryKey: false isEstimated: true} 2018-01-05T11:28:59,851 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[4]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:28:59,851 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [Case 7] STATS-GBY[4]: cardinality: 0 2018-01-05T11:28:59,851 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[4]: Equals 0 in number of rows. 0 rows will be set to 1 2018-01-05T11:28:59,851 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 2412 Basic stats: COMPLETE Column stats: NONE) on GBY[4] 2018-01-05T11:28:59,851 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-GBY[4]: numRows: 1 dataSize: 2412 basicStatsState: COMPLETE colStatsState: NONE colStats: {c_float= colName: c_float colType: float countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 1.4E-45 max: 3.4028235E38 ] isPrimaryKey: false isEstimated: true, _col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col4= colName: _col4 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 272.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, c_int= colName: c_int colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, c_boolean= colName: c_boolean colType: boolean countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 10 numFalses: 10 isPrimaryKey: false isEstimated: true} 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-FS[6]: numRows: 1 dataSize: 2412 basicStatsState: COMPLETE colStatsState: NONE colStats: {c_float= colName: c_float colType: float countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 1.4E-45 max: 3.4028235E38 ] isPrimaryKey: false isEstimated: true, _col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col4= colName: _col4 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 272.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, c_int= colName: c_int colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, c_boolean= colName: c_boolean colType: boolean countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 10 numFalses: 10 isPrimaryKey: false isEstimated: true} 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on TS[0] 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on SEL[1] 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[2] 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[3] 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[4] 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FileSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on FS[6] 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,852 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: FS, 6 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: GBY, 4 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: RS, 3 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: GBY, 2 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: SEL, 1 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:28:59,852 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Cycle free: true 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{cbo_t3}key,value: string|{cbo_t3}value,c_int: int|{cbo_t3}c_int,c_float: float|{cbo_t3}c_float,c_boolean: boolean|{cbo_t3}c_boolean,BLOCK__OFFSET__INSIDE__FILE: bigint|{cbo_t3}block__offset__inside__file,INPUT__FILE__NAME: string|{cbo_t3}input__file__name,ROW__ID: struct|{cbo_t3}row__id,RAW__DATA__SIZE: bigint|{cbo_t3}raw__data__size) 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: string|{cbo_t3}key,value: string|{cbo_t3}value,c_int: int|{cbo_t3}c_int,c_float: float|{cbo_t3}c_float,c_boolean: boolean|{cbo_t3}c_boolean) 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value] Column[c_int] Column[c_float] Column[c_boolean]) 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col c_int) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col c_float) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col c_boolean) 'hll')) 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll'),VALUE._col2: struct|{}(tok_function compute_stats (tok_table_or_col c_int) 'hll'),VALUE._col3: struct|{}(tok_function compute_stats (tok_table_or_col c_float) 'hll'),VALUE._col4: struct|{}(tok_function compute_stats (tok_table_or_col c_boolean) 'hll')) 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col c_int) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col c_float) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col c_boolean) 'hll')) 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{},_col2: struct|{},_col3: struct|{},_col4: struct|{}) 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: TS[0] 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: RS[3] 2018-01-05T11:28:59,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3 of table cbo_t3 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: RS[3] 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: GBY[4] 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: FS[6] 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding reduce work (Reducer 2) for GBY[4] 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Setting up reduce sink: RS[3] with following reduce work: Reducer 2 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Removing RS[3] as parent from GBY[4] 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: FS[6] 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping vectorization 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:28:59,853 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: llap mode: only 2018-01-05T11:28:59,853 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(key, 'hll')' 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'key' 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(value, 'hll')' 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'value' 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(c_int, 'hll')' 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'c_int' 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(c_float, 'hll')' 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'c_float' 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(c_boolean, 'hll')' 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'c_boolean' 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'key' 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'value' 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'c_int' 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'c_float' 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'c_boolean' 2018-01-05T11:28:59,853 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:59,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@3911b515 can be done in LLAP 2018-01-05T11:28:59,854 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Reducer 2 2018-01-05T11:28:59,854 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col0)' 2018-01-05T11:28:59,854 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:28:59,854 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col1)' 2018-01-05T11:28:59,854 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col1' 2018-01-05T11:28:59,854 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col2)' 2018-01-05T11:28:59,854 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col2' 2018-01-05T11:28:59,854 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col3)' 2018-01-05T11:28:59,854 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col3' 2018-01-05T11:28:59,854 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col4)' 2018-01-05T11:28:59,854 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col4' 2018-01-05T11:28:59,854 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:28:59,854 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@6da58910 can be done in LLAP 2018-01-05T11:28:59,854 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,854 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed plan generation 2018-01-05T11:28:59,854 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:28:59,854 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validation start 2018-01-05T11:28:59,854 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Not a partition. 2018-01-05T11:28:59,854 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:28:59,854 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,854 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0, type:struct, comment:null), FieldSchema(name:_c1, type:struct, comment:null), FieldSchema(name:_c2, type:struct, comment:null), FieldSchema(name:_c3, type:struct, comment:null), FieldSchema(name:_c4, type:struct, comment:null)], properties:null) 2018-01-05T11:28:59,854 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,854 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:28:59,854 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=5, flushCache_()=1, getTableColumnStatistics_(String, String, List, )=14} 2018-01-05T11:28:59,854 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112859_5f019fd9-7a31-4694-b9e6-c1ae3adfa206); Time taken: 0.067 seconds 2018-01-05T11:28:59,854 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:28:59,854 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,854 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112859_5f019fd9-7a31-4694-b9e6-c1ae3adfa206): analyze table cbo_t3 compute statistics for columns key, value, c_int, c_float, c_boolean 2018-01-05T11:28:59,855 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,855 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: analyze table cbo_t3 compute statistics for columns key, value, c_int, c_float, c_boolean 2018-01-05T11:28:59,855 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:28:59,855 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@cbo_t3 2018-01-05T11:28:59,855 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@cbo_t3 2018-01-05T11:28:59,855 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_790_2754539752752503788-1/-mr-10000 2018-01-05T11:28:59,855 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,855 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,855 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,855 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Query ID = djaiswal_20180105112859_5f019fd9-7a31-4694-b9e6-c1ae3adfa206 2018-01-05T11:28:59,855 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Total jobs = 1 2018-01-05T11:28:59,855 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,855 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:28:59,856 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MAPRED] in serial mode 2018-01-05T11:28:59,875 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000055, QueryId=djaiswal_20180105112859_7fb2c9ac-06b5-4ba2-87ba-2dd685912638-26, DagId=26, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180539737, EndTime=1515180539875 2018-01-05T11:28:59,876 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_787_3068125115156391877-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:28:59,901 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:28:59,901 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:28:59,901 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:28:59,901 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112859_5f019fd9-7a31-4694-b9e6-c1ae3adfa206 2018-01-05T11:28:59,901 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Session is already open 2018-01-05T11:28:59,901 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:28:59,901 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,901 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Dag name: analyze table cbo_t3 compute sta...c_boolean (Stage-0) 2018-01-05T11:28:59,901 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: DagInfo: {"context":"Hive","description":"\nanalyze table cbo_t3 compute statistics for columns key, value, c_int, c_float, c_boolean"} 2018-01-05T11:28:59,901 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112859_5f019fd9-7a31-4694-b9e6-c1ae3adfa206 with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:28:59,901 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,914 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,914 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:28:59,915 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,915 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 2 size: 2.70KB 2018-01-05T11:28:59,931 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,931 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,942 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:28:59,942 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,942 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:28:59,943 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,943 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 4.27KB 2018-01-05T11:28:59,949 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_790_2754539752752503788-1/-mr-10001 2018-01-05T11:28:59,961 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_790_2754539752752503788-1/-mr-10001 2018-01-05T11:28:59,961 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3 2018-01-05T11:28:59,961 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,963 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,963 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,963 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=analyze table cbo_t3 compute sta...c_boolean (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112859_5f019fd9-7a31-4694-b9e6-c1ae3adfa206 } 2018-01-05T11:28:59,963 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:59,964 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:28:59,965 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: analyze table cbo_t3 compute sta...c_boolean (Stage-0) 2018-01-05T11:28:59,965 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:28:59,966 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_27, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_27.dot 2018-01-05T11:28:59,966 INFO [IPC Server handler 1 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_27 2018-01-05T11:28:59,966 WARN [IPC Server handler 1 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:28:59,966 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Running DAG: analyze table cbo_t3 compute sta...c_boolean (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112859_5f019fd9-7a31-4694-b9e6-c1ae3adfa206 } 2018-01-05T11:28:59,989 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_27][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_27, submitTime=1515180539965, queueName=null 2018-01-05T11:28:59,990 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_27_00 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_27_00 [Map 1], Default Resources= 2018-01-05T11:28:59,990 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_27_00 [Map 1] 2018-01-05T11:28:59,990 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:28:59,990 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_27_00 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:59,990 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_27_01 [Reducer 2]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_27_01 [Reducer 2], Default Resources= 2018-01-05T11:28:59,990 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_27_01 [Reducer 2] 2018-01-05T11:28:59,990 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 2 2018-01-05T11:28:59,990 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_27_01 [Reducer 2] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:28:59,990 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:28:59,990 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_27][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_27, initTime=1515180539990 2018-01-05T11:28:59,990 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: dag_1515180518813_0001_27 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:28:59,991 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:28:59,991 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_27][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_27, startTime=1515180539991 2018-01-05T11:28:59,991 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_27 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:28:59,991 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {cbo_t3={InputName=cbo_t3}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:28:59,991 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: cbo_t3, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:28:59,991 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_27_00 [Map 1] 2018-01-05T11:28:59,991 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_27, dagName=analyze table cbo_t3 compute sta...c_boolean (Stage-0) 2018-01-05T11:28:59,991 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,991 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,991 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:28:59,995 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_27_00 [Map 1] 2018-01-05T11:28:59,995 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_27_00 [Map 1] 2018-01-05T11:28:59,995 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:28:59,995 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_27_00 [Map 1] 2018-01-05T11:28:59,998 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:28:59,998 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@420b544f 2018-01-05T11:28:59,998 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_787_3068125115156391877-1/djaiswal/_tez_scratch_dir/50a73098-a6c8-4456-8bd7-08d927f8d676/map.xml 2018-01-05T11:28:59,998 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_787_3068125115156391877-1/djaiswal/_tez_scratch_dir/50a73098-a6c8-4456-8bd7-08d927f8d676/map.xml 2018-01-05T11:28:59,998 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_787_3068125115156391877-1/djaiswal/_tez_scratch_dir/50a73098-a6c8-4456-8bd7-08d927f8d676/map.xml 2018-01-05T11:28:59,998 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:59,998 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:28:59,998 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:28:59,999 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:28:59,999 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 4.27KB 2018-01-05T11:28:59,999 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_27_00 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:28:59,999 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: cbo_t3 on vertex vertex_1515180518813_0001_27_00 [Map 1] 2018-01-05T11:29:00,000 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:29:00,000 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to ShuffleVertexManager for vertex_1515180518813_0001_27_01 [Reducer 2] 2018-01-05T11:29:00,000 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:00,000 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:29:00,000 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:00,000 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_787_3068125115156391877-1/djaiswal/_tez_scratch_dir/50a73098-a6c8-4456-8bd7-08d927f8d676/map.xml 2018-01-05T11:29:00,000 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:00,000 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias cbo_t3 2018-01-05T11:29:00,000 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3 2018-01-05T11:29:00,000 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3 2018-01-05T11:29:00,000 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 0,1,2,3,4 2018-01-05T11:29:00,000 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = key,value,c_int,c_float,c_boolean 2018-01-05T11:29:00,000 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3 2018-01-05T11:29:00,001 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:00,001 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:00,001 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:00,001 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:00,004 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.05 maxFrac: 0.75 auto: false desiredTaskIput: 104857600 2018-01-05T11:29:00,004 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:29:00,004 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 1 tasks for vertex: vertex_1515180518813_0001_27_01 [Reducer 2] 2018-01-05T11:29:00,004 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_27_01 [Reducer 2] 2018-01-05T11:29:00,004 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_27][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_27_01, reconfigureDoneTime=1515180540004, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:29:00,004 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_27_01 [Reducer 2], numAdditionalOutputs=1 2018-01-05T11:29:00,004 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_27][Event:VERTEX_INITIALIZED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_27_01, initRequestedTime=1515180539999, initedTime=1515180540004, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:00,004 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_27_01 [Reducer 2] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:29:00,015 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Time taken to get FileStatuses: 14 2018-01-05T11:29:00,015 INFO [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:29:00,015 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 14 2018-01-05T11:29:00,015 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 1 2018-01-05T11:29:00,015 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:00,015 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 1. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:29:00,015 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_787_3068125115156391877-1/djaiswal/_tez_scratch_dir/50a73098-a6c8-4456-8bd7-08d927f8d676/map.xml 2018-01-05T11:29:00,015 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_787_3068125115156391877-1/djaiswal/_tez_scratch_dir/50a73098-a6c8-4456-8bd7-08d927f8d676/map.xml 2018-01-05T11:29:00,015 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_787_3068125115156391877-1/djaiswal/_tez_scratch_dir/50a73098-a6c8-4456-8bd7-08d927f8d676/map.xml 2018-01-05T11:29:00,015 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:00,015 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:00,015 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,016 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:00,016 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 4.27KB 2018-01-05T11:29:00,016 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3/cbo_t3.txt to src new group? true 2018-01-05T11:29:00,016 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:29:00,016 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:29:00,016 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:29:00,016 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 5 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 282 Original splits: 1 2018-01-05T11:29:00,016 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Using original number of splits: 1 desired splits: 1 2018-01-05T11:29:00,016 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 1 grouped split count is 1, for bucket: 1 2018-01-05T11:29:00,016 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:29:00,016 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: cbo_t3 on vertex vertex_1515180518813_0001_27_00 [Map 1] 2018-01-05T11:29:00,017 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_27_00 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:00,017 INFO [App Shared Pool - #0] impl.VertexImpl: Got updated RootInputsSpecs: {cbo_t3=forAllWorkUnits=true, update=[1]} 2018-01-05T11:29:00,017 INFO [App Shared Pool - #0] impl.VertexImpl: Vertex vertex_1515180518813_0001_27_00 [Map 1] parallelism set to 1 2018-01-05T11:29:00,017 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_27_00 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:00,017 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_27][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_27_00, initRequestedTime=1515180539991, initedTime=1515180540017, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:00,017 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_27_00 [Map 1] with distanceFromRoot: 0 2018-01-05T11:29:00,017 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_27_00 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:29:00,018 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_27][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_27_00, startRequestedTime=1515180539999, startedTime=1515180540018 2018-01-05T11:29:00,018 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_27][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_27_00, reconfigureDoneTime=1515180540018, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:29:00,018 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_27_00 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:00,018 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_27_00 for vertex: vertex_1515180518813_0001_27_01 [Reducer 2] numStartedSources: 1 numSources: 1 2018-01-05T11:29:00,018 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_27_01 [Reducer 2] with distanceFromRoot: 1 2018-01-05T11:29:00,018 INFO [App Shared Pool - #0] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:29:00,018 INFO [App Shared Pool - #0] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:29:00,018 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_27][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_27_01, startRequestedTime=1515180540018, startedTime=1515180540018 2018-01-05T11:29:00,018 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_27_00 [Map 1] 2018-01-05T11:29:00,018 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_27_01 [Reducer 2] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:00,018 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_27][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_27_00_000000, scheduledTime=1515180540018, launchTime=1515180540018 2018-01-05T11:29:00,018 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_27_00 [Map 1] 2018-01-05T11:29:00,018 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_27_00_000000_0, priority=5, capability=, hosts=null 2018-01-05T11:29:00,018 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=5,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:00,018 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: OnVertexStarted vertex: Reducer 2 with 0 source tasks and 1 pending tasks 2018-01-05T11:29:00,018 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=5,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:00,018 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:00,019 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_27_00_000000_0 on hosts=null 2018-01-05T11:29:00,019 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Map 1 in vertex: Reducer 2 numBipartiteSourceTasks: 1 2018-01-05T11:29:00,019 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_27_00_000000_0 on hosts=null 2018-01-05T11:29:00,019 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_27_00_000000_0 2018-01-05T11:29:00,019 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:00,019 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #56, task=TaskInfo{task=attempt_1515180518813_0001_27_00_000000_0, priority=5, startTime=0, containerId=null, uniqueId=55, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000056 2018-01-05T11:29:00,019 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_27_00_000000_0; false 2018-01-05T11:29:00,019 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_27_00_000000_0, priority=5, startTime=311560528, containerId=container_111101111_0001_00_000056assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=55, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:00,019 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000056 for node: 10.22.8.182:49818 2018-01-05T11:29:00,021 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 27, name=analyze table cbo_t3 compute sta...c_boolean (Stage-0), queryId=djaiswal_20180105112859_5f019fd9-7a31-4694-b9e6-c1ae3adfa206 2018-01-05T11:29:00,021 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000056, attempt_1515180518813_0001_27_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:00,021 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000056 for node: 10.22.8.182:49818 2018-01-05T11:29:00,021 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000056 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:00,021 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000056, launchTime=1515180540021 2018-01-05T11:29:00,022 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_27_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112859_5f019fd9-7a31-4694-b9e6-c1ae3adfa206, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000056, dagName=analyze table cbo_t3 compute sta...c_boolean (Stage-0), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{cbo_t3,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{Reducer 2,org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput,1}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180539991, firstAttemptStartTime=1515180540018, currentAttemptStartTime=1515180540021} 2018-01-05T11:29:00,022 INFO [IPC Server handler 1 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000056, QueryId=djaiswal_20180105112859_5f019fd9-7a31-4694-b9e6-c1ae3adfa206-27, DagId=27, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180540022 2018-01-05T11:29:00,022 DEBUG [IPC Server handler 1 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=27} with the ShuffleHandler 2018-01-05T11:29:00,022 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_27_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000056] on NM: [10.22.8.182:49818] 2018-01-05T11:29:00,022 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:00,022 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:00,022 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_27][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_27_00_000000_0, startTime=1515180540021, containerId=container_111101111_0001_00_000056, nodeId=10.22.8.182:49818 2018-01-05T11:29:00,033 INFO [IPC Server handler 1 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:29:00,033 DEBUG [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/27] 2018-01-05T11:29:00,033 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:29:00,033 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_27_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:00,033 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_27_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180540018, dagStartTime=1515180539991, withinDagPriority=5, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:29:00,033 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_27_00_000000_0 : ACCEPTED 2018-01-05T11:29:00,033 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_27_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:29:00,033 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_27_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180540018, dagStartTime=1515180539991, withinDagPriority=5, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:29:00,033 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_27_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:00,033 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_27_00_000000_0 to pre-emption queue 2018-01-05T11:29:00,034 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_27_00_000000_0: true 2018-01-05T11:29:00,034 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:1400285116 2018-01-05T11:29:00,034 INFO [TaskCommunicator # 7] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_27_00_000000_0 2018-01-05T11:29:00,034 INFO [TaskCommunicator # 7] GuaranteedTasks: Received response for attempt_1515180518813_0001_27_00_000000_0, true 2018-01-05T11:29:00,035 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : analyze table cbo_t3 compute sta...c_boolean (Stage-0), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_27_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=cbo_t3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=Reducer 2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput }}, ] 2018-01-05T11:29:00,035 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1037041664, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:00,035 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_27_00_000000_0 vertexName: Map 1 2018-01-05T11:29:00,035 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000056, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_27_00_000000_0, eventCount=1 } 2018-01-05T11:29:00,035 DEBUG [TezTR-518813_1_27_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_27 queryId: djaiswal_20180105112859_5f019fd9-7a31-4694-b9e6-c1ae3adfa206 instanceType: TaskRunner2Callable 2018-01-05T11:29:00,036 INFO [TezTR-518813_1_27_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_27_00_000000_0 2018-01-05T11:29:00,036 DEBUG [TezTR-518813_1_27_0_0_0] log.PerfLogger: 2018-01-05T11:29:00,037 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:00,037 INFO [I/O Setup 1 Initialize: {Reducer 2}] writers.UnorderedPartitionedKVWriter: Requested BufferSize (tez.runtime.unordered.output.buffer.size-mb) : 100 2018-01-05T11:29:00,037 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:00,038 INFO [IPC Server handler 8 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_27_00_000000_0 sent events: (0-1). 2018-01-05T11:29:00,038 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:29:00,038 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_27_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:29:00,039 DEBUG [TezTR-518813_1_27_0_0_0] log.PerfLogger: 2018-01-05T11:29:00,039 INFO [TezTR-518813_1_27_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:00,039 INFO [TezTR-518813_1_27_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:00,039 INFO [TezTR-518813_1_27_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:00,039 INFO [I/O Setup 0 Initialize: {cbo_t3}] input.MRInput: cbo_t3 using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:29:00,039 INFO [I/O Setup 0 Initialize: {cbo_t3}] input.MRInputLegacy: cbo_t3 MRInputLegacy deferring initialization 2018-01-05T11:29:00,039 INFO [TezTR-518813_1_27_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:00,039 INFO [TezTR-518813_1_27_0_0_0] resources.MemoryDistributor: InitialRequests=[cbo_t3:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [Reducer 2:OUTPUT:104857600:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput] 2018-01-05T11:29:00,039 INFO [TezTR-518813_1_27_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:00,039 INFO [TezTR-518813_1_27_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:00,039 INFO [TezTR-518813_1_27_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 104857600, TotalRequestedScaled: 5.24288E7, TotalJVMHeap: 1037041664, TotalAvailable: 694817914, TotalRequested/TotalJVMHeap:0.10 2018-01-05T11:29:00,039 INFO [TezTR-518813_1_27_0_0_0] resources.MemoryDistributor: Allocations=[cbo_t3:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput:OUTPUT:104857600:104857600] 2018-01-05T11:29:00,039 INFO [TezTR-518813_1_27_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:00,039 INFO [TezTR-518813_1_27_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: cbo_t3 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:00,039 INFO [TezTR-518813_1_27_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:29:00,039 INFO [TezTR-518813_1_27_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:00,039 INFO [I/O Setup 1 Start: {cbo_t3}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: cbo_t3 2018-01-05T11:29:00,039 INFO [TezTR-518813_1_27_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:00,040 INFO [TezTR-518813_1_27_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_27_00_000000_0 2018-01-05T11:29:00,040 DEBUG [TezTR-518813_1_27_0_0_0] log.PerfLogger: 2018-01-05T11:29:00,040 DEBUG [TezTR-518813_1_27_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_27_00_000000_0_10166 2018-01-05T11:29:00,040 DEBUG [TezTR-518813_1_27_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:00,040 DEBUG [TezTR-518813_1_27_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_27_00_000000_0 2018-01-05T11:29:00,040 INFO [TezTR-518813_1_27_0_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112859_5f019fd9-7a31-4694-b9e6-c1ae3adfa206 2018-01-05T11:29:00,040 DEBUG [TezTR-518813_1_27_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:00,040 DEBUG [TezTR-518813_1_27_0_0_0] log.PerfLogger: 2018-01-05T11:29:00,040 DEBUG [TezTR-518813_1_27_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:00,040 DEBUG [TezTR-518813_1_27_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:00,040 DEBUG [TezTR-518813_1_27_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112859_5f019fd9-7a31-4694-b9e6-c1ae3adfa206_Map 1__MAP_PLAN__ 2018-01-05T11:29:00,040 INFO [TezTR-518813_1_27_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_787_3068125115156391877-1/djaiswal/_tez_scratch_dir/50a73098-a6c8-4456-8bd7-08d927f8d676/map.xml 2018-01-05T11:29:00,040 DEBUG [TezTR-518813_1_27_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_787_3068125115156391877-1/djaiswal/_tez_scratch_dir/50a73098-a6c8-4456-8bd7-08d927f8d676/map.xml 2018-01-05T11:29:00,040 DEBUG [TezTR-518813_1_27_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_787_3068125115156391877-1/djaiswal/_tez_scratch_dir/50a73098-a6c8-4456-8bd7-08d927f8d676/map.xml 2018-01-05T11:29:00,040 DEBUG [TezTR-518813_1_27_0_0_0] log.PerfLogger: 2018-01-05T11:29:00,040 INFO [TezTR-518813_1_27_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:00,040 DEBUG [TezTR-518813_1_27_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,041 DEBUG [TezTR-518813_1_27_0_0_0] log.PerfLogger: 2018-01-05T11:29:00,041 INFO [TezTR-518813_1_27_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 4.27KB 2018-01-05T11:29:00,041 DEBUG [TezTR-518813_1_27_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:00,041 INFO [TezTR-518813_1_27_0_0_0] tez.MapRecordProcessor: The input names are: [cbo_t3] 2018-01-05T11:29:00,041 INFO [TezTR-518813_1_27_0_0_0] input.MRInput: Initialized MRInput: cbo_t3 2018-01-05T11:29:00,041 DEBUG [TezTR-518813_1_27_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 1 2018-01-05T11:29:00,041 INFO [TezTR-518813_1_27_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_787_3068125115156391877-1/djaiswal/_tez_scratch_dir/50a73098-a6c8-4456-8bd7-08d927f8d676/map.xml 2018-01-05T11:29:00,041 DEBUG [TezTR-518813_1_27_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:00,041 DEBUG [TezTR-518813_1_27_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3/cbo_t3.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string:int:float:boolean, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3, columns=key,value,c_int,c_float,c_boolean, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=,, numRows=20, numFiles=1, serialization.ddl=struct cbo_t3 { string key, string value, i32 c_int, float c_float, bool c_boolean}, transient_lastDdlTime=1515180539, rawDataSize=262, field.delim=,, columns.comments=, totalSize=282, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.cbo_t3}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string:int:float:boolean, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3, columns=key,value,c_int,c_float,c_boolean, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=,, numRows=20, numFiles=1, serialization.ddl=struct cbo_t3 { string key, string value, i32 c_int, float c_float, bool c_boolean}, transient_lastDdlTime=1515180539, rawDataSize=262, field.delim=,, columns.comments=, totalSize=282, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.cbo_t3}, baseFileName=cbo_t3, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string:int:float:boolean, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3, columns=key,value,c_int,c_float,c_boolean, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=,, numRows=20, numFiles=1, serialization.ddl=struct cbo_t3 { string key, string value, i32 c_int, float c_float, bool c_boolean}, transient_lastDdlTime=1515180539, rawDataSize=262, field.delim=,, columns.comments=, totalSize=282, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.cbo_t3}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string:int:float:boolean, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3, columns=key,value,c_int,c_float,c_boolean, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=,, numRows=20, numFiles=1, serialization.ddl=struct cbo_t3 { string key, string value, i32 c_int, float c_float, bool c_boolean}, transient_lastDdlTime=1515180539, rawDataSize=262, field.delim=,, columns.comments=, totalSize=282, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.cbo_t3}, baseFileName=cbo_t3, vectorPartitionDesc=null]} 2018-01-05T11:29:00,042 DEBUG [TezTR-518813_1_27_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:29:00,042 INFO [TezTR-518813_1_27_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:29:00,042 DEBUG [TezTR-518813_1_27_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:29:00,043 DEBUG [TezTR-518813_1_27_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:29:00,043 INFO [TezTR-518813_1_27_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_787_3068125115156391877-1/djaiswal/_tez_scratch_dir/50a73098-a6c8-4456-8bd7-08d927f8d676/map.xml 2018-01-05T11:29:00,043 DEBUG [TezTR-518813_1_27_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:00,043 INFO [TezTR-518813_1_27_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_27_00_000000_0 2018-01-05T11:29:00,043 INFO [TezTR-518813_1_27_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value, c_int, c_float, c_boolean] / schema.evolution.columns.types [string, string, int, float, boolean] (isAcidRead false) 2018-01-05T11:29:00,044 DEBUG [TezTR-518813_1_27_0_0_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:00,044 DEBUG [TezTR-518813_1_27_0_0_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:00,044 DEBUG [TezTR-518813_1_27_0_0_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:00,044 DEBUG [TezTR-518813_1_27_0_0_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:00,044 WARN [TezTR-518813_1_27_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3/cbo_t3.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:29:00,044 DEBUG [TezTR-518813_1_27_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:29:00,044 WARN [TezTR-518813_1_27_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3/cbo_t3.txt:0+282 2018-01-05T11:29:00,044 DEBUG [TezTR-518813_1_27_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3/cbo_t3.txt 2018-01-05T11:29:00,044 INFO [TezTR-518813_1_27_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3/cbo_t3.txt:0+282], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=282} 2018-01-05T11:29:00,044 INFO [TezTR-518813_1_27_0_0_0] input.MRInput: cbo_t3 initialized RecordReader from event 2018-01-05T11:29:00,044 DEBUG [TezTR-518813_1_27_0_0_0] tez.MapRecordProcessor: Starting Output: Reducer 2 2018-01-05T11:29:00,045 INFO [TezTR-518813_1_27_0_0_0] writers.BaseUnorderedPartitionedKVWriter: Instantiating Partitioner: [org.apache.tez.mapreduce.partition.MRPartitioner] 2018-01-05T11:29:00,069 INFO [TezTR-518813_1_27_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: numBuffers=2, sizePerBuffer=52428800, skipBuffers=true, numPartitions=1, availableMemory=104857600, maxSingleBufferSizeBytes=2147483647, pipelinedShuffle=false, isFinalMergeEnabled=true, numPartitions=1, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:29:00,069 INFO [TezTR-518813_1_27_0_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:29:00,069 INFO [TezTR-518813_1_27_0_0_0] exec.MapOperator: Initializing operator MAP[0] 2018-01-05T11:29:00,069 DEBUG [TezTR-518813_1_27_0_0_0] exec.MapOperator: Initialization Done 0 MAP 2018-01-05T11:29:00,069 DEBUG [TezTR-518813_1_27_0_0_0] exec.MapOperator: Operator 0 MAP initialized 2018-01-05T11:29:00,069 DEBUG [TezTR-518813_1_27_0_0_0] exec.MapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:29:00,070 DEBUG [TezTR-518813_1_27_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,070 DEBUG [TezTR-518813_1_27_0_0_0] exec.MapOperator: Adding alias cbo_t3 to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3 2018-01-05T11:29:00,070 DEBUG [TezTR-518813_1_27_0_0_0] exec.MapOperator: dump TS[0] struct 2018-01-05T11:29:00,070 INFO [TezTR-518813_1_27_0_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:29:00,070 INFO [TezTR-518813_1_27_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:00,070 DEBUG [TezTR-518813_1_27_0_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:29:00,070 INFO [TezTR-518813_1_27_0_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:29:00,070 DEBUG [TezTR-518813_1_27_0_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:29:00,070 DEBUG [TezTR-518813_1_27_0_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:29:00,070 DEBUG [TezTR-518813_1_27_0_0_0] exec.TableScanOperator: Initializing children of 0 TS 2018-01-05T11:29:00,070 DEBUG [TezTR-518813_1_27_0_0_0] exec.SelectOperator: Initializing child 1 SEL 2018-01-05T11:29:00,070 INFO [TezTR-518813_1_27_0_0_0] exec.SelectOperator: Initializing operator SEL[1] 2018-01-05T11:29:00,070 INFO [TezTR-518813_1_27_0_0_0] exec.SelectOperator: SELECT struct 2018-01-05T11:29:00,070 DEBUG [TezTR-518813_1_27_0_0_0] exec.SelectOperator: Initialization Done 1 SEL 2018-01-05T11:29:00,070 DEBUG [TezTR-518813_1_27_0_0_0] exec.SelectOperator: Operator 1 SEL initialized 2018-01-05T11:29:00,070 DEBUG [TezTR-518813_1_27_0_0_0] exec.SelectOperator: Initializing children of 1 SEL 2018-01-05T11:29:00,070 DEBUG [TezTR-518813_1_27_0_0_0] exec.GroupByOperator: Initializing child 2 GBY 2018-01-05T11:29:00,070 INFO [TezTR-518813_1_27_0_0_0] exec.GroupByOperator: Initializing operator GBY[2] 2018-01-05T11:29:00,071 DEBUG [TezTR-518813_1_27_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,071 DEBUG [TezTR-518813_1_27_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,071 DEBUG [TezTR-518813_1_27_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,071 DEBUG [TezTR-518813_1_27_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,071 DEBUG [TezTR-518813_1_27_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,071 INFO [TezTR-518813_1_27_0_0_0] exec.GroupByOperator: Max hash table memory: 93860656 bytes 2018-01-05T11:29:00,071 INFO [TezTR-518813_1_27_0_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:29:00,071 DEBUG [TezTR-518813_1_27_0_0_0] exec.GroupByOperator: Initialization Done 2 GBY 2018-01-05T11:29:00,071 DEBUG [TezTR-518813_1_27_0_0_0] exec.GroupByOperator: Operator 2 GBY initialized 2018-01-05T11:29:00,071 DEBUG [TezTR-518813_1_27_0_0_0] exec.GroupByOperator: Initializing children of 2 GBY 2018-01-05T11:29:00,071 DEBUG [TezTR-518813_1_27_0_0_0] exec.ReduceSinkOperator: Initializing child 3 RS 2018-01-05T11:29:00,071 INFO [TezTR-518813_1_27_0_0_0] exec.ReduceSinkOperator: Initializing operator RS[3] 2018-01-05T11:29:00,071 DEBUG [TezTR-518813_1_27_0_0_0] exec.ReduceSinkOperator: keys size is 0 2018-01-05T11:29:00,071 INFO [TezTR-518813_1_27_0_0_0] exec.ReduceSinkOperator: Using tag = -1 2018-01-05T11:29:00,071 DEBUG [TezTR-518813_1_27_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,071 DEBUG [TezTR-518813_1_27_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,071 DEBUG [TezTR-518813_1_27_0_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1, _col2, _col3, _col4] columnTypes=[struct, struct, struct, struct, struct] 2018-01-05T11:29:00,071 DEBUG [TezTR-518813_1_27_0_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS 2018-01-05T11:29:00,071 DEBUG [TezTR-518813_1_27_0_0_0] exec.ReduceSinkOperator: Operator 3 RS initialized 2018-01-05T11:29:00,071 DEBUG [TezTR-518813_1_27_0_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS done is reset. 2018-01-05T11:29:00,071 DEBUG [TezTR-518813_1_27_0_0_0] exec.GroupByOperator: Initialization Done 2 GBY done is reset. 2018-01-05T11:29:00,071 DEBUG [TezTR-518813_1_27_0_0_0] exec.SelectOperator: Initialization Done 1 SEL done is reset. 2018-01-05T11:29:00,071 DEBUG [TezTR-518813_1_27_0_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:29:00,071 INFO [TezTR-518813_1_27_0_0_0] exec.OperatorUtils: Setting output collector: RS[3] --> Reducer 2 2018-01-05T11:29:00,071 DEBUG [TezTR-518813_1_27_0_0_0] log.PerfLogger: 2018-01-05T11:29:00,071 INFO [TezTR-518813_1_27_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_787_3068125115156391877-1/djaiswal/_tez_scratch_dir/50a73098-a6c8-4456-8bd7-08d927f8d676/map.xml 2018-01-05T11:29:00,071 DEBUG [TezTR-518813_1_27_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:00,071 DEBUG [TezTR-518813_1_27_0_0_0] exec.MapOperator: Processing alias(es) cbo_t3 for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3/cbo_t3.txt 2018-01-05T11:29:00,071 INFO [TezTR-518813_1_27_0_0_0] exec.MapOperator: MAP[0]: records read - 1 2018-01-05T11:29:00,071 INFO [TezTR-518813_1_27_0_0_0] exec.MapOperator: MAP[0]: records read - 10 2018-01-05T11:29:00,071 DEBUG [TezTR-518813_1_27_0_0_0] log.PerfLogger: 2018-01-05T11:29:00,071 DEBUG [TezTR-518813_1_27_0_0_0] mr.ObjectCache: djaiswal_20180105112859_5f019fd9-7a31-4694-b9e6-c1ae3adfa206_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:29:00,071 DEBUG [TezTR-518813_1_27_0_0_0] exec.MapOperator: close called for operator MAP[0] 2018-01-05T11:29:00,071 INFO [TezTR-518813_1_27_0_0_0] exec.MapOperator: Closing operator MAP[0] 2018-01-05T11:29:00,071 INFO [TezTR-518813_1_27_0_0_0] exec.MapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:20, 2018-01-05T11:29:00,071 DEBUG [TezTR-518813_1_27_0_0_0] exec.MapOperator: Closing child = TS[0] 2018-01-05T11:29:00,071 DEBUG [TezTR-518813_1_27_0_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:29:00,071 DEBUG [TezTR-518813_1_27_0_0_0] exec.TableScanOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:00,071 INFO [TezTR-518813_1_27_0_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:29:00,071 DEBUG [TezTR-518813_1_27_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,071 DEBUG [TezTR-518813_1_27_0_0_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_790_2754539752752503788-1/-mr-10001 2018-01-05T11:29:00,072 INFO [TezTR-518813_1_27_0_0_0] exec.TableScanOperator: publishing : default.cbo_t3/ : {numRows=20, rawDataSize=262} 2018-01-05T11:29:00,093 DEBUG [TezTR-518813_1_27_0_0_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_790_2754539752752503788-1/-mr-10001/tmpstats-0_0 2018-01-05T11:29:00,093 DEBUG [TezTR-518813_1_27_0_0_0] fs.FSStatsPublisher: Writing stats in it : {default.cbo_t3/={numRows=20, rawDataSize=262}} 2018-01-05T11:29:00,093 DEBUG [TezTR-518813_1_27_0_0_0] exec.TableScanOperator: Closing child = SEL[1] 2018-01-05T11:29:00,093 DEBUG [TezTR-518813_1_27_0_0_0] exec.SelectOperator: close called for operator SEL[1] 2018-01-05T11:29:00,093 DEBUG [TezTR-518813_1_27_0_0_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:00,093 INFO [TezTR-518813_1_27_0_0_0] exec.SelectOperator: Closing operator SEL[1] 2018-01-05T11:29:00,093 DEBUG [TezTR-518813_1_27_0_0_0] exec.SelectOperator: Closing child = GBY[2] 2018-01-05T11:29:00,093 DEBUG [TezTR-518813_1_27_0_0_0] exec.GroupByOperator: close called for operator GBY[2] 2018-01-05T11:29:00,093 DEBUG [TezTR-518813_1_27_0_0_0] exec.GroupByOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:00,093 INFO [TezTR-518813_1_27_0_0_0] exec.GroupByOperator: Closing operator GBY[2] 2018-01-05T11:29:00,093 INFO [TezTR-518813_1_27_0_0_0] exec.GroupByOperator: Begin Hash Table flush: size = 1 2018-01-05T11:29:00,093 INFO [TezTR-518813_1_27_0_0_0] exec.ReduceSinkOperator: keys are [] num distributions: 0 2018-01-05T11:29:00,093 DEBUG [TezTR-518813_1_27_0_0_0] exec.GroupByOperator: Closing child = RS[3] 2018-01-05T11:29:00,093 DEBUG [TezTR-518813_1_27_0_0_0] exec.ReduceSinkOperator: close called for operator RS[3] 2018-01-05T11:29:00,093 DEBUG [TezTR-518813_1_27_0_0_0] exec.ReduceSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:00,093 INFO [TezTR-518813_1_27_0_0_0] exec.ReduceSinkOperator: Closing operator RS[3] 2018-01-05T11:29:00,093 INFO [TezTR-518813_1_27_0_0_0] exec.ReduceSinkOperator: RECORDS_OUT_INTERMEDIATE_Map_1:1, 2018-01-05T11:29:00,093 DEBUG [TezTR-518813_1_27_0_0_0] exec.ReduceSinkOperator: 3 Close done 2018-01-05T11:29:00,094 DEBUG [TezTR-518813_1_27_0_0_0] exec.GroupByOperator: 2 Close done 2018-01-05T11:29:00,094 DEBUG [TezTR-518813_1_27_0_0_0] exec.SelectOperator: 1 Close done 2018-01-05T11:29:00,094 DEBUG [TezTR-518813_1_27_0_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:29:00,094 DEBUG [TezTR-518813_1_27_0_0_0] exec.MapOperator: 0 Close done 2018-01-05T11:29:00,094 INFO [TezTR-518813_1_27_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_27_00_000000_0 2018-01-05T11:29:00,094 INFO [TezTR-518813_1_27_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: Waiting for all spills to complete : Pending : 0 2018-01-05T11:29:00,094 INFO [TezTR-518813_1_27_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: All spills complete 2018-01-05T11:29:00,106 DEBUG [TezTR-518813_1_27_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:00,106 INFO [TezTR-518813_1_27_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:00,106 INFO [TezTR-518813_1_27_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_27_00_000000_0, askedToStop=false 2018-01-05T11:29:00,106 INFO [TezTR-518813_1_27_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_27_00_000000_0, stopRequested=false 2018-01-05T11:29:00,106 INFO [TezTR-518813_1_27_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_27_00_000000_0: Counters: 14 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=20, INPUT_SPLIT_LENGTH_BYTES=282, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=200, OUTPUT_BYTES_WITH_OVERHEAD=209, OUTPUT_BYTES_PHYSICAL=237, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=20, RECORDS_OUT_INTERMEDIATE_Map_1=1]] 2018-01-05T11:29:00,106 DEBUG [TezTR-518813_1_27_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_27_0_0_0 thread id: 1780 scheme: file bytesRead: 298 bytesWritten: 328 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:00,106 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_27_00_000000_0, isTaskDone=true 2018-01-05T11:29:00,106 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000056, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_27_00_000000_0, eventCount=4 } 2018-01-05T11:29:00,107 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_27][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_27_00_000000_0, creationTime=1515180540018, allocationTime=1515180540019, startTime=1515180540021, finishTime=1515180540107, timeTaken=86, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:00,107 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_27_00_000000_0 2018-01-05T11:29:00,107 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_27_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:00,107 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:29:00,107 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_27_00_000000_0; false 2018-01-05T11:29:00,107 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_27_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:00,107 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_27_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:00,107 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_27_00_000000_0 2018-01-05T11:29:00,107 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_27][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_27_00_000000, startTime=1515180540021, finishTime=1515180540107, timeTaken=86, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_27_00_000000_0, diagnostics=, counters=Counters: 20, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=298, FILE_BYTES_WRITTEN=328, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=20, INPUT_SPLIT_LENGTH_BYTES=282, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=200, OUTPUT_BYTES_WITH_OVERHEAD=209, OUTPUT_BYTES_PHYSICAL=237, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=20, RECORDS_OUT_INTERMEDIATE_Map_1=1 2018-01-05T11:29:00,107 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000056=73 2018-01-05T11:29:00,107 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_27_00_000000_0: true 2018-01-05T11:29:00,107 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:00,107 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:00,107 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_27_00_000000_0 2018-01-05T11:29:00,107 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000056, QueryId=djaiswal_20180105112859_5f019fd9-7a31-4694-b9e6-c1ae3adfa206-27, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:00,107 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_27_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:29:00,107 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_27_00_000000_0 2018-01-05T11:29:00,107 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_27_00 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:00,107 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_27_00 [Map 1] 2018-01-05T11:29:00,108 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_27][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_27_00, initRequestedTime=1515180539991, initedTime=1515180540017, startRequestedTime=1515180539999, startedTime=1515180540018, finishTime=1515180540107, timeTaken=89, status=SUCCEEDED, diagnostics=, counters=Counters: 20, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=298, FILE_BYTES_WRITTEN=328, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=20, INPUT_SPLIT_LENGTH_BYTES=282, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=200, OUTPUT_BYTES_WITH_OVERHEAD=209, OUTPUT_BYTES_PHYSICAL=237, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=20, RECORDS_OUT_INTERMEDIATE_Map_1=1, vertexStats=firstTaskStartTime=1515180540021, firstTasksToStart=[ task_1515180518813_0001_27_00_000000 ], lastTaskFinishTime=1515180540107, lastTasksToFinish=[ task_1515180518813_0001_27_00_000000 ], minTaskDuration=86, maxTaskDuration=86, avgTaskDuration=86.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_27_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_27_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:00,108 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_27_00 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:00,108 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_27][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000056, stoppedTime=1515180540108, exitStatus=0 2018-01-05T11:29:00,108 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000056 exited with diagnostics set to Container completed. 2018-01-05T11:29:00,108 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_27_01 [Reducer 2] attempt: attempt_1515180518813_0001_27_00_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:29:00,108 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_27_00 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2 2018-01-05T11:29:00,108 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null 2018-01-05T11:29:00,108 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000056 2018-01-05T11:29:00,108 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: All source tasks completed. Ramping up 1 remaining tasks for vertex: Reducer 2 2018-01-05T11:29:00,108 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Scheduling 1 tasks for vertex: Reducer 2 with totalTasks: 1. 1 source tasks completed out of 1. MinSourceTaskCompletedFraction: 1.0 min: 0.05 max: 0.75 2018-01-05T11:29:00,108 INFO [App Shared Pool - #0] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Map 1, destinationVertexName=Reducer 2. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.UnorderedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:29:00,108 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_27_01 [Reducer 2] 2018-01-05T11:29:00,108 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_27][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_27_01_000000, scheduledTime=1515180540108, launchTime=1515180540108 2018-01-05T11:29:00,108 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_27_01 [Reducer 2] 2018-01-05T11:29:00,108 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_27_01_000000_0, priority=14, capability=, hosts=null 2018-01-05T11:29:00,108 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:00,108 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:00,108 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:00,108 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_27_01_000000_0 on hosts=null 2018-01-05T11:29:00,108 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_27_01_000000_0 on hosts=null 2018-01-05T11:29:00,108 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_27_01_000000_0 2018-01-05T11:29:00,108 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:00,109 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #57, task=TaskInfo{task=attempt_1515180518813_0001_27_01_000000_0, priority=14, startTime=0, containerId=null, uniqueId=56, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000057 2018-01-05T11:29:00,109 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_27_01_000000_0; false 2018-01-05T11:29:00,109 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_27_01_000000_0, priority=14, startTime=311560618, containerId=container_111101111_0001_00_000057assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=56, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:00,109 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000057 for node: 10.22.8.182:49818 2018-01-05T11:29:00,109 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000057, attempt_1515180518813_0001_27_01_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:00,109 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000057 for node: 10.22.8.182:49818 2018-01-05T11:29:00,109 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000057 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:00,109 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000057, launchTime=1515180540109 2018-01-05T11:29:00,109 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=27}, sourceName=Map 1, state=S_SUCCEEDED 2018-01-05T11:29:00,110 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_27_01_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000057] on NM: [10.22.8.182:49818] 2018-01-05T11:29:00,110 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:00,110 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:00,110 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_27][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_27_01_000000_0, startTime=1515180540109, containerId=container_111101111_0001_00_000057, nodeId=10.22.8.182:49818 2018-01-05T11:29:00,110 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_27_01_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112859_5f019fd9-7a31-4694-b9e6-c1ae3adfa206, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000057, dagName=analyze table cbo_t3 compute sta...c_boolean (Stage-0), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.UnorderedKVInput,1}}, Outputs={{out_Reducer 2,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=2, completedTaskCount=1, dagStartTime=1515180539991, firstAttemptStartTime=1515180540108, currentAttemptStartTime=1515180540109} 2018-01-05T11:29:00,110 INFO [IPC Server handler 2 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000057, QueryId=djaiswal_20180105112859_5f019fd9-7a31-4694-b9e6-c1ae3adfa206-27, DagId=27, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, SubmitTime=1515180540110 2018-01-05T11:29:00,110 DEBUG [IPC Server handler 2 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=27} with the ShuffleHandler 2018-01-05T11:29:00,110 DEBUG [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/27] 2018-01-05T11:29:00,111 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:29:00,111 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_27_01_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:00,111 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_27_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180540108, dagStartTime=1515180539991, withinDagPriority=14, vertexParallelism= 1, selfAndUpstreamParallelism= 2, selfAndUpstreamComplete= 1}] 2018-01-05T11:29:00,111 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_27_01_000000_0 : ACCEPTED 2018-01-05T11:29:00,111 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_27_01_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:29:00,111 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_27_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180540108, dagStartTime=1515180539991, withinDagPriority=14, vertexParallelism= 1, selfAndUpstreamParallelism= 2, selfAndUpstreamComplete= 1} 2018-01-05T11:29:00,111 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_27_01_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:00,111 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_27_01_000000_0 to pre-emption queue 2018-01-05T11:29:00,111 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_27_01_000000_0: true 2018-01-05T11:29:00,111 INFO [TaskCommunicator # 9] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_27_01_000000_0 2018-01-05T11:29:00,111 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:1400285116 2018-01-05T11:29:00,111 INFO [TaskCommunicator # 9] GuaranteedTasks: Received response for attempt_1515180518813_0001_27_01_000000_0, true 2018-01-05T11:29:00,111 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : analyze table cbo_t3 compute sta...c_boolean (Stage-0), VertexName: Reducer 2, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_27_01_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:29:00,111 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1037041664, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:00,111 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_27_01_000000_0 vertexName: Reducer 2 2018-01-05T11:29:00,111 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000057, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_27_01_000000_0, eventCount=1 } 2018-01-05T11:29:00,111 DEBUG [TezTR-518813_1_27_1_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_27 queryId: djaiswal_20180105112859_5f019fd9-7a31-4694-b9e6-c1ae3adfa206 instanceType: TaskRunner2Callable 2018-01-05T11:29:00,111 INFO [TezTR-518813_1_27_1_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_27_01_000000_0 2018-01-05T11:29:00,112 DEBUG [TezTR-518813_1_27_1_0_0] log.PerfLogger: 2018-01-05T11:29:00,112 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:00,113 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:00,114 INFO [IPC Server handler 15 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_27_01_000000_0 1 events [0,1) total 1 vertex_1515180518813_0001_27_01 [Reducer 2] 2018-01-05T11:29:00,114 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:29:00,114 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_27_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:29:00,114 DEBUG [TezTR-518813_1_27_1_0_0] log.PerfLogger: 2018-01-05T11:29:00,114 INFO [TezTR-518813_1_27_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:00,114 INFO [TezTR-518813_1_27_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:00,114 INFO [TezTR-518813_1_27_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:00,115 INFO [I/O Setup 1 Initialize: {out_Reducer 2}] output.MROutput: out_Reducer 2: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:29:00,115 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:00,115 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:00,115 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:00,115 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:00,115 INFO [TezTR-518813_1_27_1_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:00,115 INFO [TezTR-518813_1_27_1_0_0] resources.MemoryDistributor: InitialRequests=[out_Reducer 2:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.UnorderedKVInput] 2018-01-05T11:29:00,115 INFO [TezTR-518813_1_27_1_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:00,115 INFO [TezTR-518813_1_27_1_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:00,115 INFO [TezTR-518813_1_27_1_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 131404920, TotalRequestedScaled: 6.570246E7, TotalJVMHeap: 1037041664, TotalAvailable: 694817914, TotalRequested/TotalJVMHeap:0.13 2018-01-05T11:29:00,115 INFO [TezTR-518813_1_27_1_0_0] resources.MemoryDistributor: Allocations=[out_Reducer 2:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [Map 1:org.apache.tez.runtime.library.input.UnorderedKVInput:INPUT:131404920:131404920] 2018-01-05T11:29:00,115 INFO [TezTR-518813_1_27_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:00,115 INFO [TezTR-518813_1_27_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 0 2018-01-05T11:29:00,115 INFO [TezTR-518813_1_27_1_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:00,115 INFO [TezTR-518813_1_27_1_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_27_01_000000_0 2018-01-05T11:29:00,115 DEBUG [TezTR-518813_1_27_1_0_0] log.PerfLogger: 2018-01-05T11:29:00,115 DEBUG [TezTR-518813_1_27_1_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_27_01_000000_0_10169 2018-01-05T11:29:00,115 DEBUG [TezTR-518813_1_27_1_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:00,115 DEBUG [TezTR-518813_1_27_1_0_0] mr.ObjectCache: Creating djaiswal_20180105112859_5f019fd9-7a31-4694-b9e6-c1ae3adfa206_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:29:00,115 INFO [TezTR-518813_1_27_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_787_3068125115156391877-1/djaiswal/_tez_scratch_dir/d859accd-f514-46dd-995c-51f4ea09351e/reduce.xml 2018-01-05T11:29:00,115 DEBUG [TezTR-518813_1_27_1_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_787_3068125115156391877-1/djaiswal/_tez_scratch_dir/d859accd-f514-46dd-995c-51f4ea09351e/reduce.xml 2018-01-05T11:29:00,115 DEBUG [TezTR-518813_1_27_1_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_787_3068125115156391877-1/djaiswal/_tez_scratch_dir/d859accd-f514-46dd-995c-51f4ea09351e/reduce.xml 2018-01-05T11:29:00,115 DEBUG [TezTR-518813_1_27_1_0_0] log.PerfLogger: 2018-01-05T11:29:00,116 INFO [TezTR-518813_1_27_1_0_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:29:00,116 DEBUG [TezTR-518813_1_27_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,116 DEBUG [TezTR-518813_1_27_1_0_0] log.PerfLogger: 2018-01-05T11:29:00,116 INFO [TezTR-518813_1_27_1_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 2.70KB 2018-01-05T11:29:00,116 DEBUG [TezTR-518813_1_27_1_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:00,116 DEBUG [TezTR-518813_1_27_1_0_0] log.PerfLogger: 2018-01-05T11:29:00,116 DEBUG [TezTR-518813_1_27_1_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:00,116 DEBUG [TezTR-518813_1_27_1_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:00,117 DEBUG [TezTR-518813_1_27_1_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:00,117 INFO [TezTR-518813_1_27_1_0_0] impl.SimpleFetchedInputAllocator: Map_1: RequestedMemory=131404920, AssignedMemory=131404920, maxSingleShuffleLimit=32851230 2018-01-05T11:29:00,117 INFO [TezTR-518813_1_27_1_0_0] impl.ShuffleManager: Map_1: numInputs=1, compressionCodec=NoCompressionCodec, numFetchers=1, ifileBufferSize=4096, ifileReadAheadEnabled=true, ifileReadAheadLength=4194304, localDiskFetchEnabled=false, sharedFetchEnabled=false, keepAlive=false, keepAliveMaxConnections=20, connectionTimeout=20000, readTimeout=180000, bufferSize=8192, bufferSize=8192, maxTaskOutputAtOnce=20 2018-01-05T11:29:00,117 INFO [TezTR-518813_1_27_1_0_0] impl.ShuffleInputEventHandlerImpl: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:29:00,117 INFO [TezTR-518813_1_27_1_0_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:29:00,118 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=27&reduce=0&map=attempt_1515180518813_0001_27_00_000000_0_10168 mapId: [attempt_1515180518813_0001_27_00_000000_0_10168] reduceId: [0] jobId: [job_1515180518813_0001] dagId: [27] keepAlive: false 2018-01-05T11:29:00,118 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=27&reduce=0&map=attempt_1515180518813_0001_27_00_000000_0_10168; hash=...5Z2QVPaohnUxQ 2018-01-05T11:29:00,118 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=27&reduce=0&map=attempt_1515180518813_0001_27_00_000000_0_10168;reply=oboWXcdb3PL0Y 2018-01-05T11:29:00,118 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=27, user='djaiswal', attemptId='attempt_1515180518813_0001_27_00_000000_0_10168'} via loader 2018-01-05T11:29:00,118 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=27, user='djaiswal', attemptId='attempt_1515180518813_0001_27_00_000000_0_10168'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:29:00,118 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_27_00_000000_0_10168 not found 2018-01-05T11:29:00,119 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_27_00_000000_0_10168,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/27/output/attempt_1515180518813_0001_27_00_000000_0_10168/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/27/output/attempt_1515180518813_0001_27_00_000000_0_10168/file.out.index 2018-01-05T11:29:00,119 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Content Length in shuffle : 266 2018-01-05T11:29:00,119 INFO [Fetcher_B {Map_1} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=27&reduce=0&map=attempt_1515180518813_0001_27_00_000000_0_10168 sent hash and receievd reply 0 ms 2018-01-05T11:29:00,119 INFO [Fetcher_B {Map_1} #0] ShuffleManager.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_27_00_000000_0_10168} to MEMORY, csize=213, dsize=209, EndTime=1515180540119, TimeTaken=0, Rate=0.00 MB/s 2018-01-05T11:29:00,119 INFO [Fetcher_B {Map_1} #0] impl.ShuffleManager: All inputs fetched for input vertex : Map 1 2018-01-05T11:29:00,119 INFO [TezTR-518813_1_27_1_0_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:29:00,119 INFO [Fetcher_B {Map_1} #0] impl.ShuffleManager: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:00,119 INFO [TezTR-518813_1_27_1_0_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:00,119 INFO [ShuffleRunner {Map_1}] impl.ShuffleManager: Map_1: Shutting down FetchScheduler, Was Interrupted: false 2018-01-05T11:29:00,119 DEBUG [TezTR-518813_1_27_1_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:00,119 INFO [ShuffleRunner {Map_1}] impl.ShuffleManager: Map_1: Scheduler thread completed 2018-01-05T11:29:00,119 DEBUG [TezTR-518813_1_27_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,119 DEBUG [TezTR-518813_1_27_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,119 DEBUG [TezTR-518813_1_27_1_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1, _col2, _col3, _col4] columnTypes=[struct, struct, struct, struct, struct] 2018-01-05T11:29:00,119 DEBUG [TezTR-518813_1_27_1_0_0] log.PerfLogger: 2018-01-05T11:29:00,119 INFO [TezTR-518813_1_27_1_0_0] exec.GroupByOperator: Initializing operator GBY[4] 2018-01-05T11:29:00,119 DEBUG [TezTR-518813_1_27_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,119 DEBUG [TezTR-518813_1_27_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,119 DEBUG [TezTR-518813_1_27_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,119 DEBUG [TezTR-518813_1_27_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,119 DEBUG [TezTR-518813_1_27_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,119 INFO [TezTR-518813_1_27_1_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:29:00,119 DEBUG [TezTR-518813_1_27_1_0_0] exec.GroupByOperator: Initialization Done 4 GBY 2018-01-05T11:29:00,119 DEBUG [TezTR-518813_1_27_1_0_0] exec.GroupByOperator: Operator 4 GBY initialized 2018-01-05T11:29:00,119 DEBUG [TezTR-518813_1_27_1_0_0] exec.GroupByOperator: Initializing children of 4 GBY 2018-01-05T11:29:00,119 DEBUG [TezTR-518813_1_27_1_0_0] exec.FileSinkOperator: Initializing child 6 FS 2018-01-05T11:29:00,119 INFO [TezTR-518813_1_27_1_0_0] exec.FileSinkOperator: Initializing operator FS[6] 2018-01-05T11:29:00,119 DEBUG [TezTR-518813_1_27_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,120 INFO [TezTR-518813_1_27_1_0_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@f416480]:[_col0, _col1, _col2, _col3, _col4]:[struct, struct, struct, struct, struct]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@24c6046d 2018-01-05T11:29:00,120 DEBUG [TezTR-518813_1_27_1_0_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:29:00,120 DEBUG [TezTR-518813_1_27_1_0_0] exec.FileSinkOperator: Initialization Done 6 FS 2018-01-05T11:29:00,120 DEBUG [TezTR-518813_1_27_1_0_0] exec.FileSinkOperator: Operator 6 FS initialized 2018-01-05T11:29:00,120 DEBUG [TezTR-518813_1_27_1_0_0] exec.FileSinkOperator: Initialization Done 6 FS done is reset. 2018-01-05T11:29:00,120 DEBUG [TezTR-518813_1_27_1_0_0] exec.GroupByOperator: Initialization Done 4 GBY done is reset. 2018-01-05T11:29:00,120 INFO [TezTR-518813_1_27_1_0_0] tez.ReduceRecordProcessor: Id =4 Id =6 <\Children> Id = 4 null<\Parent> <\FS> <\Children> <\Parent> <\GBY> 2018-01-05T11:29:00,120 DEBUG [TezTR-518813_1_27_1_0_0] log.PerfLogger: 2018-01-05T11:29:00,120 INFO [TezTR-518813_1_27_1_0_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 2 2018-01-05T11:29:00,120 DEBUG [TezTR-518813_1_27_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:00,120 DEBUG [TezTR-518813_1_27_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:00,120 DEBUG [TezTR-518813_1_27_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:00,120 DEBUG [TezTR-518813_1_27_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:00,120 DEBUG [TezTR-518813_1_27_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:00,120 DEBUG [TezTR-518813_1_27_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:00,120 INFO [TezTR-518813_1_27_1_0_0] readers.UnorderedKVReader: Num Records read: 1 2018-01-05T11:29:00,120 DEBUG [TezTR-518813_1_27_1_0_0] log.PerfLogger: 2018-01-05T11:29:00,120 DEBUG [TezTR-518813_1_27_1_0_0] mr.ObjectCache: djaiswal_20180105112859_5f019fd9-7a31-4694-b9e6-c1ae3adfa206_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:29:00,120 DEBUG [TezTR-518813_1_27_1_0_0] exec.GroupByOperator: close called for operator GBY[4] 2018-01-05T11:29:00,120 INFO [TezTR-518813_1_27_1_0_0] exec.GroupByOperator: Closing operator GBY[4] 2018-01-05T11:29:00,120 INFO [TezTR-518813_1_27_1_0_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_790_2754539752752503788-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-59_790_2754539752752503788-1/_tmp.-ext-10002/000000_0 2018-01-05T11:29:00,120 INFO [TezTR-518813_1_27_1_0_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_790_2754539752752503788-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-59_790_2754539752752503788-1/_task_tmp.-ext-10002/_tmp.000000_0 2018-01-05T11:29:00,120 INFO [TezTR-518813_1_27_1_0_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_790_2754539752752503788-1/-mr-10000/.hive-staging_hive_2018-01-05_11-28-59_790_2754539752752503788-1/_tmp.-ext-10002/000000_0 2018-01-05T11:29:00,156 INFO [TezTR-518813_1_27_1_0_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:29:00,156 DEBUG [TezTR-518813_1_27_1_0_0] exec.GroupByOperator: Closing child = FS[6] 2018-01-05T11:29:00,156 DEBUG [TezTR-518813_1_27_1_0_0] exec.FileSinkOperator: close called for operator FS[6] 2018-01-05T11:29:00,156 DEBUG [TezTR-518813_1_27_1_0_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:00,156 INFO [TezTR-518813_1_27_1_0_0] exec.FileSinkOperator: Closing operator FS[6] 2018-01-05T11:29:00,156 INFO [TezTR-518813_1_27_1_0_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:29:00,157 INFO [TezTR-518813_1_27_1_0_0] exec.FileSinkOperator: RECORDS_OUT_0:1, 2018-01-05T11:29:00,157 DEBUG [TezTR-518813_1_27_1_0_0] exec.FileSinkOperator: 6 Close done 2018-01-05T11:29:00,157 DEBUG [TezTR-518813_1_27_1_0_0] exec.GroupByOperator: 4 Close done 2018-01-05T11:29:00,157 INFO [TezTR-518813_1_27_1_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_27_01_000000_0 2018-01-05T11:29:00,157 INFO [TezTR-518813_1_27_1_0_0] impl.ShuffleInputEventHandlerImpl: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:29:00,157 INFO [TezTR-518813_1_27_1_0_0] impl.ShuffleManager: Shutting down pending fetchers on sourceMap_1: 0 2018-01-05T11:29:00,157 INFO [TezTR-518813_1_27_1_0_0] output.MROutput: out_Reducer 2 closed 2018-01-05T11:29:00,157 DEBUG [TezTR-518813_1_27_1_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:00,157 INFO [TezTR-518813_1_27_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:00,157 INFO [TezTR-518813_1_27_1_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_27_01_000000_0, askedToStop=false 2018-01-05T11:29:00,157 INFO [TezTR-518813_1_27_1_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_27_01_000000_0, stopRequested=false 2018-01-05T11:29:00,157 INFO [TezTR-518813_1_27_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_27_01_000000_0: Counters: 14 [[org.apache.tez.common.counters.TaskCounter NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=213, SHUFFLE_BYTES_DECOMPRESSED=209, SHUFFLE_BYTES_TO_MEM=213, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=2, FIRST_EVENT_RECEIVED=0, LAST_EVENT_RECEIVED=0][HIVE CREATED_FILES=1, RECORDS_OUT_0=1]] 2018-01-05T11:29:00,157 DEBUG [TezTR-518813_1_27_1_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_27_1_0_0 thread id: 1792 scheme: file bytesRead: 0 bytesWritten: 341 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:00,157 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_27_01_000000_0, isTaskDone=true 2018-01-05T11:29:00,157 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000057, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_27_01_000000_0, eventCount=2 } 2018-01-05T11:29:00,158 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_27][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_27_01_000000_0, creationTime=1515180540108, allocationTime=1515180540109, startTime=1515180540109, finishTime=1515180540158, timeTaken=49, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:00,158 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:00,158 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_27_01_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:00,158 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_27_01_000000_0 2018-01-05T11:29:00,158 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_27_01_000000_0; false 2018-01-05T11:29:00,158 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_27_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:00,158 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_27_01_000000_0 2018-01-05T11:29:00,158 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_27_01_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:00,158 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:00,158 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:00,158 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_27][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_27_01_000000, startTime=1515180540109, finishTime=1515180540158, timeTaken=49, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_27_01_000000_0, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=341, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=213, SHUFFLE_BYTES_DECOMPRESSED=209, SHUFFLE_BYTES_TO_MEM=213, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=2, FIRST_EVENT_RECEIVED=0, LAST_EVENT_RECEIVED=0, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1 2018-01-05T11:29:00,159 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000057=48 2018-01-05T11:29:00,159 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_27_01_000000_0: true 2018-01-05T11:29:00,159 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_27_01 [Reducer 2], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:00,159 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_27_01 [Reducer 2] 2018-01-05T11:29:00,159 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_27][Event:VERTEX_FINISHED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_27_01, initRequestedTime=1515180539999, initedTime=1515180540004, startRequestedTime=1515180540018, startedTime=1515180540018, finishTime=1515180540159, timeTaken=141, status=SUCCEEDED, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=341, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=213, SHUFFLE_BYTES_DECOMPRESSED=209, SHUFFLE_BYTES_TO_MEM=213, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=2, FIRST_EVENT_RECEIVED=0, LAST_EVENT_RECEIVED=0, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1, vertexStats=firstTaskStartTime=1515180540109, firstTasksToStart=[ task_1515180518813_0001_27_01_000000 ], lastTaskFinishTime=1515180540158, lastTasksToFinish=[ task_1515180518813_0001_27_01_000000 ], minTaskDuration=49, maxTaskDuration=49, avgTaskDuration=49.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_27_01_000000 ], longestDurationTasks=[ task_1515180518813_0001_27_01_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:00,160 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_27_01 [Reducer 2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:00,160 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_27][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000057, stoppedTime=1515180540160, exitStatus=0 2018-01-05T11:29:00,160 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000057 exited with diagnostics set to Container completed. 2018-01-05T11:29:00,160 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_27_01 [Reducer 2] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2 2018-01-05T11:29:00,160 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null 2018-01-05T11:29:00,160 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_27_00 [Map 1] 2018-01-05T11:29:00,160 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_27_01 [Reducer 2] 2018-01-05T11:29:00,161 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_27, queueSize=0 2018-01-05T11:29:00,161 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_27][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_27, startTime=1515180539991, finishTime=1515180540160, timeTaken=169, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, OTHER_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=149, AM_GC_TIME_MILLIS=0, File System Counters, FILE_BYTES_READ=298, FILE_BYTES_WRITTEN=669, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=21, INPUT_SPLIT_LENGTH_BYTES=282, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=200, OUTPUT_BYTES_WITH_OVERHEAD=209, OUTPUT_BYTES_PHYSICAL=237, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=213, SHUFFLE_BYTES_DECOMPRESSED=209, SHUFFLE_BYTES_TO_MEM=213, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=2, FIRST_EVENT_RECEIVED=0, LAST_EVENT_RECEIVED=0, HIVE, CREATED_FILES=1, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=20, RECORDS_OUT_0=1, RECORDS_OUT_INTERMEDIATE_Map_1=1 2018-01-05T11:29:00,161 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_27 finished with state: SUCCEEDED 2018-01-05T11:29:00,161 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_27 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:29:00,161 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000057 2018-01-05T11:29:00,161 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_27, dagState=SUCCEEDED 2018-01-05T11:29:00,161 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_27_post 2018-01-05T11:29:00,161 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:29:00,161 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:29:00,161 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:29:00,161 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 27 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=2, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=2,NumRequestsWithoutLocation=2, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=2} 2018-01-05T11:29:00,161 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:29:00,161 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=analyze table cbo_t3 compute sta...c_boolean (Stage-0), with id=dag_1515180518813_0001_27 2018-01-05T11:29:00,161 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 27, to 10.22.8.182:49818 2018-01-05T11:29:00,161 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=2, Remaining=0 2018-01-05T11:29:00,161 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=analyze table cbo_t3 compute sta...c_boolean (Stage-0), with id=dag_1515180518813_0001_27 2018-01-05T11:29:00,162 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=27} 2018-01-05T11:29:00,162 INFO [IPC Server handler 4 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=27}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:29:00,162 INFO [IPC Server handler 4 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/27 after 300 seconds 2018-01-05T11:29:00,162 ERROR [IPC Server handler 4 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112859_5f019fd9-7a31-4694-b9e6-c1ae3adfa206, dag_1515180518813_0001_27 2018-01-05T11:29:00,162 INFO [IPC Server handler 4 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112859_5f019fd9-7a31-4694-b9e6-c1ae3adfa206 2018-01-05T11:29:00,162 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=27} = 1 2018-01-05T11:29:00,162 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Issuing killFragment for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=27} attempt_1515180518813_0001_27_01_000000_0 2018-01-05T11:29:00,162 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Removing attempt_1515180518813_0001_27_01_000000_0 from preemptionQueue 2018-01-05T11:29:00,162 INFO [IPC Server handler 4 on 49818] impl.TaskRunnerCallable: Ignoring kill request for task attempt_1515180518813_0001_27_01_000000_0 since it's already complete 2018-01-05T11:29:00,162 DEBUG [IPC Server handler 4 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=27} 2018-01-05T11:29:00,162 DEBUG [IPC Server handler 4 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:29:00,162 DEBUG [IPC Server handler 4 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=27} 2018-01-05T11:29:00,162 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,162 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,162 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,162 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,162 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 Reducer 2: 1/1 2018-01-05T11:29:00,162 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,164 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,164 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: TaskId for 000000_0 = 000000 2018-01-05T11:29:00,164 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,164 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,164 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,165 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-2:STATS] in serial mode 2018-01-05T11:29:00,165 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@34d2b43d, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:29:00,165 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:00,165 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,165 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:29:00,165 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:29:00,166 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:00,166 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:00,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:00,166 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:29:00,166 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_790_2754539752752503788-1/-mr-10001 2018-01-05T11:29:00,166 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_790_2754539752752503788-1/-mr-10001 2018-01-05T11:29:00,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.cbo_t3/ numRows 2018-01-05T11:29:00,167 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.cbo_t3/ numRows 20 2018-01-05T11:29:00,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.cbo_t3/ rawDataSize 2018-01-05T11:29:00,167 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.cbo_t3/ rawDataSize 262 2018-01-05T11:29:00,167 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,168 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:29:00,168 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:29:00,168 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:29:00,168 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:29:00,168 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:00,168 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:00,168 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:00,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:00,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:00,169 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:29:00,169 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:29:00,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,169 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:29:00,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,169 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=cbo_t3 newtbl=cbo_t3 2018-01-05T11:29:00,169 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=cbo_t3 newtbl=cbo_t3 2018-01-05T11:29:00,172 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:00,177 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:00,177 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,177 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.cbo_t3 stats: [numFiles=1, numRows=20, totalSize=282, rawDataSize=262] 2018-01-05T11:29:00,177 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: About to delete stats tmp dir :file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_790_2754539752752503788-1/-mr-10001 2018-01-05T11:29:00,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Time taken to get FileStatuses: 15 2018-01-05T11:29:00,193 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:29:00,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 15 2018-01-05T11:29:00,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FetchOperator: Creating fetchTask with deserializer typeinfo: struct<_col0:struct,_col1:struct,_col2:struct,_col3:struct,_col4:struct> 2018-01-05T11:29:00,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FetchOperator: deserializer properties: table properties: {columns=_col0,_col1,_col2,_col3,_col4, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct:struct:struct:struct:struct, escape.delim=\} partition properties: {columns=_col0,_col1,_col2,_col3,_col4, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct:struct:struct:struct:struct, escape.delim=\} 2018-01-05T11:29:00,194 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,194 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: write_column_statistics: db=default table=cbo_t3 2018-01-05T11:29:00,194 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=write_column_statistics: db=default table=cbo_t3 2018-01-05T11:29:00,196 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=cbo_t3 tableName=default colName=key 2018-01-05T11:29:00,197 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:00,197 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=cbo_t3 tableName=default colName=value 2018-01-05T11:29:00,198 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=cbo_t3 tableName=default colName=c_int 2018-01-05T11:29:00,198 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=cbo_t3 tableName=default colName=c_float 2018-01-05T11:29:00,198 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=cbo_t3 tableName=default colName=c_boolean 2018-01-05T11:29:00,201 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:00,201 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,201 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,201 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,201 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: analyze table cbo_t3 compute statistics for columns key, value, c_int, c_float, c_boolean 2018-01-05T11:29:00,201 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:29:00,201 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@cbo_t3 2018-01-05T11:29:00,201 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@cbo_t3 2018-01-05T11:29:00,201 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_790_2754539752752503788-1/-mr-10000 2018-01-05T11:29:00,201 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,201 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,202 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:00,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=8, setPartitionColumnStatistics_(SetPartitionsStatsRequest, )=7} 2018-01-05T11:29:00,202 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112859_5f019fd9-7a31-4694-b9e6-c1ae3adfa206); Time taken: 0.347 seconds 2018-01-05T11:29:00,202 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:29:00,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,202 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:00,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query analyze table cbo_t3 compute statistics for columns key, value, c_int, c_float, c_boolean 2018-01-05T11:29:00,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-28-59_787_3068125115156391877-1 2018-01-05T11:29:00,202 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.415 seconds 2018-01-05T11:29:00,202 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:29:00,202 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:00,202 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:29:00,202 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:29:00,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:00,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: analyze table src_cbo compute statistics 2018-01-05T11:29:00,203 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112900_f7f57959-0c4f-4f84-8860-baee4a2aec40): analyze table src_cbo compute statistics 2018-01-05T11:29:00,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: analyze table src_cbo compute statistics 2018-01-05T11:29:00,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:00,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,203 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:00,203 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Invoking analyze on original query 2018-01-05T11:29:00,203 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Starting Semantic Analysis 2018-01-05T11:29:00,203 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed phase 1 of Semantic Analysis 2018-01-05T11:29:00,203 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for source tables 2018-01-05T11:29:00,203 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,203 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_cbo 2018-01-05T11:29:00,203 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_cbo 2018-01-05T11:29:00,206 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:00,207 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,207 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,207 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_cbo 2018-01-05T11:29:00,207 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_cbo 2018-01-05T11:29:00,212 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:00,212 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000056, QueryId=djaiswal_20180105112859_5f019fd9-7a31-4694-b9e6-c1ae3adfa206-27, DagId=27, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180540034, EndTime=1515180540212 2018-01-05T11:29:00,212 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_27_01_000000_0 2018-01-05T11:29:00,212 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_27_01_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:29:00,212 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_27_01_000000_0 2018-01-05T11:29:00,212 INFO [ExecutionCompletionThread #0] impl.QueryTracker: Ignoring fragmentComplete message for unknown query: QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=27} 2018-01-05T11:29:00,212 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,212 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for subqueries 2018-01-05T11:29:00,212 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for destination tables 2018-01-05T11:29:00,212 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed getting MetaData in Semantic Analysis 2018-01-05T11:29:00,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_203_2889790531036783127-1/-mr-10000 2018-01-05T11:29:00,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Table Plan for src_cbo TS[0] 2018-01-05T11:29:00,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Body Plan for Query Block null 2018-01-05T11:29:00,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Plan for Query Block null 2018-01-05T11:29:00,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Before logical optimization TS[0] 2018-01-05T11:29:00,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] lineage.Generator: Time taken for lineage transform=0 2018-01-05T11:29:00,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{src_cbo}key,value: string|{src_cbo}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src_cbo}block__offset__inside__file,INPUT__FILE__NAME: string|{src_cbo}input__file__name,ROW__ID: struct|{src_cbo}row__id,RAW__DATA__SIZE: bigint|{src_cbo}raw__data__size) 2018-01-05T11:29:00,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,224 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.PredicatePushDown: After PPD: TS[0] 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{src_cbo}key,value: string|{src_cbo}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src_cbo}block__offset__inside__file,INPUT__FILE__NAME: string|{src_cbo}input__file__name,ROW__ID: struct|{src_cbo}row__id,RAW__DATA__SIZE: bigint|{src_cbo}raw__data__size) 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{src_cbo}key,value: string|{src_cbo}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src_cbo}block__offset__inside__file,INPUT__FILE__NAME: string|{src_cbo}input__file__name,ROW__ID: struct|{src_cbo}row__id,RAW__DATA__SIZE: bigint|{src_cbo}raw__data__size) 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: After logical optimization TS[0] 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,225 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,225 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=src_cbo 2018-01-05T11:29:00,225 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=src_cbo 2018-01-05T11:29:00,228 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:00,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:00,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting stats (Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: COMPLETE) on TS[0] 2018-01-05T11:29:00,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (src_cbo): numRows: 500 dataSize: 5312 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:00,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on TS[0] 2018-01-05T11:29:00,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:29:00,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:29:00,230 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Cycle free: true 2018-01-05T11:29:00,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{src_cbo}key,value: string|{src_cbo}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src_cbo}block__offset__inside__file,INPUT__FILE__NAME: string|{src_cbo}input__file__name,ROW__ID: struct|{src_cbo}row__id,RAW__DATA__SIZE: bigint|{src_cbo}raw__data__size) 2018-01-05T11:29:00,230 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,231 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:29:00,231 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo of table src_cbo 2018-01-05T11:29:00,231 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo 2018-01-05T11:29:00,231 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:29:00,231 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,231 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,231 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:29:00,231 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:00,231 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:00,231 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:29:00,231 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:00,231 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:00,231 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:29:00,231 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping vectorization 2018-01-05T11:29:00,231 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:29:00,231 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: llap mode: only 2018-01-05T11:29:00,231 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:29:00,231 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:00,231 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@30e24816 can be done in LLAP 2018-01-05T11:29:00,231 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,231 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed plan generation 2018-01-05T11:29:00,231 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:00,231 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validation start 2018-01-05T11:29:00,231 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Not a partition. 2018-01-05T11:29:00,231 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,231 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:src_cbo.key, type:string, comment:null), FieldSchema(name:src_cbo.value, type:string, comment:null)], properties:null) 2018-01-05T11:29:00,231 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,231 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:00,231 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=9, flushCache_()=0, getTableColumnStatistics_(String, String, List, )=5} 2018-01-05T11:29:00,231 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112900_f7f57959-0c4f-4f84-8860-baee4a2aec40); Time taken: 0.028 seconds 2018-01-05T11:29:00,231 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:00,231 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,231 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112900_f7f57959-0c4f-4f84-8860-baee4a2aec40): analyze table src_cbo compute statistics 2018-01-05T11:29:00,233 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,233 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: analyze table src_cbo compute statistics 2018-01-05T11:29:00,233 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:29:00,233 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@src_cbo 2018-01-05T11:29:00,233 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@src_cbo 2018-01-05T11:29:00,233 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,233 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,233 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,233 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Query ID = djaiswal_20180105112900_f7f57959-0c4f-4f84-8860-baee4a2aec40 2018-01-05T11:29:00,233 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Total jobs = 1 2018-01-05T11:29:00,233 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,233 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:29:00,233 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MAPRED] in serial mode 2018-01-05T11:29:00,244 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_203_2889790531036783127-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:29:00,265 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:29:00,265 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:29:00,265 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:29:00,266 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112900_f7f57959-0c4f-4f84-8860-baee4a2aec40 2018-01-05T11:29:00,266 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Session is already open 2018-01-05T11:29:00,266 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:29:00,266 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,266 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Dag name: analyze table src_cbo compute statistics (Stage-0) 2018-01-05T11:29:00,266 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: DagInfo: {"context":"Hive","description":"\nanalyze table src_cbo compute statistics"} 2018-01-05T11:29:00,266 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112900_f7f57959-0c4f-4f84-8860-baee4a2aec40 with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:29:00,266 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,278 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:29:00,278 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,278 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:29:00,278 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,279 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 1.80KB 2018-01-05T11:29:00,284 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_203_2889790531036783127-1/-mr-10000 2018-01-05T11:29:00,295 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_203_2889790531036783127-1/-mr-10000 2018-01-05T11:29:00,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo 2018-01-05T11:29:00,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,297 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,297 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=analyze table src_cbo compute statistics (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112900_f7f57959-0c4f-4f84-8860-baee4a2aec40 } 2018-01-05T11:29:00,297 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:00,299 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: analyze table src_cbo compute statistics (Stage-0) 2018-01-05T11:29:00,299 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:29:00,299 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_28, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_28.dot 2018-01-05T11:29:00,300 INFO [IPC Server handler 1 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_28 2018-01-05T11:29:00,300 WARN [IPC Server handler 1 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:29:00,300 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Running DAG: analyze table src_cbo compute statistics (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112900_f7f57959-0c4f-4f84-8860-baee4a2aec40 } 2018-01-05T11:29:00,313 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000057, QueryId=djaiswal_20180105112859_5f019fd9-7a31-4694-b9e6-c1ae3adfa206-27, DagId=27, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180540111, EndTime=1515180540313 2018-01-05T11:29:00,324 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_28][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_28, submitTime=1515180540299, queueName=null 2018-01-05T11:29:00,324 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_28_00 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_28_00 [Map 1], Default Resources= 2018-01-05T11:29:00,324 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_28_00 [Map 1] 2018-01-05T11:29:00,324 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_28_00 [Map 1] 2018-01-05T11:29:00,324 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:29:00,325 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_28_00 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:00,325 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:29:00,325 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_28][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_28, initTime=1515180540325 2018-01-05T11:29:00,325 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: dag_1515180518813_0001_28 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:29:00,325 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:29:00,325 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_28][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_28, startTime=1515180540325 2018-01-05T11:29:00,325 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_28 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:29:00,325 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {src_cbo={InputName=src_cbo}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:29:00,325 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: src_cbo, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:29:00,325 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_28_00 [Map 1] 2018-01-05T11:29:00,325 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_28, dagName=analyze table src_cbo compute statistics (Stage-0) 2018-01-05T11:29:00,325 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,325 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,325 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,330 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_28_00 [Map 1] 2018-01-05T11:29:00,330 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_28_00 [Map 1] 2018-01-05T11:29:00,330 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:00,330 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_28_00 [Map 1] 2018-01-05T11:29:00,332 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:29:00,332 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@4c64f04a 2018-01-05T11:29:00,332 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_203_2889790531036783127-1/djaiswal/_tez_scratch_dir/697896fb-acbf-4e1c-9c3b-dd2dbfedb3fd/map.xml 2018-01-05T11:29:00,332 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_203_2889790531036783127-1/djaiswal/_tez_scratch_dir/697896fb-acbf-4e1c-9c3b-dd2dbfedb3fd/map.xml 2018-01-05T11:29:00,332 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_203_2889790531036783127-1/djaiswal/_tez_scratch_dir/697896fb-acbf-4e1c-9c3b-dd2dbfedb3fd/map.xml 2018-01-05T11:29:00,332 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:00,332 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:00,332 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,332 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:00,333 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 1.80KB 2018-01-05T11:29:00,333 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: src_cbo on vertex vertex_1515180518813_0001_28_00 [Map 1] 2018-01-05T11:29:00,333 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_28_00 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:29:00,333 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:29:00,333 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:00,333 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:29:00,333 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:00,333 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_203_2889790531036783127-1/djaiswal/_tez_scratch_dir/697896fb-acbf-4e1c-9c3b-dd2dbfedb3fd/map.xml 2018-01-05T11:29:00,333 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:00,333 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias src_cbo 2018-01-05T11:29:00,333 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo 2018-01-05T11:29:00,333 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo 2018-01-05T11:29:00,333 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 2018-01-05T11:29:00,333 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = 2018-01-05T11:29:00,333 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo 2018-01-05T11:29:00,334 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:00,334 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:00,334 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:00,334 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:00,347 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Time taken to get FileStatuses: 13 2018-01-05T11:29:00,347 INFO [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:29:00,347 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 13 2018-01-05T11:29:00,347 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 1 2018-01-05T11:29:00,347 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:00,347 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 1. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:29:00,347 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_203_2889790531036783127-1/djaiswal/_tez_scratch_dir/697896fb-acbf-4e1c-9c3b-dd2dbfedb3fd/map.xml 2018-01-05T11:29:00,347 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_203_2889790531036783127-1/djaiswal/_tez_scratch_dir/697896fb-acbf-4e1c-9c3b-dd2dbfedb3fd/map.xml 2018-01-05T11:29:00,347 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_203_2889790531036783127-1/djaiswal/_tez_scratch_dir/697896fb-acbf-4e1c-9c3b-dd2dbfedb3fd/map.xml 2018-01-05T11:29:00,347 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:00,347 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:00,347 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,348 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:00,348 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 1.80KB 2018-01-05T11:29:00,348 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo/000000_0 to src new group? true 2018-01-05T11:29:00,348 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:29:00,348 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:29:00,348 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:29:00,348 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 107 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 5812 Original splits: 1 2018-01-05T11:29:00,348 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Using original number of splits: 1 desired splits: 1 2018-01-05T11:29:00,348 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 1 grouped split count is 1, for bucket: 1 2018-01-05T11:29:00,348 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:29:00,348 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: src_cbo on vertex vertex_1515180518813_0001_28_00 [Map 1] 2018-01-05T11:29:00,348 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_28_00 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:00,348 INFO [App Shared Pool - #0] impl.VertexImpl: Got updated RootInputsSpecs: {src_cbo=forAllWorkUnits=true, update=[1]} 2018-01-05T11:29:00,349 INFO [App Shared Pool - #0] impl.VertexImpl: Vertex vertex_1515180518813_0001_28_00 [Map 1] parallelism set to 1 2018-01-05T11:29:00,349 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_28_00 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:00,349 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_28_00 [Map 1], numAdditionalOutputs=1 2018-01-05T11:29:00,349 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_28][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_28_00, initRequestedTime=1515180540325, initedTime=1515180540349, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:00,349 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_28_00 [Map 1] with distanceFromRoot: 0 2018-01-05T11:29:00,349 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_28_00 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:29:00,349 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_28][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_28_00, startRequestedTime=1515180540333, startedTime=1515180540349 2018-01-05T11:29:00,349 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_28][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_28_00, reconfigureDoneTime=1515180540349, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:29:00,349 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_28_00 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:00,349 INFO [App Shared Pool - #0] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:29:00,349 INFO [App Shared Pool - #0] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:29:00,349 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_28_00 [Map 1] 2018-01-05T11:29:00,349 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_28][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_28_00_000000, scheduledTime=1515180540349, launchTime=1515180540349 2018-01-05T11:29:00,349 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_28_00 [Map 1] 2018-01-05T11:29:00,349 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_28_00_000000_0, priority=2, capability=, hosts=null 2018-01-05T11:29:00,349 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=2,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:00,349 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=2,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:00,349 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:00,351 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_28_00_000000_0 on hosts=null 2018-01-05T11:29:00,351 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_28_00_000000_0 on hosts=null 2018-01-05T11:29:00,351 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_28_00_000000_0 2018-01-05T11:29:00,351 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:00,352 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #58, task=TaskInfo{task=attempt_1515180518813_0001_28_00_000000_0, priority=2, startTime=0, containerId=null, uniqueId=57, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000058 2018-01-05T11:29:00,352 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_28_00_000000_0; false 2018-01-05T11:29:00,352 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_28_00_000000_0, priority=2, startTime=311560861, containerId=container_111101111_0001_00_000058assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=57, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:00,352 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000058 for node: 10.22.8.182:49818 2018-01-05T11:29:00,354 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 28, name=analyze table src_cbo compute statistics (Stage-0), queryId=djaiswal_20180105112900_f7f57959-0c4f-4f84-8860-baee4a2aec40 2018-01-05T11:29:00,354 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000058, attempt_1515180518813_0001_28_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:00,354 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000058 for node: 10.22.8.182:49818 2018-01-05T11:29:00,354 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000058 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:00,354 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000058, launchTime=1515180540354 2018-01-05T11:29:00,355 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_28_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112900_f7f57959-0c4f-4f84-8860-baee4a2aec40, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000058, dagName=analyze table src_cbo compute statistics (Stage-0), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{src_cbo,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{out_Map 1,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180540325, firstAttemptStartTime=1515180540349, currentAttemptStartTime=1515180540354} 2018-01-05T11:29:00,355 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_28_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000058] on NM: [10.22.8.182:49818] 2018-01-05T11:29:00,355 INFO [IPC Server handler 3 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000058, QueryId=djaiswal_20180105112900_f7f57959-0c4f-4f84-8860-baee4a2aec40-28, DagId=28, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180540355 2018-01-05T11:29:00,355 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:00,355 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:00,355 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_28][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_28_00_000000_0, startTime=1515180540354, containerId=container_111101111_0001_00_000058, nodeId=10.22.8.182:49818 2018-01-05T11:29:00,355 DEBUG [IPC Server handler 3 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=28} with the ShuffleHandler 2018-01-05T11:29:00,366 INFO [IPC Server handler 3 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:29:00,366 DEBUG [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/28] 2018-01-05T11:29:00,366 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:29:00,366 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_28_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:00,366 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_28_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180540349, dagStartTime=1515180540325, withinDagPriority=2, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:29:00,366 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_28_00_000000_0 : ACCEPTED 2018-01-05T11:29:00,366 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_28_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:29:00,366 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_28_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180540349, dagStartTime=1515180540325, withinDagPriority=2, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:29:00,366 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_28_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:00,366 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_28_00_000000_0 to pre-emption queue 2018-01-05T11:29:00,366 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_28_00_000000_0: true 2018-01-05T11:29:00,366 INFO [TaskCommunicator # 1] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_28_00_000000_0 2018-01-05T11:29:00,367 INFO [TaskCommunicator # 1] GuaranteedTasks: Received response for attempt_1515180518813_0001_28_00_000000_0, true 2018-01-05T11:29:00,367 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:1726983018 2018-01-05T11:29:00,367 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : analyze table src_cbo compute statistics (Stage-0), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_28_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=src_cbo, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=out_Map 1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:29:00,367 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1037041664, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:00,367 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_28_00_000000_0 vertexName: Map 1 2018-01-05T11:29:00,368 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000058, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_28_00_000000_0, eventCount=1 } 2018-01-05T11:29:00,368 DEBUG [TezTR-518813_1_28_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_28 queryId: djaiswal_20180105112900_f7f57959-0c4f-4f84-8860-baee4a2aec40 instanceType: TaskRunner2Callable 2018-01-05T11:29:00,368 INFO [TezTR-518813_1_28_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_28_00_000000_0 2018-01-05T11:29:00,368 DEBUG [TezTR-518813_1_28_0_0_0] log.PerfLogger: 2018-01-05T11:29:00,368 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:00,369 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:00,369 DEBUG [AMReporterQueueDrainer] impl.AMReporter: Removing am 10.22.8.182:49824 with last associated dag QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=14} from heartbeat with taskCount=0, amFailed=false, isDone=true 2018-01-05T11:29:00,369 INFO [IPC Server handler 23 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_28_00_000000_0 sent events: (0-1). 2018-01-05T11:29:00,370 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:29:00,370 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_28_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:29:00,370 INFO [I/O Setup 0 Initialize: {src_cbo}] input.MRInput: src_cbo using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:29:00,370 INFO [I/O Setup 0 Initialize: {src_cbo}] input.MRInputLegacy: src_cbo MRInputLegacy deferring initialization 2018-01-05T11:29:00,370 DEBUG [TezTR-518813_1_28_0_0_0] log.PerfLogger: 2018-01-05T11:29:00,370 INFO [TezTR-518813_1_28_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:00,370 INFO [TezTR-518813_1_28_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:00,370 INFO [TezTR-518813_1_28_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:00,371 INFO [I/O Setup 1 Initialize: {out_Map 1}] output.MROutput: out_Map 1: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:29:00,371 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:00,371 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:00,371 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:00,371 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:00,371 INFO [TezTR-518813_1_28_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:00,371 INFO [TezTR-518813_1_28_0_0_0] resources.MemoryDistributor: InitialRequests=[src_cbo:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [out_Map 1:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput] 2018-01-05T11:29:00,371 INFO [TezTR-518813_1_28_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:00,371 INFO [TezTR-518813_1_28_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:00,371 INFO [TezTR-518813_1_28_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 1037041664, TotalAvailable: 694817914, TotalRequested/TotalJVMHeap:0.00 2018-01-05T11:29:00,371 INFO [TezTR-518813_1_28_0_0_0] resources.MemoryDistributor: Allocations=[src_cbo:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [out_Map 1:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0] 2018-01-05T11:29:00,371 INFO [TezTR-518813_1_28_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:00,371 INFO [TezTR-518813_1_28_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: src_cbo being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:00,371 INFO [I/O Setup 0 Start: {src_cbo}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: src_cbo 2018-01-05T11:29:00,371 INFO [TezTR-518813_1_28_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:29:00,371 INFO [TezTR-518813_1_28_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:00,371 INFO [TezTR-518813_1_28_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:00,371 INFO [TezTR-518813_1_28_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_28_00_000000_0 2018-01-05T11:29:00,371 DEBUG [TezTR-518813_1_28_0_0_0] log.PerfLogger: 2018-01-05T11:29:00,371 DEBUG [TezTR-518813_1_28_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_28_00_000000_0_10172 2018-01-05T11:29:00,371 DEBUG [TezTR-518813_1_28_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:00,371 DEBUG [TezTR-518813_1_28_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_28_00_000000_0 2018-01-05T11:29:00,371 INFO [TezTR-518813_1_28_0_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112900_f7f57959-0c4f-4f84-8860-baee4a2aec40 2018-01-05T11:29:00,371 DEBUG [TezTR-518813_1_28_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:00,371 DEBUG [TezTR-518813_1_28_0_0_0] log.PerfLogger: 2018-01-05T11:29:00,371 DEBUG [TezTR-518813_1_28_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:00,371 DEBUG [TezTR-518813_1_28_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:00,371 DEBUG [TezTR-518813_1_28_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112900_f7f57959-0c4f-4f84-8860-baee4a2aec40_Map 1__MAP_PLAN__ 2018-01-05T11:29:00,371 INFO [TezTR-518813_1_28_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_203_2889790531036783127-1/djaiswal/_tez_scratch_dir/697896fb-acbf-4e1c-9c3b-dd2dbfedb3fd/map.xml 2018-01-05T11:29:00,371 DEBUG [TezTR-518813_1_28_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_203_2889790531036783127-1/djaiswal/_tez_scratch_dir/697896fb-acbf-4e1c-9c3b-dd2dbfedb3fd/map.xml 2018-01-05T11:29:00,371 DEBUG [TezTR-518813_1_28_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_203_2889790531036783127-1/djaiswal/_tez_scratch_dir/697896fb-acbf-4e1c-9c3b-dd2dbfedb3fd/map.xml 2018-01-05T11:29:00,371 DEBUG [TezTR-518813_1_28_0_0_0] log.PerfLogger: 2018-01-05T11:29:00,371 INFO [TezTR-518813_1_28_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:00,371 DEBUG [TezTR-518813_1_28_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,372 DEBUG [TezTR-518813_1_28_0_0_0] log.PerfLogger: 2018-01-05T11:29:00,372 INFO [TezTR-518813_1_28_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 1.80KB 2018-01-05T11:29:00,372 DEBUG [TezTR-518813_1_28_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:00,372 INFO [TezTR-518813_1_28_0_0_0] tez.MapRecordProcessor: The input names are: [src_cbo] 2018-01-05T11:29:00,372 INFO [TezTR-518813_1_28_0_0_0] input.MRInput: Initialized MRInput: src_cbo 2018-01-05T11:29:00,372 DEBUG [TezTR-518813_1_28_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 1 2018-01-05T11:29:00,372 INFO [TezTR-518813_1_28_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_203_2889790531036783127-1/djaiswal/_tez_scratch_dir/697896fb-acbf-4e1c-9c3b-dd2dbfedb3fd/map.xml 2018-01-05T11:29:00,372 DEBUG [TezTR-518813_1_28_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:00,372 DEBUG [TezTR-518813_1_28_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo/000000_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct src_cbo { string key, string value}, transient_lastDdlTime=1515180537, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.src_cbo}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct src_cbo { string key, string value}, transient_lastDdlTime=1515180537, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.src_cbo}, baseFileName=src_cbo, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct src_cbo { string key, string value}, transient_lastDdlTime=1515180537, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.src_cbo}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct src_cbo { string key, string value}, transient_lastDdlTime=1515180537, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.src_cbo}, baseFileName=src_cbo, vectorPartitionDesc=null]} 2018-01-05T11:29:00,372 DEBUG [TezTR-518813_1_28_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:29:00,372 INFO [TezTR-518813_1_28_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:29:00,373 DEBUG [TezTR-518813_1_28_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:29:00,373 DEBUG [TezTR-518813_1_28_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:29:00,373 INFO [TezTR-518813_1_28_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_203_2889790531036783127-1/djaiswal/_tez_scratch_dir/697896fb-acbf-4e1c-9c3b-dd2dbfedb3fd/map.xml 2018-01-05T11:29:00,373 DEBUG [TezTR-518813_1_28_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:00,373 INFO [TezTR-518813_1_28_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_28_00_000000_0 2018-01-05T11:29:00,374 INFO [TezTR-518813_1_28_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [string, string] (isAcidRead false) 2018-01-05T11:29:00,374 WARN [TezTR-518813_1_28_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo/000000_0 (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:29:00,374 DEBUG [TezTR-518813_1_28_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:29:00,374 WARN [TezTR-518813_1_28_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo/000000_0:0+5812 2018-01-05T11:29:00,374 DEBUG [TezTR-518813_1_28_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo/000000_0 2018-01-05T11:29:00,374 INFO [TezTR-518813_1_28_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo/000000_0:0+5812], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=5812} 2018-01-05T11:29:00,374 INFO [TezTR-518813_1_28_0_0_0] input.MRInput: src_cbo initialized RecordReader from event 2018-01-05T11:29:00,374 DEBUG [TezTR-518813_1_28_0_0_0] tez.MapRecordProcessor: Starting Output: out_Map 1 2018-01-05T11:29:00,374 INFO [TezTR-518813_1_28_0_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:29:00,374 INFO [TezTR-518813_1_28_0_0_0] exec.MapOperator: Initializing operator MAP[0] 2018-01-05T11:29:00,374 DEBUG [TezTR-518813_1_28_0_0_0] exec.MapOperator: Initialization Done 0 MAP 2018-01-05T11:29:00,374 DEBUG [TezTR-518813_1_28_0_0_0] exec.MapOperator: Operator 0 MAP initialized 2018-01-05T11:29:00,374 DEBUG [TezTR-518813_1_28_0_0_0] exec.MapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:29:00,374 DEBUG [TezTR-518813_1_28_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,375 DEBUG [TezTR-518813_1_28_0_0_0] exec.MapOperator: Adding alias src_cbo to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo 2018-01-05T11:29:00,375 DEBUG [TezTR-518813_1_28_0_0_0] exec.MapOperator: dump TS[0] struct 2018-01-05T11:29:00,375 INFO [TezTR-518813_1_28_0_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:29:00,375 INFO [TezTR-518813_1_28_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:00,375 DEBUG [TezTR-518813_1_28_0_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:29:00,375 INFO [TezTR-518813_1_28_0_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:29:00,375 DEBUG [TezTR-518813_1_28_0_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:29:00,375 DEBUG [TezTR-518813_1_28_0_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:29:00,375 DEBUG [TezTR-518813_1_28_0_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:29:00,375 DEBUG [TezTR-518813_1_28_0_0_0] log.PerfLogger: 2018-01-05T11:29:00,375 INFO [TezTR-518813_1_28_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_203_2889790531036783127-1/djaiswal/_tez_scratch_dir/697896fb-acbf-4e1c-9c3b-dd2dbfedb3fd/map.xml 2018-01-05T11:29:00,375 DEBUG [TezTR-518813_1_28_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:00,375 DEBUG [TezTR-518813_1_28_0_0_0] exec.MapOperator: Processing alias(es) src_cbo for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo/000000_0 2018-01-05T11:29:00,375 INFO [TezTR-518813_1_28_0_0_0] exec.MapOperator: MAP[0]: records read - 1 2018-01-05T11:29:00,375 INFO [TezTR-518813_1_28_0_0_0] exec.MapOperator: MAP[0]: records read - 10 2018-01-05T11:29:00,375 INFO [TezTR-518813_1_28_0_0_0] exec.MapOperator: MAP[0]: records read - 100 2018-01-05T11:29:00,375 DEBUG [TezTR-518813_1_28_0_0_0] log.PerfLogger: 2018-01-05T11:29:00,375 DEBUG [TezTR-518813_1_28_0_0_0] mr.ObjectCache: djaiswal_20180105112900_f7f57959-0c4f-4f84-8860-baee4a2aec40_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:29:00,375 DEBUG [TezTR-518813_1_28_0_0_0] exec.MapOperator: close called for operator MAP[0] 2018-01-05T11:29:00,375 INFO [TezTR-518813_1_28_0_0_0] exec.MapOperator: Closing operator MAP[0] 2018-01-05T11:29:00,375 INFO [TezTR-518813_1_28_0_0_0] exec.MapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:500, 2018-01-05T11:29:00,375 DEBUG [TezTR-518813_1_28_0_0_0] exec.MapOperator: Closing child = TS[0] 2018-01-05T11:29:00,375 DEBUG [TezTR-518813_1_28_0_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:29:00,375 DEBUG [TezTR-518813_1_28_0_0_0] exec.TableScanOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:00,375 INFO [TezTR-518813_1_28_0_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:29:00,375 DEBUG [TezTR-518813_1_28_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,375 DEBUG [TezTR-518813_1_28_0_0_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_203_2889790531036783127-1/-mr-10000 2018-01-05T11:29:00,375 INFO [TezTR-518813_1_28_0_0_0] exec.TableScanOperator: publishing : default.src_cbo/ : {numRows=500, rawDataSize=5312} 2018-01-05T11:29:00,397 DEBUG [TezTR-518813_1_28_0_0_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_203_2889790531036783127-1/-mr-10000/tmpstats-0_0 2018-01-05T11:29:00,397 DEBUG [TezTR-518813_1_28_0_0_0] fs.FSStatsPublisher: Writing stats in it : {default.src_cbo/={numRows=500, rawDataSize=5312}} 2018-01-05T11:29:00,397 DEBUG [TezTR-518813_1_28_0_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:29:00,397 DEBUG [TezTR-518813_1_28_0_0_0] exec.MapOperator: 0 Close done 2018-01-05T11:29:00,398 INFO [TezTR-518813_1_28_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_28_00_000000_0 2018-01-05T11:29:00,398 INFO [TezTR-518813_1_28_0_0_0] output.MROutput: out_Map 1 closed 2018-01-05T11:29:00,398 DEBUG [TezTR-518813_1_28_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:00,398 INFO [TezTR-518813_1_28_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:00,398 INFO [TezTR-518813_1_28_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_28_00_000000_0, askedToStop=false 2018-01-05T11:29:00,398 INFO [TezTR-518813_1_28_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_28_00_000000_0, stopRequested=false 2018-01-05T11:29:00,398 INFO [TezTR-518813_1_28_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_28_00_000000_0: Counters: 5 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=500, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=0][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500]] 2018-01-05T11:29:00,398 DEBUG [TezTR-518813_1_28_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_28_0_0_0 thread id: 1821 scheme: file bytesRead: 5872 bytesWritten: 86 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:00,398 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_28_00_000000_0, isTaskDone=true 2018-01-05T11:29:00,398 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000058, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_28_00_000000_0, eventCount=2 } 2018-01-05T11:29:00,399 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_28][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_28_00_000000_0, creationTime=1515180540349, allocationTime=1515180540352, startTime=1515180540354, finishTime=1515180540399, timeTaken=45, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:00,399 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:29:00,399 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_28_00_000000_0 2018-01-05T11:29:00,399 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_28_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:00,399 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_28_00_000000_0 2018-01-05T11:29:00,399 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_28_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:00,399 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_28_00_000000_0; false 2018-01-05T11:29:00,399 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_28_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:00,399 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000058=32 2018-01-05T11:29:00,399 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_28_00_000000_0: true 2018-01-05T11:29:00,399 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:00,399 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_28_00_000000_0 2018-01-05T11:29:00,399 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:00,399 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_28][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_28_00_000000, startTime=1515180540354, finishTime=1515180540399, timeTaken=45, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_28_00_000000_0, diagnostics=, counters=Counters: 11, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=5872, FILE_BYTES_WRITTEN=86, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=500, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500 2018-01-05T11:29:00,399 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000058, QueryId=djaiswal_20180105112900_f7f57959-0c4f-4f84-8860-baee4a2aec40-28, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:00,399 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_28_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:29:00,399 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_28_00_000000_0 2018-01-05T11:29:00,399 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_28_00 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:00,399 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_28_00 [Map 1] 2018-01-05T11:29:00,399 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_28][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_28_00, initRequestedTime=1515180540325, initedTime=1515180540349, startRequestedTime=1515180540333, startedTime=1515180540349, finishTime=1515180540399, timeTaken=50, status=SUCCEEDED, diagnostics=, counters=Counters: 11, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=5872, FILE_BYTES_WRITTEN=86, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=500, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500, vertexStats=firstTaskStartTime=1515180540354, firstTasksToStart=[ task_1515180518813_0001_28_00_000000 ], lastTaskFinishTime=1515180540399, lastTasksToFinish=[ task_1515180518813_0001_28_00_000000 ], minTaskDuration=45, maxTaskDuration=45, avgTaskDuration=45.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_28_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_28_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:00,399 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_28_00 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:00,399 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_28][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000058, stoppedTime=1515180540399, exitStatus=0 2018-01-05T11:29:00,399 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000058 exited with diagnostics set to Container completed. 2018-01-05T11:29:00,400 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_28_00 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1 2018-01-05T11:29:00,400 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null 2018-01-05T11:29:00,400 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_28_00 [Map 1] 2018-01-05T11:29:00,400 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_28, queueSize=0 2018-01-05T11:29:00,401 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_28][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_28, startTime=1515180540325, finishTime=1515180540400, timeTaken=75, status=SUCCEEDED, diagnostics=, counters=Counters: 15, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, OTHER_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=61, AM_GC_TIME_MILLIS=0, File System Counters, FILE_BYTES_READ=5872, FILE_BYTES_WRITTEN=86, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=500, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500 2018-01-05T11:29:00,401 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_28 finished with state: SUCCEEDED 2018-01-05T11:29:00,401 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_28 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:29:00,401 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000058 2018-01-05T11:29:00,401 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_28, dagState=SUCCEEDED 2018-01-05T11:29:00,401 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_28_post 2018-01-05T11:29:00,401 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:29:00,401 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:29:00,401 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:29:00,401 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 28 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=1, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=1,NumRequestsWithoutLocation=1, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=1} 2018-01-05T11:29:00,401 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:29:00,401 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=analyze table src_cbo compute statistics (Stage-0), with id=dag_1515180518813_0001_28 2018-01-05T11:29:00,401 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 28, to 10.22.8.182:49818 2018-01-05T11:29:00,401 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=1, Remaining=0 2018-01-05T11:29:00,401 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=analyze table src_cbo compute statistics (Stage-0), with id=dag_1515180518813_0001_28 2018-01-05T11:29:00,401 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=28} 2018-01-05T11:29:00,401 INFO [IPC Server handler 1 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=28}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:29:00,401 INFO [IPC Server handler 1 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/28 after 300 seconds 2018-01-05T11:29:00,401 ERROR [IPC Server handler 1 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112900_f7f57959-0c4f-4f84-8860-baee4a2aec40, dag_1515180518813_0001_28 2018-01-05T11:29:00,401 INFO [IPC Server handler 1 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112900_f7f57959-0c4f-4f84-8860-baee4a2aec40 2018-01-05T11:29:00,401 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=28} = 0 2018-01-05T11:29:00,401 DEBUG [IPC Server handler 1 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=28} 2018-01-05T11:29:00,401 DEBUG [IPC Server handler 1 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:29:00,401 DEBUG [IPC Server handler 1 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=28} 2018-01-05T11:29:00,402 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,402 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,402 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 2018-01-05T11:29:00,402 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,402 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-2:STATS] in serial mode 2018-01-05T11:29:00,402 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@6079f9f, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:29:00,402 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:00,402 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,402 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:29:00,402 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:29:00,402 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:00,402 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:00,402 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,402 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:00,402 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:29:00,402 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_203_2889790531036783127-1/-mr-10000 2018-01-05T11:29:00,403 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_203_2889790531036783127-1/-mr-10000 2018-01-05T11:29:00,403 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.src_cbo/ numRows 2018-01-05T11:29:00,403 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.src_cbo/ numRows 500 2018-01-05T11:29:00,403 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.src_cbo/ rawDataSize 2018-01-05T11:29:00,403 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.src_cbo/ rawDataSize 5312 2018-01-05T11:29:00,404 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,404 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:29:00,404 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:29:00,404 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:29:00,404 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:29:00,404 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:00,404 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:00,404 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:00,405 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:00,405 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:00,405 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:29:00,405 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:29:00,405 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,406 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:29:00,406 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,406 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=src_cbo newtbl=src_cbo 2018-01-05T11:29:00,406 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=src_cbo newtbl=src_cbo 2018-01-05T11:29:00,408 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:00,412 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:00,412 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,412 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.src_cbo stats: [numFiles=1, numRows=500, totalSize=5812, rawDataSize=5312] 2018-01-05T11:29:00,412 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: About to delete stats tmp dir :file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_203_2889790531036783127-1/-mr-10000 2018-01-05T11:29:00,413 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,413 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,413 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: analyze table src_cbo compute statistics 2018-01-05T11:29:00,413 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:29:00,413 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@src_cbo 2018-01-05T11:29:00,413 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@src_cbo 2018-01-05T11:29:00,413 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,413 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,413 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:00,413 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=6} 2018-01-05T11:29:00,413 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112900_f7f57959-0c4f-4f84-8860-baee4a2aec40); Time taken: 0.182 seconds 2018-01-05T11:29:00,413 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:29:00,413 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,413 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:00,413 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,413 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query analyze table src_cbo compute statistics 2018-01-05T11:29:00,413 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_203_2889790531036783127-1 2018-01-05T11:29:00,414 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.21 seconds 2018-01-05T11:29:00,414 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:29:00,414 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:00,414 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:29:00,414 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:29:00,414 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,414 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:00,414 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,414 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,414 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,414 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: analyze table src_cbo compute statistics for columns 2018-01-05T11:29:00,414 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112900_27c20ecb-12b4-4a83-b82f-be6498310e11): analyze table src_cbo compute statistics for columns 2018-01-05T11:29:00,414 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,414 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: analyze table src_cbo compute statistics for columns 2018-01-05T11:29:00,414 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:00,414 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,414 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,414 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,414 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,414 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:00,414 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,414 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_cbo 2018-01-05T11:29:00,414 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_cbo 2018-01-05T11:29:00,417 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:00,417 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,417 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: select compute_stats(`key`, 'hll') , compute_stats(`value`, 'hll') from `default`.`src_cbo` 2018-01-05T11:29:00,417 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: select compute_stats(`key`, 'hll') , compute_stats(`value`, 'hll') from `default`.`src_cbo` 2018-01-05T11:29:00,418 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:00,418 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Invoking analyze on rewritten query 2018-01-05T11:29:00,418 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Starting Semantic Analysis 2018-01-05T11:29:00,418 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:00,418 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:00,418 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed phase 1 of Semantic Analysis 2018-01-05T11:29:00,418 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for source tables 2018-01-05T11:29:00,418 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,418 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_cbo 2018-01-05T11:29:00,418 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_cbo 2018-01-05T11:29:00,421 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:00,422 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,422 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for subqueries 2018-01-05T11:29:00,422 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for destination tables 2018-01-05T11:29:00,433 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed getting MetaData in Semantic Analysis 2018-01-05T11:29:00,433 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_417_1251941222306996201-1/-mr-10001 2018-01-05T11:29:00,433 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Table Plan for src_cbo TS[0] 2018-01-05T11:29:00,434 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:00,434 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:00,434 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: RR before GB src_cbo{(key,key: string)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)(RAW__DATA__SIZE,RAW__DATA__SIZE: bigint)} after GB {((tok_function compute_stats (tok_table_or_col key) 'hll'),_col0: struct)((tok_function compute_stats (tok_table_or_col value) 'hll'),_col1: struct)} 2018-01-05T11:29:00,434 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: tree: (tok_select (tok_selexpr (tok_function compute_stats (tok_table_or_col key) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col value) 'hll'))) 2018-01-05T11:29:00,434 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: genSelectPlan: input = {((tok_function compute_stats (tok_table_or_col key) 'hll'),_col0: struct)((tok_function compute_stats (tok_table_or_col value) 'hll'),_col1: struct)} starRr = src_cbo{(key,key: string)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)(RAW__DATA__SIZE,RAW__DATA__SIZE: bigint)} 2018-01-05T11:29:00,434 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Select Plan row schema: null{(_c0,_col0: struct)(_c1,_col1: struct)} 2018-01-05T11:29:00,434 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Select Plan for clause: insclause-0 2018-01-05T11:29:00,434 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_417_1251941222306996201-1/-mr-10000/.hive-staging_hive_2018-01-05_11-29-00_417_1251941222306996201-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_417_1251941222306996201-1/-mr-10000 2018-01-05T11:29:00,434 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_417_1251941222306996201-1/-mr-10000/.hive-staging_hive_2018-01-05_11-29-00_417_1251941222306996201-1 2018-01-05T11:29:00,456 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_417_1251941222306996201-1/-mr-10003 2018-01-05T11:29:00,456 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_417_1251941222306996201-1/-mr-10000 row schema: null{(_c0,_col0: struct)(_c1,_col1: struct)} 2018-01-05T11:29:00,456 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Body Plan for Query Block null 2018-01-05T11:29:00,456 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Plan for Query Block null 2018-01-05T11:29:00,456 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Before logical optimization TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-SEL[5]-FS[6] 2018-01-05T11:29:00,456 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,456 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,456 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,456 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] lineage.Generator: Time taken for lineage transform=0 2018-01-05T11:29:00,456 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,456 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,456 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,456 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,456 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,456 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,456 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,456 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,456 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{src_cbo}key,value: string|{src_cbo}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src_cbo}block__offset__inside__file,INPUT__FILE__NAME: string|{src_cbo}input__file__name,ROW__ID: struct|{src_cbo}row__id,RAW__DATA__SIZE: bigint|{src_cbo}raw__data__size) 2018-01-05T11:29:00,456 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: string|{src_cbo}key,value: string|{src_cbo}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src_cbo}block__offset__inside__file,INPUT__FILE__NAME: string|{src_cbo}input__file__name,ROW__ID: struct|{src_cbo}row__id,RAW__DATA__SIZE: bigint|{src_cbo}raw__data__size) 2018-01-05T11:29:00,456 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:29:00,456 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1) 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0] Column[_col1]) 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{}) 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,457 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for FS(6) 2018-01-05T11:29:00,457 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(5) 2018-01-05T11:29:00,457 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for GBY(4) 2018-01-05T11:29:00,457 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for RS(3) 2018-01-05T11:29:00,457 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for GBY(2) 2018-01-05T11:29:00,457 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(1) 2018-01-05T11:29:00,457 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.PredicatePushDown: After PPD: TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-SEL[5]-FS[6] 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{src_cbo}key,value: string|{src_cbo}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src_cbo}block__offset__inside__file,INPUT__FILE__NAME: string|{src_cbo}input__file__name,ROW__ID: struct|{src_cbo}row__id,RAW__DATA__SIZE: bigint|{src_cbo}raw__data__size) 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: string|{src_cbo}key,value: string|{src_cbo}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src_cbo}block__offset__inside__file,INPUT__FILE__NAME: string|{src_cbo}input__file__name,ROW__ID: struct|{src_cbo}row__id,RAW__DATA__SIZE: bigint|{src_cbo}raw__data__size) 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1) 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0] Column[_col1]) 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{}) 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{src_cbo}key,value: string|{src_cbo}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src_cbo}block__offset__inside__file,INPUT__FILE__NAME: string|{src_cbo}input__file__name,ROW__ID: struct|{src_cbo}row__id,RAW__DATA__SIZE: bigint|{src_cbo}raw__data__size) 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: string|{src_cbo}key,value: string|{src_cbo}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src_cbo}block__offset__inside__file,INPUT__FILE__NAME: string|{src_cbo}input__file__name,ROW__ID: struct|{src_cbo}row__id,RAW__DATA__SIZE: bigint|{src_cbo}raw__data__size) 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1) 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0] Column[_col1]) 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{}) 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 3 key:[] 2018-01-05T11:29:00,457 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 3 oldColExprMap: {VALUE._col0=Column[_col0], VALUE._col1=Column[_col1]} 2018-01-05T11:29:00,457 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 3 newColExprMap: {VALUE._col0=Column[_col0], VALUE._col1=Column[_col1]} 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,458 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,458 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,458 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,458 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,458 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,458 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,458 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,458 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,458 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,458 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,458 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,458 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,458 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.IdentityProjectRemover: Identity project remover optimization removed : SEL[5] 2018-01-05T11:29:00,458 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,458 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,458 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,458 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,458 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,458 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,458 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,458 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: After logical optimization TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-FS[6] 2018-01-05T11:29:00,458 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,458 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,458 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,458 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,458 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,458 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,458 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=src_cbo 2018-01-05T11:29:00,458 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=src_cbo 2018-01-05T11:29:00,463 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 2.636531ms + 0.006418ms, the query is [select "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "TAB_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (...)] 2018-01-05T11:29:00,464 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:00,469 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:00,469 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,469 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting stats (Num rows: 500 Data size: 175168 Basic stats: COMPLETE Column stats: NONE) on TS[0] 2018-01-05T11:29:00,469 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (src_cbo): numRows: 500 dataSize: 175168 basicStatsState: COMPLETE colStatsState: NONE colStats: {value= colName: value colType: string countDistincts: 100 numNulls: 25 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: string countDistincts: 100 numNulls: 25 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true} 2018-01-05T11:29:00,469 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting stats (Num rows: 500 Data size: 175168 Basic stats: COMPLETE Column stats: NONE) on SEL[1] 2018-01-05T11:29:00,469 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [1] STATS-SEL[1]: numRows: 500 dataSize: 175168 basicStatsState: COMPLETE colStatsState: NONE colStats: {value= colName: value colType: string countDistincts: 100 numNulls: 25 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: string countDistincts: 100 numNulls: 25 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true} 2018-01-05T11:29:00,469 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[2]: inputSize: 175168 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:00,469 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [Case 1] STATS-GBY[2]: cardinality: 500 2018-01-05T11:29:00,469 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 1248 Basic stats: COMPLETE Column stats: NONE) on GBY[2] 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-GBY[2]: numRows: 1 dataSize: 1248 basicStatsState: COMPLETE colStatsState: NONE colStats: {_col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true} 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 1248 Basic stats: COMPLETE Column stats: NONE) on RS[3] 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-RS[3]: numRows: 1 dataSize: 1248 basicStatsState: COMPLETE colStatsState: NONE colStats: {_col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true} 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[4]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [Case 7] STATS-GBY[4]: cardinality: 0 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[4]: Equals 0 in number of rows. 0 rows will be set to 1 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 1248 Basic stats: COMPLETE Column stats: NONE) on GBY[4] 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-GBY[4]: numRows: 1 dataSize: 1248 basicStatsState: COMPLETE colStatsState: NONE colStats: {_col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true} 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-FS[6]: numRows: 1 dataSize: 1248 basicStatsState: COMPLETE colStatsState: NONE colStats: {_col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, value= colName: value colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true} 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on TS[0] 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on SEL[1] 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[2] 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[3] 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[4] 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FileSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on FS[6] 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,470 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: FS, 6 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: GBY, 4 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: RS, 3 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: GBY, 2 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: SEL, 1 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:29:00,470 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Cycle free: true 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{src_cbo}key,value: string|{src_cbo}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{src_cbo}block__offset__inside__file,INPUT__FILE__NAME: string|{src_cbo}input__file__name,ROW__ID: struct|{src_cbo}row__id,RAW__DATA__SIZE: bigint|{src_cbo}raw__data__size) 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: string|{src_cbo}key,value: string|{src_cbo}value) 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value]) 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{}) 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: TS[0] 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: RS[3] 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:29:00,470 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo of table src_cbo 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: RS[3] 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: GBY[4] 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: FS[6] 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding reduce work (Reducer 2) for GBY[4] 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Setting up reduce sink: RS[3] with following reduce work: Reducer 2 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Removing RS[3] as parent from GBY[4] 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: FS[6] 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping vectorization 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:29:00,471 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: llap mode: only 2018-01-05T11:29:00,471 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(key, 'hll')' 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'key' 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(value, 'hll')' 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'value' 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'key' 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'value' 2018-01-05T11:29:00,471 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@53d37138 can be done in LLAP 2018-01-05T11:29:00,471 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Reducer 2 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col0)' 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col1)' 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col1' 2018-01-05T11:29:00,471 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@46b22e4a can be done in LLAP 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,471 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed plan generation 2018-01-05T11:29:00,471 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validation start 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Not a partition. 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,471 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0, type:struct, comment:null), FieldSchema(name:_c1, type:struct, comment:null)], properties:null) 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,471 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=7, flushCache_()=0, getTableColumnStatistics_(String, String, List, )=11} 2018-01-05T11:29:00,471 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112900_27c20ecb-12b4-4a83-b82f-be6498310e11); Time taken: 0.057 seconds 2018-01-05T11:29:00,471 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:00,471 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,471 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112900_27c20ecb-12b4-4a83-b82f-be6498310e11): analyze table src_cbo compute statistics for columns 2018-01-05T11:29:00,472 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,472 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: analyze table src_cbo compute statistics for columns 2018-01-05T11:29:00,472 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:29:00,472 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@src_cbo 2018-01-05T11:29:00,472 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@src_cbo 2018-01-05T11:29:00,472 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_417_1251941222306996201-1/-mr-10000 2018-01-05T11:29:00,472 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,472 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,472 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,472 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Query ID = djaiswal_20180105112900_27c20ecb-12b4-4a83-b82f-be6498310e11 2018-01-05T11:29:00,472 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Total jobs = 1 2018-01-05T11:29:00,473 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,473 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:29:00,473 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MAPRED] in serial mode 2018-01-05T11:29:00,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_414_4986098278565479672-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:29:00,501 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000058, QueryId=djaiswal_20180105112900_f7f57959-0c4f-4f84-8860-baee4a2aec40-28, DagId=28, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180540366, EndTime=1515180540501 2018-01-05T11:29:00,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:29:00,520 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:29:00,520 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:29:00,520 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112900_27c20ecb-12b4-4a83-b82f-be6498310e11 2018-01-05T11:29:00,520 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Session is already open 2018-01-05T11:29:00,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:29:00,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,520 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Dag name: analyze table src_cbo compute stat...columns (Stage-0) 2018-01-05T11:29:00,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: DagInfo: {"context":"Hive","description":"\nanalyze table src_cbo compute statistics for columns"} 2018-01-05T11:29:00,520 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112900_27c20ecb-12b4-4a83-b82f-be6498310e11 with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:29:00,521 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,531 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,531 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:29:00,532 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,532 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 2 size: 2.01KB 2018-01-05T11:29:00,548 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,548 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,561 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:29:00,561 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,561 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:29:00,562 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,562 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 3.19KB 2018-01-05T11:29:00,570 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_417_1251941222306996201-1/-mr-10001 2018-01-05T11:29:00,581 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_417_1251941222306996201-1/-mr-10001 2018-01-05T11:29:00,581 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo 2018-01-05T11:29:00,581 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,583 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,583 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,583 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=analyze table src_cbo compute stat...columns (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112900_27c20ecb-12b4-4a83-b82f-be6498310e11 } 2018-01-05T11:29:00,583 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:00,583 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:00,598 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: analyze table src_cbo compute stat...columns (Stage-0) 2018-01-05T11:29:00,598 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:29:00,598 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_29, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_29.dot 2018-01-05T11:29:00,598 INFO [IPC Server handler 0 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_29 2018-01-05T11:29:00,598 WARN [IPC Server handler 0 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:29:00,598 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Running DAG: analyze table src_cbo compute stat...columns (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112900_27c20ecb-12b4-4a83-b82f-be6498310e11 } 2018-01-05T11:29:00,626 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_29][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_29, submitTime=1515180540598, queueName=null 2018-01-05T11:29:00,627 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_29_00 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_29_00 [Map 1], Default Resources= 2018-01-05T11:29:00,627 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_29_00 [Map 1] 2018-01-05T11:29:00,627 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:29:00,627 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_29_00 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:00,627 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_29_01 [Reducer 2]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_29_01 [Reducer 2], Default Resources= 2018-01-05T11:29:00,627 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_29_01 [Reducer 2] 2018-01-05T11:29:00,627 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 2 2018-01-05T11:29:00,627 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_29_01 [Reducer 2] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:00,627 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:29:00,627 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_29][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_29, initTime=1515180540627 2018-01-05T11:29:00,628 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: dag_1515180518813_0001_29 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:29:00,628 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:29:00,628 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_29][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_29, startTime=1515180540628 2018-01-05T11:29:00,628 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_29 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:29:00,628 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {src_cbo={InputName=src_cbo}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:29:00,628 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_29, dagName=analyze table src_cbo compute stat...columns (Stage-0) 2018-01-05T11:29:00,628 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: src_cbo, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:29:00,628 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_29_00 [Map 1] 2018-01-05T11:29:00,628 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,628 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,628 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,633 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_29_00 [Map 1] 2018-01-05T11:29:00,633 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_29_00 [Map 1] 2018-01-05T11:29:00,633 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:00,633 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_29_00 [Map 1] 2018-01-05T11:29:00,638 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:29:00,638 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@757c145e 2018-01-05T11:29:00,638 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_414_4986098278565479672-1/djaiswal/_tez_scratch_dir/eb29269a-ce8e-4a07-8d97-0e2f4248fd57/map.xml 2018-01-05T11:29:00,638 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_414_4986098278565479672-1/djaiswal/_tez_scratch_dir/eb29269a-ce8e-4a07-8d97-0e2f4248fd57/map.xml 2018-01-05T11:29:00,638 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_414_4986098278565479672-1/djaiswal/_tez_scratch_dir/eb29269a-ce8e-4a07-8d97-0e2f4248fd57/map.xml 2018-01-05T11:29:00,638 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:00,638 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:00,638 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,639 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:00,639 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.19KB 2018-01-05T11:29:00,639 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_29_00 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:29:00,639 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: src_cbo on vertex vertex_1515180518813_0001_29_00 [Map 1] 2018-01-05T11:29:00,639 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to ShuffleVertexManager for vertex_1515180518813_0001_29_01 [Reducer 2] 2018-01-05T11:29:00,640 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:29:00,640 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:00,640 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:29:00,640 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:00,640 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_414_4986098278565479672-1/djaiswal/_tez_scratch_dir/eb29269a-ce8e-4a07-8d97-0e2f4248fd57/map.xml 2018-01-05T11:29:00,640 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:00,640 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias src_cbo 2018-01-05T11:29:00,640 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo 2018-01-05T11:29:00,640 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo 2018-01-05T11:29:00,640 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 0,1 2018-01-05T11:29:00,640 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = key,value 2018-01-05T11:29:00,640 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo 2018-01-05T11:29:00,640 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:00,641 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:00,641 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:00,641 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:00,645 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.05 maxFrac: 0.75 auto: false desiredTaskIput: 104857600 2018-01-05T11:29:00,645 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:29:00,645 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 1 tasks for vertex: vertex_1515180518813_0001_29_01 [Reducer 2] 2018-01-05T11:29:00,645 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_29_01 [Reducer 2] 2018-01-05T11:29:00,645 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_29][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_29_01, reconfigureDoneTime=1515180540645, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:29:00,645 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_29_01 [Reducer 2], numAdditionalOutputs=1 2018-01-05T11:29:00,645 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_29][Event:VERTEX_INITIALIZED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_29_01, initRequestedTime=1515180540639, initedTime=1515180540645, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:00,645 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_29_01 [Reducer 2] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:29:00,656 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Time taken to get FileStatuses: 15 2018-01-05T11:29:00,656 INFO [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:29:00,656 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 15 2018-01-05T11:29:00,656 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 1 2018-01-05T11:29:00,656 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:00,656 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 1. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:29:00,656 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_414_4986098278565479672-1/djaiswal/_tez_scratch_dir/eb29269a-ce8e-4a07-8d97-0e2f4248fd57/map.xml 2018-01-05T11:29:00,656 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_414_4986098278565479672-1/djaiswal/_tez_scratch_dir/eb29269a-ce8e-4a07-8d97-0e2f4248fd57/map.xml 2018-01-05T11:29:00,656 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_414_4986098278565479672-1/djaiswal/_tez_scratch_dir/eb29269a-ce8e-4a07-8d97-0e2f4248fd57/map.xml 2018-01-05T11:29:00,656 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:00,656 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:00,656 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,657 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:00,657 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.19KB 2018-01-05T11:29:00,657 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo/000000_0 to src new group? true 2018-01-05T11:29:00,657 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:29:00,657 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:29:00,657 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:29:00,657 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 107 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 5812 Original splits: 1 2018-01-05T11:29:00,657 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Using original number of splits: 1 desired splits: 1 2018-01-05T11:29:00,657 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 1 grouped split count is 1, for bucket: 1 2018-01-05T11:29:00,657 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:29:00,657 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: src_cbo on vertex vertex_1515180518813_0001_29_00 [Map 1] 2018-01-05T11:29:00,657 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_29_00 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:00,658 INFO [App Shared Pool - #0] impl.VertexImpl: Got updated RootInputsSpecs: {src_cbo=forAllWorkUnits=true, update=[1]} 2018-01-05T11:29:00,658 INFO [App Shared Pool - #0] impl.VertexImpl: Vertex vertex_1515180518813_0001_29_00 [Map 1] parallelism set to 1 2018-01-05T11:29:00,658 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_29_00 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:00,658 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_29][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_29_00, initRequestedTime=1515180540628, initedTime=1515180540658, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:00,658 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_29_00 [Map 1] with distanceFromRoot: 0 2018-01-05T11:29:00,658 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_29_00 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:29:00,658 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_29][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_29_00, startRequestedTime=1515180540639, startedTime=1515180540658 2018-01-05T11:29:00,658 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_29][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_29_00, reconfigureDoneTime=1515180540658, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:29:00,658 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_29_00 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:00,658 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_29_00 for vertex: vertex_1515180518813_0001_29_01 [Reducer 2] numStartedSources: 1 numSources: 1 2018-01-05T11:29:00,658 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_29_01 [Reducer 2] with distanceFromRoot: 1 2018-01-05T11:29:00,658 INFO [App Shared Pool - #0] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:29:00,658 INFO [App Shared Pool - #0] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:29:00,658 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_29][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_29_01, startRequestedTime=1515180540658, startedTime=1515180540658 2018-01-05T11:29:00,658 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_29_00 [Map 1] 2018-01-05T11:29:00,658 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_29_01 [Reducer 2] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:00,658 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: OnVertexStarted vertex: Reducer 2 with 0 source tasks and 1 pending tasks 2018-01-05T11:29:00,659 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_29][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_29_00_000000, scheduledTime=1515180540659, launchTime=1515180540659 2018-01-05T11:29:00,659 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Map 1 in vertex: Reducer 2 numBipartiteSourceTasks: 1 2018-01-05T11:29:00,659 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_29_00 [Map 1] 2018-01-05T11:29:00,659 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_29_00_000000_0, priority=5, capability=, hosts=null 2018-01-05T11:29:00,659 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=5,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:00,659 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=5,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:00,659 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:00,659 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_29_00_000000_0 on hosts=null 2018-01-05T11:29:00,659 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_29_00_000000_0 on hosts=null 2018-01-05T11:29:00,659 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_29_00_000000_0 2018-01-05T11:29:00,659 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:00,659 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #59, task=TaskInfo{task=attempt_1515180518813_0001_29_00_000000_0, priority=5, startTime=0, containerId=null, uniqueId=58, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000059 2018-01-05T11:29:00,659 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_29_00_000000_0; false 2018-01-05T11:29:00,659 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_29_00_000000_0, priority=5, startTime=311561169, containerId=container_111101111_0001_00_000059assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=58, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:00,660 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000059 for node: 10.22.8.182:49818 2018-01-05T11:29:00,662 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 29, name=analyze table src_cbo compute stat...columns (Stage-0), queryId=djaiswal_20180105112900_27c20ecb-12b4-4a83-b82f-be6498310e11 2018-01-05T11:29:00,662 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000059, attempt_1515180518813_0001_29_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:00,662 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000059 for node: 10.22.8.182:49818 2018-01-05T11:29:00,662 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000059 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:00,662 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000059, launchTime=1515180540662 2018-01-05T11:29:00,662 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_29_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112900_27c20ecb-12b4-4a83-b82f-be6498310e11, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000059, dagName=analyze table src_cbo compute stat...columns (Stage-0), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{src_cbo,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{Reducer 2,org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput,1}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180540628, firstAttemptStartTime=1515180540659, currentAttemptStartTime=1515180540662} 2018-01-05T11:29:00,662 INFO [IPC Server handler 0 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000059, QueryId=djaiswal_20180105112900_27c20ecb-12b4-4a83-b82f-be6498310e11-29, DagId=29, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180540662 2018-01-05T11:29:00,662 DEBUG [IPC Server handler 0 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=29} with the ShuffleHandler 2018-01-05T11:29:00,662 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_29_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000059] on NM: [10.22.8.182:49818] 2018-01-05T11:29:00,663 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:00,663 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:00,663 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_29][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_29_00_000000_0, startTime=1515180540662, containerId=container_111101111_0001_00_000059, nodeId=10.22.8.182:49818 2018-01-05T11:29:00,674 INFO [IPC Server handler 0 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:29:00,674 DEBUG [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/29] 2018-01-05T11:29:00,674 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:29:00,675 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_29_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:00,675 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_29_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180540659, dagStartTime=1515180540628, withinDagPriority=5, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:29:00,675 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_29_00_000000_0 : ACCEPTED 2018-01-05T11:29:00,675 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_29_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:29:00,675 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_29_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180540659, dagStartTime=1515180540628, withinDagPriority=5, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:29:00,675 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_29_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:00,675 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_29_00_000000_0 to pre-emption queue 2018-01-05T11:29:00,675 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_29_00_000000_0: true 2018-01-05T11:29:00,675 INFO [TaskCommunicator # 2] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_29_00_000000_0 2018-01-05T11:29:00,675 INFO [TaskCommunicator # 2] GuaranteedTasks: Received response for attempt_1515180518813_0001_29_00_000000_0, true 2018-01-05T11:29:00,675 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:1424654955 2018-01-05T11:29:00,676 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : analyze table src_cbo compute stat...columns (Stage-0), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_29_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=src_cbo, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=Reducer 2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput }}, ] 2018-01-05T11:29:00,676 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1038614528, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:00,676 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_29_00_000000_0 vertexName: Map 1 2018-01-05T11:29:00,676 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000059, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_29_00_000000_0, eventCount=1 } 2018-01-05T11:29:00,676 DEBUG [TezTR-518813_1_29_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_29 queryId: djaiswal_20180105112900_27c20ecb-12b4-4a83-b82f-be6498310e11 instanceType: TaskRunner2Callable 2018-01-05T11:29:00,676 INFO [TezTR-518813_1_29_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_29_00_000000_0 2018-01-05T11:29:00,676 DEBUG [TezTR-518813_1_29_0_0_0] log.PerfLogger: 2018-01-05T11:29:00,677 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:00,677 INFO [I/O Setup 1 Initialize: {Reducer 2}] writers.UnorderedPartitionedKVWriter: Requested BufferSize (tez.runtime.unordered.output.buffer.size-mb) : 100 2018-01-05T11:29:00,677 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:00,678 INFO [IPC Server handler 28 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_29_00_000000_0 sent events: (0-1). 2018-01-05T11:29:00,678 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:29:00,678 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_29_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:29:00,679 INFO [I/O Setup 0 Initialize: {src_cbo}] input.MRInput: src_cbo using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:29:00,679 INFO [I/O Setup 0 Initialize: {src_cbo}] input.MRInputLegacy: src_cbo MRInputLegacy deferring initialization 2018-01-05T11:29:00,679 DEBUG [TezTR-518813_1_29_0_0_0] log.PerfLogger: 2018-01-05T11:29:00,679 INFO [TezTR-518813_1_29_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:00,679 INFO [TezTR-518813_1_29_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:00,679 INFO [TezTR-518813_1_29_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:00,679 INFO [TezTR-518813_1_29_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:00,679 INFO [TezTR-518813_1_29_0_0_0] resources.MemoryDistributor: InitialRequests=[src_cbo:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [Reducer 2:OUTPUT:104857600:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput] 2018-01-05T11:29:00,679 INFO [TezTR-518813_1_29_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:00,679 INFO [TezTR-518813_1_29_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:00,679 INFO [TezTR-518813_1_29_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 104857600, TotalRequestedScaled: 5.24288E7, TotalJVMHeap: 1038614528, TotalAvailable: 695871733, TotalRequested/TotalJVMHeap:0.10 2018-01-05T11:29:00,679 INFO [TezTR-518813_1_29_0_0_0] resources.MemoryDistributor: Allocations=[src_cbo:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput:OUTPUT:104857600:104857600] 2018-01-05T11:29:00,679 INFO [TezTR-518813_1_29_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:00,679 INFO [TezTR-518813_1_29_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: src_cbo being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:00,679 INFO [I/O Setup 1 Start: {src_cbo}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: src_cbo 2018-01-05T11:29:00,679 INFO [TezTR-518813_1_29_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:29:00,679 INFO [TezTR-518813_1_29_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:00,679 INFO [TezTR-518813_1_29_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:00,679 INFO [TezTR-518813_1_29_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_29_00_000000_0 2018-01-05T11:29:00,679 DEBUG [TezTR-518813_1_29_0_0_0] log.PerfLogger: 2018-01-05T11:29:00,679 DEBUG [TezTR-518813_1_29_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_29_00_000000_0_10175 2018-01-05T11:29:00,679 DEBUG [TezTR-518813_1_29_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:00,679 DEBUG [TezTR-518813_1_29_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_29_00_000000_0 2018-01-05T11:29:00,679 INFO [TezTR-518813_1_29_0_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112900_27c20ecb-12b4-4a83-b82f-be6498310e11 2018-01-05T11:29:00,679 DEBUG [TezTR-518813_1_29_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:00,679 DEBUG [TezTR-518813_1_29_0_0_0] log.PerfLogger: 2018-01-05T11:29:00,679 DEBUG [TezTR-518813_1_29_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:00,679 DEBUG [TezTR-518813_1_29_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:00,679 DEBUG [TezTR-518813_1_29_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112900_27c20ecb-12b4-4a83-b82f-be6498310e11_Map 1__MAP_PLAN__ 2018-01-05T11:29:00,679 INFO [TezTR-518813_1_29_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_414_4986098278565479672-1/djaiswal/_tez_scratch_dir/eb29269a-ce8e-4a07-8d97-0e2f4248fd57/map.xml 2018-01-05T11:29:00,679 DEBUG [TezTR-518813_1_29_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_414_4986098278565479672-1/djaiswal/_tez_scratch_dir/eb29269a-ce8e-4a07-8d97-0e2f4248fd57/map.xml 2018-01-05T11:29:00,679 DEBUG [TezTR-518813_1_29_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_414_4986098278565479672-1/djaiswal/_tez_scratch_dir/eb29269a-ce8e-4a07-8d97-0e2f4248fd57/map.xml 2018-01-05T11:29:00,679 DEBUG [TezTR-518813_1_29_0_0_0] log.PerfLogger: 2018-01-05T11:29:00,679 INFO [TezTR-518813_1_29_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:00,679 DEBUG [TezTR-518813_1_29_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,680 DEBUG [TezTR-518813_1_29_0_0_0] log.PerfLogger: 2018-01-05T11:29:00,680 INFO [TezTR-518813_1_29_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.19KB 2018-01-05T11:29:00,681 DEBUG [TezTR-518813_1_29_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:00,681 INFO [TezTR-518813_1_29_0_0_0] tez.MapRecordProcessor: The input names are: [src_cbo] 2018-01-05T11:29:00,681 INFO [TezTR-518813_1_29_0_0_0] input.MRInput: Initialized MRInput: src_cbo 2018-01-05T11:29:00,681 DEBUG [TezTR-518813_1_29_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 1 2018-01-05T11:29:00,681 INFO [TezTR-518813_1_29_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_414_4986098278565479672-1/djaiswal/_tez_scratch_dir/eb29269a-ce8e-4a07-8d97-0e2f4248fd57/map.xml 2018-01-05T11:29:00,681 DEBUG [TezTR-518813_1_29_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:00,681 DEBUG [TezTR-518813_1_29_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo/000000_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct src_cbo { string key, string value}, transient_lastDdlTime=1515180540, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.src_cbo}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct src_cbo { string key, string value}, transient_lastDdlTime=1515180540, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.src_cbo}, baseFileName=src_cbo, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct src_cbo { string key, string value}, transient_lastDdlTime=1515180540, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.src_cbo}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct src_cbo { string key, string value}, transient_lastDdlTime=1515180540, rawDataSize=5312, columns.comments=, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.src_cbo}, baseFileName=src_cbo, vectorPartitionDesc=null]} 2018-01-05T11:29:00,681 DEBUG [TezTR-518813_1_29_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:29:00,681 INFO [TezTR-518813_1_29_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:29:00,681 DEBUG [TezTR-518813_1_29_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:29:00,682 DEBUG [TezTR-518813_1_29_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:29:00,682 INFO [TezTR-518813_1_29_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_414_4986098278565479672-1/djaiswal/_tez_scratch_dir/eb29269a-ce8e-4a07-8d97-0e2f4248fd57/map.xml 2018-01-05T11:29:00,682 DEBUG [TezTR-518813_1_29_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:00,682 INFO [TezTR-518813_1_29_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_29_00_000000_0 2018-01-05T11:29:00,682 INFO [TezTR-518813_1_29_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [string, string] (isAcidRead false) 2018-01-05T11:29:00,682 DEBUG [TezTR-518813_1_29_0_0_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:00,682 DEBUG [TezTR-518813_1_29_0_0_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:00,682 DEBUG [TezTR-518813_1_29_0_0_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:00,682 DEBUG [TezTR-518813_1_29_0_0_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:00,682 WARN [TezTR-518813_1_29_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo/000000_0 (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:29:00,683 DEBUG [TezTR-518813_1_29_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:29:00,683 WARN [TezTR-518813_1_29_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo/000000_0:0+5812 2018-01-05T11:29:00,683 DEBUG [TezTR-518813_1_29_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo/000000_0 2018-01-05T11:29:00,683 INFO [TezTR-518813_1_29_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo/000000_0:0+5812], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=5812} 2018-01-05T11:29:00,683 INFO [TezTR-518813_1_29_0_0_0] input.MRInput: src_cbo initialized RecordReader from event 2018-01-05T11:29:00,683 DEBUG [TezTR-518813_1_29_0_0_0] tez.MapRecordProcessor: Starting Output: Reducer 2 2018-01-05T11:29:00,683 INFO [TezTR-518813_1_29_0_0_0] writers.BaseUnorderedPartitionedKVWriter: Instantiating Partitioner: [org.apache.tez.mapreduce.partition.MRPartitioner] 2018-01-05T11:29:00,704 INFO [TezTR-518813_1_29_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: numBuffers=2, sizePerBuffer=52428800, skipBuffers=true, numPartitions=1, availableMemory=104857600, maxSingleBufferSizeBytes=2147483647, pipelinedShuffle=false, isFinalMergeEnabled=true, numPartitions=1, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:29:00,705 INFO [TezTR-518813_1_29_0_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:29:00,705 INFO [TezTR-518813_1_29_0_0_0] exec.MapOperator: Initializing operator MAP[0] 2018-01-05T11:29:00,705 DEBUG [TezTR-518813_1_29_0_0_0] exec.MapOperator: Initialization Done 0 MAP 2018-01-05T11:29:00,705 DEBUG [TezTR-518813_1_29_0_0_0] exec.MapOperator: Operator 0 MAP initialized 2018-01-05T11:29:00,705 DEBUG [TezTR-518813_1_29_0_0_0] exec.MapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:29:00,705 DEBUG [TezTR-518813_1_29_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,705 DEBUG [TezTR-518813_1_29_0_0_0] exec.MapOperator: Adding alias src_cbo to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo 2018-01-05T11:29:00,705 DEBUG [TezTR-518813_1_29_0_0_0] exec.MapOperator: dump TS[0] struct 2018-01-05T11:29:00,705 INFO [TezTR-518813_1_29_0_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:29:00,705 INFO [TezTR-518813_1_29_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:00,705 DEBUG [TezTR-518813_1_29_0_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:29:00,705 INFO [TezTR-518813_1_29_0_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:29:00,705 DEBUG [TezTR-518813_1_29_0_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:29:00,705 DEBUG [TezTR-518813_1_29_0_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:29:00,705 DEBUG [TezTR-518813_1_29_0_0_0] exec.TableScanOperator: Initializing children of 0 TS 2018-01-05T11:29:00,706 DEBUG [TezTR-518813_1_29_0_0_0] exec.SelectOperator: Initializing child 1 SEL 2018-01-05T11:29:00,706 INFO [TezTR-518813_1_29_0_0_0] exec.SelectOperator: Initializing operator SEL[1] 2018-01-05T11:29:00,706 INFO [TezTR-518813_1_29_0_0_0] exec.SelectOperator: SELECT struct 2018-01-05T11:29:00,706 DEBUG [TezTR-518813_1_29_0_0_0] exec.SelectOperator: Initialization Done 1 SEL 2018-01-05T11:29:00,706 DEBUG [TezTR-518813_1_29_0_0_0] exec.SelectOperator: Operator 1 SEL initialized 2018-01-05T11:29:00,706 DEBUG [TezTR-518813_1_29_0_0_0] exec.SelectOperator: Initializing children of 1 SEL 2018-01-05T11:29:00,706 DEBUG [TezTR-518813_1_29_0_0_0] exec.GroupByOperator: Initializing child 2 GBY 2018-01-05T11:29:00,706 INFO [TezTR-518813_1_29_0_0_0] exec.GroupByOperator: Initializing operator GBY[2] 2018-01-05T11:29:00,706 DEBUG [TezTR-518813_1_29_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,706 DEBUG [TezTR-518813_1_29_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,706 INFO [TezTR-518813_1_29_0_0_0] exec.GroupByOperator: Max hash table memory: 93860656 bytes 2018-01-05T11:29:00,706 INFO [TezTR-518813_1_29_0_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:29:00,706 DEBUG [TezTR-518813_1_29_0_0_0] exec.GroupByOperator: Initialization Done 2 GBY 2018-01-05T11:29:00,706 DEBUG [TezTR-518813_1_29_0_0_0] exec.GroupByOperator: Operator 2 GBY initialized 2018-01-05T11:29:00,706 DEBUG [TezTR-518813_1_29_0_0_0] exec.GroupByOperator: Initializing children of 2 GBY 2018-01-05T11:29:00,706 DEBUG [TezTR-518813_1_29_0_0_0] exec.ReduceSinkOperator: Initializing child 3 RS 2018-01-05T11:29:00,706 INFO [TezTR-518813_1_29_0_0_0] exec.ReduceSinkOperator: Initializing operator RS[3] 2018-01-05T11:29:00,706 DEBUG [TezTR-518813_1_29_0_0_0] exec.ReduceSinkOperator: keys size is 0 2018-01-05T11:29:00,706 INFO [TezTR-518813_1_29_0_0_0] exec.ReduceSinkOperator: Using tag = -1 2018-01-05T11:29:00,706 DEBUG [TezTR-518813_1_29_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,706 DEBUG [TezTR-518813_1_29_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,706 DEBUG [TezTR-518813_1_29_0_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:29:00,706 DEBUG [TezTR-518813_1_29_0_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS 2018-01-05T11:29:00,706 DEBUG [TezTR-518813_1_29_0_0_0] exec.ReduceSinkOperator: Operator 3 RS initialized 2018-01-05T11:29:00,706 DEBUG [TezTR-518813_1_29_0_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS done is reset. 2018-01-05T11:29:00,706 DEBUG [TezTR-518813_1_29_0_0_0] exec.GroupByOperator: Initialization Done 2 GBY done is reset. 2018-01-05T11:29:00,706 DEBUG [TezTR-518813_1_29_0_0_0] exec.SelectOperator: Initialization Done 1 SEL done is reset. 2018-01-05T11:29:00,706 DEBUG [TezTR-518813_1_29_0_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:29:00,706 INFO [TezTR-518813_1_29_0_0_0] exec.OperatorUtils: Setting output collector: RS[3] --> Reducer 2 2018-01-05T11:29:00,706 DEBUG [TezTR-518813_1_29_0_0_0] log.PerfLogger: 2018-01-05T11:29:00,706 INFO [TezTR-518813_1_29_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_414_4986098278565479672-1/djaiswal/_tez_scratch_dir/eb29269a-ce8e-4a07-8d97-0e2f4248fd57/map.xml 2018-01-05T11:29:00,706 DEBUG [TezTR-518813_1_29_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:00,706 DEBUG [TezTR-518813_1_29_0_0_0] exec.MapOperator: Processing alias(es) src_cbo for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo/000000_0 2018-01-05T11:29:00,706 INFO [TezTR-518813_1_29_0_0_0] exec.MapOperator: MAP[0]: records read - 1 2018-01-05T11:29:00,706 INFO [TezTR-518813_1_29_0_0_0] exec.MapOperator: MAP[0]: records read - 10 2018-01-05T11:29:00,706 INFO [TezTR-518813_1_29_0_0_0] exec.MapOperator: MAP[0]: records read - 100 2018-01-05T11:29:00,707 DEBUG [TezTR-518813_1_29_0_0_0] log.PerfLogger: 2018-01-05T11:29:00,707 DEBUG [TezTR-518813_1_29_0_0_0] mr.ObjectCache: djaiswal_20180105112900_27c20ecb-12b4-4a83-b82f-be6498310e11_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:29:00,707 DEBUG [TezTR-518813_1_29_0_0_0] exec.MapOperator: close called for operator MAP[0] 2018-01-05T11:29:00,707 INFO [TezTR-518813_1_29_0_0_0] exec.MapOperator: Closing operator MAP[0] 2018-01-05T11:29:00,707 INFO [TezTR-518813_1_29_0_0_0] exec.MapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:500, 2018-01-05T11:29:00,707 DEBUG [TezTR-518813_1_29_0_0_0] exec.MapOperator: Closing child = TS[0] 2018-01-05T11:29:00,707 DEBUG [TezTR-518813_1_29_0_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:29:00,707 DEBUG [TezTR-518813_1_29_0_0_0] exec.TableScanOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:00,707 INFO [TezTR-518813_1_29_0_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:29:00,707 DEBUG [TezTR-518813_1_29_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,707 DEBUG [TezTR-518813_1_29_0_0_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_417_1251941222306996201-1/-mr-10001 2018-01-05T11:29:00,707 INFO [TezTR-518813_1_29_0_0_0] exec.TableScanOperator: publishing : default.src_cbo/ : {numRows=500, rawDataSize=5312} 2018-01-05T11:29:00,731 DEBUG [TezTR-518813_1_29_0_0_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_417_1251941222306996201-1/-mr-10001/tmpstats-0_0 2018-01-05T11:29:00,731 DEBUG [TezTR-518813_1_29_0_0_0] fs.FSStatsPublisher: Writing stats in it : {default.src_cbo/={numRows=500, rawDataSize=5312}} 2018-01-05T11:29:00,731 DEBUG [TezTR-518813_1_29_0_0_0] exec.TableScanOperator: Closing child = SEL[1] 2018-01-05T11:29:00,731 DEBUG [TezTR-518813_1_29_0_0_0] exec.SelectOperator: close called for operator SEL[1] 2018-01-05T11:29:00,731 DEBUG [TezTR-518813_1_29_0_0_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:00,731 INFO [TezTR-518813_1_29_0_0_0] exec.SelectOperator: Closing operator SEL[1] 2018-01-05T11:29:00,731 DEBUG [TezTR-518813_1_29_0_0_0] exec.SelectOperator: Closing child = GBY[2] 2018-01-05T11:29:00,731 DEBUG [TezTR-518813_1_29_0_0_0] exec.GroupByOperator: close called for operator GBY[2] 2018-01-05T11:29:00,731 DEBUG [TezTR-518813_1_29_0_0_0] exec.GroupByOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:00,731 INFO [TezTR-518813_1_29_0_0_0] exec.GroupByOperator: Closing operator GBY[2] 2018-01-05T11:29:00,731 INFO [TezTR-518813_1_29_0_0_0] exec.GroupByOperator: Begin Hash Table flush: size = 1 2018-01-05T11:29:00,731 INFO [TezTR-518813_1_29_0_0_0] exec.ReduceSinkOperator: keys are [] num distributions: 0 2018-01-05T11:29:00,731 DEBUG [TezTR-518813_1_29_0_0_0] exec.GroupByOperator: Closing child = RS[3] 2018-01-05T11:29:00,731 DEBUG [TezTR-518813_1_29_0_0_0] exec.ReduceSinkOperator: close called for operator RS[3] 2018-01-05T11:29:00,731 DEBUG [TezTR-518813_1_29_0_0_0] exec.ReduceSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:00,731 INFO [TezTR-518813_1_29_0_0_0] exec.ReduceSinkOperator: Closing operator RS[3] 2018-01-05T11:29:00,731 INFO [TezTR-518813_1_29_0_0_0] exec.ReduceSinkOperator: RECORDS_OUT_INTERMEDIATE_Map_1:1, 2018-01-05T11:29:00,731 DEBUG [TezTR-518813_1_29_0_0_0] exec.ReduceSinkOperator: 3 Close done 2018-01-05T11:29:00,731 DEBUG [TezTR-518813_1_29_0_0_0] exec.GroupByOperator: 2 Close done 2018-01-05T11:29:00,731 DEBUG [TezTR-518813_1_29_0_0_0] exec.SelectOperator: 1 Close done 2018-01-05T11:29:00,731 DEBUG [TezTR-518813_1_29_0_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:29:00,731 DEBUG [TezTR-518813_1_29_0_0_0] exec.MapOperator: 0 Close done 2018-01-05T11:29:00,732 INFO [TezTR-518813_1_29_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_29_00_000000_0 2018-01-05T11:29:00,732 INFO [TezTR-518813_1_29_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: Waiting for all spills to complete : Pending : 0 2018-01-05T11:29:00,732 INFO [TezTR-518813_1_29_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: All spills complete 2018-01-05T11:29:00,744 DEBUG [TezTR-518813_1_29_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:00,744 INFO [TezTR-518813_1_29_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:00,744 INFO [TezTR-518813_1_29_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_29_00_000000_0, askedToStop=false 2018-01-05T11:29:00,744 INFO [TezTR-518813_1_29_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_29_00_000000_0, stopRequested=false 2018-01-05T11:29:00,744 INFO [TezTR-518813_1_29_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_29_00_000000_0: Counters: 14 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=500, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=2374, OUTPUT_BYTES_WITH_OVERHEAD=2384, OUTPUT_BYTES_PHYSICAL=2412, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500, RECORDS_OUT_INTERMEDIATE_Map_1=1]] 2018-01-05T11:29:00,744 DEBUG [TezTR-518813_1_29_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_29_0_0_0 thread id: 1851 scheme: file bytesRead: 5872 bytesWritten: 2506 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:00,744 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_29_00_000000_0, isTaskDone=true 2018-01-05T11:29:00,745 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000059, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_29_00_000000_0, eventCount=4 } 2018-01-05T11:29:00,745 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_29][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_29_00_000000_0, creationTime=1515180540659, allocationTime=1515180540660, startTime=1515180540662, finishTime=1515180540745, timeTaken=83, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:00,745 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:29:00,745 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_29_00_000000_0 2018-01-05T11:29:00,745 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_29_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:00,745 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_29_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:00,745 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_29_00_000000_0 2018-01-05T11:29:00,745 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_29_00_000000_0; false 2018-01-05T11:29:00,745 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_29_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:00,745 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000059=70 2018-01-05T11:29:00,745 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_29_00_000000_0: true 2018-01-05T11:29:00,746 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_29][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_29_00_000000, startTime=1515180540662, finishTime=1515180540745, timeTaken=83, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_29_00_000000_0, diagnostics=, counters=Counters: 20, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=5872, FILE_BYTES_WRITTEN=2506, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=500, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=2374, OUTPUT_BYTES_WITH_OVERHEAD=2384, OUTPUT_BYTES_PHYSICAL=2412, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500, RECORDS_OUT_INTERMEDIATE_Map_1=1 2018-01-05T11:29:00,746 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:00,746 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_29_00_000000_0 2018-01-05T11:29:00,746 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:00,746 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000059, QueryId=djaiswal_20180105112900_27c20ecb-12b4-4a83-b82f-be6498310e11-29, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:00,746 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_29_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:29:00,746 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_29_00_000000_0 2018-01-05T11:29:00,746 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_29_00 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:00,746 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_29_00 [Map 1] 2018-01-05T11:29:00,746 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_29][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_29_00, initRequestedTime=1515180540628, initedTime=1515180540658, startRequestedTime=1515180540639, startedTime=1515180540658, finishTime=1515180540746, timeTaken=88, status=SUCCEEDED, diagnostics=, counters=Counters: 20, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=5872, FILE_BYTES_WRITTEN=2506, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=500, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=2374, OUTPUT_BYTES_WITH_OVERHEAD=2384, OUTPUT_BYTES_PHYSICAL=2412, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500, RECORDS_OUT_INTERMEDIATE_Map_1=1, vertexStats=firstTaskStartTime=1515180540662, firstTasksToStart=[ task_1515180518813_0001_29_00_000000 ], lastTaskFinishTime=1515180540745, lastTasksToFinish=[ task_1515180518813_0001_29_00_000000 ], minTaskDuration=83, maxTaskDuration=83, avgTaskDuration=83.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_29_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_29_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:00,746 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_29_00 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:00,746 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_29][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000059, stoppedTime=1515180540746, exitStatus=0 2018-01-05T11:29:00,746 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000059 exited with diagnostics set to Container completed. 2018-01-05T11:29:00,746 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_29_01 [Reducer 2] attempt: attempt_1515180518813_0001_29_00_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:29:00,746 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_29_00 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2 2018-01-05T11:29:00,746 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null 2018-01-05T11:29:00,746 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000059 2018-01-05T11:29:00,746 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: All source tasks completed. Ramping up 1 remaining tasks for vertex: Reducer 2 2018-01-05T11:29:00,746 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Scheduling 1 tasks for vertex: Reducer 2 with totalTasks: 1. 1 source tasks completed out of 1. MinSourceTaskCompletedFraction: 1.0 min: 0.05 max: 0.75 2018-01-05T11:29:00,746 INFO [App Shared Pool - #0] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Map 1, destinationVertexName=Reducer 2. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.UnorderedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:29:00,746 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_29_01 [Reducer 2] 2018-01-05T11:29:00,746 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_29][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_29_01_000000, scheduledTime=1515180540746, launchTime=1515180540746 2018-01-05T11:29:00,746 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_29_01 [Reducer 2] 2018-01-05T11:29:00,746 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_29_01_000000_0, priority=14, capability=, hosts=null 2018-01-05T11:29:00,746 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:00,746 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:00,746 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:00,746 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_29_01_000000_0 on hosts=null 2018-01-05T11:29:00,746 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_29_01_000000_0 on hosts=null 2018-01-05T11:29:00,747 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_29_01_000000_0 2018-01-05T11:29:00,747 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:00,747 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #60, task=TaskInfo{task=attempt_1515180518813_0001_29_01_000000_0, priority=14, startTime=0, containerId=null, uniqueId=59, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000060 2018-01-05T11:29:00,747 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_29_01_000000_0; false 2018-01-05T11:29:00,747 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_29_01_000000_0, priority=14, startTime=311561256, containerId=container_111101111_0001_00_000060assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=59, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:00,747 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000060 for node: 10.22.8.182:49818 2018-01-05T11:29:00,747 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000060, attempt_1515180518813_0001_29_01_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:00,747 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000060 for node: 10.22.8.182:49818 2018-01-05T11:29:00,747 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000060 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:00,747 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000060, launchTime=1515180540747 2018-01-05T11:29:00,747 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=29}, sourceName=Map 1, state=S_SUCCEEDED 2018-01-05T11:29:00,748 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_29_01_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000060] on NM: [10.22.8.182:49818] 2018-01-05T11:29:00,748 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:00,748 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:00,748 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_29_01_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112900_27c20ecb-12b4-4a83-b82f-be6498310e11, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000060, dagName=analyze table src_cbo compute stat...columns (Stage-0), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.UnorderedKVInput,1}}, Outputs={{out_Reducer 2,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=2, completedTaskCount=1, dagStartTime=1515180540628, firstAttemptStartTime=1515180540746, currentAttemptStartTime=1515180540747} 2018-01-05T11:29:00,748 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_29][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_29_01_000000_0, startTime=1515180540747, containerId=container_111101111_0001_00_000060, nodeId=10.22.8.182:49818 2018-01-05T11:29:00,748 INFO [IPC Server handler 4 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000060, QueryId=djaiswal_20180105112900_27c20ecb-12b4-4a83-b82f-be6498310e11-29, DagId=29, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, SubmitTime=1515180540748 2018-01-05T11:29:00,748 DEBUG [IPC Server handler 4 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=29} with the ShuffleHandler 2018-01-05T11:29:00,748 DEBUG [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/29] 2018-01-05T11:29:00,748 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:29:00,748 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_29_01_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:00,748 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_29_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180540746, dagStartTime=1515180540628, withinDagPriority=14, vertexParallelism= 1, selfAndUpstreamParallelism= 2, selfAndUpstreamComplete= 1}] 2018-01-05T11:29:00,748 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_29_01_000000_0 : ACCEPTED 2018-01-05T11:29:00,748 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_29_01_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:29:00,748 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_29_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180540746, dagStartTime=1515180540628, withinDagPriority=14, vertexParallelism= 1, selfAndUpstreamParallelism= 2, selfAndUpstreamComplete= 1} 2018-01-05T11:29:00,748 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_29_01_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:00,748 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_29_01_000000_0 to pre-emption queue 2018-01-05T11:29:00,748 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_29_01_000000_0: true 2018-01-05T11:29:00,748 INFO [TaskCommunicator # 5] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_29_01_000000_0 2018-01-05T11:29:00,749 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:1424654955 2018-01-05T11:29:00,749 INFO [TaskCommunicator # 5] GuaranteedTasks: Received response for attempt_1515180518813_0001_29_01_000000_0, true 2018-01-05T11:29:00,749 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : analyze table src_cbo compute stat...columns (Stage-0), VertexName: Reducer 2, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_29_01_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:29:00,749 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1038614528, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:00,749 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_29_01_000000_0 vertexName: Reducer 2 2018-01-05T11:29:00,749 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000060, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_29_01_000000_0, eventCount=1 } 2018-01-05T11:29:00,749 DEBUG [TezTR-518813_1_29_1_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_29 queryId: djaiswal_20180105112900_27c20ecb-12b4-4a83-b82f-be6498310e11 instanceType: TaskRunner2Callable 2018-01-05T11:29:00,749 INFO [TezTR-518813_1_29_1_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_29_01_000000_0 2018-01-05T11:29:00,749 DEBUG [TezTR-518813_1_29_1_0_0] log.PerfLogger: 2018-01-05T11:29:00,750 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:00,750 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:00,751 INFO [IPC Server handler 20 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_29_01_000000_0 1 events [0,1) total 1 vertex_1515180518813_0001_29_01 [Reducer 2] 2018-01-05T11:29:00,751 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:29:00,751 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_29_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:29:00,752 DEBUG [TezTR-518813_1_29_1_0_0] log.PerfLogger: 2018-01-05T11:29:00,752 INFO [TezTR-518813_1_29_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:00,752 INFO [TezTR-518813_1_29_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:00,752 INFO [TezTR-518813_1_29_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:00,752 INFO [I/O Setup 1 Initialize: {out_Reducer 2}] output.MROutput: out_Reducer 2: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:29:00,752 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:00,752 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:00,752 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:00,752 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:00,752 INFO [TezTR-518813_1_29_1_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:00,752 INFO [TezTR-518813_1_29_1_0_0] resources.MemoryDistributor: InitialRequests=[out_Reducer 2:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.UnorderedKVInput] 2018-01-05T11:29:00,752 INFO [TezTR-518813_1_29_1_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:00,752 INFO [TezTR-518813_1_29_1_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:00,752 INFO [TezTR-518813_1_29_1_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 131404920, TotalRequestedScaled: 6.570246E7, TotalJVMHeap: 1038614528, TotalAvailable: 695871733, TotalRequested/TotalJVMHeap:0.13 2018-01-05T11:29:00,752 INFO [TezTR-518813_1_29_1_0_0] resources.MemoryDistributor: Allocations=[out_Reducer 2:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [Map 1:org.apache.tez.runtime.library.input.UnorderedKVInput:INPUT:131404920:131404920] 2018-01-05T11:29:00,752 INFO [TezTR-518813_1_29_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:00,752 INFO [TezTR-518813_1_29_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 0 2018-01-05T11:29:00,752 INFO [TezTR-518813_1_29_1_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:00,752 INFO [TezTR-518813_1_29_1_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_29_01_000000_0 2018-01-05T11:29:00,752 DEBUG [TezTR-518813_1_29_1_0_0] log.PerfLogger: 2018-01-05T11:29:00,752 DEBUG [TezTR-518813_1_29_1_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_29_01_000000_0_10178 2018-01-05T11:29:00,752 DEBUG [TezTR-518813_1_29_1_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:00,753 DEBUG [TezTR-518813_1_29_1_0_0] mr.ObjectCache: Creating djaiswal_20180105112900_27c20ecb-12b4-4a83-b82f-be6498310e11_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:29:00,753 INFO [TezTR-518813_1_29_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_414_4986098278565479672-1/djaiswal/_tez_scratch_dir/6220f4d3-812e-4760-818e-1570e7efedff/reduce.xml 2018-01-05T11:29:00,753 DEBUG [TezTR-518813_1_29_1_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_414_4986098278565479672-1/djaiswal/_tez_scratch_dir/6220f4d3-812e-4760-818e-1570e7efedff/reduce.xml 2018-01-05T11:29:00,753 DEBUG [TezTR-518813_1_29_1_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_414_4986098278565479672-1/djaiswal/_tez_scratch_dir/6220f4d3-812e-4760-818e-1570e7efedff/reduce.xml 2018-01-05T11:29:00,753 DEBUG [TezTR-518813_1_29_1_0_0] log.PerfLogger: 2018-01-05T11:29:00,753 INFO [TezTR-518813_1_29_1_0_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:29:00,753 DEBUG [TezTR-518813_1_29_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,754 DEBUG [TezTR-518813_1_29_1_0_0] log.PerfLogger: 2018-01-05T11:29:00,754 INFO [TezTR-518813_1_29_1_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 2.01KB 2018-01-05T11:29:00,754 DEBUG [TezTR-518813_1_29_1_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:00,754 DEBUG [TezTR-518813_1_29_1_0_0] log.PerfLogger: 2018-01-05T11:29:00,754 DEBUG [TezTR-518813_1_29_1_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:00,754 DEBUG [TezTR-518813_1_29_1_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:00,754 DEBUG [TezTR-518813_1_29_1_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:00,754 INFO [TezTR-518813_1_29_1_0_0] impl.SimpleFetchedInputAllocator: Map_1: RequestedMemory=131404920, AssignedMemory=131404920, maxSingleShuffleLimit=32851230 2018-01-05T11:29:00,754 INFO [TezTR-518813_1_29_1_0_0] impl.ShuffleManager: Map_1: numInputs=1, compressionCodec=NoCompressionCodec, numFetchers=1, ifileBufferSize=4096, ifileReadAheadEnabled=true, ifileReadAheadLength=4194304, localDiskFetchEnabled=false, sharedFetchEnabled=false, keepAlive=false, keepAliveMaxConnections=20, connectionTimeout=20000, readTimeout=180000, bufferSize=8192, bufferSize=8192, maxTaskOutputAtOnce=20 2018-01-05T11:29:00,755 INFO [TezTR-518813_1_29_1_0_0] impl.ShuffleInputEventHandlerImpl: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:29:00,755 INFO [TezTR-518813_1_29_1_0_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:29:00,756 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=29&reduce=0&map=attempt_1515180518813_0001_29_00_000000_0_10177 mapId: [attempt_1515180518813_0001_29_00_000000_0_10177] reduceId: [0] jobId: [job_1515180518813_0001] dagId: [29] keepAlive: false 2018-01-05T11:29:00,756 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=29&reduce=0&map=attempt_1515180518813_0001_29_00_000000_0_10177; hash=...vUo6zkpvqLewE 2018-01-05T11:29:00,756 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=29&reduce=0&map=attempt_1515180518813_0001_29_00_000000_0_10177;reply=UAyXF3tYxYIAg 2018-01-05T11:29:00,756 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=29, user='djaiswal', attemptId='attempt_1515180518813_0001_29_00_000000_0_10177'} via loader 2018-01-05T11:29:00,756 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=29, user='djaiswal', attemptId='attempt_1515180518813_0001_29_00_000000_0_10177'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:29:00,756 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_29_00_000000_0_10177 not found 2018-01-05T11:29:00,756 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_29_00_000000_0_10177,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/29/output/attempt_1515180518813_0001_29_00_000000_0_10177/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/29/output/attempt_1515180518813_0001_29_00_000000_0_10177/file.out.index 2018-01-05T11:29:00,756 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Content Length in shuffle : 2443 2018-01-05T11:29:00,757 INFO [Fetcher_B {Map_1} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=29&reduce=0&map=attempt_1515180518813_0001_29_00_000000_0_10177 sent hash and receievd reply 0 ms 2018-01-05T11:29:00,757 INFO [Fetcher_B {Map_1} #0] ShuffleManager.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_29_00_000000_0_10177} to MEMORY, csize=2388, dsize=2384, EndTime=1515180540757, TimeTaken=0, Rate=0.00 MB/s 2018-01-05T11:29:00,757 INFO [Fetcher_B {Map_1} #0] impl.ShuffleManager: All inputs fetched for input vertex : Map 1 2018-01-05T11:29:00,757 INFO [Fetcher_B {Map_1} #0] impl.ShuffleManager: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:00,757 INFO [TezTR-518813_1_29_1_0_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:29:00,757 INFO [ShuffleRunner {Map_1}] impl.ShuffleManager: Map_1: Shutting down FetchScheduler, Was Interrupted: false 2018-01-05T11:29:00,757 INFO [TezTR-518813_1_29_1_0_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:00,757 INFO [ShuffleRunner {Map_1}] impl.ShuffleManager: Map_1: Scheduler thread completed 2018-01-05T11:29:00,757 DEBUG [TezTR-518813_1_29_1_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:00,757 DEBUG [TezTR-518813_1_29_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,757 DEBUG [TezTR-518813_1_29_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,757 DEBUG [TezTR-518813_1_29_1_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:29:00,757 DEBUG [TezTR-518813_1_29_1_0_0] log.PerfLogger: 2018-01-05T11:29:00,757 INFO [TezTR-518813_1_29_1_0_0] exec.GroupByOperator: Initializing operator GBY[4] 2018-01-05T11:29:00,757 DEBUG [TezTR-518813_1_29_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,757 DEBUG [TezTR-518813_1_29_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,757 INFO [TezTR-518813_1_29_1_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:29:00,757 DEBUG [TezTR-518813_1_29_1_0_0] exec.GroupByOperator: Initialization Done 4 GBY 2018-01-05T11:29:00,757 DEBUG [TezTR-518813_1_29_1_0_0] exec.GroupByOperator: Operator 4 GBY initialized 2018-01-05T11:29:00,757 DEBUG [TezTR-518813_1_29_1_0_0] exec.GroupByOperator: Initializing children of 4 GBY 2018-01-05T11:29:00,757 DEBUG [TezTR-518813_1_29_1_0_0] exec.FileSinkOperator: Initializing child 6 FS 2018-01-05T11:29:00,757 INFO [TezTR-518813_1_29_1_0_0] exec.FileSinkOperator: Initializing operator FS[6] 2018-01-05T11:29:00,757 DEBUG [TezTR-518813_1_29_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,758 INFO [TezTR-518813_1_29_1_0_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@480ea370]:[_col0, _col1]:[struct, struct]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@56deee74 2018-01-05T11:29:00,758 DEBUG [TezTR-518813_1_29_1_0_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:29:00,758 DEBUG [TezTR-518813_1_29_1_0_0] exec.FileSinkOperator: Initialization Done 6 FS 2018-01-05T11:29:00,758 DEBUG [TezTR-518813_1_29_1_0_0] exec.FileSinkOperator: Operator 6 FS initialized 2018-01-05T11:29:00,758 DEBUG [TezTR-518813_1_29_1_0_0] exec.FileSinkOperator: Initialization Done 6 FS done is reset. 2018-01-05T11:29:00,758 DEBUG [TezTR-518813_1_29_1_0_0] exec.GroupByOperator: Initialization Done 4 GBY done is reset. 2018-01-05T11:29:00,758 INFO [TezTR-518813_1_29_1_0_0] tez.ReduceRecordProcessor: Id =4 Id =6 <\Children> Id = 4 null<\Parent> <\FS> <\Children> <\Parent> <\GBY> 2018-01-05T11:29:00,758 DEBUG [TezTR-518813_1_29_1_0_0] log.PerfLogger: 2018-01-05T11:29:00,758 INFO [TezTR-518813_1_29_1_0_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 2 2018-01-05T11:29:00,758 DEBUG [TezTR-518813_1_29_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:00,758 DEBUG [TezTR-518813_1_29_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:00,758 DEBUG [TezTR-518813_1_29_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:00,758 DEBUG [TezTR-518813_1_29_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:00,758 DEBUG [TezTR-518813_1_29_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:00,758 DEBUG [TezTR-518813_1_29_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:00,758 INFO [TezTR-518813_1_29_1_0_0] readers.UnorderedKVReader: Num Records read: 1 2018-01-05T11:29:00,758 DEBUG [TezTR-518813_1_29_1_0_0] log.PerfLogger: 2018-01-05T11:29:00,758 DEBUG [TezTR-518813_1_29_1_0_0] mr.ObjectCache: djaiswal_20180105112900_27c20ecb-12b4-4a83-b82f-be6498310e11_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:29:00,758 DEBUG [TezTR-518813_1_29_1_0_0] exec.GroupByOperator: close called for operator GBY[4] 2018-01-05T11:29:00,758 INFO [TezTR-518813_1_29_1_0_0] exec.GroupByOperator: Closing operator GBY[4] 2018-01-05T11:29:00,758 INFO [TezTR-518813_1_29_1_0_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_417_1251941222306996201-1/-mr-10000/.hive-staging_hive_2018-01-05_11-29-00_417_1251941222306996201-1/_tmp.-ext-10002/000000_0 2018-01-05T11:29:00,758 INFO [TezTR-518813_1_29_1_0_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_417_1251941222306996201-1/-mr-10000/.hive-staging_hive_2018-01-05_11-29-00_417_1251941222306996201-1/_task_tmp.-ext-10002/_tmp.000000_0 2018-01-05T11:29:00,758 INFO [TezTR-518813_1_29_1_0_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_417_1251941222306996201-1/-mr-10000/.hive-staging_hive_2018-01-05_11-29-00_417_1251941222306996201-1/_tmp.-ext-10002/000000_0 2018-01-05T11:29:00,776 DEBUG [AMReporterQueueDrainer] impl.AMReporter: Removing am 10.22.8.182:49824 with last associated dag QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=15} from heartbeat with taskCount=0, amFailed=false, isDone=true 2018-01-05T11:29:00,794 INFO [TezTR-518813_1_29_1_0_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:29:00,794 DEBUG [TezTR-518813_1_29_1_0_0] exec.GroupByOperator: Closing child = FS[6] 2018-01-05T11:29:00,794 DEBUG [TezTR-518813_1_29_1_0_0] exec.FileSinkOperator: close called for operator FS[6] 2018-01-05T11:29:00,794 DEBUG [TezTR-518813_1_29_1_0_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:00,794 INFO [TezTR-518813_1_29_1_0_0] exec.FileSinkOperator: Closing operator FS[6] 2018-01-05T11:29:00,794 INFO [TezTR-518813_1_29_1_0_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:29:00,795 INFO [TezTR-518813_1_29_1_0_0] exec.FileSinkOperator: RECORDS_OUT_0:1, 2018-01-05T11:29:00,795 DEBUG [TezTR-518813_1_29_1_0_0] exec.FileSinkOperator: 6 Close done 2018-01-05T11:29:00,795 DEBUG [TezTR-518813_1_29_1_0_0] exec.GroupByOperator: 4 Close done 2018-01-05T11:29:00,795 INFO [TezTR-518813_1_29_1_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_29_01_000000_0 2018-01-05T11:29:00,795 INFO [TezTR-518813_1_29_1_0_0] impl.ShuffleInputEventHandlerImpl: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:29:00,795 INFO [TezTR-518813_1_29_1_0_0] impl.ShuffleManager: Shutting down pending fetchers on sourceMap_1: 0 2018-01-05T11:29:00,795 INFO [TezTR-518813_1_29_1_0_0] output.MROutput: out_Reducer 2 closed 2018-01-05T11:29:00,795 DEBUG [TezTR-518813_1_29_1_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:00,795 INFO [TezTR-518813_1_29_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:00,795 INFO [TezTR-518813_1_29_1_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_29_01_000000_0, askedToStop=false 2018-01-05T11:29:00,795 INFO [TezTR-518813_1_29_1_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_29_01_000000_0, stopRequested=false 2018-01-05T11:29:00,795 DEBUG [HikariPool-1 housekeeper] pool.HikariPool: HikariPool-1 - Pool stats (total=10, active=0, idle=10, waiting=0) 2018-01-05T11:29:00,795 INFO [TezTR-518813_1_29_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_29_01_000000_0: Counters: 14 [[org.apache.tez.common.counters.TaskCounter NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=2388, SHUFFLE_BYTES_DECOMPRESSED=2384, SHUFFLE_BYTES_TO_MEM=2388, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=3, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1][HIVE CREATED_FILES=1, RECORDS_OUT_0=1]] 2018-01-05T11:29:00,795 DEBUG [TezTR-518813_1_29_1_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_29_1_0_0 thread id: 1863 scheme: file bytesRead: 0 bytesWritten: 3285 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:00,796 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_29_01_000000_0, isTaskDone=true 2018-01-05T11:29:00,796 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000060, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_29_01_000000_0, eventCount=2 } 2018-01-05T11:29:00,796 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_29][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_29_01_000000_0, creationTime=1515180540746, allocationTime=1515180540747, startTime=1515180540747, finishTime=1515180540796, timeTaken=49, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:00,796 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:00,796 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_29_01_000000_0 2018-01-05T11:29:00,796 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_29_01_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:00,796 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_29_01_000000_0; false 2018-01-05T11:29:00,796 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_29_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:00,796 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_29_01_000000_0 2018-01-05T11:29:00,796 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_29][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_29_01_000000, startTime=1515180540747, finishTime=1515180540796, timeTaken=49, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_29_01_000000_0, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=3285, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=2388, SHUFFLE_BYTES_DECOMPRESSED=2384, SHUFFLE_BYTES_TO_MEM=2388, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=3, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1 2018-01-05T11:29:00,796 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_29_01_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:00,797 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:00,797 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:00,797 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000060=48 2018-01-05T11:29:00,797 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_29_01_000000_0: true 2018-01-05T11:29:00,797 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_29_01 [Reducer 2], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:00,797 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_29_01 [Reducer 2] 2018-01-05T11:29:00,797 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_29][Event:VERTEX_FINISHED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_29_01, initRequestedTime=1515180540639, initedTime=1515180540645, startRequestedTime=1515180540658, startedTime=1515180540658, finishTime=1515180540797, timeTaken=139, status=SUCCEEDED, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=3285, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=2388, SHUFFLE_BYTES_DECOMPRESSED=2384, SHUFFLE_BYTES_TO_MEM=2388, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=3, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1, vertexStats=firstTaskStartTime=1515180540747, firstTasksToStart=[ task_1515180518813_0001_29_01_000000 ], lastTaskFinishTime=1515180540796, lastTasksToFinish=[ task_1515180518813_0001_29_01_000000 ], minTaskDuration=49, maxTaskDuration=49, avgTaskDuration=49.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_29_01_000000 ], longestDurationTasks=[ task_1515180518813_0001_29_01_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:00,797 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_29_01 [Reducer 2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:00,797 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_29][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000060, stoppedTime=1515180540797, exitStatus=0 2018-01-05T11:29:00,797 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000060 exited with diagnostics set to Container completed. 2018-01-05T11:29:00,797 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_29_01 [Reducer 2] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2 2018-01-05T11:29:00,797 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null 2018-01-05T11:29:00,798 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_29_00 [Map 1] 2018-01-05T11:29:00,798 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_29_01 [Reducer 2] 2018-01-05T11:29:00,798 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_29, queueSize=0 2018-01-05T11:29:00,799 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_29][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_29, startTime=1515180540628, finishTime=1515180540798, timeTaken=170, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, OTHER_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=148, AM_GC_TIME_MILLIS=0, File System Counters, FILE_BYTES_READ=5872, FILE_BYTES_WRITTEN=5791, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=501, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=2374, OUTPUT_BYTES_WITH_OVERHEAD=2384, OUTPUT_BYTES_PHYSICAL=2412, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=2388, SHUFFLE_BYTES_DECOMPRESSED=2384, SHUFFLE_BYTES_TO_MEM=2388, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=3, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1, HIVE, CREATED_FILES=1, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500, RECORDS_OUT_0=1, RECORDS_OUT_INTERMEDIATE_Map_1=1 2018-01-05T11:29:00,799 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_29 finished with state: SUCCEEDED 2018-01-05T11:29:00,799 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_29 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:29:00,799 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000060 2018-01-05T11:29:00,799 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_29, dagState=SUCCEEDED 2018-01-05T11:29:00,799 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_29_post 2018-01-05T11:29:00,799 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:29:00,799 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:29:00,799 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:29:00,799 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 29 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=2, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=2,NumRequestsWithoutLocation=2, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=2} 2018-01-05T11:29:00,799 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:29:00,799 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=analyze table src_cbo compute stat...columns (Stage-0), with id=dag_1515180518813_0001_29 2018-01-05T11:29:00,799 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 29, to 10.22.8.182:49818 2018-01-05T11:29:00,799 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=2, Remaining=0 2018-01-05T11:29:00,799 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=analyze table src_cbo compute stat...columns (Stage-0), with id=dag_1515180518813_0001_29 2018-01-05T11:29:00,799 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=29} 2018-01-05T11:29:00,799 INFO [IPC Server handler 3 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=29}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:29:00,799 INFO [IPC Server handler 3 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/29 after 300 seconds 2018-01-05T11:29:00,799 ERROR [IPC Server handler 3 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112900_27c20ecb-12b4-4a83-b82f-be6498310e11, dag_1515180518813_0001_29 2018-01-05T11:29:00,799 INFO [IPC Server handler 3 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112900_27c20ecb-12b4-4a83-b82f-be6498310e11 2018-01-05T11:29:00,799 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=29} = 1 2018-01-05T11:29:00,799 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Issuing killFragment for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=29} attempt_1515180518813_0001_29_01_000000_0 2018-01-05T11:29:00,799 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Removing attempt_1515180518813_0001_29_01_000000_0 from preemptionQueue 2018-01-05T11:29:00,799 INFO [IPC Server handler 3 on 49818] impl.TaskRunnerCallable: Ignoring kill request for task attempt_1515180518813_0001_29_01_000000_0 since it's already complete 2018-01-05T11:29:00,799 DEBUG [IPC Server handler 3 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=29} 2018-01-05T11:29:00,799 DEBUG [IPC Server handler 3 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:29:00,799 DEBUG [IPC Server handler 3 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=29} 2018-01-05T11:29:00,800 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,800 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,800 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,800 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,800 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 Reducer 2: 1/1 2018-01-05T11:29:00,800 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,801 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,801 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: TaskId for 000000_0 = 000000 2018-01-05T11:29:00,801 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,801 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,801 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,802 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-2:STATS] in serial mode 2018-01-05T11:29:00,802 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@2d52d18e, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:29:00,802 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:00,802 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,802 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:29:00,803 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:29:00,803 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:00,803 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:00,803 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,803 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:00,803 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:29:00,803 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_417_1251941222306996201-1/-mr-10001 2018-01-05T11:29:00,803 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_417_1251941222306996201-1/-mr-10001 2018-01-05T11:29:00,803 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.src_cbo/ numRows 2018-01-05T11:29:00,803 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.src_cbo/ numRows 500 2018-01-05T11:29:00,804 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.src_cbo/ rawDataSize 2018-01-05T11:29:00,804 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.src_cbo/ rawDataSize 5312 2018-01-05T11:29:00,804 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,804 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:29:00,804 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:29:00,804 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:29:00,804 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:29:00,805 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:00,805 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:00,805 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:00,805 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:00,806 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:00,806 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:29:00,806 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:29:00,806 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,806 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:29:00,806 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,806 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=src_cbo newtbl=src_cbo 2018-01-05T11:29:00,806 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=src_cbo newtbl=src_cbo 2018-01-05T11:29:00,808 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:00,811 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:00,812 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,812 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.src_cbo stats: [numFiles=1, numRows=500, totalSize=5812, rawDataSize=5312] 2018-01-05T11:29:00,812 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: About to delete stats tmp dir :file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_417_1251941222306996201-1/-mr-10001 2018-01-05T11:29:00,827 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Time taken to get FileStatuses: 14 2018-01-05T11:29:00,827 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:29:00,827 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 14 2018-01-05T11:29:00,827 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FetchOperator: Creating fetchTask with deserializer typeinfo: struct<_col0:struct,_col1:struct> 2018-01-05T11:29:00,827 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FetchOperator: deserializer properties: table properties: {columns=_col0,_col1, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct:struct, escape.delim=\} partition properties: {columns=_col0,_col1, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct:struct, escape.delim=\} 2018-01-05T11:29:00,828 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,828 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: write_column_statistics: db=default table=src_cbo 2018-01-05T11:29:00,828 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=write_column_statistics: db=default table=src_cbo 2018-01-05T11:29:00,831 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=src_cbo tableName=default colName=key 2018-01-05T11:29:00,831 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:00,832 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=src_cbo tableName=default colName=value 2018-01-05T11:29:00,833 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:00,834 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,834 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,834 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,834 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: analyze table src_cbo compute statistics for columns 2018-01-05T11:29:00,834 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:29:00,834 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@src_cbo 2018-01-05T11:29:00,834 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@src_cbo 2018-01-05T11:29:00,834 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_417_1251941222306996201-1/-mr-10000 2018-01-05T11:29:00,834 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,834 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,834 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:00,834 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=6, setPartitionColumnStatistics_(SetPartitionsStatsRequest, )=6} 2018-01-05T11:29:00,834 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112900_27c20ecb-12b4-4a83-b82f-be6498310e11); Time taken: 0.363 seconds 2018-01-05T11:29:00,834 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:29:00,834 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,834 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:00,834 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,834 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query analyze table src_cbo compute statistics for columns 2018-01-05T11:29:00,834 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_414_4986098278565479672-1 2018-01-05T11:29:00,835 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.42 seconds 2018-01-05T11:29:00,835 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:29:00,835 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:00,835 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:29:00,835 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:29:00,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:00,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: analyze table part compute statistics 2018-01-05T11:29:00,835 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112900_08173570-4964-412e-a3cb-561353455cac): analyze table part compute statistics 2018-01-05T11:29:00,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: analyze table part compute statistics 2018-01-05T11:29:00,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:00,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,835 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,836 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:00,836 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Invoking analyze on original query 2018-01-05T11:29:00,836 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Starting Semantic Analysis 2018-01-05T11:29:00,836 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed phase 1 of Semantic Analysis 2018-01-05T11:29:00,836 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for source tables 2018-01-05T11:29:00,836 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,836 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=part 2018-01-05T11:29:00,836 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=part 2018-01-05T11:29:00,838 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:00,838 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,838 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,838 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=part 2018-01-05T11:29:00,838 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=part 2018-01-05T11:29:00,840 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:00,841 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,841 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for subqueries 2018-01-05T11:29:00,841 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for destination tables 2018-01-05T11:29:00,841 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed getting MetaData in Semantic Analysis 2018-01-05T11:29:00,849 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000059, QueryId=djaiswal_20180105112900_27c20ecb-12b4-4a83-b82f-be6498310e11-29, DagId=29, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180540675, EndTime=1515180540849 2018-01-05T11:29:00,849 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_29_01_000000_0 2018-01-05T11:29:00,849 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_29_01_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:29:00,849 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_29_01_000000_0 2018-01-05T11:29:00,849 INFO [ExecutionCompletionThread #0] impl.QueryTracker: Ignoring fragmentComplete message for unknown query: QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=29} 2018-01-05T11:29:00,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_835_5577510622354784072-1/-mr-10000 2018-01-05T11:29:00,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Table Plan for part TS[0] 2018-01-05T11:29:00,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Body Plan for Query Block null 2018-01-05T11:29:00,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Plan for Query Block null 2018-01-05T11:29:00,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Before logical optimization TS[0] 2018-01-05T11:29:00,852 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] lineage.Generator: Time taken for lineage transform=0 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(p_partkey: int|{part}p_partkey,p_name: string|{part}p_name,p_mfgr: string|{part}p_mfgr,p_brand: string|{part}p_brand,p_type: string|{part}p_type,p_size: int|{part}p_size,p_container: string|{part}p_container,p_retailprice: double|{part}p_retailprice,p_comment: string|{part}p_comment,BLOCK__OFFSET__INSIDE__FILE: bigint|{part}block__offset__inside__file,INPUT__FILE__NAME: string|{part}input__file__name,ROW__ID: struct|{part}row__id,RAW__DATA__SIZE: bigint|{part}raw__data__size) 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.PredicatePushDown: After PPD: TS[0] 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(p_partkey: int|{part}p_partkey,p_name: string|{part}p_name,p_mfgr: string|{part}p_mfgr,p_brand: string|{part}p_brand,p_type: string|{part}p_type,p_size: int|{part}p_size,p_container: string|{part}p_container,p_retailprice: double|{part}p_retailprice,p_comment: string|{part}p_comment,BLOCK__OFFSET__INSIDE__FILE: bigint|{part}block__offset__inside__file,INPUT__FILE__NAME: string|{part}input__file__name,ROW__ID: struct|{part}row__id,RAW__DATA__SIZE: bigint|{part}raw__data__size) 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(p_partkey: int|{part}p_partkey,p_name: string|{part}p_name,p_mfgr: string|{part}p_mfgr,p_brand: string|{part}p_brand,p_type: string|{part}p_type,p_size: int|{part}p_size,p_container: string|{part}p_container,p_retailprice: double|{part}p_retailprice,p_comment: string|{part}p_comment,BLOCK__OFFSET__INSIDE__FILE: bigint|{part}block__offset__inside__file,INPUT__FILE__NAME: string|{part}input__file__name,ROW__ID: struct|{part}row__id,RAW__DATA__SIZE: bigint|{part}raw__data__size) 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: After logical optimization TS[0] 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,853 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,854 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,854 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=part 2018-01-05T11:29:00,854 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=part 2018-01-05T11:29:00,856 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:00,859 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:00,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:00,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting stats (Num rows: 1 Data size: 31730 Basic stats: COMPLETE Column stats: COMPLETE) on TS[0] 2018-01-05T11:29:00,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (part): numRows: 1 dataSize: 31730 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:00,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on TS[0] 2018-01-05T11:29:00,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:29:00,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:29:00,860 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Cycle free: true 2018-01-05T11:29:00,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(p_partkey: int|{part}p_partkey,p_name: string|{part}p_name,p_mfgr: string|{part}p_mfgr,p_brand: string|{part}p_brand,p_type: string|{part}p_type,p_size: int|{part}p_size,p_container: string|{part}p_container,p_retailprice: double|{part}p_retailprice,p_comment: string|{part}p_comment,BLOCK__OFFSET__INSIDE__FILE: bigint|{part}block__offset__inside__file,INPUT__FILE__NAME: string|{part}input__file__name,ROW__ID: struct|{part}row__id,RAW__DATA__SIZE: bigint|{part}raw__data__size) 2018-01-05T11:29:00,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:29:00,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part of table part 2018-01-05T11:29:00,861 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part 2018-01-05T11:29:00,861 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:29:00,861 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,861 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,861 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:29:00,861 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:00,861 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:00,861 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:29:00,861 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:00,861 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:00,861 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:29:00,861 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping vectorization 2018-01-05T11:29:00,861 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:29:00,861 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: llap mode: only 2018-01-05T11:29:00,861 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:29:00,861 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:00,861 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@7a3d444d can be done in LLAP 2018-01-05T11:29:00,861 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,861 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed plan generation 2018-01-05T11:29:00,861 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:00,861 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validation start 2018-01-05T11:29:00,861 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Not a partition. 2018-01-05T11:29:00,861 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,861 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:part.p_partkey, type:int, comment:null), FieldSchema(name:part.p_name, type:string, comment:null), FieldSchema(name:part.p_mfgr, type:string, comment:null), FieldSchema(name:part.p_brand, type:string, comment:null), FieldSchema(name:part.p_type, type:string, comment:null), FieldSchema(name:part.p_size, type:int, comment:null), FieldSchema(name:part.p_container, type:string, comment:null), FieldSchema(name:part.p_retailprice, type:double, comment:null), FieldSchema(name:part.p_comment, type:string, comment:null)], properties:null) 2018-01-05T11:29:00,861 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,861 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:00,861 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=5, flushCache_()=1, getTableColumnStatistics_(String, String, List, )=6} 2018-01-05T11:29:00,861 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112900_08173570-4964-412e-a3cb-561353455cac); Time taken: 0.026 seconds 2018-01-05T11:29:00,861 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:00,861 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,861 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112900_08173570-4964-412e-a3cb-561353455cac): analyze table part compute statistics 2018-01-05T11:29:00,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,862 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: analyze table part compute statistics 2018-01-05T11:29:00,862 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:29:00,862 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@part 2018-01-05T11:29:00,862 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@part 2018-01-05T11:29:00,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,862 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Query ID = djaiswal_20180105112900_08173570-4964-412e-a3cb-561353455cac 2018-01-05T11:29:00,862 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Total jobs = 1 2018-01-05T11:29:00,862 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,862 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:29:00,863 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MAPRED] in serial mode 2018-01-05T11:29:00,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_835_5577510622354784072-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:29:00,893 DEBUG [HikariPool-2 housekeeper] pool.HikariPool: HikariPool-2 - Pool stats (total=10, active=0, idle=10, waiting=0) 2018-01-05T11:29:00,894 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:29:00,894 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:29:00,894 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:29:00,894 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112900_08173570-4964-412e-a3cb-561353455cac 2018-01-05T11:29:00,894 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Session is already open 2018-01-05T11:29:00,894 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:29:00,894 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,894 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Dag name: analyze table part compute statistics (Stage-0) 2018-01-05T11:29:00,894 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: DagInfo: {"context":"Hive","description":"\nanalyze table part compute statistics"} 2018-01-05T11:29:00,894 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112900_08173570-4964-412e-a3cb-561353455cac with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:29:00,894 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,906 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:29:00,906 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,906 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:29:00,907 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,907 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 1.99KB 2018-01-05T11:29:00,912 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_835_5577510622354784072-1/-mr-10000 2018-01-05T11:29:00,923 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_835_5577510622354784072-1/-mr-10000 2018-01-05T11:29:00,926 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part 2018-01-05T11:29:00,926 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,926 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,926 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,926 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=analyze table part compute statistics (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112900_08173570-4964-412e-a3cb-561353455cac } 2018-01-05T11:29:00,926 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:00,927 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: analyze table part compute statistics (Stage-0) 2018-01-05T11:29:00,927 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:29:00,928 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_30, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_30.dot 2018-01-05T11:29:00,928 INFO [IPC Server handler 0 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_30 2018-01-05T11:29:00,928 WARN [IPC Server handler 0 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:29:00,928 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Running DAG: analyze table part compute statistics (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112900_08173570-4964-412e-a3cb-561353455cac } 2018-01-05T11:29:00,952 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_30][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_30, submitTime=1515180540927, queueName=null 2018-01-05T11:29:00,953 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_30_00 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_30_00 [Map 1], Default Resources= 2018-01-05T11:29:00,953 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_30_00 [Map 1] 2018-01-05T11:29:00,953 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_30_00 [Map 1] 2018-01-05T11:29:00,953 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:29:00,953 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_30_00 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:00,953 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:29:00,953 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_30][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_30, initTime=1515180540953 2018-01-05T11:29:00,953 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: dag_1515180518813_0001_30 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:29:00,954 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:29:00,954 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000060, QueryId=djaiswal_20180105112900_27c20ecb-12b4-4a83-b82f-be6498310e11-29, DagId=29, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180540748, EndTime=1515180540954 2018-01-05T11:29:00,954 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_30][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_30, startTime=1515180540954 2018-01-05T11:29:00,954 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_30 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:29:00,954 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {part={InputName=part}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:29:00,954 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: part, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:29:00,954 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_30_00 [Map 1] 2018-01-05T11:29:00,954 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_30, dagName=analyze table part compute statistics (Stage-0) 2018-01-05T11:29:00,954 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,954 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,954 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:00,958 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_30_00 [Map 1] 2018-01-05T11:29:00,958 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_30_00 [Map 1] 2018-01-05T11:29:00,958 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:00,958 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_30_00 [Map 1] 2018-01-05T11:29:00,960 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:29:00,960 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@446d15b9 2018-01-05T11:29:00,960 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_835_5577510622354784072-1/djaiswal/_tez_scratch_dir/4fbd7048-0677-4a7c-a52c-1c5a9bbcae91/map.xml 2018-01-05T11:29:00,960 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_835_5577510622354784072-1/djaiswal/_tez_scratch_dir/4fbd7048-0677-4a7c-a52c-1c5a9bbcae91/map.xml 2018-01-05T11:29:00,960 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_835_5577510622354784072-1/djaiswal/_tez_scratch_dir/4fbd7048-0677-4a7c-a52c-1c5a9bbcae91/map.xml 2018-01-05T11:29:00,960 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:00,960 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:00,960 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,961 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:00,961 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 1.99KB 2018-01-05T11:29:00,961 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_30_00 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:29:00,961 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: part on vertex vertex_1515180518813_0001_30_00 [Map 1] 2018-01-05T11:29:00,961 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:29:00,961 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:00,961 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:29:00,961 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:00,961 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_835_5577510622354784072-1/djaiswal/_tez_scratch_dir/4fbd7048-0677-4a7c-a52c-1c5a9bbcae91/map.xml 2018-01-05T11:29:00,961 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:00,961 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias part 2018-01-05T11:29:00,961 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part 2018-01-05T11:29:00,961 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part 2018-01-05T11:29:00,962 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 2018-01-05T11:29:00,962 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = 2018-01-05T11:29:00,962 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part 2018-01-05T11:29:00,962 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:00,962 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:00,962 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:00,962 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:00,976 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Time taken to get FileStatuses: 13 2018-01-05T11:29:00,976 INFO [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:29:00,976 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 14 2018-01-05T11:29:00,976 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 1 2018-01-05T11:29:00,976 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:00,976 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 1. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:29:00,976 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_835_5577510622354784072-1/djaiswal/_tez_scratch_dir/4fbd7048-0677-4a7c-a52c-1c5a9bbcae91/map.xml 2018-01-05T11:29:00,976 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_835_5577510622354784072-1/djaiswal/_tez_scratch_dir/4fbd7048-0677-4a7c-a52c-1c5a9bbcae91/map.xml 2018-01-05T11:29:00,976 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_835_5577510622354784072-1/djaiswal/_tez_scratch_dir/4fbd7048-0677-4a7c-a52c-1c5a9bbcae91/map.xml 2018-01-05T11:29:00,976 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:00,976 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:00,976 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:00,977 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:00,977 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 1.99KB 2018-01-05T11:29:00,977 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part/part_tiny.txt to src new group? true 2018-01-05T11:29:00,977 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:29:00,977 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:29:00,977 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:29:00,977 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 58 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 3173 Original splits: 1 2018-01-05T11:29:00,977 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Using original number of splits: 1 desired splits: 1 2018-01-05T11:29:00,977 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 1 grouped split count is 1, for bucket: 1 2018-01-05T11:29:00,977 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:29:00,977 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: part on vertex vertex_1515180518813_0001_30_00 [Map 1] 2018-01-05T11:29:00,977 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_30_00 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:00,977 INFO [App Shared Pool - #0] impl.VertexImpl: Got updated RootInputsSpecs: {part=forAllWorkUnits=true, update=[1]} 2018-01-05T11:29:00,978 INFO [App Shared Pool - #0] impl.VertexImpl: Vertex vertex_1515180518813_0001_30_00 [Map 1] parallelism set to 1 2018-01-05T11:29:00,978 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_30_00 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:00,978 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_30_00 [Map 1], numAdditionalOutputs=1 2018-01-05T11:29:00,978 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_30][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_30_00, initRequestedTime=1515180540954, initedTime=1515180540978, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:00,978 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_30_00 [Map 1] with distanceFromRoot: 0 2018-01-05T11:29:00,978 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_30_00 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:29:00,978 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_30][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_30_00, startRequestedTime=1515180540961, startedTime=1515180540978 2018-01-05T11:29:00,978 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_30][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_30_00, reconfigureDoneTime=1515180540978, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:29:00,978 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_30_00 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:00,978 INFO [App Shared Pool - #0] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:29:00,978 INFO [App Shared Pool - #0] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:29:00,978 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_30_00 [Map 1] 2018-01-05T11:29:00,978 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_30][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_30_00_000000, scheduledTime=1515180540978, launchTime=1515180540978 2018-01-05T11:29:00,978 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_30_00 [Map 1] 2018-01-05T11:29:00,978 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_30_00_000000_0, priority=2, capability=, hosts=null 2018-01-05T11:29:00,978 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=2,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:00,978 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=2,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:00,978 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:00,979 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_30_00_000000_0 on hosts=null 2018-01-05T11:29:00,979 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_30_00_000000_0 on hosts=null 2018-01-05T11:29:00,979 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_30_00_000000_0 2018-01-05T11:29:00,979 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:00,979 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #61, task=TaskInfo{task=attempt_1515180518813_0001_30_00_000000_0, priority=2, startTime=0, containerId=null, uniqueId=60, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000061 2018-01-05T11:29:00,979 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_30_00_000000_0; false 2018-01-05T11:29:00,979 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_30_00_000000_0, priority=2, startTime=311561488, containerId=container_111101111_0001_00_000061assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=60, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:00,979 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000061 for node: 10.22.8.182:49818 2018-01-05T11:29:00,981 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 30, name=analyze table part compute statistics (Stage-0), queryId=djaiswal_20180105112900_08173570-4964-412e-a3cb-561353455cac 2018-01-05T11:29:00,981 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000061, attempt_1515180518813_0001_30_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:00,981 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000061 for node: 10.22.8.182:49818 2018-01-05T11:29:00,981 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000061 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:00,981 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000061, launchTime=1515180540981 2018-01-05T11:29:00,982 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_30_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112900_08173570-4964-412e-a3cb-561353455cac, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000061, dagName=analyze table part compute statistics (Stage-0), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{part,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{out_Map 1,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180540954, firstAttemptStartTime=1515180540978, currentAttemptStartTime=1515180540981} 2018-01-05T11:29:00,982 INFO [IPC Server handler 1 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000061, QueryId=djaiswal_20180105112900_08173570-4964-412e-a3cb-561353455cac-30, DagId=30, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180540982 2018-01-05T11:29:00,982 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_30_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000061] on NM: [10.22.8.182:49818] 2018-01-05T11:29:00,982 DEBUG [IPC Server handler 1 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=30} with the ShuffleHandler 2018-01-05T11:29:00,982 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:00,982 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:00,982 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_30][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_30_00_000000_0, startTime=1515180540982, containerId=container_111101111_0001_00_000061, nodeId=10.22.8.182:49818 2018-01-05T11:29:00,994 INFO [IPC Server handler 1 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:29:00,994 DEBUG [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/30] 2018-01-05T11:29:00,994 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:29:00,994 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_30_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:00,994 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_30_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180540978, dagStartTime=1515180540954, withinDagPriority=2, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:29:00,994 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_30_00_000000_0 : ACCEPTED 2018-01-05T11:29:00,994 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_30_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:29:00,994 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_30_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180540978, dagStartTime=1515180540954, withinDagPriority=2, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:29:00,994 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_30_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:00,994 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_30_00_000000_0 to pre-emption queue 2018-01-05T11:29:00,994 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_30_00_000000_0: true 2018-01-05T11:29:00,995 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:460446173 2018-01-05T11:29:00,995 INFO [TaskCommunicator # 7] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_30_00_000000_0 2018-01-05T11:29:00,995 INFO [TaskCommunicator # 7] GuaranteedTasks: Received response for attempt_1515180518813_0001_30_00_000000_0, true 2018-01-05T11:29:00,995 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : analyze table part compute statistics (Stage-0), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_30_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=part, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=out_Map 1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:29:00,996 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1038614528, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:00,996 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_30_00_000000_0 vertexName: Map 1 2018-01-05T11:29:00,996 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000061, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_30_00_000000_0, eventCount=1 } 2018-01-05T11:29:00,996 DEBUG [TezTR-518813_1_30_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_30 queryId: djaiswal_20180105112900_08173570-4964-412e-a3cb-561353455cac instanceType: TaskRunner2Callable 2018-01-05T11:29:00,996 INFO [TezTR-518813_1_30_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_30_00_000000_0 2018-01-05T11:29:00,996 DEBUG [TezTR-518813_1_30_0_0_0] log.PerfLogger: 2018-01-05T11:29:00,997 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:00,997 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:00,998 INFO [IPC Server handler 24 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_30_00_000000_0 sent events: (0-1). 2018-01-05T11:29:00,998 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:29:00,998 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_30_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:29:00,999 INFO [I/O Setup 0 Initialize: {part}] input.MRInput: part using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:29:00,999 INFO [I/O Setup 0 Initialize: {part}] input.MRInputLegacy: part MRInputLegacy deferring initialization 2018-01-05T11:29:00,999 DEBUG [TezTR-518813_1_30_0_0_0] log.PerfLogger: 2018-01-05T11:29:00,999 INFO [TezTR-518813_1_30_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:00,999 INFO [I/O Setup 1 Initialize: {out_Map 1}] output.MROutput: out_Map 1: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:29:00,999 INFO [TezTR-518813_1_30_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:00,999 INFO [TezTR-518813_1_30_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:00,999 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:00,999 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:00,999 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:01,000 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:01,000 INFO [TezTR-518813_1_30_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:01,000 INFO [TezTR-518813_1_30_0_0_0] resources.MemoryDistributor: InitialRequests=[part:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [out_Map 1:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput] 2018-01-05T11:29:01,000 INFO [TezTR-518813_1_30_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:01,000 INFO [TezTR-518813_1_30_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:01,000 INFO [TezTR-518813_1_30_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 1038614528, TotalAvailable: 695871733, TotalRequested/TotalJVMHeap:0.00 2018-01-05T11:29:01,000 INFO [TezTR-518813_1_30_0_0_0] resources.MemoryDistributor: Allocations=[part:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [out_Map 1:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0] 2018-01-05T11:29:01,000 INFO [TezTR-518813_1_30_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:01,000 INFO [TezTR-518813_1_30_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: part being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:01,000 INFO [TezTR-518813_1_30_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:29:01,000 INFO [TezTR-518813_1_30_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:01,000 INFO [I/O Setup 0 Start: {part}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: part 2018-01-05T11:29:01,000 INFO [TezTR-518813_1_30_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:01,000 INFO [TezTR-518813_1_30_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_30_00_000000_0 2018-01-05T11:29:01,000 DEBUG [TezTR-518813_1_30_0_0_0] log.PerfLogger: 2018-01-05T11:29:01,000 DEBUG [TezTR-518813_1_30_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_30_00_000000_0_10181 2018-01-05T11:29:01,000 DEBUG [TezTR-518813_1_30_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:01,000 DEBUG [TezTR-518813_1_30_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_30_00_000000_0 2018-01-05T11:29:01,000 INFO [TezTR-518813_1_30_0_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112900_08173570-4964-412e-a3cb-561353455cac 2018-01-05T11:29:01,000 DEBUG [TezTR-518813_1_30_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:01,000 DEBUG [TezTR-518813_1_30_0_0_0] log.PerfLogger: 2018-01-05T11:29:01,000 DEBUG [TezTR-518813_1_30_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:01,000 DEBUG [TezTR-518813_1_30_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:01,000 DEBUG [TezTR-518813_1_30_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112900_08173570-4964-412e-a3cb-561353455cac_Map 1__MAP_PLAN__ 2018-01-05T11:29:01,000 INFO [TezTR-518813_1_30_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_835_5577510622354784072-1/djaiswal/_tez_scratch_dir/4fbd7048-0677-4a7c-a52c-1c5a9bbcae91/map.xml 2018-01-05T11:29:01,000 DEBUG [TezTR-518813_1_30_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_835_5577510622354784072-1/djaiswal/_tez_scratch_dir/4fbd7048-0677-4a7c-a52c-1c5a9bbcae91/map.xml 2018-01-05T11:29:01,000 DEBUG [TezTR-518813_1_30_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_835_5577510622354784072-1/djaiswal/_tez_scratch_dir/4fbd7048-0677-4a7c-a52c-1c5a9bbcae91/map.xml 2018-01-05T11:29:01,000 DEBUG [TezTR-518813_1_30_0_0_0] log.PerfLogger: 2018-01-05T11:29:01,000 INFO [TezTR-518813_1_30_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:01,000 DEBUG [TezTR-518813_1_30_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,001 DEBUG [TezTR-518813_1_30_0_0_0] log.PerfLogger: 2018-01-05T11:29:01,001 INFO [TezTR-518813_1_30_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 1.99KB 2018-01-05T11:29:01,001 DEBUG [TezTR-518813_1_30_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:01,001 INFO [TezTR-518813_1_30_0_0_0] tez.MapRecordProcessor: The input names are: [part] 2018-01-05T11:29:01,001 INFO [TezTR-518813_1_30_0_0_0] input.MRInput: Initialized MRInput: part 2018-01-05T11:29:01,001 DEBUG [TezTR-518813_1_30_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 1 2018-01-05T11:29:01,001 INFO [TezTR-518813_1_30_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_835_5577510622354784072-1/djaiswal/_tez_scratch_dir/4fbd7048-0677-4a7c-a52c-1c5a9bbcae91/map.xml 2018-01-05T11:29:01,001 DEBUG [TezTR-518813_1_30_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:01,002 DEBUG [TezTR-518813_1_30_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part/part_tiny.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.part, numFiles=1, columns.types=int:string:string:string:string:int:string:double:string, serialization.format=1, serialization.ddl=struct part { i32 p_partkey, string p_name, string p_mfgr, string p_brand, string p_type, i32 p_size, string p_container, double p_retailprice, string p_comment}, columns=p_partkey,p_name,p_mfgr,p_brand,p_type,p_size,p_container,p_retailprice,p_comment, columns.comments=, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, totalSize=3173, column.name.delimiter=,, transient_lastDdlTime=1515180537}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.part, numFiles=1, columns.types=int:string:string:string:string:int:string:double:string, columns=p_partkey,p_name,p_mfgr,p_brand,p_type,p_size,p_container,p_retailprice,p_comment, serialization.format=1, serialization.ddl=struct part { i32 p_partkey, string p_name, string p_mfgr, string p_brand, string p_type, i32 p_size, string p_container, double p_retailprice, string p_comment}, columns.comments=, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, totalSize=3173, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part, column.name.delimiter=,, transient_lastDdlTime=1515180537}, baseFileName=part, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.part, numFiles=1, columns.types=int:string:string:string:string:int:string:double:string, serialization.format=1, serialization.ddl=struct part { i32 p_partkey, string p_name, string p_mfgr, string p_brand, string p_type, i32 p_size, string p_container, double p_retailprice, string p_comment}, columns=p_partkey,p_name,p_mfgr,p_brand,p_type,p_size,p_container,p_retailprice,p_comment, columns.comments=, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, totalSize=3173, column.name.delimiter=,, transient_lastDdlTime=1515180537}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.part, numFiles=1, columns.types=int:string:string:string:string:int:string:double:string, columns=p_partkey,p_name,p_mfgr,p_brand,p_type,p_size,p_container,p_retailprice,p_comment, serialization.format=1, serialization.ddl=struct part { i32 p_partkey, string p_name, string p_mfgr, string p_brand, string p_type, i32 p_size, string p_container, double p_retailprice, string p_comment}, columns.comments=, rawDataSize=0, numRows=0, bucket_count=-1, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, totalSize=3173, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part, column.name.delimiter=,, transient_lastDdlTime=1515180537}, baseFileName=part, vectorPartitionDesc=null]} 2018-01-05T11:29:01,002 DEBUG [TezTR-518813_1_30_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:29:01,002 INFO [TezTR-518813_1_30_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:29:01,002 DEBUG [TezTR-518813_1_30_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:29:01,003 DEBUG [TezTR-518813_1_30_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:29:01,003 INFO [TezTR-518813_1_30_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_835_5577510622354784072-1/djaiswal/_tez_scratch_dir/4fbd7048-0677-4a7c-a52c-1c5a9bbcae91/map.xml 2018-01-05T11:29:01,003 DEBUG [TezTR-518813_1_30_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:01,003 INFO [TezTR-518813_1_30_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_30_00_000000_0 2018-01-05T11:29:01,003 INFO [TezTR-518813_1_30_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [p_partkey, p_name, p_mfgr, p_brand, p_type, p_size, p_container, p_retailprice, p_comment] / schema.evolution.columns.types [int, string, string, string, string, int, string, double, string] (isAcidRead false) 2018-01-05T11:29:01,003 WARN [TezTR-518813_1_30_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part/part_tiny.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:29:01,004 DEBUG [TezTR-518813_1_30_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:29:01,004 WARN [TezTR-518813_1_30_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part/part_tiny.txt:0+3173 2018-01-05T11:29:01,004 DEBUG [TezTR-518813_1_30_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part/part_tiny.txt 2018-01-05T11:29:01,004 INFO [TezTR-518813_1_30_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part/part_tiny.txt:0+3173], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=3173} 2018-01-05T11:29:01,004 INFO [TezTR-518813_1_30_0_0_0] input.MRInput: part initialized RecordReader from event 2018-01-05T11:29:01,004 DEBUG [TezTR-518813_1_30_0_0_0] tez.MapRecordProcessor: Starting Output: out_Map 1 2018-01-05T11:29:01,004 INFO [TezTR-518813_1_30_0_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:29:01,004 INFO [TezTR-518813_1_30_0_0_0] exec.MapOperator: Initializing operator MAP[0] 2018-01-05T11:29:01,004 DEBUG [TezTR-518813_1_30_0_0_0] exec.MapOperator: Initialization Done 0 MAP 2018-01-05T11:29:01,004 DEBUG [TezTR-518813_1_30_0_0_0] exec.MapOperator: Operator 0 MAP initialized 2018-01-05T11:29:01,004 DEBUG [TezTR-518813_1_30_0_0_0] exec.MapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:29:01,004 DEBUG [TezTR-518813_1_30_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,004 DEBUG [TezTR-518813_1_30_0_0_0] exec.MapOperator: Adding alias part to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part 2018-01-05T11:29:01,004 DEBUG [TezTR-518813_1_30_0_0_0] exec.MapOperator: dump TS[0] struct 2018-01-05T11:29:01,004 INFO [TezTR-518813_1_30_0_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:29:01,004 INFO [TezTR-518813_1_30_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:01,004 DEBUG [TezTR-518813_1_30_0_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:29:01,004 INFO [TezTR-518813_1_30_0_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:29:01,004 DEBUG [TezTR-518813_1_30_0_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:29:01,004 DEBUG [TezTR-518813_1_30_0_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:29:01,004 DEBUG [TezTR-518813_1_30_0_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:29:01,004 DEBUG [TezTR-518813_1_30_0_0_0] log.PerfLogger: 2018-01-05T11:29:01,004 INFO [TezTR-518813_1_30_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_835_5577510622354784072-1/djaiswal/_tez_scratch_dir/4fbd7048-0677-4a7c-a52c-1c5a9bbcae91/map.xml 2018-01-05T11:29:01,004 DEBUG [TezTR-518813_1_30_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:01,005 DEBUG [TezTR-518813_1_30_0_0_0] exec.MapOperator: Processing alias(es) part for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part/part_tiny.txt 2018-01-05T11:29:01,005 INFO [TezTR-518813_1_30_0_0_0] exec.MapOperator: MAP[0]: records read - 1 2018-01-05T11:29:01,005 INFO [TezTR-518813_1_30_0_0_0] exec.MapOperator: MAP[0]: records read - 10 2018-01-05T11:29:01,005 DEBUG [TezTR-518813_1_30_0_0_0] log.PerfLogger: 2018-01-05T11:29:01,005 DEBUG [TezTR-518813_1_30_0_0_0] mr.ObjectCache: djaiswal_20180105112900_08173570-4964-412e-a3cb-561353455cac_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:29:01,005 DEBUG [TezTR-518813_1_30_0_0_0] exec.MapOperator: close called for operator MAP[0] 2018-01-05T11:29:01,005 INFO [TezTR-518813_1_30_0_0_0] exec.MapOperator: Closing operator MAP[0] 2018-01-05T11:29:01,005 INFO [TezTR-518813_1_30_0_0_0] exec.MapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:26, 2018-01-05T11:29:01,005 DEBUG [TezTR-518813_1_30_0_0_0] exec.MapOperator: Closing child = TS[0] 2018-01-05T11:29:01,005 DEBUG [TezTR-518813_1_30_0_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:29:01,005 DEBUG [TezTR-518813_1_30_0_0_0] exec.TableScanOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:01,005 INFO [TezTR-518813_1_30_0_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:29:01,005 DEBUG [TezTR-518813_1_30_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,005 DEBUG [TezTR-518813_1_30_0_0_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_835_5577510622354784072-1/-mr-10000 2018-01-05T11:29:01,005 INFO [TezTR-518813_1_30_0_0_0] exec.TableScanOperator: publishing : default.part/ : {numRows=26, rawDataSize=3147} 2018-01-05T11:29:01,028 DEBUG [TezTR-518813_1_30_0_0_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_835_5577510622354784072-1/-mr-10000/tmpstats-0_0 2018-01-05T11:29:01,028 DEBUG [TezTR-518813_1_30_0_0_0] fs.FSStatsPublisher: Writing stats in it : {default.part/={numRows=26, rawDataSize=3147}} 2018-01-05T11:29:01,029 DEBUG [TezTR-518813_1_30_0_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:29:01,029 DEBUG [TezTR-518813_1_30_0_0_0] exec.MapOperator: 0 Close done 2018-01-05T11:29:01,029 INFO [TezTR-518813_1_30_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_30_00_000000_0 2018-01-05T11:29:01,029 INFO [TezTR-518813_1_30_0_0_0] output.MROutput: out_Map 1 closed 2018-01-05T11:29:01,029 DEBUG [TezTR-518813_1_30_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:01,029 INFO [TezTR-518813_1_30_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:01,029 INFO [TezTR-518813_1_30_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_30_00_000000_0, askedToStop=false 2018-01-05T11:29:01,029 INFO [TezTR-518813_1_30_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_30_00_000000_0, stopRequested=false 2018-01-05T11:29:01,029 INFO [TezTR-518813_1_30_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_30_00_000000_0: Counters: 5 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=26, INPUT_SPLIT_LENGTH_BYTES=3173, OUTPUT_RECORDS=0][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=26]] 2018-01-05T11:29:01,029 DEBUG [TezTR-518813_1_30_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_30_0_0_0 thread id: 1892 scheme: file bytesRead: 3213 bytesWritten: 82 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:01,029 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_30_00_000000_0, isTaskDone=true 2018-01-05T11:29:01,029 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000061, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_30_00_000000_0, eventCount=2 } 2018-01-05T11:29:01,030 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_30][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_30_00_000000_0, creationTime=1515180540978, allocationTime=1515180540979, startTime=1515180540982, finishTime=1515180541030, timeTaken=48, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:01,030 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:29:01,030 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_30_00_000000_0 2018-01-05T11:29:01,030 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_30_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:01,030 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_30_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:01,030 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_30_00_000000_0 2018-01-05T11:29:01,030 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_30_00_000000_0; false 2018-01-05T11:29:01,030 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000061=35 2018-01-05T11:29:01,030 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_30_00_000000_0: true 2018-01-05T11:29:01,030 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_30][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_30_00_000000, startTime=1515180540982, finishTime=1515180541030, timeTaken=48, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_30_00_000000_0, diagnostics=, counters=Counters: 11, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=3213, FILE_BYTES_WRITTEN=82, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=26, INPUT_SPLIT_LENGTH_BYTES=3173, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=26 2018-01-05T11:29:01,030 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_30_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:01,030 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_30_00_000000_0 2018-01-05T11:29:01,030 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:01,030 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:01,030 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000061, QueryId=djaiswal_20180105112900_08173570-4964-412e-a3cb-561353455cac-30, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:01,030 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_30_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:29:01,030 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_30_00_000000_0 2018-01-05T11:29:01,030 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_30_00 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:01,030 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_30_00 [Map 1] 2018-01-05T11:29:01,031 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_30][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_30_00, initRequestedTime=1515180540954, initedTime=1515180540978, startRequestedTime=1515180540961, startedTime=1515180540978, finishTime=1515180541030, timeTaken=52, status=SUCCEEDED, diagnostics=, counters=Counters: 11, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=3213, FILE_BYTES_WRITTEN=82, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=26, INPUT_SPLIT_LENGTH_BYTES=3173, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=26, vertexStats=firstTaskStartTime=1515180540982, firstTasksToStart=[ task_1515180518813_0001_30_00_000000 ], lastTaskFinishTime=1515180541030, lastTasksToFinish=[ task_1515180518813_0001_30_00_000000 ], minTaskDuration=48, maxTaskDuration=48, avgTaskDuration=48.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_30_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_30_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:01,031 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_30_00 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:01,031 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_30][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000061, stoppedTime=1515180541031, exitStatus=0 2018-01-05T11:29:01,031 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000061 exited with diagnostics set to Container completed. 2018-01-05T11:29:01,031 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_30_00 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1 2018-01-05T11:29:01,031 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null 2018-01-05T11:29:01,031 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_30_00 [Map 1] 2018-01-05T11:29:01,031 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_30, queueSize=0 2018-01-05T11:29:01,032 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_30][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_30, startTime=1515180540954, finishTime=1515180541031, timeTaken=77, status=SUCCEEDED, diagnostics=, counters=Counters: 15, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, OTHER_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=101, AM_GC_TIME_MILLIS=0, File System Counters, FILE_BYTES_READ=3213, FILE_BYTES_WRITTEN=82, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=26, INPUT_SPLIT_LENGTH_BYTES=3173, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=26 2018-01-05T11:29:01,032 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_30 finished with state: SUCCEEDED 2018-01-05T11:29:01,032 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_30 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:29:01,032 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000061 2018-01-05T11:29:01,032 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_30, dagState=SUCCEEDED 2018-01-05T11:29:01,032 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_30_post 2018-01-05T11:29:01,032 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:29:01,032 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:29:01,032 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:29:01,032 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 30 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=1, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=1,NumRequestsWithoutLocation=1, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=1} 2018-01-05T11:29:01,032 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:29:01,032 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=analyze table part compute statistics (Stage-0), with id=dag_1515180518813_0001_30 2018-01-05T11:29:01,032 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 30, to 10.22.8.182:49818 2018-01-05T11:29:01,032 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=1, Remaining=0 2018-01-05T11:29:01,032 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=analyze table part compute statistics (Stage-0), with id=dag_1515180518813_0001_30 2018-01-05T11:29:01,032 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=30} 2018-01-05T11:29:01,032 INFO [IPC Server handler 0 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=30}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:29:01,032 INFO [IPC Server handler 0 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/30 after 300 seconds 2018-01-05T11:29:01,032 ERROR [IPC Server handler 0 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112900_08173570-4964-412e-a3cb-561353455cac, dag_1515180518813_0001_30 2018-01-05T11:29:01,032 INFO [IPC Server handler 0 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112900_08173570-4964-412e-a3cb-561353455cac 2018-01-05T11:29:01,032 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=30} = 0 2018-01-05T11:29:01,032 DEBUG [IPC Server handler 0 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=30} 2018-01-05T11:29:01,032 DEBUG [IPC Server handler 0 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:29:01,032 DEBUG [IPC Server handler 0 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=30} 2018-01-05T11:29:01,032 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,032 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,032 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 2018-01-05T11:29:01,032 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,033 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-2:STATS] in serial mode 2018-01-05T11:29:01,033 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@1c31eec5, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:29:01,033 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:01,033 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,033 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:29:01,033 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:29:01,033 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:01,033 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:01,033 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,033 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:01,033 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:29:01,033 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_835_5577510622354784072-1/-mr-10000 2018-01-05T11:29:01,033 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_835_5577510622354784072-1/-mr-10000 2018-01-05T11:29:01,034 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.part/ numRows 2018-01-05T11:29:01,034 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.part/ numRows 26 2018-01-05T11:29:01,034 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.part/ rawDataSize 2018-01-05T11:29:01,034 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.part/ rawDataSize 3147 2018-01-05T11:29:01,035 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,035 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:29:01,035 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:29:01,035 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:29:01,035 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:29:01,035 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:01,035 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:01,035 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:01,036 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:01,036 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:01,036 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:29:01,036 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:29:01,036 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,036 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:29:01,036 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,036 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=part newtbl=part 2018-01-05T11:29:01,036 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=part newtbl=part 2018-01-05T11:29:01,039 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:01,043 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:01,044 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,044 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.part stats: [numFiles=1, numRows=26, totalSize=3173, rawDataSize=3147] 2018-01-05T11:29:01,044 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: About to delete stats tmp dir :file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_835_5577510622354784072-1/-mr-10000 2018-01-05T11:29:01,044 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,044 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,044 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: analyze table part compute statistics 2018-01-05T11:29:01,044 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:29:01,044 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@part 2018-01-05T11:29:01,044 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@part 2018-01-05T11:29:01,044 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,044 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,044 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:01,044 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=8} 2018-01-05T11:29:01,044 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112900_08173570-4964-412e-a3cb-561353455cac); Time taken: 0.183 seconds 2018-01-05T11:29:01,044 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:29:01,044 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,044 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:01,044 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,044 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query analyze table part compute statistics 2018-01-05T11:29:01,045 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-00_835_5577510622354784072-1 2018-01-05T11:29:01,045 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.21 seconds 2018-01-05T11:29:01,045 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:29:01,045 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:01,045 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:29:01,045 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:29:01,045 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,045 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:01,045 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,045 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,045 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,045 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: analyze table part compute statistics for columns 2018-01-05T11:29:01,045 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112901_10c480d0-6dc9-477b-879b-acf80ce89912): analyze table part compute statistics for columns 2018-01-05T11:29:01,045 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,046 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: analyze table part compute statistics for columns 2018-01-05T11:29:01,046 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:01,046 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,046 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,046 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,046 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,046 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:01,046 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,046 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=part 2018-01-05T11:29:01,046 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=part 2018-01-05T11:29:01,048 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:01,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: select compute_stats(`p_partkey`, 'hll') , compute_stats(`p_name`, 'hll') , compute_stats(`p_mfgr`, 'hll') , compute_stats(`p_brand`, 'hll') , compute_stats(`p_type`, 'hll') , compute_stats(`p_size`, 'hll') , compute_stats(`p_container`, 'hll') , compute_stats(`p_retailprice`, 'hll') , compute_stats(`p_comment`, 'hll') from `default`.`part` 2018-01-05T11:29:01,049 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: select compute_stats(`p_partkey`, 'hll') , compute_stats(`p_name`, 'hll') , compute_stats(`p_mfgr`, 'hll') , compute_stats(`p_brand`, 'hll') , compute_stats(`p_type`, 'hll') , compute_stats(`p_size`, 'hll') , compute_stats(`p_container`, 'hll') , compute_stats(`p_retailprice`, 'hll') , compute_stats(`p_comment`, 'hll') from `default`.`part` 2018-01-05T11:29:01,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:01,050 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Invoking analyze on rewritten query 2018-01-05T11:29:01,050 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Starting Semantic Analysis 2018-01-05T11:29:01,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,050 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed phase 1 of Semantic Analysis 2018-01-05T11:29:01,050 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for source tables 2018-01-05T11:29:01,050 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,050 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=part 2018-01-05T11:29:01,050 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=part 2018-01-05T11:29:01,053 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:01,053 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,053 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for subqueries 2018-01-05T11:29:01,053 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for destination tables 2018-01-05T11:29:01,064 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed getting MetaData in Semantic Analysis 2018-01-05T11:29:01,064 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_049_1860637755816674007-1/-mr-10001 2018-01-05T11:29:01,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Table Plan for part TS[0] 2018-01-05T11:29:01,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,065 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,066 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,066 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,066 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: RR before GB part{(p_partkey,p_partkey: int)(p_name,p_name: string)(p_mfgr,p_mfgr: string)(p_brand,p_brand: string)(p_type,p_type: string)(p_size,p_size: int)(p_container,p_container: string)(p_retailprice,p_retailprice: double)(p_comment,p_comment: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)(RAW__DATA__SIZE,RAW__DATA__SIZE: bigint)} after GB {((tok_function compute_stats (tok_table_or_col p_partkey) 'hll'),_col0: struct)((tok_function compute_stats (tok_table_or_col p_name) 'hll'),_col1: struct)((tok_function compute_stats (tok_table_or_col p_mfgr) 'hll'),_col2: struct)((tok_function compute_stats (tok_table_or_col p_brand) 'hll'),_col3: struct)((tok_function compute_stats (tok_table_or_col p_type) 'hll'),_col4: struct)((tok_function compute_stats (tok_table_or_col p_size) 'hll'),_col5: struct)((tok_function compute_stats (tok_table_or_col p_container) 'hll'),_col6: struct)((tok_function compute_stats (tok_table_or_col p_retailprice) 'hll'),_col7: struct)((tok_function compute_stats (tok_table_or_col p_comment) 'hll'),_col8: struct)} 2018-01-05T11:29:01,066 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: tree: (tok_select (tok_selexpr (tok_function compute_stats (tok_table_or_col p_partkey) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col p_name) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col p_mfgr) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col p_brand) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col p_type) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col p_size) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col p_container) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col p_retailprice) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col p_comment) 'hll'))) 2018-01-05T11:29:01,066 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: genSelectPlan: input = {((tok_function compute_stats (tok_table_or_col p_partkey) 'hll'),_col0: struct)((tok_function compute_stats (tok_table_or_col p_name) 'hll'),_col1: struct)((tok_function compute_stats (tok_table_or_col p_mfgr) 'hll'),_col2: struct)((tok_function compute_stats (tok_table_or_col p_brand) 'hll'),_col3: struct)((tok_function compute_stats (tok_table_or_col p_type) 'hll'),_col4: struct)((tok_function compute_stats (tok_table_or_col p_size) 'hll'),_col5: struct)((tok_function compute_stats (tok_table_or_col p_container) 'hll'),_col6: struct)((tok_function compute_stats (tok_table_or_col p_retailprice) 'hll'),_col7: struct)((tok_function compute_stats (tok_table_or_col p_comment) 'hll'),_col8: struct)} starRr = part{(p_partkey,p_partkey: int)(p_name,p_name: string)(p_mfgr,p_mfgr: string)(p_brand,p_brand: string)(p_type,p_type: string)(p_size,p_size: int)(p_container,p_container: string)(p_retailprice,p_retailprice: double)(p_comment,p_comment: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)(RAW__DATA__SIZE,RAW__DATA__SIZE: bigint)} 2018-01-05T11:29:01,066 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Select Plan row schema: null{(_c0,_col0: struct)(_c1,_col1: struct)(_c2,_col2: struct)(_c3,_col3: struct)(_c4,_col4: struct)(_c5,_col5: struct)(_c6,_col6: struct)(_c7,_col7: struct)(_c8,_col8: struct)} 2018-01-05T11:29:01,066 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Select Plan for clause: insclause-0 2018-01-05T11:29:01,066 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_049_1860637755816674007-1/-mr-10000/.hive-staging_hive_2018-01-05_11-29-01_049_1860637755816674007-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_049_1860637755816674007-1/-mr-10000 2018-01-05T11:29:01,066 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_049_1860637755816674007-1/-mr-10000/.hive-staging_hive_2018-01-05_11-29-01_049_1860637755816674007-1 2018-01-05T11:29:01,089 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_049_1860637755816674007-1/-mr-10003 2018-01-05T11:29:01,089 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_049_1860637755816674007-1/-mr-10000 row schema: null{(_c0,_col0: struct)(_c1,_col1: struct)(_c2,_col2: struct)(_c3,_col3: struct)(_c4,_col4: struct)(_c5,_col5: struct)(_c6,_col6: struct)(_c7,_col7: struct)(_c8,_col8: struct)} 2018-01-05T11:29:01,089 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Body Plan for Query Block null 2018-01-05T11:29:01,089 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Plan for Query Block null 2018-01-05T11:29:01,089 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Before logical optimization TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-SEL[5]-FS[6] 2018-01-05T11:29:01,089 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,089 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,089 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] lineage.Generator: Time taken for lineage transform=1 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(p_partkey: int|{part}p_partkey,p_name: string|{part}p_name,p_mfgr: string|{part}p_mfgr,p_brand: string|{part}p_brand,p_type: string|{part}p_type,p_size: int|{part}p_size,p_container: string|{part}p_container,p_retailprice: double|{part}p_retailprice,p_comment: string|{part}p_comment,BLOCK__OFFSET__INSIDE__FILE: bigint|{part}block__offset__inside__file,INPUT__FILE__NAME: string|{part}input__file__name,ROW__ID: struct|{part}row__id,RAW__DATA__SIZE: bigint|{part}raw__data__size) 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(p_partkey: int|{part}p_partkey,p_name: string|{part}p_name,p_mfgr: string|{part}p_mfgr,p_brand: string|{part}p_brand,p_type: string|{part}p_type,p_size: int|{part}p_size,p_container: string|{part}p_container,p_retailprice: double|{part}p_retailprice,p_comment: string|{part}p_comment,BLOCK__OFFSET__INSIDE__FILE: bigint|{part}block__offset__inside__file,INPUT__FILE__NAME: string|{part}input__file__name,ROW__ID: struct|{part}row__id,RAW__DATA__SIZE: bigint|{part}raw__data__size) 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[p_partkey] Column[p_name] Column[p_mfgr] Column[p_brand] Column[p_type] Column[p_size] Column[p_container] Column[p_retailprice] Column[p_comment] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col p_partkey) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col p_name) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col p_mfgr) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col p_brand) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col p_type) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col p_size) 'hll'),_col6: struct|{}(tok_function compute_stats (tok_table_or_col p_container) 'hll'),_col7: struct|{}(tok_function compute_stats (tok_table_or_col p_retailprice) 'hll'),_col8: struct|{}(tok_function compute_stats (tok_table_or_col p_comment) 'hll')) 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col p_partkey) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col p_name) 'hll'),VALUE._col2: struct|{}(tok_function compute_stats (tok_table_or_col p_mfgr) 'hll'),VALUE._col3: struct|{}(tok_function compute_stats (tok_table_or_col p_brand) 'hll'),VALUE._col4: struct|{}(tok_function compute_stats (tok_table_or_col p_type) 'hll'),VALUE._col5: struct|{}(tok_function compute_stats (tok_table_or_col p_size) 'hll'),VALUE._col6: struct|{}(tok_function compute_stats (tok_table_or_col p_container) 'hll'),VALUE._col7: struct|{}(tok_function compute_stats (tok_table_or_col p_retailprice) 'hll'),VALUE._col8: struct|{}(tok_function compute_stats (tok_table_or_col p_comment) 'hll')) 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col p_partkey) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col p_name) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col p_mfgr) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col p_brand) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col p_type) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col p_size) 'hll'),_col6: struct|{}(tok_function compute_stats (tok_table_or_col p_container) 'hll'),_col7: struct|{}(tok_function compute_stats (tok_table_or_col p_retailprice) 'hll'),_col8: struct|{}(tok_function compute_stats (tok_table_or_col p_comment) 'hll')) 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1,_col2: struct|{null}_c2,_col3: struct|{null}_c3,_col4: struct|{null}_c4,_col5: struct|{null}_c5,_col6: struct|{null}_c6,_col7: struct|{null}_c7,_col8: struct|{null}_c8) 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0] Column[_col1] Column[_col2] Column[_col3] Column[_col4] Column[_col5] Column[_col6] Column[_col7] Column[_col8]) 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{},_col2: struct|{},_col3: struct|{},_col4: struct|{},_col5: struct|{},_col6: struct|{},_col7: struct|{},_col8: struct|{}) 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,090 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for FS(6) 2018-01-05T11:29:01,090 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(5) 2018-01-05T11:29:01,090 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for GBY(4) 2018-01-05T11:29:01,090 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for RS(3) 2018-01-05T11:29:01,090 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for GBY(2) 2018-01-05T11:29:01,090 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(1) 2018-01-05T11:29:01,090 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.PredicatePushDown: After PPD: TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-SEL[5]-FS[6] 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(p_partkey: int|{part}p_partkey,p_name: string|{part}p_name,p_mfgr: string|{part}p_mfgr,p_brand: string|{part}p_brand,p_type: string|{part}p_type,p_size: int|{part}p_size,p_container: string|{part}p_container,p_retailprice: double|{part}p_retailprice,p_comment: string|{part}p_comment,BLOCK__OFFSET__INSIDE__FILE: bigint|{part}block__offset__inside__file,INPUT__FILE__NAME: string|{part}input__file__name,ROW__ID: struct|{part}row__id,RAW__DATA__SIZE: bigint|{part}raw__data__size) 2018-01-05T11:29:01,090 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(p_partkey: int|{part}p_partkey,p_name: string|{part}p_name,p_mfgr: string|{part}p_mfgr,p_brand: string|{part}p_brand,p_type: string|{part}p_type,p_size: int|{part}p_size,p_container: string|{part}p_container,p_retailprice: double|{part}p_retailprice,p_comment: string|{part}p_comment,BLOCK__OFFSET__INSIDE__FILE: bigint|{part}block__offset__inside__file,INPUT__FILE__NAME: string|{part}input__file__name,ROW__ID: struct|{part}row__id,RAW__DATA__SIZE: bigint|{part}raw__data__size) 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[p_partkey] Column[p_name] Column[p_mfgr] Column[p_brand] Column[p_type] Column[p_size] Column[p_container] Column[p_retailprice] Column[p_comment] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col p_partkey) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col p_name) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col p_mfgr) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col p_brand) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col p_type) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col p_size) 'hll'),_col6: struct|{}(tok_function compute_stats (tok_table_or_col p_container) 'hll'),_col7: struct|{}(tok_function compute_stats (tok_table_or_col p_retailprice) 'hll'),_col8: struct|{}(tok_function compute_stats (tok_table_or_col p_comment) 'hll')) 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col p_partkey) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col p_name) 'hll'),VALUE._col2: struct|{}(tok_function compute_stats (tok_table_or_col p_mfgr) 'hll'),VALUE._col3: struct|{}(tok_function compute_stats (tok_table_or_col p_brand) 'hll'),VALUE._col4: struct|{}(tok_function compute_stats (tok_table_or_col p_type) 'hll'),VALUE._col5: struct|{}(tok_function compute_stats (tok_table_or_col p_size) 'hll'),VALUE._col6: struct|{}(tok_function compute_stats (tok_table_or_col p_container) 'hll'),VALUE._col7: struct|{}(tok_function compute_stats (tok_table_or_col p_retailprice) 'hll'),VALUE._col8: struct|{}(tok_function compute_stats (tok_table_or_col p_comment) 'hll')) 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col p_partkey) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col p_name) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col p_mfgr) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col p_brand) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col p_type) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col p_size) 'hll'),_col6: struct|{}(tok_function compute_stats (tok_table_or_col p_container) 'hll'),_col7: struct|{}(tok_function compute_stats (tok_table_or_col p_retailprice) 'hll'),_col8: struct|{}(tok_function compute_stats (tok_table_or_col p_comment) 'hll')) 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1,_col2: struct|{null}_c2,_col3: struct|{null}_c3,_col4: struct|{null}_c4,_col5: struct|{null}_c5,_col6: struct|{null}_c6,_col7: struct|{null}_c7,_col8: struct|{null}_c8) 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0] Column[_col1] Column[_col2] Column[_col3] Column[_col4] Column[_col5] Column[_col6] Column[_col7] Column[_col8]) 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{},_col2: struct|{},_col3: struct|{},_col4: struct|{},_col5: struct|{},_col6: struct|{},_col7: struct|{},_col8: struct|{}) 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(p_partkey: int|{part}p_partkey,p_name: string|{part}p_name,p_mfgr: string|{part}p_mfgr,p_brand: string|{part}p_brand,p_type: string|{part}p_type,p_size: int|{part}p_size,p_container: string|{part}p_container,p_retailprice: double|{part}p_retailprice,p_comment: string|{part}p_comment,BLOCK__OFFSET__INSIDE__FILE: bigint|{part}block__offset__inside__file,INPUT__FILE__NAME: string|{part}input__file__name,ROW__ID: struct|{part}row__id,RAW__DATA__SIZE: bigint|{part}raw__data__size) 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(p_partkey: int|{part}p_partkey,p_name: string|{part}p_name,p_mfgr: string|{part}p_mfgr,p_brand: string|{part}p_brand,p_type: string|{part}p_type,p_size: int|{part}p_size,p_container: string|{part}p_container,p_retailprice: double|{part}p_retailprice,p_comment: string|{part}p_comment,BLOCK__OFFSET__INSIDE__FILE: bigint|{part}block__offset__inside__file,INPUT__FILE__NAME: string|{part}input__file__name,ROW__ID: struct|{part}row__id,RAW__DATA__SIZE: bigint|{part}raw__data__size) 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[p_partkey] Column[p_name] Column[p_mfgr] Column[p_brand] Column[p_type] Column[p_size] Column[p_container] Column[p_retailprice] Column[p_comment] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col p_partkey) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col p_name) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col p_mfgr) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col p_brand) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col p_type) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col p_size) 'hll'),_col6: struct|{}(tok_function compute_stats (tok_table_or_col p_container) 'hll'),_col7: struct|{}(tok_function compute_stats (tok_table_or_col p_retailprice) 'hll'),_col8: struct|{}(tok_function compute_stats (tok_table_or_col p_comment) 'hll')) 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col p_partkey) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col p_name) 'hll'),VALUE._col2: struct|{}(tok_function compute_stats (tok_table_or_col p_mfgr) 'hll'),VALUE._col3: struct|{}(tok_function compute_stats (tok_table_or_col p_brand) 'hll'),VALUE._col4: struct|{}(tok_function compute_stats (tok_table_or_col p_type) 'hll'),VALUE._col5: struct|{}(tok_function compute_stats (tok_table_or_col p_size) 'hll'),VALUE._col6: struct|{}(tok_function compute_stats (tok_table_or_col p_container) 'hll'),VALUE._col7: struct|{}(tok_function compute_stats (tok_table_or_col p_retailprice) 'hll'),VALUE._col8: struct|{}(tok_function compute_stats (tok_table_or_col p_comment) 'hll')) 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col p_partkey) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col p_name) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col p_mfgr) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col p_brand) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col p_type) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col p_size) 'hll'),_col6: struct|{}(tok_function compute_stats (tok_table_or_col p_container) 'hll'),_col7: struct|{}(tok_function compute_stats (tok_table_or_col p_retailprice) 'hll'),_col8: struct|{}(tok_function compute_stats (tok_table_or_col p_comment) 'hll')) 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1,_col2: struct|{null}_c2,_col3: struct|{null}_c3,_col4: struct|{null}_c4,_col5: struct|{null}_c5,_col6: struct|{null}_c6,_col7: struct|{null}_c7,_col8: struct|{null}_c8) 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0] Column[_col1] Column[_col2] Column[_col3] Column[_col4] Column[_col5] Column[_col6] Column[_col7] Column[_col8]) 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{},_col2: struct|{},_col3: struct|{},_col4: struct|{},_col5: struct|{},_col6: struct|{},_col7: struct|{},_col8: struct|{}) 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 3 key:[] 2018-01-05T11:29:01,091 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 3 oldColExprMap: {VALUE._col2=Column[_col2], VALUE._col3=Column[_col3], VALUE._col4=Column[_col4], VALUE._col5=Column[_col5], VALUE._col0=Column[_col0], VALUE._col1=Column[_col1], VALUE._col6=Column[_col6], VALUE._col7=Column[_col7], VALUE._col8=Column[_col8]} 2018-01-05T11:29:01,091 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 3 newColExprMap: {VALUE._col2=Column[_col2], VALUE._col3=Column[_col3], VALUE._col4=Column[_col4], VALUE._col5=Column[_col5], VALUE._col0=Column[_col0], VALUE._col1=Column[_col1], VALUE._col6=Column[_col6], VALUE._col7=Column[_col7], VALUE._col8=Column[_col8]} 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,091 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.IdentityProjectRemover: Identity project remover optimization removed : SEL[5] 2018-01-05T11:29:01,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: After logical optimization TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-FS[6] 2018-01-05T11:29:01,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,092 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,092 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=part 2018-01-05T11:29:01,092 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=part 2018-01-05T11:29:01,097 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 2.202694ms + 0.007475ms, the query is [select "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "TAB_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (...)] 2018-01-05T11:29:01,097 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:01,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:01,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting stats (Num rows: 26 Data size: 29120 Basic stats: COMPLETE Column stats: NONE) on TS[0] 2018-01-05T11:29:01,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (part): numRows: 26 dataSize: 29120 basicStatsState: COMPLETE colStatsState: NONE colStats: {p_partkey= colName: p_partkey colType: int countDistincts: 5 numNulls: 1 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, p_container= colName: p_container colType: string countDistincts: 5 numNulls: 1 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, p_name= colName: p_name colType: string countDistincts: 5 numNulls: 1 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, p_comment= colName: p_comment colType: string countDistincts: 5 numNulls: 1 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, p_brand= colName: p_brand colType: string countDistincts: 5 numNulls: 1 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, p_type= colName: p_type colType: string countDistincts: 5 numNulls: 1 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, p_retailprice= colName: p_retailprice colType: double countDistincts: 5 numNulls: 1 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: 4.9E-324 max: 1.7976931348623157E308 ] isPrimaryKey: false isEstimated: true, p_mfgr= colName: p_mfgr colType: string countDistincts: 5 numNulls: 1 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, p_size= colName: p_size colType: int countDistincts: 5 numNulls: 1 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true} 2018-01-05T11:29:01,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting stats (Num rows: 26 Data size: 29120 Basic stats: COMPLETE Column stats: NONE) on SEL[1] 2018-01-05T11:29:01,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [1] STATS-SEL[1]: numRows: 26 dataSize: 29120 basicStatsState: COMPLETE colStatsState: NONE colStats: {p_partkey= colName: p_partkey colType: int countDistincts: 5 numNulls: 1 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, p_container= colName: p_container colType: string countDistincts: 5 numNulls: 1 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, p_name= colName: p_name colType: string countDistincts: 5 numNulls: 1 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, p_comment= colName: p_comment colType: string countDistincts: 5 numNulls: 1 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, p_brand= colName: p_brand colType: string countDistincts: 5 numNulls: 1 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, p_type= colName: p_type colType: string countDistincts: 5 numNulls: 1 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, p_retailprice= colName: p_retailprice colType: double countDistincts: 5 numNulls: 1 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: 4.9E-324 max: 1.7976931348623157E308 ] isPrimaryKey: false isEstimated: true, p_mfgr= colName: p_mfgr colType: string countDistincts: 5 numNulls: 1 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, p_size= colName: p_size colType: int countDistincts: 5 numNulls: 1 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true} 2018-01-05T11:29:01,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[2]: inputSize: 29120 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:01,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [Case 1] STATS-GBY[2]: cardinality: 26 2018-01-05T11:29:01,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 5032 Basic stats: COMPLETE Column stats: NONE) on GBY[2] 2018-01-05T11:29:01,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-GBY[2]: numRows: 1 dataSize: 5032 basicStatsState: COMPLETE colStatsState: NONE colStats: {p_partkey= colName: p_partkey colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, p_container= colName: p_container colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, _col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, p_brand= colName: p_brand colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, p_retailprice= colName: p_retailprice colType: double countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: 4.9E-324 max: 1.7976931348623157E308 ] isPrimaryKey: false isEstimated: true, p_name= colName: p_name colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, p_comment= colName: p_comment colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, _col2= colName: _col2 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, p_type= colName: p_type colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col4= colName: _col4 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col6= colName: _col6 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col5= colName: _col5 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col8= colName: _col8 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, p_mfgr= colName: p_mfgr colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, p_size= colName: p_size colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, _col7= colName: _col7 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:01,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 5032 Basic stats: COMPLETE Column stats: NONE) on RS[3] 2018-01-05T11:29:01,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-RS[3]: numRows: 1 dataSize: 5032 basicStatsState: COMPLETE colStatsState: NONE colStats: {p_partkey= colName: p_partkey colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, p_container= colName: p_container colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, _col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, p_brand= colName: p_brand colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, p_retailprice= colName: p_retailprice colType: double countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: 4.9E-324 max: 1.7976931348623157E308 ] isPrimaryKey: false isEstimated: true, p_name= colName: p_name colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, p_comment= colName: p_comment colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, _col2= colName: _col2 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, p_type= colName: p_type colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col4= colName: _col4 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col6= colName: _col6 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col5= colName: _col5 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col8= colName: _col8 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, p_mfgr= colName: p_mfgr colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, p_size= colName: p_size colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, _col7= colName: _col7 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:01,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[4]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:01,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [Case 7] STATS-GBY[4]: cardinality: 0 2018-01-05T11:29:01,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[4]: Equals 0 in number of rows. 0 rows will be set to 1 2018-01-05T11:29:01,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 5080 Basic stats: COMPLETE Column stats: NONE) on GBY[4] 2018-01-05T11:29:01,105 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-GBY[4]: numRows: 1 dataSize: 5080 basicStatsState: COMPLETE colStatsState: NONE colStats: {p_partkey= colName: p_partkey colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, p_container= colName: p_container colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, _col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, p_brand= colName: p_brand colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, p_retailprice= colName: p_retailprice colType: double countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: 4.9E-324 max: 1.7976931348623157E308 ] isPrimaryKey: false isEstimated: true, p_name= colName: p_name colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, p_comment= colName: p_comment colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, _col2= colName: _col2 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, p_type= colName: p_type colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col4= colName: _col4 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col6= colName: _col6 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col5= colName: _col5 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col8= colName: _col8 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, p_mfgr= colName: p_mfgr colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, p_size= colName: p_size colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, _col7= colName: _col7 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-FS[6]: numRows: 1 dataSize: 5080 basicStatsState: COMPLETE colStatsState: NONE colStats: {p_partkey= colName: p_partkey colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, p_container= colName: p_container colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, _col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, p_brand= colName: p_brand colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, p_retailprice= colName: p_retailprice colType: double countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: 4.9E-324 max: 1.7976931348623157E308 ] isPrimaryKey: false isEstimated: true, p_name= colName: p_name colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, p_comment= colName: p_comment colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, _col2= colName: _col2 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, p_type= colName: p_type colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col4= colName: _col4 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col6= colName: _col6 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col5= colName: _col5 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col8= colName: _col8 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, p_mfgr= colName: p_mfgr colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, p_size= colName: p_size colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, _col7= colName: _col7 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on TS[0] 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on SEL[1] 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[2] 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[3] 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[4] 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FileSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on FS[6] 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,106 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: FS, 6 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: GBY, 4 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: RS, 3 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: GBY, 2 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: SEL, 1 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:29:01,106 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Cycle free: true 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(p_partkey: int|{part}p_partkey,p_name: string|{part}p_name,p_mfgr: string|{part}p_mfgr,p_brand: string|{part}p_brand,p_type: string|{part}p_type,p_size: int|{part}p_size,p_container: string|{part}p_container,p_retailprice: double|{part}p_retailprice,p_comment: string|{part}p_comment,BLOCK__OFFSET__INSIDE__FILE: bigint|{part}block__offset__inside__file,INPUT__FILE__NAME: string|{part}input__file__name,ROW__ID: struct|{part}row__id,RAW__DATA__SIZE: bigint|{part}raw__data__size) 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(p_partkey: int|{part}p_partkey,p_name: string|{part}p_name,p_mfgr: string|{part}p_mfgr,p_brand: string|{part}p_brand,p_type: string|{part}p_type,p_size: int|{part}p_size,p_container: string|{part}p_container,p_retailprice: double|{part}p_retailprice,p_comment: string|{part}p_comment) 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[p_partkey] Column[p_name] Column[p_mfgr] Column[p_brand] Column[p_type] Column[p_size] Column[p_container] Column[p_retailprice] Column[p_comment]) 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col p_partkey) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col p_name) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col p_mfgr) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col p_brand) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col p_type) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col p_size) 'hll'),_col6: struct|{}(tok_function compute_stats (tok_table_or_col p_container) 'hll'),_col7: struct|{}(tok_function compute_stats (tok_table_or_col p_retailprice) 'hll'),_col8: struct|{}(tok_function compute_stats (tok_table_or_col p_comment) 'hll')) 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col p_partkey) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col p_name) 'hll'),VALUE._col2: struct|{}(tok_function compute_stats (tok_table_or_col p_mfgr) 'hll'),VALUE._col3: struct|{}(tok_function compute_stats (tok_table_or_col p_brand) 'hll'),VALUE._col4: struct|{}(tok_function compute_stats (tok_table_or_col p_type) 'hll'),VALUE._col5: struct|{}(tok_function compute_stats (tok_table_or_col p_size) 'hll'),VALUE._col6: struct|{}(tok_function compute_stats (tok_table_or_col p_container) 'hll'),VALUE._col7: struct|{}(tok_function compute_stats (tok_table_or_col p_retailprice) 'hll'),VALUE._col8: struct|{}(tok_function compute_stats (tok_table_or_col p_comment) 'hll')) 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col p_partkey) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col p_name) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col p_mfgr) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col p_brand) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col p_type) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col p_size) 'hll'),_col6: struct|{}(tok_function compute_stats (tok_table_or_col p_container) 'hll'),_col7: struct|{}(tok_function compute_stats (tok_table_or_col p_retailprice) 'hll'),_col8: struct|{}(tok_function compute_stats (tok_table_or_col p_comment) 'hll')) 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{},_col2: struct|{},_col3: struct|{},_col4: struct|{},_col5: struct|{},_col6: struct|{},_col7: struct|{},_col8: struct|{}) 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:29:01,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: TS[0] 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: RS[3] 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part of table part 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: RS[3] 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: GBY[4] 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: FS[6] 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding reduce work (Reducer 2) for GBY[4] 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Setting up reduce sink: RS[3] with following reduce work: Reducer 2 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Removing RS[3] as parent from GBY[4] 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: FS[6] 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping vectorization 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:29:01,107 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: llap mode: only 2018-01-05T11:29:01,107 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(p_partkey, 'hll')' 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'p_partkey' 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(p_name, 'hll')' 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'p_name' 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(p_mfgr, 'hll')' 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'p_mfgr' 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(p_brand, 'hll')' 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'p_brand' 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(p_type, 'hll')' 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'p_type' 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(p_size, 'hll')' 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'p_size' 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(p_container, 'hll')' 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'p_container' 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(p_retailprice, 'hll')' 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'p_retailprice' 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(p_comment, 'hll')' 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'p_comment' 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'p_partkey' 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'p_name' 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'p_mfgr' 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'p_brand' 2018-01-05T11:29:01,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'p_type' 2018-01-05T11:29:01,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'p_size' 2018-01-05T11:29:01,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'p_container' 2018-01-05T11:29:01,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'p_retailprice' 2018-01-05T11:29:01,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'p_comment' 2018-01-05T11:29:01,108 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:01,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@4e14a84 can be done in LLAP 2018-01-05T11:29:01,108 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Reducer 2 2018-01-05T11:29:01,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col0)' 2018-01-05T11:29:01,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:29:01,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col1)' 2018-01-05T11:29:01,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col1' 2018-01-05T11:29:01,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col2)' 2018-01-05T11:29:01,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col2' 2018-01-05T11:29:01,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col3)' 2018-01-05T11:29:01,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col3' 2018-01-05T11:29:01,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col4)' 2018-01-05T11:29:01,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col4' 2018-01-05T11:29:01,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col5)' 2018-01-05T11:29:01,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col5' 2018-01-05T11:29:01,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col6)' 2018-01-05T11:29:01,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col6' 2018-01-05T11:29:01,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col7)' 2018-01-05T11:29:01,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col7' 2018-01-05T11:29:01,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col8)' 2018-01-05T11:29:01,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col8' 2018-01-05T11:29:01,108 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:01,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@8f6d59 can be done in LLAP 2018-01-05T11:29:01,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,108 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed plan generation 2018-01-05T11:29:01,108 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:01,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validation start 2018-01-05T11:29:01,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Not a partition. 2018-01-05T11:29:01,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:29:01,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,108 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0, type:struct, comment:null), FieldSchema(name:_c1, type:struct, comment:null), FieldSchema(name:_c2, type:struct, comment:null), FieldSchema(name:_c3, type:struct, comment:null), FieldSchema(name:_c4, type:struct, comment:null), FieldSchema(name:_c5, type:struct, comment:null), FieldSchema(name:_c6, type:struct, comment:null), FieldSchema(name:_c7, type:struct, comment:null), FieldSchema(name:_c8, type:struct, comment:null)], properties:null) 2018-01-05T11:29:01,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,108 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:01,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=6, flushCache_()=0, getTableColumnStatistics_(String, String, List, )=13} 2018-01-05T11:29:01,108 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112901_10c480d0-6dc9-477b-879b-acf80ce89912); Time taken: 0.063 seconds 2018-01-05T11:29:01,108 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:01,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,108 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112901_10c480d0-6dc9-477b-879b-acf80ce89912): analyze table part compute statistics for columns 2018-01-05T11:29:01,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: analyze table part compute statistics for columns 2018-01-05T11:29:01,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:29:01,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@part 2018-01-05T11:29:01,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@part 2018-01-05T11:29:01,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_049_1860637755816674007-1/-mr-10000 2018-01-05T11:29:01,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Query ID = djaiswal_20180105112901_10c480d0-6dc9-477b-879b-acf80ce89912 2018-01-05T11:29:01,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Total jobs = 1 2018-01-05T11:29:01,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:29:01,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MAPRED] in serial mode 2018-01-05T11:29:01,132 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_045_5682828745345364-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:29:01,135 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000061, QueryId=djaiswal_20180105112900_08173570-4964-412e-a3cb-561353455cac-30, DagId=30, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180540994, EndTime=1515180541135 2018-01-05T11:29:01,157 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:29:01,157 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:29:01,157 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:29:01,157 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112901_10c480d0-6dc9-477b-879b-acf80ce89912 2018-01-05T11:29:01,157 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Session is already open 2018-01-05T11:29:01,157 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:29:01,157 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,157 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Dag name: analyze table part compute statist...columns (Stage-0) 2018-01-05T11:29:01,157 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: DagInfo: {"context":"Hive","description":"\nanalyze table part compute statistics for columns"} 2018-01-05T11:29:01,157 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112901_10c480d0-6dc9-477b-879b-acf80ce89912 with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:29:01,157 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,169 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:29:01,169 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,169 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 2 size: 2.96KB 2018-01-05T11:29:01,185 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,185 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,196 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:29:01,196 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,196 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:29:01,197 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,197 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 4.72KB 2018-01-05T11:29:01,202 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_049_1860637755816674007-1/-mr-10001 2018-01-05T11:29:01,212 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_049_1860637755816674007-1/-mr-10001 2018-01-05T11:29:01,212 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part 2018-01-05T11:29:01,212 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,213 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,213 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,213 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=analyze table part compute statist...columns (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112901_10c480d0-6dc9-477b-879b-acf80ce89912 } 2018-01-05T11:29:01,214 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:01,214 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:01,216 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: analyze table part compute statist...columns (Stage-0) 2018-01-05T11:29:01,216 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:29:01,216 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_31, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_31.dot 2018-01-05T11:29:01,216 INFO [IPC Server handler 1 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_31 2018-01-05T11:29:01,216 WARN [IPC Server handler 1 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:29:01,216 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Running DAG: analyze table part compute statist...columns (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112901_10c480d0-6dc9-477b-879b-acf80ce89912 } 2018-01-05T11:29:01,238 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_31][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_31, submitTime=1515180541216, queueName=null 2018-01-05T11:29:01,239 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_31_00 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_31_00 [Map 1], Default Resources= 2018-01-05T11:29:01,239 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_31_00 [Map 1] 2018-01-05T11:29:01,239 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:29:01,239 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_31_00 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:01,239 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_31_01 [Reducer 2]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_31_01 [Reducer 2], Default Resources= 2018-01-05T11:29:01,239 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_31_01 [Reducer 2] 2018-01-05T11:29:01,239 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 2 2018-01-05T11:29:01,239 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_31_01 [Reducer 2] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:01,239 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:29:01,239 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_31][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_31, initTime=1515180541239 2018-01-05T11:29:01,239 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: dag_1515180518813_0001_31 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:29:01,240 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:29:01,240 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_31][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_31, startTime=1515180541240 2018-01-05T11:29:01,240 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_31 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:29:01,240 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {part={InputName=part}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:29:01,240 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: part, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:29:01,240 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_31_00 [Map 1] 2018-01-05T11:29:01,240 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_31, dagName=analyze table part compute statist...columns (Stage-0) 2018-01-05T11:29:01,240 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,240 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,240 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,244 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_31_00 [Map 1] 2018-01-05T11:29:01,244 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_31_00 [Map 1] 2018-01-05T11:29:01,244 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:01,244 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_31_00 [Map 1] 2018-01-05T11:29:01,246 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:29:01,246 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@1e59b94d 2018-01-05T11:29:01,246 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_045_5682828745345364-1/djaiswal/_tez_scratch_dir/34afe3d2-5c28-46a2-9314-3c543bc2351e/map.xml 2018-01-05T11:29:01,246 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_045_5682828745345364-1/djaiswal/_tez_scratch_dir/34afe3d2-5c28-46a2-9314-3c543bc2351e/map.xml 2018-01-05T11:29:01,246 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_045_5682828745345364-1/djaiswal/_tez_scratch_dir/34afe3d2-5c28-46a2-9314-3c543bc2351e/map.xml 2018-01-05T11:29:01,246 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:01,246 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:01,246 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,247 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:01,247 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 4.72KB 2018-01-05T11:29:01,247 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_31_00 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:29:01,247 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: part on vertex vertex_1515180518813_0001_31_00 [Map 1] 2018-01-05T11:29:01,248 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to ShuffleVertexManager for vertex_1515180518813_0001_31_01 [Reducer 2] 2018-01-05T11:29:01,248 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:29:01,248 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:01,248 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:29:01,248 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:01,248 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_045_5682828745345364-1/djaiswal/_tez_scratch_dir/34afe3d2-5c28-46a2-9314-3c543bc2351e/map.xml 2018-01-05T11:29:01,248 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:01,248 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias part 2018-01-05T11:29:01,248 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part 2018-01-05T11:29:01,248 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part 2018-01-05T11:29:01,249 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 0,1,2,3,4,5,6,7,8 2018-01-05T11:29:01,249 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = p_partkey,p_name,p_mfgr,p_brand,p_type,p_size,p_container,p_retailprice,p_comment 2018-01-05T11:29:01,249 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part 2018-01-05T11:29:01,249 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:01,249 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:01,249 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:01,249 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:01,252 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.05 maxFrac: 0.75 auto: false desiredTaskIput: 104857600 2018-01-05T11:29:01,252 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:29:01,252 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 1 tasks for vertex: vertex_1515180518813_0001_31_01 [Reducer 2] 2018-01-05T11:29:01,252 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_31_01 [Reducer 2] 2018-01-05T11:29:01,252 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_31][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_31_01, reconfigureDoneTime=1515180541252, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:29:01,252 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_31_01 [Reducer 2], numAdditionalOutputs=1 2018-01-05T11:29:01,252 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_31][Event:VERTEX_INITIALIZED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_31_01, initRequestedTime=1515180541247, initedTime=1515180541252, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:01,252 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_31_01 [Reducer 2] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:29:01,264 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Time taken to get FileStatuses: 14 2018-01-05T11:29:01,264 INFO [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:29:01,264 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 15 2018-01-05T11:29:01,264 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 1 2018-01-05T11:29:01,264 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:01,264 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 1. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:29:01,264 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_045_5682828745345364-1/djaiswal/_tez_scratch_dir/34afe3d2-5c28-46a2-9314-3c543bc2351e/map.xml 2018-01-05T11:29:01,264 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_045_5682828745345364-1/djaiswal/_tez_scratch_dir/34afe3d2-5c28-46a2-9314-3c543bc2351e/map.xml 2018-01-05T11:29:01,264 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_045_5682828745345364-1/djaiswal/_tez_scratch_dir/34afe3d2-5c28-46a2-9314-3c543bc2351e/map.xml 2018-01-05T11:29:01,264 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:01,264 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:01,264 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,265 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:01,265 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 4.72KB 2018-01-05T11:29:01,265 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part/part_tiny.txt to src new group? true 2018-01-05T11:29:01,265 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:29:01,265 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:29:01,265 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:29:01,265 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 58 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 3173 Original splits: 1 2018-01-05T11:29:01,265 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Using original number of splits: 1 desired splits: 1 2018-01-05T11:29:01,265 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 1 grouped split count is 1, for bucket: 1 2018-01-05T11:29:01,265 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:29:01,265 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: part on vertex vertex_1515180518813_0001_31_00 [Map 1] 2018-01-05T11:29:01,266 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_31_00 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:01,266 INFO [App Shared Pool - #0] impl.VertexImpl: Got updated RootInputsSpecs: {part=forAllWorkUnits=true, update=[1]} 2018-01-05T11:29:01,266 INFO [App Shared Pool - #0] impl.VertexImpl: Vertex vertex_1515180518813_0001_31_00 [Map 1] parallelism set to 1 2018-01-05T11:29:01,266 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_31_00 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:01,266 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_31][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_31_00, initRequestedTime=1515180541240, initedTime=1515180541266, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:01,266 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_31_00 [Map 1] with distanceFromRoot: 0 2018-01-05T11:29:01,266 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_31_00 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:29:01,266 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_31][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_31_00, startRequestedTime=1515180541247, startedTime=1515180541266 2018-01-05T11:29:01,266 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_31][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_31_00, reconfigureDoneTime=1515180541266, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:29:01,266 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_31_00 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:01,266 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_31_00 for vertex: vertex_1515180518813_0001_31_01 [Reducer 2] numStartedSources: 1 numSources: 1 2018-01-05T11:29:01,266 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_31_01 [Reducer 2] with distanceFromRoot: 1 2018-01-05T11:29:01,266 INFO [App Shared Pool - #0] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:29:01,266 INFO [App Shared Pool - #0] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:29:01,266 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_31][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_31_01, startRequestedTime=1515180541266, startedTime=1515180541266 2018-01-05T11:29:01,266 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_31_00 [Map 1] 2018-01-05T11:29:01,266 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_31_01 [Reducer 2] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:01,267 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: OnVertexStarted vertex: Reducer 2 with 0 source tasks and 1 pending tasks 2018-01-05T11:29:01,267 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_31][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_31_00_000000, scheduledTime=1515180541267, launchTime=1515180541267 2018-01-05T11:29:01,267 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_31_00 [Map 1] 2018-01-05T11:29:01,267 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Map 1 in vertex: Reducer 2 numBipartiteSourceTasks: 1 2018-01-05T11:29:01,267 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_31_00_000000_0, priority=5, capability=, hosts=null 2018-01-05T11:29:01,267 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=5,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:01,267 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=5,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:01,267 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:01,267 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_31_00_000000_0 on hosts=null 2018-01-05T11:29:01,267 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_31_00_000000_0 on hosts=null 2018-01-05T11:29:01,267 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_31_00_000000_0 2018-01-05T11:29:01,267 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:01,267 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #62, task=TaskInfo{task=attempt_1515180518813_0001_31_00_000000_0, priority=5, startTime=0, containerId=null, uniqueId=61, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000062 2018-01-05T11:29:01,267 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_31_00_000000_0; false 2018-01-05T11:29:01,267 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_31_00_000000_0, priority=5, startTime=311561777, containerId=container_111101111_0001_00_000062assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=61, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:01,268 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000062 for node: 10.22.8.182:49818 2018-01-05T11:29:01,270 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 31, name=analyze table part compute statist...columns (Stage-0), queryId=djaiswal_20180105112901_10c480d0-6dc9-477b-879b-acf80ce89912 2018-01-05T11:29:01,270 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000062, attempt_1515180518813_0001_31_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:01,270 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000062 for node: 10.22.8.182:49818 2018-01-05T11:29:01,270 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000062 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:01,270 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000062, launchTime=1515180541270 2018-01-05T11:29:01,270 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_31_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112901_10c480d0-6dc9-477b-879b-acf80ce89912, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000062, dagName=analyze table part compute statist...columns (Stage-0), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{part,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{Reducer 2,org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput,1}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180541240, firstAttemptStartTime=1515180541267, currentAttemptStartTime=1515180541270} 2018-01-05T11:29:01,271 INFO [IPC Server handler 2 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000062, QueryId=djaiswal_20180105112901_10c480d0-6dc9-477b-879b-acf80ce89912-31, DagId=31, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180541271 2018-01-05T11:29:01,271 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_31_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000062] on NM: [10.22.8.182:49818] 2018-01-05T11:29:01,271 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:01,271 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:01,271 DEBUG [IPC Server handler 2 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=31} with the ShuffleHandler 2018-01-05T11:29:01,271 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_31][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_31_00_000000_0, startTime=1515180541270, containerId=container_111101111_0001_00_000062, nodeId=10.22.8.182:49818 2018-01-05T11:29:01,281 INFO [IPC Server handler 2 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:29:01,282 DEBUG [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/31] 2018-01-05T11:29:01,282 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:29:01,282 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_31_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:01,282 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_31_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180541267, dagStartTime=1515180541240, withinDagPriority=5, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:29:01,282 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_31_00_000000_0 : ACCEPTED 2018-01-05T11:29:01,282 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_31_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:29:01,282 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_31_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180541267, dagStartTime=1515180541240, withinDagPriority=5, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:29:01,282 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_31_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:01,282 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_31_00_000000_0 to pre-emption queue 2018-01-05T11:29:01,282 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_31_00_000000_0: true 2018-01-05T11:29:01,282 INFO [TaskCommunicator # 9] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_31_00_000000_0 2018-01-05T11:29:01,282 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:1794872305 2018-01-05T11:29:01,282 INFO [TaskCommunicator # 9] GuaranteedTasks: Received response for attempt_1515180518813_0001_31_00_000000_0, true 2018-01-05T11:29:01,283 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : analyze table part compute statist...columns (Stage-0), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_31_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=part, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=Reducer 2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput }}, ] 2018-01-05T11:29:01,283 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1038614528, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:01,283 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_31_00_000000_0 vertexName: Map 1 2018-01-05T11:29:01,283 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000062, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_31_00_000000_0, eventCount=1 } 2018-01-05T11:29:01,283 DEBUG [TezTR-518813_1_31_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_31 queryId: djaiswal_20180105112901_10c480d0-6dc9-477b-879b-acf80ce89912 instanceType: TaskRunner2Callable 2018-01-05T11:29:01,283 INFO [TezTR-518813_1_31_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_31_00_000000_0 2018-01-05T11:29:01,284 DEBUG [TezTR-518813_1_31_0_0_0] log.PerfLogger: 2018-01-05T11:29:01,284 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:01,284 INFO [I/O Setup 1 Initialize: {Reducer 2}] writers.UnorderedPartitionedKVWriter: Requested BufferSize (tez.runtime.unordered.output.buffer.size-mb) : 100 2018-01-05T11:29:01,285 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:01,285 INFO [IPC Server handler 23 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_31_00_000000_0 sent events: (0-1). 2018-01-05T11:29:01,285 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:29:01,285 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_31_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:29:01,286 DEBUG [TezTR-518813_1_31_0_0_0] log.PerfLogger: 2018-01-05T11:29:01,286 INFO [TezTR-518813_1_31_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:01,286 INFO [TezTR-518813_1_31_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:01,286 INFO [TezTR-518813_1_31_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:01,286 INFO [I/O Setup 0 Initialize: {part}] input.MRInput: part using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:29:01,286 INFO [I/O Setup 0 Initialize: {part}] input.MRInputLegacy: part MRInputLegacy deferring initialization 2018-01-05T11:29:01,286 INFO [TezTR-518813_1_31_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:01,286 INFO [TezTR-518813_1_31_0_0_0] resources.MemoryDistributor: InitialRequests=[part:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [Reducer 2:OUTPUT:104857600:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput] 2018-01-05T11:29:01,286 INFO [TezTR-518813_1_31_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:01,286 INFO [TezTR-518813_1_31_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:01,286 INFO [TezTR-518813_1_31_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 104857600, TotalRequestedScaled: 5.24288E7, TotalJVMHeap: 1038614528, TotalAvailable: 695871733, TotalRequested/TotalJVMHeap:0.10 2018-01-05T11:29:01,287 INFO [TezTR-518813_1_31_0_0_0] resources.MemoryDistributor: Allocations=[part:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput:OUTPUT:104857600:104857600] 2018-01-05T11:29:01,287 INFO [TezTR-518813_1_31_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:01,287 INFO [TezTR-518813_1_31_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: part being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:01,287 INFO [TezTR-518813_1_31_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:29:01,287 INFO [TezTR-518813_1_31_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:01,287 INFO [I/O Setup 1 Start: {part}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: part 2018-01-05T11:29:01,287 INFO [TezTR-518813_1_31_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:01,287 INFO [TezTR-518813_1_31_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_31_00_000000_0 2018-01-05T11:29:01,287 DEBUG [TezTR-518813_1_31_0_0_0] log.PerfLogger: 2018-01-05T11:29:01,287 DEBUG [TezTR-518813_1_31_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_31_00_000000_0_10184 2018-01-05T11:29:01,287 DEBUG [TezTR-518813_1_31_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:01,287 DEBUG [TezTR-518813_1_31_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_31_00_000000_0 2018-01-05T11:29:01,287 INFO [TezTR-518813_1_31_0_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112901_10c480d0-6dc9-477b-879b-acf80ce89912 2018-01-05T11:29:01,287 DEBUG [TezTR-518813_1_31_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:01,287 DEBUG [TezTR-518813_1_31_0_0_0] log.PerfLogger: 2018-01-05T11:29:01,287 DEBUG [TezTR-518813_1_31_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:01,287 DEBUG [TezTR-518813_1_31_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:01,287 DEBUG [TezTR-518813_1_31_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112901_10c480d0-6dc9-477b-879b-acf80ce89912_Map 1__MAP_PLAN__ 2018-01-05T11:29:01,287 INFO [TezTR-518813_1_31_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_045_5682828745345364-1/djaiswal/_tez_scratch_dir/34afe3d2-5c28-46a2-9314-3c543bc2351e/map.xml 2018-01-05T11:29:01,287 DEBUG [TezTR-518813_1_31_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_045_5682828745345364-1/djaiswal/_tez_scratch_dir/34afe3d2-5c28-46a2-9314-3c543bc2351e/map.xml 2018-01-05T11:29:01,287 DEBUG [TezTR-518813_1_31_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_045_5682828745345364-1/djaiswal/_tez_scratch_dir/34afe3d2-5c28-46a2-9314-3c543bc2351e/map.xml 2018-01-05T11:29:01,287 DEBUG [TezTR-518813_1_31_0_0_0] log.PerfLogger: 2018-01-05T11:29:01,287 INFO [TezTR-518813_1_31_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:01,287 DEBUG [TezTR-518813_1_31_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,288 DEBUG [TezTR-518813_1_31_0_0_0] log.PerfLogger: 2018-01-05T11:29:01,288 INFO [TezTR-518813_1_31_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 4.72KB 2018-01-05T11:29:01,288 DEBUG [TezTR-518813_1_31_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:01,288 INFO [TezTR-518813_1_31_0_0_0] tez.MapRecordProcessor: The input names are: [part] 2018-01-05T11:29:01,288 INFO [TezTR-518813_1_31_0_0_0] input.MRInput: Initialized MRInput: part 2018-01-05T11:29:01,288 DEBUG [TezTR-518813_1_31_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 1 2018-01-05T11:29:01,288 INFO [TezTR-518813_1_31_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_045_5682828745345364-1/djaiswal/_tez_scratch_dir/34afe3d2-5c28-46a2-9314-3c543bc2351e/map.xml 2018-01-05T11:29:01,288 DEBUG [TezTR-518813_1_31_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:01,288 DEBUG [TezTR-518813_1_31_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part/part_tiny.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string:string:string:string:int:string:double:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part, columns=p_partkey,p_name,p_mfgr,p_brand,p_type,p_size,p_container,p_retailprice,p_comment, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=26, numFiles=1, serialization.ddl=struct part { i32 p_partkey, string p_name, string p_mfgr, string p_brand, string p_type, i32 p_size, string p_container, double p_retailprice, string p_comment}, transient_lastDdlTime=1515180541, rawDataSize=3147, columns.comments=, totalSize=3173, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.part}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string:string:string:string:int:string:double:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part, columns=p_partkey,p_name,p_mfgr,p_brand,p_type,p_size,p_container,p_retailprice,p_comment, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=26, numFiles=1, serialization.ddl=struct part { i32 p_partkey, string p_name, string p_mfgr, string p_brand, string p_type, i32 p_size, string p_container, double p_retailprice, string p_comment}, transient_lastDdlTime=1515180541, rawDataSize=3147, columns.comments=, totalSize=3173, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.part}, baseFileName=part, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string:string:string:string:int:string:double:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part, columns=p_partkey,p_name,p_mfgr,p_brand,p_type,p_size,p_container,p_retailprice,p_comment, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=26, numFiles=1, serialization.ddl=struct part { i32 p_partkey, string p_name, string p_mfgr, string p_brand, string p_type, i32 p_size, string p_container, double p_retailprice, string p_comment}, transient_lastDdlTime=1515180541, rawDataSize=3147, columns.comments=, totalSize=3173, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.part}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string:string:string:string:int:string:double:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part, columns=p_partkey,p_name,p_mfgr,p_brand,p_type,p_size,p_container,p_retailprice,p_comment, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=1, numRows=26, numFiles=1, serialization.ddl=struct part { i32 p_partkey, string p_name, string p_mfgr, string p_brand, string p_type, i32 p_size, string p_container, double p_retailprice, string p_comment}, transient_lastDdlTime=1515180541, rawDataSize=3147, columns.comments=, totalSize=3173, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.part}, baseFileName=part, vectorPartitionDesc=null]} 2018-01-05T11:29:01,289 DEBUG [TezTR-518813_1_31_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:29:01,289 INFO [TezTR-518813_1_31_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:29:01,289 DEBUG [TezTR-518813_1_31_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:29:01,290 DEBUG [TezTR-518813_1_31_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:29:01,290 INFO [TezTR-518813_1_31_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_045_5682828745345364-1/djaiswal/_tez_scratch_dir/34afe3d2-5c28-46a2-9314-3c543bc2351e/map.xml 2018-01-05T11:29:01,290 DEBUG [TezTR-518813_1_31_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:01,290 INFO [TezTR-518813_1_31_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_31_00_000000_0 2018-01-05T11:29:01,290 INFO [TezTR-518813_1_31_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [p_partkey, p_name, p_mfgr, p_brand, p_type, p_size, p_container, p_retailprice, p_comment] / schema.evolution.columns.types [int, string, string, string, string, int, string, double, string] (isAcidRead false) 2018-01-05T11:29:01,290 DEBUG [TezTR-518813_1_31_0_0_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:01,290 DEBUG [TezTR-518813_1_31_0_0_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:01,290 DEBUG [TezTR-518813_1_31_0_0_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:01,290 DEBUG [TezTR-518813_1_31_0_0_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:01,290 WARN [TezTR-518813_1_31_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part/part_tiny.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:29:01,290 DEBUG [TezTR-518813_1_31_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:29:01,290 WARN [TezTR-518813_1_31_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part/part_tiny.txt:0+3173 2018-01-05T11:29:01,291 DEBUG [TezTR-518813_1_31_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part/part_tiny.txt 2018-01-05T11:29:01,291 INFO [TezTR-518813_1_31_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part/part_tiny.txt:0+3173], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=3173} 2018-01-05T11:29:01,291 INFO [TezTR-518813_1_31_0_0_0] input.MRInput: part initialized RecordReader from event 2018-01-05T11:29:01,291 DEBUG [TezTR-518813_1_31_0_0_0] tez.MapRecordProcessor: Starting Output: Reducer 2 2018-01-05T11:29:01,292 INFO [TezTR-518813_1_31_0_0_0] writers.BaseUnorderedPartitionedKVWriter: Instantiating Partitioner: [org.apache.tez.mapreduce.partition.MRPartitioner] 2018-01-05T11:29:01,316 INFO [TezTR-518813_1_31_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: numBuffers=2, sizePerBuffer=52428800, skipBuffers=true, numPartitions=1, availableMemory=104857600, maxSingleBufferSizeBytes=2147483647, pipelinedShuffle=false, isFinalMergeEnabled=true, numPartitions=1, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:29:01,316 INFO [TezTR-518813_1_31_0_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:29:01,316 INFO [TezTR-518813_1_31_0_0_0] exec.MapOperator: Initializing operator MAP[0] 2018-01-05T11:29:01,316 DEBUG [TezTR-518813_1_31_0_0_0] exec.MapOperator: Initialization Done 0 MAP 2018-01-05T11:29:01,316 DEBUG [TezTR-518813_1_31_0_0_0] exec.MapOperator: Operator 0 MAP initialized 2018-01-05T11:29:01,316 DEBUG [TezTR-518813_1_31_0_0_0] exec.MapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:29:01,316 DEBUG [TezTR-518813_1_31_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,316 DEBUG [TezTR-518813_1_31_0_0_0] exec.MapOperator: Adding alias part to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part 2018-01-05T11:29:01,317 DEBUG [TezTR-518813_1_31_0_0_0] exec.MapOperator: dump TS[0] struct 2018-01-05T11:29:01,317 INFO [TezTR-518813_1_31_0_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:29:01,317 INFO [TezTR-518813_1_31_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:01,317 DEBUG [TezTR-518813_1_31_0_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:29:01,317 INFO [TezTR-518813_1_31_0_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:29:01,317 DEBUG [TezTR-518813_1_31_0_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:29:01,317 DEBUG [TezTR-518813_1_31_0_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:29:01,317 DEBUG [TezTR-518813_1_31_0_0_0] exec.TableScanOperator: Initializing children of 0 TS 2018-01-05T11:29:01,317 DEBUG [TezTR-518813_1_31_0_0_0] exec.SelectOperator: Initializing child 1 SEL 2018-01-05T11:29:01,317 INFO [TezTR-518813_1_31_0_0_0] exec.SelectOperator: Initializing operator SEL[1] 2018-01-05T11:29:01,317 INFO [TezTR-518813_1_31_0_0_0] exec.SelectOperator: SELECT struct 2018-01-05T11:29:01,317 DEBUG [TezTR-518813_1_31_0_0_0] exec.SelectOperator: Initialization Done 1 SEL 2018-01-05T11:29:01,317 DEBUG [TezTR-518813_1_31_0_0_0] exec.SelectOperator: Operator 1 SEL initialized 2018-01-05T11:29:01,317 DEBUG [TezTR-518813_1_31_0_0_0] exec.SelectOperator: Initializing children of 1 SEL 2018-01-05T11:29:01,317 DEBUG [TezTR-518813_1_31_0_0_0] exec.GroupByOperator: Initializing child 2 GBY 2018-01-05T11:29:01,317 INFO [TezTR-518813_1_31_0_0_0] exec.GroupByOperator: Initializing operator GBY[2] 2018-01-05T11:29:01,317 DEBUG [TezTR-518813_1_31_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,317 DEBUG [TezTR-518813_1_31_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,317 DEBUG [TezTR-518813_1_31_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,317 DEBUG [TezTR-518813_1_31_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,317 DEBUG [TezTR-518813_1_31_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,317 DEBUG [TezTR-518813_1_31_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,317 DEBUG [TezTR-518813_1_31_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,317 DEBUG [TezTR-518813_1_31_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,317 DEBUG [TezTR-518813_1_31_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,317 INFO [TezTR-518813_1_31_0_0_0] exec.GroupByOperator: Max hash table memory: 93860656 bytes 2018-01-05T11:29:01,317 INFO [TezTR-518813_1_31_0_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:29:01,317 DEBUG [TezTR-518813_1_31_0_0_0] exec.GroupByOperator: Initialization Done 2 GBY 2018-01-05T11:29:01,317 DEBUG [TezTR-518813_1_31_0_0_0] exec.GroupByOperator: Operator 2 GBY initialized 2018-01-05T11:29:01,317 DEBUG [TezTR-518813_1_31_0_0_0] exec.GroupByOperator: Initializing children of 2 GBY 2018-01-05T11:29:01,317 DEBUG [TezTR-518813_1_31_0_0_0] exec.ReduceSinkOperator: Initializing child 3 RS 2018-01-05T11:29:01,317 INFO [TezTR-518813_1_31_0_0_0] exec.ReduceSinkOperator: Initializing operator RS[3] 2018-01-05T11:29:01,317 DEBUG [TezTR-518813_1_31_0_0_0] exec.ReduceSinkOperator: keys size is 0 2018-01-05T11:29:01,317 INFO [TezTR-518813_1_31_0_0_0] exec.ReduceSinkOperator: Using tag = -1 2018-01-05T11:29:01,317 DEBUG [TezTR-518813_1_31_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,317 DEBUG [TezTR-518813_1_31_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,317 DEBUG [TezTR-518813_1_31_0_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8] columnTypes=[struct, struct, struct, struct, struct, struct, struct, struct, struct] 2018-01-05T11:29:01,317 DEBUG [TezTR-518813_1_31_0_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS 2018-01-05T11:29:01,317 DEBUG [TezTR-518813_1_31_0_0_0] exec.ReduceSinkOperator: Operator 3 RS initialized 2018-01-05T11:29:01,317 DEBUG [TezTR-518813_1_31_0_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS done is reset. 2018-01-05T11:29:01,317 DEBUG [TezTR-518813_1_31_0_0_0] exec.GroupByOperator: Initialization Done 2 GBY done is reset. 2018-01-05T11:29:01,317 DEBUG [TezTR-518813_1_31_0_0_0] exec.SelectOperator: Initialization Done 1 SEL done is reset. 2018-01-05T11:29:01,317 DEBUG [TezTR-518813_1_31_0_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:29:01,317 INFO [TezTR-518813_1_31_0_0_0] exec.OperatorUtils: Setting output collector: RS[3] --> Reducer 2 2018-01-05T11:29:01,317 DEBUG [TezTR-518813_1_31_0_0_0] log.PerfLogger: 2018-01-05T11:29:01,317 INFO [TezTR-518813_1_31_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_045_5682828745345364-1/djaiswal/_tez_scratch_dir/34afe3d2-5c28-46a2-9314-3c543bc2351e/map.xml 2018-01-05T11:29:01,317 DEBUG [TezTR-518813_1_31_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:01,317 DEBUG [TezTR-518813_1_31_0_0_0] exec.MapOperator: Processing alias(es) part for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part/part_tiny.txt 2018-01-05T11:29:01,318 INFO [TezTR-518813_1_31_0_0_0] exec.MapOperator: MAP[0]: records read - 1 2018-01-05T11:29:01,318 INFO [TezTR-518813_1_31_0_0_0] exec.MapOperator: MAP[0]: records read - 10 2018-01-05T11:29:01,318 DEBUG [TezTR-518813_1_31_0_0_0] log.PerfLogger: 2018-01-05T11:29:01,318 DEBUG [TezTR-518813_1_31_0_0_0] mr.ObjectCache: djaiswal_20180105112901_10c480d0-6dc9-477b-879b-acf80ce89912_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:29:01,318 DEBUG [TezTR-518813_1_31_0_0_0] exec.MapOperator: close called for operator MAP[0] 2018-01-05T11:29:01,318 INFO [TezTR-518813_1_31_0_0_0] exec.MapOperator: Closing operator MAP[0] 2018-01-05T11:29:01,318 INFO [TezTR-518813_1_31_0_0_0] exec.MapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:26, 2018-01-05T11:29:01,318 DEBUG [TezTR-518813_1_31_0_0_0] exec.MapOperator: Closing child = TS[0] 2018-01-05T11:29:01,318 DEBUG [TezTR-518813_1_31_0_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:29:01,318 DEBUG [TezTR-518813_1_31_0_0_0] exec.TableScanOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:01,318 INFO [TezTR-518813_1_31_0_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:29:01,318 DEBUG [TezTR-518813_1_31_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,318 DEBUG [TezTR-518813_1_31_0_0_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_049_1860637755816674007-1/-mr-10001 2018-01-05T11:29:01,318 INFO [TezTR-518813_1_31_0_0_0] exec.TableScanOperator: publishing : default.part/ : {numRows=26, rawDataSize=3147} 2018-01-05T11:29:01,341 DEBUG [TezTR-518813_1_31_0_0_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_049_1860637755816674007-1/-mr-10001/tmpstats-0_0 2018-01-05T11:29:01,341 DEBUG [TezTR-518813_1_31_0_0_0] fs.FSStatsPublisher: Writing stats in it : {default.part/={numRows=26, rawDataSize=3147}} 2018-01-05T11:29:01,341 DEBUG [TezTR-518813_1_31_0_0_0] exec.TableScanOperator: Closing child = SEL[1] 2018-01-05T11:29:01,341 DEBUG [TezTR-518813_1_31_0_0_0] exec.SelectOperator: close called for operator SEL[1] 2018-01-05T11:29:01,341 DEBUG [TezTR-518813_1_31_0_0_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:01,341 INFO [TezTR-518813_1_31_0_0_0] exec.SelectOperator: Closing operator SEL[1] 2018-01-05T11:29:01,341 DEBUG [TezTR-518813_1_31_0_0_0] exec.SelectOperator: Closing child = GBY[2] 2018-01-05T11:29:01,341 DEBUG [TezTR-518813_1_31_0_0_0] exec.GroupByOperator: close called for operator GBY[2] 2018-01-05T11:29:01,341 DEBUG [TezTR-518813_1_31_0_0_0] exec.GroupByOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:01,341 INFO [TezTR-518813_1_31_0_0_0] exec.GroupByOperator: Closing operator GBY[2] 2018-01-05T11:29:01,341 INFO [TezTR-518813_1_31_0_0_0] exec.GroupByOperator: Begin Hash Table flush: size = 1 2018-01-05T11:29:01,341 INFO [TezTR-518813_1_31_0_0_0] exec.ReduceSinkOperator: keys are [] num distributions: 0 2018-01-05T11:29:01,341 DEBUG [TezTR-518813_1_31_0_0_0] exec.GroupByOperator: Closing child = RS[3] 2018-01-05T11:29:01,341 DEBUG [TezTR-518813_1_31_0_0_0] exec.ReduceSinkOperator: close called for operator RS[3] 2018-01-05T11:29:01,341 DEBUG [TezTR-518813_1_31_0_0_0] exec.ReduceSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:01,341 INFO [TezTR-518813_1_31_0_0_0] exec.ReduceSinkOperator: Closing operator RS[3] 2018-01-05T11:29:01,341 INFO [TezTR-518813_1_31_0_0_0] exec.ReduceSinkOperator: RECORDS_OUT_INTERMEDIATE_Map_1:1, 2018-01-05T11:29:01,341 DEBUG [TezTR-518813_1_31_0_0_0] exec.ReduceSinkOperator: 3 Close done 2018-01-05T11:29:01,341 DEBUG [TezTR-518813_1_31_0_0_0] exec.GroupByOperator: 2 Close done 2018-01-05T11:29:01,341 DEBUG [TezTR-518813_1_31_0_0_0] exec.SelectOperator: 1 Close done 2018-01-05T11:29:01,341 DEBUG [TezTR-518813_1_31_0_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:29:01,341 DEBUG [TezTR-518813_1_31_0_0_0] exec.MapOperator: 0 Close done 2018-01-05T11:29:01,341 INFO [TezTR-518813_1_31_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_31_00_000000_0 2018-01-05T11:29:01,341 INFO [TezTR-518813_1_31_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: Waiting for all spills to complete : Pending : 0 2018-01-05T11:29:01,341 INFO [TezTR-518813_1_31_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: All spills complete 2018-01-05T11:29:01,353 DEBUG [TezTR-518813_1_31_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:01,353 INFO [TezTR-518813_1_31_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:01,353 INFO [TezTR-518813_1_31_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_31_00_000000_0, askedToStop=false 2018-01-05T11:29:01,353 INFO [TezTR-518813_1_31_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_31_00_000000_0, stopRequested=false 2018-01-05T11:29:01,353 INFO [TezTR-518813_1_31_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_31_00_000000_0: Counters: 14 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=26, INPUT_SPLIT_LENGTH_BYTES=3173, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=974, OUTPUT_BYTES_WITH_OVERHEAD=984, OUTPUT_BYTES_PHYSICAL=1012, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=26, RECORDS_OUT_INTERMEDIATE_Map_1=1]] 2018-01-05T11:29:01,353 DEBUG [TezTR-518813_1_31_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_31_0_0_0 thread id: 1922 scheme: file bytesRead: 3213 bytesWritten: 1102 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:01,353 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_31_00_000000_0, isTaskDone=true 2018-01-05T11:29:01,354 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000062, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_31_00_000000_0, eventCount=4 } 2018-01-05T11:29:01,354 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_31][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_31_00_000000_0, creationTime=1515180541267, allocationTime=1515180541268, startTime=1515180541270, finishTime=1515180541354, timeTaken=84, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:01,354 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:29:01,354 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_31_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:01,354 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_31_00_000000_0 2018-01-05T11:29:01,354 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_31_00_000000_0 2018-01-05T11:29:01,354 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000062=71 2018-01-05T11:29:01,354 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_31_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:01,354 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_31_00_000000_0: true 2018-01-05T11:29:01,354 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_31_00_000000_0; false 2018-01-05T11:29:01,354 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_31_00_000000_0 2018-01-05T11:29:01,354 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_31][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_31_00_000000, startTime=1515180541270, finishTime=1515180541354, timeTaken=84, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_31_00_000000_0, diagnostics=, counters=Counters: 20, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=3213, FILE_BYTES_WRITTEN=1102, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=26, INPUT_SPLIT_LENGTH_BYTES=3173, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=974, OUTPUT_BYTES_WITH_OVERHEAD=984, OUTPUT_BYTES_PHYSICAL=1012, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=26, RECORDS_OUT_INTERMEDIATE_Map_1=1 2018-01-05T11:29:01,354 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_31_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:01,355 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000062, QueryId=djaiswal_20180105112901_10c480d0-6dc9-477b-879b-acf80ce89912-31, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:01,355 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_31_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:29:01,355 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:01,355 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_31_00_000000_0 2018-01-05T11:29:01,355 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:01,355 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_31_00 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:01,355 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_31_00 [Map 1] 2018-01-05T11:29:01,355 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_31][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_31_00, initRequestedTime=1515180541240, initedTime=1515180541266, startRequestedTime=1515180541247, startedTime=1515180541266, finishTime=1515180541355, timeTaken=89, status=SUCCEEDED, diagnostics=, counters=Counters: 20, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=3213, FILE_BYTES_WRITTEN=1102, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=26, INPUT_SPLIT_LENGTH_BYTES=3173, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=974, OUTPUT_BYTES_WITH_OVERHEAD=984, OUTPUT_BYTES_PHYSICAL=1012, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=26, RECORDS_OUT_INTERMEDIATE_Map_1=1, vertexStats=firstTaskStartTime=1515180541270, firstTasksToStart=[ task_1515180518813_0001_31_00_000000 ], lastTaskFinishTime=1515180541354, lastTasksToFinish=[ task_1515180518813_0001_31_00_000000 ], minTaskDuration=84, maxTaskDuration=84, avgTaskDuration=84.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_31_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_31_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:01,355 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_31_00 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:01,355 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_31][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000062, stoppedTime=1515180541355, exitStatus=0 2018-01-05T11:29:01,355 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_31_01 [Reducer 2] attempt: attempt_1515180518813_0001_31_00_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:29:01,355 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000062 exited with diagnostics set to Container completed. 2018-01-05T11:29:01,355 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_31_00 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2 2018-01-05T11:29:01,355 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null 2018-01-05T11:29:01,355 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: All source tasks completed. Ramping up 1 remaining tasks for vertex: Reducer 2 2018-01-05T11:29:01,355 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Scheduling 1 tasks for vertex: Reducer 2 with totalTasks: 1. 1 source tasks completed out of 1. MinSourceTaskCompletedFraction: 1.0 min: 0.05 max: 0.75 2018-01-05T11:29:01,355 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000062 2018-01-05T11:29:01,355 INFO [App Shared Pool - #0] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Map 1, destinationVertexName=Reducer 2. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.UnorderedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:29:01,355 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_31_01 [Reducer 2] 2018-01-05T11:29:01,355 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_31][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_31_01_000000, scheduledTime=1515180541355, launchTime=1515180541355 2018-01-05T11:29:01,355 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_31_01 [Reducer 2] 2018-01-05T11:29:01,355 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_31_01_000000_0, priority=14, capability=, hosts=null 2018-01-05T11:29:01,355 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:01,356 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:01,356 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:01,356 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_31_01_000000_0 on hosts=null 2018-01-05T11:29:01,356 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_31_01_000000_0 on hosts=null 2018-01-05T11:29:01,356 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_31_01_000000_0 2018-01-05T11:29:01,356 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:01,356 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #63, task=TaskInfo{task=attempt_1515180518813_0001_31_01_000000_0, priority=14, startTime=0, containerId=null, uniqueId=62, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000063 2018-01-05T11:29:01,356 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_31_01_000000_0; false 2018-01-05T11:29:01,356 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_31_01_000000_0, priority=14, startTime=311561865, containerId=container_111101111_0001_00_000063assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=62, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:01,356 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000063 for node: 10.22.8.182:49818 2018-01-05T11:29:01,356 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000063, attempt_1515180518813_0001_31_01_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:01,356 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000063 for node: 10.22.8.182:49818 2018-01-05T11:29:01,356 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000063 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:01,356 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000063, launchTime=1515180541356 2018-01-05T11:29:01,356 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=31}, sourceName=Map 1, state=S_SUCCEEDED 2018-01-05T11:29:01,357 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_31_01_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000063] on NM: [10.22.8.182:49818] 2018-01-05T11:29:01,357 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:01,357 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:01,357 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_31_01_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112901_10c480d0-6dc9-477b-879b-acf80ce89912, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000063, dagName=analyze table part compute statist...columns (Stage-0), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.UnorderedKVInput,1}}, Outputs={{out_Reducer 2,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=2, completedTaskCount=1, dagStartTime=1515180541240, firstAttemptStartTime=1515180541355, currentAttemptStartTime=1515180541356} 2018-01-05T11:29:01,357 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_31][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_31_01_000000_0, startTime=1515180541356, containerId=container_111101111_0001_00_000063, nodeId=10.22.8.182:49818 2018-01-05T11:29:01,357 INFO [IPC Server handler 3 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000063, QueryId=djaiswal_20180105112901_10c480d0-6dc9-477b-879b-acf80ce89912-31, DagId=31, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, SubmitTime=1515180541357 2018-01-05T11:29:01,357 DEBUG [IPC Server handler 3 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=31} with the ShuffleHandler 2018-01-05T11:29:01,357 DEBUG [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/31] 2018-01-05T11:29:01,357 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:29:01,357 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_31_01_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:01,357 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_31_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180541355, dagStartTime=1515180541240, withinDagPriority=14, vertexParallelism= 1, selfAndUpstreamParallelism= 2, selfAndUpstreamComplete= 1}] 2018-01-05T11:29:01,358 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_31_01_000000_0 : ACCEPTED 2018-01-05T11:29:01,358 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_31_01_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:29:01,358 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_31_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180541355, dagStartTime=1515180541240, withinDagPriority=14, vertexParallelism= 1, selfAndUpstreamParallelism= 2, selfAndUpstreamComplete= 1} 2018-01-05T11:29:01,358 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_31_01_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:01,358 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_31_01_000000_0 to pre-emption queue 2018-01-05T11:29:01,358 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_31_01_000000_0: true 2018-01-05T11:29:01,358 INFO [TaskCommunicator # 1] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_31_01_000000_0 2018-01-05T11:29:01,358 INFO [TaskCommunicator # 1] GuaranteedTasks: Received response for attempt_1515180518813_0001_31_01_000000_0, true 2018-01-05T11:29:01,358 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:1794872305 2018-01-05T11:29:01,358 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : analyze table part compute statist...columns (Stage-0), VertexName: Reducer 2, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_31_01_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:29:01,358 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1038614528, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:01,358 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_31_01_000000_0 vertexName: Reducer 2 2018-01-05T11:29:01,358 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000063, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_31_01_000000_0, eventCount=1 } 2018-01-05T11:29:01,358 DEBUG [TezTR-518813_1_31_1_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_31 queryId: djaiswal_20180105112901_10c480d0-6dc9-477b-879b-acf80ce89912 instanceType: TaskRunner2Callable 2018-01-05T11:29:01,358 INFO [TezTR-518813_1_31_1_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_31_01_000000_0 2018-01-05T11:29:01,359 DEBUG [TezTR-518813_1_31_1_0_0] log.PerfLogger: 2018-01-05T11:29:01,359 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:01,360 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:01,361 INFO [IPC Server handler 13 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_31_01_000000_0 1 events [0,1) total 1 vertex_1515180518813_0001_31_01 [Reducer 2] 2018-01-05T11:29:01,361 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:29:01,361 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_31_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:29:01,362 DEBUG [TezTR-518813_1_31_1_0_0] log.PerfLogger: 2018-01-05T11:29:01,362 INFO [TezTR-518813_1_31_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:01,362 INFO [TezTR-518813_1_31_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:01,362 INFO [TezTR-518813_1_31_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:01,362 INFO [I/O Setup 1 Initialize: {out_Reducer 2}] output.MROutput: out_Reducer 2: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:29:01,362 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:01,362 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:01,362 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:01,362 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:01,362 INFO [TezTR-518813_1_31_1_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:01,362 INFO [TezTR-518813_1_31_1_0_0] resources.MemoryDistributor: InitialRequests=[out_Reducer 2:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.UnorderedKVInput] 2018-01-05T11:29:01,362 INFO [TezTR-518813_1_31_1_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:01,362 INFO [TezTR-518813_1_31_1_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:01,362 INFO [TezTR-518813_1_31_1_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 131404920, TotalRequestedScaled: 6.570246E7, TotalJVMHeap: 1038614528, TotalAvailable: 695871733, TotalRequested/TotalJVMHeap:0.13 2018-01-05T11:29:01,362 INFO [TezTR-518813_1_31_1_0_0] resources.MemoryDistributor: Allocations=[out_Reducer 2:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [Map 1:org.apache.tez.runtime.library.input.UnorderedKVInput:INPUT:131404920:131404920] 2018-01-05T11:29:01,362 INFO [TezTR-518813_1_31_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:01,362 INFO [TezTR-518813_1_31_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 0 2018-01-05T11:29:01,362 INFO [TezTR-518813_1_31_1_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:01,362 INFO [TezTR-518813_1_31_1_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_31_01_000000_0 2018-01-05T11:29:01,362 DEBUG [TezTR-518813_1_31_1_0_0] log.PerfLogger: 2018-01-05T11:29:01,362 DEBUG [TezTR-518813_1_31_1_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_31_01_000000_0_10187 2018-01-05T11:29:01,362 DEBUG [TezTR-518813_1_31_1_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:01,362 DEBUG [TezTR-518813_1_31_1_0_0] mr.ObjectCache: Creating djaiswal_20180105112901_10c480d0-6dc9-477b-879b-acf80ce89912_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:29:01,362 INFO [TezTR-518813_1_31_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_045_5682828745345364-1/djaiswal/_tez_scratch_dir/74d5b0d3-cbe3-4cc0-969c-44f31a0339ec/reduce.xml 2018-01-05T11:29:01,363 DEBUG [TezTR-518813_1_31_1_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_045_5682828745345364-1/djaiswal/_tez_scratch_dir/74d5b0d3-cbe3-4cc0-969c-44f31a0339ec/reduce.xml 2018-01-05T11:29:01,363 DEBUG [TezTR-518813_1_31_1_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_045_5682828745345364-1/djaiswal/_tez_scratch_dir/74d5b0d3-cbe3-4cc0-969c-44f31a0339ec/reduce.xml 2018-01-05T11:29:01,364 DEBUG [TezTR-518813_1_31_1_0_0] log.PerfLogger: 2018-01-05T11:29:01,364 INFO [TezTR-518813_1_31_1_0_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:29:01,364 DEBUG [TezTR-518813_1_31_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,364 DEBUG [TezTR-518813_1_31_1_0_0] log.PerfLogger: 2018-01-05T11:29:01,364 INFO [TezTR-518813_1_31_1_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 2.96KB 2018-01-05T11:29:01,364 DEBUG [TezTR-518813_1_31_1_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:01,364 DEBUG [TezTR-518813_1_31_1_0_0] log.PerfLogger: 2018-01-05T11:29:01,364 DEBUG [TezTR-518813_1_31_1_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:01,364 DEBUG [TezTR-518813_1_31_1_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:01,365 DEBUG [TezTR-518813_1_31_1_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:01,365 INFO [TezTR-518813_1_31_1_0_0] impl.SimpleFetchedInputAllocator: Map_1: RequestedMemory=131404920, AssignedMemory=131404920, maxSingleShuffleLimit=32851230 2018-01-05T11:29:01,365 INFO [TezTR-518813_1_31_1_0_0] impl.ShuffleManager: Map_1: numInputs=1, compressionCodec=NoCompressionCodec, numFetchers=1, ifileBufferSize=4096, ifileReadAheadEnabled=true, ifileReadAheadLength=4194304, localDiskFetchEnabled=false, sharedFetchEnabled=false, keepAlive=false, keepAliveMaxConnections=20, connectionTimeout=20000, readTimeout=180000, bufferSize=8192, bufferSize=8192, maxTaskOutputAtOnce=20 2018-01-05T11:29:01,366 INFO [TezTR-518813_1_31_1_0_0] impl.ShuffleInputEventHandlerImpl: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:29:01,366 INFO [TezTR-518813_1_31_1_0_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:29:01,367 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=31&reduce=0&map=attempt_1515180518813_0001_31_00_000000_0_10186 mapId: [attempt_1515180518813_0001_31_00_000000_0_10186] reduceId: [0] jobId: [job_1515180518813_0001] dagId: [31] keepAlive: false 2018-01-05T11:29:01,367 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=31&reduce=0&map=attempt_1515180518813_0001_31_00_000000_0_10186; hash=...Tto1tCEDrTW0k 2018-01-05T11:29:01,367 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=31&reduce=0&map=attempt_1515180518813_0001_31_00_000000_0_10186;reply=n8Ekuxw3iRTts 2018-01-05T11:29:01,367 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=31, user='djaiswal', attemptId='attempt_1515180518813_0001_31_00_000000_0_10186'} via loader 2018-01-05T11:29:01,367 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=31, user='djaiswal', attemptId='attempt_1515180518813_0001_31_00_000000_0_10186'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:29:01,367 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_31_00_000000_0_10186 not found 2018-01-05T11:29:01,368 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_31_00_000000_0_10186,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/31/output/attempt_1515180518813_0001_31_00_000000_0_10186/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/31/output/attempt_1515180518813_0001_31_00_000000_0_10186/file.out.index 2018-01-05T11:29:01,368 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Content Length in shuffle : 1043 2018-01-05T11:29:01,368 INFO [Fetcher_B {Map_1} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=31&reduce=0&map=attempt_1515180518813_0001_31_00_000000_0_10186 sent hash and receievd reply 0 ms 2018-01-05T11:29:01,368 INFO [Fetcher_B {Map_1} #0] ShuffleManager.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_31_00_000000_0_10186} to MEMORY, csize=988, dsize=984, EndTime=1515180541368, TimeTaken=0, Rate=0.00 MB/s 2018-01-05T11:29:01,368 INFO [Fetcher_B {Map_1} #0] impl.ShuffleManager: All inputs fetched for input vertex : Map 1 2018-01-05T11:29:01,368 INFO [TezTR-518813_1_31_1_0_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:29:01,368 INFO [Fetcher_B {Map_1} #0] impl.ShuffleManager: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:01,368 INFO [TezTR-518813_1_31_1_0_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:01,368 INFO [ShuffleRunner {Map_1}] impl.ShuffleManager: Map_1: Shutting down FetchScheduler, Was Interrupted: false 2018-01-05T11:29:01,368 DEBUG [TezTR-518813_1_31_1_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:01,368 INFO [ShuffleRunner {Map_1}] impl.ShuffleManager: Map_1: Scheduler thread completed 2018-01-05T11:29:01,368 DEBUG [TezTR-518813_1_31_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,368 DEBUG [TezTR-518813_1_31_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,368 DEBUG [TezTR-518813_1_31_1_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8] columnTypes=[struct, struct, struct, struct, struct, struct, struct, struct, struct] 2018-01-05T11:29:01,368 DEBUG [TezTR-518813_1_31_1_0_0] log.PerfLogger: 2018-01-05T11:29:01,369 INFO [TezTR-518813_1_31_1_0_0] exec.GroupByOperator: Initializing operator GBY[4] 2018-01-05T11:29:01,369 DEBUG [TezTR-518813_1_31_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,369 DEBUG [TezTR-518813_1_31_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,369 DEBUG [TezTR-518813_1_31_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,369 DEBUG [TezTR-518813_1_31_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,369 DEBUG [TezTR-518813_1_31_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,369 DEBUG [TezTR-518813_1_31_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,369 DEBUG [TezTR-518813_1_31_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,369 DEBUG [TezTR-518813_1_31_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,369 DEBUG [TezTR-518813_1_31_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,369 INFO [TezTR-518813_1_31_1_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:29:01,369 DEBUG [TezTR-518813_1_31_1_0_0] exec.GroupByOperator: Initialization Done 4 GBY 2018-01-05T11:29:01,369 DEBUG [TezTR-518813_1_31_1_0_0] exec.GroupByOperator: Operator 4 GBY initialized 2018-01-05T11:29:01,369 DEBUG [TezTR-518813_1_31_1_0_0] exec.GroupByOperator: Initializing children of 4 GBY 2018-01-05T11:29:01,369 DEBUG [TezTR-518813_1_31_1_0_0] exec.FileSinkOperator: Initializing child 6 FS 2018-01-05T11:29:01,369 INFO [TezTR-518813_1_31_1_0_0] exec.FileSinkOperator: Initializing operator FS[6] 2018-01-05T11:29:01,369 DEBUG [TezTR-518813_1_31_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,369 INFO [TezTR-518813_1_31_1_0_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@34b4bebe]:[_col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8]:[struct, struct, struct, struct, struct, struct, struct, struct, struct]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@71388e2a 2018-01-05T11:29:01,369 DEBUG [TezTR-518813_1_31_1_0_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:29:01,369 DEBUG [TezTR-518813_1_31_1_0_0] exec.FileSinkOperator: Initialization Done 6 FS 2018-01-05T11:29:01,369 DEBUG [TezTR-518813_1_31_1_0_0] exec.FileSinkOperator: Operator 6 FS initialized 2018-01-05T11:29:01,369 DEBUG [TezTR-518813_1_31_1_0_0] exec.FileSinkOperator: Initialization Done 6 FS done is reset. 2018-01-05T11:29:01,369 DEBUG [TezTR-518813_1_31_1_0_0] exec.GroupByOperator: Initialization Done 4 GBY done is reset. 2018-01-05T11:29:01,369 INFO [TezTR-518813_1_31_1_0_0] tez.ReduceRecordProcessor: Id =4 Id =6 <\Children> Id = 4 null<\Parent> <\FS> <\Children> <\Parent> <\GBY> 2018-01-05T11:29:01,369 DEBUG [TezTR-518813_1_31_1_0_0] log.PerfLogger: 2018-01-05T11:29:01,369 INFO [TezTR-518813_1_31_1_0_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 2 2018-01-05T11:29:01,369 DEBUG [TezTR-518813_1_31_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:01,369 DEBUG [TezTR-518813_1_31_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:01,369 DEBUG [TezTR-518813_1_31_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:01,369 DEBUG [TezTR-518813_1_31_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:01,369 DEBUG [TezTR-518813_1_31_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:01,369 DEBUG [TezTR-518813_1_31_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:01,370 INFO [TezTR-518813_1_31_1_0_0] readers.UnorderedKVReader: Num Records read: 1 2018-01-05T11:29:01,371 DEBUG [TezTR-518813_1_31_1_0_0] log.PerfLogger: 2018-01-05T11:29:01,371 DEBUG [TezTR-518813_1_31_1_0_0] mr.ObjectCache: djaiswal_20180105112901_10c480d0-6dc9-477b-879b-acf80ce89912_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:29:01,371 DEBUG [TezTR-518813_1_31_1_0_0] exec.GroupByOperator: close called for operator GBY[4] 2018-01-05T11:29:01,371 INFO [TezTR-518813_1_31_1_0_0] exec.GroupByOperator: Closing operator GBY[4] 2018-01-05T11:29:01,371 INFO [TezTR-518813_1_31_1_0_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_049_1860637755816674007-1/-mr-10000/.hive-staging_hive_2018-01-05_11-29-01_049_1860637755816674007-1/_tmp.-ext-10002/000000_0 2018-01-05T11:29:01,371 INFO [TezTR-518813_1_31_1_0_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_049_1860637755816674007-1/-mr-10000/.hive-staging_hive_2018-01-05_11-29-01_049_1860637755816674007-1/_task_tmp.-ext-10002/_tmp.000000_0 2018-01-05T11:29:01,371 INFO [TezTR-518813_1_31_1_0_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_049_1860637755816674007-1/-mr-10000/.hive-staging_hive_2018-01-05_11-29-01_049_1860637755816674007-1/_tmp.-ext-10002/000000_0 2018-01-05T11:29:01,406 INFO [TezTR-518813_1_31_1_0_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:29:01,406 DEBUG [TezTR-518813_1_31_1_0_0] exec.GroupByOperator: Closing child = FS[6] 2018-01-05T11:29:01,406 DEBUG [TezTR-518813_1_31_1_0_0] exec.FileSinkOperator: close called for operator FS[6] 2018-01-05T11:29:01,406 DEBUG [TezTR-518813_1_31_1_0_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:01,406 INFO [TezTR-518813_1_31_1_0_0] exec.FileSinkOperator: Closing operator FS[6] 2018-01-05T11:29:01,406 INFO [TezTR-518813_1_31_1_0_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:29:01,407 INFO [TezTR-518813_1_31_1_0_0] exec.FileSinkOperator: RECORDS_OUT_0:1, 2018-01-05T11:29:01,407 DEBUG [TezTR-518813_1_31_1_0_0] exec.FileSinkOperator: 6 Close done 2018-01-05T11:29:01,407 DEBUG [TezTR-518813_1_31_1_0_0] exec.GroupByOperator: 4 Close done 2018-01-05T11:29:01,407 INFO [TezTR-518813_1_31_1_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_31_01_000000_0 2018-01-05T11:29:01,407 INFO [TezTR-518813_1_31_1_0_0] impl.ShuffleInputEventHandlerImpl: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:29:01,407 INFO [TezTR-518813_1_31_1_0_0] impl.ShuffleManager: Shutting down pending fetchers on sourceMap_1: 0 2018-01-05T11:29:01,407 INFO [TezTR-518813_1_31_1_0_0] output.MROutput: out_Reducer 2 closed 2018-01-05T11:29:01,407 DEBUG [TezTR-518813_1_31_1_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:01,407 INFO [TezTR-518813_1_31_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:01,407 INFO [TezTR-518813_1_31_1_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_31_01_000000_0, askedToStop=false 2018-01-05T11:29:01,407 INFO [TezTR-518813_1_31_1_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_31_01_000000_0, stopRequested=false 2018-01-05T11:29:01,407 INFO [TezTR-518813_1_31_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_31_01_000000_0: Counters: 14 [[org.apache.tez.common.counters.TaskCounter NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=988, SHUFFLE_BYTES_DECOMPRESSED=984, SHUFFLE_BYTES_TO_MEM=988, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=3, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1][HIVE CREATED_FILES=1, RECORDS_OUT_0=1]] 2018-01-05T11:29:01,407 DEBUG [TezTR-518813_1_31_1_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_31_1_0_0 thread id: 1934 scheme: file bytesRead: 0 bytesWritten: 1430 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:01,407 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_31_01_000000_0, isTaskDone=true 2018-01-05T11:29:01,407 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000063, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_31_01_000000_0, eventCount=2 } 2018-01-05T11:29:01,408 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_31][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_31_01_000000_0, creationTime=1515180541355, allocationTime=1515180541356, startTime=1515180541356, finishTime=1515180541408, timeTaken=52, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:01,408 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_31_01_000000_0 2018-01-05T11:29:01,408 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_31_01_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:01,408 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_31_01_000000_0; false 2018-01-05T11:29:01,408 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:01,408 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_31_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:01,408 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_31_01_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:01,408 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_31_01_000000_0 2018-01-05T11:29:01,408 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_31][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_31_01_000000, startTime=1515180541356, finishTime=1515180541408, timeTaken=52, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_31_01_000000_0, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=1430, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=988, SHUFFLE_BYTES_DECOMPRESSED=984, SHUFFLE_BYTES_TO_MEM=988, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=3, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1 2018-01-05T11:29:01,408 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:01,409 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:01,409 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000063=51 2018-01-05T11:29:01,409 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_31_01 [Reducer 2], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:01,409 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_31_01_000000_0: true 2018-01-05T11:29:01,409 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_31_01 [Reducer 2] 2018-01-05T11:29:01,409 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_31][Event:VERTEX_FINISHED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_31_01, initRequestedTime=1515180541247, initedTime=1515180541252, startRequestedTime=1515180541266, startedTime=1515180541266, finishTime=1515180541409, timeTaken=143, status=SUCCEEDED, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=1430, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=988, SHUFFLE_BYTES_DECOMPRESSED=984, SHUFFLE_BYTES_TO_MEM=988, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=3, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1, vertexStats=firstTaskStartTime=1515180541356, firstTasksToStart=[ task_1515180518813_0001_31_01_000000 ], lastTaskFinishTime=1515180541408, lastTasksToFinish=[ task_1515180518813_0001_31_01_000000 ], minTaskDuration=52, maxTaskDuration=52, avgTaskDuration=52.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_31_01_000000 ], longestDurationTasks=[ task_1515180518813_0001_31_01_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:01,409 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_31_01 [Reducer 2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:01,410 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_31][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000063, stoppedTime=1515180541409, exitStatus=0 2018-01-05T11:29:01,410 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000063 exited with diagnostics set to Container completed. 2018-01-05T11:29:01,410 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_31_01 [Reducer 2] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2 2018-01-05T11:29:01,410 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null 2018-01-05T11:29:01,410 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_31_00 [Map 1] 2018-01-05T11:29:01,410 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_31_01 [Reducer 2] 2018-01-05T11:29:01,410 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_31, queueSize=0 2018-01-05T11:29:01,411 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_31][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_31, startTime=1515180541240, finishTime=1515180541410, timeTaken=170, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, OTHER_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=375, AM_GC_TIME_MILLIS=0, File System Counters, FILE_BYTES_READ=3213, FILE_BYTES_WRITTEN=2532, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=27, INPUT_SPLIT_LENGTH_BYTES=3173, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=974, OUTPUT_BYTES_WITH_OVERHEAD=984, OUTPUT_BYTES_PHYSICAL=1012, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=988, SHUFFLE_BYTES_DECOMPRESSED=984, SHUFFLE_BYTES_TO_MEM=988, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=3, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1, HIVE, CREATED_FILES=1, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=26, RECORDS_OUT_0=1, RECORDS_OUT_INTERMEDIATE_Map_1=1 2018-01-05T11:29:01,411 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_31 finished with state: SUCCEEDED 2018-01-05T11:29:01,411 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_31 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:29:01,411 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000063 2018-01-05T11:29:01,411 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_31, dagState=SUCCEEDED 2018-01-05T11:29:01,411 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_31_post 2018-01-05T11:29:01,411 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:29:01,411 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:29:01,411 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:29:01,411 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 31 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=2, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=2,NumRequestsWithoutLocation=2, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=2} 2018-01-05T11:29:01,411 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:29:01,411 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=analyze table part compute statist...columns (Stage-0), with id=dag_1515180518813_0001_31 2018-01-05T11:29:01,411 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 31, to 10.22.8.182:49818 2018-01-05T11:29:01,411 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=2, Remaining=0 2018-01-05T11:29:01,411 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=analyze table part compute statist...columns (Stage-0), with id=dag_1515180518813_0001_31 2018-01-05T11:29:01,411 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=31} 2018-01-05T11:29:01,411 INFO [IPC Server handler 1 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=31}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:29:01,411 INFO [IPC Server handler 1 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/31 after 300 seconds 2018-01-05T11:29:01,411 ERROR [IPC Server handler 1 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112901_10c480d0-6dc9-477b-879b-acf80ce89912, dag_1515180518813_0001_31 2018-01-05T11:29:01,411 INFO [IPC Server handler 1 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112901_10c480d0-6dc9-477b-879b-acf80ce89912 2018-01-05T11:29:01,411 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=31} = 1 2018-01-05T11:29:01,411 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Issuing killFragment for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=31} attempt_1515180518813_0001_31_01_000000_0 2018-01-05T11:29:01,411 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Removing attempt_1515180518813_0001_31_01_000000_0 from preemptionQueue 2018-01-05T11:29:01,412 INFO [IPC Server handler 1 on 49818] impl.TaskRunnerCallable: Ignoring kill request for task attempt_1515180518813_0001_31_01_000000_0 since it's already complete 2018-01-05T11:29:01,412 DEBUG [IPC Server handler 1 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=31} 2018-01-05T11:29:01,412 DEBUG [IPC Server handler 1 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:29:01,412 DEBUG [IPC Server handler 1 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=31} 2018-01-05T11:29:01,412 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,412 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,412 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,412 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,412 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 Reducer 2: 1/1 2018-01-05T11:29:01,412 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,414 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,414 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: TaskId for 000000_0 = 000000 2018-01-05T11:29:01,414 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,414 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,414 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,416 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-2:STATS] in serial mode 2018-01-05T11:29:01,416 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@1e2097b3, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:29:01,416 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:01,416 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,416 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:29:01,416 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:29:01,416 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:01,416 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:01,416 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,416 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:01,416 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:29:01,416 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_049_1860637755816674007-1/-mr-10001 2018-01-05T11:29:01,416 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_049_1860637755816674007-1/-mr-10001 2018-01-05T11:29:01,417 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.part/ numRows 2018-01-05T11:29:01,417 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.part/ numRows 26 2018-01-05T11:29:01,417 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.part/ rawDataSize 2018-01-05T11:29:01,417 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.part/ rawDataSize 3147 2018-01-05T11:29:01,418 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,419 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:29:01,419 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:29:01,419 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:29:01,419 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:29:01,419 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:01,419 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:01,419 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:01,420 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:01,420 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:01,420 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:29:01,420 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:29:01,420 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,420 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:29:01,420 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,420 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=part newtbl=part 2018-01-05T11:29:01,420 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=part newtbl=part 2018-01-05T11:29:01,423 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:01,427 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:01,427 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,427 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.part stats: [numFiles=1, numRows=26, totalSize=3173, rawDataSize=3147] 2018-01-05T11:29:01,427 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: About to delete stats tmp dir :file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_049_1860637755816674007-1/-mr-10001 2018-01-05T11:29:01,444 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Time taken to get FileStatuses: 16 2018-01-05T11:29:01,444 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:29:01,444 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 16 2018-01-05T11:29:01,444 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FetchOperator: Creating fetchTask with deserializer typeinfo: struct<_col0:struct,_col1:struct,_col2:struct,_col3:struct,_col4:struct,_col5:struct,_col6:struct,_col7:struct,_col8:struct> 2018-01-05T11:29:01,444 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FetchOperator: deserializer properties: table properties: {columns=_col0,_col1,_col2,_col3,_col4,_col5,_col6,_col7,_col8, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct:struct:struct:struct:struct:struct:struct:struct:struct, escape.delim=\} partition properties: {columns=_col0,_col1,_col2,_col3,_col4,_col5,_col6,_col7,_col8, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct:struct:struct:struct:struct:struct:struct:struct:struct, escape.delim=\} 2018-01-05T11:29:01,445 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,445 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: write_column_statistics: db=default table=part 2018-01-05T11:29:01,445 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=write_column_statistics: db=default table=part 2018-01-05T11:29:01,448 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=part tableName=default colName=p_partkey 2018-01-05T11:29:01,448 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=part tableName=default colName=p_name 2018-01-05T11:29:01,449 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=part tableName=default colName=p_mfgr 2018-01-05T11:29:01,449 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=part tableName=default colName=p_brand 2018-01-05T11:29:01,449 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:01,450 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=part tableName=default colName=p_type 2018-01-05T11:29:01,450 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=part tableName=default colName=p_size 2018-01-05T11:29:01,450 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=part tableName=default colName=p_container 2018-01-05T11:29:01,451 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=part tableName=default colName=p_retailprice 2018-01-05T11:29:01,451 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=part tableName=default colName=p_comment 2018-01-05T11:29:01,451 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:01,453 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:01,454 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,454 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,454 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,454 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: analyze table part compute statistics for columns 2018-01-05T11:29:01,454 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:29:01,454 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@part 2018-01-05T11:29:01,454 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@part 2018-01-05T11:29:01,454 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_049_1860637755816674007-1/-mr-10000 2018-01-05T11:29:01,454 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,454 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,454 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:01,454 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=7, setPartitionColumnStatistics_(SetPartitionsStatsRequest, )=9} 2018-01-05T11:29:01,454 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112901_10c480d0-6dc9-477b-879b-acf80ce89912); Time taken: 0.346 seconds 2018-01-05T11:29:01,454 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:29:01,454 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,454 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:01,454 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,454 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query analyze table part compute statistics for columns 2018-01-05T11:29:01,455 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_045_5682828745345364-1 2018-01-05T11:29:01,455 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000062, QueryId=djaiswal_20180105112901_10c480d0-6dc9-477b-879b-acf80ce89912-31, DagId=31, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180541282, EndTime=1515180541455 2018-01-05T11:29:01,455 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_31_01_000000_0 2018-01-05T11:29:01,455 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_31_01_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:29:01,455 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.409 seconds 2018-01-05T11:29:01,456 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:29:01,455 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_31_01_000000_0 2018-01-05T11:29:01,456 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:01,456 INFO [ExecutionCompletionThread #0] impl.QueryTracker: Ignoring fragmentComplete message for unknown query: QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=31} 2018-01-05T11:29:01,456 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:29:01,456 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:29:01,456 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,456 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:01,456 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,456 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,456 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,456 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: analyze table lineitem compute statistics 2018-01-05T11:29:01,456 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112901_56be7895-b321-4016-9b72-c1f4d0580d42): analyze table lineitem compute statistics 2018-01-05T11:29:01,456 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,456 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: analyze table lineitem compute statistics 2018-01-05T11:29:01,456 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:01,456 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,456 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,456 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,456 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,456 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:01,456 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Invoking analyze on original query 2018-01-05T11:29:01,456 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Starting Semantic Analysis 2018-01-05T11:29:01,457 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed phase 1 of Semantic Analysis 2018-01-05T11:29:01,457 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for source tables 2018-01-05T11:29:01,457 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,457 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=lineitem 2018-01-05T11:29:01,457 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=lineitem 2018-01-05T11:29:01,459 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:01,460 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,460 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,460 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=lineitem 2018-01-05T11:29:01,460 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=lineitem 2018-01-05T11:29:01,463 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:01,464 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,464 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for subqueries 2018-01-05T11:29:01,464 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for destination tables 2018-01-05T11:29:01,464 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed getting MetaData in Semantic Analysis 2018-01-05T11:29:01,478 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_456_2706990394584359273-1/-mr-10000 2018-01-05T11:29:01,478 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Table Plan for lineitem TS[0] 2018-01-05T11:29:01,478 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Body Plan for Query Block null 2018-01-05T11:29:01,478 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Plan for Query Block null 2018-01-05T11:29:01,478 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Before logical optimization TS[0] 2018-01-05T11:29:01,478 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,478 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,478 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,478 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] lineage.Generator: Time taken for lineage transform=0 2018-01-05T11:29:01,478 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,478 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,478 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,478 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,478 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,478 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,478 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,478 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,478 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(l_orderkey: int|{lineitem}l_orderkey,l_partkey: int|{lineitem}l_partkey,l_suppkey: int|{lineitem}l_suppkey,l_linenumber: int|{lineitem}l_linenumber,l_quantity: double|{lineitem}l_quantity,l_extendedprice: double|{lineitem}l_extendedprice,l_discount: double|{lineitem}l_discount,l_tax: double|{lineitem}l_tax,l_returnflag: string|{lineitem}l_returnflag,l_linestatus: string|{lineitem}l_linestatus,l_shipdate: string|{lineitem}l_shipdate,l_commitdate: string|{lineitem}l_commitdate,l_receiptdate: string|{lineitem}l_receiptdate,l_shipinstruct: string|{lineitem}l_shipinstruct,l_shipmode: string|{lineitem}l_shipmode,l_comment: string|{lineitem}l_comment,BLOCK__OFFSET__INSIDE__FILE: bigint|{lineitem}block__offset__inside__file,INPUT__FILE__NAME: string|{lineitem}input__file__name,ROW__ID: struct|{lineitem}row__id,RAW__DATA__SIZE: bigint|{lineitem}raw__data__size) 2018-01-05T11:29:01,478 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,478 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,478 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,478 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,478 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:29:01,478 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.PredicatePushDown: After PPD: TS[0] 2018-01-05T11:29:01,478 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,478 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(l_orderkey: int|{lineitem}l_orderkey,l_partkey: int|{lineitem}l_partkey,l_suppkey: int|{lineitem}l_suppkey,l_linenumber: int|{lineitem}l_linenumber,l_quantity: double|{lineitem}l_quantity,l_extendedprice: double|{lineitem}l_extendedprice,l_discount: double|{lineitem}l_discount,l_tax: double|{lineitem}l_tax,l_returnflag: string|{lineitem}l_returnflag,l_linestatus: string|{lineitem}l_linestatus,l_shipdate: string|{lineitem}l_shipdate,l_commitdate: string|{lineitem}l_commitdate,l_receiptdate: string|{lineitem}l_receiptdate,l_shipinstruct: string|{lineitem}l_shipinstruct,l_shipmode: string|{lineitem}l_shipmode,l_comment: string|{lineitem}l_comment,BLOCK__OFFSET__INSIDE__FILE: bigint|{lineitem}block__offset__inside__file,INPUT__FILE__NAME: string|{lineitem}input__file__name,ROW__ID: struct|{lineitem}row__id,RAW__DATA__SIZE: bigint|{lineitem}raw__data__size) 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(l_orderkey: int|{lineitem}l_orderkey,l_partkey: int|{lineitem}l_partkey,l_suppkey: int|{lineitem}l_suppkey,l_linenumber: int|{lineitem}l_linenumber,l_quantity: double|{lineitem}l_quantity,l_extendedprice: double|{lineitem}l_extendedprice,l_discount: double|{lineitem}l_discount,l_tax: double|{lineitem}l_tax,l_returnflag: string|{lineitem}l_returnflag,l_linestatus: string|{lineitem}l_linestatus,l_shipdate: string|{lineitem}l_shipdate,l_commitdate: string|{lineitem}l_commitdate,l_receiptdate: string|{lineitem}l_receiptdate,l_shipinstruct: string|{lineitem}l_shipinstruct,l_shipmode: string|{lineitem}l_shipmode,l_comment: string|{lineitem}l_comment,BLOCK__OFFSET__INSIDE__FILE: bigint|{lineitem}block__offset__inside__file,INPUT__FILE__NAME: string|{lineitem}input__file__name,ROW__ID: struct|{lineitem}row__id,RAW__DATA__SIZE: bigint|{lineitem}raw__data__size) 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: After logical optimization TS[0] 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,479 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,479 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=lineitem 2018-01-05T11:29:01,479 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=lineitem 2018-01-05T11:29:01,482 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:01,484 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:01,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting stats (Num rows: 1 Data size: 120990 Basic stats: COMPLETE Column stats: COMPLETE) on TS[0] 2018-01-05T11:29:01,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (lineitem): numRows: 1 dataSize: 120990 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:01,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on TS[0] 2018-01-05T11:29:01,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:29:01,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:29:01,485 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Cycle free: true 2018-01-05T11:29:01,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(l_orderkey: int|{lineitem}l_orderkey,l_partkey: int|{lineitem}l_partkey,l_suppkey: int|{lineitem}l_suppkey,l_linenumber: int|{lineitem}l_linenumber,l_quantity: double|{lineitem}l_quantity,l_extendedprice: double|{lineitem}l_extendedprice,l_discount: double|{lineitem}l_discount,l_tax: double|{lineitem}l_tax,l_returnflag: string|{lineitem}l_returnflag,l_linestatus: string|{lineitem}l_linestatus,l_shipdate: string|{lineitem}l_shipdate,l_commitdate: string|{lineitem}l_commitdate,l_receiptdate: string|{lineitem}l_receiptdate,l_shipinstruct: string|{lineitem}l_shipinstruct,l_shipmode: string|{lineitem}l_shipmode,l_comment: string|{lineitem}l_comment,BLOCK__OFFSET__INSIDE__FILE: bigint|{lineitem}block__offset__inside__file,INPUT__FILE__NAME: string|{lineitem}input__file__name,ROW__ID: struct|{lineitem}row__id,RAW__DATA__SIZE: bigint|{lineitem}raw__data__size) 2018-01-05T11:29:01,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:29:01,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem of table lineitem 2018-01-05T11:29:01,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem 2018-01-05T11:29:01,485 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:29:01,486 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,486 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,486 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:29:01,486 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:01,486 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:01,486 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:29:01,486 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:01,486 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:01,486 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:29:01,486 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping vectorization 2018-01-05T11:29:01,486 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:29:01,486 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: llap mode: only 2018-01-05T11:29:01,486 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:29:01,486 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:01,486 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@36f70ce4 can be done in LLAP 2018-01-05T11:29:01,486 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,486 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed plan generation 2018-01-05T11:29:01,486 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:01,486 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validation start 2018-01-05T11:29:01,486 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Not a partition. 2018-01-05T11:29:01,486 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,486 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:lineitem.l_orderkey, type:int, comment:null), FieldSchema(name:lineitem.l_partkey, type:int, comment:null), FieldSchema(name:lineitem.l_suppkey, type:int, comment:null), FieldSchema(name:lineitem.l_linenumber, type:int, comment:null), FieldSchema(name:lineitem.l_quantity, type:double, comment:null), FieldSchema(name:lineitem.l_extendedprice, type:double, comment:null), FieldSchema(name:lineitem.l_discount, type:double, comment:null), FieldSchema(name:lineitem.l_tax, type:double, comment:null), FieldSchema(name:lineitem.l_returnflag, type:string, comment:null), FieldSchema(name:lineitem.l_linestatus, type:string, comment:null), FieldSchema(name:lineitem.l_shipdate, type:string, comment:null), FieldSchema(name:lineitem.l_commitdate, type:string, comment:null), FieldSchema(name:lineitem.l_receiptdate, type:string, comment:null), FieldSchema(name:lineitem.l_shipinstruct, type:string, comment:null), FieldSchema(name:lineitem.l_shipmode, type:string, comment:null), FieldSchema(name:lineitem.l_comment, type:string, comment:null)], properties:null) 2018-01-05T11:29:01,486 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,486 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:01,486 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=7, flushCache_()=0, getTableColumnStatistics_(String, String, List, )=6} 2018-01-05T11:29:01,486 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112901_56be7895-b321-4016-9b72-c1f4d0580d42); Time taken: 0.03 seconds 2018-01-05T11:29:01,486 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:01,486 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,486 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112901_56be7895-b321-4016-9b72-c1f4d0580d42): analyze table lineitem compute statistics 2018-01-05T11:29:01,487 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,487 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: analyze table lineitem compute statistics 2018-01-05T11:29:01,487 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:29:01,487 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@lineitem 2018-01-05T11:29:01,487 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@lineitem 2018-01-05T11:29:01,487 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,487 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,487 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,487 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Query ID = djaiswal_20180105112901_56be7895-b321-4016-9b72-c1f4d0580d42 2018-01-05T11:29:01,487 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Total jobs = 1 2018-01-05T11:29:01,487 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,487 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:29:01,487 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MAPRED] in serial mode 2018-01-05T11:29:01,498 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_456_2706990394584359273-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:29:01,522 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:29:01,522 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:29:01,522 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:29:01,522 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112901_56be7895-b321-4016-9b72-c1f4d0580d42 2018-01-05T11:29:01,522 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Session is already open 2018-01-05T11:29:01,522 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:29:01,522 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,522 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Dag name: analyze table lineitem compute statistics (Stage-0) 2018-01-05T11:29:01,522 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: DagInfo: {"context":"Hive","description":"\nanalyze table lineitem compute statistics"} 2018-01-05T11:29:01,522 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112901_56be7895-b321-4016-9b72-c1f4d0580d42 with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:29:01,522 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,534 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:29:01,534 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,534 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:29:01,535 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,535 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 2.26KB 2018-01-05T11:29:01,540 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_456_2706990394584359273-1/-mr-10000 2018-01-05T11:29:01,553 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_456_2706990394584359273-1/-mr-10000 2018-01-05T11:29:01,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem 2018-01-05T11:29:01,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,556 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,556 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=analyze table lineitem compute statistics (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112901_56be7895-b321-4016-9b72-c1f4d0580d42 } 2018-01-05T11:29:01,556 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:01,557 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000063, QueryId=djaiswal_20180105112901_10c480d0-6dc9-477b-879b-acf80ce89912-31, DagId=31, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180541358, EndTime=1515180541557 2018-01-05T11:29:01,558 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: analyze table lineitem compute statistics (Stage-0) 2018-01-05T11:29:01,558 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:29:01,558 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_32, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_32.dot 2018-01-05T11:29:01,559 INFO [IPC Server handler 1 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_32 2018-01-05T11:29:01,559 WARN [IPC Server handler 1 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:29:01,559 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Running DAG: analyze table lineitem compute statistics (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112901_56be7895-b321-4016-9b72-c1f4d0580d42 } 2018-01-05T11:29:01,586 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_32][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_32, submitTime=1515180541558, queueName=null 2018-01-05T11:29:01,587 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_32_00 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_32_00 [Map 1], Default Resources= 2018-01-05T11:29:01,587 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_32_00 [Map 1] 2018-01-05T11:29:01,587 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_32_00 [Map 1] 2018-01-05T11:29:01,587 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:29:01,587 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_32_00 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:01,587 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:29:01,587 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_32][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_32, initTime=1515180541587 2018-01-05T11:29:01,587 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: dag_1515180518813_0001_32 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:29:01,588 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:29:01,588 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_32][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_32, startTime=1515180541588 2018-01-05T11:29:01,588 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_32 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:29:01,588 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {lineitem={InputName=lineitem}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:29:01,588 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: lineitem, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:29:01,588 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_32_00 [Map 1] 2018-01-05T11:29:01,588 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_32, dagName=analyze table lineitem compute statistics (Stage-0) 2018-01-05T11:29:01,588 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,588 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,588 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,592 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_32_00 [Map 1] 2018-01-05T11:29:01,592 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_32_00 [Map 1] 2018-01-05T11:29:01,593 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:01,593 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_32_00 [Map 1] 2018-01-05T11:29:01,595 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:29:01,595 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@1ce06ae3 2018-01-05T11:29:01,595 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_456_2706990394584359273-1/djaiswal/_tez_scratch_dir/9f49db21-aded-4207-b421-5993ce511afb/map.xml 2018-01-05T11:29:01,595 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_456_2706990394584359273-1/djaiswal/_tez_scratch_dir/9f49db21-aded-4207-b421-5993ce511afb/map.xml 2018-01-05T11:29:01,595 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_456_2706990394584359273-1/djaiswal/_tez_scratch_dir/9f49db21-aded-4207-b421-5993ce511afb/map.xml 2018-01-05T11:29:01,595 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:01,595 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:01,595 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,596 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:01,596 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 2.26KB 2018-01-05T11:29:01,596 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_32_00 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:29:01,596 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: lineitem on vertex vertex_1515180518813_0001_32_00 [Map 1] 2018-01-05T11:29:01,596 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:29:01,596 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:01,596 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:29:01,596 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:01,596 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_456_2706990394584359273-1/djaiswal/_tez_scratch_dir/9f49db21-aded-4207-b421-5993ce511afb/map.xml 2018-01-05T11:29:01,596 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:01,596 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias lineitem 2018-01-05T11:29:01,596 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem 2018-01-05T11:29:01,596 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem 2018-01-05T11:29:01,597 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 2018-01-05T11:29:01,597 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = 2018-01-05T11:29:01,597 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem 2018-01-05T11:29:01,597 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:01,597 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:01,597 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:01,597 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:01,617 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Time taken to get FileStatuses: 19 2018-01-05T11:29:01,617 INFO [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:29:01,617 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 20 2018-01-05T11:29:01,617 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 1 2018-01-05T11:29:01,617 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:01,617 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 1. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:29:01,618 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_456_2706990394584359273-1/djaiswal/_tez_scratch_dir/9f49db21-aded-4207-b421-5993ce511afb/map.xml 2018-01-05T11:29:01,618 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_456_2706990394584359273-1/djaiswal/_tez_scratch_dir/9f49db21-aded-4207-b421-5993ce511afb/map.xml 2018-01-05T11:29:01,618 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_456_2706990394584359273-1/djaiswal/_tez_scratch_dir/9f49db21-aded-4207-b421-5993ce511afb/map.xml 2018-01-05T11:29:01,618 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:01,618 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:01,618 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,618 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:01,618 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 2.26KB 2018-01-05T11:29:01,619 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem/lineitem.txt to src new group? true 2018-01-05T11:29:01,619 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:29:01,619 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:29:01,619 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:29:01,619 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 224 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 12099 Original splits: 1 2018-01-05T11:29:01,619 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Using original number of splits: 1 desired splits: 1 2018-01-05T11:29:01,619 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 1 grouped split count is 1, for bucket: 1 2018-01-05T11:29:01,619 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:29:01,619 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: lineitem on vertex vertex_1515180518813_0001_32_00 [Map 1] 2018-01-05T11:29:01,619 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_32_00 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:01,619 INFO [App Shared Pool - #0] impl.VertexImpl: Got updated RootInputsSpecs: {lineitem=forAllWorkUnits=true, update=[1]} 2018-01-05T11:29:01,620 INFO [App Shared Pool - #0] impl.VertexImpl: Vertex vertex_1515180518813_0001_32_00 [Map 1] parallelism set to 1 2018-01-05T11:29:01,620 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_32_00 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:01,620 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_32_00 [Map 1], numAdditionalOutputs=1 2018-01-05T11:29:01,620 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_32][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_32_00, initRequestedTime=1515180541588, initedTime=1515180541620, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:01,620 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_32_00 [Map 1] with distanceFromRoot: 0 2018-01-05T11:29:01,620 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_32_00 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:29:01,620 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_32][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_32_00, startRequestedTime=1515180541596, startedTime=1515180541620 2018-01-05T11:29:01,620 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_32][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_32_00, reconfigureDoneTime=1515180541620, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:29:01,620 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_32_00 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:01,621 INFO [App Shared Pool - #0] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:29:01,621 INFO [App Shared Pool - #0] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:29:01,621 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_32_00 [Map 1] 2018-01-05T11:29:01,621 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_32][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_32_00_000000, scheduledTime=1515180541621, launchTime=1515180541621 2018-01-05T11:29:01,621 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_32_00 [Map 1] 2018-01-05T11:29:01,621 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_32_00_000000_0, priority=2, capability=, hosts=null 2018-01-05T11:29:01,621 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=2,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:01,621 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=2,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:01,621 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:01,621 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_32_00_000000_0 on hosts=null 2018-01-05T11:29:01,621 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_32_00_000000_0 on hosts=null 2018-01-05T11:29:01,621 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_32_00_000000_0 2018-01-05T11:29:01,621 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:01,621 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #64, task=TaskInfo{task=attempt_1515180518813_0001_32_00_000000_0, priority=2, startTime=0, containerId=null, uniqueId=63, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000064 2018-01-05T11:29:01,621 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_32_00_000000_0; false 2018-01-05T11:29:01,622 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_32_00_000000_0, priority=2, startTime=311562131, containerId=container_111101111_0001_00_000064assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=63, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:01,622 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000064 for node: 10.22.8.182:49818 2018-01-05T11:29:01,625 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 32, name=analyze table lineitem compute statistics (Stage-0), queryId=djaiswal_20180105112901_56be7895-b321-4016-9b72-c1f4d0580d42 2018-01-05T11:29:01,625 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000064, attempt_1515180518813_0001_32_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:01,625 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000064 for node: 10.22.8.182:49818 2018-01-05T11:29:01,626 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000064 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:01,626 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000064, launchTime=1515180541626 2018-01-05T11:29:01,627 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_32_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000064] on NM: [10.22.8.182:49818] 2018-01-05T11:29:01,627 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:01,627 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:01,627 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_32][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_32_00_000000_0, startTime=1515180541626, containerId=container_111101111_0001_00_000064, nodeId=10.22.8.182:49818 2018-01-05T11:29:01,627 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_32_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112901_56be7895-b321-4016-9b72-c1f4d0580d42, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000064, dagName=analyze table lineitem compute statistics (Stage-0), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{lineitem,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{out_Map 1,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180541588, firstAttemptStartTime=1515180541621, currentAttemptStartTime=1515180541626} 2018-01-05T11:29:01,627 INFO [IPC Server handler 0 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000064, QueryId=djaiswal_20180105112901_56be7895-b321-4016-9b72-c1f4d0580d42-32, DagId=32, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180541627 2018-01-05T11:29:01,628 DEBUG [IPC Server handler 0 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=32} with the ShuffleHandler 2018-01-05T11:29:01,646 INFO [IPC Server handler 0 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:29:01,646 DEBUG [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/32] 2018-01-05T11:29:01,647 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:29:01,647 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_32_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:01,647 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_32_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180541621, dagStartTime=1515180541588, withinDagPriority=2, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:29:01,647 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_32_00_000000_0 : ACCEPTED 2018-01-05T11:29:01,647 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_32_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:29:01,647 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_32_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180541621, dagStartTime=1515180541588, withinDagPriority=2, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:29:01,647 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_32_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:01,647 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_32_00_000000_0 to pre-emption queue 2018-01-05T11:29:01,647 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_32_00_000000_0: true 2018-01-05T11:29:01,648 INFO [TaskCommunicator # 2] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_32_00_000000_0 2018-01-05T11:29:01,648 INFO [TaskCommunicator # 2] GuaranteedTasks: Received response for attempt_1515180518813_0001_32_00_000000_0, true 2018-01-05T11:29:01,648 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:2115348201 2018-01-05T11:29:01,649 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : analyze table lineitem compute statistics (Stage-0), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_32_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=lineitem, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=out_Map 1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:29:01,649 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1038614528, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:01,649 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_32_00_000000_0 vertexName: Map 1 2018-01-05T11:29:01,650 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000064, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_32_00_000000_0, eventCount=1 } 2018-01-05T11:29:01,650 DEBUG [TezTR-518813_1_32_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_32 queryId: djaiswal_20180105112901_56be7895-b321-4016-9b72-c1f4d0580d42 instanceType: TaskRunner2Callable 2018-01-05T11:29:01,650 INFO [TezTR-518813_1_32_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_32_00_000000_0 2018-01-05T11:29:01,650 DEBUG [TezTR-518813_1_32_0_0_0] log.PerfLogger: 2018-01-05T11:29:01,652 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:01,654 DEBUG [TezTR-518813_1_32_0_0_0] log.PerfLogger: 2018-01-05T11:29:01,654 INFO [TezTR-518813_1_32_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:01,654 INFO [TezTR-518813_1_32_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:01,654 INFO [I/O Setup 0 Initialize: {lineitem}] input.MRInput: lineitem using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:29:01,654 INFO [I/O Setup 0 Initialize: {lineitem}] input.MRInputLegacy: lineitem MRInputLegacy deferring initialization 2018-01-05T11:29:01,654 INFO [TezTR-518813_1_32_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:01,654 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:01,654 INFO [I/O Setup 1 Initialize: {out_Map 1}] output.MROutput: out_Map 1: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:29:01,654 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:01,654 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:01,655 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:01,655 DEBUG [I/O Setup 1 Initialize: {out_Map 1}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:01,655 INFO [TezTR-518813_1_32_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:01,655 INFO [TezTR-518813_1_32_0_0_0] resources.MemoryDistributor: InitialRequests=[lineitem:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [out_Map 1:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput] 2018-01-05T11:29:01,655 INFO [TezTR-518813_1_32_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:01,655 INFO [TezTR-518813_1_32_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:01,655 INFO [TezTR-518813_1_32_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 1038614528, TotalAvailable: 695871733, TotalRequested/TotalJVMHeap:0.00 2018-01-05T11:29:01,655 INFO [TezTR-518813_1_32_0_0_0] resources.MemoryDistributor: Allocations=[lineitem:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [out_Map 1:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0] 2018-01-05T11:29:01,655 INFO [TezTR-518813_1_32_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:01,655 INFO [TezTR-518813_1_32_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: lineitem being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:01,655 INFO [TezTR-518813_1_32_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:29:01,655 INFO [TezTR-518813_1_32_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:01,655 INFO [IPC Server handler 26 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_32_00_000000_0 sent events: (0-1). 2018-01-05T11:29:01,655 INFO [I/O Setup 0 Start: {lineitem}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: lineitem 2018-01-05T11:29:01,655 INFO [TezTR-518813_1_32_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:01,655 INFO [TezTR-518813_1_32_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_32_00_000000_0 2018-01-05T11:29:01,655 DEBUG [TezTR-518813_1_32_0_0_0] log.PerfLogger: 2018-01-05T11:29:01,655 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:29:01,655 DEBUG [TezTR-518813_1_32_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_32_00_000000_0_10190 2018-01-05T11:29:01,655 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_32_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:29:01,655 DEBUG [TezTR-518813_1_32_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:01,656 DEBUG [TezTR-518813_1_32_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_32_00_000000_0 2018-01-05T11:29:01,656 INFO [TezTR-518813_1_32_0_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112901_56be7895-b321-4016-9b72-c1f4d0580d42 2018-01-05T11:29:01,656 DEBUG [TezTR-518813_1_32_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:01,656 DEBUG [TezTR-518813_1_32_0_0_0] log.PerfLogger: 2018-01-05T11:29:01,656 DEBUG [TezTR-518813_1_32_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:01,656 DEBUG [TezTR-518813_1_32_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:01,656 DEBUG [TezTR-518813_1_32_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112901_56be7895-b321-4016-9b72-c1f4d0580d42_Map 1__MAP_PLAN__ 2018-01-05T11:29:01,656 INFO [TezTR-518813_1_32_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_456_2706990394584359273-1/djaiswal/_tez_scratch_dir/9f49db21-aded-4207-b421-5993ce511afb/map.xml 2018-01-05T11:29:01,656 DEBUG [TezTR-518813_1_32_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_456_2706990394584359273-1/djaiswal/_tez_scratch_dir/9f49db21-aded-4207-b421-5993ce511afb/map.xml 2018-01-05T11:29:01,656 DEBUG [TezTR-518813_1_32_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_456_2706990394584359273-1/djaiswal/_tez_scratch_dir/9f49db21-aded-4207-b421-5993ce511afb/map.xml 2018-01-05T11:29:01,656 DEBUG [TezTR-518813_1_32_0_0_0] log.PerfLogger: 2018-01-05T11:29:01,656 INFO [TezTR-518813_1_32_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:01,656 DEBUG [TezTR-518813_1_32_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,657 DEBUG [TezTR-518813_1_32_0_0_0] log.PerfLogger: 2018-01-05T11:29:01,657 INFO [TezTR-518813_1_32_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 2.26KB 2018-01-05T11:29:01,657 DEBUG [TezTR-518813_1_32_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:01,657 INFO [TezTR-518813_1_32_0_0_0] tez.MapRecordProcessor: The input names are: [lineitem] 2018-01-05T11:29:01,657 INFO [TezTR-518813_1_32_0_0_0] input.MRInput: Initialized MRInput: lineitem 2018-01-05T11:29:01,657 DEBUG [TezTR-518813_1_32_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 1 2018-01-05T11:29:01,657 INFO [TezTR-518813_1_32_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_456_2706990394584359273-1/djaiswal/_tez_scratch_dir/9f49db21-aded-4207-b421-5993ce511afb/map.xml 2018-01-05T11:29:01,657 DEBUG [TezTR-518813_1_32_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:01,658 DEBUG [TezTR-518813_1_32_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem/lineitem.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:int:int:int:double:double:double:double:string:string:string:string:string:string:string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem, columns=l_orderkey,l_partkey,l_suppkey,l_linenumber,l_quantity,l_extendedprice,l_discount,l_tax,l_returnflag,l_linestatus,l_shipdate,l_commitdate,l_receiptdate,l_shipinstruct,l_shipmode,l_comment, serialization.format=|, numRows=0, numFiles=1, serialization.ddl=struct lineitem { i32 l_orderkey, i32 l_partkey, i32 l_suppkey, i32 l_linenumber, double l_quantity, double l_extendedprice, double l_discount, double l_tax, string l_returnflag, string l_linestatus, string l_shipdate, string l_commitdate, string l_receiptdate, string l_shipinstruct, string l_shipmode, string l_comment}, transient_lastDdlTime=1515180537, rawDataSize=0, field.delim=|, columns.comments=, totalSize=12099, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.lineitem}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:int:int:int:double:double:double:double:string:string:string:string:string:string:string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem, columns=l_orderkey,l_partkey,l_suppkey,l_linenumber,l_quantity,l_extendedprice,l_discount,l_tax,l_returnflag,l_linestatus,l_shipdate,l_commitdate,l_receiptdate,l_shipinstruct,l_shipmode,l_comment, serialization.format=|, numRows=0, numFiles=1, serialization.ddl=struct lineitem { i32 l_orderkey, i32 l_partkey, i32 l_suppkey, i32 l_linenumber, double l_quantity, double l_extendedprice, double l_discount, double l_tax, string l_returnflag, string l_linestatus, string l_shipdate, string l_commitdate, string l_receiptdate, string l_shipinstruct, string l_shipmode, string l_comment}, transient_lastDdlTime=1515180537, rawDataSize=0, field.delim=|, columns.comments=, totalSize=12099, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.lineitem}, baseFileName=lineitem, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:int:int:int:double:double:double:double:string:string:string:string:string:string:string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem, columns=l_orderkey,l_partkey,l_suppkey,l_linenumber,l_quantity,l_extendedprice,l_discount,l_tax,l_returnflag,l_linestatus,l_shipdate,l_commitdate,l_receiptdate,l_shipinstruct,l_shipmode,l_comment, serialization.format=|, numRows=0, numFiles=1, serialization.ddl=struct lineitem { i32 l_orderkey, i32 l_partkey, i32 l_suppkey, i32 l_linenumber, double l_quantity, double l_extendedprice, double l_discount, double l_tax, string l_returnflag, string l_linestatus, string l_shipdate, string l_commitdate, string l_receiptdate, string l_shipinstruct, string l_shipmode, string l_comment}, transient_lastDdlTime=1515180537, rawDataSize=0, field.delim=|, columns.comments=, totalSize=12099, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.lineitem}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:int:int:int:double:double:double:double:string:string:string:string:string:string:string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem, columns=l_orderkey,l_partkey,l_suppkey,l_linenumber,l_quantity,l_extendedprice,l_discount,l_tax,l_returnflag,l_linestatus,l_shipdate,l_commitdate,l_receiptdate,l_shipinstruct,l_shipmode,l_comment, serialization.format=|, numRows=0, numFiles=1, serialization.ddl=struct lineitem { i32 l_orderkey, i32 l_partkey, i32 l_suppkey, i32 l_linenumber, double l_quantity, double l_extendedprice, double l_discount, double l_tax, string l_returnflag, string l_linestatus, string l_shipdate, string l_commitdate, string l_receiptdate, string l_shipinstruct, string l_shipmode, string l_comment}, transient_lastDdlTime=1515180537, rawDataSize=0, field.delim=|, columns.comments=, totalSize=12099, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.lineitem}, baseFileName=lineitem, vectorPartitionDesc=null]} 2018-01-05T11:29:01,658 DEBUG [TezTR-518813_1_32_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:29:01,658 INFO [TezTR-518813_1_32_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:29:01,658 DEBUG [TezTR-518813_1_32_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:29:01,659 DEBUG [TezTR-518813_1_32_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:29:01,659 INFO [TezTR-518813_1_32_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_456_2706990394584359273-1/djaiswal/_tez_scratch_dir/9f49db21-aded-4207-b421-5993ce511afb/map.xml 2018-01-05T11:29:01,659 DEBUG [TezTR-518813_1_32_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:01,659 INFO [TezTR-518813_1_32_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_32_00_000000_0 2018-01-05T11:29:01,659 INFO [TezTR-518813_1_32_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [l_orderkey, l_partkey, l_suppkey, l_linenumber, l_quantity, l_extendedprice, l_discount, l_tax, l_returnflag, l_linestatus, l_shipdate, l_commitdate, l_receiptdate, l_shipinstruct, l_shipmode, l_comment] / schema.evolution.columns.types [int, int, int, int, double, double, double, double, string, string, string, string, string, string, string, string] (isAcidRead false) 2018-01-05T11:29:01,660 WARN [TezTR-518813_1_32_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem/lineitem.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:29:01,660 DEBUG [TezTR-518813_1_32_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:29:01,660 WARN [TezTR-518813_1_32_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem/lineitem.txt:0+12099 2018-01-05T11:29:01,661 DEBUG [TezTR-518813_1_32_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem/lineitem.txt 2018-01-05T11:29:01,661 INFO [TezTR-518813_1_32_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem/lineitem.txt:0+12099], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=12099} 2018-01-05T11:29:01,661 INFO [TezTR-518813_1_32_0_0_0] input.MRInput: lineitem initialized RecordReader from event 2018-01-05T11:29:01,661 DEBUG [TezTR-518813_1_32_0_0_0] tez.MapRecordProcessor: Starting Output: out_Map 1 2018-01-05T11:29:01,661 INFO [TezTR-518813_1_32_0_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:29:01,661 INFO [TezTR-518813_1_32_0_0_0] exec.MapOperator: Initializing operator MAP[0] 2018-01-05T11:29:01,661 DEBUG [TezTR-518813_1_32_0_0_0] exec.MapOperator: Initialization Done 0 MAP 2018-01-05T11:29:01,661 DEBUG [TezTR-518813_1_32_0_0_0] exec.MapOperator: Operator 0 MAP initialized 2018-01-05T11:29:01,661 DEBUG [TezTR-518813_1_32_0_0_0] exec.MapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:29:01,661 DEBUG [TezTR-518813_1_32_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,661 DEBUG [TezTR-518813_1_32_0_0_0] exec.MapOperator: Adding alias lineitem to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem 2018-01-05T11:29:01,661 DEBUG [TezTR-518813_1_32_0_0_0] exec.MapOperator: dump TS[0] struct 2018-01-05T11:29:01,662 INFO [TezTR-518813_1_32_0_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:29:01,662 INFO [TezTR-518813_1_32_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:01,662 DEBUG [TezTR-518813_1_32_0_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:29:01,662 INFO [TezTR-518813_1_32_0_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:29:01,662 DEBUG [TezTR-518813_1_32_0_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:29:01,662 DEBUG [TezTR-518813_1_32_0_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:29:01,662 DEBUG [TezTR-518813_1_32_0_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:29:01,662 DEBUG [TezTR-518813_1_32_0_0_0] log.PerfLogger: 2018-01-05T11:29:01,662 INFO [TezTR-518813_1_32_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_456_2706990394584359273-1/djaiswal/_tez_scratch_dir/9f49db21-aded-4207-b421-5993ce511afb/map.xml 2018-01-05T11:29:01,662 DEBUG [TezTR-518813_1_32_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:01,662 DEBUG [TezTR-518813_1_32_0_0_0] exec.MapOperator: Processing alias(es) lineitem for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem/lineitem.txt 2018-01-05T11:29:01,662 INFO [TezTR-518813_1_32_0_0_0] exec.MapOperator: MAP[0]: records read - 1 2018-01-05T11:29:01,662 INFO [TezTR-518813_1_32_0_0_0] exec.MapOperator: MAP[0]: records read - 10 2018-01-05T11:29:01,662 INFO [TezTR-518813_1_32_0_0_0] exec.MapOperator: MAP[0]: records read - 100 2018-01-05T11:29:01,662 DEBUG [TezTR-518813_1_32_0_0_0] log.PerfLogger: 2018-01-05T11:29:01,662 DEBUG [TezTR-518813_1_32_0_0_0] mr.ObjectCache: djaiswal_20180105112901_56be7895-b321-4016-9b72-c1f4d0580d42_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:29:01,662 DEBUG [TezTR-518813_1_32_0_0_0] exec.MapOperator: close called for operator MAP[0] 2018-01-05T11:29:01,662 INFO [TezTR-518813_1_32_0_0_0] exec.MapOperator: Closing operator MAP[0] 2018-01-05T11:29:01,662 INFO [TezTR-518813_1_32_0_0_0] exec.MapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:100, 2018-01-05T11:29:01,662 DEBUG [TezTR-518813_1_32_0_0_0] exec.MapOperator: Closing child = TS[0] 2018-01-05T11:29:01,662 DEBUG [TezTR-518813_1_32_0_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:29:01,662 DEBUG [TezTR-518813_1_32_0_0_0] exec.TableScanOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:01,662 INFO [TezTR-518813_1_32_0_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:29:01,662 DEBUG [TezTR-518813_1_32_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,662 DEBUG [TezTR-518813_1_32_0_0_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_456_2706990394584359273-1/-mr-10000 2018-01-05T11:29:01,662 INFO [TezTR-518813_1_32_0_0_0] exec.TableScanOperator: publishing : default.lineitem/ : {numRows=100, rawDataSize=11999} 2018-01-05T11:29:01,688 DEBUG [TezTR-518813_1_32_0_0_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_456_2706990394584359273-1/-mr-10000/tmpstats-0_0 2018-01-05T11:29:01,688 DEBUG [TezTR-518813_1_32_0_0_0] fs.FSStatsPublisher: Writing stats in it : {default.lineitem/={numRows=100, rawDataSize=11999}} 2018-01-05T11:29:01,688 DEBUG [TezTR-518813_1_32_0_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:29:01,688 DEBUG [TezTR-518813_1_32_0_0_0] exec.MapOperator: 0 Close done 2018-01-05T11:29:01,688 INFO [TezTR-518813_1_32_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_32_00_000000_0 2018-01-05T11:29:01,688 INFO [TezTR-518813_1_32_0_0_0] output.MROutput: out_Map 1 closed 2018-01-05T11:29:01,688 DEBUG [TezTR-518813_1_32_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:01,688 INFO [TezTR-518813_1_32_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:01,688 INFO [TezTR-518813_1_32_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_32_00_000000_0, askedToStop=false 2018-01-05T11:29:01,688 INFO [TezTR-518813_1_32_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_32_00_000000_0, stopRequested=false 2018-01-05T11:29:01,689 INFO [TezTR-518813_1_32_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_32_00_000000_0: Counters: 5 [[org.apache.tez.common.counters.TaskCounter INPUT_RECORDS_PROCESSED=100, INPUT_SPLIT_LENGTH_BYTES=12099, OUTPUT_RECORDS=0][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=100]] 2018-01-05T11:29:01,689 DEBUG [TezTR-518813_1_32_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_32_0_0_0 thread id: 1963 scheme: file bytesRead: 12207 bytesWritten: 88 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:01,689 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_32_00_000000_0, isTaskDone=true 2018-01-05T11:29:01,689 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000064, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_32_00_000000_0, eventCount=2 } 2018-01-05T11:29:01,690 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_32][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_32_00_000000_0, creationTime=1515180541621, allocationTime=1515180541622, startTime=1515180541626, finishTime=1515180541689, timeTaken=63, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:01,690 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_32_00_000000_0 2018-01-05T11:29:01,690 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:29:01,690 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_32_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:01,690 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_32_00_000000_0; false 2018-01-05T11:29:01,690 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_32_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:01,690 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_32_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:01,690 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_32_00_000000_0 2018-01-05T11:29:01,690 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:01,690 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:01,690 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000064=42 2018-01-05T11:29:01,690 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_32_00_000000_0: true 2018-01-05T11:29:01,690 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_32_00_000000_0 2018-01-05T11:29:01,690 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000064, QueryId=djaiswal_20180105112901_56be7895-b321-4016-9b72-c1f4d0580d42-32, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:01,690 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_32_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:29:01,690 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_32_00_000000_0 2018-01-05T11:29:01,690 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_32][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_32_00_000000, startTime=1515180541626, finishTime=1515180541690, timeTaken=64, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_32_00_000000_0, diagnostics=, counters=Counters: 11, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=12207, FILE_BYTES_WRITTEN=88, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=100, INPUT_SPLIT_LENGTH_BYTES=12099, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=100 2018-01-05T11:29:01,690 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_32_00 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:01,690 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_32_00 [Map 1] 2018-01-05T11:29:01,690 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_32][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_32_00, initRequestedTime=1515180541588, initedTime=1515180541620, startRequestedTime=1515180541596, startedTime=1515180541620, finishTime=1515180541690, timeTaken=70, status=SUCCEEDED, diagnostics=, counters=Counters: 11, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=12207, FILE_BYTES_WRITTEN=88, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=100, INPUT_SPLIT_LENGTH_BYTES=12099, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=100, vertexStats=firstTaskStartTime=1515180541626, firstTasksToStart=[ task_1515180518813_0001_32_00_000000 ], lastTaskFinishTime=1515180541689, lastTasksToFinish=[ task_1515180518813_0001_32_00_000000 ], minTaskDuration=63, maxTaskDuration=63, avgTaskDuration=63.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_32_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_32_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:01,690 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_32_00 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:01,691 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_32][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000064, stoppedTime=1515180541691, exitStatus=0 2018-01-05T11:29:01,691 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000064 exited with diagnostics set to Container completed. 2018-01-05T11:29:01,691 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_32_00 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1 2018-01-05T11:29:01,691 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1, commitInProgress=0, terminationCause=null 2018-01-05T11:29:01,691 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_32_00 [Map 1] 2018-01-05T11:29:01,691 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_32, queueSize=0 2018-01-05T11:29:01,692 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_32][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_32, startTime=1515180541588, finishTime=1515180541691, timeTaken=103, status=SUCCEEDED, diagnostics=, counters=Counters: 15, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, OTHER_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=137, AM_GC_TIME_MILLIS=0, File System Counters, FILE_BYTES_READ=12207, FILE_BYTES_WRITTEN=88, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, INPUT_RECORDS_PROCESSED=100, INPUT_SPLIT_LENGTH_BYTES=12099, OUTPUT_RECORDS=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=100 2018-01-05T11:29:01,692 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_32 finished with state: SUCCEEDED 2018-01-05T11:29:01,692 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_32 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:29:01,692 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000064 2018-01-05T11:29:01,692 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_32, dagState=SUCCEEDED 2018-01-05T11:29:01,692 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_32_post 2018-01-05T11:29:01,692 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:29:01,692 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:29:01,692 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:29:01,692 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 32 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=1, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=1,NumRequestsWithoutLocation=1, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=1} 2018-01-05T11:29:01,692 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:29:01,692 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=analyze table lineitem compute statistics (Stage-0), with id=dag_1515180518813_0001_32 2018-01-05T11:29:01,692 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 32, to 10.22.8.182:49818 2018-01-05T11:29:01,692 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=1, Remaining=0 2018-01-05T11:29:01,692 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=analyze table lineitem compute statistics (Stage-0), with id=dag_1515180518813_0001_32 2018-01-05T11:29:01,692 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=32} 2018-01-05T11:29:01,692 INFO [IPC Server handler 2 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=32}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:29:01,692 INFO [IPC Server handler 2 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/32 after 300 seconds 2018-01-05T11:29:01,693 ERROR [IPC Server handler 2 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112901_56be7895-b321-4016-9b72-c1f4d0580d42, dag_1515180518813_0001_32 2018-01-05T11:29:01,693 INFO [IPC Server handler 2 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112901_56be7895-b321-4016-9b72-c1f4d0580d42 2018-01-05T11:29:01,693 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=32} = 0 2018-01-05T11:29:01,693 DEBUG [IPC Server handler 2 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=32} 2018-01-05T11:29:01,693 DEBUG [IPC Server handler 2 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:29:01,693 DEBUG [IPC Server handler 2 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=32} 2018-01-05T11:29:01,693 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,693 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,693 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 2018-01-05T11:29:01,693 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,693 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-2:STATS] in serial mode 2018-01-05T11:29:01,693 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@5c4f2b59, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:29:01,693 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:01,693 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,693 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:29:01,694 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:29:01,694 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:01,694 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:01,694 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,694 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:01,694 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:29:01,694 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_456_2706990394584359273-1/-mr-10000 2018-01-05T11:29:01,694 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_456_2706990394584359273-1/-mr-10000 2018-01-05T11:29:01,695 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.lineitem/ numRows 2018-01-05T11:29:01,695 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.lineitem/ numRows 100 2018-01-05T11:29:01,695 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.lineitem/ rawDataSize 2018-01-05T11:29:01,695 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.lineitem/ rawDataSize 11999 2018-01-05T11:29:01,695 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,696 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:29:01,696 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:29:01,696 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:29:01,696 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:29:01,696 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:01,696 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:01,696 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:01,697 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:01,698 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:01,698 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:29:01,698 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:29:01,698 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,698 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:29:01,698 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,698 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=lineitem newtbl=lineitem 2018-01-05T11:29:01,698 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=lineitem newtbl=lineitem 2018-01-05T11:29:01,701 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:01,706 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:01,706 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,706 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.lineitem stats: [numFiles=1, numRows=100, totalSize=12099, rawDataSize=11999] 2018-01-05T11:29:01,707 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: About to delete stats tmp dir :file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_456_2706990394584359273-1/-mr-10000 2018-01-05T11:29:01,707 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,707 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,707 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: analyze table lineitem compute statistics 2018-01-05T11:29:01,707 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:29:01,707 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@lineitem 2018-01-05T11:29:01,707 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@lineitem 2018-01-05T11:29:01,707 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,707 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,707 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:01,707 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=8} 2018-01-05T11:29:01,707 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112901_56be7895-b321-4016-9b72-c1f4d0580d42); Time taken: 0.221 seconds 2018-01-05T11:29:01,707 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:29:01,707 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,707 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:01,707 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,707 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query analyze table lineitem compute statistics 2018-01-05T11:29:01,707 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_456_2706990394584359273-1 2018-01-05T11:29:01,708 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.251 seconds 2018-01-05T11:29:01,708 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:29:01,708 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:01,708 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:29:01,708 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:29:01,708 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,708 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:01,708 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,708 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,708 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,708 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: analyze table lineitem compute statistics for columns 2018-01-05T11:29:01,708 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Compiling command(queryId=djaiswal_20180105112901_531e127a-818d-4fb7-ad60-8d20036d1617): analyze table lineitem compute statistics for columns 2018-01-05T11:29:01,708 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,708 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: analyze table lineitem compute statistics for columns 2018-01-05T11:29:01,708 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:01,708 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,708 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,708 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,708 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,708 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:01,708 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,708 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=lineitem 2018-01-05T11:29:01,708 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=lineitem 2018-01-05T11:29:01,711 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:01,711 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,711 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: select compute_stats(`l_orderkey`, 'hll') , compute_stats(`l_partkey`, 'hll') , compute_stats(`l_suppkey`, 'hll') , compute_stats(`l_linenumber`, 'hll') , compute_stats(`l_quantity`, 'hll') , compute_stats(`l_extendedprice`, 'hll') , compute_stats(`l_discount`, 'hll') , compute_stats(`l_tax`, 'hll') , compute_stats(`l_returnflag`, 'hll') , compute_stats(`l_linestatus`, 'hll') , compute_stats(`l_shipdate`, 'hll') , compute_stats(`l_commitdate`, 'hll') , compute_stats(`l_receiptdate`, 'hll') , compute_stats(`l_shipinstruct`, 'hll') , compute_stats(`l_shipmode`, 'hll') , compute_stats(`l_comment`, 'hll') from `default`.`lineitem` 2018-01-05T11:29:01,712 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parsing command: select compute_stats(`l_orderkey`, 'hll') , compute_stats(`l_partkey`, 'hll') , compute_stats(`l_suppkey`, 'hll') , compute_stats(`l_linenumber`, 'hll') , compute_stats(`l_quantity`, 'hll') , compute_stats(`l_extendedprice`, 'hll') , compute_stats(`l_discount`, 'hll') , compute_stats(`l_tax`, 'hll') , compute_stats(`l_returnflag`, 'hll') , compute_stats(`l_linestatus`, 'hll') , compute_stats(`l_shipdate`, 'hll') , compute_stats(`l_commitdate`, 'hll') , compute_stats(`l_receiptdate`, 'hll') , compute_stats(`l_shipinstruct`, 'hll') , compute_stats(`l_shipmode`, 'hll') , compute_stats(`l_comment`, 'hll') from `default`.`lineitem` 2018-01-05T11:29:01,715 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:01,715 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Invoking analyze on rewritten query 2018-01-05T11:29:01,715 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Starting Semantic Analysis 2018-01-05T11:29:01,715 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,715 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,715 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,715 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,715 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,715 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,715 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,715 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,715 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,715 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,715 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,715 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,715 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,715 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,715 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,715 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,715 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed phase 1 of Semantic Analysis 2018-01-05T11:29:01,715 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for source tables 2018-01-05T11:29:01,715 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,715 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table : db=default tbl=lineitem 2018-01-05T11:29:01,715 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=lineitem 2018-01-05T11:29:01,718 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:01,718 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,718 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for subqueries 2018-01-05T11:29:01,719 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Get metadata for destination tables 2018-01-05T11:29:01,733 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed getting MetaData in Semantic Analysis 2018-01-05T11:29:01,733 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_712_4485361284563650037-1/-mr-10001 2018-01-05T11:29:01,733 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Table Plan for lineitem TS[0] 2018-01-05T11:29:01,733 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,734 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,734 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,734 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,734 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,734 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,734 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,734 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,734 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,735 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,735 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,735 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,735 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,735 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,735 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,735 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:01,736 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: RR before GB lineitem{(l_orderkey,l_orderkey: int)(l_partkey,l_partkey: int)(l_suppkey,l_suppkey: int)(l_linenumber,l_linenumber: int)(l_quantity,l_quantity: double)(l_extendedprice,l_extendedprice: double)(l_discount,l_discount: double)(l_tax,l_tax: double)(l_returnflag,l_returnflag: string)(l_linestatus,l_linestatus: string)(l_shipdate,l_shipdate: string)(l_commitdate,l_commitdate: string)(l_receiptdate,l_receiptdate: string)(l_shipinstruct,l_shipinstruct: string)(l_shipmode,l_shipmode: string)(l_comment,l_comment: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)(RAW__DATA__SIZE,RAW__DATA__SIZE: bigint)} after GB {((tok_function compute_stats (tok_table_or_col l_orderkey) 'hll'),_col0: struct)((tok_function compute_stats (tok_table_or_col l_partkey) 'hll'),_col1: struct)((tok_function compute_stats (tok_table_or_col l_suppkey) 'hll'),_col2: struct)((tok_function compute_stats (tok_table_or_col l_linenumber) 'hll'),_col3: struct)((tok_function compute_stats (tok_table_or_col l_quantity) 'hll'),_col4: struct)((tok_function compute_stats (tok_table_or_col l_extendedprice) 'hll'),_col5: struct)((tok_function compute_stats (tok_table_or_col l_discount) 'hll'),_col6: struct)((tok_function compute_stats (tok_table_or_col l_tax) 'hll'),_col7: struct)((tok_function compute_stats (tok_table_or_col l_returnflag) 'hll'),_col8: struct)((tok_function compute_stats (tok_table_or_col l_linestatus) 'hll'),_col9: struct)((tok_function compute_stats (tok_table_or_col l_shipdate) 'hll'),_col10: struct)((tok_function compute_stats (tok_table_or_col l_commitdate) 'hll'),_col11: struct)((tok_function compute_stats (tok_table_or_col l_receiptdate) 'hll'),_col12: struct)((tok_function compute_stats (tok_table_or_col l_shipinstruct) 'hll'),_col13: struct)((tok_function compute_stats (tok_table_or_col l_shipmode) 'hll'),_col14: struct)((tok_function compute_stats (tok_table_or_col l_comment) 'hll'),_col15: struct)} 2018-01-05T11:29:01,736 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: tree: (tok_select (tok_selexpr (tok_function compute_stats (tok_table_or_col l_orderkey) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col l_partkey) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col l_suppkey) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col l_linenumber) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col l_quantity) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col l_extendedprice) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col l_discount) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col l_tax) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col l_returnflag) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col l_linestatus) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col l_shipdate) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col l_commitdate) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col l_receiptdate) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col l_shipinstruct) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col l_shipmode) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col l_comment) 'hll'))) 2018-01-05T11:29:01,736 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: genSelectPlan: input = {((tok_function compute_stats (tok_table_or_col l_orderkey) 'hll'),_col0: struct)((tok_function compute_stats (tok_table_or_col l_partkey) 'hll'),_col1: struct)((tok_function compute_stats (tok_table_or_col l_suppkey) 'hll'),_col2: struct)((tok_function compute_stats (tok_table_or_col l_linenumber) 'hll'),_col3: struct)((tok_function compute_stats (tok_table_or_col l_quantity) 'hll'),_col4: struct)((tok_function compute_stats (tok_table_or_col l_extendedprice) 'hll'),_col5: struct)((tok_function compute_stats (tok_table_or_col l_discount) 'hll'),_col6: struct)((tok_function compute_stats (tok_table_or_col l_tax) 'hll'),_col7: struct)((tok_function compute_stats (tok_table_or_col l_returnflag) 'hll'),_col8: struct)((tok_function compute_stats (tok_table_or_col l_linestatus) 'hll'),_col9: struct)((tok_function compute_stats (tok_table_or_col l_shipdate) 'hll'),_col10: struct)((tok_function compute_stats (tok_table_or_col l_commitdate) 'hll'),_col11: struct)((tok_function compute_stats (tok_table_or_col l_receiptdate) 'hll'),_col12: struct)((tok_function compute_stats (tok_table_or_col l_shipinstruct) 'hll'),_col13: struct)((tok_function compute_stats (tok_table_or_col l_shipmode) 'hll'),_col14: struct)((tok_function compute_stats (tok_table_or_col l_comment) 'hll'),_col15: struct)} starRr = lineitem{(l_orderkey,l_orderkey: int)(l_partkey,l_partkey: int)(l_suppkey,l_suppkey: int)(l_linenumber,l_linenumber: int)(l_quantity,l_quantity: double)(l_extendedprice,l_extendedprice: double)(l_discount,l_discount: double)(l_tax,l_tax: double)(l_returnflag,l_returnflag: string)(l_linestatus,l_linestatus: string)(l_shipdate,l_shipdate: string)(l_commitdate,l_commitdate: string)(l_receiptdate,l_receiptdate: string)(l_shipinstruct,l_shipinstruct: string)(l_shipmode,l_shipmode: string)(l_comment,l_comment: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)(RAW__DATA__SIZE,RAW__DATA__SIZE: bigint)} 2018-01-05T11:29:01,737 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Select Plan row schema: null{(_c0,_col0: struct)(_c1,_col1: struct)(_c2,_col2: struct)(_c3,_col3: struct)(_c4,_col4: struct)(_c5,_col5: struct)(_c6,_col6: struct)(_c7,_col7: struct)(_c8,_col8: struct)(_c9,_col9: struct)(_c10,_col10: struct)(_c11,_col11: struct)(_c12,_col12: struct)(_c13,_col13: struct)(_c14,_col14: struct)(_c15,_col15: struct)} 2018-01-05T11:29:01,737 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Select Plan for clause: insclause-0 2018-01-05T11:29:01,737 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_712_4485361284563650037-1/-mr-10000/.hive-staging_hive_2018-01-05_11-29-01_712_4485361284563650037-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_712_4485361284563650037-1/-mr-10000 2018-01-05T11:29:01,737 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_712_4485361284563650037-1/-mr-10000/.hive-staging_hive_2018-01-05_11-29-01_712_4485361284563650037-1 2018-01-05T11:29:01,761 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_712_4485361284563650037-1/-mr-10003 2018-01-05T11:29:01,761 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_712_4485361284563650037-1/-mr-10000 row schema: null{(_c0,_col0: struct)(_c1,_col1: struct)(_c2,_col2: struct)(_c3,_col3: struct)(_c4,_col4: struct)(_c5,_col5: struct)(_c6,_col6: struct)(_c7,_col7: struct)(_c8,_col8: struct)(_c9,_col9: struct)(_c10,_col10: struct)(_c11,_col11: struct)(_c12,_col12: struct)(_c13,_col13: struct)(_c14,_col14: struct)(_c15,_col15: struct)} 2018-01-05T11:29:01,761 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Body Plan for Query Block null 2018-01-05T11:29:01,761 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Created Plan for Query Block null 2018-01-05T11:29:01,761 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Before logical optimization TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-SEL[5]-FS[6] 2018-01-05T11:29:01,761 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,761 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,761 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,762 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] lineage.Generator: Time taken for lineage transform=1 2018-01-05T11:29:01,762 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,762 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,762 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,762 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,762 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,762 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,762 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,762 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,762 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(l_orderkey: int|{lineitem}l_orderkey,l_partkey: int|{lineitem}l_partkey,l_suppkey: int|{lineitem}l_suppkey,l_linenumber: int|{lineitem}l_linenumber,l_quantity: double|{lineitem}l_quantity,l_extendedprice: double|{lineitem}l_extendedprice,l_discount: double|{lineitem}l_discount,l_tax: double|{lineitem}l_tax,l_returnflag: string|{lineitem}l_returnflag,l_linestatus: string|{lineitem}l_linestatus,l_shipdate: string|{lineitem}l_shipdate,l_commitdate: string|{lineitem}l_commitdate,l_receiptdate: string|{lineitem}l_receiptdate,l_shipinstruct: string|{lineitem}l_shipinstruct,l_shipmode: string|{lineitem}l_shipmode,l_comment: string|{lineitem}l_comment,BLOCK__OFFSET__INSIDE__FILE: bigint|{lineitem}block__offset__inside__file,INPUT__FILE__NAME: string|{lineitem}input__file__name,ROW__ID: struct|{lineitem}row__id,RAW__DATA__SIZE: bigint|{lineitem}raw__data__size) 2018-01-05T11:29:01,762 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(l_orderkey: int|{lineitem}l_orderkey,l_partkey: int|{lineitem}l_partkey,l_suppkey: int|{lineitem}l_suppkey,l_linenumber: int|{lineitem}l_linenumber,l_quantity: double|{lineitem}l_quantity,l_extendedprice: double|{lineitem}l_extendedprice,l_discount: double|{lineitem}l_discount,l_tax: double|{lineitem}l_tax,l_returnflag: string|{lineitem}l_returnflag,l_linestatus: string|{lineitem}l_linestatus,l_shipdate: string|{lineitem}l_shipdate,l_commitdate: string|{lineitem}l_commitdate,l_receiptdate: string|{lineitem}l_receiptdate,l_shipinstruct: string|{lineitem}l_shipinstruct,l_shipmode: string|{lineitem}l_shipmode,l_comment: string|{lineitem}l_comment,BLOCK__OFFSET__INSIDE__FILE: bigint|{lineitem}block__offset__inside__file,INPUT__FILE__NAME: string|{lineitem}input__file__name,ROW__ID: struct|{lineitem}row__id,RAW__DATA__SIZE: bigint|{lineitem}raw__data__size) 2018-01-05T11:29:01,762 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:29:01,762 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:29:01,762 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[l_orderkey] Column[l_partkey] Column[l_suppkey] Column[l_linenumber] Column[l_quantity] Column[l_extendedprice] Column[l_discount] Column[l_tax] Column[l_returnflag] Column[l_linestatus] Column[l_shipdate] Column[l_commitdate] Column[l_receiptdate] Column[l_shipinstruct] Column[l_shipmode] Column[l_comment] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col l_orderkey) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col l_partkey) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col l_suppkey) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col l_linenumber) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col l_quantity) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col l_extendedprice) 'hll'),_col6: struct|{}(tok_function compute_stats (tok_table_or_col l_discount) 'hll'),_col7: struct|{}(tok_function compute_stats (tok_table_or_col l_tax) 'hll'),_col8: struct|{}(tok_function compute_stats (tok_table_or_col l_returnflag) 'hll'),_col9: struct|{}(tok_function compute_stats (tok_table_or_col l_linestatus) 'hll'),_col10: struct|{}(tok_function compute_stats (tok_table_or_col l_shipdate) 'hll'),_col11: struct|{}(tok_function compute_stats (tok_table_or_col l_commitdate) 'hll'),_col12: struct|{}(tok_function compute_stats (tok_table_or_col l_receiptdate) 'hll'),_col13: struct|{}(tok_function compute_stats (tok_table_or_col l_shipinstruct) 'hll'),_col14: struct|{}(tok_function compute_stats (tok_table_or_col l_shipmode) 'hll'),_col15: struct|{}(tok_function compute_stats (tok_table_or_col l_comment) 'hll')) 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col l_orderkey) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col l_partkey) 'hll'),VALUE._col2: struct|{}(tok_function compute_stats (tok_table_or_col l_suppkey) 'hll'),VALUE._col3: struct|{}(tok_function compute_stats (tok_table_or_col l_linenumber) 'hll'),VALUE._col4: struct|{}(tok_function compute_stats (tok_table_or_col l_quantity) 'hll'),VALUE._col5: struct|{}(tok_function compute_stats (tok_table_or_col l_extendedprice) 'hll'),VALUE._col6: struct|{}(tok_function compute_stats (tok_table_or_col l_discount) 'hll'),VALUE._col7: struct|{}(tok_function compute_stats (tok_table_or_col l_tax) 'hll'),VALUE._col8: struct|{}(tok_function compute_stats (tok_table_or_col l_returnflag) 'hll'),VALUE._col9: struct|{}(tok_function compute_stats (tok_table_or_col l_linestatus) 'hll'),VALUE._col10: struct|{}(tok_function compute_stats (tok_table_or_col l_shipdate) 'hll'),VALUE._col11: struct|{}(tok_function compute_stats (tok_table_or_col l_commitdate) 'hll'),VALUE._col12: struct|{}(tok_function compute_stats (tok_table_or_col l_receiptdate) 'hll'),VALUE._col13: struct|{}(tok_function compute_stats (tok_table_or_col l_shipinstruct) 'hll'),VALUE._col14: struct|{}(tok_function compute_stats (tok_table_or_col l_shipmode) 'hll'),VALUE._col15: struct|{}(tok_function compute_stats (tok_table_or_col l_comment) 'hll')) 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col l_orderkey) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col l_partkey) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col l_suppkey) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col l_linenumber) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col l_quantity) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col l_extendedprice) 'hll'),_col6: struct|{}(tok_function compute_stats (tok_table_or_col l_discount) 'hll'),_col7: struct|{}(tok_function compute_stats (tok_table_or_col l_tax) 'hll'),_col8: struct|{}(tok_function compute_stats (tok_table_or_col l_returnflag) 'hll'),_col9: struct|{}(tok_function compute_stats (tok_table_or_col l_linestatus) 'hll'),_col10: struct|{}(tok_function compute_stats (tok_table_or_col l_shipdate) 'hll'),_col11: struct|{}(tok_function compute_stats (tok_table_or_col l_commitdate) 'hll'),_col12: struct|{}(tok_function compute_stats (tok_table_or_col l_receiptdate) 'hll'),_col13: struct|{}(tok_function compute_stats (tok_table_or_col l_shipinstruct) 'hll'),_col14: struct|{}(tok_function compute_stats (tok_table_or_col l_shipmode) 'hll'),_col15: struct|{}(tok_function compute_stats (tok_table_or_col l_comment) 'hll')) 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1,_col2: struct|{null}_c2,_col3: struct|{null}_c3,_col4: struct|{null}_c4,_col5: struct|{null}_c5,_col6: struct|{null}_c6,_col7: struct|{null}_c7,_col8: struct|{null}_c8,_col9: struct|{null}_c9,_col10: struct|{null}_c10,_col11: struct|{null}_c11,_col12: struct|{null}_c12,_col13: struct|{null}_c13,_col14: struct|{null}_c14,_col15: struct|{null}_c15) 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0] Column[_col1] Column[_col2] Column[_col3] Column[_col4] Column[_col5] Column[_col6] Column[_col7] Column[_col8] Column[_col9] Column[_col10] Column[_col11] Column[_col12] Column[_col13] Column[_col14] Column[_col15]) 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{},_col2: struct|{},_col3: struct|{},_col4: struct|{},_col5: struct|{},_col6: struct|{},_col7: struct|{},_col8: struct|{},_col9: struct|{},_col10: struct|{},_col11: struct|{},_col12: struct|{},_col13: struct|{},_col14: struct|{},_col15: struct|{}) 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,763 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for FS(6) 2018-01-05T11:29:01,763 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(5) 2018-01-05T11:29:01,763 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for GBY(4) 2018-01-05T11:29:01,763 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for RS(3) 2018-01-05T11:29:01,763 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for GBY(2) 2018-01-05T11:29:01,763 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for SEL(1) 2018-01-05T11:29:01,763 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ppd.PredicatePushDown: After PPD: TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-SEL[5]-FS[6] 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(l_orderkey: int|{lineitem}l_orderkey,l_partkey: int|{lineitem}l_partkey,l_suppkey: int|{lineitem}l_suppkey,l_linenumber: int|{lineitem}l_linenumber,l_quantity: double|{lineitem}l_quantity,l_extendedprice: double|{lineitem}l_extendedprice,l_discount: double|{lineitem}l_discount,l_tax: double|{lineitem}l_tax,l_returnflag: string|{lineitem}l_returnflag,l_linestatus: string|{lineitem}l_linestatus,l_shipdate: string|{lineitem}l_shipdate,l_commitdate: string|{lineitem}l_commitdate,l_receiptdate: string|{lineitem}l_receiptdate,l_shipinstruct: string|{lineitem}l_shipinstruct,l_shipmode: string|{lineitem}l_shipmode,l_comment: string|{lineitem}l_comment,BLOCK__OFFSET__INSIDE__FILE: bigint|{lineitem}block__offset__inside__file,INPUT__FILE__NAME: string|{lineitem}input__file__name,ROW__ID: struct|{lineitem}row__id,RAW__DATA__SIZE: bigint|{lineitem}raw__data__size) 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(l_orderkey: int|{lineitem}l_orderkey,l_partkey: int|{lineitem}l_partkey,l_suppkey: int|{lineitem}l_suppkey,l_linenumber: int|{lineitem}l_linenumber,l_quantity: double|{lineitem}l_quantity,l_extendedprice: double|{lineitem}l_extendedprice,l_discount: double|{lineitem}l_discount,l_tax: double|{lineitem}l_tax,l_returnflag: string|{lineitem}l_returnflag,l_linestatus: string|{lineitem}l_linestatus,l_shipdate: string|{lineitem}l_shipdate,l_commitdate: string|{lineitem}l_commitdate,l_receiptdate: string|{lineitem}l_receiptdate,l_shipinstruct: string|{lineitem}l_shipinstruct,l_shipmode: string|{lineitem}l_shipmode,l_comment: string|{lineitem}l_comment,BLOCK__OFFSET__INSIDE__FILE: bigint|{lineitem}block__offset__inside__file,INPUT__FILE__NAME: string|{lineitem}input__file__name,ROW__ID: struct|{lineitem}row__id,RAW__DATA__SIZE: bigint|{lineitem}raw__data__size) 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[l_orderkey] Column[l_partkey] Column[l_suppkey] Column[l_linenumber] Column[l_quantity] Column[l_extendedprice] Column[l_discount] Column[l_tax] Column[l_returnflag] Column[l_linestatus] Column[l_shipdate] Column[l_commitdate] Column[l_receiptdate] Column[l_shipinstruct] Column[l_shipmode] Column[l_comment] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col l_orderkey) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col l_partkey) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col l_suppkey) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col l_linenumber) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col l_quantity) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col l_extendedprice) 'hll'),_col6: struct|{}(tok_function compute_stats (tok_table_or_col l_discount) 'hll'),_col7: struct|{}(tok_function compute_stats (tok_table_or_col l_tax) 'hll'),_col8: struct|{}(tok_function compute_stats (tok_table_or_col l_returnflag) 'hll'),_col9: struct|{}(tok_function compute_stats (tok_table_or_col l_linestatus) 'hll'),_col10: struct|{}(tok_function compute_stats (tok_table_or_col l_shipdate) 'hll'),_col11: struct|{}(tok_function compute_stats (tok_table_or_col l_commitdate) 'hll'),_col12: struct|{}(tok_function compute_stats (tok_table_or_col l_receiptdate) 'hll'),_col13: struct|{}(tok_function compute_stats (tok_table_or_col l_shipinstruct) 'hll'),_col14: struct|{}(tok_function compute_stats (tok_table_or_col l_shipmode) 'hll'),_col15: struct|{}(tok_function compute_stats (tok_table_or_col l_comment) 'hll')) 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col l_orderkey) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col l_partkey) 'hll'),VALUE._col2: struct|{}(tok_function compute_stats (tok_table_or_col l_suppkey) 'hll'),VALUE._col3: struct|{}(tok_function compute_stats (tok_table_or_col l_linenumber) 'hll'),VALUE._col4: struct|{}(tok_function compute_stats (tok_table_or_col l_quantity) 'hll'),VALUE._col5: struct|{}(tok_function compute_stats (tok_table_or_col l_extendedprice) 'hll'),VALUE._col6: struct|{}(tok_function compute_stats (tok_table_or_col l_discount) 'hll'),VALUE._col7: struct|{}(tok_function compute_stats (tok_table_or_col l_tax) 'hll'),VALUE._col8: struct|{}(tok_function compute_stats (tok_table_or_col l_returnflag) 'hll'),VALUE._col9: struct|{}(tok_function compute_stats (tok_table_or_col l_linestatus) 'hll'),VALUE._col10: struct|{}(tok_function compute_stats (tok_table_or_col l_shipdate) 'hll'),VALUE._col11: struct|{}(tok_function compute_stats (tok_table_or_col l_commitdate) 'hll'),VALUE._col12: struct|{}(tok_function compute_stats (tok_table_or_col l_receiptdate) 'hll'),VALUE._col13: struct|{}(tok_function compute_stats (tok_table_or_col l_shipinstruct) 'hll'),VALUE._col14: struct|{}(tok_function compute_stats (tok_table_or_col l_shipmode) 'hll'),VALUE._col15: struct|{}(tok_function compute_stats (tok_table_or_col l_comment) 'hll')) 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col l_orderkey) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col l_partkey) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col l_suppkey) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col l_linenumber) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col l_quantity) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col l_extendedprice) 'hll'),_col6: struct|{}(tok_function compute_stats (tok_table_or_col l_discount) 'hll'),_col7: struct|{}(tok_function compute_stats (tok_table_or_col l_tax) 'hll'),_col8: struct|{}(tok_function compute_stats (tok_table_or_col l_returnflag) 'hll'),_col9: struct|{}(tok_function compute_stats (tok_table_or_col l_linestatus) 'hll'),_col10: struct|{}(tok_function compute_stats (tok_table_or_col l_shipdate) 'hll'),_col11: struct|{}(tok_function compute_stats (tok_table_or_col l_commitdate) 'hll'),_col12: struct|{}(tok_function compute_stats (tok_table_or_col l_receiptdate) 'hll'),_col13: struct|{}(tok_function compute_stats (tok_table_or_col l_shipinstruct) 'hll'),_col14: struct|{}(tok_function compute_stats (tok_table_or_col l_shipmode) 'hll'),_col15: struct|{}(tok_function compute_stats (tok_table_or_col l_comment) 'hll')) 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1,_col2: struct|{null}_c2,_col3: struct|{null}_c3,_col4: struct|{null}_c4,_col5: struct|{null}_c5,_col6: struct|{null}_c6,_col7: struct|{null}_c7,_col8: struct|{null}_c8,_col9: struct|{null}_c9,_col10: struct|{null}_c10,_col11: struct|{null}_c11,_col12: struct|{null}_c12,_col13: struct|{null}_c13,_col14: struct|{null}_c14,_col15: struct|{null}_c15) 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0] Column[_col1] Column[_col2] Column[_col3] Column[_col4] Column[_col5] Column[_col6] Column[_col7] Column[_col8] Column[_col9] Column[_col10] Column[_col11] Column[_col12] Column[_col13] Column[_col14] Column[_col15]) 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{},_col2: struct|{},_col3: struct|{},_col4: struct|{},_col5: struct|{},_col6: struct|{},_col7: struct|{},_col8: struct|{},_col9: struct|{},_col10: struct|{},_col11: struct|{},_col12: struct|{},_col13: struct|{},_col14: struct|{},_col15: struct|{}) 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(l_orderkey: int|{lineitem}l_orderkey,l_partkey: int|{lineitem}l_partkey,l_suppkey: int|{lineitem}l_suppkey,l_linenumber: int|{lineitem}l_linenumber,l_quantity: double|{lineitem}l_quantity,l_extendedprice: double|{lineitem}l_extendedprice,l_discount: double|{lineitem}l_discount,l_tax: double|{lineitem}l_tax,l_returnflag: string|{lineitem}l_returnflag,l_linestatus: string|{lineitem}l_linestatus,l_shipdate: string|{lineitem}l_shipdate,l_commitdate: string|{lineitem}l_commitdate,l_receiptdate: string|{lineitem}l_receiptdate,l_shipinstruct: string|{lineitem}l_shipinstruct,l_shipmode: string|{lineitem}l_shipmode,l_comment: string|{lineitem}l_comment,BLOCK__OFFSET__INSIDE__FILE: bigint|{lineitem}block__offset__inside__file,INPUT__FILE__NAME: string|{lineitem}input__file__name,ROW__ID: struct|{lineitem}row__id,RAW__DATA__SIZE: bigint|{lineitem}raw__data__size) 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(l_orderkey: int|{lineitem}l_orderkey,l_partkey: int|{lineitem}l_partkey,l_suppkey: int|{lineitem}l_suppkey,l_linenumber: int|{lineitem}l_linenumber,l_quantity: double|{lineitem}l_quantity,l_extendedprice: double|{lineitem}l_extendedprice,l_discount: double|{lineitem}l_discount,l_tax: double|{lineitem}l_tax,l_returnflag: string|{lineitem}l_returnflag,l_linestatus: string|{lineitem}l_linestatus,l_shipdate: string|{lineitem}l_shipdate,l_commitdate: string|{lineitem}l_commitdate,l_receiptdate: string|{lineitem}l_receiptdate,l_shipinstruct: string|{lineitem}l_shipinstruct,l_shipmode: string|{lineitem}l_shipmode,l_comment: string|{lineitem}l_comment,BLOCK__OFFSET__INSIDE__FILE: bigint|{lineitem}block__offset__inside__file,INPUT__FILE__NAME: string|{lineitem}input__file__name,ROW__ID: struct|{lineitem}row__id,RAW__DATA__SIZE: bigint|{lineitem}raw__data__size) 2018-01-05T11:29:01,763 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:29:01,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:29:01,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[l_orderkey] Column[l_partkey] Column[l_suppkey] Column[l_linenumber] Column[l_quantity] Column[l_extendedprice] Column[l_discount] Column[l_tax] Column[l_returnflag] Column[l_linestatus] Column[l_shipdate] Column[l_commitdate] Column[l_receiptdate] Column[l_shipinstruct] Column[l_shipmode] Column[l_comment] Column[BLOCK__OFFSET__INSIDE__FILE] Column[INPUT__FILE__NAME] Column[ROW__ID] Column[RAW__DATA__SIZE]) 2018-01-05T11:29:01,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col l_orderkey) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col l_partkey) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col l_suppkey) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col l_linenumber) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col l_quantity) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col l_extendedprice) 'hll'),_col6: struct|{}(tok_function compute_stats (tok_table_or_col l_discount) 'hll'),_col7: struct|{}(tok_function compute_stats (tok_table_or_col l_tax) 'hll'),_col8: struct|{}(tok_function compute_stats (tok_table_or_col l_returnflag) 'hll'),_col9: struct|{}(tok_function compute_stats (tok_table_or_col l_linestatus) 'hll'),_col10: struct|{}(tok_function compute_stats (tok_table_or_col l_shipdate) 'hll'),_col11: struct|{}(tok_function compute_stats (tok_table_or_col l_commitdate) 'hll'),_col12: struct|{}(tok_function compute_stats (tok_table_or_col l_receiptdate) 'hll'),_col13: struct|{}(tok_function compute_stats (tok_table_or_col l_shipinstruct) 'hll'),_col14: struct|{}(tok_function compute_stats (tok_table_or_col l_shipmode) 'hll'),_col15: struct|{}(tok_function compute_stats (tok_table_or_col l_comment) 'hll')) 2018-01-05T11:29:01,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:29:01,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:29:01,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col l_orderkey) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col l_partkey) 'hll'),VALUE._col2: struct|{}(tok_function compute_stats (tok_table_or_col l_suppkey) 'hll'),VALUE._col3: struct|{}(tok_function compute_stats (tok_table_or_col l_linenumber) 'hll'),VALUE._col4: struct|{}(tok_function compute_stats (tok_table_or_col l_quantity) 'hll'),VALUE._col5: struct|{}(tok_function compute_stats (tok_table_or_col l_extendedprice) 'hll'),VALUE._col6: struct|{}(tok_function compute_stats (tok_table_or_col l_discount) 'hll'),VALUE._col7: struct|{}(tok_function compute_stats (tok_table_or_col l_tax) 'hll'),VALUE._col8: struct|{}(tok_function compute_stats (tok_table_or_col l_returnflag) 'hll'),VALUE._col9: struct|{}(tok_function compute_stats (tok_table_or_col l_linestatus) 'hll'),VALUE._col10: struct|{}(tok_function compute_stats (tok_table_or_col l_shipdate) 'hll'),VALUE._col11: struct|{}(tok_function compute_stats (tok_table_or_col l_commitdate) 'hll'),VALUE._col12: struct|{}(tok_function compute_stats (tok_table_or_col l_receiptdate) 'hll'),VALUE._col13: struct|{}(tok_function compute_stats (tok_table_or_col l_shipinstruct) 'hll'),VALUE._col14: struct|{}(tok_function compute_stats (tok_table_or_col l_shipmode) 'hll'),VALUE._col15: struct|{}(tok_function compute_stats (tok_table_or_col l_comment) 'hll')) 2018-01-05T11:29:01,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:29:01,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:29:01,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col l_orderkey) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col l_partkey) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col l_suppkey) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col l_linenumber) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col l_quantity) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col l_extendedprice) 'hll'),_col6: struct|{}(tok_function compute_stats (tok_table_or_col l_discount) 'hll'),_col7: struct|{}(tok_function compute_stats (tok_table_or_col l_tax) 'hll'),_col8: struct|{}(tok_function compute_stats (tok_table_or_col l_returnflag) 'hll'),_col9: struct|{}(tok_function compute_stats (tok_table_or_col l_linestatus) 'hll'),_col10: struct|{}(tok_function compute_stats (tok_table_or_col l_shipdate) 'hll'),_col11: struct|{}(tok_function compute_stats (tok_table_or_col l_commitdate) 'hll'),_col12: struct|{}(tok_function compute_stats (tok_table_or_col l_receiptdate) 'hll'),_col13: struct|{}(tok_function compute_stats (tok_table_or_col l_shipinstruct) 'hll'),_col14: struct|{}(tok_function compute_stats (tok_table_or_col l_shipmode) 'hll'),_col15: struct|{}(tok_function compute_stats (tok_table_or_col l_comment) 'hll')) 2018-01-05T11:29:01,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:29:01,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:29:01,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1,_col2: struct|{null}_c2,_col3: struct|{null}_c3,_col4: struct|{null}_c4,_col5: struct|{null}_c5,_col6: struct|{null}_c6,_col7: struct|{null}_c7,_col8: struct|{null}_c8,_col9: struct|{null}_c9,_col10: struct|{null}_c10,_col11: struct|{null}_c11,_col12: struct|{null}_c12,_col13: struct|{null}_c13,_col14: struct|{null}_c14,_col15: struct|{null}_c15) 2018-01-05T11:29:01,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:29:01,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:29:01,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0] Column[_col1] Column[_col2] Column[_col3] Column[_col4] Column[_col5] Column[_col6] Column[_col7] Column[_col8] Column[_col9] Column[_col10] Column[_col11] Column[_col12] Column[_col13] Column[_col14] Column[_col15]) 2018-01-05T11:29:01,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{},_col2: struct|{},_col3: struct|{},_col4: struct|{},_col5: struct|{},_col6: struct|{},_col7: struct|{},_col8: struct|{},_col9: struct|{},_col10: struct|{},_col11: struct|{},_col12: struct|{},_col13: struct|{},_col14: struct|{},_col15: struct|{}) 2018-01-05T11:29:01,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:29:01,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:29:01,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 3 key:[] 2018-01-05T11:29:01,764 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 3 oldColExprMap: {VALUE._col10=Column[_col10], VALUE._col11=Column[_col11], VALUE._col12=Column[_col12], VALUE._col2=Column[_col2], VALUE._col3=Column[_col3], VALUE._col4=Column[_col4], VALUE._col5=Column[_col5], VALUE._col0=Column[_col0], VALUE._col1=Column[_col1], VALUE._col13=Column[_col13], VALUE._col14=Column[_col14], VALUE._col15=Column[_col15], VALUE._col6=Column[_col6], VALUE._col7=Column[_col7], VALUE._col8=Column[_col8], VALUE._col9=Column[_col9]} 2018-01-05T11:29:01,764 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ColumnPrunerProcFactory: RS 3 newColExprMap: {VALUE._col10=Column[_col10], VALUE._col11=Column[_col11], VALUE._col12=Column[_col12], VALUE._col2=Column[_col2], VALUE._col3=Column[_col3], VALUE._col4=Column[_col4], VALUE._col5=Column[_col5], VALUE._col0=Column[_col0], VALUE._col1=Column[_col1], VALUE._col13=Column[_col13], VALUE._col14=Column[_col14], VALUE._col15=Column[_col15], VALUE._col6=Column[_col6], VALUE._col7=Column[_col7], VALUE._col8=Column[_col8], VALUE._col9=Column[_col9]} 2018-01-05T11:29:01,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,764 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,765 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,765 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,765 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,765 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,765 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,765 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,765 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,765 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,765 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,765 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,765 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,765 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,765 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,765 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,765 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.IdentityProjectRemover: Identity project remover optimization removed : SEL[5] 2018-01-05T11:29:01,765 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,765 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,765 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,765 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,765 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,765 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,765 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,765 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: After logical optimization TS[0]-SEL[1]-GBY[2]-RS[3]-GBY[4]-FS[6] 2018-01-05T11:29:01,765 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,765 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,765 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,765 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,765 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,765 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,765 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=lineitem 2018-01-05T11:29:01,765 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=lineitem 2018-01-05T11:29:01,770 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Direct SQL query in 2.720929ms + 0.00869ms, the query is [select "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "TAB_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (...)] 2018-01-05T11:29:01,771 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:01,780 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:01,780 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:01,780 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting stats (Num rows: 100 Data size: 145920 Basic stats: COMPLETE Column stats: NONE) on TS[0] 2018-01-05T11:29:01,781 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (lineitem): numRows: 100 dataSize: 145920 basicStatsState: COMPLETE colStatsState: NONE colStats: {l_returnflag= colName: l_returnflag colType: string countDistincts: 20 numNulls: 5 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, l_receiptdate= colName: l_receiptdate colType: string countDistincts: 20 numNulls: 5 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, l_tax= colName: l_tax colType: double countDistincts: 20 numNulls: 5 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: 4.9E-324 max: 1.7976931348623157E308 ] isPrimaryKey: false isEstimated: true, l_shipmode= colName: l_shipmode colType: string countDistincts: 20 numNulls: 5 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, l_suppkey= colName: l_suppkey colType: int countDistincts: 20 numNulls: 5 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, l_shipdate= colName: l_shipdate colType: string countDistincts: 20 numNulls: 5 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, l_commitdate= colName: l_commitdate colType: string countDistincts: 20 numNulls: 5 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, l_partkey= colName: l_partkey colType: int countDistincts: 20 numNulls: 5 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, l_orderkey= colName: l_orderkey colType: int countDistincts: 20 numNulls: 5 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, l_quantity= colName: l_quantity colType: double countDistincts: 20 numNulls: 5 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: 4.9E-324 max: 1.7976931348623157E308 ] isPrimaryKey: false isEstimated: true, l_linestatus= colName: l_linestatus colType: string countDistincts: 20 numNulls: 5 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, l_comment= colName: l_comment colType: string countDistincts: 20 numNulls: 5 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, l_extendedprice= colName: l_extendedprice colType: double countDistincts: 20 numNulls: 5 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: 4.9E-324 max: 1.7976931348623157E308 ] isPrimaryKey: false isEstimated: true, l_linenumber= colName: l_linenumber colType: int countDistincts: 20 numNulls: 5 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, l_discount= colName: l_discount colType: double countDistincts: 20 numNulls: 5 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: 4.9E-324 max: 1.7976931348623157E308 ] isPrimaryKey: false isEstimated: true, l_shipinstruct= colName: l_shipinstruct colType: string countDistincts: 20 numNulls: 5 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true} 2018-01-05T11:29:01,781 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting stats (Num rows: 100 Data size: 145920 Basic stats: COMPLETE Column stats: NONE) on SEL[1] 2018-01-05T11:29:01,781 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [1] STATS-SEL[1]: numRows: 100 dataSize: 145920 basicStatsState: COMPLETE colStatsState: NONE colStats: {l_returnflag= colName: l_returnflag colType: string countDistincts: 20 numNulls: 5 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, l_receiptdate= colName: l_receiptdate colType: string countDistincts: 20 numNulls: 5 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, l_tax= colName: l_tax colType: double countDistincts: 20 numNulls: 5 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: 4.9E-324 max: 1.7976931348623157E308 ] isPrimaryKey: false isEstimated: true, l_shipmode= colName: l_shipmode colType: string countDistincts: 20 numNulls: 5 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, l_suppkey= colName: l_suppkey colType: int countDistincts: 20 numNulls: 5 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, l_shipdate= colName: l_shipdate colType: string countDistincts: 20 numNulls: 5 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, l_commitdate= colName: l_commitdate colType: string countDistincts: 20 numNulls: 5 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, l_partkey= colName: l_partkey colType: int countDistincts: 20 numNulls: 5 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, l_orderkey= colName: l_orderkey colType: int countDistincts: 20 numNulls: 5 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, l_quantity= colName: l_quantity colType: double countDistincts: 20 numNulls: 5 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: 4.9E-324 max: 1.7976931348623157E308 ] isPrimaryKey: false isEstimated: true, l_linestatus= colName: l_linestatus colType: string countDistincts: 20 numNulls: 5 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, l_comment= colName: l_comment colType: string countDistincts: 20 numNulls: 5 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, l_extendedprice= colName: l_extendedprice colType: double countDistincts: 20 numNulls: 5 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: 4.9E-324 max: 1.7976931348623157E308 ] isPrimaryKey: false isEstimated: true, l_linenumber= colName: l_linenumber colType: int countDistincts: 20 numNulls: 5 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, l_discount= colName: l_discount colType: double countDistincts: 20 numNulls: 5 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: 4.9E-324 max: 1.7976931348623157E308 ] isPrimaryKey: false isEstimated: true, l_shipinstruct= colName: l_shipinstruct colType: string countDistincts: 20 numNulls: 5 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true} 2018-01-05T11:29:01,781 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[2]: inputSize: 145920 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:01,781 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [Case 1] STATS-GBY[2]: cardinality: 100 2018-01-05T11:29:01,781 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 8432 Basic stats: COMPLETE Column stats: NONE) on GBY[2] 2018-01-05T11:29:01,782 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-GBY[2]: numRows: 1 dataSize: 8432 basicStatsState: COMPLETE colStatsState: NONE colStats: {l_returnflag= colName: l_returnflag colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, l_tax= colName: l_tax colType: double countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: 4.9E-324 max: 1.7976931348623157E308 ] isPrimaryKey: false isEstimated: true, l_shipmode= colName: l_shipmode colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, l_suppkey= colName: l_suppkey colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, l_shipdate= colName: l_shipdate colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, l_commitdate= colName: l_commitdate colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, l_partkey= colName: l_partkey colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, l_linestatus= colName: l_linestatus colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, l_comment= colName: l_comment colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, l_discount= colName: l_discount colType: double countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: 4.9E-324 max: 1.7976931348623157E308 ] isPrimaryKey: false isEstimated: true, _col13= colName: _col13 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col12= colName: _col12 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col11= colName: _col11 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, l_receiptdate= colName: l_receiptdate colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, _col10= colName: _col10 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col15= colName: _col15 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col14= colName: _col14 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col9= colName: _col9 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, l_orderkey= colName: l_orderkey colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, l_quantity= colName: l_quantity colType: double countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: 4.9E-324 max: 1.7976931348623157E308 ] isPrimaryKey: false isEstimated: true, _col2= colName: _col2 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, l_extendedprice= colName: l_extendedprice colType: double countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: 4.9E-324 max: 1.7976931348623157E308 ] isPrimaryKey: false isEstimated: true, _col4= colName: _col4 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, l_linenumber= colName: l_linenumber colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, _col6= colName: _col6 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, l_shipinstruct= colName: l_shipinstruct colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, _col5= colName: _col5 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col8= colName: _col8 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col7= colName: _col7 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:01,782 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 8432 Basic stats: COMPLETE Column stats: NONE) on RS[3] 2018-01-05T11:29:01,782 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-RS[3]: numRows: 1 dataSize: 8432 basicStatsState: COMPLETE colStatsState: NONE colStats: {l_returnflag= colName: l_returnflag colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, l_tax= colName: l_tax colType: double countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: 4.9E-324 max: 1.7976931348623157E308 ] isPrimaryKey: false isEstimated: true, l_shipmode= colName: l_shipmode colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, l_suppkey= colName: l_suppkey colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, l_shipdate= colName: l_shipdate colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, l_commitdate= colName: l_commitdate colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, l_partkey= colName: l_partkey colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, l_linestatus= colName: l_linestatus colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, l_comment= colName: l_comment colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, l_discount= colName: l_discount colType: double countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: 4.9E-324 max: 1.7976931348623157E308 ] isPrimaryKey: false isEstimated: true, _col13= colName: _col13 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col12= colName: _col12 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col11= colName: _col11 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, l_receiptdate= colName: l_receiptdate colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, _col10= colName: _col10 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col15= colName: _col15 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col14= colName: _col14 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col9= colName: _col9 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, l_orderkey= colName: l_orderkey colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, l_quantity= colName: l_quantity colType: double countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: 4.9E-324 max: 1.7976931348623157E308 ] isPrimaryKey: false isEstimated: true, _col2= colName: _col2 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, l_extendedprice= colName: l_extendedprice colType: double countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: 4.9E-324 max: 1.7976931348623157E308 ] isPrimaryKey: false isEstimated: true, _col4= colName: _col4 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, l_linenumber= colName: l_linenumber colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, _col6= colName: _col6 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, l_shipinstruct= colName: l_shipinstruct colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, _col5= colName: _col5 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col8= colName: _col8 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col7= colName: _col7 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:01,782 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[4]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:01,782 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [Case 7] STATS-GBY[4]: cardinality: 0 2018-01-05T11:29:01,782 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: STATS-GBY[4]: Equals 0 in number of rows. 0 rows will be set to 1 2018-01-05T11:29:01,782 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 8560 Basic stats: COMPLETE Column stats: NONE) on GBY[4] 2018-01-05T11:29:01,782 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-GBY[4]: numRows: 1 dataSize: 8560 basicStatsState: COMPLETE colStatsState: NONE colStats: {l_returnflag= colName: l_returnflag colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, l_tax= colName: l_tax colType: double countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: 4.9E-324 max: 1.7976931348623157E308 ] isPrimaryKey: false isEstimated: true, l_shipmode= colName: l_shipmode colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, l_suppkey= colName: l_suppkey colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, l_shipdate= colName: l_shipdate colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, l_commitdate= colName: l_commitdate colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, l_partkey= colName: l_partkey colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, l_linestatus= colName: l_linestatus colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, l_comment= colName: l_comment colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, l_discount= colName: l_discount colType: double countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: 4.9E-324 max: 1.7976931348623157E308 ] isPrimaryKey: false isEstimated: true, _col13= colName: _col13 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col12= colName: _col12 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col11= colName: _col11 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, l_receiptdate= colName: l_receiptdate colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, _col10= colName: _col10 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col15= colName: _col15 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col14= colName: _col14 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col9= colName: _col9 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, l_orderkey= colName: l_orderkey colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, l_quantity= colName: l_quantity colType: double countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: 4.9E-324 max: 1.7976931348623157E308 ] isPrimaryKey: false isEstimated: true, _col2= colName: _col2 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, l_extendedprice= colName: l_extendedprice colType: double countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: 4.9E-324 max: 1.7976931348623157E308 ] isPrimaryKey: false isEstimated: true, _col4= colName: _col4 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, l_linenumber= colName: l_linenumber colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, _col6= colName: _col6 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, l_shipinstruct= colName: l_shipinstruct colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, _col5= colName: _col5 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col8= colName: _col8 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col7= colName: _col7 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:01,782 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] annotation.StatsRulesProcFactory: [0] STATS-FS[6]: numRows: 1 dataSize: 8560 basicStatsState: COMPLETE colStatsState: NONE colStats: {l_returnflag= colName: l_returnflag colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, l_tax= colName: l_tax colType: double countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: 4.9E-324 max: 1.7976931348623157E308 ] isPrimaryKey: false isEstimated: true, l_shipmode= colName: l_shipmode colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, l_suppkey= colName: l_suppkey colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, l_shipdate= colName: l_shipdate colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, l_commitdate= colName: l_commitdate colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, l_partkey= colName: l_partkey colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, l_linestatus= colName: l_linestatus colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, l_comment= colName: l_comment colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, l_discount= colName: l_discount colType: double countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: 4.9E-324 max: 1.7976931348623157E308 ] isPrimaryKey: false isEstimated: true, _col13= colName: _col13 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col12= colName: _col12 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col11= colName: _col11 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, l_receiptdate= colName: l_receiptdate colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, _col10= colName: _col10 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col0= colName: _col0 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col15= colName: _col15 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col14= colName: _col14 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col9= colName: _col9 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, l_orderkey= colName: l_orderkey colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, l_quantity= colName: l_quantity colType: double countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: 4.9E-324 max: 1.7976931348623157E308 ] isPrimaryKey: false isEstimated: true, _col2= colName: _col2 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, l_extendedprice= colName: l_extendedprice colType: double countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 Range: [ min: 4.9E-324 max: 1.7976931348623157E308 ] isPrimaryKey: false isEstimated: true, _col4= colName: _col4 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, l_linenumber= colName: l_linenumber colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true, _col6= colName: _col6 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, l_shipinstruct= colName: l_shipinstruct colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, _col5= colName: _col5 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col8= colName: _col8 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col7= colName: _col7 colType: struct countDistincts: 1 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.TableScanOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on TS[0] 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SelectOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on SEL[1] 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[2] 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[3] 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[4] 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FileSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on FS[6] 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,783 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: FS, 6 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: GBY, 4 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: RS, 3 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: GBY, 2 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: SEL, 1 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Component: 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:29:01,783 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Cycle free: true 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(l_orderkey: int|{lineitem}l_orderkey,l_partkey: int|{lineitem}l_partkey,l_suppkey: int|{lineitem}l_suppkey,l_linenumber: int|{lineitem}l_linenumber,l_quantity: double|{lineitem}l_quantity,l_extendedprice: double|{lineitem}l_extendedprice,l_discount: double|{lineitem}l_discount,l_tax: double|{lineitem}l_tax,l_returnflag: string|{lineitem}l_returnflag,l_linestatus: string|{lineitem}l_linestatus,l_shipdate: string|{lineitem}l_shipdate,l_commitdate: string|{lineitem}l_commitdate,l_receiptdate: string|{lineitem}l_receiptdate,l_shipinstruct: string|{lineitem}l_shipinstruct,l_shipmode: string|{lineitem}l_shipmode,l_comment: string|{lineitem}l_comment,BLOCK__OFFSET__INSIDE__FILE: bigint|{lineitem}block__offset__inside__file,INPUT__FILE__NAME: string|{lineitem}input__file__name,ROW__ID: struct|{lineitem}row__id,RAW__DATA__SIZE: bigint|{lineitem}raw__data__size) 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(l_orderkey: int|{lineitem}l_orderkey,l_partkey: int|{lineitem}l_partkey,l_suppkey: int|{lineitem}l_suppkey,l_linenumber: int|{lineitem}l_linenumber,l_quantity: double|{lineitem}l_quantity,l_extendedprice: double|{lineitem}l_extendedprice,l_discount: double|{lineitem}l_discount,l_tax: double|{lineitem}l_tax,l_returnflag: string|{lineitem}l_returnflag,l_linestatus: string|{lineitem}l_linestatus,l_shipdate: string|{lineitem}l_shipdate,l_commitdate: string|{lineitem}l_commitdate,l_receiptdate: string|{lineitem}l_receiptdate,l_shipinstruct: string|{lineitem}l_shipinstruct,l_shipmode: string|{lineitem}l_shipmode,l_comment: string|{lineitem}l_comment) 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcFactory: New column list:(Column[l_orderkey] Column[l_partkey] Column[l_suppkey] Column[l_linenumber] Column[l_quantity] Column[l_extendedprice] Column[l_discount] Column[l_tax] Column[l_returnflag] Column[l_linestatus] Column[l_shipdate] Column[l_commitdate] Column[l_receiptdate] Column[l_shipinstruct] Column[l_shipmode] Column[l_comment]) 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col l_orderkey) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col l_partkey) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col l_suppkey) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col l_linenumber) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col l_quantity) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col l_extendedprice) 'hll'),_col6: struct|{}(tok_function compute_stats (tok_table_or_col l_discount) 'hll'),_col7: struct|{}(tok_function compute_stats (tok_table_or_col l_tax) 'hll'),_col8: struct|{}(tok_function compute_stats (tok_table_or_col l_returnflag) 'hll'),_col9: struct|{}(tok_function compute_stats (tok_table_or_col l_linestatus) 'hll'),_col10: struct|{}(tok_function compute_stats (tok_table_or_col l_shipdate) 'hll'),_col11: struct|{}(tok_function compute_stats (tok_table_or_col l_commitdate) 'hll'),_col12: struct|{}(tok_function compute_stats (tok_table_or_col l_receiptdate) 'hll'),_col13: struct|{}(tok_function compute_stats (tok_table_or_col l_shipinstruct) 'hll'),_col14: struct|{}(tok_function compute_stats (tok_table_or_col l_shipmode) 'hll'),_col15: struct|{}(tok_function compute_stats (tok_table_or_col l_comment) 'hll')) 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[2] 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col l_orderkey) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col l_partkey) 'hll'),VALUE._col2: struct|{}(tok_function compute_stats (tok_table_or_col l_suppkey) 'hll'),VALUE._col3: struct|{}(tok_function compute_stats (tok_table_or_col l_linenumber) 'hll'),VALUE._col4: struct|{}(tok_function compute_stats (tok_table_or_col l_quantity) 'hll'),VALUE._col5: struct|{}(tok_function compute_stats (tok_table_or_col l_extendedprice) 'hll'),VALUE._col6: struct|{}(tok_function compute_stats (tok_table_or_col l_discount) 'hll'),VALUE._col7: struct|{}(tok_function compute_stats (tok_table_or_col l_tax) 'hll'),VALUE._col8: struct|{}(tok_function compute_stats (tok_table_or_col l_returnflag) 'hll'),VALUE._col9: struct|{}(tok_function compute_stats (tok_table_or_col l_linestatus) 'hll'),VALUE._col10: struct|{}(tok_function compute_stats (tok_table_or_col l_shipdate) 'hll'),VALUE._col11: struct|{}(tok_function compute_stats (tok_table_or_col l_commitdate) 'hll'),VALUE._col12: struct|{}(tok_function compute_stats (tok_table_or_col l_receiptdate) 'hll'),VALUE._col13: struct|{}(tok_function compute_stats (tok_table_or_col l_shipinstruct) 'hll'),VALUE._col14: struct|{}(tok_function compute_stats (tok_table_or_col l_shipmode) 'hll'),VALUE._col15: struct|{}(tok_function compute_stats (tok_table_or_col l_comment) 'hll')) 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {} 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[3] 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: struct|{}(tok_function compute_stats (tok_table_or_col l_orderkey) 'hll'),_col1: struct|{}(tok_function compute_stats (tok_table_or_col l_partkey) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col l_suppkey) 'hll'),_col3: struct|{}(tok_function compute_stats (tok_table_or_col l_linenumber) 'hll'),_col4: struct|{}(tok_function compute_stats (tok_table_or_col l_quantity) 'hll'),_col5: struct|{}(tok_function compute_stats (tok_table_or_col l_extendedprice) 'hll'),_col6: struct|{}(tok_function compute_stats (tok_table_or_col l_discount) 'hll'),_col7: struct|{}(tok_function compute_stats (tok_table_or_col l_tax) 'hll'),_col8: struct|{}(tok_function compute_stats (tok_table_or_col l_returnflag) 'hll'),_col9: struct|{}(tok_function compute_stats (tok_table_or_col l_linestatus) 'hll'),_col10: struct|{}(tok_function compute_stats (tok_table_or_col l_shipdate) 'hll'),_col11: struct|{}(tok_function compute_stats (tok_table_or_col l_commitdate) 'hll'),_col12: struct|{}(tok_function compute_stats (tok_table_or_col l_receiptdate) 'hll'),_col13: struct|{}(tok_function compute_stats (tok_table_or_col l_shipinstruct) 'hll'),_col14: struct|{}(tok_function compute_stats (tok_table_or_col l_shipmode) 'hll'),_col15: struct|{}(tok_function compute_stats (tok_table_or_col l_comment) 'hll')) 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {} 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[4] 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{},_col2: struct|{},_col3: struct|{},_col4: struct|{},_col5: struct|{},_col6: struct|{},_col7: struct|{},_col8: struct|{},_col9: struct|{},_col10: struct|{},_col11: struct|{},_col12: struct|{},_col13: struct|{},_col14: struct|{},_col15: struct|{}) 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {} 2018-01-05T11:29:01,783 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: TS[0] 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: RS[3] 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem of table lineitem 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: RS[3] 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Root operator: GBY[4] 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Leaf operator: FS[6] 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Adding reduce work (Reducer 2) for GBY[4] 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezUtils: Setting up reduce sink: RS[3] with following reduce work: Reducer 2 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: Removing RS[3] as parent from GBY[4] 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.GenTezWork: First pass. Leaf operator: FS[6] 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping vectorization 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:29:01,784 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: llap mode: only 2018-01-05T11:29:01,784 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(l_orderkey, 'hll')' 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'l_orderkey' 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(l_partkey, 'hll')' 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'l_partkey' 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(l_suppkey, 'hll')' 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'l_suppkey' 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(l_linenumber, 'hll')' 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'l_linenumber' 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(l_quantity, 'hll')' 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'l_quantity' 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(l_extendedprice, 'hll')' 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'l_extendedprice' 2018-01-05T11:29:01,784 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(l_discount, 'hll')' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'l_discount' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(l_tax, 'hll')' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'l_tax' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(l_returnflag, 'hll')' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'l_returnflag' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(l_linestatus, 'hll')' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'l_linestatus' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(l_shipdate, 'hll')' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'l_shipdate' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(l_commitdate, 'hll')' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'l_commitdate' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(l_receiptdate, 'hll')' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'l_receiptdate' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(l_shipinstruct, 'hll')' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'l_shipinstruct' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(l_shipmode, 'hll')' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'l_shipmode' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(l_comment, 'hll')' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'l_comment' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'l_orderkey' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'l_partkey' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'l_suppkey' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'l_linenumber' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'l_quantity' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'l_extendedprice' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'l_discount' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'l_tax' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'l_returnflag' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'l_linestatus' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'l_shipdate' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'l_commitdate' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'l_receiptdate' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'l_shipinstruct' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'l_shipmode' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'l_comment' 2018-01-05T11:29:01,785 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@749ad467 can be done in LLAP 2018-01-05T11:29:01,785 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Evaluating work item: Reducer 2 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col0)' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col1)' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col1' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col2)' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col2' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col3)' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col3' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col4)' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col4' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col5)' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col5' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col6)' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col6' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col7)' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col7' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col8)' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col8' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col9)' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col9' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col10)' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col10' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col11)' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col11' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col12)' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col12' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col13)' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col13' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col14)' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col14' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'compute_stats(VALUE._col15)' 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Checking 'VALUE._col15' 2018-01-05T11:29:01,785 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@6a68159e can be done in LLAP 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,785 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Completed plan generation 2018-01-05T11:29:01,785 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: validation start 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: Not a partition. 2018-01-05T11:29:01,785 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] parse.ColumnStatsSemanticAnalyzer: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:29:01,786 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,786 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0, type:struct, comment:null), FieldSchema(name:_c1, type:struct, comment:null), FieldSchema(name:_c2, type:struct, comment:null), FieldSchema(name:_c3, type:struct, comment:null), FieldSchema(name:_c4, type:struct, comment:null), FieldSchema(name:_c5, type:struct, comment:null), FieldSchema(name:_c6, type:struct, comment:null), FieldSchema(name:_c7, type:struct, comment:null), FieldSchema(name:_c8, type:struct, comment:null), FieldSchema(name:_c9, type:struct, comment:null), FieldSchema(name:_c10, type:struct, comment:null), FieldSchema(name:_c11, type:struct, comment:null), FieldSchema(name:_c12, type:struct, comment:null), FieldSchema(name:_c13, type:struct, comment:null), FieldSchema(name:_c14, type:struct, comment:null), FieldSchema(name:_c15, type:struct, comment:null)], properties:null) 2018-01-05T11:29:01,786 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,786 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:01,786 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=6, flushCache_()=0, getTableColumnStatistics_(String, String, List, )=15} 2018-01-05T11:29:01,786 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112901_531e127a-818d-4fb7-ad60-8d20036d1617); Time taken: 0.078 seconds 2018-01-05T11:29:01,786 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:01,786 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,786 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Executing command(queryId=djaiswal_20180105112901_531e127a-818d-4fb7-ad60-8d20036d1617): analyze table lineitem compute statistics for columns 2018-01-05T11:29:01,787 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,787 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: query: analyze table lineitem compute statistics for columns 2018-01-05T11:29:01,787 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:29:01,787 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Input: default@lineitem 2018-01-05T11:29:01,787 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: default@lineitem 2018-01-05T11:29:01,787 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: PREHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_712_4485361284563650037-1/-mr-10000 2018-01-05T11:29:01,787 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,787 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,787 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,787 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Query ID = djaiswal_20180105112901_531e127a-818d-4fb7-ad60-8d20036d1617 2018-01-05T11:29:01,787 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Total jobs = 1 2018-01-05T11:29:01,787 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,787 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:29:01,787 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-0:MAPRED] in serial mode 2018-01-05T11:29:01,790 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000064, QueryId=djaiswal_20180105112901_56be7895-b321-4016-9b72-c1f4d0580d42-32, DagId=32, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180541647, EndTime=1515180541790 2018-01-05T11:29:01,812 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_708_2053710569094398989-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:29:01,833 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:29:01,833 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:29:01,833 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:29:01,833 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112901_531e127a-818d-4fb7-ad60-8d20036d1617 2018-01-05T11:29:01,833 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Session is already open 2018-01-05T11:29:01,833 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:29:01,833 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,833 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Dag name: analyze table lineitem compute sta...columns (Stage-0) 2018-01-05T11:29:01,833 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: DagInfo: {"context":"Hive","description":"\nanalyze table lineitem compute statistics for columns"} 2018-01-05T11:29:01,833 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112901_531e127a-818d-4fb7-ad60-8d20036d1617 with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:29:01,833 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,844 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,844 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:29:01,845 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,845 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 2 size: 3.67KB 2018-01-05T11:29:01,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,860 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,871 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:29:01,871 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,871 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:29:01,872 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,872 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 5.97KB 2018-01-05T11:29:01,877 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_712_4485361284563650037-1/-mr-10001 2018-01-05T11:29:01,889 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_712_4485361284563650037-1/-mr-10001 2018-01-05T11:29:01,889 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem 2018-01-05T11:29:01,889 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,891 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,891 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,891 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=analyze table lineitem compute sta...columns (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112901_531e127a-818d-4fb7-ad60-8d20036d1617 } 2018-01-05T11:29:01,891 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:01,891 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:01,893 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: analyze table lineitem compute sta...columns (Stage-0) 2018-01-05T11:29:01,893 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:29:01,893 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_33, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_33.dot 2018-01-05T11:29:01,893 INFO [IPC Server handler 0 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_33 2018-01-05T11:29:01,893 WARN [IPC Server handler 0 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:29:01,893 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Running DAG: analyze table lineitem compute sta...columns (Stage-0), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112901_531e127a-818d-4fb7-ad60-8d20036d1617 } 2018-01-05T11:29:01,916 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_33][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_33, submitTime=1515180541893, queueName=null 2018-01-05T11:29:01,916 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_33_00 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_33_00 [Map 1], Default Resources= 2018-01-05T11:29:01,916 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_33_00 [Map 1] 2018-01-05T11:29:01,916 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:29:01,916 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_33_00 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:01,917 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_33_01 [Reducer 2]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_33_01 [Reducer 2], Default Resources= 2018-01-05T11:29:01,917 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_33_01 [Reducer 2] 2018-01-05T11:29:01,917 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 2 2018-01-05T11:29:01,917 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_33_01 [Reducer 2] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:01,917 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:29:01,917 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_33][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_33, initTime=1515180541917 2018-01-05T11:29:01,917 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: dag_1515180518813_0001_33 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:29:01,917 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:29:01,917 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_33][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_33, startTime=1515180541917 2018-01-05T11:29:01,917 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_33 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:29:01,917 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {lineitem={InputName=lineitem}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:29:01,917 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: lineitem, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:29:01,917 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_33_00 [Map 1] 2018-01-05T11:29:01,918 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_33, dagName=analyze table lineitem compute sta...columns (Stage-0) 2018-01-05T11:29:01,918 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,918 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,918 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:01,922 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_33_00 [Map 1] 2018-01-05T11:29:01,922 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_33_00 [Map 1] 2018-01-05T11:29:01,922 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:01,922 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_33_00 [Map 1] 2018-01-05T11:29:01,924 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:29:01,924 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@65cde638 2018-01-05T11:29:01,924 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_708_2053710569094398989-1/djaiswal/_tez_scratch_dir/45b67c21-48cc-42c0-a3cf-50262330ed8b/map.xml 2018-01-05T11:29:01,924 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_708_2053710569094398989-1/djaiswal/_tez_scratch_dir/45b67c21-48cc-42c0-a3cf-50262330ed8b/map.xml 2018-01-05T11:29:01,924 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_708_2053710569094398989-1/djaiswal/_tez_scratch_dir/45b67c21-48cc-42c0-a3cf-50262330ed8b/map.xml 2018-01-05T11:29:01,924 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:01,924 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:01,924 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,925 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:01,925 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 5.97KB 2018-01-05T11:29:01,925 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_33_00 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:29:01,925 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: lineitem on vertex vertex_1515180518813_0001_33_00 [Map 1] 2018-01-05T11:29:01,926 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to ShuffleVertexManager for vertex_1515180518813_0001_33_01 [Reducer 2] 2018-01-05T11:29:01,926 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:29:01,926 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:01,926 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:29:01,926 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:01,926 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_708_2053710569094398989-1/djaiswal/_tez_scratch_dir/45b67c21-48cc-42c0-a3cf-50262330ed8b/map.xml 2018-01-05T11:29:01,926 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:01,926 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias lineitem 2018-01-05T11:29:01,926 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem 2018-01-05T11:29:01,926 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem 2018-01-05T11:29:01,927 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15 2018-01-05T11:29:01,927 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = l_orderkey,l_partkey,l_suppkey,l_linenumber,l_quantity,l_extendedprice,l_discount,l_tax,l_returnflag,l_linestatus,l_shipdate,l_commitdate,l_receiptdate,l_shipinstruct,l_shipmode,l_comment 2018-01-05T11:29:01,927 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem 2018-01-05T11:29:01,927 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:01,927 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:01,927 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:01,927 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:01,931 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.05 maxFrac: 0.75 auto: false desiredTaskIput: 104857600 2018-01-05T11:29:01,931 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:29:01,931 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 1 tasks for vertex: vertex_1515180518813_0001_33_01 [Reducer 2] 2018-01-05T11:29:01,931 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_33_01 [Reducer 2] 2018-01-05T11:29:01,931 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_33][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_33_01, reconfigureDoneTime=1515180541931, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:29:01,931 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_33_01 [Reducer 2], numAdditionalOutputs=1 2018-01-05T11:29:01,931 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_33][Event:VERTEX_INITIALIZED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_33_01, initRequestedTime=1515180541925, initedTime=1515180541931, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:01,931 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_33_01 [Reducer 2] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:29:01,943 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Time taken to get FileStatuses: 15 2018-01-05T11:29:01,943 INFO [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:29:01,943 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 15 2018-01-05T11:29:01,943 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 1 2018-01-05T11:29:01,943 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:01,943 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 1. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:29:01,943 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_708_2053710569094398989-1/djaiswal/_tez_scratch_dir/45b67c21-48cc-42c0-a3cf-50262330ed8b/map.xml 2018-01-05T11:29:01,943 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_708_2053710569094398989-1/djaiswal/_tez_scratch_dir/45b67c21-48cc-42c0-a3cf-50262330ed8b/map.xml 2018-01-05T11:29:01,943 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_708_2053710569094398989-1/djaiswal/_tez_scratch_dir/45b67c21-48cc-42c0-a3cf-50262330ed8b/map.xml 2018-01-05T11:29:01,943 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:01,943 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:01,943 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,944 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:01,944 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 5.97KB 2018-01-05T11:29:01,944 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem/lineitem.txt to src new group? true 2018-01-05T11:29:01,944 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:29:01,944 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:29:01,944 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:29:01,944 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 224 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 12099 Original splits: 1 2018-01-05T11:29:01,944 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Using original number of splits: 1 desired splits: 1 2018-01-05T11:29:01,944 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 1 grouped split count is 1, for bucket: 1 2018-01-05T11:29:01,944 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:29:01,944 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: lineitem on vertex vertex_1515180518813_0001_33_00 [Map 1] 2018-01-05T11:29:01,945 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_33_00 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:01,945 INFO [App Shared Pool - #0] impl.VertexImpl: Got updated RootInputsSpecs: {lineitem=forAllWorkUnits=true, update=[1]} 2018-01-05T11:29:01,945 INFO [App Shared Pool - #0] impl.VertexImpl: Vertex vertex_1515180518813_0001_33_00 [Map 1] parallelism set to 1 2018-01-05T11:29:01,945 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_33_00 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:01,945 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_33][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_33_00, initRequestedTime=1515180541917, initedTime=1515180541945, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:01,945 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_33_00 [Map 1] with distanceFromRoot: 0 2018-01-05T11:29:01,945 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_33_00 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:29:01,945 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_33][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_33_00, startRequestedTime=1515180541925, startedTime=1515180541945 2018-01-05T11:29:01,945 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_33][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_33_00, reconfigureDoneTime=1515180541945, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:29:01,945 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_33_00 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:01,945 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_33_00 for vertex: vertex_1515180518813_0001_33_01 [Reducer 2] numStartedSources: 1 numSources: 1 2018-01-05T11:29:01,945 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_33_01 [Reducer 2] with distanceFromRoot: 1 2018-01-05T11:29:01,945 INFO [App Shared Pool - #0] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:29:01,945 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_33][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_33_01, startRequestedTime=1515180541945, startedTime=1515180541945 2018-01-05T11:29:01,945 INFO [App Shared Pool - #0] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:29:01,946 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_33_01 [Reducer 2] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:01,946 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_33_00 [Map 1] 2018-01-05T11:29:01,946 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_33][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_33_00_000000, scheduledTime=1515180541946, launchTime=1515180541946 2018-01-05T11:29:01,946 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: OnVertexStarted vertex: Reducer 2 with 0 source tasks and 1 pending tasks 2018-01-05T11:29:01,946 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_33_00 [Map 1] 2018-01-05T11:29:01,947 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Map 1 in vertex: Reducer 2 numBipartiteSourceTasks: 1 2018-01-05T11:29:01,947 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_33_00_000000_0, priority=5, capability=, hosts=null 2018-01-05T11:29:01,947 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=5,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:01,947 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=5,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:01,947 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:01,947 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_33_00_000000_0 on hosts=null 2018-01-05T11:29:01,947 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_33_00_000000_0 on hosts=null 2018-01-05T11:29:01,947 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_33_00_000000_0 2018-01-05T11:29:01,947 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:01,947 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #65, task=TaskInfo{task=attempt_1515180518813_0001_33_00_000000_0, priority=5, startTime=0, containerId=null, uniqueId=64, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000065 2018-01-05T11:29:01,947 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_33_00_000000_0; false 2018-01-05T11:29:01,947 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_33_00_000000_0, priority=5, startTime=311562457, containerId=container_111101111_0001_00_000065assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=64, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:01,948 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000065 for node: 10.22.8.182:49818 2018-01-05T11:29:01,950 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 33, name=analyze table lineitem compute sta...columns (Stage-0), queryId=djaiswal_20180105112901_531e127a-818d-4fb7-ad60-8d20036d1617 2018-01-05T11:29:01,950 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000065, attempt_1515180518813_0001_33_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:01,950 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000065 for node: 10.22.8.182:49818 2018-01-05T11:29:01,950 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000065 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:01,950 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000065, launchTime=1515180541950 2018-01-05T11:29:01,951 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_33_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112901_531e127a-818d-4fb7-ad60-8d20036d1617, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000065, dagName=analyze table lineitem compute sta...columns (Stage-0), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{lineitem,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{Reducer 2,org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput,1}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180541917, firstAttemptStartTime=1515180541947, currentAttemptStartTime=1515180541950} 2018-01-05T11:29:01,951 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_33_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000065] on NM: [10.22.8.182:49818] 2018-01-05T11:29:01,951 INFO [IPC Server handler 4 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000065, QueryId=djaiswal_20180105112901_531e127a-818d-4fb7-ad60-8d20036d1617-33, DagId=33, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180541951 2018-01-05T11:29:01,951 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:01,951 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:01,951 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_33][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_33_00_000000_0, startTime=1515180541950, containerId=container_111101111_0001_00_000065, nodeId=10.22.8.182:49818 2018-01-05T11:29:01,951 DEBUG [IPC Server handler 4 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=33} with the ShuffleHandler 2018-01-05T11:29:01,962 INFO [IPC Server handler 4 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:29:01,962 DEBUG [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/33] 2018-01-05T11:29:01,962 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:29:01,962 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_33_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:01,963 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_33_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180541947, dagStartTime=1515180541917, withinDagPriority=5, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:29:01,963 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_33_00_000000_0 : ACCEPTED 2018-01-05T11:29:01,963 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_33_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:29:01,963 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_33_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180541947, dagStartTime=1515180541917, withinDagPriority=5, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:29:01,963 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_33_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:01,963 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_33_00_000000_0 to pre-emption queue 2018-01-05T11:29:01,963 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_33_00_000000_0: true 2018-01-05T11:29:01,963 INFO [TaskCommunicator # 5] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_33_00_000000_0 2018-01-05T11:29:01,963 INFO [TaskCommunicator # 5] GuaranteedTasks: Received response for attempt_1515180518813_0001_33_00_000000_0, true 2018-01-05T11:29:01,963 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:1578095647 2018-01-05T11:29:01,964 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : analyze table lineitem compute sta...columns (Stage-0), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_33_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=lineitem, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=Reducer 2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput }}, ] 2018-01-05T11:29:01,964 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1038614528, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:01,964 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_33_00_000000_0 vertexName: Map 1 2018-01-05T11:29:01,964 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000065, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_33_00_000000_0, eventCount=1 } 2018-01-05T11:29:01,964 DEBUG [TezTR-518813_1_33_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_33 queryId: djaiswal_20180105112901_531e127a-818d-4fb7-ad60-8d20036d1617 instanceType: TaskRunner2Callable 2018-01-05T11:29:01,964 INFO [TezTR-518813_1_33_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_33_00_000000_0 2018-01-05T11:29:01,964 DEBUG [TezTR-518813_1_33_0_0_0] log.PerfLogger: 2018-01-05T11:29:01,965 INFO [I/O Setup 1 Initialize: {Reducer 2}] writers.UnorderedPartitionedKVWriter: Requested BufferSize (tez.runtime.unordered.output.buffer.size-mb) : 100 2018-01-05T11:29:01,965 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:01,966 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:01,966 INFO [IPC Server handler 24 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_33_00_000000_0 sent events: (0-1). 2018-01-05T11:29:01,967 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:29:01,967 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_33_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:29:01,967 DEBUG [TezTR-518813_1_33_0_0_0] log.PerfLogger: 2018-01-05T11:29:01,967 INFO [TezTR-518813_1_33_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:01,967 INFO [TezTR-518813_1_33_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:01,967 INFO [TezTR-518813_1_33_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:01,968 INFO [I/O Setup 0 Initialize: {lineitem}] input.MRInput: lineitem using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:29:01,968 INFO [I/O Setup 0 Initialize: {lineitem}] input.MRInputLegacy: lineitem MRInputLegacy deferring initialization 2018-01-05T11:29:01,968 INFO [TezTR-518813_1_33_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:01,968 INFO [TezTR-518813_1_33_0_0_0] resources.MemoryDistributor: InitialRequests=[lineitem:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [Reducer 2:OUTPUT:104857600:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput] 2018-01-05T11:29:01,968 INFO [TezTR-518813_1_33_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:01,968 INFO [TezTR-518813_1_33_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:01,968 INFO [TezTR-518813_1_33_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 104857600, TotalRequestedScaled: 5.24288E7, TotalJVMHeap: 1038614528, TotalAvailable: 695871733, TotalRequested/TotalJVMHeap:0.10 2018-01-05T11:29:01,968 INFO [TezTR-518813_1_33_0_0_0] resources.MemoryDistributor: Allocations=[lineitem:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput:OUTPUT:104857600:104857600] 2018-01-05T11:29:01,968 INFO [TezTR-518813_1_33_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:01,968 INFO [TezTR-518813_1_33_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: lineitem being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:01,968 INFO [TezTR-518813_1_33_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:29:01,968 INFO [TezTR-518813_1_33_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:01,968 INFO [I/O Setup 1 Start: {lineitem}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: lineitem 2018-01-05T11:29:01,968 INFO [TezTR-518813_1_33_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:01,968 INFO [TezTR-518813_1_33_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_33_00_000000_0 2018-01-05T11:29:01,968 DEBUG [TezTR-518813_1_33_0_0_0] log.PerfLogger: 2018-01-05T11:29:01,968 DEBUG [TezTR-518813_1_33_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_33_00_000000_0_10193 2018-01-05T11:29:01,968 DEBUG [TezTR-518813_1_33_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:01,968 DEBUG [TezTR-518813_1_33_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_33_00_000000_0 2018-01-05T11:29:01,968 INFO [TezTR-518813_1_33_0_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112901_531e127a-818d-4fb7-ad60-8d20036d1617 2018-01-05T11:29:01,968 DEBUG [TezTR-518813_1_33_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:01,968 DEBUG [TezTR-518813_1_33_0_0_0] log.PerfLogger: 2018-01-05T11:29:01,968 DEBUG [TezTR-518813_1_33_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:01,968 DEBUG [TezTR-518813_1_33_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:01,968 DEBUG [TezTR-518813_1_33_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112901_531e127a-818d-4fb7-ad60-8d20036d1617_Map 1__MAP_PLAN__ 2018-01-05T11:29:01,968 INFO [TezTR-518813_1_33_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_708_2053710569094398989-1/djaiswal/_tez_scratch_dir/45b67c21-48cc-42c0-a3cf-50262330ed8b/map.xml 2018-01-05T11:29:01,968 DEBUG [TezTR-518813_1_33_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_708_2053710569094398989-1/djaiswal/_tez_scratch_dir/45b67c21-48cc-42c0-a3cf-50262330ed8b/map.xml 2018-01-05T11:29:01,968 DEBUG [TezTR-518813_1_33_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_708_2053710569094398989-1/djaiswal/_tez_scratch_dir/45b67c21-48cc-42c0-a3cf-50262330ed8b/map.xml 2018-01-05T11:29:01,969 DEBUG [TezTR-518813_1_33_0_0_0] log.PerfLogger: 2018-01-05T11:29:01,969 INFO [TezTR-518813_1_33_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:01,969 DEBUG [TezTR-518813_1_33_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:01,970 DEBUG [TezTR-518813_1_33_0_0_0] log.PerfLogger: 2018-01-05T11:29:01,970 INFO [TezTR-518813_1_33_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 5.97KB 2018-01-05T11:29:01,970 DEBUG [TezTR-518813_1_33_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:01,970 INFO [TezTR-518813_1_33_0_0_0] tez.MapRecordProcessor: The input names are: [lineitem] 2018-01-05T11:29:01,970 INFO [TezTR-518813_1_33_0_0_0] input.MRInput: Initialized MRInput: lineitem 2018-01-05T11:29:01,970 DEBUG [TezTR-518813_1_33_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 1 2018-01-05T11:29:01,970 INFO [TezTR-518813_1_33_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_708_2053710569094398989-1/djaiswal/_tez_scratch_dir/45b67c21-48cc-42c0-a3cf-50262330ed8b/map.xml 2018-01-05T11:29:01,970 DEBUG [TezTR-518813_1_33_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:01,970 DEBUG [TezTR-518813_1_33_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem/lineitem.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:int:int:int:double:double:double:double:string:string:string:string:string:string:string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem, columns=l_orderkey,l_partkey,l_suppkey,l_linenumber,l_quantity,l_extendedprice,l_discount,l_tax,l_returnflag,l_linestatus,l_shipdate,l_commitdate,l_receiptdate,l_shipinstruct,l_shipmode,l_comment, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=|, numRows=100, numFiles=1, serialization.ddl=struct lineitem { i32 l_orderkey, i32 l_partkey, i32 l_suppkey, i32 l_linenumber, double l_quantity, double l_extendedprice, double l_discount, double l_tax, string l_returnflag, string l_linestatus, string l_shipdate, string l_commitdate, string l_receiptdate, string l_shipinstruct, string l_shipmode, string l_comment}, transient_lastDdlTime=1515180541, rawDataSize=11999, field.delim=|, columns.comments=, totalSize=12099, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.lineitem}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:int:int:int:double:double:double:double:string:string:string:string:string:string:string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem, columns=l_orderkey,l_partkey,l_suppkey,l_linenumber,l_quantity,l_extendedprice,l_discount,l_tax,l_returnflag,l_linestatus,l_shipdate,l_commitdate,l_receiptdate,l_shipinstruct,l_shipmode,l_comment, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=|, numRows=100, numFiles=1, serialization.ddl=struct lineitem { i32 l_orderkey, i32 l_partkey, i32 l_suppkey, i32 l_linenumber, double l_quantity, double l_extendedprice, double l_discount, double l_tax, string l_returnflag, string l_linestatus, string l_shipdate, string l_commitdate, string l_receiptdate, string l_shipinstruct, string l_shipmode, string l_comment}, transient_lastDdlTime=1515180541, rawDataSize=11999, field.delim=|, columns.comments=, totalSize=12099, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.lineitem}, baseFileName=lineitem, vectorPartitionDesc=null] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:int:int:int:double:double:double:double:string:string:string:string:string:string:string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem, columns=l_orderkey,l_partkey,l_suppkey,l_linenumber,l_quantity,l_extendedprice,l_discount,l_tax,l_returnflag,l_linestatus,l_shipdate,l_commitdate,l_receiptdate,l_shipinstruct,l_shipmode,l_comment, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=|, numRows=100, numFiles=1, serialization.ddl=struct lineitem { i32 l_orderkey, i32 l_partkey, i32 l_suppkey, i32 l_linenumber, double l_quantity, double l_extendedprice, double l_discount, double l_tax, string l_returnflag, string l_linestatus, string l_shipdate, string l_commitdate, string l_receiptdate, string l_shipinstruct, string l_shipmode, string l_comment}, transient_lastDdlTime=1515180541, rawDataSize=11999, field.delim=|, columns.comments=, totalSize=12099, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.lineitem}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:int:int:int:double:double:double:double:string:string:string:string:string:string:string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem, columns=l_orderkey,l_partkey,l_suppkey,l_linenumber,l_quantity,l_extendedprice,l_discount,l_tax,l_returnflag,l_linestatus,l_shipdate,l_commitdate,l_receiptdate,l_shipinstruct,l_shipmode,l_comment, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true"}, serialization.format=|, numRows=100, numFiles=1, serialization.ddl=struct lineitem { i32 l_orderkey, i32 l_partkey, i32 l_suppkey, i32 l_linenumber, double l_quantity, double l_extendedprice, double l_discount, double l_tax, string l_returnflag, string l_linestatus, string l_shipdate, string l_commitdate, string l_receiptdate, string l_shipinstruct, string l_shipmode, string l_comment}, transient_lastDdlTime=1515180541, rawDataSize=11999, field.delim=|, columns.comments=, totalSize=12099, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.lineitem}, baseFileName=lineitem, vectorPartitionDesc=null]} 2018-01-05T11:29:01,970 DEBUG [TezTR-518813_1_33_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:29:01,971 INFO [TezTR-518813_1_33_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:29:01,971 DEBUG [TezTR-518813_1_33_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:29:01,972 DEBUG [TezTR-518813_1_33_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:29:01,972 INFO [TezTR-518813_1_33_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_708_2053710569094398989-1/djaiswal/_tez_scratch_dir/45b67c21-48cc-42c0-a3cf-50262330ed8b/map.xml 2018-01-05T11:29:01,972 DEBUG [TezTR-518813_1_33_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:01,972 INFO [TezTR-518813_1_33_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_33_00_000000_0 2018-01-05T11:29:01,973 INFO [TezTR-518813_1_33_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [l_orderkey, l_partkey, l_suppkey, l_linenumber, l_quantity, l_extendedprice, l_discount, l_tax, l_returnflag, l_linestatus, l_shipdate, l_commitdate, l_receiptdate, l_shipinstruct, l_shipmode, l_comment] / schema.evolution.columns.types [int, int, int, int, double, double, double, double, string, string, string, string, string, string, string, string] (isAcidRead false) 2018-01-05T11:29:01,974 DEBUG [TezTR-518813_1_33_0_0_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:01,974 DEBUG [TezTR-518813_1_33_0_0_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:01,974 DEBUG [TezTR-518813_1_33_0_0_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:01,974 DEBUG [TezTR-518813_1_33_0_0_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:01,974 WARN [TezTR-518813_1_33_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem/lineitem.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:29:01,975 DEBUG [TezTR-518813_1_33_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:29:01,975 WARN [TezTR-518813_1_33_0_0_0] LlapIoImpl: Not using LLAP IO for an unsupported split: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem/lineitem.txt:0+12099 2018-01-05T11:29:01,975 DEBUG [TezTR-518813_1_33_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem/lineitem.txt 2018-01-05T11:29:01,975 INFO [TezTR-518813_1_33_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem/lineitem.txt:0+12099], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=12099} 2018-01-05T11:29:01,975 INFO [TezTR-518813_1_33_0_0_0] input.MRInput: lineitem initialized RecordReader from event 2018-01-05T11:29:01,975 DEBUG [TezTR-518813_1_33_0_0_0] tez.MapRecordProcessor: Starting Output: Reducer 2 2018-01-05T11:29:01,975 INFO [TezTR-518813_1_33_0_0_0] writers.BaseUnorderedPartitionedKVWriter: Instantiating Partitioner: [org.apache.tez.mapreduce.partition.MRPartitioner] 2018-01-05T11:29:02,010 INFO [TezTR-518813_1_33_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: numBuffers=2, sizePerBuffer=52428800, skipBuffers=true, numPartitions=1, availableMemory=104857600, maxSingleBufferSizeBytes=2147483647, pipelinedShuffle=false, isFinalMergeEnabled=true, numPartitions=1, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:29:02,011 INFO [TezTR-518813_1_33_0_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:29:02,011 INFO [TezTR-518813_1_33_0_0_0] exec.MapOperator: Initializing operator MAP[0] 2018-01-05T11:29:02,011 DEBUG [TezTR-518813_1_33_0_0_0] exec.MapOperator: Initialization Done 0 MAP 2018-01-05T11:29:02,011 DEBUG [TezTR-518813_1_33_0_0_0] exec.MapOperator: Operator 0 MAP initialized 2018-01-05T11:29:02,011 DEBUG [TezTR-518813_1_33_0_0_0] exec.MapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:29:02,011 DEBUG [TezTR-518813_1_33_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,011 DEBUG [TezTR-518813_1_33_0_0_0] exec.MapOperator: Adding alias lineitem to work list for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem 2018-01-05T11:29:02,011 DEBUG [TezTR-518813_1_33_0_0_0] exec.MapOperator: dump TS[0] struct 2018-01-05T11:29:02,011 INFO [TezTR-518813_1_33_0_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:29:02,012 INFO [TezTR-518813_1_33_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:29:02,012 INFO [TezTR-518813_1_33_0_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] exec.TableScanOperator: Initializing children of 0 TS 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] exec.SelectOperator: Initializing child 1 SEL 2018-01-05T11:29:02,012 INFO [TezTR-518813_1_33_0_0_0] exec.SelectOperator: Initializing operator SEL[1] 2018-01-05T11:29:02,012 INFO [TezTR-518813_1_33_0_0_0] exec.SelectOperator: SELECT struct 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] exec.SelectOperator: Initialization Done 1 SEL 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] exec.SelectOperator: Operator 1 SEL initialized 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] exec.SelectOperator: Initializing children of 1 SEL 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] exec.GroupByOperator: Initializing child 2 GBY 2018-01-05T11:29:02,012 INFO [TezTR-518813_1_33_0_0_0] exec.GroupByOperator: Initializing operator GBY[2] 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,012 INFO [TezTR-518813_1_33_0_0_0] exec.GroupByOperator: Max hash table memory: 93860656 bytes 2018-01-05T11:29:02,012 INFO [TezTR-518813_1_33_0_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] exec.GroupByOperator: Initialization Done 2 GBY 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] exec.GroupByOperator: Operator 2 GBY initialized 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] exec.GroupByOperator: Initializing children of 2 GBY 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] exec.ReduceSinkOperator: Initializing child 3 RS 2018-01-05T11:29:02,012 INFO [TezTR-518813_1_33_0_0_0] exec.ReduceSinkOperator: Initializing operator RS[3] 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] exec.ReduceSinkOperator: keys size is 0 2018-01-05T11:29:02,012 INFO [TezTR-518813_1_33_0_0_0] exec.ReduceSinkOperator: Using tag = -1 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11, _col12, _col13, _col14, _col15] columnTypes=[struct, struct, struct, struct, struct, struct, struct, struct, struct, struct, struct, struct, struct, struct, struct, struct] 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] exec.ReduceSinkOperator: Operator 3 RS initialized 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS done is reset. 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] exec.GroupByOperator: Initialization Done 2 GBY done is reset. 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] exec.SelectOperator: Initialization Done 1 SEL done is reset. 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:29:02,012 INFO [TezTR-518813_1_33_0_0_0] exec.OperatorUtils: Setting output collector: RS[3] --> Reducer 2 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] log.PerfLogger: 2018-01-05T11:29:02,012 INFO [TezTR-518813_1_33_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_708_2053710569094398989-1/djaiswal/_tez_scratch_dir/45b67c21-48cc-42c0-a3cf-50262330ed8b/map.xml 2018-01-05T11:29:02,012 DEBUG [TezTR-518813_1_33_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:02,013 DEBUG [TezTR-518813_1_33_0_0_0] exec.MapOperator: Processing alias(es) lineitem for file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem/lineitem.txt 2018-01-05T11:29:02,013 WARN [TezTR-518813_1_33_0_0_0] lazy.LazyStruct: Extra bytes detected at the end of the row! Ignoring similar problems. 2018-01-05T11:29:02,013 INFO [TezTR-518813_1_33_0_0_0] exec.MapOperator: MAP[0]: records read - 1 2018-01-05T11:29:02,013 INFO [TezTR-518813_1_33_0_0_0] exec.MapOperator: MAP[0]: records read - 10 2018-01-05T11:29:02,014 INFO [TezTR-518813_1_33_0_0_0] exec.MapOperator: MAP[0]: records read - 100 2018-01-05T11:29:02,014 DEBUG [TezTR-518813_1_33_0_0_0] log.PerfLogger: 2018-01-05T11:29:02,014 DEBUG [TezTR-518813_1_33_0_0_0] mr.ObjectCache: djaiswal_20180105112901_531e127a-818d-4fb7-ad60-8d20036d1617_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:29:02,014 DEBUG [TezTR-518813_1_33_0_0_0] exec.MapOperator: close called for operator MAP[0] 2018-01-05T11:29:02,014 INFO [TezTR-518813_1_33_0_0_0] exec.MapOperator: Closing operator MAP[0] 2018-01-05T11:29:02,014 INFO [TezTR-518813_1_33_0_0_0] exec.MapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:100, 2018-01-05T11:29:02,014 DEBUG [TezTR-518813_1_33_0_0_0] exec.MapOperator: Closing child = TS[0] 2018-01-05T11:29:02,014 DEBUG [TezTR-518813_1_33_0_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:29:02,014 DEBUG [TezTR-518813_1_33_0_0_0] exec.TableScanOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:02,014 INFO [TezTR-518813_1_33_0_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:29:02,014 DEBUG [TezTR-518813_1_33_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,014 DEBUG [TezTR-518813_1_33_0_0_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_712_4485361284563650037-1/-mr-10001 2018-01-05T11:29:02,015 INFO [TezTR-518813_1_33_0_0_0] exec.TableScanOperator: publishing : default.lineitem/ : {numRows=100, rawDataSize=11999} 2018-01-05T11:29:02,036 DEBUG [TezTR-518813_1_33_0_0_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_712_4485361284563650037-1/-mr-10001/tmpstats-0_0 2018-01-05T11:29:02,036 DEBUG [TezTR-518813_1_33_0_0_0] fs.FSStatsPublisher: Writing stats in it : {default.lineitem/={numRows=100, rawDataSize=11999}} 2018-01-05T11:29:02,036 DEBUG [TezTR-518813_1_33_0_0_0] exec.TableScanOperator: Closing child = SEL[1] 2018-01-05T11:29:02,036 DEBUG [TezTR-518813_1_33_0_0_0] exec.SelectOperator: close called for operator SEL[1] 2018-01-05T11:29:02,036 DEBUG [TezTR-518813_1_33_0_0_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:02,036 INFO [TezTR-518813_1_33_0_0_0] exec.SelectOperator: Closing operator SEL[1] 2018-01-05T11:29:02,036 DEBUG [TezTR-518813_1_33_0_0_0] exec.SelectOperator: Closing child = GBY[2] 2018-01-05T11:29:02,036 DEBUG [TezTR-518813_1_33_0_0_0] exec.GroupByOperator: close called for operator GBY[2] 2018-01-05T11:29:02,036 DEBUG [TezTR-518813_1_33_0_0_0] exec.GroupByOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:02,036 INFO [TezTR-518813_1_33_0_0_0] exec.GroupByOperator: Closing operator GBY[2] 2018-01-05T11:29:02,036 INFO [TezTR-518813_1_33_0_0_0] exec.GroupByOperator: Begin Hash Table flush: size = 1 2018-01-05T11:29:02,037 INFO [TezTR-518813_1_33_0_0_0] exec.ReduceSinkOperator: keys are [] num distributions: 0 2018-01-05T11:29:02,037 DEBUG [TezTR-518813_1_33_0_0_0] exec.GroupByOperator: Closing child = RS[3] 2018-01-05T11:29:02,037 DEBUG [TezTR-518813_1_33_0_0_0] exec.ReduceSinkOperator: close called for operator RS[3] 2018-01-05T11:29:02,037 DEBUG [TezTR-518813_1_33_0_0_0] exec.ReduceSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:02,037 INFO [TezTR-518813_1_33_0_0_0] exec.ReduceSinkOperator: Closing operator RS[3] 2018-01-05T11:29:02,037 INFO [TezTR-518813_1_33_0_0_0] exec.ReduceSinkOperator: RECORDS_OUT_INTERMEDIATE_Map_1:1, 2018-01-05T11:29:02,037 DEBUG [TezTR-518813_1_33_0_0_0] exec.ReduceSinkOperator: 3 Close done 2018-01-05T11:29:02,037 DEBUG [TezTR-518813_1_33_0_0_0] exec.GroupByOperator: 2 Close done 2018-01-05T11:29:02,037 DEBUG [TezTR-518813_1_33_0_0_0] exec.SelectOperator: 1 Close done 2018-01-05T11:29:02,037 DEBUG [TezTR-518813_1_33_0_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:29:02,037 DEBUG [TezTR-518813_1_33_0_0_0] exec.MapOperator: 0 Close done 2018-01-05T11:29:02,037 INFO [TezTR-518813_1_33_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_33_00_000000_0 2018-01-05T11:29:02,037 INFO [TezTR-518813_1_33_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: Waiting for all spills to complete : Pending : 0 2018-01-05T11:29:02,037 INFO [TezTR-518813_1_33_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_2: All spills complete 2018-01-05T11:29:02,042 DEBUG [AMReporterQueueDrainer] impl.AMReporter: Removing am 10.22.8.182:49824 with last associated dag QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=16} from heartbeat with taskCount=0, amFailed=false, isDone=true 2018-01-05T11:29:02,049 DEBUG [TezTR-518813_1_33_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:02,049 INFO [TezTR-518813_1_33_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:02,049 INFO [TezTR-518813_1_33_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_33_00_000000_0, askedToStop=false 2018-01-05T11:29:02,050 INFO [TezTR-518813_1_33_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_33_00_000000_0, stopRequested=false 2018-01-05T11:29:02,050 INFO [TezTR-518813_1_33_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_33_00_000000_0: Counters: 14 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=100, INPUT_SPLIT_LENGTH_BYTES=12099, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=3596, OUTPUT_BYTES_WITH_OVERHEAD=3606, OUTPUT_BYTES_PHYSICAL=3634, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=100, RECORDS_OUT_INTERMEDIATE_Map_1=1]] 2018-01-05T11:29:02,050 DEBUG [TezTR-518813_1_33_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_33_0_0_0 thread id: 1993 scheme: file bytesRead: 12207 bytesWritten: 3730 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:02,050 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_33_00_000000_0, isTaskDone=true 2018-01-05T11:29:02,050 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000065, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_33_00_000000_0, eventCount=4 } 2018-01-05T11:29:02,051 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_33][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_33_00_000000_0, creationTime=1515180541946, allocationTime=1515180541948, startTime=1515180541950, finishTime=1515180542050, timeTaken=100, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:02,051 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:29:02,051 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_33_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:02,051 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_33_00_000000_0 2018-01-05T11:29:02,051 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000065=87 2018-01-05T11:29:02,051 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_33_00_000000_0: true 2018-01-05T11:29:02,051 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_33_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:02,051 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_33_00_000000_0 2018-01-05T11:29:02,051 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_33_00_000000_0 2018-01-05T11:29:02,051 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_33_00_000000_0; false 2018-01-05T11:29:02,051 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_33_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:02,051 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000065, QueryId=djaiswal_20180105112901_531e127a-818d-4fb7-ad60-8d20036d1617-33, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:02,051 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_33_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:29:02,051 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_33_00_000000_0 2018-01-05T11:29:02,051 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:02,051 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:02,051 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_33][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_33_00_000000, startTime=1515180541950, finishTime=1515180542051, timeTaken=101, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_33_00_000000_0, diagnostics=, counters=Counters: 20, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=12207, FILE_BYTES_WRITTEN=3730, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=100, INPUT_SPLIT_LENGTH_BYTES=12099, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=3596, OUTPUT_BYTES_WITH_OVERHEAD=3606, OUTPUT_BYTES_PHYSICAL=3634, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=100, RECORDS_OUT_INTERMEDIATE_Map_1=1 2018-01-05T11:29:02,051 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_33_00 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:02,051 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_33_00 [Map 1] 2018-01-05T11:29:02,051 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_33][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_33_00, initRequestedTime=1515180541917, initedTime=1515180541945, startRequestedTime=1515180541925, startedTime=1515180541945, finishTime=1515180542051, timeTaken=106, status=SUCCEEDED, diagnostics=, counters=Counters: 20, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=12207, FILE_BYTES_WRITTEN=3730, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=100, INPUT_SPLIT_LENGTH_BYTES=12099, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=3596, OUTPUT_BYTES_WITH_OVERHEAD=3606, OUTPUT_BYTES_PHYSICAL=3634, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=100, RECORDS_OUT_INTERMEDIATE_Map_1=1, vertexStats=firstTaskStartTime=1515180541950, firstTasksToStart=[ task_1515180518813_0001_33_00_000000 ], lastTaskFinishTime=1515180542050, lastTasksToFinish=[ task_1515180518813_0001_33_00_000000 ], minTaskDuration=100, maxTaskDuration=100, avgTaskDuration=100.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_33_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_33_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:02,051 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_33_00 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:02,051 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_33][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000065, stoppedTime=1515180542051, exitStatus=0 2018-01-05T11:29:02,051 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000065 exited with diagnostics set to Container completed. 2018-01-05T11:29:02,051 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_33_01 [Reducer 2] attempt: attempt_1515180518813_0001_33_00_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:29:02,051 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_33_00 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2 2018-01-05T11:29:02,051 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null 2018-01-05T11:29:02,051 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000065 2018-01-05T11:29:02,051 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: All source tasks completed. Ramping up 1 remaining tasks for vertex: Reducer 2 2018-01-05T11:29:02,052 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Scheduling 1 tasks for vertex: Reducer 2 with totalTasks: 1. 1 source tasks completed out of 1. MinSourceTaskCompletedFraction: 1.0 min: 0.05 max: 0.75 2018-01-05T11:29:02,052 INFO [App Shared Pool - #0] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Map 1, destinationVertexName=Reducer 2. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.UnorderedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:29:02,052 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_33_01 [Reducer 2] 2018-01-05T11:29:02,052 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_33][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_33_01_000000, scheduledTime=1515180542052, launchTime=1515180542052 2018-01-05T11:29:02,052 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_33_01 [Reducer 2] 2018-01-05T11:29:02,052 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_33_01_000000_0, priority=14, capability=, hosts=null 2018-01-05T11:29:02,052 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:02,052 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:02,052 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:02,052 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_33_01_000000_0 on hosts=null 2018-01-05T11:29:02,052 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_33_01_000000_0 on hosts=null 2018-01-05T11:29:02,052 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_33_01_000000_0 2018-01-05T11:29:02,052 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:02,052 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #66, task=TaskInfo{task=attempt_1515180518813_0001_33_01_000000_0, priority=14, startTime=0, containerId=null, uniqueId=65, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000066 2018-01-05T11:29:02,052 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_33_01_000000_0; false 2018-01-05T11:29:02,052 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_33_01_000000_0, priority=14, startTime=311562561, containerId=container_111101111_0001_00_000066assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=65, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:02,052 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000066 for node: 10.22.8.182:49818 2018-01-05T11:29:02,052 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000066, attempt_1515180518813_0001_33_01_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:02,052 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000066 for node: 10.22.8.182:49818 2018-01-05T11:29:02,052 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000066 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:02,052 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=33}, sourceName=Map 1, state=S_SUCCEEDED 2018-01-05T11:29:02,052 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000066, launchTime=1515180542052 2018-01-05T11:29:02,053 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_33_01_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000066] on NM: [10.22.8.182:49818] 2018-01-05T11:29:02,053 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_33_01_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112901_531e127a-818d-4fb7-ad60-8d20036d1617, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000066, dagName=analyze table lineitem compute sta...columns (Stage-0), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.UnorderedKVInput,1}}, Outputs={{out_Reducer 2,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=2, completedTaskCount=1, dagStartTime=1515180541917, firstAttemptStartTime=1515180542052, currentAttemptStartTime=1515180542052} 2018-01-05T11:29:02,053 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:02,053 INFO [IPC Server handler 1 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000066, QueryId=djaiswal_20180105112901_531e127a-818d-4fb7-ad60-8d20036d1617-33, DagId=33, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, SubmitTime=1515180542053 2018-01-05T11:29:02,053 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:02,053 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_33][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_33_01_000000_0, startTime=1515180542052, containerId=container_111101111_0001_00_000066, nodeId=10.22.8.182:49818 2018-01-05T11:29:02,053 DEBUG [IPC Server handler 1 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=33} with the ShuffleHandler 2018-01-05T11:29:02,053 DEBUG [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/33] 2018-01-05T11:29:02,054 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:29:02,054 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_33_01_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:02,054 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_33_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180542052, dagStartTime=1515180541917, withinDagPriority=14, vertexParallelism= 1, selfAndUpstreamParallelism= 2, selfAndUpstreamComplete= 1}] 2018-01-05T11:29:02,054 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_33_01_000000_0 : ACCEPTED 2018-01-05T11:29:02,054 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_33_01_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:29:02,054 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_33_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180542052, dagStartTime=1515180541917, withinDagPriority=14, vertexParallelism= 1, selfAndUpstreamParallelism= 2, selfAndUpstreamComplete= 1} 2018-01-05T11:29:02,054 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_33_01_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:02,054 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_33_01_000000_0 to pre-emption queue 2018-01-05T11:29:02,054 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_33_01_000000_0: true 2018-01-05T11:29:02,054 INFO [TaskCommunicator # 7] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_33_01_000000_0 2018-01-05T11:29:02,054 INFO [TaskCommunicator # 7] GuaranteedTasks: Received response for attempt_1515180518813_0001_33_01_000000_0, true 2018-01-05T11:29:02,054 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:1578095647 2018-01-05T11:29:02,054 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : analyze table lineitem compute sta...columns (Stage-0), VertexName: Reducer 2, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_33_01_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:29:02,054 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1035468800, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:02,054 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_33_01_000000_0 vertexName: Reducer 2 2018-01-05T11:29:02,054 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000066, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_33_01_000000_0, eventCount=1 } 2018-01-05T11:29:02,054 DEBUG [TezTR-518813_1_33_1_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_33 queryId: djaiswal_20180105112901_531e127a-818d-4fb7-ad60-8d20036d1617 instanceType: TaskRunner2Callable 2018-01-05T11:29:02,055 INFO [TezTR-518813_1_33_1_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_33_01_000000_0 2018-01-05T11:29:02,055 DEBUG [TezTR-518813_1_33_1_0_0] log.PerfLogger: 2018-01-05T11:29:02,055 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:02,056 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:02,057 INFO [IPC Server handler 7 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_33_01_000000_0 1 events [0,1) total 1 vertex_1515180518813_0001_33_01 [Reducer 2] 2018-01-05T11:29:02,057 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:29:02,057 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_33_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:29:02,058 DEBUG [TezTR-518813_1_33_1_0_0] log.PerfLogger: 2018-01-05T11:29:02,058 INFO [TezTR-518813_1_33_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:02,058 INFO [TezTR-518813_1_33_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:02,058 INFO [TezTR-518813_1_33_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:02,058 INFO [I/O Setup 1 Initialize: {out_Reducer 2}] output.MROutput: out_Reducer 2: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:29:02,058 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:02,058 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:02,058 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:02,058 DEBUG [I/O Setup 1 Initialize: {out_Reducer 2}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:02,058 INFO [TezTR-518813_1_33_1_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:02,058 INFO [TezTR-518813_1_33_1_0_0] resources.MemoryDistributor: InitialRequests=[out_Reducer 2:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.UnorderedKVInput] 2018-01-05T11:29:02,058 INFO [TezTR-518813_1_33_1_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:02,058 INFO [TezTR-518813_1_33_1_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:02,058 INFO [TezTR-518813_1_33_1_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 131404920, TotalRequestedScaled: 6.570246E7, TotalJVMHeap: 1035468800, TotalAvailable: 693764096, TotalRequested/TotalJVMHeap:0.13 2018-01-05T11:29:02,058 INFO [TezTR-518813_1_33_1_0_0] resources.MemoryDistributor: Allocations=[out_Reducer 2:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [Map 1:org.apache.tez.runtime.library.input.UnorderedKVInput:INPUT:131404920:131404920] 2018-01-05T11:29:02,058 INFO [TezTR-518813_1_33_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:02,058 INFO [TezTR-518813_1_33_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 0 2018-01-05T11:29:02,058 INFO [TezTR-518813_1_33_1_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:02,058 INFO [TezTR-518813_1_33_1_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_33_01_000000_0 2018-01-05T11:29:02,058 DEBUG [TezTR-518813_1_33_1_0_0] log.PerfLogger: 2018-01-05T11:29:02,058 DEBUG [TezTR-518813_1_33_1_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_33_01_000000_0_10196 2018-01-05T11:29:02,058 DEBUG [TezTR-518813_1_33_1_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:02,058 DEBUG [TezTR-518813_1_33_1_0_0] mr.ObjectCache: Creating djaiswal_20180105112901_531e127a-818d-4fb7-ad60-8d20036d1617_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:29:02,058 INFO [TezTR-518813_1_33_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_708_2053710569094398989-1/djaiswal/_tez_scratch_dir/3841e73f-a310-40d8-8351-28a357fea35e/reduce.xml 2018-01-05T11:29:02,058 DEBUG [TezTR-518813_1_33_1_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_708_2053710569094398989-1/djaiswal/_tez_scratch_dir/3841e73f-a310-40d8-8351-28a357fea35e/reduce.xml 2018-01-05T11:29:02,058 DEBUG [TezTR-518813_1_33_1_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_708_2053710569094398989-1/djaiswal/_tez_scratch_dir/3841e73f-a310-40d8-8351-28a357fea35e/reduce.xml 2018-01-05T11:29:02,059 DEBUG [TezTR-518813_1_33_1_0_0] log.PerfLogger: 2018-01-05T11:29:02,059 INFO [TezTR-518813_1_33_1_0_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:29:02,059 DEBUG [TezTR-518813_1_33_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,059 DEBUG [TezTR-518813_1_33_1_0_0] log.PerfLogger: 2018-01-05T11:29:02,059 INFO [TezTR-518813_1_33_1_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 3.67KB 2018-01-05T11:29:02,059 DEBUG [TezTR-518813_1_33_1_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:02,059 DEBUG [TezTR-518813_1_33_1_0_0] log.PerfLogger: 2018-01-05T11:29:02,059 DEBUG [TezTR-518813_1_33_1_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:02,059 DEBUG [TezTR-518813_1_33_1_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:02,060 DEBUG [TezTR-518813_1_33_1_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:02,060 INFO [TezTR-518813_1_33_1_0_0] impl.SimpleFetchedInputAllocator: Map_1: RequestedMemory=131404920, AssignedMemory=131404920, maxSingleShuffleLimit=32851230 2018-01-05T11:29:02,060 INFO [TezTR-518813_1_33_1_0_0] impl.ShuffleManager: Map_1: numInputs=1, compressionCodec=NoCompressionCodec, numFetchers=1, ifileBufferSize=4096, ifileReadAheadEnabled=true, ifileReadAheadLength=4194304, localDiskFetchEnabled=false, sharedFetchEnabled=false, keepAlive=false, keepAliveMaxConnections=20, connectionTimeout=20000, readTimeout=180000, bufferSize=8192, bufferSize=8192, maxTaskOutputAtOnce=20 2018-01-05T11:29:02,060 INFO [TezTR-518813_1_33_1_0_0] impl.ShuffleInputEventHandlerImpl: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:29:02,060 INFO [TezTR-518813_1_33_1_0_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:29:02,061 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=33&reduce=0&map=attempt_1515180518813_0001_33_00_000000_0_10195 mapId: [attempt_1515180518813_0001_33_00_000000_0_10195] reduceId: [0] jobId: [job_1515180518813_0001] dagId: [33] keepAlive: false 2018-01-05T11:29:02,061 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=33&reduce=0&map=attempt_1515180518813_0001_33_00_000000_0_10195; hash=...LtVIy21Gsa14w 2018-01-05T11:29:02,061 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=33&reduce=0&map=attempt_1515180518813_0001_33_00_000000_0_10195;reply=6r9f6vTZ7FrS8 2018-01-05T11:29:02,061 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=33, user='djaiswal', attemptId='attempt_1515180518813_0001_33_00_000000_0_10195'} via loader 2018-01-05T11:29:02,061 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=33, user='djaiswal', attemptId='attempt_1515180518813_0001_33_00_000000_0_10195'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:29:02,061 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_33_00_000000_0_10195 not found 2018-01-05T11:29:02,062 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_33_00_000000_0_10195,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/33/output/attempt_1515180518813_0001_33_00_000000_0_10195/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/33/output/attempt_1515180518813_0001_33_00_000000_0_10195/file.out.index 2018-01-05T11:29:02,062 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Content Length in shuffle : 3665 2018-01-05T11:29:02,062 INFO [Fetcher_B {Map_1} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=33&reduce=0&map=attempt_1515180518813_0001_33_00_000000_0_10195 sent hash and receievd reply 0 ms 2018-01-05T11:29:02,062 INFO [Fetcher_B {Map_1} #0] ShuffleManager.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_33_00_000000_0_10195} to MEMORY, csize=3610, dsize=3606, EndTime=1515180542062, TimeTaken=0, Rate=0.00 MB/s 2018-01-05T11:29:02,062 INFO [Fetcher_B {Map_1} #0] impl.ShuffleManager: All inputs fetched for input vertex : Map 1 2018-01-05T11:29:02,062 INFO [TezTR-518813_1_33_1_0_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:29:02,062 INFO [Fetcher_B {Map_1} #0] impl.ShuffleManager: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:02,062 INFO [TezTR-518813_1_33_1_0_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:02,062 INFO [ShuffleRunner {Map_1}] impl.ShuffleManager: Map_1: Shutting down FetchScheduler, Was Interrupted: false 2018-01-05T11:29:02,062 DEBUG [TezTR-518813_1_33_1_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:02,062 INFO [ShuffleRunner {Map_1}] impl.ShuffleManager: Map_1: Scheduler thread completed 2018-01-05T11:29:02,062 DEBUG [TezTR-518813_1_33_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,062 DEBUG [TezTR-518813_1_33_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,062 DEBUG [TezTR-518813_1_33_1_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11, _col12, _col13, _col14, _col15] columnTypes=[struct, struct, struct, struct, struct, struct, struct, struct, struct, struct, struct, struct, struct, struct, struct, struct] 2018-01-05T11:29:02,062 DEBUG [TezTR-518813_1_33_1_0_0] log.PerfLogger: 2018-01-05T11:29:02,062 INFO [TezTR-518813_1_33_1_0_0] exec.GroupByOperator: Initializing operator GBY[4] 2018-01-05T11:29:02,062 DEBUG [TezTR-518813_1_33_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,062 DEBUG [TezTR-518813_1_33_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,062 DEBUG [TezTR-518813_1_33_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,062 DEBUG [TezTR-518813_1_33_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,062 DEBUG [TezTR-518813_1_33_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,062 DEBUG [TezTR-518813_1_33_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,062 DEBUG [TezTR-518813_1_33_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,062 DEBUG [TezTR-518813_1_33_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,062 DEBUG [TezTR-518813_1_33_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,062 DEBUG [TezTR-518813_1_33_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,062 DEBUG [TezTR-518813_1_33_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,062 DEBUG [TezTR-518813_1_33_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,062 DEBUG [TezTR-518813_1_33_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,063 DEBUG [TezTR-518813_1_33_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,063 DEBUG [TezTR-518813_1_33_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,063 DEBUG [TezTR-518813_1_33_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,063 INFO [TezTR-518813_1_33_1_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:29:02,063 DEBUG [TezTR-518813_1_33_1_0_0] exec.GroupByOperator: Initialization Done 4 GBY 2018-01-05T11:29:02,063 DEBUG [TezTR-518813_1_33_1_0_0] exec.GroupByOperator: Operator 4 GBY initialized 2018-01-05T11:29:02,063 DEBUG [TezTR-518813_1_33_1_0_0] exec.GroupByOperator: Initializing children of 4 GBY 2018-01-05T11:29:02,063 DEBUG [TezTR-518813_1_33_1_0_0] exec.FileSinkOperator: Initializing child 6 FS 2018-01-05T11:29:02,063 INFO [TezTR-518813_1_33_1_0_0] exec.FileSinkOperator: Initializing operator FS[6] 2018-01-05T11:29:02,063 DEBUG [TezTR-518813_1_33_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:02,063 INFO [TezTR-518813_1_33_1_0_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@4f5fe6fa]:[_col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11, _col12, _col13, _col14, _col15]:[struct, struct, struct, struct, struct, struct, struct, struct, struct, struct, struct, struct, struct, struct, struct, struct]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@54875d30 2018-01-05T11:29:02,063 DEBUG [TezTR-518813_1_33_1_0_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:29:02,063 DEBUG [TezTR-518813_1_33_1_0_0] exec.FileSinkOperator: Initialization Done 6 FS 2018-01-05T11:29:02,063 DEBUG [TezTR-518813_1_33_1_0_0] exec.FileSinkOperator: Operator 6 FS initialized 2018-01-05T11:29:02,063 DEBUG [TezTR-518813_1_33_1_0_0] exec.FileSinkOperator: Initialization Done 6 FS done is reset. 2018-01-05T11:29:02,063 DEBUG [TezTR-518813_1_33_1_0_0] exec.GroupByOperator: Initialization Done 4 GBY done is reset. 2018-01-05T11:29:02,063 INFO [TezTR-518813_1_33_1_0_0] tez.ReduceRecordProcessor: Id =4 Id =6 <\Children> Id = 4 null<\Parent> <\FS> <\Children> <\Parent> <\GBY> 2018-01-05T11:29:02,063 DEBUG [TezTR-518813_1_33_1_0_0] log.PerfLogger: 2018-01-05T11:29:02,063 INFO [TezTR-518813_1_33_1_0_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 2 2018-01-05T11:29:02,063 DEBUG [TezTR-518813_1_33_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:02,063 DEBUG [TezTR-518813_1_33_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:02,063 DEBUG [TezTR-518813_1_33_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:02,063 DEBUG [TezTR-518813_1_33_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:02,063 DEBUG [TezTR-518813_1_33_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:02,063 DEBUG [TezTR-518813_1_33_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:02,063 INFO [TezTR-518813_1_33_1_0_0] lazybinary.LazyBinaryStruct: Missing fields! Expected 16 fields but only got 16! Last field end 3596 and serialize buffer end 3595. Ignoring similar problems. 2018-01-05T11:29:02,064 INFO [TezTR-518813_1_33_1_0_0] readers.UnorderedKVReader: Num Records read: 1 2018-01-05T11:29:02,064 DEBUG [TezTR-518813_1_33_1_0_0] log.PerfLogger: 2018-01-05T11:29:02,064 DEBUG [TezTR-518813_1_33_1_0_0] mr.ObjectCache: djaiswal_20180105112901_531e127a-818d-4fb7-ad60-8d20036d1617_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:29:02,064 DEBUG [TezTR-518813_1_33_1_0_0] exec.GroupByOperator: close called for operator GBY[4] 2018-01-05T11:29:02,064 INFO [TezTR-518813_1_33_1_0_0] exec.GroupByOperator: Closing operator GBY[4] 2018-01-05T11:29:02,064 INFO [TezTR-518813_1_33_1_0_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_712_4485361284563650037-1/-mr-10000/.hive-staging_hive_2018-01-05_11-29-01_712_4485361284563650037-1/_tmp.-ext-10002/000000_0 2018-01-05T11:29:02,064 INFO [TezTR-518813_1_33_1_0_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_712_4485361284563650037-1/-mr-10000/.hive-staging_hive_2018-01-05_11-29-01_712_4485361284563650037-1/_task_tmp.-ext-10002/_tmp.000000_0 2018-01-05T11:29:02,064 INFO [TezTR-518813_1_33_1_0_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_712_4485361284563650037-1/-mr-10000/.hive-staging_hive_2018-01-05_11-29-01_712_4485361284563650037-1/_tmp.-ext-10002/000000_0 2018-01-05T11:29:02,098 INFO [TezTR-518813_1_33_1_0_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:29:02,098 DEBUG [TezTR-518813_1_33_1_0_0] exec.GroupByOperator: Closing child = FS[6] 2018-01-05T11:29:02,098 DEBUG [TezTR-518813_1_33_1_0_0] exec.FileSinkOperator: close called for operator FS[6] 2018-01-05T11:29:02,098 DEBUG [TezTR-518813_1_33_1_0_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:02,098 INFO [TezTR-518813_1_33_1_0_0] exec.FileSinkOperator: Closing operator FS[6] 2018-01-05T11:29:02,098 INFO [TezTR-518813_1_33_1_0_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:29:02,099 INFO [TezTR-518813_1_33_1_0_0] exec.FileSinkOperator: RECORDS_OUT_0:1, 2018-01-05T11:29:02,099 DEBUG [TezTR-518813_1_33_1_0_0] exec.FileSinkOperator: 6 Close done 2018-01-05T11:29:02,099 DEBUG [TezTR-518813_1_33_1_0_0] exec.GroupByOperator: 4 Close done 2018-01-05T11:29:02,099 INFO [TezTR-518813_1_33_1_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_33_01_000000_0 2018-01-05T11:29:02,099 INFO [TezTR-518813_1_33_1_0_0] impl.ShuffleInputEventHandlerImpl: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:29:02,099 INFO [TezTR-518813_1_33_1_0_0] impl.ShuffleManager: Shutting down pending fetchers on sourceMap_1: 0 2018-01-05T11:29:02,099 INFO [TezTR-518813_1_33_1_0_0] output.MROutput: out_Reducer 2 closed 2018-01-05T11:29:02,099 DEBUG [TezTR-518813_1_33_1_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:02,099 INFO [TezTR-518813_1_33_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:02,099 INFO [TezTR-518813_1_33_1_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_33_01_000000_0, askedToStop=false 2018-01-05T11:29:02,099 INFO [TezTR-518813_1_33_1_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_33_01_000000_0, stopRequested=false 2018-01-05T11:29:02,099 INFO [TezTR-518813_1_33_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_33_01_000000_0: Counters: 14 [[org.apache.tez.common.counters.TaskCounter NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=3610, SHUFFLE_BYTES_DECOMPRESSED=3606, SHUFFLE_BYTES_TO_MEM=3610, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=2, FIRST_EVENT_RECEIVED=0, LAST_EVENT_RECEIVED=0][HIVE CREATED_FILES=1, RECORDS_OUT_0=1]] 2018-01-05T11:29:02,099 DEBUG [TezTR-518813_1_33_1_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_33_1_0_0 thread id: 2005 scheme: file bytesRead: 0 bytesWritten: 4825 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:02,099 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_33_01_000000_0, isTaskDone=true 2018-01-05T11:29:02,099 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000066, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_33_01_000000_0, eventCount=2 } 2018-01-05T11:29:02,100 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_33][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_33_01_000000_0, creationTime=1515180542052, allocationTime=1515180542052, startTime=1515180542052, finishTime=1515180542100, timeTaken=48, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:02,100 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:02,100 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_33_01_000000_0 2018-01-05T11:29:02,100 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_33_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:02,100 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_33_01_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:02,100 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_33_01_000000_0 2018-01-05T11:29:02,100 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_33_01_000000_0; false 2018-01-05T11:29:02,100 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_33_01_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:02,101 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:02,101 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:02,101 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000066=47 2018-01-05T11:29:02,101 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_33_01_000000_0: true 2018-01-05T11:29:02,101 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_33][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_33_01_000000, startTime=1515180542052, finishTime=1515180542100, timeTaken=48, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_33_01_000000_0, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=4825, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=3610, SHUFFLE_BYTES_DECOMPRESSED=3606, SHUFFLE_BYTES_TO_MEM=3610, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=2, FIRST_EVENT_RECEIVED=0, LAST_EVENT_RECEIVED=0, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1 2018-01-05T11:29:02,101 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_33_01 [Reducer 2], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:02,101 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_33_01 [Reducer 2] 2018-01-05T11:29:02,101 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_33][Event:VERTEX_FINISHED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_33_01, initRequestedTime=1515180541925, initedTime=1515180541931, startRequestedTime=1515180541945, startedTime=1515180541945, finishTime=1515180542101, timeTaken=156, status=SUCCEEDED, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=4825, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=3610, SHUFFLE_BYTES_DECOMPRESSED=3606, SHUFFLE_BYTES_TO_MEM=3610, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=2, FIRST_EVENT_RECEIVED=0, LAST_EVENT_RECEIVED=0, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1, vertexStats=firstTaskStartTime=1515180542052, firstTasksToStart=[ task_1515180518813_0001_33_01_000000 ], lastTaskFinishTime=1515180542100, lastTasksToFinish=[ task_1515180518813_0001_33_01_000000 ], minTaskDuration=48, maxTaskDuration=48, avgTaskDuration=48.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_33_01_000000 ], longestDurationTasks=[ task_1515180518813_0001_33_01_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:02,102 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_33_01 [Reducer 2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:02,102 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_33][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000066, stoppedTime=1515180542102, exitStatus=0 2018-01-05T11:29:02,102 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000066 exited with diagnostics set to Container completed. 2018-01-05T11:29:02,102 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_33_01 [Reducer 2] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2 2018-01-05T11:29:02,102 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2, commitInProgress=0, terminationCause=null 2018-01-05T11:29:02,102 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_33_00 [Map 1] 2018-01-05T11:29:02,102 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_33_01 [Reducer 2] 2018-01-05T11:29:02,103 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_33, queueSize=0 2018-01-05T11:29:02,103 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_33][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_33, startTime=1515180541917, finishTime=1515180542102, timeTaken=185, status=SUCCEEDED, diagnostics=, counters=Counters: 36, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, OTHER_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=297, AM_GC_TIME_MILLIS=12, File System Counters, FILE_BYTES_READ=12207, FILE_BYTES_WRITTEN=8555, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=101, INPUT_SPLIT_LENGTH_BYTES=12099, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=3596, OUTPUT_BYTES_WITH_OVERHEAD=3606, OUTPUT_BYTES_PHYSICAL=3634, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=3610, SHUFFLE_BYTES_DECOMPRESSED=3606, SHUFFLE_BYTES_TO_MEM=3610, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=2, FIRST_EVENT_RECEIVED=0, LAST_EVENT_RECEIVED=0, HIVE, CREATED_FILES=1, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=100, RECORDS_OUT_0=1, RECORDS_OUT_INTERMEDIATE_Map_1=1 2018-01-05T11:29:02,103 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_33 finished with state: SUCCEEDED 2018-01-05T11:29:02,103 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_33 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:29:02,103 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000066 2018-01-05T11:29:02,103 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_33, dagState=SUCCEEDED 2018-01-05T11:29:02,103 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_33_post 2018-01-05T11:29:02,103 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:29:02,103 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:29:02,103 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:29:02,103 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 33 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=2, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=2,NumRequestsWithoutLocation=2, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=2} 2018-01-05T11:29:02,103 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:29:02,103 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=analyze table lineitem compute sta...columns (Stage-0), with id=dag_1515180518813_0001_33 2018-01-05T11:29:02,103 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 33, to 10.22.8.182:49818 2018-01-05T11:29:02,103 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=2, Remaining=0 2018-01-05T11:29:02,103 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=analyze table lineitem compute sta...columns (Stage-0), with id=dag_1515180518813_0001_33 2018-01-05T11:29:02,104 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=33} 2018-01-05T11:29:02,104 INFO [IPC Server handler 0 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=33}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:29:02,104 INFO [IPC Server handler 0 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/33 after 300 seconds 2018-01-05T11:29:02,104 ERROR [IPC Server handler 0 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112901_531e127a-818d-4fb7-ad60-8d20036d1617, dag_1515180518813_0001_33 2018-01-05T11:29:02,104 INFO [IPC Server handler 0 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112901_531e127a-818d-4fb7-ad60-8d20036d1617 2018-01-05T11:29:02,104 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=33} = 1 2018-01-05T11:29:02,104 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Issuing killFragment for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=33} attempt_1515180518813_0001_33_01_000000_0 2018-01-05T11:29:02,104 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Removing attempt_1515180518813_0001_33_01_000000_0 from preemptionQueue 2018-01-05T11:29:02,104 INFO [IPC Server handler 0 on 49818] impl.TaskRunnerCallable: Ignoring kill request for task attempt_1515180518813_0001_33_01_000000_0 since it's already complete 2018-01-05T11:29:02,104 DEBUG [IPC Server handler 0 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=33} 2018-01-05T11:29:02,104 DEBUG [IPC Server handler 0 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:29:02,104 DEBUG [IPC Server handler 0 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=33} 2018-01-05T11:29:02,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:02,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:02,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:02,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:02,104 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 Reducer 2: 1/1 2018-01-05T11:29:02,104 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:02,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:02,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.Utilities: TaskId for 000000_0 = 000000 2018-01-05T11:29:02,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:02,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:02,106 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:02,107 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Starting task [Stage-2:STATS] in serial mode 2018-01-05T11:29:02,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@789e7ca, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:29:02,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:02,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:02,107 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:29:02,107 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:29:02,107 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:02,107 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:02,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:02,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:02,107 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:29:02,107 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_712_4485361284563650037-1/-mr-10001 2018-01-05T11:29:02,107 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_712_4485361284563650037-1/-mr-10001 2018-01-05T11:29:02,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.lineitem/ numRows 2018-01-05T11:29:02,108 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.lineitem/ numRows 100 2018-01-05T11:29:02,108 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Part ID: default.lineitem/ rawDataSize 2018-01-05T11:29:02,108 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: Read stats for : default.lineitem/ rawDataSize 11999 2018-01-05T11:29:02,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:02,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:29:02,109 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:29:02,109 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:29:02,109 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:29:02,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,109 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,110 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,110 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,110 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,111 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:29:02,111 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:29:02,111 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:02,111 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:29:02,111 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:02,111 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: alter_table: db=default tbl=lineitem newtbl=lineitem 2018-01-05T11:29:02,111 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_table: db=default tbl=lineitem newtbl=lineitem 2018-01-05T11:29:02,116 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:02,120 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] stats.BasicStatsTask: Table default.lineitem stats: [numFiles=1, numRows=100, totalSize=12099, rawDataSize=11999] 2018-01-05T11:29:02,120 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] fs.FSStatsAggregator: About to delete stats tmp dir :file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_712_4485361284563650037-1/-mr-10001 2018-01-05T11:29:02,136 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Time taken to get FileStatuses: 14 2018-01-05T11:29:02,136 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:29:02,136 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 14 2018-01-05T11:29:02,136 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FetchOperator: Creating fetchTask with deserializer typeinfo: struct<_col0:struct,_col1:struct,_col2:struct,_col3:struct,_col4:struct,_col5:struct,_col6:struct,_col7:struct,_col8:struct,_col9:struct,_col10:struct,_col11:struct,_col12:struct,_col13:struct,_col14:struct,_col15:struct> 2018-01-05T11:29:02,136 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] exec.FetchOperator: deserializer properties: table properties: {columns=_col0,_col1,_col2,_col3,_col4,_col5,_col6,_col7,_col8,_col9,_col10,_col11,_col12,_col13,_col14,_col15, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct:struct:struct:struct:struct:struct:struct:struct:struct:struct:struct:struct:struct:struct:struct:struct, escape.delim=\} partition properties: {columns=_col0,_col1,_col2,_col3,_col4,_col5,_col6,_col7,_col8,_col9,_col10,_col11,_col12,_col13,_col14,_col15, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct:struct:struct:struct:struct:struct:struct:struct:struct:struct:struct:struct:struct:struct:struct:struct, escape.delim=\} 2018-01-05T11:29:02,137 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:02,137 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.HiveMetaStore: 0: write_column_statistics: db=default table=lineitem 2018-01-05T11:29:02,137 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=write_column_statistics: db=default table=lineitem 2018-01-05T11:29:02,139 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=lineitem tableName=default colName=l_orderkey 2018-01-05T11:29:02,140 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=lineitem tableName=default colName=l_partkey 2018-01-05T11:29:02,140 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=lineitem tableName=default colName=l_suppkey 2018-01-05T11:29:02,140 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=lineitem tableName=default colName=l_linenumber 2018-01-05T11:29:02,141 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=lineitem tableName=default colName=l_quantity 2018-01-05T11:29:02,141 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,141 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=lineitem tableName=default colName=l_extendedprice 2018-01-05T11:29:02,142 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=lineitem tableName=default colName=l_discount 2018-01-05T11:29:02,142 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=lineitem tableName=default colName=l_tax 2018-01-05T11:29:02,142 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=lineitem tableName=default colName=l_returnflag 2018-01-05T11:29:02,143 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=lineitem tableName=default colName=l_linestatus 2018-01-05T11:29:02,143 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,143 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=lineitem tableName=default colName=l_shipdate 2018-01-05T11:29:02,144 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=lineitem tableName=default colName=l_commitdate 2018-01-05T11:29:02,144 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=lineitem tableName=default colName=l_receiptdate 2018-01-05T11:29:02,144 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=lineitem tableName=default colName=l_shipinstruct 2018-01-05T11:29:02,145 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=lineitem tableName=default colName=l_shipmode 2018-01-05T11:29:02,145 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,145 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metastore.ObjectStore: Updating table level column statistics for db=lineitem tableName=default colName=l_comment 2018-01-05T11:29:02,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metrics.PerfLogger: 2018-01-05T11:29:02,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:02,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:02,148 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: query: analyze table lineitem compute statistics for columns 2018-01-05T11:29:02,148 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:29:02,148 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Input: default@lineitem 2018-01-05T11:29:02,148 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: default@lineitem 2018-01-05T11:29:02,148 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] SessionState: POSTHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_712_4485361284563650037-1/-mr-10000 2018-01-05T11:29:02,148 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:02,149 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:02,149 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:02,149 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] metadata.Hive: Total time spent in each metastore function (ms): {alter_table_with_environmentContext_(String, String, Table, EnvironmentContext, )=9, setPartitionColumnStatistics_(SetPartitionsStatsRequest, )=11} 2018-01-05T11:29:02,149 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112901_531e127a-818d-4fb7-ad60-8d20036d1617); Time taken: 0.363 seconds 2018-01-05T11:29:02,149 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: OK 2018-01-05T11:29:02,149 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:02,149 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:02,149 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] log.PerfLogger: 2018-01-05T11:29:02,149 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Driver: Shutting down query analyze table lineitem compute statistics for columns 2018-01-05T11:29:02,149 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb/hive_2018-01-05_11-29-01_708_2053710569094398989-1 2018-01-05T11:29:02,149 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] CliDriver: Time taken: 0.441 seconds 2018-01-05T11:29:02,149 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:29:02,149 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:02,149 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:29:02,149 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:29:02,155 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000065, QueryId=djaiswal_20180105112901_531e127a-818d-4fb7-ad60-8d20036d1617-33, DagId=33, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180541963, EndTime=1515180542155 2018-01-05T11:29:02,155 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_33_01_000000_0 2018-01-05T11:29:02,156 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_33_01_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:29:02,156 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_33_01_000000_0 2018-01-05T11:29:02,156 INFO [ExecutionCompletionThread #0] impl.QueryTracker: Ignoring fragmentComplete message for unknown query: QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=33} 2018-01-05T11:29:02,192 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: HiveConf of name hive.dummyparam.test.server.specific.config.override does not exist 2018-01-05T11:29:02,192 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: HiveConf of name hive.metastore.metadb.dir does not exist 2018-01-05T11:29:02,193 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: HiveConf of name hive.dummyparam.test.server.specific.config.metastoresite does not exist 2018-01-05T11:29:02,193 WARN [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: HiveConf of name hive.llap.daemon.service.port does not exist 2018-01-05T11:29:02,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: /Users/djaiswal/hive/hive/itests/qtest/../../testutils/hadoop 2018-01-05T11:29:02,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: fs 2018-01-05T11:29:02,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: true 2018-01-05T11:29:02,193 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:29:02,193 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:02,193 INFO [main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:29:02,193 INFO [main] session.SessionState: Updating thread name to 53565c10-a9a1-4685-81b4-afcf82ce77fb main 2018-01-05T11:29:02,193 DEBUG [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.VariableSubstitution: Substitution is on: fs 2018-01-05T11:29:02,193 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] conf.HiveConf: Using the default value passed in for log id: 53565c10-a9a1-4685-81b4-afcf82ce77fb 2018-01-05T11:29:02,193 INFO [53565c10-a9a1-4685-81b4-afcf82ce77fb main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:02,193 INFO [main] QTestUtil: Result from cliDrriver.processLine in createSources=0 2018-01-05T11:29:02,193 INFO [main] control.CoreCliDriver: Initialization createSources done. ElapsedTime(ms)=25055 2018-01-05T11:29:02,238 WARN [main] conf.HiveConf: HiveConf of name hive.dummyparam.test.server.specific.config.override does not exist 2018-01-05T11:29:02,238 WARN [main] conf.HiveConf: HiveConf of name hive.metastore.metadb.dir does not exist 2018-01-05T11:29:02,238 WARN [main] conf.HiveConf: HiveConf of name hive.dummyparam.test.server.specific.config.metastoresite does not exist 2018-01-05T11:29:02,238 WARN [main] conf.HiveConf: HiveConf of name hive.llap.daemon.service.port does not exist 2018-01-05T11:29:02,240 INFO [ProcessThread(sid:0 cport:-1):] server.PrepRequestProcessor: Processed session termination for sessionid: 0x160c7cbc1600000 2018-01-05T11:29:02,240 INFO [main] zookeeper.ZooKeeper: Session: 0x160c7cbc1600000 closed 2018-01-05T11:29:02,240 INFO [main] zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:49384 sessionTimeout=1200000 watcher=org.apache.hadoop.hive.ql.QTestUtil$QTestSetup$1@9e1674a 2018-01-05T11:29:02,241 INFO [main] metastore.HiveMetaStoreClient: Mestastore configuration metastore.filter.hook changed from org.apache.hadoop.hive.ql.security.authorization.plugin.AuthorizationMetaStoreFilterHook to org.apache.hadoop.hive.metastore.DefaultMetaStoreFilterHookImpl 2018-01-05T11:29:02,241 DEBUG [main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@6a68ccf3, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:29:02,241 DEBUG [main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:02,241 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,241 INFO [main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:29:02,241 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:29:02,242 INFO [main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:02,242 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:02,242 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,242 DEBUG [main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:02,242 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:49384] server.NIOServerCnxnFactory: Accepted socket connection from /127.0.0.1:49967 2018-01-05T11:29:02,242 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:49384] server.ZooKeeperServer: Client attempting to establish new session at /127.0.0.1:49967 2018-01-05T11:29:02,242 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,243 INFO [SyncThread:0] server.ZooKeeperServer: Established session 0x160c7cbc1600005 with negotiated timeout 40000 for client /127.0.0.1:49967 2018-01-05T11:29:02,243 INFO [main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:29:02,243 WARN [main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:29:02,243 WARN [main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:29:02,243 INFO [main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:29:02,243 DEBUG [main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,243 DEBUG [main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,243 DEBUG [main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,244 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,244 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,244 INFO [main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:29:02,244 INFO [main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:29:02,245 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,245 INFO [main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:29:02,245 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,245 INFO [main] metastore.HiveMetaStore: 0: get_all_databases 2018-01-05T11:29:02,245 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_all_databases 2018-01-05T11:29:02,246 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,246 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,246 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,246 INFO [main] metastore.HiveMetaStore: 0: get_tables: db=default pat=.* 2018-01-05T11:29:02,246 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_tables: db=default pat=.* 2018-01-05T11:29:02,249 WARN [main] metastore.ObjectStore: Falling back to ORM path due to direct SQL failure (this is not an error): java.lang.String cannot be cast to [Ljava.lang.Object; at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.getTables(MetaStoreDirectSql.java:411) at org.apache.hadoop.hive.metastore.ObjectStore$2.getSqlResult(ObjectStore.java:1313) 2018-01-05T11:29:02,250 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,256 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,256 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000066, QueryId=djaiswal_20180105112901_531e127a-818d-4fb7-ad60-8d20036d1617-33, DagId=33, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180542054, EndTime=1515180542256 2018-01-05T11:29:02,256 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,256 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,256 INFO [main] metastore.HiveMetaStore: 0: get_indexes : db=default tbl=alltypesorc 2018-01-05T11:29:02,256 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_indexes : db=default tbl=alltypesorc 2018-01-05T11:29:02,261 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,261 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,261 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,261 INFO [main] metastore.HiveMetaStore: 0: get_indexes : db=default tbl=alltypesparquet 2018-01-05T11:29:02,261 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_indexes : db=default tbl=alltypesparquet 2018-01-05T11:29:02,261 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,261 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,261 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,261 INFO [main] metastore.HiveMetaStore: 0: get_indexes : db=default tbl=cbo_t1 2018-01-05T11:29:02,261 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_indexes : db=default tbl=cbo_t1 2018-01-05T11:29:02,262 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,262 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,262 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,262 INFO [main] metastore.HiveMetaStore: 0: get_indexes : db=default tbl=cbo_t2 2018-01-05T11:29:02,262 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_indexes : db=default tbl=cbo_t2 2018-01-05T11:29:02,262 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,262 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,262 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,262 INFO [main] metastore.HiveMetaStore: 0: get_indexes : db=default tbl=cbo_t3 2018-01-05T11:29:02,262 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_indexes : db=default tbl=cbo_t3 2018-01-05T11:29:02,262 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,262 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,262 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,262 INFO [main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest1 2018-01-05T11:29:02,262 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest1 2018-01-05T11:29:02,265 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,266 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,266 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,266 INFO [main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest1 2018-01-05T11:29:02,266 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest1 2018-01-05T11:29:02,268 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,269 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,269 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,269 INFO [main] metastore.HiveMetaStore: 0: drop_table : db=default tbl=dest1 2018-01-05T11:29:02,269 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=drop_table : db=default tbl=dest1 2018-01-05T11:29:02,277 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 5.831479ms + 0.007187ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc] 2018-01-05T11:29:02,371 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,372 DEBUG [main] utils.FileUtils: deleting file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/dest1 2018-01-05T11:29:02,372 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,372 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,372 INFO [main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest2 2018-01-05T11:29:02,372 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest2 2018-01-05T11:29:02,375 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,375 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,375 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,375 INFO [main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest2 2018-01-05T11:29:02,375 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest2 2018-01-05T11:29:02,378 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,379 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,379 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,379 INFO [main] metastore.HiveMetaStore: 0: drop_table : db=default tbl=dest2 2018-01-05T11:29:02,379 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=drop_table : db=default tbl=dest2 2018-01-05T11:29:02,382 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 0.215091ms + 0.007356ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc] 2018-01-05T11:29:02,394 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,395 DEBUG [main] utils.FileUtils: deleting file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/dest2 2018-01-05T11:29:02,395 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,395 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,395 INFO [main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest3 2018-01-05T11:29:02,395 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest3 2018-01-05T11:29:02,397 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,398 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,398 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,398 INFO [main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest3 2018-01-05T11:29:02,398 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest3 2018-01-05T11:29:02,400 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,401 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,401 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,401 INFO [main] metastore.HiveMetaStore: 0: drop_table : db=default tbl=dest3 2018-01-05T11:29:02,401 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=drop_table : db=default tbl=dest3 2018-01-05T11:29:02,404 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 0.17862ms + 0.008611ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc] 2018-01-05T11:29:02,408 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 3.809083ms + 0.064191ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (9) order by "PART_NAME" asc] 2018-01-05T11:29:02,411 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 2.626213ms + 0.116916ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (9) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:29:02,413 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 2.045755ms + 0.052711ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (9) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:02,415 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 1.657212ms + 0.015374ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (25) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:29:02,416 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 1.463405ms + 0.01451ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (25) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:02,418 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 1.830515ms + 0.021119ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (25) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:02,420 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 1.532034ms + 0.015777ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (25) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:02,422 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 2.2177ms + 0.048155ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (16) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:02,424 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 1.98944ms + 0.044441ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (25) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:29:02,462 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 0.666388ms + 0.005389ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc] 2018-01-05T11:29:02,471 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,471 DEBUG [main] utils.FileUtils: deleting file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/dest3 2018-01-05T11:29:02,472 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,472 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,472 INFO [main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest4 2018-01-05T11:29:02,472 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest4 2018-01-05T11:29:02,474 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,475 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,475 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,475 INFO [main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest4 2018-01-05T11:29:02,475 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest4 2018-01-05T11:29:02,477 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,477 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,477 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,477 INFO [main] metastore.HiveMetaStore: 0: drop_table : db=default tbl=dest4 2018-01-05T11:29:02,478 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=drop_table : db=default tbl=dest4 2018-01-05T11:29:02,480 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 0.141752ms + 0.004317ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc] 2018-01-05T11:29:02,486 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,486 DEBUG [main] utils.FileUtils: deleting file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/dest4 2018-01-05T11:29:02,486 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,486 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,486 INFO [main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest4_sequencefile 2018-01-05T11:29:02,486 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest4_sequencefile 2018-01-05T11:29:02,488 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,489 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,489 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,489 INFO [main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest4_sequencefile 2018-01-05T11:29:02,489 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest4_sequencefile 2018-01-05T11:29:02,491 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,491 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,491 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,491 INFO [main] metastore.HiveMetaStore: 0: drop_table : db=default tbl=dest4_sequencefile 2018-01-05T11:29:02,491 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=drop_table : db=default tbl=dest4_sequencefile 2018-01-05T11:29:02,493 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 0.12833ms + 0.003902ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc] 2018-01-05T11:29:02,500 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,500 DEBUG [main] utils.FileUtils: deleting file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/dest4_sequencefile 2018-01-05T11:29:02,500 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,500 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,500 INFO [main] metastore.HiveMetaStore: 0: get_indexes : db=default tbl=lineitem 2018-01-05T11:29:02,500 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_indexes : db=default tbl=lineitem 2018-01-05T11:29:02,501 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,501 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,501 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,501 INFO [main] metastore.HiveMetaStore: 0: get_indexes : db=default tbl=part 2018-01-05T11:29:02,501 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_indexes : db=default tbl=part 2018-01-05T11:29:02,501 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,501 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,501 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,501 INFO [main] metastore.HiveMetaStore: 0: get_table : db=default tbl=primitives 2018-01-05T11:29:02,501 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=primitives 2018-01-05T11:29:02,503 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,504 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,504 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,504 INFO [main] metastore.HiveMetaStore: 0: get_table : db=default tbl=primitives 2018-01-05T11:29:02,504 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=primitives 2018-01-05T11:29:02,506 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,507 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,507 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,507 INFO [main] metastore.HiveMetaStore: 0: drop_table : db=default tbl=primitives 2018-01-05T11:29:02,507 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=drop_table : db=default tbl=primitives 2018-01-05T11:29:02,509 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 0.137837ms + 0.004557ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc] 2018-01-05T11:29:02,512 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 3.357496ms + 0.090308ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (5,6,7,8) order by "PART_NAME" asc] 2018-01-05T11:29:02,514 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 1.671663ms + 0.188091ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (5,6,7,8) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:29:02,516 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 2.185927ms + 0.051118ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (5,6,7,8) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:02,518 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 1.957279ms + 0.032923ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (18,19,20,21) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:29:02,520 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 1.891397ms + 0.017509ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (18,19,20,21) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:02,522 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 1.984543ms + 0.018485ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (18,19,20,21) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:02,524 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 1.661846ms + 0.01468ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (18,19,20,21) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:02,526 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 1.577897ms + 0.060005ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (13) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:02,527 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 1.408542ms + 0.162153ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (18,19,20,21) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:29:02,562 DEBUG [AMReporterQueueDrainer] impl.AMReporter: Removing am 10.22.8.182:49824 with last associated dag QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=17} from heartbeat with taskCount=0, amFailed=false, isDone=true 2018-01-05T11:29:02,571 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 0.303396ms + 0.007333ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc] 2018-01-05T11:29:02,580 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,580 DEBUG [main] utils.FileUtils: deleting file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/primitives 2018-01-05T11:29:02,582 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,582 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,582 INFO [main] metastore.HiveMetaStore: 0: get_indexes : db=default tbl=src 2018-01-05T11:29:02,582 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_indexes : db=default tbl=src 2018-01-05T11:29:02,583 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,583 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,583 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,583 INFO [main] metastore.HiveMetaStore: 0: get_indexes : db=default tbl=src1 2018-01-05T11:29:02,583 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_indexes : db=default tbl=src1 2018-01-05T11:29:02,583 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,583 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,583 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,583 INFO [main] metastore.HiveMetaStore: 0: get_indexes : db=default tbl=src_cbo 2018-01-05T11:29:02,583 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_indexes : db=default tbl=src_cbo 2018-01-05T11:29:02,584 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,584 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,584 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,584 INFO [main] metastore.HiveMetaStore: 0: get_indexes : db=default tbl=src_json 2018-01-05T11:29:02,584 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_indexes : db=default tbl=src_json 2018-01-05T11:29:02,584 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,584 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,584 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,584 INFO [main] metastore.HiveMetaStore: 0: get_indexes : db=default tbl=src_sequencefile 2018-01-05T11:29:02,584 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_indexes : db=default tbl=src_sequencefile 2018-01-05T11:29:02,585 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,585 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,585 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,585 INFO [main] metastore.HiveMetaStore: 0: get_indexes : db=default tbl=src_thrift 2018-01-05T11:29:02,585 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_indexes : db=default tbl=src_thrift 2018-01-05T11:29:02,585 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,585 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,585 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,585 INFO [main] metastore.HiveMetaStore: 0: get_indexes : db=default tbl=srcbucket 2018-01-05T11:29:02,585 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_indexes : db=default tbl=srcbucket 2018-01-05T11:29:02,585 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,585 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,585 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,585 INFO [main] metastore.HiveMetaStore: 0: get_indexes : db=default tbl=srcbucket2 2018-01-05T11:29:02,585 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_indexes : db=default tbl=srcbucket2 2018-01-05T11:29:02,586 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,586 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,586 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,586 INFO [main] metastore.HiveMetaStore: 0: get_indexes : db=default tbl=srcpart 2018-01-05T11:29:02,586 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_indexes : db=default tbl=srcpart 2018-01-05T11:29:02,586 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,587 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,587 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,590 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,590 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,590 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,590 INFO [main] metastore.HiveMetaStore: 0: get_functions: db=default pat=.* 2018-01-05T11:29:02,590 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_functions: db=default pat=.* 2018-01-05T11:29:02,594 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,594 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,594 INFO [main] control.CoreCliDriver: PerTestSetup done. ElapsedTime(ms)=392 2018-01-05T11:29:02,595 INFO [main] control.CoreCliDriver: Begin query: mergejoin.q 2018-01-05T11:29:02,595 DEBUG [main] session.SessionState: SessionState user: null 2018-01-05T11:29:02,596 INFO [main] DependencyResolver: ivysettings.xml file not found in HIVE_HOME or HIVE_CONF_DIR,/Users/djaiswal/hive/hive/conf/ivysettings.xml will be used 2018-01-05T11:29:02,599 DEBUG [main] session.SessionState: Removing resource dir /Users/djaiswal/hive/hive/itests/qtest/target/tmp/53565c10-a9a1-4685-81b4-afcf82ce77fb_resources 2018-01-05T11:29:02,600 INFO [main] session.SessionState: Deleted directory: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/53565c10-a9a1-4685-81b4-afcf82ce77fb on fs with scheme file 2018-01-05T11:29:02,616 INFO [main] session.SessionState: Deleted directory: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/53565c10-a9a1-4685-81b4-afcf82ce77fb on fs with scheme file 2018-01-05T11:29:02,616 DEBUG [main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:02,616 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,616 INFO [main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:29:02,616 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:29:02,616 INFO [main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:02,616 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:02,616 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:02,630 DEBUG [main] session.SessionState: HDFS root scratch dir: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir with schema null, permission: rwx-wx-wx 2018-01-05T11:29:02,642 INFO [main] session.SessionState: Created HDFS directory: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:02,653 INFO [main] session.SessionState: Created local directory: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:02,667 INFO [main] session.SessionState: Created HDFS directory: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/dfdd8629-6c16-4f61-9218-1f1e0156f79c/_tmp_space.db 2018-01-05T11:29:02,668 WARN [main] session.SessionState: Tez session was already present in SessionState before start: sessionId=9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, queueName=null, user=djaiswal, doAs=true, isOpen=true, isDefault=false 2018-01-05T11:29:02,668 DEBUG [main] CliDriver: CliDriver inited with classpath /Users/djaiswal/hive/hive/itests/qtest/target/test-classes:/Users/djaiswal/hive/hive/itests/qtest/target/classes:/Users/djaiswal/.m2/repository/org/apache/hive/hive-common/3.0.0-SNAPSHOT/hive-common-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-classification/3.0.0-SNAPSHOT/hive-classification-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-shims/3.0.0-SNAPSHOT/hive-shims-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/shims/hive-shims-common/3.0.0-SNAPSHOT/hive-shims-common-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/shims/hive-shims-0.23/3.0.0-SNAPSHOT/hive-shims-0.23-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/shims/hive-shims-scheduler/3.0.0-SNAPSHOT/hive-shims-scheduler-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-storage-api/3.0.0-SNAPSHOT/hive-storage-api-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/commons-cli/commons-cli/1.2/commons-cli-1.2.jar:/Users/djaiswal/.m2/repository/commons-lang/commons-lang/2.6/commons-lang-2.6.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-lang3/3.2/commons-lang3-3.2.jar:/Users/djaiswal/.m2/repository/org/apache/orc/orc-core/1.4.1/orc-core-1.4.1.jar:/Users/djaiswal/.m2/repository/io/airlift/aircompressor/0.8/aircompressor-0.8.jar:/Users/djaiswal/.m2/repository/io/airlift/slice/0.29/slice-0.29.jar:/Users/djaiswal/.m2/repository/org/openjdk/jol/jol-core/0.2/jol-core-0.2.jar:/Users/djaiswal/.m2/repository/jline/jline/2.12/jline-2.12.jar:/Users/djaiswal/.m2/repository/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-rewrite/9.3.8.v20160314/jetty-rewrite-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-client/9.3.8.v20160314/jetty-client-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-server/9.3.8.v20160314/jetty-server-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-http/9.3.8.v20160314/jetty-http-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-io/9.3.8.v20160314/jetty-io-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-servlet/9.3.8.v20160314/jetty-servlet-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-webapp/9.3.8.v20160314/jetty-webapp-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-xml/9.3.8.v20160314/jetty-xml-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/joda-time/joda-time/2.8.1/joda-time-2.8.1.jar:/Users/djaiswal/.m2/repository/org/apache/logging/log4j/log4j-1.2-api/2.8.2/log4j-1.2-api-2.8.2.jar:/Users/djaiswal/.m2/repository/org/apache/logging/log4j/log4j-web/2.8.2/log4j-web-2.8.2.jar:/Users/djaiswal/.m2/repository/org/apache/logging/log4j/log4j-slf4j-impl/2.8.2/log4j-slf4j-impl-2.8.2.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-compress/1.9/commons-compress-1.9.jar:/Users/djaiswal/.m2/repository/org/apache/ant/ant/1.9.1/ant-1.9.1.jar:/Users/djaiswal/.m2/repository/org/apache/ant/ant-launcher/1.9.1/ant-launcher-1.9.1.jar:/Users/djaiswal/.m2/repository/com/tdunning/json/1.8/json-1.8.jar:/Users/djaiswal/.m2/repository/io/dropwizard/metrics/metrics-core/3.1.0/metrics-core-3.1.0.jar:/Users/djaiswal/.m2/repository/io/dropwizard/metrics/metrics-jvm/3.1.0/metrics-jvm-3.1.0.jar:/Users/djaiswal/.m2/repository/io/dropwizard/metrics/metrics-json/3.1.0/metrics-json-3.1.0.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/core/jackson-databind/2.6.5/jackson-databind-2.6.5.jar:/Users/djaiswal/.m2/repository/com/github/joshelser/dropwizard-metrics-hadoop-metrics2-reporter/0.1.2/dropwizard-metrics-hadoop-metrics2-reporter-0.1.2.jar:/Users/djaiswal/.m2/repository/javolution/javolution/5.5.1/javolution-5.5.1.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-contrib/3.0.0-SNAPSHOT/hive-contrib-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/commons-codec/commons-codec/1.4/commons-codec-1.4.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-standalone-metastore/3.0.0-SNAPSHOT/hive-standalone-metastore-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/com/google/guava/guava/19.0/guava-19.0.jar:/Users/djaiswal/.m2/repository/com/google/protobuf/protobuf-java/2.5.0/protobuf-java-2.5.0.jar:/Users/djaiswal/.m2/repository/com/jolbox/bonecp/0.8.0.RELEASE/bonecp-0.8.0.RELEASE.jar:/Users/djaiswal/.m2/repository/com/zaxxer/HikariCP/2.6.1/HikariCP-2.6.1.jar:/Users/djaiswal/.m2/repository/commons-dbcp/commons-dbcp/1.4/commons-dbcp-1.4.jar:/Users/djaiswal/.m2/repository/commons-pool/commons-pool/1.5.4/commons-pool-1.5.4.jar:/Users/djaiswal/.m2/repository/org/antlr/antlr-runtime/3.5.2/antlr-runtime-3.5.2.jar:/Users/djaiswal/.m2/repository/org/apache/derby/derby/10.11.1.1/derby-10.11.1.1.jar:/Users/djaiswal/.m2/repository/org/apache/thrift/libfb303/0.9.3/libfb303-0.9.3.jar:/Users/djaiswal/.m2/repository/org/apache/thrift/libthrift/0.9.3/libthrift-0.9.3.jar:/Users/djaiswal/.m2/repository/org/apache/httpcomponents/httpcore/4.4.4/httpcore-4.4.4.jar:/Users/djaiswal/.m2/repository/org/datanucleus/datanucleus-api-jdo/4.2.4/datanucleus-api-jdo-4.2.4.jar:/Users/djaiswal/.m2/repository/org/datanucleus/datanucleus-core/4.1.17/datanucleus-core-4.1.17.jar:/Users/djaiswal/.m2/repository/org/datanucleus/datanucleus-rdbms/4.1.19/datanucleus-rdbms-4.1.19.jar:/Users/djaiswal/.m2/repository/org/datanucleus/javax.jdo/3.2.0-m3/javax.jdo-3.2.0-m3.jar:/Users/djaiswal/.m2/repository/javax/transaction/transaction-api/1.1/transaction-api-1.1.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-standalone-metastore/3.0.0-SNAPSHOT/hive-standalone-metastore-3.0.0-SNAPSHOT-tests.jar:/Users/djaiswal/hive/hive/itests/custom-serde/target/classes:/Users/djaiswal/hive/hive/itests/hive-unit/target/test-classes:/Users/djaiswal/.m2/repository/org/apache/hive/hive-jdbc/3.0.0-SNAPSHOT/hive-jdbc-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-service/3.0.0-SNAPSHOT/hive-service-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/javax/servlet/jsp/javax.servlet.jsp-api/2.3.1/javax.servlet.jsp-api-2.3.1.jar:/Users/djaiswal/.m2/repository/net/sf/jpam/jpam/1.1/jpam-1.1.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-runner/9.3.8.v20160314/jetty-runner-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-plus/9.3.8.v20160314/jetty-plus-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-annotations/9.3.8.v20160314/jetty-annotations-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-jaas/9.3.8.v20160314/jetty-jaas-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/websocket/websocket-server/9.3.8.v20160314/websocket-server-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/websocket/websocket-common/9.3.8.v20160314/websocket-common-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/websocket/websocket-api/9.3.8.v20160314/websocket-api-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/websocket/websocket-client/9.3.8.v20160314/websocket-client-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/websocket/websocket-servlet/9.3.8.v20160314/websocket-servlet-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-jndi/9.3.8.v20160314/jetty-jndi-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/apache-jsp/9.3.8.v20160314/apache-jsp-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/toolchain/jetty-schemas/3.1/jetty-schemas-3.1.jar:/Users/djaiswal/.m2/repository/org/eclipse/jdt/core/compiler/ecj/4.4.2/ecj-4.4.2.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/apache-jstl/9.3.8.v20160314/apache-jstl-9.3.8.v20160314.jar:/Users/djaiswal/.m2/repository/org/apache/taglibs/taglibs-standard-spec/1.2.5/taglibs-standard-spec-1.2.5.jar:/Users/djaiswal/.m2/repository/org/apache/taglibs/taglibs-standard-impl/1.2.5/taglibs-standard-impl-1.2.5.jar:/Users/djaiswal/.m2/repository/tomcat/jasper-compiler/5.5.23/jasper-compiler-5.5.23.jar:/Users/djaiswal/.m2/repository/javax/servlet/jsp-api/2.0/jsp-api-2.0.jar:/Users/djaiswal/.m2/repository/ant/ant/1.6.5/ant-1.6.5.jar:/Users/djaiswal/.m2/repository/tomcat/jasper-runtime/5.5.23/jasper-runtime-5.5.23.jar:/Users/djaiswal/.m2/repository/commons-el/commons-el/1.0/commons-el-1.0.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-llap-ext-client/3.0.0-SNAPSHOT/hive-llap-ext-client-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hcatalog/hive-hcatalog-core/3.0.0-SNAPSHOT/hive-hcatalog-core-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.9.13/jackson-mapper-asl-1.9.13.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hcatalog/hive-hcatalog-streaming/3.0.0-SNAPSHOT/hive-hcatalog-streaming-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hcatalog/hive-hcatalog-server-extensions/3.0.0-SNAPSHOT/hive-hcatalog-server-extensions-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/javax/jms/jms/1.1/jms-1.1.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hcatalog/hive-webhcat-java-client/3.0.0-SNAPSHOT/hive-webhcat-java-client-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-service/3.0.0-SNAPSHOT/hive-service-3.0.0-SNAPSHOT-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-exec/3.0.0-SNAPSHOT/hive-exec-3.0.0-SNAPSHOT-tests.jar:/Users/djaiswal/hive/hive/itests/util/target/classes:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-minicluster/1.7.3/accumulo-minicluster-1.7.3.jar:/Users/djaiswal/.m2/repository/com/beust/jcommander/1.32/jcommander-1.32.jar:/Users/djaiswal/.m2/repository/commons-configuration/commons-configuration/1.6/commons-configuration-1.6.jar:/Users/djaiswal/.m2/repository/commons-digester/commons-digester/1.8/commons-digester-1.8.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-core/1.7.3/accumulo-core-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-math/2.1/commons-math-2.1.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-gc/1.7.3/accumulo-gc-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-master/1.7.3/accumulo-master-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-monitor/1.7.3/accumulo-monitor-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-shell/1.7.3/accumulo-shell-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-server-base/1.7.3/accumulo-server-base-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-start/1.7.3/accumulo-start-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-tracer/1.7.3/accumulo-tracer-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-tserver/1.7.3/accumulo-tserver-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-vfs2/2.1/commons-vfs2-2.1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-minicluster/3.0.0-beta1/hadoop-minicluster-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-accumulo-handler/3.0.0-SNAPSHOT/hive-accumulo-handler-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-fate/1.7.3/accumulo-fate-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/accumulo/accumulo-trace/1.7.3/accumulo-trace-1.7.3.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-accumulo-handler/3.0.0-SNAPSHOT/hive-accumulo-handler-3.0.0-SNAPSHOT-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-cli/3.0.0-SNAPSHOT/hive-cli-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-beeline/3.0.0-SNAPSHOT/hive-beeline-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/net/sf/supercsv/super-csv/2.2.0/super-csv-2.2.0.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-hbase-handler/3.0.0-SNAPSHOT/hive-hbase-handler-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-hbase-handler/3.0.0-SNAPSHOT/hive-hbase-handler-3.0.0-SNAPSHOT-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-metastore/3.0.0-SNAPSHOT/hive-metastore-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/javax/jdo/jdo-api/3.0.1/jdo-api-3.0.1.jar:/Users/djaiswal/.m2/repository/javax/transaction/jta/1.1/jta-1.1.jar:/Users/djaiswal/.m2/repository/co/cask/tephra/tephra-api/0.6.0/tephra-api-0.6.0.jar:/Users/djaiswal/.m2/repository/co/cask/tephra/tephra-core/0.6.0/tephra-core-0.6.0.jar:/Users/djaiswal/.m2/repository/com/google/inject/extensions/guice-assistedinject/3.0/guice-assistedinject-3.0.jar:/Users/djaiswal/.m2/repository/org/apache/twill/twill-common/0.6.0-incubating/twill-common-0.6.0-incubating.jar:/Users/djaiswal/.m2/repository/org/apache/twill/twill-core/0.6.0-incubating/twill-core-0.6.0-incubating.jar:/Users/djaiswal/.m2/repository/org/apache/twill/twill-api/0.6.0-incubating/twill-api-0.6.0-incubating.jar:/Users/djaiswal/.m2/repository/org/apache/twill/twill-discovery-api/0.6.0-incubating/twill-discovery-api-0.6.0-incubating.jar:/Users/djaiswal/.m2/repository/org/apache/twill/twill-discovery-core/0.6.0-incubating/twill-discovery-core-0.6.0-incubating.jar:/Users/djaiswal/.m2/repository/org/apache/twill/twill-zookeeper/0.6.0-incubating/twill-zookeeper-0.6.0-incubating.jar:/Users/djaiswal/.m2/repository/co/cask/tephra/tephra-hbase-compat-1.0/0.6.0/tephra-hbase-compat-1.0-0.6.0.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-client/2.0.0-alpha4/hbase-client-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/jruby/jcodings/jcodings/1.0.18/jcodings-1.0.18.jar:/Users/djaiswal/.m2/repository/org/jruby/joni/joni/2.1.11/joni-2.1.11.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-mapreduce/2.0.0-alpha4/hbase-mapreduce-2.0.0-alpha4-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-serde/3.0.0-SNAPSHOT/hive-serde-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-service-rpc/3.0.0-SNAPSHOT/hive-service-rpc-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/com/google/code/findbugs/jsr305/3.0.0/jsr305-3.0.0.jar:/Users/djaiswal/.m2/repository/org/apache/avro/avro/1.7.7/avro-1.7.7.jar:/Users/djaiswal/.m2/repository/org/codehaus/jackson/jackson-core-asl/1.9.13/jackson-core-asl-1.9.13.jar:/Users/djaiswal/.m2/repository/com/thoughtworks/paranamer/paranamer/2.3/paranamer-2.3.jar:/Users/djaiswal/.m2/repository/org/xerial/snappy/snappy-java/1.1.4/snappy-java-1.1.4.jar:/Users/djaiswal/.m2/repository/net/sf/opencsv/opencsv/2.3/opencsv-2.3.jar:/Users/djaiswal/.m2/repository/org/apache/parquet/parquet-hadoop-bundle/1.9.0/parquet-hadoop-bundle-1.9.0.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-exec/3.0.0-SNAPSHOT/hive-exec-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-vector-code-gen/3.0.0-SNAPSHOT/hive-vector-code-gen-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/velocity/velocity/1.5/velocity-1.5.jar:/Users/djaiswal/.m2/repository/oro/oro/2.0.8/oro-2.0.8.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-llap-tez/3.0.0-SNAPSHOT/hive-llap-tez-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/commons-httpclient/commons-httpclient/3.0.1/commons-httpclient-3.0.1.jar:/Users/djaiswal/.m2/repository/commons-io/commons-io/2.4/commons-io-2.4.jar:/Users/djaiswal/.m2/repository/org/antlr/ST4/4.0.4/ST4-4.0.4.jar:/Users/djaiswal/.m2/repository/org/apache/ivy/ivy/2.4.0/ivy-2.4.0.jar:/Users/djaiswal/.m2/repository/org/apache/zookeeper/zookeeper/3.4.6/zookeeper-3.4.6.jar:/Users/djaiswal/.m2/repository/org/apache/curator/curator-framework/2.12.0/curator-framework-2.12.0.jar:/Users/djaiswal/.m2/repository/org/codehaus/groovy/groovy-all/2.4.11/groovy-all-2.4.11.jar:/Users/djaiswal/.m2/repository/org/apache/calcite/calcite-core/1.15.0/calcite-core-1.15.0.jar:/Users/djaiswal/.m2/repository/org/apache/calcite/calcite-linq4j/1.15.0/calcite-linq4j-1.15.0.jar:/Users/djaiswal/.m2/repository/com/esri/geometry/esri-geometry-api/2.0.0/esri-geometry-api-2.0.0.jar:/Users/djaiswal/.m2/repository/com/yahoo/datasketches/sketches-core/0.9.0/sketches-core-0.9.0.jar:/Users/djaiswal/.m2/repository/com/yahoo/datasketches/memory/0.9.0/memory-0.9.0.jar:/Users/djaiswal/.m2/repository/org/codehaus/janino/janino/2.7.6/janino-2.7.6.jar:/Users/djaiswal/.m2/repository/org/codehaus/janino/commons-compiler/2.7.6/commons-compiler-2.7.6.jar:/Users/djaiswal/.m2/repository/org/apache/calcite/calcite-druid/1.15.0/calcite-druid-1.15.0.jar:/Users/djaiswal/.m2/repository/org/apache/calcite/avatica/avatica/1.10.0/avatica-1.10.0.jar:/Users/djaiswal/.m2/repository/com/google/code/gson/gson/2.2.4/gson-2.2.4.jar:/Users/djaiswal/.m2/repository/stax/stax-api/1.0.1/stax-api-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-registry/3.0.0-beta1/hadoop-yarn-registry-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-druid-handler/3.0.0-SNAPSHOT/hive-druid-handler-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/com/metamx/java-util/0.27.10/java-util-0.27.10.jar:/Users/djaiswal/.m2/repository/org/skife/config/config-magic/0.9/config-magic-0.9.jar:/Users/djaiswal/.m2/repository/org/mozilla/rhino/1.7R5/rhino-1.7R5.jar:/Users/djaiswal/.m2/repository/com/jayway/jsonpath/json-path/2.1.0/json-path-2.1.0.jar:/Users/djaiswal/.m2/repository/io/druid/druid-server/0.10.1/druid-server-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/druid-console/0.0.2/druid-console-0.0.2.jar:/Users/djaiswal/.m2/repository/com/metamx/http-client/1.1.0/http-client-1.1.0.jar:/Users/djaiswal/.m2/repository/com/metamx/server-metrics/0.2.8/server-metrics-0.2.8.jar:/Users/djaiswal/.m2/repository/javax/inject/javax.inject/1/javax.inject-1.jar:/Users/djaiswal/.m2/repository/org/glassfish/javax.el/3.0.0/javax.el-3.0.0.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/jaxrs/jackson-jaxrs-smile-provider/2.4.6/jackson-jaxrs-smile-provider-2.4.6.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/dataformat/jackson-dataformat-smile/2.4.6/jackson-dataformat-smile-2.4.6.jar:/Users/djaiswal/.m2/repository/io/tesla/aether/tesla-aether/0.0.5/tesla-aether-0.0.5.jar:/Users/djaiswal/.m2/repository/org/eclipse/aether/aether-spi/0.9.0.M2/aether-spi-0.9.0.M2.jar:/Users/djaiswal/.m2/repository/org/eclipse/aether/aether-util/0.9.0.M2/aether-util-0.9.0.M2.jar:/Users/djaiswal/.m2/repository/org/eclipse/aether/aether-impl/0.9.0.M2/aether-impl-0.9.0.M2.jar:/Users/djaiswal/.m2/repository/org/eclipse/aether/aether-connector-file/0.9.0.M2/aether-connector-file-0.9.0.M2.jar:/Users/djaiswal/.m2/repository/io/tesla/aether/aether-connector-okhttp/0.0.9/aether-connector-okhttp-0.0.9.jar:/Users/djaiswal/.m2/repository/org/apache/maven/wagon/wagon-provider-api/2.4/wagon-provider-api-2.4.jar:/Users/djaiswal/.m2/repository/org/apache/maven/maven-aether-provider/3.1.1/maven-aether-provider-3.1.1.jar:/Users/djaiswal/.m2/repository/org/apache/maven/maven-model/3.1.1/maven-model-3.1.1.jar:/Users/djaiswal/.m2/repository/org/apache/maven/maven-model-builder/3.1.1/maven-model-builder-3.1.1.jar:/Users/djaiswal/.m2/repository/org/apache/maven/maven-repository-metadata/3.1.1/maven-repository-metadata-3.1.1.jar:/Users/djaiswal/.m2/repository/org/codehaus/plexus/plexus-utils/3.0.15/plexus-utils-3.0.15.jar:/Users/djaiswal/.m2/repository/org/apache/maven/maven-settings-builder/3.1.1/maven-settings-builder-3.1.1.jar:/Users/djaiswal/.m2/repository/org/codehaus/plexus/plexus-interpolation/1.19/plexus-interpolation-1.19.jar:/Users/djaiswal/.m2/repository/org/apache/maven/maven-settings/3.1.1/maven-settings-3.1.1.jar:/Users/djaiswal/.m2/repository/org/eclipse/aether/aether-api/0.9.0.M2/aether-api-0.9.0.M2.jar:/Users/djaiswal/.m2/repository/net/spy/spymemcached/2.12.3/spymemcached-2.12.3.jar:/Users/djaiswal/.m2/repository/net/jpountz/lz4/lz4/1.3.0/lz4-1.3.0.jar:/Users/djaiswal/.m2/repository/com/ircclouds/irc/irc-api/1.0-0014/irc-api-1.0-0014.jar:/Users/djaiswal/.m2/repository/com/maxmind/geoip2/geoip2/0.4.0/geoip2-0.4.0.jar:/Users/djaiswal/.m2/repository/com/maxmind/maxminddb/maxminddb/0.2.0/maxminddb-0.2.0.jar:/Users/djaiswal/.m2/repository/org/apache/derby/derbynet/10.11.1.1/derbynet-10.11.1.1.jar:/Users/djaiswal/.m2/repository/org/apache/derby/derbyclient/10.11.1.1/derbyclient-10.11.1.1.jar:/Users/djaiswal/.m2/repository/it/unimi/dsi/fastutil/7.2.0/fastutil-7.2.0.jar:/Users/djaiswal/.m2/repository/io/druid/druid-processing/0.10.1/druid-processing-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/druid-common/0.10.1/druid-common-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/java-util/0.10.1/java-util-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/druid-api/0.10.1/druid-api-0.10.1.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-dbcp2/2.0.1/commons-dbcp2-2.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-pool2/2.2/commons-pool2-2.2.jar:/Users/djaiswal/.m2/repository/org/hibernate/hibernate-validator/5.1.3.Final/hibernate-validator-5.1.3.Final.jar:/Users/djaiswal/.m2/repository/org/jboss/logging/jboss-logging/3.1.3.GA/jboss-logging-3.1.3.GA.jar:/Users/djaiswal/.m2/repository/com/fasterxml/classmate/1.0.0/classmate-1.0.0.jar:/Users/djaiswal/.m2/repository/javax/el/javax.el-api/3.0.0/javax.el-api-3.0.0.jar:/Users/djaiswal/.m2/repository/javax/validation/validation-api/1.1.0.Final/validation-api-1.1.0.Final.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/datatype/jackson-datatype-guava/2.4.6/jackson-datatype-guava-2.4.6.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/datatype/jackson-datatype-joda/2.4.6/jackson-datatype-joda-2.4.6.jar:/Users/djaiswal/.m2/repository/com/google/inject/extensions/guice-multibindings/4.1.0/guice-multibindings-4.1.0.jar:/Users/djaiswal/.m2/repository/org/jdbi/jdbi/2.63.1/jdbi-2.63.1.jar:/Users/djaiswal/.m2/repository/org/apache/logging/log4j/log4j-jul/2.5/log4j-jul-2.5.jar:/Users/djaiswal/.m2/repository/org/slf4j/jcl-over-slf4j/1.7.12/jcl-over-slf4j-1.7.12.jar:/Users/djaiswal/.m2/repository/net/java/dev/jets3t/jets3t/0.9.4/jets3t-0.9.4.jar:/Users/djaiswal/.m2/repository/javax/activation/activation/1.1.1/activation-1.1.1.jar:/Users/djaiswal/.m2/repository/org/bouncycastle/bcprov-jdk15on/1.52/bcprov-jdk15on-1.52.jar:/Users/djaiswal/.m2/repository/com/jamesmurty/utils/java-xmlbuilder/1.1/java-xmlbuilder-1.1.jar:/Users/djaiswal/.m2/repository/net/iharder/base64/2.3.8/base64-2.3.8.jar:/Users/djaiswal/.m2/repository/org/antlr/antlr4-runtime/4.5.1/antlr4-runtime-4.5.1.jar:/Users/djaiswal/.m2/repository/io/druid/druid-hll/0.10.1/druid-hll-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/bytebuffer-collections/0.10.1/bytebuffer-collections-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/extendedset/0.10.1/extendedset-0.10.1.jar:/Users/djaiswal/.m2/repository/com/metamx/emitter/0.4.5/emitter-0.4.5.jar:/Users/djaiswal/.m2/repository/com/ning/compress-lzf/1.0.3/compress-lzf-1.0.3.jar:/Users/djaiswal/.m2/repository/com/ibm/icu/icu4j/54.1.1/icu4j-54.1.1.jar:/Users/djaiswal/.m2/repository/org/mapdb/mapdb/1.0.8/mapdb-1.0.8.jar:/Users/djaiswal/.m2/repository/org/ow2/asm/asm/5.2/asm-5.2.jar:/Users/djaiswal/.m2/repository/org/ow2/asm/asm-commons/5.2/asm-commons-5.2.jar:/Users/djaiswal/.m2/repository/org/ow2/asm/asm-tree/5.2/asm-tree-5.2.jar:/Users/djaiswal/.m2/repository/io/druid/extensions/druid-hdfs-storage/0.10.1/druid-hdfs-storage-0.10.1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-aws/2.7.3/hadoop-aws-2.7.3.jar:/Users/djaiswal/.m2/repository/com/amazonaws/aws-java-sdk-s3/1.10.77/aws-java-sdk-s3-1.10.77.jar:/Users/djaiswal/.m2/repository/com/amazonaws/aws-java-sdk-kms/1.10.77/aws-java-sdk-kms-1.10.77.jar:/Users/djaiswal/.m2/repository/com/amazonaws/aws-java-sdk-core/1.10.77/aws-java-sdk-core-1.10.77.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/dataformat/jackson-dataformat-cbor/2.5.3/jackson-dataformat-cbor-2.5.3.jar:/Users/djaiswal/.m2/repository/io/druid/extensions/mysql-metadata-storage/0.10.1/mysql-metadata-storage-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/extensions/postgresql-metadata-storage/0.10.1/postgresql-metadata-storage-0.10.1.jar:/Users/djaiswal/.m2/repository/org/postgresql/postgresql/9.4.1208.jre7/postgresql-9.4.1208.jre7.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-druid-handler/3.0.0-SNAPSHOT/hive-druid-handler-3.0.0-SNAPSHOT-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-jdbc-handler/3.0.0-SNAPSHOT/hive-jdbc-handler-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/junit/junit/4.11/junit-4.11.jar:/Users/djaiswal/.m2/repository/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar:/Users/djaiswal/.m2/repository/com/sun/jersey/jersey-servlet/1.19/jersey-servlet-1.19.jar:/Users/djaiswal/.m2/repository/com/sun/jersey/jersey-server/1.19/jersey-server-1.19.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-archives/3.0.0-beta1/hadoop-archives-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-common/3.0.0-beta1/hadoop-common-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-annotations/3.0.0-beta1/hadoop-annotations-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-math3/3.1.1/commons-math3-3.1.1.jar:/Users/djaiswal/.m2/repository/commons-net/commons-net/3.1/commons-net-3.1.jar:/Users/djaiswal/.m2/repository/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-util/9.3.19.v20170502/jetty-util-9.3.19.v20170502.jar:/Users/djaiswal/.m2/repository/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.jar:/Users/djaiswal/.m2/repository/com/sun/jersey/jersey-core/1.19/jersey-core-1.19.jar:/Users/djaiswal/.m2/repository/javax/ws/rs/jsr311-api/1.1.1/jsr311-api-1.1.1.jar:/Users/djaiswal/.m2/repository/com/sun/jersey/jersey-json/1.19/jersey-json-1.19.jar:/Users/djaiswal/.m2/repository/com/sun/xml/bind/jaxb-impl/2.2.3-1/jaxb-impl-2.2.3-1.jar:/Users/djaiswal/.m2/repository/org/codehaus/jackson/jackson-jaxrs/1.9.13/jackson-jaxrs-1.9.13.jar:/Users/djaiswal/.m2/repository/org/codehaus/jackson/jackson-xc/1.9.13/jackson-xc-1.9.13.jar:/Users/djaiswal/.m2/repository/commons-logging/commons-logging/1.1.3/commons-logging-1.1.3.jar:/Users/djaiswal/.m2/repository/log4j/log4j/1.2.17/log4j-1.2.17.jar:/Users/djaiswal/.m2/repository/commons-beanutils/commons-beanutils/1.9.3/commons-beanutils-1.9.3.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-configuration2/2.1.1/commons-configuration2-2.1.1.jar:/Users/djaiswal/.m2/repository/com/google/re2j/re2j/1.1/re2j-1.1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-auth/3.0.0-beta1/hadoop-auth-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/com/nimbusds/nimbus-jose-jwt/4.41.1/nimbus-jose-jwt-4.41.1.jar:/Users/djaiswal/.m2/repository/com/github/stephenc/jcip/jcip-annotations/1.0-1/jcip-annotations-1.0-1.jar:/Users/djaiswal/.m2/repository/net/minidev/json-smart/2.3/json-smart-2.3.jar:/Users/djaiswal/.m2/repository/net/minidev/accessors-smart/1.2/accessors-smart-1.2.jar:/Users/djaiswal/.m2/repository/com/jcraft/jsch/0.1.54/jsch-0.1.54.jar:/Users/djaiswal/.m2/repository/org/apache/curator/curator-client/2.12.0/curator-client-2.12.0.jar:/Users/djaiswal/.m2/repository/org/apache/curator/curator-recipes/2.12.0/curator-recipes-2.12.0.jar:/Users/djaiswal/.m2/repository/org/apache/htrace/htrace-core4/4.1.0-incubating/htrace-core4-4.1.0-incubating.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerb-simplekdc/1.0.1/kerb-simplekdc-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerb-client/1.0.1/kerb-client-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerby-config/1.0.1/kerby-config-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerb-core/1.0.1/kerb-core-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerby-pkix/1.0.1/kerby-pkix-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerby-asn1/1.0.1/kerby-asn1-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerby-util/1.0.1/kerby-util-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerb-common/1.0.1/kerb-common-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerb-crypto/1.0.1/kerb-crypto-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerb-util/1.0.1/kerb-util-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/token-provider/1.0.1/token-provider-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerb-admin/1.0.1/kerb-admin-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerb-server/1.0.1/kerb-server-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerb-identity/1.0.1/kerb-identity-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/kerby/kerby-xdr/1.0.1/kerby-xdr-1.0.1.jar:/Users/djaiswal/.m2/repository/org/codehaus/woodstox/stax2-api/3.1.4/stax2-api-3.1.4.jar:/Users/djaiswal/.m2/repository/com/fasterxml/woodstox/woodstox-core/5.0.3/woodstox-core-5.0.3.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-common/3.0.0-beta1/hadoop-common-3.0.0-beta1-tests.jar:/Users/djaiswal/.m2/repository/org/apache/httpcomponents/httpclient/4.5.2/httpclient-4.5.2.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-distcp/3.0.0-beta1/hadoop-distcp-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-hdfs/3.0.0-beta1/hadoop-hdfs-3.0.0-beta1-tests.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-util-ajax/9.3.19.v20170502/jetty-util-ajax-9.3.19.v20170502.jar:/Users/djaiswal/.m2/repository/commons-daemon/commons-daemon/1.0.13/commons-daemon-1.0.13.jar:/Users/djaiswal/.m2/repository/io/netty/netty/3.10.5.Final/netty-3.10.5.Final.jar:/Users/djaiswal/.m2/repository/io/netty/netty-all/4.0.52.Final/netty-all-4.0.52.Final.jar:/Users/djaiswal/.m2/repository/org/fusesource/leveldbjni/leveldbjni-all/1.8/leveldbjni-all-1.8.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-hdfs/3.0.0-beta1/hadoop-hdfs-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-jobclient/3.0.0-beta1/hadoop-mapreduce-client-jobclient-3.0.0-beta1-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-common/3.0.0-beta1/hadoop-mapreduce-client-common-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/com/google/inject/extensions/guice-servlet/4.0/guice-servlet-4.0.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-hs/3.0.0-beta1/hadoop-mapreduce-client-hs-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-hdfs-client/3.0.0-beta1/hadoop-hdfs-client-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/com/squareup/okhttp/okhttp/2.4.0/okhttp-2.4.0.jar:/Users/djaiswal/.m2/repository/com/squareup/okio/okio/1.4.0/okio-1.4.0.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-app/3.0.0-beta1/hadoop-mapreduce-client-app-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-shuffle/3.0.0-beta1/hadoop-mapreduce-client-shuffle-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-core/3.0.0-beta1/hadoop-mapreduce-client-core-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-llap-server/3.0.0-SNAPSHOT/hive-llap-server-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-llap-common/3.0.0-SNAPSHOT/hive-llap-common-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-llap-client/3.0.0-SNAPSHOT/hive-llap-client-3.0.0-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/slider/slider-core/0.92.0-incubating/slider-core-0.92.0-incubating.jar:/Users/djaiswal/.m2/repository/org/yaml/snakeyaml/1.16/snakeyaml-1.16.jar:/Users/djaiswal/.m2/repository/org/codehaus/jettison/jettison/1.1/jettison-1.1.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-llap-common/3.0.0-SNAPSHOT/hive-llap-common-3.0.0-SNAPSHOT-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hive/hive-llap-server/3.0.0-SNAPSHOT/hive-llap-server-3.0.0-SNAPSHOT-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-server-tests/3.0.0-beta1/hadoop-yarn-server-tests-3.0.0-beta1-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-server-common/3.0.0-beta1/hadoop-yarn-server-common-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/geronimo/specs/geronimo-jcache_1.0_spec/1.0-alpha-1/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/Users/djaiswal/.m2/repository/org/ehcache/ehcache/3.3.1/ehcache-3.3.1.jar:/Users/djaiswal/.m2/repository/com/zaxxer/HikariCP-java7/2.4.12/HikariCP-java7-2.4.12.jar:/Users/djaiswal/.m2/repository/com/microsoft/sqlserver/mssql-jdbc/6.2.1.jre7/mssql-jdbc-6.2.1.jre7.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-server-nodemanager/3.0.0-beta1/hadoop-yarn-server-nodemanager-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/com/codahale/metrics/metrics-core/3.0.1/metrics-core-3.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-server-resourcemanager/3.0.0-beta1/hadoop-yarn-server-resourcemanager-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-server-applicationhistoryservice/3.0.0-beta1/hadoop-yarn-server-applicationhistoryservice-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/de/ruedigermoeller/fst/2.50/fst-2.50.jar:/Users/djaiswal/.m2/repository/com/cedarsoftware/java-util/1.9.0/java-util-1.9.0.jar:/Users/djaiswal/.m2/repository/com/cedarsoftware/json-io/2.5.1/json-io-2.5.1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-server-timelineservice/3.0.0-beta1/hadoop-yarn-server-timelineservice-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-csv/1.0/commons-csv-1.0.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-client/3.0.0-beta1/hadoop-yarn-client-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-common/3.0.0-beta1/hadoop-yarn-common-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/javax/xml/bind/jaxb-api/2.2.11/jaxb-api-2.2.11.jar:/Users/djaiswal/.m2/repository/com/sun/jersey/jersey-client/1.19/jersey-client-1.19.jar:/Users/djaiswal/.m2/repository/com/google/inject/guice/4.0/guice-4.0.jar:/Users/djaiswal/.m2/repository/aopalliance/aopalliance/1.0/aopalliance-1.0.jar:/Users/djaiswal/.m2/repository/com/sun/jersey/contribs/jersey-guice/1.19/jersey-guice-1.19.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/core/jackson-core/2.7.8/jackson-core-2.7.8.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/module/jackson-module-jaxb-annotations/2.7.8/jackson-module-jaxb-annotations-2.7.8.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/jaxrs/jackson-jaxrs-json-provider/2.7.8/jackson-jaxrs-json-provider-2.7.8.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/jaxrs/jackson-jaxrs-base/2.7.8/jackson-jaxrs-base-2.7.8.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-api/3.0.0-beta1/hadoop-yarn-api-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/com/fasterxml/jackson/core/jackson-annotations/2.7.8/jackson-annotations-2.7.8.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-common/2.0.0-alpha4/hbase-common-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/thirdparty/hbase-shaded-miscellaneous/1.0.1/hbase-shaded-miscellaneous-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-collections4/4.1/commons-collections4-4.1.jar:/Users/djaiswal/.m2/repository/org/apache/htrace/htrace-core/3.2.0-incubating/htrace-core-3.2.0-incubating.jar:/Users/djaiswal/.m2/repository/org/apache/commons/commons-crypto/1.0.0/commons-crypto-1.0.0.jar:/Users/djaiswal/.m2/repository/com/github/stephenc/findbugs/findbugs-annotations/1.3.9-1/findbugs-annotations-1.3.9-1.jar:/Users/djaiswal/.m2/repository/org/apache/yetus/audience-annotations/0.5.0/audience-annotations-0.5.0.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-common/2.0.0-alpha4/hbase-common-2.0.0-alpha4-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-hadoop-compat/2.0.0-alpha4/hbase-hadoop-compat-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-metrics-api/2.0.0-alpha4/hbase-metrics-api-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-hadoop-compat/2.0.0-alpha4/hbase-hadoop-compat-2.0.0-alpha4-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-hadoop2-compat/2.0.0-alpha4/hbase-hadoop2-compat-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-metrics/2.0.0-alpha4/hbase-metrics-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-hadoop2-compat/2.0.0-alpha4/hbase-hadoop2-compat-2.0.0-alpha4-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-server/2.0.0-alpha4/hbase-server-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/thirdparty/hbase-shaded-protobuf/1.0.1/hbase-shaded-protobuf-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/thirdparty/hbase-shaded-netty/1.0.1/hbase-shaded-netty-1.0.1.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-http/2.0.0-alpha4/hbase-http-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/glassfish/jersey/core/jersey-server/2.25.1/jersey-server-2.25.1.jar:/Users/djaiswal/.m2/repository/org/glassfish/jersey/core/jersey-common/2.25.1/jersey-common-2.25.1.jar:/Users/djaiswal/.m2/repository/org/glassfish/jersey/bundles/repackaged/jersey-guava/2.25.1/jersey-guava-2.25.1.jar:/Users/djaiswal/.m2/repository/org/glassfish/hk2/osgi-resource-locator/1.0.1/osgi-resource-locator-1.0.1.jar:/Users/djaiswal/.m2/repository/org/glassfish/jersey/core/jersey-client/2.25.1/jersey-client-2.25.1.jar:/Users/djaiswal/.m2/repository/org/glassfish/jersey/media/jersey-media-jaxb/2.25.1/jersey-media-jaxb-2.25.1.jar:/Users/djaiswal/.m2/repository/javax/annotation/javax.annotation-api/1.2/javax.annotation-api-1.2.jar:/Users/djaiswal/.m2/repository/org/glassfish/hk2/hk2-api/2.5.0-b32/hk2-api-2.5.0-b32.jar:/Users/djaiswal/.m2/repository/org/glassfish/hk2/hk2-utils/2.5.0-b32/hk2-utils-2.5.0-b32.jar:/Users/djaiswal/.m2/repository/org/glassfish/hk2/external/aopalliance-repackaged/2.5.0-b32/aopalliance-repackaged-2.5.0-b32.jar:/Users/djaiswal/.m2/repository/org/glassfish/hk2/external/javax.inject/2.5.0-b32/javax.inject-2.5.0-b32.jar:/Users/djaiswal/.m2/repository/org/glassfish/hk2/hk2-locator/2.5.0-b32/hk2-locator-2.5.0-b32.jar:/Users/djaiswal/.m2/repository/org/javassist/javassist/3.20.0-GA/javassist-3.20.0-GA.jar:/Users/djaiswal/.m2/repository/org/glassfish/jersey/containers/jersey-container-servlet-core/2.25.1/jersey-container-servlet-core-2.25.1.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-protocol/2.0.0-alpha4/hbase-protocol-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-protocol-shaded/2.0.0-alpha4/hbase-protocol-shaded-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-procedure/2.0.0-alpha4/hbase-procedure-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-replication/2.0.0-alpha4/hbase-replication-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-prefix-tree/2.0.0-alpha4/hbase-prefix-tree-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/glassfish/web/javax.servlet.jsp/2.3.2/javax.servlet.jsp-2.3.2.jar:/Users/djaiswal/.m2/repository/org/jamon/jamon-runtime/2.3.1/jamon-runtime-2.3.1.jar:/Users/djaiswal/.m2/repository/javax/ws/rs/javax.ws.rs-api/2.0.1/javax.ws.rs-api-2.0.1.jar:/Users/djaiswal/.m2/repository/com/lmax/disruptor/3.3.6/disruptor-3.3.6.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-client/3.0.0-beta1/hadoop-client-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-jobclient/3.0.0-beta1/hadoop-mapreduce-client-jobclient-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-server/2.0.0-alpha4/hbase-server-2.0.0-alpha4-tests.jar:/Users/djaiswal/.m2/repository/org/apache/hbase/hbase-mapreduce/2.0.0-alpha4/hbase-mapreduce-2.0.0-alpha4.jar:/Users/djaiswal/.m2/repository/org/apache/tez/tez-tests/0.9.1-SNAPSHOT/tez-tests-0.9.1-SNAPSHOT-tests.jar:/Users/djaiswal/.m2/repository/org/apache/tez/tez-common/0.9.1-SNAPSHOT/tez-common-0.9.1-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/slf4j/slf4j-log4j12/1.7.10/slf4j-log4j12-1.7.10.jar:/Users/djaiswal/.m2/repository/org/apache/tez/tez-examples/0.9.1-SNAPSHOT/tez-examples-0.9.1-SNAPSHOT.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_101.jdk/Contents/Home/jre/../lib/tools.jar:/Users/djaiswal/.m2/repository/org/apache/tez/tez-api/0.9.1-SNAPSHOT/tez-api-0.9.1-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/tez/tez-runtime-library/0.9.1-SNAPSHOT/tez-runtime-library-0.9.1-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/roaringbitmap/RoaringBitmap/0.4.9/RoaringBitmap-0.4.9.jar:/Users/djaiswal/.m2/repository/com/ning/async-http-client/1.8.16/async-http-client-1.8.16.jar:/Users/djaiswal/.m2/repository/org/apache/tez/tez-mapreduce/0.9.1-SNAPSHOT/tez-mapreduce-0.9.1-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/tez/tez-dag/0.9.1-SNAPSHOT/tez-dag-0.9.1-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/tez/hadoop-shim/0.9.1-SNAPSHOT/hadoop-shim-0.9.1-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/tez/tez-runtime-internals/0.9.1-SNAPSHOT/tez-runtime-internals-0.9.1-SNAPSHOT.jar:/Users/djaiswal/.m2/repository/org/apache/hadoop/hadoop-yarn-server-web-proxy/3.0.0-beta1/hadoop-yarn-server-web-proxy-3.0.0-beta1.jar:/Users/djaiswal/.m2/repository/org/mortbay/jetty/jetty/6.1.26/jetty-6.1.26.jar:/Users/djaiswal/.m2/repository/org/mortbay/jetty/servlet-api/2.5-20081211/servlet-api-2.5-20081211.jar:/Users/djaiswal/.m2/repository/org/mortbay/jetty/jetty-util/6.1.26/jetty-util-6.1.26.jar:/Users/djaiswal/.m2/repository/javax/servlet/servlet-api/2.5/servlet-api-2.5.jar:/Users/djaiswal/hive/hive/itests/qtest-druid/target/classes:/Users/djaiswal/.m2/repository/io/druid/druid-services/0.10.1/druid-services-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/druid-indexing-hadoop/0.10.1/druid-indexing-hadoop-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/druid-indexing-service/0.10.1/druid-indexing-service-0.10.1.jar:/Users/djaiswal/.m2/repository/io/druid/druid-sql/0.10.1/druid-sql-0.10.1.jar:/Users/djaiswal/.m2/repository/org/apache/calcite/avatica/avatica-core/1.9.0/avatica-core-1.9.0.jar:/Users/djaiswal/.m2/repository/org/apache/calcite/avatica/avatica-metrics/1.9.0/avatica-metrics-1.9.0.jar:/Users/djaiswal/.m2/repository/org/apache/calcite/avatica/avatica-server/1.9.0/avatica-server-1.9.0.jar:/Users/djaiswal/.m2/repository/io/airlift/airline/0.7/airline-0.7.jar:/Users/djaiswal/.m2/repository/org/apache/logging/log4j/log4j-api/2.8.2/log4j-api-2.8.2.jar:/Users/djaiswal/.m2/repository/org/apache/logging/log4j/log4j-core/2.8.2/log4j-core-2.8.2.jar:/Users/djaiswal/.m2/repository/org/apache/curator/curator-x-discovery/2.11.0/curator-x-discovery-2.11.0.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-servlets/9.3.19.v20170502/jetty-servlets-9.3.19.v20170502.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-continuation/9.3.19.v20170502/jetty-continuation-9.3.19.v20170502.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-proxy/9.3.19.v20170502/jetty-proxy-9.3.19.v20170502.jar:/Users/djaiswal/.m2/repository/org/eclipse/jetty/jetty-security/9.3.19.v20170502/jetty-security-9.3.19.v20170502.jar:/Users/djaiswal/.m2/repository/org/slf4j/slf4j-api/1.7.10/slf4j-api-1.7.10.jar:/Users/djaiswal/.m2/repository/org/mockito/mockito-all/1.10.19/mockito-all-1.10.19.jar:/Users/djaiswal/hive/hive/itests/qtest/target/testconf:/Users/djaiswal/hive/hive/itests/qtest/../../conf: 2018-01-05T11:29:02,670 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:02,670 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:02,670 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: false 2018-01-05T11:29:02,670 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:02,670 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:02,670 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:02,670 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:02,670 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: nonstrict 2018-01-05T11:29:02,670 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:02,670 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:02,670 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:02,670 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:02,670 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: false 2018-01-05T11:29:02,670 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:02,670 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:02,670 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:02,670 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:02,670 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: 100000 2018-01-05T11:29:02,670 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:02,670 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:02,670 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:02,670 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:02,670 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: true 2018-01-05T11:29:02,670 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:02,670 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:02,670 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:02,670 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:02,670 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: true 2018-01-05T11:29:02,670 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:02,670 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:02,670 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:02,670 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:02,671 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: true 2018-01-05T11:29:02,671 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:02,671 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:02,671 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:02,671 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:02,671 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: false 2018-01-05T11:29:02,671 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:02,671 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:02,671 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:02,671 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:02,671 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: true 2018-01-05T11:29:02,671 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:02,671 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:02,671 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:02,671 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:02,671 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: true 2018-01-05T11:29:02,671 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:02,671 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:02,671 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:02,671 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:02,671 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: 1 2018-01-05T11:29:02,671 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:02,671 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:02,671 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:02,671 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:02,671 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: 1 2018-01-05T11:29:02,671 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:02,671 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:02,671 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:02,671 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:02,671 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:02,671 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:02,671 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:02,671 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:02,671 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:02,671 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: explain vectorization detail select * from src a join src1 b on a.key = b.key 2018-01-05T11:29:02,671 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112902_f73e508d-7879-43ec-a4dc-1ca6a9418703): explain vectorization detail select * from src a join src1 b on a.key = b.key 2018-01-05T11:29:02,672 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:02,672 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: explain vectorization detail select * from src a join src1 b on a.key = b.key 2018-01-05T11:29:02,673 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:02,673 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:02,673 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:02,706 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:02,707 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:29:02,707 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:29:02,707 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:29:02,707 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:29:02,707 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,707 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,707 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,721 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,722 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,722 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:29:02,722 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:29:02,722 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:02,723 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:29:02,723 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:02,723 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:02,723 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:02,723 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:29:02,723 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] sqlstd.SQLStdHiveAccessController: Created SQLStdHiveAccessController for session context : HiveAuthzSessionContext [sessionString=dfdd8629-6c16-4f61-9218-1f1e0156f79c, clientType=HIVECLI] 2018-01-05T11:29:02,723 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: METASTORE_FILTER_HOOK will be ignored, since hive.security.authorization.manager is set to instance of HiveAuthorizerFactory. 2018-01-05T11:29:02,723 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStoreClient: Mestastore configuration metastore.filter.hook changed from org.apache.hadoop.hive.metastore.DefaultMetaStoreFilterHookImpl to org.apache.hadoop.hive.ql.security.authorization.plugin.AuthorizationMetaStoreFilterHook 2018-01-05T11:29:02,723 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@48d35c40, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:29:02,723 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:02,723 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:02,723 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:29:02,723 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:29:02,723 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:02,723 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:02,723 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:02,723 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:02,724 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:02,724 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:29:02,724 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:29:02,724 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:29:02,724 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:29:02,724 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,724 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,724 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,725 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,725 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,725 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:29:02,725 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:29:02,725 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:02,725 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:29:02,725 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Session is using authorization class class org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAuthorizerImpl 2018-01-05T11:29:02,725 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed phase 1 of Semantic Analysis 2018-01-05T11:29:02,725 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:02,726 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:29:02,726 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:29:02,726 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,726 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,726 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,726 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,726 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,726 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:29:02,726 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:29:02,726 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:02,727 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:02,727 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:29:02,727 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:02,727 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src 2018-01-05T11:29:02,727 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src 2018-01-05T11:29:02,730 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,731 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:02,731 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:02,731 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src1 2018-01-05T11:29:02,731 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src1 2018-01-05T11:29:02,733 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,734 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:02,734 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:02,734 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:02,734 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed getting MetaData in Semantic Analysis 2018-01-05T11:29:02,735 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:02,736 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:02,736 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_not_null_constraints : db=default tbl=src 2018-01-05T11:29:02,736 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_not_null_constraints : db=default tbl=src 2018-01-05T11:29:02,741 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,741 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:02,741 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:02,741 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=src 2018-01-05T11:29:02,741 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=src 2018-01-05T11:29:02,746 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,746 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:02,746 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:02,746 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=src 2018-01-05T11:29:02,746 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=src 2018-01-05T11:29:02,747 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,747 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:02,747 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:02,747 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_unique_constraints : db=default tbl=src 2018-01-05T11:29:02,747 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_unique_constraints : db=default tbl=src 2018-01-05T11:29:02,752 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,752 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:02,752 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:02,752 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=src 2018-01-05T11:29:02,752 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=src 2018-01-05T11:29:02,760 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,760 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:02,761 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:02,761 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_not_null_constraints : db=default tbl=src1 2018-01-05T11:29:02,761 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_not_null_constraints : db=default tbl=src1 2018-01-05T11:29:02,761 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,761 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:02,761 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:02,761 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=src1 2018-01-05T11:29:02,761 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=src1 2018-01-05T11:29:02,762 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,762 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:02,762 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:02,762 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=src1 2018-01-05T11:29:02,762 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=src1 2018-01-05T11:29:02,762 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,762 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:02,762 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:02,762 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_unique_constraints : db=default tbl=src1 2018-01-05T11:29:02,762 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_unique_constraints : db=default tbl=src1 2018-01-05T11:29:02,762 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,762 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:02,762 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:02,762 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=src1 2018-01-05T11:29:02,762 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=src1 2018-01-05T11:29:02,763 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:02,763 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,008 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:29:03,008 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,032 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.sql2rel: Plan after trimming unused fields HiveJoin(condition=[=($0, $2)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.src]], table:alias=[a]) HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.src1]], table:alias=[b]) 2018-01-05T11:29:03,032 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan before removing subquery: HiveProject(key=[$0], value=[$1], key1=[$5], value1=[$6]) HiveJoin(condition=[=($0, $5)], joinType=[inner], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.src]], table:alias=[a]) HiveTableScan(table=[[default.src1]], table:alias=[b]) 2018-01-05T11:29:03,033 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan just after removing subquery: HiveProject(key=[$0], value=[$1], key1=[$5], value1=[$6]) HiveJoin(condition=[=($0, $5)], joinType=[inner], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.src]], table:alias=[a]) HiveTableScan(table=[[default.src1]], table:alias=[b]) 2018-01-05T11:29:03,033 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan after decorrelation: HiveProject(key=[$0], value=[$1], key1=[$5], value1=[$6]) HiveJoin(condition=[=($0, $5)], joinType=[inner], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.src]], table:alias=[a]) HiveTableScan(table=[[default.src1]], table:alias=[b]) 2018-01-05T11:29:03,033 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,034 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,034 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,034 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,034 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,035 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,035 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,035 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,035 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,036 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,036 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,093 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,093 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,093 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,093 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,093 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,093 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,129 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.sql2rel: Plan after trimming unused fields HiveJoin(condition=[=($0, $2)], joinType=[inner], algorithm=[none], cost=[not available]) HiveFilter(condition=[IS NOT NULL($0)]) HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.src]], table:alias=[a]) HiveFilter(condition=[IS NOT NULL($0)]) HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.src1]], table:alias=[b]) 2018-01-05T11:29:03,129 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,129 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,129 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,129 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,133 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,133 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,133 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,133 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,134 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,134 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,135 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,135 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,135 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,135 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,136 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,136 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Original plan for PlanModifier HiveJoin(condition=[=($0, $2)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.src]], table:alias=[a]) HiveProject(key=[$0], value=[$1]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.src1]], table:alias=[b]) 2018-01-05T11:29:03,137 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after top-level introduceDerivedTable HiveProject(key=[$0], value=[$1], key0=[$2], value0=[$3]) HiveJoin(condition=[=($0, $2)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.src]], table:alias=[a]) HiveProject(key=[$0], value=[$1]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.src1]], table:alias=[b]) 2018-01-05T11:29:03,137 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after nested convertOpTree HiveProject(key=[$0], value=[$1], key0=[$2], value0=[$3]) HiveJoin(condition=[=($0, $2)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.src]], table:alias=[a]) HiveProject(key=[$0], value=[$1]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.src1]], table:alias=[b]) 2018-01-05T11:29:03,138 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after propagating order HiveProject(key=[$0], value=[$1], key0=[$2], value0=[$3]) HiveJoin(condition=[=($0, $2)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.src]], table:alias=[a]) HiveProject(key=[$0], value=[$1]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.src1]], table:alias=[b]) 2018-01-05T11:29:03,138 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after fixTopOBSchema HiveProject(key=[$0], value=[$1], key0=[$2], value0=[$3]) HiveJoin(condition=[=($0, $2)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.src]], table:alias=[a]) HiveProject(key=[$0], value=[$1]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.src1]], table:alias=[b]) 2018-01-05T11:29:03,138 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Final plan after modifier HiveProject(a.key=[$0], a.value=[$1], b.key=[$2], b.value=[$3]) HiveJoin(condition=[=($0, $2)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.src]], table:alias=[a]) HiveProject(key=[$0], value=[$1]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.src1]], table:alias=[b]) 2018-01-05T11:29:03,142 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:03,142 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:03,142 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:03,142 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,142 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src 2018-01-05T11:29:03,142 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src 2018-01-05T11:29:03,145 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:03,145 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,145 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:03,145 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:03,145 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:03,145 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,145 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src1 2018-01-05T11:29:03,145 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src1 2018-01-05T11:29:03,147 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:03,148 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,148 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:03,148 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:03,148 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:03,148 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Table Plan for a TS[0] 2018-01-05T11:29:03,150 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Filter Plan for $hdt$_0 row schema: a{(key,key: string)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:03,150 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB a{(key,key: string)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB a{(key,key: string)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:03,150 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (. (tok_table_or_col a) key) key) (tok_selexpr (. (tok_table_or_col a) value) value)) 2018-01-05T11:29:03,150 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = a{(key,key: string)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} starRr = null 2018-01-05T11:29:03,150 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(key,_col0: string)(value,_col1: string)} 2018-01-05T11:29:03,150 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:03,150 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block $hdt$_0 2018-01-05T11:29:03,150 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block $hdt$_0 2018-01-05T11:29:03,150 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Table Plan for b TS[3] 2018-01-05T11:29:03,150 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Filter Plan for $hdt$_1 row schema: b{(key,key: string)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:03,150 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB b{(key,key: string)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB b{(key,key: string)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:03,150 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (. (tok_table_or_col b) key) key) (tok_selexpr (. (tok_table_or_col b) value) value)) 2018-01-05T11:29:03,150 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = b{(key,key: string)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} starRr = null 2018-01-05T11:29:03,150 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(key,_col0: string)(value,_col1: string)} 2018-01-05T11:29:03,150 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:03,150 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block $hdt$_1 2018-01-05T11:29:03,150 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block $hdt$_1 2018-01-05T11:29:03,156 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB $hdt$_0{(key,_col0: string)(value,_col1: string)} $hdt$_1{(key,_col2: string)(value,_col3: string)} after GB $hdt$_0{(key,_col0: string)(value,_col1: string)} $hdt$_1{(key,_col2: string)(value,_col3: string)} 2018-01-05T11:29:03,156 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (. (tok_table_or_col $hdt$_0) key) a.key) (tok_selexpr (. (tok_table_or_col $hdt$_0) value) a.value) (tok_selexpr (. (tok_table_or_col $hdt$_1) key) b.key) (tok_selexpr (. (tok_table_or_col $hdt$_1) value) b.value)) 2018-01-05T11:29:03,156 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = $hdt$_0{(key,_col0: string)(value,_col1: string)} $hdt$_1{(key,_col2: string)(value,_col3: string)} starRr = null 2018-01-05T11:29:03,156 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(a.key,_col0: string)(a.value,_col1: string)(b.key,_col2: string)(b.value,_col3: string)} 2018-01-05T11:29:03,156 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:03,156 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-02_671_246690629183113225-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-02_671_246690629183113225-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-02_671_246690629183113225-1/-mr-10001 2018-01-05T11:29:03,157 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-02_671_246690629183113225-1/-mr-10003 2018-01-05T11:29:03,157 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-02_671_246690629183113225-1/-mr-10001 row schema: null{(a.key,_col0: string)(a.value,_col1: string)(b.key,_col2: string)(b.value,_col3: string)} 2018-01-05T11:29:03,157 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block null 2018-01-05T11:29:03,157 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:29:03,157 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: CBO Succeeded; optimized logical plan. 2018-01-05T11:29:03,157 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Before logical optimization TS[0]-FIL[1]-SEL[2]-RS[6]-JOIN[8]-SEL[9]-FS[10] TS[3]-FIL[4]-SEL[5]-RS[7]-JOIN[8] 2018-01-05T11:29:03,157 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,157 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,157 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,158 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] lineage.Generator: Time taken for lineage transform=1 2018-01-05T11:29:03,158 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,158 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,160 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.PartitionColumnsSeparator: Partition columns not separated for null, is not IN operator : 2018-01-05T11:29:03,160 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.PartitionColumnsSeparator: Partition columns not separated for null, is not IN operator : 2018-01-05T11:29:03,160 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,160 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,160 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SyntheticJoinPredicate: Synthetic predicate: 0 --> 1 2018-01-05T11:29:03,161 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SyntheticJoinPredicate: Synthetic predicate: 1 --> 0 2018-01-05T11:29:03,161 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,161 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,161 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FS(10) 2018-01-05T11:29:03,161 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(9) 2018-01-05T11:29:03,161 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for JOIN(8) 2018-01-05T11:29:03,161 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(6) 2018-01-05T11:29:03,161 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(11) 2018-01-05T11:29:03,167 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias $hdt$_0: (_col0) IN (RS[7]) 2018-01-05T11:29:03,167 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(2) 2018-01-05T11:29:03,167 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of SEL for alias $hdt$_0: (key) IN (RS[7]) 2018-01-05T11:29:03,168 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(1) 2018-01-05T11:29:03,168 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias a: key is not null 2018-01-05T11:29:03,168 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias a: (key) IN (RS[7]) 2018-01-05T11:29:03,168 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:29:03,168 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of TS for alias a: key is not null; (key) IN (RS[7]) 2018-01-05T11:29:03,168 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(7) 2018-01-05T11:29:03,168 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(12) 2018-01-05T11:29:03,168 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias $hdt$_1: (_col0) IN (RS[6]) 2018-01-05T11:29:03,168 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(5) 2018-01-05T11:29:03,168 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of SEL for alias $hdt$_1: (key) IN (RS[6]) 2018-01-05T11:29:03,168 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(4) 2018-01-05T11:29:03,168 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias b: key is not null 2018-01-05T11:29:03,168 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias b: (key) IN (RS[6]) 2018-01-05T11:29:03,168 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for TS(3) 2018-01-05T11:29:03,168 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of TS for alias b: key is not null; (key) IN (RS[6]) 2018-01-05T11:29:03,168 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SimplePredicatePushDown: After PPD: TS[0]-FIL[13]-SEL[2]-RS[6]-JOIN[8]-SEL[9]-FS[10] TS[3]-FIL[14]-SEL[5]-RS[7]-JOIN[8] 2018-01-05T11:29:03,168 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,169 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,169 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,169 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,169 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,169 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,171 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,171 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,171 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,171 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,172 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,172 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,172 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: JOIN 8 oldExprs: {0=[Column[KEY.reducesinkkey0], Column[VALUE._col0]], 1=[Column[KEY.reducesinkkey0], Column[VALUE._col0]]} 2018-01-05T11:29:03,172 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: JOIN 8 newExprs: {0=[Column[KEY.reducesinkkey0], Column[VALUE._col0]], 1=[Column[KEY.reducesinkkey0], Column[VALUE._col0]]} 2018-01-05T11:29:03,172 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 6 key:[Column[_col0]] 2018-01-05T11:29:03,172 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 6 oldColExprMap: {VALUE._col0=Column[_col1], KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:03,172 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 6 newColExprMap: {VALUE._col0=Column[_col1], KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:03,172 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 7 key:[Column[_col0]] 2018-01-05T11:29:03,172 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 7 oldColExprMap: {VALUE._col0=Column[_col1], KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:03,172 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 7 newColExprMap: {VALUE._col0=Column[_col1], KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:03,172 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,172 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,172 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,172 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,172 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,172 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,172 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,172 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,172 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,172 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,172 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,172 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,172 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,172 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.IdentityProjectRemover: Identity project remover optimization removed : SEL[9] 2018-01-05T11:29:03,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,183 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,183 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,183 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,183 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: After logical optimization TS[0]-FIL[13]-SEL[2]-RS[6]-JOIN[8]-FS[10] TS[3]-FIL[14]-SEL[5]-RS[7]-JOIN[8] 2018-01-05T11:29:03,183 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,184 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Parent: TS[0] 2018-01-05T11:29:03,184 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Filter: (key is not null and (key) IN (RS[7])) (type: boolean) 2018-01-05T11:29:03,184 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: TableScan: TS[0] 2018-01-05T11:29:03,186 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Column key is not a partition column 2018-01-05T11:29:03,186 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Initiate semijoin reduction for key ((key is not null and (key) IN (RS[7])) 2018-01-05T11:29:03,187 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:03,191 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:03,191 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:03,192 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:03,192 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:03,192 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:03,192 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: DynamicSemiJoinPushdown: Saving RS to TS mapping: RS[19]: TS[0] 2018-01-05T11:29:03,200 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Parent: TS[3] 2018-01-05T11:29:03,200 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Filter: (key is not null and (key) IN (RS[6])) (type: boolean) 2018-01-05T11:29:03,200 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: TableScan: TS[3] 2018-01-05T11:29:03,200 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Column key is not a partition column 2018-01-05T11:29:03,200 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Initiate semijoin reduction for key ((key is not null and (key) IN (RS[6])) 2018-01-05T11:29:03,200 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:03,200 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:03,200 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:03,200 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:03,200 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:03,200 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:03,200 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: DynamicSemiJoinPushdown: Saving RS to TS mapping: RS[24]: TS[3] 2018-01-05T11:29:03,201 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,201 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,201 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,201 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=src 2018-01-05T11:29:03,201 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=src 2018-01-05T11:29:03,208 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 2.760536ms + 0.009461ms, the query is [select "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "TAB_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (...)] 2018-01-05T11:29:03,209 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:03,220 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:03,221 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,221 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting stats (Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE) on TS[0] 2018-01-05T11:29:03,221 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (src): numRows: 500 dataSize: 89000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {value= colName: value colType: string countDistincts: 309 numNulls: 0 avgColLen: 6.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, key= colName: key colType: string countDistincts: 309 numNulls: 0 avgColLen: 2.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:03,221 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,221 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=src1 2018-01-05T11:29:03,221 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=src1 2018-01-05T11:29:03,224 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.161819ms + 0.006176ms, the query is [select "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "TAB_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (...)] 2018-01-05T11:29:03,226 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:03,231 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:03,232 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,232 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting stats (Num rows: 25 Data size: 4375 Basic stats: COMPLETE Column stats: COMPLETE) on TS[3] 2018-01-05T11:29:03,232 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-TS[3] (src1): numRows: 25 dataSize: 4375 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {value= colName: value colType: string countDistincts: 19 numNulls: 0 avgColLen: 4.92 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, key= colName: key colType: string countDistincts: 16 numNulls: 0 avgColLen: 1.72 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:03,232 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_7_b_key_min), DynamicValue(RS_7_b_key_max)) Original num rows: 500 New num rows: 500 2018-01-05T11:29:03,232 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_7_b_key_min), DynamicValue(RS_7_b_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_7_b_key_bloom_filter))) Original num rows: 500 New num rows: 500 2018-01-05T11:29:03,232 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_7_b_key_min), DynamicValue(RS_7_b_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_7_b_key_bloom_filter)))) Original num rows: 500 New num rows: 500 2018-01-05T11:29:03,232 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-FIL[13]: numRows: 500 dataSize: 89000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {value= colName: value colType: string countDistincts: 309 numNulls: 0 avgColLen: 6.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, key= colName: key colType: string countDistincts: 309 numNulls: 0 avgColLen: 2.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:03,232 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting stats (Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE) on FIL[13] 2018-01-05T11:29:03,232 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_6_a_key_min), DynamicValue(RS_6_a_key_max)) Original num rows: 25 New num rows: 25 2018-01-05T11:29:03,232 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_6_a_key_min), DynamicValue(RS_6_a_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_6_a_key_bloom_filter))) Original num rows: 25 New num rows: 25 2018-01-05T11:29:03,232 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_6_a_key_min), DynamicValue(RS_6_a_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_6_a_key_bloom_filter)))) Original num rows: 25 New num rows: 25 2018-01-05T11:29:03,232 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-FIL[14]: numRows: 25 dataSize: 4375 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {value= colName: value colType: string countDistincts: 19 numNulls: 0 avgColLen: 4.92 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, key= colName: key colType: string countDistincts: 16 numNulls: 0 avgColLen: 1.72 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:03,233 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting stats (Num rows: 25 Data size: 4375 Basic stats: COMPLETE Column stats: COMPLETE) on FIL[14] 2018-01-05T11:29:03,233 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[2] 2018-01-05T11:29:03,233 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[2]: numRows: 500 dataSize: 89000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: string countDistincts: 309 numNulls: 0 avgColLen: 2.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: string countDistincts: 309 numNulls: 0 avgColLen: 6.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:03,233 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 25 Data size: 4375 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[5] 2018-01-05T11:29:03,233 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[5]: numRows: 25 dataSize: 4375 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: string countDistincts: 16 numNulls: 0 avgColLen: 1.72 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: string countDistincts: 19 numNulls: 0 avgColLen: 4.92 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:03,233 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE) on RS[6] 2018-01-05T11:29:03,233 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[6]: numRows: 500 dataSize: 89000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {VALUE._col0= colName: VALUE._col0 colType: string countDistincts: 309 numNulls: 0 avgColLen: 6.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, KEY.reducesinkkey0= colName: KEY.reducesinkkey0 colType: string countDistincts: 309 numNulls: 0 avgColLen: 2.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:03,233 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[20] 2018-01-05T11:29:03,233 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[20]: numRows: 500 dataSize: 43500 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: string countDistincts: 309 numNulls: 0 avgColLen: 2.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:03,233 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 25 Data size: 4375 Basic stats: COMPLETE Column stats: COMPLETE) on RS[7] 2018-01-05T11:29:03,233 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[7]: numRows: 25 dataSize: 4375 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {VALUE._col0= colName: VALUE._col0 colType: string countDistincts: 19 numNulls: 0 avgColLen: 4.92 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, KEY.reducesinkkey0= colName: KEY.reducesinkkey0 colType: string countDistincts: 16 numNulls: 0 avgColLen: 1.72 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:03,233 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[15] 2018-01-05T11:29:03,233 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[15]: numRows: 25 dataSize: 2150 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: string countDistincts: 16 numNulls: 0 avgColLen: 1.72 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:03,233 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-JOIN[8]: detects none/multiple PK parents. 2018-01-05T11:29:03,233 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.JoinOperator: Setting stats (Num rows: 40 Data size: 14120 Basic stats: COMPLETE Column stats: COMPLETE) on JOIN[8] 2018-01-05T11:29:03,233 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-JOIN[8]: numRows: 40 dataSize: 14120 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: string countDistincts: 2 numNulls: 0 avgColLen: 2.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: string countDistincts: 16 numNulls: 0 avgColLen: 1.72 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: string countDistincts: 25 numNulls: 0 avgColLen: 6.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: string countDistincts: 19 numNulls: 0 avgColLen: 4.92 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:03,234 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[21]: inputSize: 89000 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:03,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[21] hashAgg: true 2018-01-05T11:29:03,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 3] STATS-GBY[21]: cardinality: 1 2018-01-05T11:29:03,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 552 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[21] 2018-01-05T11:29:03,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[21]: numRows: 1 dataSize: 552 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:03,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[16]: inputSize: 4375 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:03,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[16] hashAgg: true 2018-01-05T11:29:03,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 3] STATS-GBY[16]: cardinality: 1 2018-01-05T11:29:03,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 552 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[16] 2018-01-05T11:29:03,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[16]: numRows: 1 dataSize: 552 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:03,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-FS[10]: numRows: 40 dataSize: 14120 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: string countDistincts: 2 numNulls: 0 avgColLen: 2.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: string countDistincts: 16 numNulls: 0 avgColLen: 1.72 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: string countDistincts: 25 numNulls: 0 avgColLen: 6.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: string countDistincts: 19 numNulls: 0 avgColLen: 4.92 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:03,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 552 Basic stats: COMPLETE Column stats: COMPLETE) on RS[22] 2018-01-05T11:29:03,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[22]: numRows: 1 dataSize: 552 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:03,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 552 Basic stats: COMPLETE Column stats: COMPLETE) on RS[17] 2018-01-05T11:29:03,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[17]: numRows: 1 dataSize: 552 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:03,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[23]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:03,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[23] hashAgg: false 2018-01-05T11:29:03,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 9] STATS-GBY[23]: cardinality: 1 2018-01-05T11:29:03,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 552 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[23] 2018-01-05T11:29:03,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[23]: numRows: 1 dataSize: 552 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:03,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[18]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:03,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[18] hashAgg: false 2018-01-05T11:29:03,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 9] STATS-GBY[18]: cardinality: 1 2018-01-05T11:29:03,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 552 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[18] 2018-01-05T11:29:03,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[18]: numRows: 1 dataSize: 552 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:03,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 552 Basic stats: COMPLETE Column stats: COMPLETE) on RS[24] 2018-01-05T11:29:03,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[24]: numRows: 1 dataSize: 552 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:03,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 552 Basic stats: COMPLETE Column stats: COMPLETE) on RS[19] 2018-01-05T11:29:03,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[19]: numRows: 1 dataSize: 552 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:03,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on TS[0] 2018-01-05T11:29:03,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on TS[3] 2018-01-05T11:29:03,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on FIL[13] 2018-01-05T11:29:03,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on FIL[14] 2018-01-05T11:29:03,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on SEL[2] 2018-01-05T11:29:03,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on SEL[5] 2018-01-05T11:29:03,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[KEY.reducesinkkey0]]; sort column names: [[KEY.reducesinkkey0]]; bucket count: -1 }) on RS[6] 2018-01-05T11:29:03,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on SEL[20] 2018-01-05T11:29:03,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[KEY.reducesinkkey0]]; sort column names: [[KEY.reducesinkkey0]]; bucket count: -1 }) on RS[7] 2018-01-05T11:29:03,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on SEL[15] 2018-01-05T11:29:03,246 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.JoinOperator: Setting traits ({ bucket column names: [[_col0], [_col2]]; sort column names: [[_col0], [_col2]]; bucket count: -1 }) on JOIN[8] 2018-01-05T11:29:03,246 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[21] 2018-01-05T11:29:03,246 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[16] 2018-01-05T11:29:03,246 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FileSinkOperator: Setting traits ({ bucket column names: [[_col0], [_col2]]; sort column names: [[_col0], [_col2]]; bucket count: -1 }) on FS[10] 2018-01-05T11:29:03,246 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[22] 2018-01-05T11:29:03,246 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[17] 2018-01-05T11:29:03,246 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[23] 2018-01-05T11:29:03,246 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[18] 2018-01-05T11:29:03,246 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[24] 2018-01-05T11:29:03,246 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[19] 2018-01-05T11:29:03,246 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,246 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,246 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Set parallelism for reduce sink RS[6] to: 1 2018-01-05T11:29:03,246 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Set parallelism for reduce sink RS[7] to: 1 2018-01-05T11:29:03,246 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConvertJoinMapJoin: Memory monitor info set to : { isLlap: true executorsPerNode: 4 maxExecutorsOverSubscribeMemory: 3 memoryOverSubscriptionFactor: 0.20000000298023224 memoryCheckInterval: 100000 noConditionalTaskSize: 10000000 adjustedNoConditionalTaskSize: 16000000 hashTableInflationFactor: 2.0 threshold: 32000000 } 2018-01-05T11:29:03,249 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConvertJoinMapJoin: Fallback to common merge join operator 2018-01-05T11:29:03,250 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.CommonMergeJoinOperator: Setting traits ({ bucket column names: [[_col0], [_col2]]; sort column names: [[_col0], [_col2]]; bucket count: 0 }) on MERGEJOIN[25] 2018-01-05T11:29:03,250 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.CommonMergeJoinOperator: Setting stats (Num rows: 40 Data size: 14120 Basic stats: COMPLETE Column stats: COMPLETE) on MERGEJOIN[25] 2018-01-05T11:29:03,250 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:03,250 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:03,250 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:03,250 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:03,250 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,250 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,250 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,250 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,250 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,250 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,250 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,250 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,250 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing BloomFilter cost/benefit for RS[19] - TS[0] (a) (Column[key]) 2018-01-05T11:29:03,251 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing key domain cardinality, keyDomainCardinality=325, semiJoinKeyIsPK=false, selColStat= colName: _col0 colType: string countDistincts: 16 numNulls: 0 avgColLen: 1.72 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, selColSourceStat= colName: key colType: string countDistincts: 16 numNulls: 0 avgColLen: 1.72 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, tsColStat= colName: key colType: string countDistincts: 309 numNulls: 0 avgColLen: 2.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false 2018-01-05T11:29:03,251 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit for Column[_col0] to Column[key], selKeyCardinality=16, tsKeyCardinality=309, tsRows=500, keyDomainCardinality=325 2018-01-05T11:29:03,251 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: SemiJoin key selectivity=0.04923076923076923, benefit=475.3846153846154 2018-01-05T11:29:03,251 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit=475.3846153846154, cost=25.0, tsDataSize=500, netBenefit=450.3846153846154 2018-01-05T11:29:03,251 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: netBenefit=0.9007692307692309 2018-01-05T11:29:03,251 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing BloomFilter cost/benefit for RS[24] - TS[3] (b) (Column[key]) 2018-01-05T11:29:03,251 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing key domain cardinality, keyDomainCardinality=325, semiJoinKeyIsPK=false, selColStat= colName: _col0 colType: string countDistincts: 309 numNulls: 0 avgColLen: 2.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, selColSourceStat= colName: key colType: string countDistincts: 309 numNulls: 0 avgColLen: 2.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, tsColStat= colName: key colType: string countDistincts: 16 numNulls: 0 avgColLen: 1.72 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false 2018-01-05T11:29:03,251 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit for Column[_col0] to Column[key], selKeyCardinality=309, tsKeyCardinality=16, tsRows=25, keyDomainCardinality=325 2018-01-05T11:29:03,251 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: SemiJoin key selectivity=0.9507692307692308, benefit=1.2307692307692297 2018-01-05T11:29:03,251 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit=1.2307692307692297, cost=500.0, tsDataSize=25, netBenefit=-498.7692307692308 2018-01-05T11:29:03,251 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: netBenefit=-19.950769230769232 2018-01-05T11:29:03,251 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Reduction factor not satisfied for RS[24]-TS[3] (b). Removing semijoin optimization. 2018-01-05T11:29:03,251 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Removing ReduceSink RS[24] and TableScan TS[3] 2018-01-05T11:29:03,252 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,252 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,252 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,252 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,253 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,253 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,257 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,257 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,257 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,257 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Adding special edge: RS --> TS[0] 2018-01-05T11:29:03,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:03,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: FS, 10 2018-01-05T11:29:03,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:03,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: MERGEJOIN, 25 2018-01-05T11:29:03,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:03,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 6 2018-01-05T11:29:03,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:03,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 2 2018-01-05T11:29:03,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:03,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: FIL, 13 2018-01-05T11:29:03,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:03,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:29:03,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:03,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 7 2018-01-05T11:29:03,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:03,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 19 2018-01-05T11:29:03,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:03,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: GBY, 18 2018-01-05T11:29:03,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:03,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 17 2018-01-05T11:29:03,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:03,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: GBY, 16 2018-01-05T11:29:03,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:03,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 15 2018-01-05T11:29:03,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:03,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 5 2018-01-05T11:29:03,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:03,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: FIL, 14 2018-01-05T11:29:03,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:03,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: TS, 3 2018-01-05T11:29:03,258 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Cycle free: true 2018-01-05T11:29:03,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: Before SharedWorkOptimizer: TS[0]-FIL[13]-SEL[2]-RS[6]-MERGEJOIN[25]-FS[10] TS[3]-FIL[14]-SEL[5]-RS[7]-MERGEJOIN[25] -SEL[15]-GBY[16]-RS[17]-GBY[18]-RS[19] 2018-01-05T11:29:03,259 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: DPP information stored in the cache: {TS[0]=[RS[19]]} 2018-01-05T11:29:03,262 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: Sorted tables by size: [default.src=89000, default.src1=4375] 2018-01-05T11:29:03,262 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: After SharedWorkOptimizer: TS[0]-FIL[13]-SEL[2]-RS[6]-MERGEJOIN[25]-FS[10] TS[3]-FIL[14]-SEL[5]-RS[7]-MERGEJOIN[25] -SEL[15]-GBY[16]-RS[17]-GBY[18]-RS[19] 2018-01-05T11:29:03,262 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,262 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{a}key,value: string|{a}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{a}block__offset__inside__file,INPUT__FILE__NAME: string|{a}input__file__name,ROW__ID: struct|{a}row__id) 2018-01-05T11:29:03,263 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[3] with rs:(key: string|{b}key,value: string|{b}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{b}block__offset__inside__file,INPUT__FILE__NAME: string|{b}input__file__name,ROW__ID: struct|{b}row__id) 2018-01-05T11:29:03,263 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(Const boolean true, Const boolean true) -> Const boolean true 2018-01-05T11:29:03,263 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPAnd(Const boolean true, Const boolean true)) -> GenericUDFOPNotNull(Column[key]) 2018-01-05T11:29:03,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FIL[13] with rs:(key: string|{a}key,value: string|{a}value) 2018-01-05T11:29:03,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:29:03,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FIL[13] 2018-01-05T11:29:03,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Old filter FIL[13] conditions:(key is not null and (key BETWEEN DynamicValue(RS_7_b_key_min) AND DynamicValue(RS_7_b_key_max) and in_bloom_filter(key, DynamicValue(RS_7_b_key_bloom_filter)))) 2018-01-05T11:29:03,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New filter FIL[13] conditions:(key is not null and (key BETWEEN DynamicValue(RS_7_b_key_min) AND DynamicValue(RS_7_b_key_max) and in_bloom_filter(key, DynamicValue(RS_7_b_key_bloom_filter)))) 2018-01-05T11:29:03,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FIL[14] with rs:(key: string|{b}key,value: string|{b}value) 2018-01-05T11:29:03,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_3 {} 2018-01-05T11:29:03,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FIL[14] 2018-01-05T11:29:03,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Old filter FIL[14] conditions:(key is not null and (true and true)) 2018-01-05T11:29:03,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(Const boolean true, Const boolean true) -> Const boolean true 2018-01-05T11:29:03,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPAnd(Const boolean true, Const boolean true)) -> GenericUDFOPNotNull(Column[key]) 2018-01-05T11:29:03,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New filter FIL[14] conditions:key is not null 2018-01-05T11:29:03,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[2] with rs:(_col0: string|{$hdt$_0}key,_col1: string|{$hdt$_0}value) 2018-01-05T11:29:03,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op FIL_13 {} 2018-01-05T11:29:03,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[2] 2018-01-05T11:29:03,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value]) 2018-01-05T11:29:03,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: string|{$hdt$_1}key,_col1: string|{$hdt$_1}value) 2018-01-05T11:29:03,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op FIL_14 {} 2018-01-05T11:29:03,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:29:03,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value]) 2018-01-05T11:29:03,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[6] with rs:(KEY.reducesinkkey0: string|{$hdt$_0}key,VALUE._col0: string|{$hdt$_0}value) 2018-01-05T11:29:03,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_2 {} 2018-01-05T11:29:03,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[6] 2018-01-05T11:29:03,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[7] with rs:(KEY.reducesinkkey0: string|{$hdt$_1}key,VALUE._col0: string|{$hdt$_1}value) 2018-01-05T11:29:03,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:29:03,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[7] 2018-01-05T11:29:03,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[15] with rs:(_col0: string|{$hdt$_1}key) 2018-01-05T11:29:03,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:29:03,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[15] 2018-01-05T11:29:03,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0]) 2018-01-05T11:29:03,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:MERGEJOIN[25] with rs:(_col0: string|{$hdt$_0}key,_col1: string|{$hdt$_0}value,_col2: string|{$hdt$_1}key,_col3: string|{$hdt$_1}value) 2018-01-05T11:29:03,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_6 {} 2018-01-05T11:29:03,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_7 {} 2018-01-05T11:29:03,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator MERGEJOIN[25] 2018-01-05T11:29:03,265 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[16] with rs:(_col0: string|{},_col1: string|{},_col2: string|{}) 2018-01-05T11:29:03,265 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_15 {} 2018-01-05T11:29:03,265 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[16] 2018-01-05T11:29:03,265 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[10] with rs:(_col0: string|{},_col1: string|{},_col2: string|{},_col3: string|{}) 2018-01-05T11:29:03,265 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op MERGEJOIN_25 {} 2018-01-05T11:29:03,265 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[10] 2018-01-05T11:29:03,265 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[17] with rs:(_col0: string|{},_col1: string|{},_col2: string|{}) 2018-01-05T11:29:03,265 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_16 {} 2018-01-05T11:29:03,265 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[17] 2018-01-05T11:29:03,265 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[18] with rs:(_col0: string|{},_col1: string|{},_col2: string|{}) 2018-01-05T11:29:03,265 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_17 {} 2018-01-05T11:29:03,265 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[18] 2018-01-05T11:29:03,265 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[19] with rs:(_col0: string|{},_col1: string|{},_col2: string|{}) 2018-01-05T11:29:03,265 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_18 {} 2018-01-05T11:29:03,265 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[19] 2018-01-05T11:29:03,267 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,267 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: TS[0] 2018-01-05T11:29:03,267 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[6] 2018-01-05T11:29:03,267 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:29:03,268 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src of table a 2018-01-05T11:29:03,268 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src 2018-01-05T11:29:03,268 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[6] 2018-01-05T11:29:03,268 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: MERGEJOIN[25] 2018-01-05T11:29:03,268 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: FS[10] 2018-01-05T11:29:03,268 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding reduce work (Reducer 2) for MERGEJOIN[25] 2018-01-05T11:29:03,268 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Setting up reduce sink: RS[6] with following reduce work: Reducer 2 2018-01-05T11:29:03,269 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[6] as parent from MERGEJOIN[25] 2018-01-05T11:29:03,269 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[7] as parent from MERGEJOIN[25] 2018-01-05T11:29:03,269 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: FS[10] 2018-01-05T11:29:03,269 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: TS[3] 2018-01-05T11:29:03,269 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[7] 2018-01-05T11:29:03,269 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding map work (Map 3) for TS[3] 2018-01-05T11:29:03,269 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1 of table b 2018-01-05T11:29:03,269 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1 2018-01-05T11:29:03,269 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Second pass. Leaf operator: RS[7] has common downstream work: org.apache.hadoop.hive.ql.plan.MergeJoinWork@6e4ccc5f 2018-01-05T11:29:03,269 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: TS[3] 2018-01-05T11:29:03,269 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[17] 2018-01-05T11:29:03,269 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[17] 2018-01-05T11:29:03,269 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: GBY[18] 2018-01-05T11:29:03,269 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[19] 2018-01-05T11:29:03,269 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding reduce work (Reducer 4) for GBY[18] 2018-01-05T11:29:03,269 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Setting up reduce sink: RS[17] with following reduce work: Reducer 4 2018-01-05T11:29:03,269 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[17] as parent from GBY[18] 2018-01-05T11:29:03,269 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[19] 2018-01-05T11:29:03,269 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: ResduceSink RS[19] to TableScan TS[0] 2018-01-05T11:29:03,269 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Connecting Baswork - Reducer 4 to Map 1 2018-01-05T11:29:03,269 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:29:03,269 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,269 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,269 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 3 2018-01-05T11:29:03,269 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:03,270 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:03,270 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 3 2018-01-05T11:29:03,270 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:03,270 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:03,270 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:29:03,294 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Examining input format to see if vectorization is enabled. 2018-01-05T11:29:03,299 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorization is enabled for input format(s) [org.apache.hadoop.mapred.TextInputFormat] 2018-01-05T11:29:03,299 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing MapWork... 2018-01-05T11:29:03,322 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFOPNotNull(Column[key]), Vectorized Expression = SelectColumnIsNotNull(col 0:string) 2018-01-05T11:29:03,324 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator 2018-01-05T11:29:03,324 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.FilterDesc 2018-01-05T11:29:03,324 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[key], Vectorized Expression = col 0:string 2018-01-05T11:29:03,324 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[value], Vectorized Expression = col 1:string 2018-01-05T11:29:03,326 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator 2018-01-05T11:29:03,326 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.SelectDesc 2018-01-05T11:29:03,327 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:string 2018-01-05T11:29:03,327 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col1], Vectorized Expression = col 1:string 2018-01-05T11:29:03,327 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator groupby typeName string 2018-01-05T11:29:03,329 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkStringOperator 2018-01-05T11:29:03,333 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:03,334 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkStringOperator 2018-01-05T11:29:03,334 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:03,334 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:string 2018-01-05T11:29:03,334 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator 2018-01-05T11:29:03,334 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.SelectDesc 2018-01-05T11:29:03,335 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vector GROUP BY operator will use processing mode HASH 2018-01-05T11:29:03,341 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:string 2018-01-05T11:29:03,346 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:string 2018-01-05T11:29:03,349 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:string 2018-01-05T11:29:03,353 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator 2018-01-05T11:29:03,353 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.GroupByDesc 2018-01-05T11:29:03,353 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:string 2018-01-05T11:29:03,353 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col1], Vectorized Expression = col 1:string 2018-01-05T11:29:03,353 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col2], Vectorized Expression = col 2:binary 2018-01-05T11:29:03,354 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkEmptyKeyOperator 2018-01-05T11:29:03,355 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:03,355 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkEmptyKeyOperator: VectorReduceSinkEmptyKeyOperator constructor vectorReduceSinkInfo org.apache.hadoop.hive.ql.plan.VectorReduceSinkInfo@511aa137 2018-01-05T11:29:03,355 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkEmptyKeyOperator 2018-01-05T11:29:03,355 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:03,355 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [key, value, ROW__ID] 2018-01-05T11:29:03,355 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [string, string, struct] 2018-01-05T11:29:03,355 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE, NONE, NONE] 2018-01-05T11:29:03,355 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 0 2018-01-05T11:29:03,355 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 1 2018-01-05T11:29:03,356 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:03,356 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:03,356 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Using reduce tag 0 2018-01-05T11:29:03,356 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1, _col2] columnTypes=[string, string, binary] 2018-01-05T11:29:03,356 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing ReduceWork... 2018-01-05T11:29:03,357 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vector GROUP BY operator will use processing mode STREAMING 2018-01-05T11:29:03,357 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[VALUE._col0], Vectorized Expression = col 0:string 2018-01-05T11:29:03,357 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[VALUE._col1], Vectorized Expression = col 1:string 2018-01-05T11:29:03,357 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[VALUE._col2], Vectorized Expression = col 2:binary 2018-01-05T11:29:03,357 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator 2018-01-05T11:29:03,357 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.GroupByDesc 2018-01-05T11:29:03,357 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:string 2018-01-05T11:29:03,357 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col1], Vectorized Expression = col 1:string 2018-01-05T11:29:03,357 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col2], Vectorized Expression = col 2:binary 2018-01-05T11:29:03,357 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkEmptyKeyOperator 2018-01-05T11:29:03,357 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:03,357 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkEmptyKeyOperator: VectorReduceSinkEmptyKeyOperator constructor vectorReduceSinkInfo org.apache.hadoop.hive.ql.plan.VectorReduceSinkInfo@6a0ab4cc 2018-01-05T11:29:03,357 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkEmptyKeyOperator 2018-01-05T11:29:03,357 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:03,357 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [VALUE._col0, VALUE._col1, VALUE._col2] 2018-01-05T11:29:03,357 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [string, string, binary] 2018-01-05T11:29:03,357 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE, NONE, NONE] 2018-01-05T11:29:03,357 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 0 2018-01-05T11:29:03,357 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 0 2018-01-05T11:29:03,357 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:03,357 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:03,357 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Examining input format to see if vectorization is enabled. 2018-01-05T11:29:03,358 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorization is enabled for input format(s) [org.apache.hadoop.mapred.TextInputFormat] 2018-01-05T11:29:03,358 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing MapWork... 2018-01-05T11:29:03,358 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFOPNotNull(Column[key]), Vectorized Expression = SelectColumnIsNotNull(col 0:string) 2018-01-05T11:29:03,360 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_7_b_key_min), DynamicValue(RS_7_b_key_max)), Vectorized Expression = FilterStringColumnBetweenDynamicValue(col 0:string, left NULL, right NULL) 2018-01-05T11:29:03,362 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFInBloomFilter(Column[key], DynamicValue(RS_7_b_key_bloom_filter)), Vectorized Expression = VectorInBloomFilterColDynamicValue 2018-01-05T11:29:03,362 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_7_b_key_min), DynamicValue(RS_7_b_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_7_b_key_bloom_filter))), Vectorized Expression = FilterExprAndExpr(children: FilterStringColumnBetweenDynamicValue(col 0:string, left NULL, right NULL), VectorInBloomFilterColDynamicValue) 2018-01-05T11:29:03,362 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_7_b_key_min), DynamicValue(RS_7_b_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_7_b_key_bloom_filter)))), Vectorized Expression = FilterExprAndExpr(children: SelectColumnIsNotNull(col 0:string), FilterExprAndExpr(children: FilterStringColumnBetweenDynamicValue(col 0:string, left NULL, right NULL), VectorInBloomFilterColDynamicValue)) 2018-01-05T11:29:03,362 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator 2018-01-05T11:29:03,362 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.FilterDesc 2018-01-05T11:29:03,362 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[key], Vectorized Expression = col 0:string 2018-01-05T11:29:03,362 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[value], Vectorized Expression = col 1:string 2018-01-05T11:29:03,362 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator 2018-01-05T11:29:03,362 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.SelectDesc 2018-01-05T11:29:03,362 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:string 2018-01-05T11:29:03,362 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col1], Vectorized Expression = col 1:string 2018-01-05T11:29:03,362 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator groupby typeName string 2018-01-05T11:29:03,363 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkStringOperator 2018-01-05T11:29:03,363 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkStringOperator 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [key, value, ROW__ID] 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [string, string, struct] 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE, NONE, NONE] 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 0 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 1 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:29:03,363 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: llap mode: only 2018-01-05T11:29:03,363 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Map 3 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'min(_col0)' 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '_col0' 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'max(_col0)' 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '_col0' 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'bloom_filter(_col0, expectedEntries=16)' 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '_col0' 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '_col0' 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key' 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'value' 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key is not null' 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key is not null' 2018-01-05T11:29:03,363 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@74627ef6 can be done in LLAP 2018-01-05T11:29:03,363 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Reducer 4 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'min(VALUE._col0)' 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'max(VALUE._col1)' 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'VALUE._col1' 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'bloom_filter(VALUE._col2, expectedEntries=16)' 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'VALUE._col2' 2018-01-05T11:29:03,363 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@28ebaa4b can be done in LLAP 2018-01-05T11:29:03,363 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key' 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'value' 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_7_b_key_min) AND DynamicValue(RS_7_b_key_max) and in_bloom_filter(key, DynamicValue(RS_7_b_key_bloom_filter))))' 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_7_b_key_min) AND DynamicValue(RS_7_b_key_max) and in_bloom_filter(key, DynamicValue(RS_7_b_key_bloom_filter))))' 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_7_b_key_min) AND DynamicValue(RS_7_b_key_max) and in_bloom_filter(key, DynamicValue(RS_7_b_key_bloom_filter))))' 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_7_b_key_min) AND DynamicValue(RS_7_b_key_max) and in_bloom_filter(key, DynamicValue(RS_7_b_key_bloom_filter))))' 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_7_b_key_min) AND DynamicValue(RS_7_b_key_max) and in_bloom_filter(key, DynamicValue(RS_7_b_key_bloom_filter))))' 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_7_b_key_min) AND DynamicValue(RS_7_b_key_max) and in_bloom_filter(key, DynamicValue(RS_7_b_key_bloom_filter))))' 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_7_b_key_min) AND DynamicValue(RS_7_b_key_max) and in_bloom_filter(key, DynamicValue(RS_7_b_key_bloom_filter))))' 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_7_b_key_min) AND DynamicValue(RS_7_b_key_max) and in_bloom_filter(key, DynamicValue(RS_7_b_key_bloom_filter))))' 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_7_b_key_min) AND DynamicValue(RS_7_b_key_max) and in_bloom_filter(key, DynamicValue(RS_7_b_key_bloom_filter))))' 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_7_b_key_min) AND DynamicValue(RS_7_b_key_max) and in_bloom_filter(key, DynamicValue(RS_7_b_key_bloom_filter))))' 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_7_b_key_min) AND DynamicValue(RS_7_b_key_max) and in_bloom_filter(key, DynamicValue(RS_7_b_key_bloom_filter))))' 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_7_b_key_min) AND DynamicValue(RS_7_b_key_max) and in_bloom_filter(key, DynamicValue(RS_7_b_key_bloom_filter))))' 2018-01-05T11:29:03,363 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@4132bec9 can be done in LLAP 2018-01-05T11:29:03,363 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Reducer 2 2018-01-05T11:29:03,363 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:03,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MergeJoinWork@6e4ccc5f can be done in LLAP 2018-01-05T11:29:03,364 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.SerializeFilter: Serializing: key is not null 2018-01-05T11:29:03,364 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.SerializeFilter: Serializing: (key is not null and (key BETWEEN DynamicValue(RS_7_b_key_min) AND DynamicValue(RS_7_b_key_max) and in_bloom_filter(key, DynamicValue(RS_7_b_key_bloom_filter)))) 2018-01-05T11:29:03,365 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,365 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed plan generation 2018-01-05T11:29:03,365 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: validation start 2018-01-05T11:29:03,365 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:29:03,368 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:03,368 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,368 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:Explain, type:string, comment:null)], properties:null) 2018-01-05T11:29:03,368 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,368 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:03,368 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {getNotNullConstraints_(NotNullConstraintsRequest, )=5, getUniqueConstraints_(UniqueConstraintsRequest, )=5, getPrimaryKeys_(PrimaryKeysRequest, )=7, getTableColumnStatistics_(String, String, List, )=31, getForeignKeys_(ForeignKeysRequest, )=9} 2018-01-05T11:29:03,368 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112902_f73e508d-7879-43ec-a4dc-1ca6a9418703); Time taken: 0.697 seconds 2018-01-05T11:29:03,368 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:03,368 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,368 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112902_f73e508d-7879-43ec-a4dc-1ca6a9418703): explain vectorization detail select * from src a join src1 b on a.key = b.key 2018-01-05T11:29:03,370 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,370 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: explain vectorization detail select * from src a join src1 b on a.key = b.key 2018-01-05T11:29:03,370 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:29:03,370 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,370 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,370 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,370 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,370 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-3:EXPLAIN] in serial mode 2018-01-05T11:29:03,449 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,449 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,449 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: explain vectorization detail select * from src a join src1 b on a.key = b.key 2018-01-05T11:29:03,449 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:29:03,449 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,449 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,449 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:03,449 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {} 2018-01-05T11:29:03,449 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112902_f73e508d-7879-43ec-a4dc-1ca6a9418703); Time taken: 0.081 seconds 2018-01-05T11:29:03,449 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:03,449 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,449 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:03,449 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,449 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query explain vectorization detail select * from src a join src1 b on a.key = b.key 2018-01-05T11:29:03,453 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting result file: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-02_671_246690629183113225-1/-local-10004 2018-01-05T11:29:03,453 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-02_671_246690629183113225-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-02_671_246690629183113225-1 2018-01-05T11:29:03,453 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-02_671_246690629183113225-1 2018-01-05T11:29:03,453 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.778 seconds, Fetched: 219 row(s) 2018-01-05T11:29:03,453 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:03,453 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:03,453 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:03,453 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:03,453 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,453 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:03,453 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,453 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,453 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,453 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: select * from src a join src1 b on a.key = b.key 2018-01-05T11:29:03,453 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d): select * from src a join src1 b on a.key = b.key 2018-01-05T11:29:03,454 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,454 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: select * from src a join src1 b on a.key = b.key 2018-01-05T11:29:03,454 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:03,454 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,454 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,454 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,454 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,454 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:03,454 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:29:03,454 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed phase 1 of Semantic Analysis 2018-01-05T11:29:03,454 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:03,454 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,454 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src 2018-01-05T11:29:03,454 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src 2018-01-05T11:29:03,457 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:03,458 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,458 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,458 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src1 2018-01-05T11:29:03,458 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src1 2018-01-05T11:29:03,460 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:03,460 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,460 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:03,460 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:03,470 DEBUG [AMReporterQueueDrainer] impl.AMReporter: Removing am 10.22.8.182:49824 with last associated dag QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=18} from heartbeat with taskCount=0, amFailed=false, isDone=true 2018-01-05T11:29:03,473 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed getting MetaData in Semantic Analysis 2018-01-05T11:29:03,475 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,475 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,475 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_not_null_constraints : db=default tbl=src 2018-01-05T11:29:03,475 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_not_null_constraints : db=default tbl=src 2018-01-05T11:29:03,476 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:03,476 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,476 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,476 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=src 2018-01-05T11:29:03,476 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=src 2018-01-05T11:29:03,476 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:03,476 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,476 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,476 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=src 2018-01-05T11:29:03,476 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=src 2018-01-05T11:29:03,477 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:03,477 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,477 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,477 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_unique_constraints : db=default tbl=src 2018-01-05T11:29:03,477 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_unique_constraints : db=default tbl=src 2018-01-05T11:29:03,477 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:03,477 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,477 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,477 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=src 2018-01-05T11:29:03,477 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=src 2018-01-05T11:29:03,477 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:03,477 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,478 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,478 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_not_null_constraints : db=default tbl=src1 2018-01-05T11:29:03,478 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_not_null_constraints : db=default tbl=src1 2018-01-05T11:29:03,478 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:03,478 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,478 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,478 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=src1 2018-01-05T11:29:03,478 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=src1 2018-01-05T11:29:03,478 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:03,478 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,479 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,479 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=src1 2018-01-05T11:29:03,479 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=src1 2018-01-05T11:29:03,479 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:03,479 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,480 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,480 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_unique_constraints : db=default tbl=src1 2018-01-05T11:29:03,480 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_unique_constraints : db=default tbl=src1 2018-01-05T11:29:03,480 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:03,480 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,480 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,480 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=src1 2018-01-05T11:29:03,480 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=src1 2018-01-05T11:29:03,481 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:03,481 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,482 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:29:03,482 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,483 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.sql2rel: Plan after trimming unused fields HiveJoin(condition=[=($0, $2)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.src]], table:alias=[a]) HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.src1]], table:alias=[b]) 2018-01-05T11:29:03,483 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan before removing subquery: HiveProject(key=[$0], value=[$1], key1=[$5], value1=[$6]) HiveJoin(condition=[=($0, $5)], joinType=[inner], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.src]], table:alias=[a]) HiveTableScan(table=[[default.src1]], table:alias=[b]) 2018-01-05T11:29:03,484 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan just after removing subquery: HiveProject(key=[$0], value=[$1], key1=[$5], value1=[$6]) HiveJoin(condition=[=($0, $5)], joinType=[inner], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.src]], table:alias=[a]) HiveTableScan(table=[[default.src1]], table:alias=[b]) 2018-01-05T11:29:03,484 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan after decorrelation: HiveProject(key=[$0], value=[$1], key1=[$5], value1=[$6]) HiveJoin(condition=[=($0, $5)], joinType=[inner], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.src]], table:alias=[a]) HiveTableScan(table=[[default.src1]], table:alias=[b]) 2018-01-05T11:29:03,484 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,485 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,485 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,485 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,485 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,486 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,486 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,486 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,486 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,487 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,487 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,502 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,502 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,502 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,503 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,503 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,503 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,503 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,503 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,526 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.sql2rel: Plan after trimming unused fields HiveJoin(condition=[=($0, $2)], joinType=[inner], algorithm=[none], cost=[not available]) HiveFilter(condition=[IS NOT NULL($0)]) HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.src]], table:alias=[a]) HiveFilter(condition=[IS NOT NULL($0)]) HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.src1]], table:alias=[b]) 2018-01-05T11:29:03,526 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,526 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,527 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,527 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,528 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,528 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,529 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,529 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,530 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,530 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,530 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,530 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,531 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,531 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,531 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,532 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Original plan for PlanModifier HiveJoin(condition=[=($0, $2)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.src]], table:alias=[a]) HiveProject(key=[$0], value=[$1]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.src1]], table:alias=[b]) 2018-01-05T11:29:03,532 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after top-level introduceDerivedTable HiveProject(key=[$0], value=[$1], key0=[$2], value0=[$3]) HiveJoin(condition=[=($0, $2)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.src]], table:alias=[a]) HiveProject(key=[$0], value=[$1]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.src1]], table:alias=[b]) 2018-01-05T11:29:03,532 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after nested convertOpTree HiveProject(key=[$0], value=[$1], key0=[$2], value0=[$3]) HiveJoin(condition=[=($0, $2)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.src]], table:alias=[a]) HiveProject(key=[$0], value=[$1]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.src1]], table:alias=[b]) 2018-01-05T11:29:03,533 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after propagating order HiveProject(key=[$0], value=[$1], key0=[$2], value0=[$3]) HiveJoin(condition=[=($0, $2)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.src]], table:alias=[a]) HiveProject(key=[$0], value=[$1]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.src1]], table:alias=[b]) 2018-01-05T11:29:03,533 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after fixTopOBSchema HiveProject(key=[$0], value=[$1], key0=[$2], value0=[$3]) HiveJoin(condition=[=($0, $2)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.src]], table:alias=[a]) HiveProject(key=[$0], value=[$1]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.src1]], table:alias=[b]) 2018-01-05T11:29:03,533 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Final plan after modifier HiveProject(a.key=[$0], a.value=[$1], b.key=[$2], b.value=[$3]) HiveJoin(condition=[=($0, $2)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.src]], table:alias=[a]) HiveProject(key=[$0], value=[$1]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.src1]], table:alias=[b]) 2018-01-05T11:29:03,533 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:03,533 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:03,533 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:03,533 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,533 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src 2018-01-05T11:29:03,533 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src 2018-01-05T11:29:03,536 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:03,537 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,537 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:03,537 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:03,537 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:03,537 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,537 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src1 2018-01-05T11:29:03,537 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src1 2018-01-05T11:29:03,539 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:03,540 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,540 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:03,540 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:03,540 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:03,540 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Table Plan for a TS[0] 2018-01-05T11:29:03,541 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Filter Plan for $hdt$_0 row schema: a{(key,key: string)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:03,541 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB a{(key,key: string)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB a{(key,key: string)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:03,541 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (. (tok_table_or_col a) key) key) (tok_selexpr (. (tok_table_or_col a) value) value)) 2018-01-05T11:29:03,541 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = a{(key,key: string)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} starRr = null 2018-01-05T11:29:03,541 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(key,_col0: string)(value,_col1: string)} 2018-01-05T11:29:03,541 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:03,541 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block $hdt$_0 2018-01-05T11:29:03,541 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block $hdt$_0 2018-01-05T11:29:03,541 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Table Plan for b TS[3] 2018-01-05T11:29:03,541 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Filter Plan for $hdt$_1 row schema: b{(key,key: string)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:03,541 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB b{(key,key: string)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB b{(key,key: string)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:03,541 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (. (tok_table_or_col b) key) key) (tok_selexpr (. (tok_table_or_col b) value) value)) 2018-01-05T11:29:03,541 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = b{(key,key: string)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} starRr = null 2018-01-05T11:29:03,542 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(key,_col0: string)(value,_col1: string)} 2018-01-05T11:29:03,542 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:03,542 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block $hdt$_1 2018-01-05T11:29:03,542 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block $hdt$_1 2018-01-05T11:29:03,542 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB $hdt$_0{(key,_col0: string)(value,_col1: string)} $hdt$_1{(key,_col2: string)(value,_col3: string)} after GB $hdt$_0{(key,_col0: string)(value,_col1: string)} $hdt$_1{(key,_col2: string)(value,_col3: string)} 2018-01-05T11:29:03,542 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (. (tok_table_or_col $hdt$_0) key) a.key) (tok_selexpr (. (tok_table_or_col $hdt$_0) value) a.value) (tok_selexpr (. (tok_table_or_col $hdt$_1) key) b.key) (tok_selexpr (. (tok_table_or_col $hdt$_1) value) b.value)) 2018-01-05T11:29:03,542 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = $hdt$_0{(key,_col0: string)(value,_col1: string)} $hdt$_1{(key,_col2: string)(value,_col3: string)} starRr = null 2018-01-05T11:29:03,543 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(a.key,_col0: string)(a.value,_col1: string)(b.key,_col2: string)(b.value,_col3: string)} 2018-01-05T11:29:03,543 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:03,543 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-03_453_7803144398595179850-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/-mr-10001 2018-01-05T11:29:03,543 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-03_453_7803144398595179850-1 2018-01-05T11:29:03,568 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/-mr-10003 2018-01-05T11:29:03,568 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/-mr-10001 row schema: null{(a.key,_col0: string)(a.value,_col1: string)(b.key,_col2: string)(b.value,_col3: string)} 2018-01-05T11:29:03,568 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block null 2018-01-05T11:29:03,568 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:29:03,568 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: CBO Succeeded; optimized logical plan. 2018-01-05T11:29:03,568 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Before logical optimization TS[0]-FIL[1]-SEL[2]-RS[6]-JOIN[8]-SEL[9]-FS[10] TS[3]-FIL[4]-SEL[5]-RS[7]-JOIN[8] 2018-01-05T11:29:03,568 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,568 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,568 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,569 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] lineage.Generator: Time taken for lineage transform=1 2018-01-05T11:29:03,569 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,569 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,569 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.PartitionColumnsSeparator: Partition columns not separated for null, is not IN operator : 2018-01-05T11:29:03,570 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.PartitionColumnsSeparator: Partition columns not separated for null, is not IN operator : 2018-01-05T11:29:03,570 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,570 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,570 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SyntheticJoinPredicate: Synthetic predicate: 0 --> 1 2018-01-05T11:29:03,570 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SyntheticJoinPredicate: Synthetic predicate: 1 --> 0 2018-01-05T11:29:03,570 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,570 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,570 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FS(10) 2018-01-05T11:29:03,570 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(9) 2018-01-05T11:29:03,570 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for JOIN(8) 2018-01-05T11:29:03,570 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(6) 2018-01-05T11:29:03,570 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(11) 2018-01-05T11:29:03,570 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias $hdt$_0: (_col0) IN (RS[7]) 2018-01-05T11:29:03,570 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(2) 2018-01-05T11:29:03,571 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of SEL for alias $hdt$_0: (key) IN (RS[7]) 2018-01-05T11:29:03,571 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(1) 2018-01-05T11:29:03,571 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias a: key is not null 2018-01-05T11:29:03,571 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias a: (key) IN (RS[7]) 2018-01-05T11:29:03,571 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:29:03,571 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of TS for alias a: key is not null; (key) IN (RS[7]) 2018-01-05T11:29:03,571 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(7) 2018-01-05T11:29:03,571 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(12) 2018-01-05T11:29:03,571 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias $hdt$_1: (_col0) IN (RS[6]) 2018-01-05T11:29:03,571 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(5) 2018-01-05T11:29:03,571 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of SEL for alias $hdt$_1: (key) IN (RS[6]) 2018-01-05T11:29:03,571 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(4) 2018-01-05T11:29:03,571 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias b: key is not null 2018-01-05T11:29:03,571 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias b: (key) IN (RS[6]) 2018-01-05T11:29:03,571 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for TS(3) 2018-01-05T11:29:03,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of TS for alias b: key is not null; (key) IN (RS[6]) 2018-01-05T11:29:03,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SimplePredicatePushDown: After PPD: TS[0]-FIL[13]-SEL[2]-RS[6]-JOIN[8]-SEL[9]-FS[10] TS[3]-FIL[14]-SEL[5]-RS[7]-JOIN[8] 2018-01-05T11:29:03,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,572 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: JOIN 8 oldExprs: {0=[Column[KEY.reducesinkkey0], Column[VALUE._col0]], 1=[Column[KEY.reducesinkkey0], Column[VALUE._col0]]} 2018-01-05T11:29:03,572 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: JOIN 8 newExprs: {0=[Column[KEY.reducesinkkey0], Column[VALUE._col0]], 1=[Column[KEY.reducesinkkey0], Column[VALUE._col0]]} 2018-01-05T11:29:03,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 6 key:[Column[_col0]] 2018-01-05T11:29:03,572 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 6 oldColExprMap: {VALUE._col0=Column[_col1], KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:03,572 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 6 newColExprMap: {VALUE._col0=Column[_col1], KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:03,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 7 key:[Column[_col0]] 2018-01-05T11:29:03,572 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 7 oldColExprMap: {VALUE._col0=Column[_col1], KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:03,572 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 7 newColExprMap: {VALUE._col0=Column[_col1], KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:03,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.IdentityProjectRemover: Identity project remover optimization removed : SEL[9] 2018-01-05T11:29:03,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: After logical optimization TS[0]-FIL[13]-SEL[2]-RS[6]-JOIN[8]-FS[10] TS[3]-FIL[14]-SEL[5]-RS[7]-JOIN[8] 2018-01-05T11:29:03,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Parent: TS[0] 2018-01-05T11:29:03,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Filter: (key is not null and (key) IN (RS[7])) (type: boolean) 2018-01-05T11:29:03,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: TableScan: TS[0] 2018-01-05T11:29:03,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Column key is not a partition column 2018-01-05T11:29:03,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Initiate semijoin reduction for key ((key is not null and (key) IN (RS[7])) 2018-01-05T11:29:03,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:03,574 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:03,574 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:03,574 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:03,574 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:03,574 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:03,574 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: DynamicSemiJoinPushdown: Saving RS to TS mapping: RS[19]: TS[0] 2018-01-05T11:29:03,574 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Parent: TS[3] 2018-01-05T11:29:03,574 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Filter: (key is not null and (key) IN (RS[6])) (type: boolean) 2018-01-05T11:29:03,574 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: TableScan: TS[3] 2018-01-05T11:29:03,574 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Column key is not a partition column 2018-01-05T11:29:03,574 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Initiate semijoin reduction for key ((key is not null and (key) IN (RS[6])) 2018-01-05T11:29:03,574 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:03,574 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:03,574 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:03,574 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:03,574 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:03,574 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:03,575 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: DynamicSemiJoinPushdown: Saving RS to TS mapping: RS[24]: TS[3] 2018-01-05T11:29:03,575 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,575 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,575 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,575 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=src 2018-01-05T11:29:03,575 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=src 2018-01-05T11:29:03,578 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.182787ms + 0.007466ms, the query is [select "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "TAB_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (...)] 2018-01-05T11:29:03,580 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:03,584 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:03,585 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,585 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting stats (Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE) on TS[0] 2018-01-05T11:29:03,585 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (src): numRows: 500 dataSize: 89000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {value= colName: value colType: string countDistincts: 309 numNulls: 0 avgColLen: 6.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, key= colName: key colType: string countDistincts: 309 numNulls: 0 avgColLen: 2.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:03,585 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,585 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=src1 2018-01-05T11:29:03,585 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=src1 2018-01-05T11:29:03,587 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.145988ms + 0.004417ms, the query is [select "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "TAB_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (...)] 2018-01-05T11:29:03,588 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:03,593 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:03,593 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:03,593 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting stats (Num rows: 25 Data size: 4375 Basic stats: COMPLETE Column stats: COMPLETE) on TS[3] 2018-01-05T11:29:03,593 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-TS[3] (src1): numRows: 25 dataSize: 4375 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {value= colName: value colType: string countDistincts: 19 numNulls: 0 avgColLen: 4.92 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, key= colName: key colType: string countDistincts: 16 numNulls: 0 avgColLen: 1.72 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:03,593 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_7_b_key_min), DynamicValue(RS_7_b_key_max)) Original num rows: 500 New num rows: 500 2018-01-05T11:29:03,593 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_7_b_key_min), DynamicValue(RS_7_b_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_7_b_key_bloom_filter))) Original num rows: 500 New num rows: 500 2018-01-05T11:29:03,593 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_7_b_key_min), DynamicValue(RS_7_b_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_7_b_key_bloom_filter)))) Original num rows: 500 New num rows: 500 2018-01-05T11:29:03,593 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-FIL[13]: numRows: 500 dataSize: 89000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {value= colName: value colType: string countDistincts: 309 numNulls: 0 avgColLen: 6.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, key= colName: key colType: string countDistincts: 309 numNulls: 0 avgColLen: 2.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting stats (Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE) on FIL[13] 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_6_a_key_min), DynamicValue(RS_6_a_key_max)) Original num rows: 25 New num rows: 25 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_6_a_key_min), DynamicValue(RS_6_a_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_6_a_key_bloom_filter))) Original num rows: 25 New num rows: 25 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_6_a_key_min), DynamicValue(RS_6_a_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_6_a_key_bloom_filter)))) Original num rows: 25 New num rows: 25 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-FIL[14]: numRows: 25 dataSize: 4375 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {value= colName: value colType: string countDistincts: 19 numNulls: 0 avgColLen: 4.92 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, key= colName: key colType: string countDistincts: 16 numNulls: 0 avgColLen: 1.72 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting stats (Num rows: 25 Data size: 4375 Basic stats: COMPLETE Column stats: COMPLETE) on FIL[14] 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[2] 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[2]: numRows: 500 dataSize: 89000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: string countDistincts: 309 numNulls: 0 avgColLen: 2.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: string countDistincts: 309 numNulls: 0 avgColLen: 6.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 25 Data size: 4375 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[5] 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[5]: numRows: 25 dataSize: 4375 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: string countDistincts: 16 numNulls: 0 avgColLen: 1.72 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: string countDistincts: 19 numNulls: 0 avgColLen: 4.92 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE) on RS[6] 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[6]: numRows: 500 dataSize: 89000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {VALUE._col0= colName: VALUE._col0 colType: string countDistincts: 309 numNulls: 0 avgColLen: 6.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, KEY.reducesinkkey0= colName: KEY.reducesinkkey0 colType: string countDistincts: 309 numNulls: 0 avgColLen: 2.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[20] 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[20]: numRows: 500 dataSize: 43500 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: string countDistincts: 309 numNulls: 0 avgColLen: 2.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 25 Data size: 4375 Basic stats: COMPLETE Column stats: COMPLETE) on RS[7] 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[7]: numRows: 25 dataSize: 4375 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {VALUE._col0= colName: VALUE._col0 colType: string countDistincts: 19 numNulls: 0 avgColLen: 4.92 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, KEY.reducesinkkey0= colName: KEY.reducesinkkey0 colType: string countDistincts: 16 numNulls: 0 avgColLen: 1.72 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[15] 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[15]: numRows: 25 dataSize: 2150 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: string countDistincts: 16 numNulls: 0 avgColLen: 1.72 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-JOIN[8]: detects none/multiple PK parents. 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.JoinOperator: Setting stats (Num rows: 40 Data size: 14120 Basic stats: COMPLETE Column stats: COMPLETE) on JOIN[8] 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-JOIN[8]: numRows: 40 dataSize: 14120 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: string countDistincts: 2 numNulls: 0 avgColLen: 2.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: string countDistincts: 16 numNulls: 0 avgColLen: 1.72 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: string countDistincts: 25 numNulls: 0 avgColLen: 6.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: string countDistincts: 19 numNulls: 0 avgColLen: 4.92 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[21]: inputSize: 89000 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[21] hashAgg: true 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 3] STATS-GBY[21]: cardinality: 1 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 552 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[21] 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[21]: numRows: 1 dataSize: 552 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[16]: inputSize: 4375 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[16] hashAgg: true 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 3] STATS-GBY[16]: cardinality: 1 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 552 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[16] 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[16]: numRows: 1 dataSize: 552 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-FS[10]: numRows: 40 dataSize: 14120 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: string countDistincts: 2 numNulls: 0 avgColLen: 2.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: string countDistincts: 16 numNulls: 0 avgColLen: 1.72 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: string countDistincts: 25 numNulls: 0 avgColLen: 6.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: string countDistincts: 19 numNulls: 0 avgColLen: 4.92 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 552 Basic stats: COMPLETE Column stats: COMPLETE) on RS[22] 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[22]: numRows: 1 dataSize: 552 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 552 Basic stats: COMPLETE Column stats: COMPLETE) on RS[17] 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[17]: numRows: 1 dataSize: 552 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[23]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[23] hashAgg: false 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 9] STATS-GBY[23]: cardinality: 1 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 552 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[23] 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[23]: numRows: 1 dataSize: 552 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[18]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[18] hashAgg: false 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 9] STATS-GBY[18]: cardinality: 1 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 552 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[18] 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[18]: numRows: 1 dataSize: 552 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 552 Basic stats: COMPLETE Column stats: COMPLETE) on RS[24] 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[24]: numRows: 1 dataSize: 552 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:03,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 552 Basic stats: COMPLETE Column stats: COMPLETE) on RS[19] 2018-01-05T11:29:03,595 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[19]: numRows: 1 dataSize: 552 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:03,595 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on TS[0] 2018-01-05T11:29:03,595 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on TS[3] 2018-01-05T11:29:03,595 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on FIL[13] 2018-01-05T11:29:03,595 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on FIL[14] 2018-01-05T11:29:03,595 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on SEL[2] 2018-01-05T11:29:03,595 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on SEL[5] 2018-01-05T11:29:03,595 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[KEY.reducesinkkey0]]; sort column names: [[KEY.reducesinkkey0]]; bucket count: -1 }) on RS[6] 2018-01-05T11:29:03,595 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on SEL[20] 2018-01-05T11:29:03,595 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[KEY.reducesinkkey0]]; sort column names: [[KEY.reducesinkkey0]]; bucket count: -1 }) on RS[7] 2018-01-05T11:29:03,595 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on SEL[15] 2018-01-05T11:29:03,595 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.JoinOperator: Setting traits ({ bucket column names: [[_col0], [_col2]]; sort column names: [[_col0], [_col2]]; bucket count: -1 }) on JOIN[8] 2018-01-05T11:29:03,595 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[21] 2018-01-05T11:29:03,595 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[16] 2018-01-05T11:29:03,595 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FileSinkOperator: Setting traits ({ bucket column names: [[_col0], [_col2]]; sort column names: [[_col0], [_col2]]; bucket count: -1 }) on FS[10] 2018-01-05T11:29:03,595 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[22] 2018-01-05T11:29:03,595 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[17] 2018-01-05T11:29:03,595 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[23] 2018-01-05T11:29:03,595 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[18] 2018-01-05T11:29:03,595 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[24] 2018-01-05T11:29:03,595 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[19] 2018-01-05T11:29:03,595 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,595 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,595 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Set parallelism for reduce sink RS[6] to: 1 2018-01-05T11:29:03,595 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Set parallelism for reduce sink RS[7] to: 1 2018-01-05T11:29:03,595 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConvertJoinMapJoin: Memory monitor info set to : { isLlap: true executorsPerNode: 4 maxExecutorsOverSubscribeMemory: 3 memoryOverSubscriptionFactor: 0.20000000298023224 memoryCheckInterval: 100000 noConditionalTaskSize: 10000000 adjustedNoConditionalTaskSize: 16000000 hashTableInflationFactor: 2.0 threshold: 32000000 } 2018-01-05T11:29:03,595 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConvertJoinMapJoin: Fallback to common merge join operator 2018-01-05T11:29:03,595 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.CommonMergeJoinOperator: Setting traits ({ bucket column names: [[_col0], [_col2]]; sort column names: [[_col0], [_col2]]; bucket count: 0 }) on MERGEJOIN[25] 2018-01-05T11:29:03,595 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.CommonMergeJoinOperator: Setting stats (Num rows: 40 Data size: 14120 Basic stats: COMPLETE Column stats: COMPLETE) on MERGEJOIN[25] 2018-01-05T11:29:03,595 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:03,595 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:03,595 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:03,595 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing BloomFilter cost/benefit for RS[19] - TS[0] (a) (Column[key]) 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing key domain cardinality, keyDomainCardinality=325, semiJoinKeyIsPK=false, selColStat= colName: _col0 colType: string countDistincts: 16 numNulls: 0 avgColLen: 1.72 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, selColSourceStat= colName: key colType: string countDistincts: 16 numNulls: 0 avgColLen: 1.72 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, tsColStat= colName: key colType: string countDistincts: 309 numNulls: 0 avgColLen: 2.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit for Column[_col0] to Column[key], selKeyCardinality=16, tsKeyCardinality=309, tsRows=500, keyDomainCardinality=325 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: SemiJoin key selectivity=0.04923076923076923, benefit=475.3846153846154 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit=475.3846153846154, cost=25.0, tsDataSize=500, netBenefit=450.3846153846154 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: netBenefit=0.9007692307692309 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing BloomFilter cost/benefit for RS[24] - TS[3] (b) (Column[key]) 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing key domain cardinality, keyDomainCardinality=325, semiJoinKeyIsPK=false, selColStat= colName: _col0 colType: string countDistincts: 309 numNulls: 0 avgColLen: 2.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, selColSourceStat= colName: key colType: string countDistincts: 309 numNulls: 0 avgColLen: 2.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, tsColStat= colName: key colType: string countDistincts: 16 numNulls: 0 avgColLen: 1.72 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit for Column[_col0] to Column[key], selKeyCardinality=309, tsKeyCardinality=16, tsRows=25, keyDomainCardinality=325 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: SemiJoin key selectivity=0.9507692307692308, benefit=1.2307692307692297 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit=1.2307692307692297, cost=500.0, tsDataSize=25, netBenefit=-498.7692307692308 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: netBenefit=-19.950769230769232 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Reduction factor not satisfied for RS[24]-TS[3] (b). Removing semijoin optimization. 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Removing ReduceSink RS[24] and TableScan TS[3] 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Adding special edge: RS --> TS[0] 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: FS, 10 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: MERGEJOIN, 25 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 6 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 2 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: FIL, 13 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 7 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 19 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: GBY, 18 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 17 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: GBY, 16 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 15 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 5 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: FIL, 14 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: TS, 3 2018-01-05T11:29:03,596 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Cycle free: true 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: Before SharedWorkOptimizer: TS[0]-FIL[13]-SEL[2]-RS[6]-MERGEJOIN[25]-FS[10] TS[3]-FIL[14]-SEL[5]-RS[7]-MERGEJOIN[25] -SEL[15]-GBY[16]-RS[17]-GBY[18]-RS[19] 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: DPP information stored in the cache: {TS[0]=[RS[19]]} 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: Sorted tables by size: [default.src=89000, default.src1=4375] 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: After SharedWorkOptimizer: TS[0]-FIL[13]-SEL[2]-RS[6]-MERGEJOIN[25]-FS[10] TS[3]-FIL[14]-SEL[5]-RS[7]-MERGEJOIN[25] -SEL[15]-GBY[16]-RS[17]-GBY[18]-RS[19] 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: string|{a}key,value: string|{a}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{a}block__offset__inside__file,INPUT__FILE__NAME: string|{a}input__file__name,ROW__ID: struct|{a}row__id) 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[3] with rs:(key: string|{b}key,value: string|{b}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{b}block__offset__inside__file,INPUT__FILE__NAME: string|{b}input__file__name,ROW__ID: struct|{b}row__id) 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(Const boolean true, Const boolean true) -> Const boolean true 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPAnd(Const boolean true, Const boolean true)) -> GenericUDFOPNotNull(Column[key]) 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FIL[13] with rs:(key: string|{a}key,value: string|{a}value) 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FIL[13] 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Old filter FIL[13] conditions:(key is not null and (key BETWEEN DynamicValue(RS_7_b_key_min) AND DynamicValue(RS_7_b_key_max) and in_bloom_filter(key, DynamicValue(RS_7_b_key_bloom_filter)))) 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New filter FIL[13] conditions:(key is not null and (key BETWEEN DynamicValue(RS_7_b_key_min) AND DynamicValue(RS_7_b_key_max) and in_bloom_filter(key, DynamicValue(RS_7_b_key_bloom_filter)))) 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FIL[14] with rs:(key: string|{b}key,value: string|{b}value) 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_3 {} 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FIL[14] 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Old filter FIL[14] conditions:(key is not null and (true and true)) 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(Const boolean true, Const boolean true) -> Const boolean true 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPAnd(Const boolean true, Const boolean true)) -> GenericUDFOPNotNull(Column[key]) 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New filter FIL[14] conditions:key is not null 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[2] with rs:(_col0: string|{$hdt$_0}key,_col1: string|{$hdt$_0}value) 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op FIL_13 {} 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[2] 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value]) 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: string|{$hdt$_1}key,_col1: string|{$hdt$_1}value) 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op FIL_14 {} 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value]) 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[6] with rs:(KEY.reducesinkkey0: string|{$hdt$_0}key,VALUE._col0: string|{$hdt$_0}value) 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_2 {} 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[6] 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[7] with rs:(KEY.reducesinkkey0: string|{$hdt$_1}key,VALUE._col0: string|{$hdt$_1}value) 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[7] 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[15] with rs:(_col0: string|{$hdt$_1}key) 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[15] 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0]) 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:MERGEJOIN[25] with rs:(_col0: string|{$hdt$_0}key,_col1: string|{$hdt$_0}value,_col2: string|{$hdt$_1}key,_col3: string|{$hdt$_1}value) 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_6 {} 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_7 {} 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator MERGEJOIN[25] 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[16] with rs:(_col0: string|{},_col1: string|{},_col2: string|{}) 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_15 {} 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[16] 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[10] with rs:(_col0: string|{},_col1: string|{},_col2: string|{},_col3: string|{}) 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op MERGEJOIN_25 {} 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[10] 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[17] with rs:(_col0: string|{},_col1: string|{},_col2: string|{}) 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_16 {} 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[17] 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[18] with rs:(_col0: string|{},_col1: string|{},_col2: string|{}) 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_17 {} 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[18] 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[19] with rs:(_col0: string|{},_col1: string|{},_col2: string|{}) 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_18 {} 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[19] 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: TS[0] 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[6] 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:29:03,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src of table a 2018-01-05T11:29:03,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src 2018-01-05T11:29:03,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[6] 2018-01-05T11:29:03,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: MERGEJOIN[25] 2018-01-05T11:29:03,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: FS[10] 2018-01-05T11:29:03,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding reduce work (Reducer 2) for MERGEJOIN[25] 2018-01-05T11:29:03,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Setting up reduce sink: RS[6] with following reduce work: Reducer 2 2018-01-05T11:29:03,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[6] as parent from MERGEJOIN[25] 2018-01-05T11:29:03,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[7] as parent from MERGEJOIN[25] 2018-01-05T11:29:03,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: FS[10] 2018-01-05T11:29:03,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: TS[3] 2018-01-05T11:29:03,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[7] 2018-01-05T11:29:03,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding map work (Map 3) for TS[3] 2018-01-05T11:29:03,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1 of table b 2018-01-05T11:29:03,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1 2018-01-05T11:29:03,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Second pass. Leaf operator: RS[7] has common downstream work: org.apache.hadoop.hive.ql.plan.MergeJoinWork@1830302e 2018-01-05T11:29:03,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: TS[3] 2018-01-05T11:29:03,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[17] 2018-01-05T11:29:03,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[17] 2018-01-05T11:29:03,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: GBY[18] 2018-01-05T11:29:03,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[19] 2018-01-05T11:29:03,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding reduce work (Reducer 4) for GBY[18] 2018-01-05T11:29:03,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Setting up reduce sink: RS[17] with following reduce work: Reducer 4 2018-01-05T11:29:03,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[17] as parent from GBY[18] 2018-01-05T11:29:03,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[19] 2018-01-05T11:29:03,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: ResduceSink RS[19] to TableScan TS[0] 2018-01-05T11:29:03,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Connecting Baswork - Reducer 4 to Map 1 2018-01-05T11:29:03,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:29:03,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 3 2018-01-05T11:29:03,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:03,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:03,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 3 2018-01-05T11:29:03,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:03,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:03,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:29:03,599 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Examining input format to see if vectorization is enabled. 2018-01-05T11:29:03,599 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorization is enabled for input format(s) [org.apache.hadoop.mapred.TextInputFormat] 2018-01-05T11:29:03,599 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing MapWork... 2018-01-05T11:29:03,599 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFOPNotNull(Column[key]), Vectorized Expression = SelectColumnIsNotNull(col 0:string) 2018-01-05T11:29:03,599 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator 2018-01-05T11:29:03,599 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.FilterDesc 2018-01-05T11:29:03,599 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[key], Vectorized Expression = col 0:string 2018-01-05T11:29:03,599 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[value], Vectorized Expression = col 1:string 2018-01-05T11:29:03,599 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator 2018-01-05T11:29:03,599 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.SelectDesc 2018-01-05T11:29:03,599 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:string 2018-01-05T11:29:03,599 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col1], Vectorized Expression = col 1:string 2018-01-05T11:29:03,599 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator groupby typeName string 2018-01-05T11:29:03,599 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkStringOperator 2018-01-05T11:29:03,599 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:03,599 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkStringOperator 2018-01-05T11:29:03,599 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:03,599 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:string 2018-01-05T11:29:03,599 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator 2018-01-05T11:29:03,599 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.SelectDesc 2018-01-05T11:29:03,599 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vector GROUP BY operator will use processing mode HASH 2018-01-05T11:29:03,599 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:string 2018-01-05T11:29:03,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:string 2018-01-05T11:29:03,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:string 2018-01-05T11:29:03,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator 2018-01-05T11:29:03,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.GroupByDesc 2018-01-05T11:29:03,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:string 2018-01-05T11:29:03,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col1], Vectorized Expression = col 1:string 2018-01-05T11:29:03,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col2], Vectorized Expression = col 2:binary 2018-01-05T11:29:03,600 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkEmptyKeyOperator 2018-01-05T11:29:03,600 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:03,600 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkEmptyKeyOperator: VectorReduceSinkEmptyKeyOperator constructor vectorReduceSinkInfo org.apache.hadoop.hive.ql.plan.VectorReduceSinkInfo@1ad5b8a5 2018-01-05T11:29:03,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkEmptyKeyOperator 2018-01-05T11:29:03,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:03,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [key, value, ROW__ID] 2018-01-05T11:29:03,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [string, string, struct] 2018-01-05T11:29:03,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE, NONE, NONE] 2018-01-05T11:29:03,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 0 2018-01-05T11:29:03,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 1 2018-01-05T11:29:03,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:03,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:03,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Using reduce tag 0 2018-01-05T11:29:03,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1, _col2] columnTypes=[string, string, binary] 2018-01-05T11:29:03,600 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing ReduceWork... 2018-01-05T11:29:03,600 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vector GROUP BY operator will use processing mode STREAMING 2018-01-05T11:29:03,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[VALUE._col0], Vectorized Expression = col 0:string 2018-01-05T11:29:03,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[VALUE._col1], Vectorized Expression = col 1:string 2018-01-05T11:29:03,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[VALUE._col2], Vectorized Expression = col 2:binary 2018-01-05T11:29:03,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator 2018-01-05T11:29:03,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.GroupByDesc 2018-01-05T11:29:03,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:string 2018-01-05T11:29:03,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col1], Vectorized Expression = col 1:string 2018-01-05T11:29:03,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col2], Vectorized Expression = col 2:binary 2018-01-05T11:29:03,600 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkEmptyKeyOperator 2018-01-05T11:29:03,600 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:03,600 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkEmptyKeyOperator: VectorReduceSinkEmptyKeyOperator constructor vectorReduceSinkInfo org.apache.hadoop.hive.ql.plan.VectorReduceSinkInfo@50cfa74d 2018-01-05T11:29:03,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkEmptyKeyOperator 2018-01-05T11:29:03,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:03,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [VALUE._col0, VALUE._col1, VALUE._col2] 2018-01-05T11:29:03,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [string, string, binary] 2018-01-05T11:29:03,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE, NONE, NONE] 2018-01-05T11:29:03,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 0 2018-01-05T11:29:03,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 0 2018-01-05T11:29:03,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:03,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:03,600 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Examining input format to see if vectorization is enabled. 2018-01-05T11:29:03,601 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorization is enabled for input format(s) [org.apache.hadoop.mapred.TextInputFormat] 2018-01-05T11:29:03,601 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing MapWork... 2018-01-05T11:29:03,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFOPNotNull(Column[key]), Vectorized Expression = SelectColumnIsNotNull(col 0:string) 2018-01-05T11:29:03,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_7_b_key_min), DynamicValue(RS_7_b_key_max)), Vectorized Expression = FilterStringColumnBetweenDynamicValue(col 0:string, left NULL, right NULL) 2018-01-05T11:29:03,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFInBloomFilter(Column[key], DynamicValue(RS_7_b_key_bloom_filter)), Vectorized Expression = VectorInBloomFilterColDynamicValue 2018-01-05T11:29:03,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_7_b_key_min), DynamicValue(RS_7_b_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_7_b_key_bloom_filter))), Vectorized Expression = FilterExprAndExpr(children: FilterStringColumnBetweenDynamicValue(col 0:string, left NULL, right NULL), VectorInBloomFilterColDynamicValue) 2018-01-05T11:29:03,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_7_b_key_min), DynamicValue(RS_7_b_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_7_b_key_bloom_filter)))), Vectorized Expression = FilterExprAndExpr(children: SelectColumnIsNotNull(col 0:string), FilterExprAndExpr(children: FilterStringColumnBetweenDynamicValue(col 0:string, left NULL, right NULL), VectorInBloomFilterColDynamicValue)) 2018-01-05T11:29:03,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator 2018-01-05T11:29:03,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.FilterDesc 2018-01-05T11:29:03,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[key], Vectorized Expression = col 0:string 2018-01-05T11:29:03,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[value], Vectorized Expression = col 1:string 2018-01-05T11:29:03,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator 2018-01-05T11:29:03,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.SelectDesc 2018-01-05T11:29:03,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:string 2018-01-05T11:29:03,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col1], Vectorized Expression = col 1:string 2018-01-05T11:29:03,601 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator groupby typeName string 2018-01-05T11:29:03,601 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkStringOperator 2018-01-05T11:29:03,601 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:03,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkStringOperator 2018-01-05T11:29:03,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:03,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [key, value, ROW__ID] 2018-01-05T11:29:03,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [string, string, struct] 2018-01-05T11:29:03,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE, NONE, NONE] 2018-01-05T11:29:03,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 0 2018-01-05T11:29:03,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 1 2018-01-05T11:29:03,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:03,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:03,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:29:03,602 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: llap mode: only 2018-01-05T11:29:03,602 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Map 3 2018-01-05T11:29:03,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'min(_col0)' 2018-01-05T11:29:03,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '_col0' 2018-01-05T11:29:03,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'max(_col0)' 2018-01-05T11:29:03,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '_col0' 2018-01-05T11:29:03,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'bloom_filter(_col0, expectedEntries=16)' 2018-01-05T11:29:03,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '_col0' 2018-01-05T11:29:03,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '_col0' 2018-01-05T11:29:03,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key' 2018-01-05T11:29:03,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'value' 2018-01-05T11:29:03,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key is not null' 2018-01-05T11:29:03,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key is not null' 2018-01-05T11:29:03,602 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:03,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@5dd2a02e can be done in LLAP 2018-01-05T11:29:03,602 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Reducer 4 2018-01-05T11:29:03,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'min(VALUE._col0)' 2018-01-05T11:29:03,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:29:03,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'max(VALUE._col1)' 2018-01-05T11:29:03,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'VALUE._col1' 2018-01-05T11:29:03,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'bloom_filter(VALUE._col2, expectedEntries=16)' 2018-01-05T11:29:03,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'VALUE._col2' 2018-01-05T11:29:03,602 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:03,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@617659a can be done in LLAP 2018-01-05T11:29:03,602 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:29:03,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key' 2018-01-05T11:29:03,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'value' 2018-01-05T11:29:03,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_7_b_key_min) AND DynamicValue(RS_7_b_key_max) and in_bloom_filter(key, DynamicValue(RS_7_b_key_bloom_filter))))' 2018-01-05T11:29:03,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_7_b_key_min) AND DynamicValue(RS_7_b_key_max) and in_bloom_filter(key, DynamicValue(RS_7_b_key_bloom_filter))))' 2018-01-05T11:29:03,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_7_b_key_min) AND DynamicValue(RS_7_b_key_max) and in_bloom_filter(key, DynamicValue(RS_7_b_key_bloom_filter))))' 2018-01-05T11:29:03,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_7_b_key_min) AND DynamicValue(RS_7_b_key_max) and in_bloom_filter(key, DynamicValue(RS_7_b_key_bloom_filter))))' 2018-01-05T11:29:03,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_7_b_key_min) AND DynamicValue(RS_7_b_key_max) and in_bloom_filter(key, DynamicValue(RS_7_b_key_bloom_filter))))' 2018-01-05T11:29:03,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_7_b_key_min) AND DynamicValue(RS_7_b_key_max) and in_bloom_filter(key, DynamicValue(RS_7_b_key_bloom_filter))))' 2018-01-05T11:29:03,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_7_b_key_min) AND DynamicValue(RS_7_b_key_max) and in_bloom_filter(key, DynamicValue(RS_7_b_key_bloom_filter))))' 2018-01-05T11:29:03,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_7_b_key_min) AND DynamicValue(RS_7_b_key_max) and in_bloom_filter(key, DynamicValue(RS_7_b_key_bloom_filter))))' 2018-01-05T11:29:03,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_7_b_key_min) AND DynamicValue(RS_7_b_key_max) and in_bloom_filter(key, DynamicValue(RS_7_b_key_bloom_filter))))' 2018-01-05T11:29:03,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_7_b_key_min) AND DynamicValue(RS_7_b_key_max) and in_bloom_filter(key, DynamicValue(RS_7_b_key_bloom_filter))))' 2018-01-05T11:29:03,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_7_b_key_min) AND DynamicValue(RS_7_b_key_max) and in_bloom_filter(key, DynamicValue(RS_7_b_key_bloom_filter))))' 2018-01-05T11:29:03,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_7_b_key_min) AND DynamicValue(RS_7_b_key_max) and in_bloom_filter(key, DynamicValue(RS_7_b_key_bloom_filter))))' 2018-01-05T11:29:03,602 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:03,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@1a1820de can be done in LLAP 2018-01-05T11:29:03,602 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Reducer 2 2018-01-05T11:29:03,602 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:03,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MergeJoinWork@1830302e can be done in LLAP 2018-01-05T11:29:03,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.SerializeFilter: Serializing: key is not null 2018-01-05T11:29:03,603 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.SerializeFilter: Serializing: (key is not null and (key BETWEEN DynamicValue(RS_7_b_key_min) AND DynamicValue(RS_7_b_key_max) and in_bloom_filter(key, DynamicValue(RS_7_b_key_bloom_filter)))) 2018-01-05T11:29:03,604 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,604 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed plan generation 2018-01-05T11:29:03,604 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:03,604 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: validation start 2018-01-05T11:29:03,604 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:29:03,604 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,604 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:a.key, type:string, comment:null), FieldSchema(name:a.value, type:string, comment:null), FieldSchema(name:b.key, type:string, comment:null), FieldSchema(name:b.value, type:string, comment:null)], properties:null) 2018-01-05T11:29:03,605 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: Initializing operator LIST_SINK[37] 2018-01-05T11:29:03,611 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: Initialization Done 37 LIST_SINK 2018-01-05T11:29:03,611 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: Operator 37 LIST_SINK initialized 2018-01-05T11:29:03,611 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: Initialization Done 37 LIST_SINK done is reset. 2018-01-05T11:29:03,611 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,611 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:03,611 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {getNotNullConstraints_(NotNullConstraintsRequest, )=1, isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=13, flushCache_()=0, getUniqueConstraints_(UniqueConstraintsRequest, )=0, getPrimaryKeys_(PrimaryKeysRequest, )=2, getTableColumnStatistics_(String, String, List, )=18, getForeignKeys_(ForeignKeysRequest, )=1} 2018-01-05T11:29:03,611 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d); Time taken: 0.158 seconds 2018-01-05T11:29:03,611 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:03,611 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,611 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d): select * from src a join src1 b on a.key = b.key 2018-01-05T11:29:03,612 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,612 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: select * from src a join src1 b on a.key = b.key 2018-01-05T11:29:03,612 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:29:03,613 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@src 2018-01-05T11:29:03,613 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@src1 2018-01-05T11:29:03,613 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/-mr-10001 2018-01-05T11:29:03,613 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,613 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,613 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,613 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Query ID = djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d 2018-01-05T11:29:03,613 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Total jobs = 1 2018-01-05T11:29:03,613 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,613 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:29:03,613 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-1:MAPRED] in serial mode 2018-01-05T11:29:03,630 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:29:03,663 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:29:03,663 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:29:03,663 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:29:03,663 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d 2018-01-05T11:29:03,664 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Session is already open 2018-01-05T11:29:03,664 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:29:03,664 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,664 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Dag name: select * from src a join src1 b on a...b.key (Stage-1) 2018-01-05T11:29:03,664 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: DagInfo: {"context":"Hive","description":"\n\nselect * from src a join src1 b on a.key = b.key"} 2018-01-05T11:29:03,664 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:29:03,664 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,677 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,677 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:29:03,685 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,685 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 2 size: 1.81KB 2018-01-05T11:29:03,704 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,705 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,717 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:29:03,717 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,717 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:29:03,725 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,725 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 5.34KB 2018-01-05T11:29:03,732 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src 2018-01-05T11:29:03,732 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,734 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,747 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,747 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:29:03,755 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,755 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 4 size: 5.10KB 2018-01-05T11:29:03,759 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,771 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,785 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:29:03,785 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,785 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:29:03,789 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,789 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: Serialized plan (via RPC) - name: Map 3 size: 5.42KB 2018-01-05T11:29:03,795 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1 2018-01-05T11:29:03,795 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,800 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,800 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,800 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=select * from src a join src1 b on a...b.key (Stage-1), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d } 2018-01-05T11:29:03,800 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:03,800 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:03,801 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:03,801 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:03,803 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: select * from src a join src1 b on a...b.key (Stage-1) 2018-01-05T11:29:03,803 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:29:03,804 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_34, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_34.dot 2018-01-05T11:29:03,804 INFO [IPC Server handler 0 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_34 2018-01-05T11:29:03,804 WARN [IPC Server handler 0 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:29:03,804 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Running DAG: select * from src a join src1 b on a...b.key (Stage-1), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d } 2018-01-05T11:29:03,830 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_34, submitTime=1515180543803, queueName=null 2018-01-05T11:29:03,830 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_34_00 [Map 3]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_34_00 [Map 3], Default Resources= 2018-01-05T11:29:03,830 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_34_00 [Map 3] 2018-01-05T11:29:03,831 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 3 2018-01-05T11:29:03,831 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_34_00 [Map 3] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:03,831 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_34_01 [Reducer 4]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_34_01 [Reducer 4], Default Resources= 2018-01-05T11:29:03,831 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 4 2018-01-05T11:29:03,831 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_34_01 [Reducer 4] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:03,831 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_34_02 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_34_02 [Map 1], Default Resources= 2018-01-05T11:29:03,831 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_34_02 [Map 1] 2018-01-05T11:29:03,831 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:29:03,831 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_34_02 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:03,832 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_34_03 [Reducer 2]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_34_03 [Reducer 2], Default Resources= 2018-01-05T11:29:03,832 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_34_03 [Reducer 2] 2018-01-05T11:29:03,832 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 2 2018-01-05T11:29:03,832 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_34_03 [Reducer 2] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:03,832 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:29:03,832 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_34, initTime=1515180543832 2018-01-05T11:29:03,832 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: dag_1515180518813_0001_34 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:29:03,833 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:29:03,833 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_34, startTime=1515180543833 2018-01-05T11:29:03,833 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_34 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:29:03,833 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 3 : {b={InputName=b}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:29:03,833 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: b, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:29:03,833 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_34_00 [Map 3] 2018-01-05T11:29:03,833 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_34, dagName=select * from src a join src1 b on a...b.key (Stage-1) 2018-01-05T11:29:03,833 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,833 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,833 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:03,839 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_34_00 [Map 3] 2018-01-05T11:29:03,839 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_34_00 [Map 3] 2018-01-05T11:29:03,839 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:03,839 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_34_00 [Map 3] 2018-01-05T11:29:03,841 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:29:03,841 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@6ca50e66 2018-01-05T11:29:03,842 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir/0d27c3bb-5a71-454d-ba1d-9144cc0bd13d/map.xml 2018-01-05T11:29:03,842 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir/0d27c3bb-5a71-454d-ba1d-9144cc0bd13d/map.xml 2018-01-05T11:29:03,842 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir/0d27c3bb-5a71-454d-ba1d-9144cc0bd13d/map.xml 2018-01-05T11:29:03,842 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:03,842 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:03,842 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:03,871 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:03,871 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 3 size: 5.42KB 2018-01-05T11:29:03,871 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_34_00 [Map 3] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:29:03,871 INFO [InputInitializer {Map 3} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: b on vertex vertex_1515180518813_0001_34_00 [Map 3] 2018-01-05T11:29:03,871 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting user vertex manager plugin: org.apache.tez.dag.library.vertexmanager.ShuffleVertexManager on vertex: vertex_1515180518813_0001_34_01 [Reducer 4] 2018-01-05T11:29:03,871 INFO [InputInitializer {Map 3} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:29:03,872 DEBUG [InputInitializer {Map 3} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:03,872 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.0 maxFrac: 0.0 auto: false desiredTaskIput: 104857600 2018-01-05T11:29:03,872 INFO [InputInitializer {Map 3} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:29:03,872 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:29:03,872 DEBUG [InputInitializer {Map 3} #0] log.PerfLogger: 2018-01-05T11:29:03,872 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 1 tasks for vertex: vertex_1515180518813_0001_34_01 [Reducer 4] 2018-01-05T11:29:03,872 INFO [InputInitializer {Map 3} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir/0d27c3bb-5a71-454d-ba1d-9144cc0bd13d/map.xml 2018-01-05T11:29:03,872 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_34_01 [Reducer 4] 2018-01-05T11:29:03,872 DEBUG [InputInitializer {Map 3} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:03,872 INFO [InputInitializer {Map 3} #0] exec.Utilities: Processing alias b 2018-01-05T11:29:03,872 DEBUG [InputInitializer {Map 3} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1 2018-01-05T11:29:03,872 INFO [InputInitializer {Map 3} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1 2018-01-05T11:29:03,872 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_34_01, reconfigureDoneTime=1515180543872, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:29:03,872 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:VERTEX_INITIALIZED]: vertexName=Reducer 4, vertexId=vertex_1515180518813_0001_34_01, initRequestedTime=1515180543871, initedTime=1515180543872, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:03,872 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_34_01 [Reducer 4] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:29:03,872 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {a={InputName=a}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:29:03,872 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: a, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:29:03,872 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_34_02 [Map 1] 2018-01-05T11:29:03,873 DEBUG [InputInitializer {Map 3} #0] io.HiveInputFormat: Pushdown initiated with filterText = key is not null, filterExpr = GenericUDFOPNotNull(Column[key]), serializedFilterExpr = AQEAamF2YS51dGlsLkFycmF5TGlz9AEBAQFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVDb2x1bW5EZXPjAQFrZfkAAAGCYgECb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5zZXJkZTIudHlwZWluZm8uUHJpbWl0aXZlVHlwZUluZu8BAXN0cmlu5wEDb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGT1BOb3ROdWzsAQEEb3JnLmFwYWNoZS5oYWRvb3AuaW8uQm9vbGVhbldyaXRhYmzlAQAAAQIBAWJvb2xlYe4= (desc) 2018-01-05T11:29:03,873 INFO [InputInitializer {Map 3} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 0,1 2018-01-05T11:29:03,873 INFO [InputInitializer {Map 3} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = key,value 2018-01-05T11:29:03,873 INFO [InputInitializer {Map 3} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1 2018-01-05T11:29:03,873 DEBUG [InputInitializer {Map 3} #0] io.HiveInputFormat: Pushdown initiated with filterText = key is not null, filterExpr = GenericUDFOPNotNull(Column[key]), serializedFilterExpr = AQEAamF2YS51dGlsLkFycmF5TGlz9AEBAQFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVDb2x1bW5EZXPjAQFrZfkAAAGCYgECb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5zZXJkZTIudHlwZWluZm8uUHJpbWl0aXZlVHlwZUluZu8BAXN0cmlu5wEDb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGT1BOb3ROdWzsAQEEb3JnLmFwYWNoZS5oYWRvb3AuaW8uQm9vbGVhbldyaXRhYmzlAQAAAQIBAWJvb2xlYe4= (desc) 2018-01-05T11:29:03,873 DEBUG [InputInitializer {Map 3} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:03,873 DEBUG [InputInitializer {Map 3} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:03,873 DEBUG [InputInitializer {Map 3} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:03,873 DEBUG [InputInitializer {Map 3} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:03,879 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_34_02 [Map 1] 2018-01-05T11:29:03,879 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_34_02 [Map 1] 2018-01-05T11:29:03,879 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:03,879 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_34_02 [Map 1] 2018-01-05T11:29:03,882 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:29:03,882 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@65b6c3cc 2018-01-05T11:29:03,882 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir/e0b5a45c-1c80-452b-acaf-ad6f51dd68d8/map.xml 2018-01-05T11:29:03,882 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir/e0b5a45c-1c80-452b-acaf-ad6f51dd68d8/map.xml 2018-01-05T11:29:03,882 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir/e0b5a45c-1c80-452b-acaf-ad6f51dd68d8/map.xml 2018-01-05T11:29:03,882 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:03,882 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:03,882 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:03,891 DEBUG [InputInitializer {Map 3} #0] mapred.FileInputFormat: Time taken to get FileStatuses: 18 2018-01-05T11:29:03,891 INFO [InputInitializer {Map 3} #0] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:29:03,892 DEBUG [InputInitializer {Map 3} #0] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 18 2018-01-05T11:29:03,892 INFO [InputInitializer {Map 3} #0] io.HiveInputFormat: number of splits 1 2018-01-05T11:29:03,892 DEBUG [InputInitializer {Map 3} #0] log.PerfLogger: 2018-01-05T11:29:03,892 INFO [InputInitializer {Map 3} #0] tez.HiveSplitGenerator: Number of input splits: 1. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:29:03,892 INFO [InputInitializer {Map 3} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir/0d27c3bb-5a71-454d-ba1d-9144cc0bd13d/map.xml 2018-01-05T11:29:03,892 DEBUG [InputInitializer {Map 3} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir/0d27c3bb-5a71-454d-ba1d-9144cc0bd13d/map.xml 2018-01-05T11:29:03,892 DEBUG [InputInitializer {Map 3} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir/0d27c3bb-5a71-454d-ba1d-9144cc0bd13d/map.xml 2018-01-05T11:29:03,892 DEBUG [InputInitializer {Map 3} #0] log.PerfLogger: 2018-01-05T11:29:03,892 INFO [InputInitializer {Map 3} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:03,892 DEBUG [InputInitializer {Map 3} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:03,897 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:03,897 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 5.34KB 2018-01-05T11:29:03,897 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_34_02 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:29:03,898 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: a on vertex vertex_1515180518813_0001_34_02 [Map 1] 2018-01-05T11:29:03,898 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:29:03,898 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to ShuffleVertexManager for vertex_1515180518813_0001_34_03 [Reducer 2] 2018-01-05T11:29:03,898 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:03,898 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:29:03,898 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:03,898 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir/e0b5a45c-1c80-452b-acaf-ad6f51dd68d8/map.xml 2018-01-05T11:29:03,898 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:03,898 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias a 2018-01-05T11:29:03,898 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src 2018-01-05T11:29:03,898 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src 2018-01-05T11:29:03,899 DEBUG [InputInitializer {Map 1} #0] io.HiveInputFormat: Pushdown initiated with filterText = (key is not null and (key BETWEEN DynamicValue(RS_7_b_key_min) AND DynamicValue(RS_7_b_key_max) and in_bloom_filter(key, DynamicValue(RS_7_b_key_bloom_filter)))), filterExpr = GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_7_b_key_min), DynamicValue(RS_7_b_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_7_b_key_bloom_filter)))), serializedFilterExpr = AQEAamF2YS51dGlsLkFycmF5TGlz9AECAQFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVHZW5lcmljRnVuY0Rlc+MBAQABAQECb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC5wbGFuLkV4cHJOb2RlQ29sdW1uRGVz4wEBa2X5AAABgmEBA29yZy5hcGFjaGUuaGFkb29wLmhpdmUuc2VyZGUyLnR5cGVpbmZvLlByaW1pdGl2ZVR5cGVJbmbvAQFzdHJpbucBBG9yZy5hcGFjaGUuaGFkb29wLmhpdmUucWwudWRmLmdlbmVyaWMuR2VuZXJpY1VERk9QTm90TnVs7AEBBW9yZy5hcGFjaGUuaGFkb29wLmlvLkJvb2xlYW5Xcml0YWJs5QEAAAEDAQFib29sZWHuAQEBAQABAgEBAQEAAQQBBm9yZy5hcGFjaGUuaGFkb29wLmhpdmUucWwucGxhbi5FeHByTm9kZUNvbnN0YW50RGVz4wEBAw0FAAECAQcAAAgBAwkBB29yZy5hcGFjaGUuaGFkb29wLmhpdmUucWwucGxhbi5FeHByTm9kZUR5bmFtaWNWYWx1ZURlc+MBAQhvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRHluYW1pY1ZhbHXlAQFSU183X2Jfa2V5X21p7gEJ1gFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnNlcmRlMi5vYmplY3RpbnNwZWN0b3IucHJpbWl0aXZlLldyaXRhYmxlU3RyaW5nT2JqZWN0SW5zcGVjdG9yAQEDCQEDCQEDCQEHAQEIAQFSU183X2Jfa2V5X21h+AEJGAEDCQEDCQEKb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGQmV0d2Vl7gEBC8EBW0xvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnNlcmRlMi5vYmplY3RpbnNwZWN0b3IuT2JqZWN0SW5zcGVjdG9yOwEFAQzfAW9yZy5hcGFjaGUuaGFkb29wLmhpdmUuc2VyZGUyLm9iamVjdGluc3BlY3Rvci5wcmltaXRpdmUuV3JpdGFibGVDb25zdGFudEJvb2xlYW5PYmplY3RJbnNwZWN0b3IBAQMNAQUBAAEJGAEJGAEJGAENxQFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnVkZi5nZW5lcmljLkdlbmVyaWNVREZPUEVxdWFsT3JHcmVhdGVyVGhhbgEAAQIBPr0BRVFVQUwgT1IgR1JFQVRFUiBUSEHOAQUBAAEOwgFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnVkZi5nZW5lcmljLkdlbmVyaWNVREZPUEVxdWFsT3JMZXNzVGhhbgEAIQE8vQFFUVVBTCBPUiBMRVNTIFRIQc4BBQEAAQUBAAABAw0BAQEBAAECAQIUAQcBAQgBAVJTXzdfYl9rZXlfYmxvb21fZmlsdGXyAQ/WAW9yZy5hcGFjaGUuaGFkb29wLmhpdmUuc2VyZGUyLm9iamVjdGluc3BlY3Rvci5wcmltaXRpdmUuV3JpdGFibGVCaW5hcnlPYmplY3RJbnNwZWN0b3IBAQMBAWJpbmFy+QEDMAEDMAEQb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGSW5CbG9vbUZpbHRl8gEAAQMNARFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnVkZi5nZW5lcmljLkdlbmVyaWNVREZPUEFu5AEBBQEAAAEDDQERAQEFAQAAAQMN (desc) 2018-01-05T11:29:03,899 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 0,1 2018-01-05T11:29:03,899 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = key,value 2018-01-05T11:29:03,899 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src 2018-01-05T11:29:03,899 DEBUG [InputInitializer {Map 1} #0] io.HiveInputFormat: Pushdown initiated with filterText = (key is not null and (key BETWEEN DynamicValue(RS_7_b_key_min) AND DynamicValue(RS_7_b_key_max) and in_bloom_filter(key, DynamicValue(RS_7_b_key_bloom_filter)))), filterExpr = GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_7_b_key_min), DynamicValue(RS_7_b_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_7_b_key_bloom_filter)))), serializedFilterExpr = AQEAamF2YS51dGlsLkFycmF5TGlz9AECAQFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVHZW5lcmljRnVuY0Rlc+MBAQABAQECb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC5wbGFuLkV4cHJOb2RlQ29sdW1uRGVz4wEBa2X5AAABgmEBA29yZy5hcGFjaGUuaGFkb29wLmhpdmUuc2VyZGUyLnR5cGVpbmZvLlByaW1pdGl2ZVR5cGVJbmbvAQFzdHJpbucBBG9yZy5hcGFjaGUuaGFkb29wLmhpdmUucWwudWRmLmdlbmVyaWMuR2VuZXJpY1VERk9QTm90TnVs7AEBBW9yZy5hcGFjaGUuaGFkb29wLmlvLkJvb2xlYW5Xcml0YWJs5QEAAAEDAQFib29sZWHuAQEBAQABAgEBAQEAAQQBBm9yZy5hcGFjaGUuaGFkb29wLmhpdmUucWwucGxhbi5FeHByTm9kZUNvbnN0YW50RGVz4wEBAw0FAAECAQcAAAgBAwkBB29yZy5hcGFjaGUuaGFkb29wLmhpdmUucWwucGxhbi5FeHByTm9kZUR5bmFtaWNWYWx1ZURlc+MBAQhvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRHluYW1pY1ZhbHXlAQFSU183X2Jfa2V5X21p7gEJ1gFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnNlcmRlMi5vYmplY3RpbnNwZWN0b3IucHJpbWl0aXZlLldyaXRhYmxlU3RyaW5nT2JqZWN0SW5zcGVjdG9yAQEDCQEDCQEDCQEHAQEIAQFSU183X2Jfa2V5X21h+AEJGAEDCQEDCQEKb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGQmV0d2Vl7gEBC8EBW0xvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnNlcmRlMi5vYmplY3RpbnNwZWN0b3IuT2JqZWN0SW5zcGVjdG9yOwEFAQzfAW9yZy5hcGFjaGUuaGFkb29wLmhpdmUuc2VyZGUyLm9iamVjdGluc3BlY3Rvci5wcmltaXRpdmUuV3JpdGFibGVDb25zdGFudEJvb2xlYW5PYmplY3RJbnNwZWN0b3IBAQMNAQUBAAEJGAEJGAEJGAENxQFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnVkZi5nZW5lcmljLkdlbmVyaWNVREZPUEVxdWFsT3JHcmVhdGVyVGhhbgEAAQIBPr0BRVFVQUwgT1IgR1JFQVRFUiBUSEHOAQUBAAEOwgFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnVkZi5nZW5lcmljLkdlbmVyaWNVREZPUEVxdWFsT3JMZXNzVGhhbgEAIQE8vQFFUVVBTCBPUiBMRVNTIFRIQc4BBQEAAQUBAAABAw0BAQEBAAECAQIUAQcBAQgBAVJTXzdfYl9rZXlfYmxvb21fZmlsdGXyAQ/WAW9yZy5hcGFjaGUuaGFkb29wLmhpdmUuc2VyZGUyLm9iamVjdGluc3BlY3Rvci5wcmltaXRpdmUuV3JpdGFibGVCaW5hcnlPYmplY3RJbnNwZWN0b3IBAQMBAWJpbmFy+QEDMAEDMAEQb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGSW5CbG9vbUZpbHRl8gEAAQMNARFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnVkZi5nZW5lcmljLkdlbmVyaWNVREZPUEFu5AEBBQEAAAEDDQERAQEFAQAAAQMN (desc) 2018-01-05T11:29:03,902 DEBUG [InputInitializer {Map 3} #0] log.PerfLogger: 2018-01-05T11:29:03,902 INFO [InputInitializer {Map 3} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 3 size: 5.42KB 2018-01-05T11:29:03,902 DEBUG [InputInitializer {Map 3} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1/kv3.txt to src new group? true 2018-01-05T11:29:03,902 INFO [InputInitializer {Map 3} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:29:03,902 INFO [InputInitializer {Map 3} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:29:03,902 INFO [InputInitializer {Map 3} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:29:03,902 INFO [InputInitializer {Map 3} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 4 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 216 Original splits: 1 2018-01-05T11:29:03,902 INFO [InputInitializer {Map 3} #0] grouper.TezSplitGrouper: Using original number of splits: 1 desired splits: 1 2018-01-05T11:29:03,902 INFO [InputInitializer {Map 3} #0] tez.SplitGrouper: Original split count is 1 grouped split count is 1, for bucket: 1 2018-01-05T11:29:03,902 INFO [InputInitializer {Map 3} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:29:03,903 INFO [InputInitializer {Map 3} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: b on vertex vertex_1515180518813_0001_34_00 [Map 3] 2018-01-05T11:29:03,904 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.05 maxFrac: 0.75 auto: false desiredTaskIput: 104857600 2018-01-05T11:29:03,904 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:29:03,904 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 1 tasks for vertex: vertex_1515180518813_0001_34_03 [Reducer 2] 2018-01-05T11:29:03,904 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_34_03 [Reducer 2] 2018-01-05T11:29:03,904 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_34_03, reconfigureDoneTime=1515180543904, numTasks=1, vertexLocationHint=null, edgeManagersCount=2, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:29:03,904 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_34_03 [Reducer 2], numAdditionalOutputs=1 2018-01-05T11:29:03,904 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:VERTEX_INITIALIZED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_34_03, initRequestedTime=1515180543898, initedTime=1515180543904, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:03,904 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_34_03 [Reducer 2] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:29:03,904 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_34_00 [Map 3] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:03,904 INFO [App Shared Pool - #0] impl.VertexImpl: Got updated RootInputsSpecs: {b=forAllWorkUnits=true, update=[1]} 2018-01-05T11:29:03,905 INFO [App Shared Pool - #0] impl.VertexImpl: Vertex vertex_1515180518813_0001_34_00 [Map 3] parallelism set to 1 2018-01-05T11:29:03,905 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_34_00 [Map 3] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:03,905 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:VERTEX_INITIALIZED]: vertexName=Map 3, vertexId=vertex_1515180518813_0001_34_00, initRequestedTime=1515180543833, initedTime=1515180543905, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:03,905 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_34_00 [Map 3] with distanceFromRoot: 0 2018-01-05T11:29:03,905 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_34_00 [Map 3] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:29:03,905 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_34_00, startRequestedTime=1515180543871, startedTime=1515180543905 2018-01-05T11:29:03,905 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_34_00, reconfigureDoneTime=1515180543905, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:29:03,905 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_34_00 [Map 3] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:03,905 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_34_00 for vertex: vertex_1515180518813_0001_34_01 [Reducer 4] numStartedSources: 1 numSources: 1 2018-01-05T11:29:03,905 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_34_01 [Reducer 4] with distanceFromRoot: 1 2018-01-05T11:29:03,905 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_34_00 for vertex: vertex_1515180518813_0001_34_03 [Reducer 2] numStartedSources: 1 numSources: 2 2018-01-05T11:29:03,905 INFO [Dispatcher thread {Central}] impl.VertexImpl: Cannot start vertex: vertex_1515180518813_0001_34_03 [Reducer 2] numStartedSources: 1 numSources: 2 2018-01-05T11:29:03,905 INFO [App Shared Pool - #0] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 3 2018-01-05T11:29:03,905 INFO [App Shared Pool - #0] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 3 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:29:03,905 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_34_01, startRequestedTime=1515180543905, startedTime=1515180543905 2018-01-05T11:29:03,905 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_34_00 [Map 3] 2018-01-05T11:29:03,905 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_34_01 [Reducer 4] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:03,905 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_34_01 for vertex: vertex_1515180518813_0001_34_02 [Map 1] numStartedSources: 1 numSources: 1 2018-01-05T11:29:03,905 INFO [Dispatcher thread {Central}] impl.VertexImpl: Cannot start vertex. Not in inited state. vertex_1515180518813_0001_34_02 [Map 1] . VertesState: INITIALIZING numTasks: -1 Num uninitialized edges: 0 2018-01-05T11:29:03,905 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:TASK_STARTED]: vertexName=Map 3, taskId=task_1515180518813_0001_34_00_000000, scheduledTime=1515180543905, launchTime=1515180543905 2018-01-05T11:29:03,905 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: OnVertexStarted vertex: Reducer 4 with 0 source tasks and 1 pending tasks 2018-01-05T11:29:03,905 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_34_00 [Map 3] 2018-01-05T11:29:03,905 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Map 3 in vertex: Reducer 4 numBipartiteSourceTasks: 1 2018-01-05T11:29:03,905 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: Scheduling 1 tasks for vertex: Reducer 4 with totalTasks: 1. 0 source tasks completed out of 1. MinSourceTaskCompletedFraction: 0.0 min: 0.0 max: 0.0 2018-01-05T11:29:03,905 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_34_00_000000_0, priority=11, capability=, hosts=null 2018-01-05T11:29:03,905 INFO [App Shared Pool - #2] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Map 3, destinationVertexName=Reducer 4. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.UnorderedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:29:03,906 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=11,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:03,906 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=11,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:03,906 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:03,906 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_34_00_000000_0 on hosts=null 2018-01-05T11:29:03,906 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:TASK_STARTED]: vertexName=Reducer 4, taskId=task_1515180518813_0001_34_01_000000, scheduledTime=1515180543906, launchTime=1515180543906 2018-01-05T11:29:03,906 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_34_00_000000_0 on hosts=null 2018-01-05T11:29:03,906 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_34_00_000000_0 2018-01-05T11:29:03,906 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:03,906 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_34_01 [Reducer 4] 2018-01-05T11:29:03,906 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #67, task=TaskInfo{task=attempt_1515180518813_0001_34_00_000000_0, priority=11, startTime=0, containerId=null, uniqueId=66, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000067 2018-01-05T11:29:03,906 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_34_00_000000_0; false 2018-01-05T11:29:03,906 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_34_00_000000_0, priority=11, startTime=311564415, containerId=container_111101111_0001_00_000067assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=66, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:03,906 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_34_01_000000_0, priority=26, capability=, hosts=null 2018-01-05T11:29:03,906 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=26,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:03,906 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=26,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:03,906 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:03,906 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_34_01_000000_0 on hosts=null 2018-01-05T11:29:03,906 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_34_01_000000_0 on hosts=null 2018-01-05T11:29:03,906 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_34_01_000000_0 2018-01-05T11:29:03,906 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:03,906 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000067 for node: 10.22.8.182:49818 2018-01-05T11:29:03,906 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #68, task=TaskInfo{task=attempt_1515180518813_0001_34_01_000000_0, priority=26, startTime=0, containerId=null, uniqueId=67, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, to container=container_111101111_0001_00_000068 2018-01-05T11:29:03,906 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_34_01_000000_0; false 2018-01-05T11:29:03,906 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_34_01_000000_0, priority=26, startTime=311564415, containerId=container_111101111_0001_00_000068assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=2, ac=6, commF=false, disabled=false}, uniqueId=67, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:03,909 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 34, name=select * from src a join src1 b on a...b.key (Stage-1), queryId=djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d 2018-01-05T11:29:03,909 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000067, attempt_1515180518813_0001_34_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:03,909 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000067 for node: 10.22.8.182:49818 2018-01-05T11:29:03,909 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000067 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:03,909 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000067, launchTime=1515180543909 2018-01-05T11:29:03,909 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000068 for node: 10.22.8.182:49818 2018-01-05T11:29:03,910 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000068, attempt_1515180518813_0001_34_01_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:03,910 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000068 for node: 10.22.8.182:49818 2018-01-05T11:29:03,910 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_34_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000067, dagName=select * from src a join src1 b on a...b.key (Stage-1), vertexName=Map 3, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=2, numGroupedInputs=0, Inputs={{b,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{Reducer 4,org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput,1}{Reducer 2,org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput,1}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180543833, firstAttemptStartTime=1515180543905, currentAttemptStartTime=1515180543909} 2018-01-05T11:29:03,910 INFO [IPC Server handler 4 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000067, QueryId=djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d-34, DagId=34, VertexName=Map 3, TaskId=0, TaskAttemptId=0, SubmitTime=1515180543910 2018-01-05T11:29:03,910 DEBUG [IPC Server handler 4 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=34} with the ShuffleHandler 2018-01-05T11:29:03,911 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_34_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000067] on NM: [10.22.8.182:49818] 2018-01-05T11:29:03,911 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:03,911 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:03,911 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 3, taskAttemptId=attempt_1515180518813_0001_34_00_000000_0, startTime=1515180543910, containerId=container_111101111_0001_00_000067, nodeId=10.22.8.182:49818 2018-01-05T11:29:03,911 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000068 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:03,911 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000068, launchTime=1515180543911 2018-01-05T11:29:03,911 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_34_01_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000068] on NM: [10.22.8.182:49818] 2018-01-05T11:29:03,912 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:03,912 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:03,912 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 4, taskAttemptId=attempt_1515180518813_0001_34_01_000000_0, startTime=1515180543911, containerId=container_111101111_0001_00_000068, nodeId=10.22.8.182:49818 2018-01-05T11:29:03,918 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Time taken to get FileStatuses: 18 2018-01-05T11:29:03,918 INFO [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:29:03,918 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 19 2018-01-05T11:29:03,919 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 1 2018-01-05T11:29:03,919 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:03,919 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 1. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:29:03,919 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir/e0b5a45c-1c80-452b-acaf-ad6f51dd68d8/map.xml 2018-01-05T11:29:03,919 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir/e0b5a45c-1c80-452b-acaf-ad6f51dd68d8/map.xml 2018-01-05T11:29:03,919 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir/e0b5a45c-1c80-452b-acaf-ad6f51dd68d8/map.xml 2018-01-05T11:29:03,919 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:03,919 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:03,919 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:03,926 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:03,926 INFO [IPC Server handler 4 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:29:03,926 DEBUG [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/34] 2018-01-05T11:29:03,926 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 5.34KB 2018-01-05T11:29:03,926 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src/kv1.txt to src new group? true 2018-01-05T11:29:03,926 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:29:03,926 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:29:03,926 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:29:03,926 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 107 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 5812 Original splits: 1 2018-01-05T11:29:03,926 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Using original number of splits: 1 desired splits: 1 2018-01-05T11:29:03,926 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 1 grouped split count is 1, for bucket: 1 2018-01-05T11:29:03,926 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:29:03,926 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:29:03,926 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_34_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:03,926 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_34_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180543905, dagStartTime=1515180543833, withinDagPriority=11, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:29:03,926 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_34_00_000000_0 : ACCEPTED 2018-01-05T11:29:03,926 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_34_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:29:03,926 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: a on vertex vertex_1515180518813_0001_34_02 [Map 1] 2018-01-05T11:29:03,926 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_34_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180543905, dagStartTime=1515180543833, withinDagPriority=11, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:29:03,926 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_34_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:03,926 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_34_00_000000_0 to pre-emption queue 2018-01-05T11:29:03,926 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_34_00_000000_0: true 2018-01-05T11:29:03,926 INFO [App Shared Pool - #2] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_34_02 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:03,928 INFO [TaskCommunicator # 9] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_34_00_000000_0 2018-01-05T11:29:03,928 INFO [App Shared Pool - #2] impl.VertexImpl: Got updated RootInputsSpecs: {a=forAllWorkUnits=true, update=[1]} 2018-01-05T11:29:03,928 INFO [TaskCommunicator # 9] GuaranteedTasks: Received response for attempt_1515180518813_0001_34_00_000000_0, true 2018-01-05T11:29:03,928 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:797465925 2018-01-05T11:29:03,928 INFO [App Shared Pool - #2] impl.VertexImpl: Vertex vertex_1515180518813_0001_34_02 [Map 1] parallelism set to 1 2018-01-05T11:29:03,928 INFO [App Shared Pool - #2] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_34_02 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:03,928 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_34_02, initRequestedTime=1515180543872, initedTime=1515180543928, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:03,928 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_34_02 [Map 1] with distanceFromRoot: 2 2018-01-05T11:29:03,928 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_34_02 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:29:03,928 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_34_02, startRequestedTime=1515180543905, startedTime=1515180543928 2018-01-05T11:29:03,928 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_34_02, reconfigureDoneTime=1515180543928, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=1, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:29:03,928 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_34_02 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:03,928 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_34_02 for vertex: vertex_1515180518813_0001_34_03 [Reducer 2] numStartedSources: 2 numSources: 2 2018-01-05T11:29:03,929 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_34_03 [Reducer 2] with distanceFromRoot: 3 2018-01-05T11:29:03,929 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_34_03, startRequestedTime=1515180543928, startedTime=1515180543929 2018-01-05T11:29:03,929 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_34_03 [Reducer 2] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:03,929 INFO [App Shared Pool - #2] impl.RootInputVertexManager: Task count in Reducer 4: 1 2018-01-05T11:29:03,929 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : select * from src a join src1 b on a...b.key (Stage-1), VertexName: Map 3, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_34_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=2, inputSpecList=[{{ sourceVertexName=b, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=Reducer 4, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput }}, {{ destinationVertexName=Reducer 2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ] 2018-01-05T11:29:03,929 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: OnVertexStarted vertex: Reducer 2 with 0 source tasks and 1 pending tasks 2018-01-05T11:29:03,929 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=2, JVM.maxFree=1031274496, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:03,929 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_34_00_000000_0 vertexName: Map 3 2018-01-05T11:29:03,929 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Map 3 in vertex: Reducer 2 numBipartiteSourceTasks: 1 2018-01-05T11:29:03,929 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_34_01_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000068, dagName=select * from src a join src1 b on a...b.key (Stage-1), vertexName=Reducer 4, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 3,org.apache.tez.runtime.library.input.UnorderedKVInput,1}}, Outputs={{Map 1,org.apache.tez.runtime.library.output.UnorderedKVOutput,1}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=2, completedTaskCount=0, dagStartTime=1515180543833, firstAttemptStartTime=1515180543906, currentAttemptStartTime=1515180543910} 2018-01-05T11:29:03,930 INFO [IPC Server handler 3 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000068, QueryId=djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d-34, DagId=34, VertexName=Reducer 4, TaskId=0, TaskAttemptId=0, SubmitTime=1515180543930 2018-01-05T11:29:03,929 INFO [App Shared Pool - #1] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Map 1 in vertex: Reducer 2 numBipartiteSourceTasks: 2 2018-01-05T11:29:03,930 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000067, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_34_00_000000_0, eventCount=1 } 2018-01-05T11:29:03,930 DEBUG [IPC Server handler 3 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=34} with the ShuffleHandler 2018-01-05T11:29:03,930 DEBUG [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/34] 2018-01-05T11:29:03,930 DEBUG [TezTR-518813_1_34_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_34 queryId: djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d instanceType: TaskRunner2Callable 2018-01-05T11:29:03,930 INFO [TezTR-518813_1_34_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_34_00_000000_0 2018-01-05T11:29:03,930 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=3, runningFragmentCount=1 2018-01-05T11:29:03,930 DEBUG [IPC Server handler 3 on 49818] impl.QueryFragmentInfo: Cannot finish due to source: Map 3 2018-01-05T11:29:03,930 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_34_01_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:03,930 DEBUG [IPC Server handler 3 on 49818] impl.QueryFragmentInfo: Cannot finish due to source: Map 3 2018-01-05T11:29:03,931 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:03,931 DEBUG [TezTR-518813_1_34_0_0_0] log.PerfLogger: 2018-01-05T11:29:03,931 INFO [App Shared Pool - #2] impl.RootInputVertexManager: Received configured notification : CONFIGURED for vertex: Reducer 4 in vertex: Map 1 numjourceTasks: 1 2018-01-05T11:29:03,931 INFO [App Shared Pool - #2] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 1. MinSourceTaskCompletedFraction: 0.0 min: 0.0 max: 0.0 2018-01-05T11:29:03,931 INFO [I/O Setup 1 Initialize: {Reducer 4}] writers.UnorderedPartitionedKVWriter: Requested BufferSize (tez.runtime.unordered.output.buffer.size-mb) : 100 2018-01-05T11:29:03,930 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_34_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=false, canFinish(in queue)=false, isGuaranteed=false, firstAttemptStartTime=1515180543906, dagStartTime=1515180543833, withinDagPriority=26, vertexParallelism= 1, selfAndUpstreamParallelism= 2, selfAndUpstreamComplete= 0}] 2018-01-05T11:29:03,932 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_34_01_000000_0 : ACCEPTED 2018-01-05T11:29:03,932 INFO [App Shared Pool - #2] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Reducer 4, destinationVertexName=Map 1. EdgeProperty: { BROADCAST : org.apache.tez.runtime.library.input.UnorderedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.UnorderedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:29:03,932 INFO [App Shared Pool - #2] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_34_02 [Map 1] 2018-01-05T11:29:03,932 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:03,932 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_34_02_000000, scheduledTime=1515180543932, launchTime=1515180543932 2018-01-05T11:29:03,932 INFO [TaskCommunicator # 0] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_34_01_000000_0 2018-01-05T11:29:03,932 INFO [TaskCommunicator # 0] GuaranteedTasks: Received response for attempt_1515180518813_0001_34_01_000000_0, true 2018-01-05T11:29:03,932 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_34_02 [Map 1] 2018-01-05T11:29:03,933 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_34_02_000000_0, priority=41, capability=, hosts=null 2018-01-05T11:29:03,933 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=41,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:03,933 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=41,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:03,933 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:03,933 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_34_02_000000_0 on hosts=null 2018-01-05T11:29:03,933 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_34_02_000000_0 on hosts=null 2018-01-05T11:29:03,933 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_34_02_000000_0 2018-01-05T11:29:03,933 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=2, ac=6, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:03,933 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #69, task=TaskInfo{task=attempt_1515180518813_0001_34_02_000000_0, priority=41, startTime=0, containerId=null, uniqueId=68, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=2, ac=6, commF=false, disabled=false}, to container=container_111101111_0001_00_000069 2018-01-05T11:29:03,933 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_34_02_000000_0; false 2018-01-05T11:29:03,933 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_34_02_000000_0, priority=41, startTime=311564442, containerId=container_111101111_0001_00_000069assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=3, ac=5, commF=false, disabled=false}, uniqueId=68, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:03,933 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000069 for node: 10.22.8.182:49818 2018-01-05T11:29:03,934 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000069, attempt_1515180518813_0001_34_02_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:03,934 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000069 for node: 10.22.8.182:49818 2018-01-05T11:29:03,932 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_34_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=false, canFinish(in queue)=false, isGuaranteed=false, firstAttemptStartTime=1515180543906, dagStartTime=1515180543833, withinDagPriority=26, vertexParallelism= 1, selfAndUpstreamParallelism= 2, selfAndUpstreamComplete= 0} 2018-01-05T11:29:03,934 DEBUG [Wait-Queue-Scheduler-0] impl.QueryFragmentInfo: Cannot finish due to source: Map 3 2018-01-05T11:29:03,934 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_34_01_000000_0 scheduled for execution. canFinish=false, isGuaranteed=false 2018-01-05T11:29:03,934 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_34_01_000000_0 to pre-emption queue 2018-01-05T11:29:03,934 DEBUG [Task-Executor-3] impl.QueryFragmentInfo: Cannot finish due to source: Map 3 2018-01-05T11:29:03,934 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_34_01_000000_0: false 2018-01-05T11:29:03,934 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: taskOwner hashCode:1897247494 2018-01-05T11:29:03,934 INFO [IPC Server handler 24 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_34_00_000000_0 sent events: (0-1). 2018-01-05T11:29:03,934 INFO [I/O Setup 0 Initialize: {b}] input.MRInput: b using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:29:03,934 INFO [I/O Setup 0 Initialize: {b}] input.MRInputLegacy: b MRInputLegacy deferring initialization 2018-01-05T11:29:03,934 DEBUG [TezTR-518813_1_34_0_0_0] log.PerfLogger: 2018-01-05T11:29:03,934 INFO [TezTR-518813_1_34_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:03,934 INFO [TezTR-518813_1_34_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 3 initializers to finish 2018-01-05T11:29:03,934 INFO [TezTR-518813_1_34_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:03,934 INFO [TezTR-518813_1_34_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:03,934 INFO [TezTR-518813_1_34_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:03,934 INFO [TezTR-518813_1_34_0_0_0] resources.MemoryDistributor: InitialRequests=[b:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [Reducer 4:OUTPUT:104857600:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput], [Reducer 2:OUTPUT:10485760:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput] 2018-01-05T11:29:03,934 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000069 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:03,934 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000069, launchTime=1515180543934 2018-01-05T11:29:03,934 INFO [TezTR-518813_1_34_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:03,935 INFO [TezTR-518813_1_34_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.045, finalReserveFractionUsed=0.345 2018-01-05T11:29:03,935 INFO [TezTR-518813_1_34_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 3, numScaledRequests: 14, TotalRequested: 115343360, TotalRequestedScaled: 1.6477622857142856E7, TotalJVMHeap: 1031274496, TotalAvailable: 675484794, TotalRequested/TotalJVMHeap:0.11 2018-01-05T11:29:03,935 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:29:03,935 INFO [TezTR-518813_1_34_0_0_0] resources.MemoryDistributor: Allocations=[b:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [Reducer 4:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput:OUTPUT:104857600:104857600], [Reducer 2:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:10485760:10485760] 2018-01-05T11:29:03,935 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_34_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:29:03,935 INFO [TezTR-518813_1_34_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:03,935 INFO [TezTR-518813_1_34_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: b being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:03,935 INFO [Task-Executor-3] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : select * from src a join src1 b on a...b.key (Stage-1), VertexName: Reducer 4, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_34_01_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 3, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }}, ], outputSpecList=[{{ destinationVertexName=Map 1, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.UnorderedKVOutput }}, ] 2018-01-05T11:29:03,935 INFO [TezTR-518813_1_34_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:29:03,935 INFO [TezTR-518813_1_34_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:03,935 INFO [Task-Executor-3] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1031274496, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:03,935 INFO [I/O Setup 1 Start: {b}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: b 2018-01-05T11:29:03,935 INFO [Task-Executor-3] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_34_01_000000_0 vertexName: Reducer 4 2018-01-05T11:29:03,935 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_34_02_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000069, dagName=select * from src a join src1 b on a...b.key (Stage-1), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=2, numOutputs=1, numGroupedInputs=0, Inputs={{a,org.apache.tez.mapreduce.input.MRInputLegacy,1}{Reducer 4,org.apache.tez.runtime.library.input.UnorderedKVInput,1}}, Outputs={{Reducer 2,org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput,1}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=3, completedTaskCount=0, dagStartTime=1515180543833, firstAttemptStartTime=1515180543932, currentAttemptStartTime=1515180543934} 2018-01-05T11:29:03,935 INFO [TezTR-518813_1_34_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:03,935 INFO [IPC Server handler 1 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000069, QueryId=djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d-34, DagId=34, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180543935 2018-01-05T11:29:03,935 DEBUG [IPC Server handler 1 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=34} with the ShuffleHandler 2018-01-05T11:29:03,935 DEBUG [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/34] 2018-01-05T11:29:03,935 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000068, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_34_01_000000_0, eventCount=1 } 2018-01-05T11:29:03,935 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_34_02_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000069] on NM: [10.22.8.182:49818] 2018-01-05T11:29:03,935 DEBUG [TezTR-518813_1_34_1_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_34 queryId: djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d instanceType: TaskRunner2Callable 2018-01-05T11:29:03,935 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:03,935 INFO [TezTR-518813_1_34_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_34_00_000000_0 2018-01-05T11:29:03,935 DEBUG [TezTR-518813_1_34_0_0_0] log.PerfLogger: 2018-01-05T11:29:03,935 INFO [TezTR-518813_1_34_1_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_34_01_000000_0 2018-01-05T11:29:03,935 DEBUG [TezTR-518813_1_34_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_34_00_000000_0_10199 2018-01-05T11:29:03,935 DEBUG [TezTR-518813_1_34_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:03,935 DEBUG [TezTR-518813_1_34_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_34_00_000000_0 2018-01-05T11:29:03,935 INFO [TezTR-518813_1_34_0_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d 2018-01-05T11:29:03,935 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:03,936 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=2, runningFragmentCount=2 2018-01-05T11:29:03,936 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_34_02_000000_0, startTime=1515180543934, containerId=container_111101111_0001_00_000069, nodeId=10.22.8.182:49818 2018-01-05T11:29:03,936 DEBUG [IPC Server handler 1 on 49818] impl.QueryFragmentInfo: Cannot finish due to source: Reducer 4 2018-01-05T11:29:03,936 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_34_02_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:03,936 DEBUG [IPC Server handler 1 on 49818] impl.QueryFragmentInfo: Cannot finish due to source: Reducer 4 2018-01-05T11:29:03,936 DEBUG [TezTR-518813_1_34_1_0_0] log.PerfLogger: 2018-01-05T11:29:03,936 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_34_02_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=false, canFinish(in queue)=false, isGuaranteed=false, firstAttemptStartTime=1515180543932, dagStartTime=1515180543833, withinDagPriority=41, vertexParallelism= 1, selfAndUpstreamParallelism= 3, selfAndUpstreamComplete= 0}] 2018-01-05T11:29:03,936 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_34_02_000000_0 : ACCEPTED 2018-01-05T11:29:03,936 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:03,936 DEBUG [TezTR-518813_1_34_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:03,936 DEBUG [TezTR-518813_1_34_0_0_0] log.PerfLogger: 2018-01-05T11:29:03,936 DEBUG [TezTR-518813_1_34_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:03,937 DEBUG [TezTR-518813_1_34_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:03,937 DEBUG [TezTR-518813_1_34_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d_Map 3__MAP_PLAN__ 2018-01-05T11:29:03,937 INFO [TaskCommunicator # 1] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_34_02_000000_0 2018-01-05T11:29:03,936 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_34_02_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=false, canFinish(in queue)=false, isGuaranteed=false, firstAttemptStartTime=1515180543932, dagStartTime=1515180543833, withinDagPriority=41, vertexParallelism= 1, selfAndUpstreamParallelism= 3, selfAndUpstreamComplete= 0} 2018-01-05T11:29:03,937 DEBUG [Wait-Queue-Scheduler-0] impl.QueryFragmentInfo: Cannot finish due to source: Reducer 4 2018-01-05T11:29:03,937 INFO [TezTR-518813_1_34_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir/0d27c3bb-5a71-454d-ba1d-9144cc0bd13d/map.xml 2018-01-05T11:29:03,937 DEBUG [TezTR-518813_1_34_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir/0d27c3bb-5a71-454d-ba1d-9144cc0bd13d/map.xml 2018-01-05T11:29:03,937 DEBUG [TezTR-518813_1_34_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir/0d27c3bb-5a71-454d-ba1d-9144cc0bd13d/map.xml 2018-01-05T11:29:03,937 INFO [TaskCommunicator # 1] GuaranteedTasks: Received response for attempt_1515180518813_0001_34_02_000000_0, true 2018-01-05T11:29:03,937 DEBUG [TezTR-518813_1_34_0_0_0] log.PerfLogger: 2018-01-05T11:29:03,937 INFO [TezTR-518813_1_34_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:03,937 DEBUG [TezTR-518813_1_34_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:03,937 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_34_02_000000_0 scheduled for execution. canFinish=false, isGuaranteed=false 2018-01-05T11:29:03,937 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_34_02_000000_0 to pre-emption queue 2018-01-05T11:29:03,937 DEBUG [Task-Executor-1] impl.QueryFragmentInfo: Cannot finish due to source: Reducer 4 2018-01-05T11:29:03,937 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_34_02_000000_0: false 2018-01-05T11:29:03,938 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: taskOwner hashCode:2074771863 2018-01-05T11:29:03,938 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:03,940 INFO [Task-Executor-1] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : select * from src a join src1 b on a...b.key (Stage-1), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_34_02_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=a, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, {{ sourceVertexName=Reducer 4, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }}, ], outputSpecList=[{{ destinationVertexName=Reducer 2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ] 2018-01-05T11:29:03,940 INFO [Task-Executor-1] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=2, numOutputs=1, JVM.maxFree=1031274496, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:03,940 INFO [Task-Executor-1] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_34_02_000000_0 vertexName: Map 1 2018-01-05T11:29:03,941 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000069, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_34_02_000000_0, eventCount=1 } 2018-01-05T11:29:03,941 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:03,941 DEBUG [TezTR-518813_1_34_2_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_34 queryId: djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d instanceType: TaskRunner2Callable 2018-01-05T11:29:03,941 INFO [TezTR-518813_1_34_2_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_34_02_000000_0 2018-01-05T11:29:03,942 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:03,942 DEBUG [TezTR-518813_1_34_2_0_0] log.PerfLogger: 2018-01-05T11:29:03,942 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:03,942 DEBUG [TezTR-518813_1_34_1_0_0] log.PerfLogger: 2018-01-05T11:29:03,942 INFO [TezTR-518813_1_34_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:03,942 INFO [TezTR-518813_1_34_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:03,942 INFO [TezTR-518813_1_34_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:03,942 INFO [TezTR-518813_1_34_1_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:03,942 INFO [TezTR-518813_1_34_1_0_0] resources.MemoryDistributor: InitialRequests=[Map 3:INPUT:131404920:org.apache.tez.runtime.library.input.UnorderedKVInput], [Map 1:OUTPUT:0:org.apache.tez.runtime.library.output.UnorderedKVOutput] 2018-01-05T11:29:03,942 INFO [TezTR-518813_1_34_1_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:03,942 INFO [TezTR-518813_1_34_1_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:03,942 INFO [TezTR-518813_1_34_1_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 131404920, TotalRequestedScaled: 6.570246E7, TotalJVMHeap: 1031274496, TotalAvailable: 690953912, TotalRequested/TotalJVMHeap:0.13 2018-01-05T11:29:03,943 INFO [TezTR-518813_1_34_1_0_0] resources.MemoryDistributor: Allocations=[Map 3:org.apache.tez.runtime.library.input.UnorderedKVInput:INPUT:131404920:131404920], [Map 1:org.apache.tez.runtime.library.output.UnorderedKVOutput:OUTPUT:0:0] 2018-01-05T11:29:03,943 INFO [TezTR-518813_1_34_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:03,943 INFO [TezTR-518813_1_34_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 0 2018-01-05T11:29:03,943 INFO [TezTR-518813_1_34_1_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:03,943 INFO [TezTR-518813_1_34_1_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_34_01_000000_0 2018-01-05T11:29:03,943 DEBUG [TezTR-518813_1_34_1_0_0] log.PerfLogger: 2018-01-05T11:29:03,943 DEBUG [TezTR-518813_1_34_1_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_34_01_000000_0_10203 2018-01-05T11:29:03,943 DEBUG [TezTR-518813_1_34_1_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:03,943 DEBUG [TezTR-518813_1_34_1_0_0] mr.ObjectCache: Creating djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d_Reducer 4__REDUCE_PLAN__ 2018-01-05T11:29:03,943 INFO [TezTR-518813_1_34_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir/0945123a-e0af-4773-8b55-9afb14888ffa/reduce.xml 2018-01-05T11:29:03,943 DEBUG [TezTR-518813_1_34_1_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir/0945123a-e0af-4773-8b55-9afb14888ffa/reduce.xml 2018-01-05T11:29:03,943 DEBUG [TezTR-518813_1_34_1_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir/0945123a-e0af-4773-8b55-9afb14888ffa/reduce.xml 2018-01-05T11:29:03,943 DEBUG [TezTR-518813_1_34_1_0_0] log.PerfLogger: 2018-01-05T11:29:03,943 INFO [TezTR-518813_1_34_1_0_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:29:03,943 DEBUG [TezTR-518813_1_34_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:03,944 DEBUG [TezTR-518813_1_34_0_0_0] log.PerfLogger: 2018-01-05T11:29:03,944 INFO [TezTR-518813_1_34_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 3 size: 5.42KB 2018-01-05T11:29:03,945 DEBUG [TezTR-518813_1_34_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:03,945 INFO [TezTR-518813_1_34_0_0_0] tez.MapRecordProcessor: The input names are: [b] 2018-01-05T11:29:03,945 INFO [TezTR-518813_1_34_0_0_0] input.MRInput: Initialized MRInput: b 2018-01-05T11:29:03,945 INFO [I/O Setup 0 Initialize: {a}] input.MRInput: a using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:29:03,945 INFO [I/O Setup 0 Initialize: {a}] input.MRInputLegacy: a MRInputLegacy deferring initialization 2018-01-05T11:29:03,945 DEBUG [TezTR-518813_1_34_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 1 2018-01-05T11:29:03,945 INFO [TezTR-518813_1_34_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir/0d27c3bb-5a71-454d-ba1d-9144cc0bd13d/map.xml 2018-01-05T11:29:03,945 DEBUG [TezTR-518813_1_34_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:03,945 INFO [IPC Server handler 8 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_34_02_000000_0 sent events: (0-1). 2018-01-05T11:29:03,945 DEBUG [TezTR-518813_1_34_2_0_0] log.PerfLogger: 2018-01-05T11:29:03,945 INFO [TezTR-518813_1_34_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:03,946 INFO [TezTR-518813_1_34_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 3 initializers to finish 2018-01-05T11:29:03,946 INFO [TezTR-518813_1_34_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:03,946 INFO [TezTR-518813_1_34_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:03,946 INFO [TezTR-518813_1_34_2_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:03,946 INFO [TezTR-518813_1_34_2_0_0] resources.MemoryDistributor: InitialRequests=[a:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [Reducer 4:INPUT:131404920:org.apache.tez.runtime.library.input.UnorderedKVInput], [Reducer 2:OUTPUT:10485760:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput] 2018-01-05T11:29:03,946 INFO [TezTR-518813_1_34_2_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:03,946 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:29:03,946 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_34_02_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:29:03,946 INFO [TezTR-518813_1_34_2_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.045, finalReserveFractionUsed=0.345 2018-01-05T11:29:03,946 INFO [TezTR-518813_1_34_2_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 3, numScaledRequests: 14, TotalRequested: 141890680, TotalRequestedScaled: 1.837386E7, TotalJVMHeap: 1031274496, TotalAvailable: 675484794, TotalRequested/TotalJVMHeap:0.14 2018-01-05T11:29:03,946 INFO [TezTR-518813_1_34_2_0_0] resources.MemoryDistributor: Allocations=[a:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [Reducer 4:org.apache.tez.runtime.library.input.UnorderedKVInput:INPUT:131404920:131404920], [Reducer 2:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:10485760:10485760] 2018-01-05T11:29:03,946 INFO [TezTR-518813_1_34_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:03,946 INFO [TezTR-518813_1_34_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: a being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:03,946 INFO [TezTR-518813_1_34_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:29:03,946 INFO [TezTR-518813_1_34_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:03,946 INFO [I/O Setup 1 Start: {a}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: a 2018-01-05T11:29:03,946 DEBUG [TezTR-518813_1_34_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1/kv3.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=25, numFiles=1, serialization.ddl=struct src1 { string key, string value}, transient_lastDdlTime=1515180522, rawDataSize=191, columns.comments=defaultdefault, totalSize=216, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.src1}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=25, numFiles=1, serialization.ddl=struct src1 { string key, string value}, transient_lastDdlTime=1515180522, rawDataSize=191, columns.comments=defaultdefault, totalSize=216, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.src1}, baseFileName=src1, vectorPartitionDesc=vector map operator read type VECTOR_DESERIALIZE, input file format class name org.apache.hadoop.mapred.TextInputFormat, deserialize type LAZY_SIMPLE] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=25, numFiles=1, serialization.ddl=struct src1 { string key, string value}, transient_lastDdlTime=1515180522, rawDataSize=191, columns.comments=defaultdefault, totalSize=216, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.src1}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=25, numFiles=1, serialization.ddl=struct src1 { string key, string value}, transient_lastDdlTime=1515180522, rawDataSize=191, columns.comments=defaultdefault, totalSize=216, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.src1}, baseFileName=src1, vectorPartitionDesc=vector map operator read type VECTOR_DESERIALIZE, input file format class name org.apache.hadoop.mapred.TextInputFormat, deserialize type LAZY_SIMPLE]} 2018-01-05T11:29:03,946 INFO [TezTR-518813_1_34_2_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:03,946 INFO [TezTR-518813_1_34_2_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_34_02_000000_0 2018-01-05T11:29:03,946 DEBUG [TezTR-518813_1_34_2_0_0] log.PerfLogger: 2018-01-05T11:29:03,946 DEBUG [TezTR-518813_1_34_2_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_34_02_000000_0_10206 2018-01-05T11:29:03,946 DEBUG [TezTR-518813_1_34_0_0_0] io.HiveInputFormat: Pushdown initiated with filterText = key is not null, filterExpr = GenericUDFOPNotNull(Column[key]), serializedFilterExpr = AQEAamF2YS51dGlsLkFycmF5TGlz9AEBAQFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVDb2x1bW5EZXPjAQFrZfkAAAGCYgECb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5zZXJkZTIudHlwZWluZm8uUHJpbWl0aXZlVHlwZUluZu8BAXN0cmlu5wEDb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGT1BOb3ROdWzsAQEEb3JnLmFwYWNoZS5oYWRvb3AuaW8uQm9vbGVhbldyaXRhYmzlAQAAAQIBAWJvb2xlYe4= (desc) 2018-01-05T11:29:03,946 DEBUG [TezTR-518813_1_34_2_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:03,946 DEBUG [TezTR-518813_1_34_2_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_34_02_000000_0 2018-01-05T11:29:03,946 INFO [TezTR-518813_1_34_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir/0d27c3bb-5a71-454d-ba1d-9144cc0bd13d/map.xml 2018-01-05T11:29:03,946 DEBUG [TezTR-518813_1_34_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:03,946 DEBUG [TezTR-518813_1_34_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:29:03,946 INFO [TezTR-518813_1_34_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:29:03,946 DEBUG [TezTR-518813_1_34_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:29:03,946 DEBUG [TezTR-518813_1_34_1_0_0] log.PerfLogger: 2018-01-05T11:29:03,946 INFO [TezTR-518813_1_34_0_0_0] io.HiveInputFormat: Resetting VectorMapOperatorReadType from VECTOR_DESERIALIZE for partition default.src1 {} 2018-01-05T11:29:03,946 INFO [TezTR-518813_1_34_1_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 4 size: 5.10KB 2018-01-05T11:29:03,946 DEBUG [TezTR-518813_1_34_2_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:03,946 DEBUG [TezTR-518813_1_34_2_0_0] log.PerfLogger: 2018-01-05T11:29:03,946 DEBUG [TezTR-518813_1_34_2_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:03,946 DEBUG [TezTR-518813_1_34_2_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:03,946 DEBUG [TezTR-518813_1_34_2_0_0] mr.ObjectCache: Creating djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d_Map 1__MAP_PLAN__ 2018-01-05T11:29:03,946 INFO [TezTR-518813_1_34_2_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir/e0b5a45c-1c80-452b-acaf-ad6f51dd68d8/map.xml 2018-01-05T11:29:03,947 DEBUG [TezTR-518813_1_34_2_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir/e0b5a45c-1c80-452b-acaf-ad6f51dd68d8/map.xml 2018-01-05T11:29:03,947 DEBUG [TezTR-518813_1_34_2_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir/e0b5a45c-1c80-452b-acaf-ad6f51dd68d8/map.xml 2018-01-05T11:29:03,947 DEBUG [TezTR-518813_1_34_1_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:03,947 DEBUG [TezTR-518813_1_34_1_0_0] log.PerfLogger: 2018-01-05T11:29:03,947 DEBUG [TezTR-518813_1_34_2_0_0] log.PerfLogger: 2018-01-05T11:29:03,947 DEBUG [TezTR-518813_1_34_1_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:03,947 INFO [TezTR-518813_1_34_2_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:03,947 DEBUG [TezTR-518813_1_34_1_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:03,947 DEBUG [TezTR-518813_1_34_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:03,947 DEBUG [TezTR-518813_1_34_1_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:03,948 DEBUG [TezTR-518813_1_34_0_0_0] impl.LlapRecordReader: Initializing for input Map 3 2018-01-05T11:29:03,948 INFO [TezTR-518813_1_34_1_0_0] impl.SimpleFetchedInputAllocator: Map_3: RequestedMemory=131404920, AssignedMemory=131404920, maxSingleShuffleLimit=32851230 2018-01-05T11:29:03,948 INFO [TezTR-518813_1_34_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir/0d27c3bb-5a71-454d-ba1d-9144cc0bd13d/map.xml 2018-01-05T11:29:03,948 DEBUG [TezTR-518813_1_34_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:03,948 DEBUG [TezTR-518813_1_34_1_0_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:03,948 DEBUG [TezTR-518813_1_34_1_0_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:03,948 DEBUG [TezTR-518813_1_34_1_0_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:03,948 DEBUG [TezTR-518813_1_34_1_0_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:03,948 DEBUG [TezTR-518813_1_34_2_0_0] log.PerfLogger: 2018-01-05T11:29:03,949 INFO [TezTR-518813_1_34_2_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 5.34KB 2018-01-05T11:29:03,949 DEBUG [TezTR-518813_1_34_2_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:03,949 INFO [TezTR-518813_1_34_2_0_0] tez.MapRecordProcessor: The input names are: [a, Reducer 4] 2018-01-05T11:29:03,949 INFO [TezTR-518813_1_34_2_0_0] input.MRInput: Initialized MRInput: a 2018-01-05T11:29:03,949 DEBUG [TezTR-518813_1_34_2_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 1 2018-01-05T11:29:03,949 INFO [TezTR-518813_1_34_2_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir/e0b5a45c-1c80-452b-acaf-ad6f51dd68d8/map.xml 2018-01-05T11:29:03,949 DEBUG [TezTR-518813_1_34_2_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:03,949 DEBUG [TezTR-518813_1_34_2_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src/kv1.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct src { string key, string value}, transient_lastDdlTime=1515180521, rawDataSize=5312, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.src}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct src { string key, string value}, transient_lastDdlTime=1515180521, rawDataSize=5312, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.src}, baseFileName=src, vectorPartitionDesc=vector map operator read type VECTOR_DESERIALIZE, input file format class name org.apache.hadoop.mapred.TextInputFormat, deserialize type LAZY_SIMPLE] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct src { string key, string value}, transient_lastDdlTime=1515180521, rawDataSize=5312, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.src}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=500, numFiles=1, serialization.ddl=struct src { string key, string value}, transient_lastDdlTime=1515180521, rawDataSize=5312, columns.comments=defaultdefault, totalSize=5812, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.src}, baseFileName=src, vectorPartitionDesc=vector map operator read type VECTOR_DESERIALIZE, input file format class name org.apache.hadoop.mapred.TextInputFormat, deserialize type LAZY_SIMPLE]} 2018-01-05T11:29:03,950 DEBUG [TezTR-518813_1_34_2_0_0] io.HiveInputFormat: Pushdown initiated with filterText = (key is not null and (key BETWEEN DynamicValue(RS_7_b_key_min) AND DynamicValue(RS_7_b_key_max) and in_bloom_filter(key, DynamicValue(RS_7_b_key_bloom_filter)))), filterExpr = GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_7_b_key_min), DynamicValue(RS_7_b_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_7_b_key_bloom_filter)))), serializedFilterExpr = AQEAamF2YS51dGlsLkFycmF5TGlz9AECAQFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVHZW5lcmljRnVuY0Rlc+MBAQABAQECb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC5wbGFuLkV4cHJOb2RlQ29sdW1uRGVz4wEBa2X5AAABgmEBA29yZy5hcGFjaGUuaGFkb29wLmhpdmUuc2VyZGUyLnR5cGVpbmZvLlByaW1pdGl2ZVR5cGVJbmbvAQFzdHJpbucBBG9yZy5hcGFjaGUuaGFkb29wLmhpdmUucWwudWRmLmdlbmVyaWMuR2VuZXJpY1VERk9QTm90TnVs7AEBBW9yZy5hcGFjaGUuaGFkb29wLmlvLkJvb2xlYW5Xcml0YWJs5QEAAAEDAQFib29sZWHuAQEBAQABAgEBAQEAAQQBBm9yZy5hcGFjaGUuaGFkb29wLmhpdmUucWwucGxhbi5FeHByTm9kZUNvbnN0YW50RGVz4wEBAw0FAAECAQcAAAgBAwkBB29yZy5hcGFjaGUuaGFkb29wLmhpdmUucWwucGxhbi5FeHByTm9kZUR5bmFtaWNWYWx1ZURlc+MBAQhvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRHluYW1pY1ZhbHXlAQFSU183X2Jfa2V5X21p7gEJ1gFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnNlcmRlMi5vYmplY3RpbnNwZWN0b3IucHJpbWl0aXZlLldyaXRhYmxlU3RyaW5nT2JqZWN0SW5zcGVjdG9yAQEDCQEDCQEDCQEHAQEIAQFSU183X2Jfa2V5X21h+AEJGAEDCQEDCQEKb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGQmV0d2Vl7gEBC8EBW0xvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnNlcmRlMi5vYmplY3RpbnNwZWN0b3IuT2JqZWN0SW5zcGVjdG9yOwEFAQzfAW9yZy5hcGFjaGUuaGFkb29wLmhpdmUuc2VyZGUyLm9iamVjdGluc3BlY3Rvci5wcmltaXRpdmUuV3JpdGFibGVDb25zdGFudEJvb2xlYW5PYmplY3RJbnNwZWN0b3IBAQMNAQUBAAEJGAEJGAEJGAENxQFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnVkZi5nZW5lcmljLkdlbmVyaWNVREZPUEVxdWFsT3JHcmVhdGVyVGhhbgEAAQIBPr0BRVFVQUwgT1IgR1JFQVRFUiBUSEHOAQUBAAEOwgFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnVkZi5nZW5lcmljLkdlbmVyaWNVREZPUEVxdWFsT3JMZXNzVGhhbgEAIQE8vQFFUVVBTCBPUiBMRVNTIFRIQc4BBQEAAQUBAAABAw0BAQEBAAECAQIUAQcBAQgBAVJTXzdfYl9rZXlfYmxvb21fZmlsdGXyAQ/WAW9yZy5hcGFjaGUuaGFkb29wLmhpdmUuc2VyZGUyLm9iamVjdGluc3BlY3Rvci5wcmltaXRpdmUuV3JpdGFibGVCaW5hcnlPYmplY3RJbnNwZWN0b3IBAQMBAWJpbmFy+QEDMAEDMAEQb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGSW5CbG9vbUZpbHRl8gEAAQMNARFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnVkZi5nZW5lcmljLkdlbmVyaWNVREZPUEFu5AEBBQEAAAEDDQERAQEFAQAAAQMN (desc) 2018-01-05T11:29:03,950 INFO [TezTR-518813_1_34_2_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir/e0b5a45c-1c80-452b-acaf-ad6f51dd68d8/map.xml 2018-01-05T11:29:03,950 DEBUG [TezTR-518813_1_34_2_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:03,950 DEBUG [TezTR-518813_1_34_2_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:29:03,950 INFO [TezTR-518813_1_34_2_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:29:03,950 DEBUG [TezTR-518813_1_34_2_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:29:03,950 INFO [TezTR-518813_1_34_2_0_0] io.HiveInputFormat: Resetting VectorMapOperatorReadType from VECTOR_DESERIALIZE for partition default.src {} 2018-01-05T11:29:03,951 INFO [TezTR-518813_1_34_1_0_0] impl.ShuffleManager: Map_3: numInputs=1, compressionCodec=NoCompressionCodec, numFetchers=1, ifileBufferSize=4096, ifileReadAheadEnabled=true, ifileReadAheadLength=4194304, localDiskFetchEnabled=false, sharedFetchEnabled=false, keepAlive=false, keepAliveMaxConnections=20, connectionTimeout=180000, readTimeout=180000, bufferSize=8192, bufferSize=8192, maxTaskOutputAtOnce=20 2018-01-05T11:29:03,951 INFO [TezTR-518813_1_34_1_0_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:29:03,951 DEBUG [TezTR-518813_1_34_2_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:29:03,951 INFO [TezTR-518813_1_34_2_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir/e0b5a45c-1c80-452b-acaf-ad6f51dd68d8/map.xml 2018-01-05T11:29:03,951 DEBUG [TezTR-518813_1_34_2_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:03,957 INFO [TezTR-518813_1_34_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_34_00_000000_0 2018-01-05T11:29:03,957 INFO [TezTR-518813_1_34_2_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_34_02_000000_0 2018-01-05T11:29:03,957 INFO [TezTR-518813_1_34_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [string, string] (isAcidRead false) 2018-01-05T11:29:03,957 INFO [TezTR-518813_1_34_2_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [string, string] (isAcidRead false) 2018-01-05T11:29:03,957 DEBUG [TezTR-518813_1_34_2_0_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:03,957 DEBUG [TezTR-518813_1_34_2_0_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:03,957 DEBUG [TezTR-518813_1_34_0_0_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:03,957 DEBUG [TezTR-518813_1_34_2_0_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:03,958 DEBUG [TezTR-518813_1_34_2_0_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:03,957 DEBUG [TezTR-518813_1_34_0_0_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:03,958 DEBUG [TezTR-518813_1_34_0_0_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:03,958 DEBUG [TezTR-518813_1_34_0_0_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:03,958 WARN [TezTR-518813_1_34_2_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src/kv1.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:29:03,958 WARN [TezTR-518813_1_34_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1/kv3.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:29:03,958 DEBUG [TezTR-518813_1_34_2_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:29:03,958 INFO [TezTR-518813_1_34_2_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir/e0b5a45c-1c80-452b-acaf-ad6f51dd68d8/map.xml 2018-01-05T11:29:03,958 DEBUG [TezTR-518813_1_34_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:29:03,958 DEBUG [TezTR-518813_1_34_2_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:03,958 INFO [TezTR-518813_1_34_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir/0d27c3bb-5a71-454d-ba1d-9144cc0bd13d/map.xml 2018-01-05T11:29:03,958 DEBUG [TezTR-518813_1_34_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:03,958 DEBUG [TezTR-518813_1_34_2_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src/kv1.txt 2018-01-05T11:29:03,958 INFO [TezTR-518813_1_34_2_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src/kv1.txt:0+5812], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=5812} 2018-01-05T11:29:03,958 DEBUG [TezTR-518813_1_34_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1/kv3.txt 2018-01-05T11:29:03,958 INFO [TezTR-518813_1_34_2_0_0] input.MRInput: a initialized RecordReader from event 2018-01-05T11:29:03,958 DEBUG [IO-Elevator-Thread-2] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_34 queryId: djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d instanceType: SerDeEncodedDataReader 2018-01-05T11:29:03,958 DEBUG [IO-Elevator-Thread-3] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_34 queryId: djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d instanceType: SerDeEncodedDataReader 2018-01-05T11:29:03,958 INFO [TezTR-518813_1_34_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1/kv3.txt:0+216], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=216} 2018-01-05T11:29:03,958 DEBUG [TezTR-518813_1_34_2_0_0] tez.MapRecordProcessor: Starting Output: Reducer 2 2018-01-05T11:29:03,958 INFO [TezTR-518813_1_34_0_0_0] input.MRInput: b initialized RecordReader from event 2018-01-05T11:29:03,959 DEBUG [TezTR-518813_1_34_0_0_0] tez.MapRecordProcessor: Starting Output: Reducer 4 2018-01-05T11:29:03,959 INFO [TezTR-518813_1_34_2_0_0] impl.ExternalSorter: Reducer 2 using: memoryMb=10, keySerializerClass=class org.apache.hadoop.hive.ql.io.HiveKey, valueSerializerClass=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization$TezBytesWritableSerializer@13ccf536, comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator@3d2a5495, partitioner=org.apache.tez.mapreduce.partition.MRPartitioner, serialization=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.hadoop.io.serializer.WritableSerialization, org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization, org.apache.hadoop.io.serializer.avro.AvroReflectSerialization, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:29:03,959 INFO [TezTR-518813_1_34_0_0_0] writers.BaseUnorderedPartitionedKVWriter: Instantiating Partitioner: [org.apache.tez.mapreduce.partition.MRPartitioner] 2018-01-05T11:29:03,959 INFO [TezTR-518813_1_34_2_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 2: , UsingHashComparator=true 2018-01-05T11:29:03,960 INFO [IO-Elevator-Thread-2] LlapIoImpl: Processing data for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src/kv1.txt 2018-01-05T11:29:03,960 INFO [IO-Elevator-Thread-3] LlapIoImpl: Processing data for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1/kv3.txt 2018-01-05T11:29:03,960 INFO [IO-Elevator-Thread-2] LlapIoImpl: Processing one split {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src/kv1.txt, 0, 5812} 2018-01-05T11:29:03,960 INFO [IO-Elevator-Thread-3] LlapIoImpl: Processing one split {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1/kv3.txt, 0, 216} 2018-01-05T11:29:03,961 INFO [TezTR-518813_1_34_2_0_0] impl.PipelinedSorter: Newly allocated block size=10485760, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=10485760, total=10485760 2018-01-05T11:29:03,961 INFO [TezTR-518813_1_34_2_0_0] impl.PipelinedSorter: Pre allocating rest of memory buffers upfront 2018-01-05T11:29:03,961 INFO [TezTR-518813_1_34_2_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 2: , UsingHashComparator=true#blocks=1, maxMemUsage=10485760, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=10485760, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=10 2018-01-05T11:29:03,961 INFO [TezTR-518813_1_34_2_0_0] impl.PipelinedSorter: Reducer 2: reserved.remaining()=10485760, reserved.metasize=5242880 2018-01-05T11:29:03,961 DEBUG [IO-Elevator-Thread-2] LlapIoImpl: Using LineRecordReader to read data 2018-01-05T11:29:03,961 DEBUG [IO-Elevator-Thread-3] LlapIoImpl: Using LineRecordReader to read data 2018-01-05T11:29:03,966 INFO [TezTR-518813_1_34_2_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:29:03,966 INFO [TezTR-518813_1_34_2_0_0] vector.VectorMapOperator: Initializing operator MAP[0] 2018-01-05T11:29:03,966 DEBUG [TezTR-518813_1_34_2_0_0] vector.VectorMapOperator: Initialization Done 0 MAP 2018-01-05T11:29:03,966 DEBUG [TezTR-518813_1_34_2_0_0] vector.VectorMapOperator: Operator 0 MAP initialized 2018-01-05T11:29:03,966 DEBUG [TezTR-518813_1_34_2_0_0] vector.VectorMapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:29:03,967 INFO [TezTR-518813_1_34_2_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:29:03,967 INFO [TezTR-518813_1_34_2_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:03,967 DEBUG [TezTR-518813_1_34_2_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:29:03,967 INFO [TezTR-518813_1_34_2_0_0] tez.DynamicValueRegistryTez: Runtime value source: Reducer 4 2018-01-05T11:29:03,967 DEBUG [TezTR-518813_1_34_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:03,967 DEBUG [TezTR-518813_1_34_2_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1, _col2] columnTypes=[string, string, binary] 2018-01-05T11:29:03,968 INFO [TezTR-518813_1_34_2_0_0] impl.SimpleFetchedInputAllocator: Reducer_4: RequestedMemory=131404920, AssignedMemory=131404920, maxSingleShuffleLimit=32851230 2018-01-05T11:29:03,968 INFO [TezTR-518813_1_34_2_0_0] impl.ShuffleManager: Reducer_4: numInputs=1, compressionCodec=NoCompressionCodec, numFetchers=1, ifileBufferSize=4096, ifileReadAheadEnabled=true, ifileReadAheadLength=4194304, localDiskFetchEnabled=false, sharedFetchEnabled=false, keepAlive=false, keepAliveMaxConnections=20, connectionTimeout=180000, readTimeout=180000, bufferSize=8192, bufferSize=8192, maxTaskOutputAtOnce=20 2018-01-05T11:29:03,970 INFO [IO-Elevator-Thread-3] LlapIoImpl: Creating VertorDeserializeOrcWriter for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1/kv3.txt 2018-01-05T11:29:03,970 INFO [IO-Elevator-Thread-2] LlapIoImpl: Creating VertorDeserializeOrcWriter for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src/kv1.txt 2018-01-05T11:29:03,981 INFO [TezTR-518813_1_34_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_4: numBuffers=2, sizePerBuffer=52428800, skipBuffers=true, numPartitions=1, availableMemory=104857600, maxSingleBufferSizeBytes=2147483647, pipelinedShuffle=false, isFinalMergeEnabled=true, numPartitions=1, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:29:03,981 DEBUG [TezTR-518813_1_34_0_0_0] tez.MapRecordProcessor: Starting Output: Reducer 2 2018-01-05T11:29:03,982 INFO [TezTR-518813_1_34_0_0_0] impl.ExternalSorter: Reducer 2 using: memoryMb=10, keySerializerClass=class org.apache.hadoop.hive.ql.io.HiveKey, valueSerializerClass=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization$TezBytesWritableSerializer@5b22eb26, comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator@4018e8cf, partitioner=org.apache.tez.mapreduce.partition.MRPartitioner, serialization=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.hadoop.io.serializer.WritableSerialization, org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization, org.apache.hadoop.io.serializer.avro.AvroReflectSerialization, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:29:03,982 INFO [TezTR-518813_1_34_0_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 2: , UsingHashComparator=true 2018-01-05T11:29:03,983 INFO [TezTR-518813_1_34_0_0_0] impl.PipelinedSorter: Newly allocated block size=10485760, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=10485760, total=10485760 2018-01-05T11:29:03,983 INFO [TezTR-518813_1_34_0_0_0] impl.PipelinedSorter: Pre allocating rest of memory buffers upfront 2018-01-05T11:29:03,983 INFO [TezTR-518813_1_34_0_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 2: , UsingHashComparator=true#blocks=1, maxMemUsage=10485760, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=10485760, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=10 2018-01-05T11:29:03,983 INFO [TezTR-518813_1_34_0_0_0] impl.PipelinedSorter: Reducer 2: reserved.remaining()=10485760, reserved.metasize=5242880 2018-01-05T11:29:03,983 INFO [TezTR-518813_1_34_0_0_0] tez.MapRecordProcessor: Main input name is Map 3 2018-01-05T11:29:03,983 INFO [TezTR-518813_1_34_0_0_0] vector.VectorMapOperator: Initializing operator MAP[0] 2018-01-05T11:29:03,983 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorMapOperator: Initialization Done 0 MAP 2018-01-05T11:29:03,983 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorMapOperator: Operator 0 MAP initialized 2018-01-05T11:29:03,983 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorMapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:29:03,983 INFO [TezTR-518813_1_34_0_0_0] tez.MapRecordProcessor: Id =0 Id =3 Id =26 Id =27 Id =28 <\Children> Id = 27 null<\Parent> <\RS> Id =29 Id =30 Id =31 <\Children> Id = 30 null<\Parent> <\RS> <\Children> Id = 29 null<\Parent> <\GBY> <\Children> Id = 27 null<\Parent> <\SEL> <\Children> Id = 26 null<\Parent> <\SEL> <\Children> Id = 3 null<\Parent> <\FIL> <\Children> <\Parent> <\TS> <\Children> <\Parent> <\MAP> 2018-01-05T11:29:03,984 INFO [TezTR-518813_1_34_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:03,984 DEBUG [TezTR-518813_1_34_0_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:03,984 INFO [TezTR-518813_1_34_0_0_0] exec.TableScanOperator: Initializing operator TS[3] 2018-01-05T11:29:03,984 DEBUG [TezTR-518813_1_34_0_0_0] exec.TableScanOperator: Initialization Done 3 TS 2018-01-05T11:29:03,984 DEBUG [TezTR-518813_1_34_0_0_0] exec.TableScanOperator: Operator 3 TS initialized 2018-01-05T11:29:03,984 DEBUG [TezTR-518813_1_34_0_0_0] exec.TableScanOperator: Initializing children of 3 TS 2018-01-05T11:29:03,984 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorFilterOperator: Initializing child 26 FIL 2018-01-05T11:29:03,984 INFO [TezTR-518813_1_34_0_0_0] vector.VectorFilterOperator: Initializing operator FIL[26] 2018-01-05T11:29:03,985 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorFilterOperator: Initialization Done 26 FIL 2018-01-05T11:29:03,985 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorFilterOperator: Operator 26 FIL initialized 2018-01-05T11:29:03,985 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorFilterOperator: Initializing children of 26 FIL 2018-01-05T11:29:03,985 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorSelectOperator: Initializing child 27 SEL 2018-01-05T11:29:03,985 INFO [TezTR-518813_1_34_0_0_0] vector.VectorSelectOperator: Initializing operator SEL[27] 2018-01-05T11:29:03,989 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorSelectOperator: Initialization Done 27 SEL 2018-01-05T11:29:03,989 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorSelectOperator: Operator 27 SEL initialized 2018-01-05T11:29:03,989 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorSelectOperator: Initializing children of 27 SEL 2018-01-05T11:29:03,989 DEBUG [TezTR-518813_1_34_0_0_0] reducesink.VectorReduceSinkStringOperator: Initializing child 28 RS 2018-01-05T11:29:03,989 INFO [TezTR-518813_1_34_0_0_0] reducesink.VectorReduceSinkStringOperator: Initializing operator RS[28] 2018-01-05T11:29:03,989 INFO [TezTR-518813_1_34_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir/0d27c3bb-5a71-454d-ba1d-9144cc0bd13d/map.xml 2018-01-05T11:29:03,989 DEBUG [TezTR-518813_1_34_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:03,989 INFO [TezTR-518813_1_34_0_0_0] reducesink.VectorReduceSinkCommonOperator: Using tag = 1 2018-01-05T11:29:03,996 DEBUG [TezTR-518813_1_34_0_0_0] reducesink.VectorReduceSinkStringOperator: Initialization Done 28 RS 2018-01-05T11:29:03,996 DEBUG [TezTR-518813_1_34_0_0_0] reducesink.VectorReduceSinkStringOperator: Operator 28 RS initialized 2018-01-05T11:29:03,996 DEBUG [TezTR-518813_1_34_0_0_0] reducesink.VectorReduceSinkStringOperator: Initialization Done 28 RS done is reset. 2018-01-05T11:29:03,996 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorSelectOperator: Initializing child 29 SEL 2018-01-05T11:29:03,996 INFO [TezTR-518813_1_34_0_0_0] vector.VectorSelectOperator: Initializing operator SEL[29] 2018-01-05T11:29:03,996 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorSelectOperator: Initialization Done 29 SEL 2018-01-05T11:29:03,996 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorSelectOperator: Operator 29 SEL initialized 2018-01-05T11:29:03,996 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorSelectOperator: Initializing children of 29 SEL 2018-01-05T11:29:03,996 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorGroupByOperator: Initializing child 30 GBY 2018-01-05T11:29:03,996 INFO [TezTR-518813_1_34_0_0_0] vector.VectorGroupByOperator: Initializing operator GBY[30] 2018-01-05T11:29:04,002 INFO [IO-Elevator-Thread-3] impl.WriterImpl: ORC writer created for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1/kv3.txt with stripeSize: 9223372036854775807 blockSize: 9223372036854775807 compression: NONE bufferSize: 262144 2018-01-05T11:29:04,002 INFO [IO-Elevator-Thread-2] impl.WriterImpl: ORC writer created for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src/kv1.txt with stripeSize: 9223372036854775807 blockSize: 9223372036854775807 compression: NONE bufferSize: 262144 2018-01-05T11:29:04,003 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorGroupByOperator: maxMemory:885Mb (983 * 0.900000) fixSize:884 (key:456 agg:384) 2018-01-05T11:29:04,003 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorGroupByOperator: using hash aggregation processing mode 2018-01-05T11:29:04,003 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorGroupByOperator: Initialization Done 30 GBY 2018-01-05T11:29:04,003 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorGroupByOperator: Operator 30 GBY initialized 2018-01-05T11:29:04,003 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorGroupByOperator: Initializing children of 30 GBY 2018-01-05T11:29:04,003 DEBUG [TezTR-518813_1_34_0_0_0] reducesink.VectorReduceSinkEmptyKeyOperator: Initializing child 31 RS 2018-01-05T11:29:04,003 INFO [TezTR-518813_1_34_0_0_0] reducesink.VectorReduceSinkEmptyKeyOperator: Initializing operator RS[31] 2018-01-05T11:29:04,003 INFO [TezTR-518813_1_34_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir/0d27c3bb-5a71-454d-ba1d-9144cc0bd13d/map.xml 2018-01-05T11:29:04,003 DEBUG [TezTR-518813_1_34_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:04,003 INFO [TezTR-518813_1_34_0_0_0] reducesink.VectorReduceSinkCommonOperator: Using tag = -1 2018-01-05T11:29:04,003 DEBUG [TezTR-518813_1_34_0_0_0] reducesink.VectorReduceSinkEmptyKeyOperator: Initialization Done 31 RS 2018-01-05T11:29:04,003 DEBUG [TezTR-518813_1_34_0_0_0] reducesink.VectorReduceSinkEmptyKeyOperator: Operator 31 RS initialized 2018-01-05T11:29:04,003 DEBUG [TezTR-518813_1_34_0_0_0] reducesink.VectorReduceSinkEmptyKeyOperator: Initialization Done 31 RS done is reset. 2018-01-05T11:29:04,003 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorGroupByOperator: Initialization Done 30 GBY done is reset. 2018-01-05T11:29:04,003 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorSelectOperator: Initialization Done 29 SEL done is reset. 2018-01-05T11:29:04,003 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorSelectOperator: Initialization Done 27 SEL done is reset. 2018-01-05T11:29:04,003 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorFilterOperator: Initialization Done 26 FIL done is reset. 2018-01-05T11:29:04,003 DEBUG [TezTR-518813_1_34_0_0_0] exec.TableScanOperator: Initialization Done 3 TS done is reset. 2018-01-05T11:29:04,003 INFO [TezTR-518813_1_34_0_0_0] exec.OperatorUtils: Setting output collector: RS[28] --> Reducer 2 2018-01-05T11:29:04,003 INFO [TezTR-518813_1_34_0_0_0] exec.OperatorUtils: Setting output collector: RS[31] --> Reducer 4 2018-01-05T11:29:04,003 DEBUG [TezTR-518813_1_34_0_0_0] log.PerfLogger: 2018-01-05T11:29:04,006 DEBUG [IO-Elevator-Thread-2] LlapIoImpl: Processing slice #0 null; has no cache data; has disk data 2018-01-05T11:29:04,006 DEBUG [IO-Elevator-Thread-3] LlapIoImpl: Processing slice #0 null; has no cache data; has disk data 2018-01-05T11:29:04,007 DEBUG [IO-Elevator-Thread-3] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: string streamBuffers.length: 0 vectors: 1 columnEncoding: null present: false data: null dictionary: false lengths: false secondary: false tz: US/Pacific 2018-01-05T11:29:04,007 DEBUG [IO-Elevator-Thread-2] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: string streamBuffers.length: 0 vectors: 1 columnEncoding: null present: false data: null dictionary: false lengths: false secondary: false tz: US/Pacific 2018-01-05T11:29:04,007 DEBUG [IO-Elevator-Thread-3] encoded.EncodedTreeReaderFactory: columnIndex: 2 columnType: string streamBuffers.length: 0 vectors: 1 columnEncoding: null present: false data: null dictionary: false lengths: false secondary: false tz: US/Pacific 2018-01-05T11:29:04,007 DEBUG [IO-Elevator-Thread-2] encoded.EncodedTreeReaderFactory: columnIndex: 2 columnType: string streamBuffers.length: 0 vectors: 1 columnEncoding: null present: false data: null dictionary: false lengths: false secondary: false tz: US/Pacific 2018-01-05T11:29:04,007 DEBUG [IO-Elevator-Thread-2] LlapIoOrc: Done with decode 2018-01-05T11:29:04,007 DEBUG [IO-Elevator-Thread-3] LlapIoOrc: Done with decode 2018-01-05T11:29:04,008 DEBUG [IO-Elevator-Thread-2] LlapIoImpl: setDone called; closed false, done false, err null, pending 1 2018-01-05T11:29:04,008 DEBUG [IO-Elevator-Thread-3] LlapIoImpl: setDone called; closed false, done false, err null, pending 0 2018-01-05T11:29:04,008 DEBUG [IO-Elevator-Thread-2] impl.StatsRecordingThreadPool: Updated stats: instance: SerDeEncodedDataReader thread name: IO-Elevator-Thread-2 thread id: 2076 scheme: file bytesRead: 5872 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:04,008 DEBUG [IO-Elevator-Thread-3] impl.StatsRecordingThreadPool: Updated stats: instance: SerDeEncodedDataReader thread name: IO-Elevator-Thread-3 thread id: 2077 scheme: file bytesRead: 232 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:04,009 INFO [TezTR-518813_1_34_0_0_0] reducesink.VectorReduceSinkCommonOperator: RS[28]: records written - 1 2018-01-05T11:29:04,009 INFO [TezTR-518813_1_34_0_0_0] reducesink.VectorReduceSinkCommonOperator: RS[28]: records written - 10 2018-01-05T11:29:04,011 INFO [TezTR-518813_1_34_0_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:04,011 INFO [TezTR-518813_1_34_0_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, , , ]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=0, NUM_ERRORS=0, ROWS_EMITTED=25, METADATA_CACHE_HIT=0, METADATA_CACHE_MISS=0, CACHE_HIT_BYTES=0, CACHE_MISS_BYTES=216, ALLOCATED_BYTES=0, ALLOCATED_USED_BYTES=0, TOTAL_IO_TIME_NS=47467837, DECODE_TIME_NS=640997, HDFS_TIME_NS=0, CONSUMER_TIME_NS=2749682 ] 2018-01-05T11:29:04,011 DEBUG [TezTR-518813_1_34_0_0_0] LlapIoImpl: Encoded reader is being stopped 2018-01-05T11:29:04,011 DEBUG [TezTR-518813_1_34_0_0_0] log.PerfLogger: 2018-01-05T11:29:04,011 DEBUG [TezTR-518813_1_34_0_0_0] mr.ObjectCache: djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d_Map 3__MAP_PLAN__ no longer needed 2018-01-05T11:29:04,011 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorMapOperator: close called for operator MAP[0] 2018-01-05T11:29:04,011 INFO [TezTR-518813_1_34_0_0_0] vector.VectorMapOperator: Closing operator MAP[0] 2018-01-05T11:29:04,011 INFO [TezTR-518813_1_34_0_0_0] vector.VectorMapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_3:25, 2018-01-05T11:29:04,011 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorMapOperator: Closing child = TS[3] 2018-01-05T11:29:04,011 DEBUG [TezTR-518813_1_34_0_0_0] exec.TableScanOperator: close called for operator TS[3] 2018-01-05T11:29:04,011 INFO [TezTR-518813_1_34_0_0_0] exec.TableScanOperator: Closing operator TS[3] 2018-01-05T11:29:04,011 DEBUG [TezTR-518813_1_34_0_0_0] exec.TableScanOperator: Closing child = FIL[26] 2018-01-05T11:29:04,011 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorFilterOperator: close called for operator FIL[26] 2018-01-05T11:29:04,011 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorFilterOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:04,011 INFO [TezTR-518813_1_34_0_0_0] vector.VectorFilterOperator: Closing operator FIL[26] 2018-01-05T11:29:04,011 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorFilterOperator: Closing child = SEL[27] 2018-01-05T11:29:04,011 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorSelectOperator: close called for operator SEL[27] 2018-01-05T11:29:04,011 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorSelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:04,011 INFO [TezTR-518813_1_34_0_0_0] vector.VectorSelectOperator: Closing operator SEL[27] 2018-01-05T11:29:04,011 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorSelectOperator: Closing child = RS[28] 2018-01-05T11:29:04,011 DEBUG [TezTR-518813_1_34_0_0_0] reducesink.VectorReduceSinkStringOperator: close called for operator RS[28] 2018-01-05T11:29:04,011 DEBUG [TezTR-518813_1_34_0_0_0] reducesink.VectorReduceSinkStringOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:04,011 INFO [TezTR-518813_1_34_0_0_0] reducesink.VectorReduceSinkStringOperator: Closing operator RS[28] 2018-01-05T11:29:04,011 INFO [TezTR-518813_1_34_0_0_0] reducesink.VectorReduceSinkCommonOperator: RS[28]: records written - 25 2018-01-05T11:29:04,011 INFO [TezTR-518813_1_34_0_0_0] reducesink.VectorReduceSinkStringOperator: RECORDS_OUT_INTERMEDIATE_Map_3:25, 2018-01-05T11:29:04,011 DEBUG [TezTR-518813_1_34_0_0_0] reducesink.VectorReduceSinkStringOperator: 28 Close done 2018-01-05T11:29:04,011 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorSelectOperator: Closing child = SEL[29] 2018-01-05T11:29:04,011 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorSelectOperator: close called for operator SEL[29] 2018-01-05T11:29:04,011 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorSelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:04,011 INFO [TezTR-518813_1_34_0_0_0] vector.VectorSelectOperator: Closing operator SEL[29] 2018-01-05T11:29:04,011 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorSelectOperator: Closing child = GBY[30] 2018-01-05T11:29:04,011 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorGroupByOperator: close called for operator GBY[30] 2018-01-05T11:29:04,011 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorGroupByOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:04,011 INFO [TezTR-518813_1_34_0_0_0] vector.VectorGroupByOperator: Closing operator GBY[30] 2018-01-05T11:29:04,011 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorGroupByOperator: Flush 1 (all) entries:1 fixed:884 variable:112 (used:0Mb max:885Mb) gcCanary:alive 2018-01-05T11:29:04,011 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorGroupByOperator: GC canary caused 0 flushes 2018-01-05T11:29:04,011 INFO [TezTR-518813_1_34_0_0_0] reducesink.VectorReduceSinkCommonOperator: RS[31]: records written - 1 2018-01-05T11:29:04,011 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorGroupByOperator: Closing child = RS[31] 2018-01-05T11:29:04,011 DEBUG [TezTR-518813_1_34_0_0_0] reducesink.VectorReduceSinkEmptyKeyOperator: close called for operator RS[31] 2018-01-05T11:29:04,011 DEBUG [TezTR-518813_1_34_0_0_0] reducesink.VectorReduceSinkEmptyKeyOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:04,011 INFO [TezTR-518813_1_34_0_0_0] reducesink.VectorReduceSinkEmptyKeyOperator: Closing operator RS[31] 2018-01-05T11:29:04,011 INFO [TezTR-518813_1_34_0_0_0] reducesink.VectorReduceSinkCommonOperator: RS[31]: records written - 1 2018-01-05T11:29:04,011 INFO [TezTR-518813_1_34_0_0_0] reducesink.VectorReduceSinkEmptyKeyOperator: RECORDS_OUT_INTERMEDIATE_Map_3:1, 2018-01-05T11:29:04,011 DEBUG [TezTR-518813_1_34_0_0_0] reducesink.VectorReduceSinkEmptyKeyOperator: 31 Close done 2018-01-05T11:29:04,011 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorGroupByOperator: 30 Close done 2018-01-05T11:29:04,011 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorSelectOperator: 29 Close done 2018-01-05T11:29:04,011 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorSelectOperator: 27 Close done 2018-01-05T11:29:04,011 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorFilterOperator: 26 Close done 2018-01-05T11:29:04,011 DEBUG [TezTR-518813_1_34_0_0_0] exec.TableScanOperator: 3 Close done 2018-01-05T11:29:04,011 DEBUG [TezTR-518813_1_34_0_0_0] vector.VectorMapOperator: 0 Close done 2018-01-05T11:29:04,012 INFO [TezTR-518813_1_34_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_34_00_000000_0 2018-01-05T11:29:04,012 INFO [TezTR-518813_1_34_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_4: Waiting for all spills to complete : Pending : 0 2018-01-05T11:29:04,012 INFO [TezTR-518813_1_34_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_4: All spills complete 2018-01-05T11:29:04,029 INFO [TezTR-518813_1_34_0_0_0] impl.PipelinedSorter: Reducer 2: Starting flush of map output 2018-01-05T11:29:04,029 INFO [TezTR-518813_1_34_0_0_0] impl.PipelinedSorter: Reducer 2: Span0.length = 25, perItem = 11 2018-01-05T11:29:04,029 INFO [TezTR-518813_1_34_0_0_0] impl.PipelinedSorter: Reducer 2: done sorting span=0, length=25, time=0 2018-01-05T11:29:04,029 INFO [TezTR-518813_1_34_0_0_0] impl.PipelinedSorter: Reducer 2: Heap = SpanIterator<0:24> (span=Span[5242880,291]), 2018-01-05T11:29:04,035 INFO [IO-Elevator-Thread-2-OrcEncode] LlapIoImpl: ORC encoder received a exit event 2018-01-05T11:29:04,035 INFO [IO-Elevator-Thread-3-OrcEncode] LlapIoImpl: ORC encoder received a exit event 2018-01-05T11:29:04,038 DEBUG [IO-Elevator-Thread-3-OrcEncode] LlapIoImpl: Discarding disk data (if any wasn't cached) 2018-01-05T11:29:04,038 DEBUG [IO-Elevator-Thread-2-OrcEncode] LlapIoImpl: Discarding disk data (if any wasn't cached) 2018-01-05T11:29:04,039 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000067, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_34_00_000000_0, eventCount=3 } 2018-01-05T11:29:04,040 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:29:04,043 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000068, requestId=2, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_34_01_000000_0, eventCount=1 } 2018-01-05T11:29:04,044 INFO [IPC Server handler 7 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_34_01_000000_0 1 events [0,1) total 1 vertex_1515180518813_0001_34_01 [Reducer 4] 2018-01-05T11:29:04,044 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:29:04,044 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_34_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:29:04,044 INFO [TezTaskEventRouter{attempt_1515180518813_0001_34_01_000000_0}] impl.ShuffleInputEventHandlerImpl: Map 3: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:29:04,045 INFO [TezTR-518813_1_34_0_0_0] impl.PipelinedSorter: Reducer 2: Spilling to /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/34/output/attempt_1515180518813_0001_34_00_000000_0_10202_0/file.out 2018-01-05T11:29:04,045 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=34&reduce=0&map=attempt_1515180518813_0001_34_00_000000_0_10201 mapId: [attempt_1515180518813_0001_34_00_000000_0_10201] reduceId: [0] jobId: [job_1515180518813_0001] dagId: [34] keepAlive: false 2018-01-05T11:29:04,045 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=34&reduce=0&map=attempt_1515180518813_0001_34_00_000000_0_10201; hash=...kol3zzoXtE/Qw 2018-01-05T11:29:04,046 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=34&reduce=0&map=attempt_1515180518813_0001_34_00_000000_0_10201;reply=vsuHQfVppF6rs 2018-01-05T11:29:04,046 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=34, user='djaiswal', attemptId='attempt_1515180518813_0001_34_00_000000_0_10201'} via loader 2018-01-05T11:29:04,046 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=34, user='djaiswal', attemptId='attempt_1515180518813_0001_34_00_000000_0_10201'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:29:04,046 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_34_00_000000_0_10201 not found 2018-01-05T11:29:04,046 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_34_00_000000_0_10201,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/34/output/attempt_1515180518813_0001_34_00_000000_0_10201/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/34/output/attempt_1515180518813_0001_34_00_000000_0_10201/file.out.index 2018-01-05T11:29:04,046 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Content Length in shuffle : 139 2018-01-05T11:29:04,047 INFO [Fetcher_B {Map_3} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=34&reduce=0&map=attempt_1515180518813_0001_34_00_000000_0_10201 sent hash and receievd reply 0 ms 2018-01-05T11:29:04,047 INFO [Fetcher_B {Map_3} #0] ShuffleManager.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_34_00_000000_0_10201} to MEMORY, csize=88, dsize=84, EndTime=1515180544047, TimeTaken=0, Rate=0.00 MB/s 2018-01-05T11:29:04,047 INFO [Fetcher_B {Map_3} #0] impl.ShuffleManager: All inputs fetched for input vertex : Map 3 2018-01-05T11:29:04,047 INFO [TezTR-518813_1_34_1_0_0] tez.ReduceRecordProcessor: Main work is Reducer 4 2018-01-05T11:29:04,047 INFO [Fetcher_B {Map_3} #0] impl.ShuffleManager: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:04,047 INFO [TezTR-518813_1_34_1_0_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:04,047 INFO [ShuffleRunner {Map_3}] impl.ShuffleManager: Map_3: Shutting down FetchScheduler, Was Interrupted: false 2018-01-05T11:29:04,047 DEBUG [TezTR-518813_1_34_1_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:04,047 INFO [ShuffleRunner {Map_3}] impl.ShuffleManager: Map_3: Scheduler thread completed 2018-01-05T11:29:04,047 DEBUG [TezTR-518813_1_34_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:04,047 DEBUG [TezTR-518813_1_34_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:04,047 DEBUG [TezTR-518813_1_34_1_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1, _col2] columnTypes=[string, string, binary] 2018-01-05T11:29:04,049 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000069, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_34_02_000000_0, eventCount=1 } 2018-01-05T11:29:04,049 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:29:04,052 DEBUG [TezTR-518813_1_34_1_0_0] log.PerfLogger: 2018-01-05T11:29:04,052 INFO [TezTR-518813_1_34_1_0_0] vector.VectorGroupByOperator: Initializing operator GBY[32] 2018-01-05T11:29:04,055 INFO [TezTR-518813_1_34_1_0_0] vector.VectorGroupByOperator: using unsorted streaming aggregation processing mode 2018-01-05T11:29:04,055 DEBUG [TezTR-518813_1_34_1_0_0] vector.VectorGroupByOperator: Initialization Done 32 GBY 2018-01-05T11:29:04,055 DEBUG [TezTR-518813_1_34_1_0_0] vector.VectorGroupByOperator: Operator 32 GBY initialized 2018-01-05T11:29:04,055 DEBUG [TezTR-518813_1_34_1_0_0] vector.VectorGroupByOperator: Initializing children of 32 GBY 2018-01-05T11:29:04,055 DEBUG [TezTR-518813_1_34_1_0_0] reducesink.VectorReduceSinkEmptyKeyOperator: Initializing child 33 RS 2018-01-05T11:29:04,055 INFO [TezTR-518813_1_34_1_0_0] reducesink.VectorReduceSinkEmptyKeyOperator: Initializing operator RS[33] 2018-01-05T11:29:04,055 INFO [TezTR-518813_1_34_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir/0945123a-e0af-4773-8b55-9afb14888ffa/reduce.xml 2018-01-05T11:29:04,055 DEBUG [TezTR-518813_1_34_1_0_0] exec.Utilities: Found plan in cache for name: reduce.xml 2018-01-05T11:29:04,055 INFO [TezTR-518813_1_34_1_0_0] reducesink.VectorReduceSinkCommonOperator: Using tag = -1 2018-01-05T11:29:04,055 DEBUG [TezTR-518813_1_34_1_0_0] reducesink.VectorReduceSinkEmptyKeyOperator: Initialization Done 33 RS 2018-01-05T11:29:04,055 DEBUG [TezTR-518813_1_34_1_0_0] reducesink.VectorReduceSinkEmptyKeyOperator: Operator 33 RS initialized 2018-01-05T11:29:04,055 DEBUG [TezTR-518813_1_34_1_0_0] reducesink.VectorReduceSinkEmptyKeyOperator: Initialization Done 33 RS done is reset. 2018-01-05T11:29:04,055 DEBUG [TezTR-518813_1_34_1_0_0] vector.VectorGroupByOperator: Initialization Done 32 GBY done is reset. 2018-01-05T11:29:04,055 INFO [TezTR-518813_1_34_1_0_0] tez.ReduceRecordProcessor: Id =32 Id =33 <\Children> Id = 32 null<\Parent> <\RS> <\Children> <\Parent> <\GBY> 2018-01-05T11:29:04,055 INFO [TezTR-518813_1_34_1_0_0] exec.OperatorUtils: Setting output collector: RS[33] --> Map 1 2018-01-05T11:29:04,055 DEBUG [TezTR-518813_1_34_1_0_0] log.PerfLogger: 2018-01-05T11:29:04,055 INFO [TezTR-518813_1_34_1_0_0] tez.ReduceRecordProcessor: Starting Output: Map 1 2018-01-05T11:29:04,056 INFO [TezTR-518813_1_34_1_0_0] writers.BaseUnorderedPartitionedKVWriter: Instantiating Partitioner: [org.apache.tez.runtime.library.output.UnorderedKVOutput$CustomPartitioner] 2018-01-05T11:29:04,065 DEBUG [TezTR-518813_1_34_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:04,065 INFO [TezTR-518813_1_34_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:04,065 INFO [TezTR-518813_1_34_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_34_00_000000_0, askedToStop=false 2018-01-05T11:29:04,065 INFO [TezTR-518813_1_34_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_34_00_000000_0, stopRequested=false 2018-01-05T11:29:04,065 INFO [TezTR-518813_1_34_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_34_00_000000_0: Counters: 27 [[File System Counters FILE_BYTES_READ=232, FILE_BYTES_WRITTEN=0, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=25, INPUT_RECORDS_PROCESSED=1, INPUT_SPLIT_LENGTH_BYTES=216, OUTPUT_RECORDS=26, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=367, OUTPUT_BYTES_WITH_OVERHEAD=397, OUTPUT_BYTES_PHYSICAL=429, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_3=25, RECORDS_OUT_INTERMEDIATE_Map_3=26][org.apache.hadoop.hive.llap.counters.LlapIOCounters CACHE_MISS_BYTES=216, CONSUMER_TIME_NS=2749682, DECODE_TIME_NS=640997, NUM_DECODED_BATCHES=1, NUM_VECTOR_BATCHES=1, ROWS_EMITTED=25, TOTAL_IO_TIME_NS=47467837]] 2018-01-05T11:29:04,065 DEBUG [TezTR-518813_1_34_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_34_0_0_0 thread id: 2051 scheme: file bytesRead: 64 bytesWritten: 469 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:04,065 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_34_00_000000_0, isTaskDone=true 2018-01-05T11:29:04,065 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000067, requestId=3, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_34_00_000000_0, eventCount=4 } 2018-01-05T11:29:04,066 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 3, taskAttemptId=attempt_1515180518813_0001_34_00_000000_0, creationTime=1515180543905, allocationTime=1515180543906, startTime=1515180543910, finishTime=1515180544066, timeTaken=156, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:04,066 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_34_00_000000_0 2018-01-05T11:29:04,066 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:TASK_FINISHED]: vertexName=Map 3, taskId=task_1515180518813_0001_34_00_000000, startTime=1515180543910, finishTime=1515180544066, timeTaken=156, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_34_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=296, FILE_BYTES_WRITTEN=469, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=25, INPUT_RECORDS_PROCESSED=1, INPUT_SPLIT_LENGTH_BYTES=216, OUTPUT_RECORDS=26, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=367, OUTPUT_BYTES_WITH_OVERHEAD=397, OUTPUT_BYTES_PHYSICAL=429, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_3=25, RECORDS_OUT_INTERMEDIATE_Map_3=26, org.apache.hadoop.hive.llap.counters.LlapIOCounters, CACHE_MISS_BYTES=216, CONSUMER_TIME_NS=2749682, DECODE_TIME_NS=640997, NUM_DECODED_BATCHES=1, NUM_VECTOR_BATCHES=1, ROWS_EMITTED=25, TOTAL_IO_TIME_NS=47467837 2018-01-05T11:29:04,066 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_34_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:04,066 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_34_00_000000_0; false 2018-01-05T11:29:04,066 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=3, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:29:04,066 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_34_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:04,066 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_34_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:04,066 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_34_00_000000_0 2018-01-05T11:29:04,066 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_34_00 [Map 3], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:04,066 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_34_00 [Map 3] 2018-01-05T11:29:04,066 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000067=138 2018-01-05T11:29:04,066 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_34_00_000000_0: true 2018-01-05T11:29:04,067 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:04,067 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:04,067 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:VERTEX_FINISHED]: vertexName=Map 3, vertexId=vertex_1515180518813_0001_34_00, initRequestedTime=1515180543833, initedTime=1515180543905, startRequestedTime=1515180543871, startedTime=1515180543905, finishTime=1515180544066, timeTaken=161, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=296, FILE_BYTES_WRITTEN=469, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=25, INPUT_RECORDS_PROCESSED=1, INPUT_SPLIT_LENGTH_BYTES=216, OUTPUT_RECORDS=26, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=367, OUTPUT_BYTES_WITH_OVERHEAD=397, OUTPUT_BYTES_PHYSICAL=429, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_3=25, RECORDS_OUT_INTERMEDIATE_Map_3=26, org.apache.hadoop.hive.llap.counters.LlapIOCounters, CACHE_MISS_BYTES=216, CONSUMER_TIME_NS=2749682, DECODE_TIME_NS=640997, NUM_DECODED_BATCHES=1, NUM_VECTOR_BATCHES=1, ROWS_EMITTED=25, TOTAL_IO_TIME_NS=47467837, vertexStats=firstTaskStartTime=1515180543910, firstTasksToStart=[ task_1515180518813_0001_34_00_000000 ], lastTaskFinishTime=1515180544066, lastTasksToFinish=[ task_1515180518813_0001_34_00_000000 ], minTaskDuration=156, maxTaskDuration=156, avgTaskDuration=156.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_34_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_34_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:04,067 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_34_00_000000_0 2018-01-05T11:29:04,067 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_34_00 [Map 3] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:04,067 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000067, QueryId=djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d-34, Vertex=Map 3, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:04,067 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_34_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=2, preemptionQueueSize=2 2018-01-05T11:29:04,067 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_34_00_000000_0 2018-01-05T11:29:04,067 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_34_01 [Reducer 4] attempt: attempt_1515180518813_0001_34_00_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:29:04,067 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000067, QueryId=djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d-34, DagId=34, VertexName=Map 3, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180543926, EndTime=1515180544067 2018-01-05T11:29:04,067 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_34_03 [Reducer 2] attempt: attempt_1515180518813_0001_34_00_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:29:04,067 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: All source tasks completed. Ramping up 0 remaining tasks for vertex: Reducer 4 2018-01-05T11:29:04,067 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000067, stoppedTime=1515180544067, exitStatus=0 2018-01-05T11:29:04,067 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000067 exited with diagnostics set to Container completed. 2018-01-05T11:29:04,067 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_34_00 [Map 3] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=4 2018-01-05T11:29:04,067 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=4, commitInProgress=0, terminationCause=null 2018-01-05T11:29:04,067 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000067 2018-01-05T11:29:04,067 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=34}, sourceName=Map 3, state=S_SUCCEEDED 2018-01-05T11:29:04,067 INFO [IPC Server handler 0 on 49818] impl.TaskExecutorService: Received finishable state update for attempt_1515180518813_0001_34_01_000000_0, state=true 2018-01-05T11:29:04,067 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Fragment attempt_1515180518813_0001_34_01_000000_0 guaranteed state changed to false; finishable true, in wait queue false, in preemption queue true 2018-01-05T11:29:04,070 INFO [TezTR-518813_1_34_1_0_0] writers.UnorderedPartitionedKVWriter: Map_1: numBuffers=2, sizePerBuffer=0, skipBuffers=true, numPartitions=1, availableMemory=0, maxSingleBufferSizeBytes=2147483647, pipelinedShuffle=false, isFinalMergeEnabled=true, numPartitions=1, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:29:04,070 INFO [TezTR-518813_1_34_1_0_0] output.UnorderedKVOutput: Map 1 started. MemoryAssigned=0 2018-01-05T11:29:04,071 INFO [TezTR-518813_1_34_1_0_0] readers.UnorderedKVReader: Num Records read: 1 2018-01-05T11:29:04,071 DEBUG [TezTR-518813_1_34_1_0_0] log.PerfLogger: 2018-01-05T11:29:04,071 DEBUG [TezTR-518813_1_34_1_0_0] mr.ObjectCache: djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d_Reducer 4__REDUCE_PLAN__ no longer needed 2018-01-05T11:29:04,071 DEBUG [TezTR-518813_1_34_1_0_0] vector.VectorGroupByOperator: close called for operator GBY[32] 2018-01-05T11:29:04,071 INFO [TezTR-518813_1_34_1_0_0] vector.VectorGroupByOperator: Closing operator GBY[32] 2018-01-05T11:29:04,071 INFO [TezTR-518813_1_34_1_0_0] reducesink.VectorReduceSinkCommonOperator: RS[33]: records written - 1 2018-01-05T11:29:04,071 DEBUG [TezTR-518813_1_34_1_0_0] vector.VectorGroupByOperator: Closing child = RS[33] 2018-01-05T11:29:04,071 DEBUG [TezTR-518813_1_34_1_0_0] reducesink.VectorReduceSinkEmptyKeyOperator: close called for operator RS[33] 2018-01-05T11:29:04,071 DEBUG [TezTR-518813_1_34_1_0_0] reducesink.VectorReduceSinkEmptyKeyOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:04,071 INFO [TezTR-518813_1_34_1_0_0] reducesink.VectorReduceSinkEmptyKeyOperator: Closing operator RS[33] 2018-01-05T11:29:04,071 INFO [TezTR-518813_1_34_1_0_0] reducesink.VectorReduceSinkCommonOperator: RS[33]: records written - 1 2018-01-05T11:29:04,071 INFO [TezTR-518813_1_34_1_0_0] reducesink.VectorReduceSinkEmptyKeyOperator: RECORDS_OUT_INTERMEDIATE_Reducer_4:1, 2018-01-05T11:29:04,071 DEBUG [TezTR-518813_1_34_1_0_0] reducesink.VectorReduceSinkEmptyKeyOperator: 33 Close done 2018-01-05T11:29:04,071 DEBUG [TezTR-518813_1_34_1_0_0] vector.VectorGroupByOperator: 32 Close done 2018-01-05T11:29:04,071 INFO [TezTR-518813_1_34_1_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_34_01_000000_0 2018-01-05T11:29:04,071 INFO [TezTR-518813_1_34_1_0_0] impl.ShuffleInputEventHandlerImpl: Map 3: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:29:04,071 INFO [TezTR-518813_1_34_1_0_0] impl.ShuffleManager: Shutting down pending fetchers on sourceMap_3: 0 2018-01-05T11:29:04,071 INFO [TezTR-518813_1_34_1_0_0] writers.UnorderedPartitionedKVWriter: Map_1: Waiting for all spills to complete : Pending : 0 2018-01-05T11:29:04,071 INFO [TezTR-518813_1_34_1_0_0] writers.UnorderedPartitionedKVWriter: Map_1: All spills complete 2018-01-05T11:29:04,084 DEBUG [TezTR-518813_1_34_1_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:04,084 INFO [TezTR-518813_1_34_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:04,084 INFO [TezTR-518813_1_34_1_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_34_01_000000_0, askedToStop=false 2018-01-05T11:29:04,084 INFO [TezTR-518813_1_34_1_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_34_01_000000_0, stopRequested=false 2018-01-05T11:29:04,084 INFO [TezTR-518813_1_34_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_34_01_000000_0: Counters: 21 [[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=76, OUTPUT_BYTES_WITH_OVERHEAD=84, OUTPUT_BYTES_PHYSICAL=112, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=88, SHUFFLE_BYTES_DECOMPRESSED=84, SHUFFLE_BYTES_TO_MEM=88, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=99, FIRST_EVENT_RECEIVED=96, LAST_EVENT_RECEIVED=96][HIVE RECORDS_OUT_INTERMEDIATE_Reducer_4=1]] 2018-01-05T11:29:04,084 DEBUG [TezTR-518813_1_34_1_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_34_1_0_0 thread id: 2058 scheme: file bytesRead: 0 bytesWritten: 120 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:04,084 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_34_01_000000_0, isTaskDone=true 2018-01-05T11:29:04,084 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000068, requestId=3, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_34_01_000000_0, eventCount=4 } 2018-01-05T11:29:04,085 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 4, taskAttemptId=attempt_1515180518813_0001_34_01_000000_0, creationTime=1515180543906, allocationTime=1515180543909, startTime=1515180543911, finishTime=1515180544085, timeTaken=174, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:04,085 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_34_01_000000_0 2018-01-05T11:29:04,085 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_34_01_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:04,085 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_34_01_000000_0; false 2018-01-05T11:29:04,085 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_34_01_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:04,085 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=3, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:04,085 INFO [Task-Executor-3] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_34_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:04,085 INFO [Task-Executor-3] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_34_01_000000_0 2018-01-05T11:29:04,085 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:TASK_FINISHED]: vertexName=Reducer 4, taskId=task_1515180518813_0001_34_01_000000, startTime=1515180543911, finishTime=1515180544085, timeTaken=174, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_34_01_000000_0, diagnostics=, counters=Counters: 26, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=120, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=76, OUTPUT_BYTES_WITH_OVERHEAD=84, OUTPUT_BYTES_PHYSICAL=112, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=88, SHUFFLE_BYTES_DECOMPRESSED=84, SHUFFLE_BYTES_TO_MEM=88, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=99, FIRST_EVENT_RECEIVED=96, LAST_EVENT_RECEIVED=96, HIVE, RECORDS_OUT_INTERMEDIATE_Reducer_4=1 2018-01-05T11:29:04,085 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:04,085 INFO [Task-Executor-3] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000068=151 2018-01-05T11:29:04,086 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:04,086 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_34_01_000000_0: true 2018-01-05T11:29:04,086 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_34_01_000000_0 2018-01-05T11:29:04,086 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_34_01 [Reducer 4], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:04,086 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_34_01 [Reducer 4] 2018-01-05T11:29:04,086 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000068, QueryId=djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d-34, Vertex=Reducer 4, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:04,086 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_34_01_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=3, preemptionQueueSize=1 2018-01-05T11:29:04,086 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_34_01_000000_0 2018-01-05T11:29:04,086 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000068, QueryId=djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d-34, DagId=34, VertexName=Reducer 4, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-3, Succeeded=true, StartTime=1515180543934, EndTime=1515180544086 2018-01-05T11:29:04,086 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:VERTEX_FINISHED]: vertexName=Reducer 4, vertexId=vertex_1515180518813_0001_34_01, initRequestedTime=1515180543871, initedTime=1515180543872, startRequestedTime=1515180543905, startedTime=1515180543905, finishTime=1515180544086, timeTaken=181, status=SUCCEEDED, diagnostics=, counters=Counters: 26, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=120, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=76, OUTPUT_BYTES_WITH_OVERHEAD=84, OUTPUT_BYTES_PHYSICAL=112, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=88, SHUFFLE_BYTES_DECOMPRESSED=84, SHUFFLE_BYTES_TO_MEM=88, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=99, FIRST_EVENT_RECEIVED=96, LAST_EVENT_RECEIVED=96, HIVE, RECORDS_OUT_INTERMEDIATE_Reducer_4=1, vertexStats=firstTaskStartTime=1515180543911, firstTasksToStart=[ task_1515180518813_0001_34_01_000000 ], lastTaskFinishTime=1515180544085, lastTasksToFinish=[ task_1515180518813_0001_34_01_000000 ], minTaskDuration=174, maxTaskDuration=174, avgTaskDuration=174.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_34_01_000000 ], longestDurationTasks=[ task_1515180518813_0001_34_01_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:04,086 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_34_01 [Reducer 4] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:04,086 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000068, stoppedTime=1515180544086, exitStatus=0 2018-01-05T11:29:04,086 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000068 exited with diagnostics set to Container completed. 2018-01-05T11:29:04,086 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_34_02 [Map 1] attempt: attempt_1515180518813_0001_34_01_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:29:04,086 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_34_01 [Reducer 4] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=4 2018-01-05T11:29:04,086 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=4, commitInProgress=0, terminationCause=null 2018-01-05T11:29:04,086 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000068 2018-01-05T11:29:04,086 INFO [App Shared Pool - #1] impl.RootInputVertexManager: All source tasks completed. Ramping up 0 remaining tasks for vertex: Map 1 2018-01-05T11:29:04,086 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=34}, sourceName=Reducer 4, state=S_SUCCEEDED 2018-01-05T11:29:04,086 INFO [IPC Server handler 2 on 49818] impl.TaskExecutorService: Received finishable state update for attempt_1515180518813_0001_34_02_000000_0, state=true 2018-01-05T11:29:04,086 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Fragment attempt_1515180518813_0001_34_02_000000_0 guaranteed state changed to false; finishable true, in wait queue false, in preemption queue true 2018-01-05T11:29:04,110 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,110 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: Status: Running (Executing on YARN cluster with App id application_1515180518813_0001) 2018-01-05T11:29:04,111 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,111 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,111 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,111 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,111 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,111 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 0(+1)/1 Map 3: 1/1 Reducer 2: 0/1 Reducer 4: 1/1 2018-01-05T11:29:04,152 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000069, requestId=3, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_34_02_000000_0, eventCount=1 } 2018-01-05T11:29:04,153 INFO [IPC Server handler 14 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_34_02_000000_0 1 events [0,1) total 1 vertex_1515180518813_0001_34_02 [Map 1] 2018-01-05T11:29:04,153 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=3, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:29:04,153 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_34_02_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:29:04,153 INFO [TezTaskEventRouter{attempt_1515180518813_0001_34_02_000000_0}] impl.ShuffleInputEventHandlerImpl: Reducer 4: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:29:04,154 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=34&reduce=0&map=attempt_1515180518813_0001_34_01_000000_0_10205 mapId: [attempt_1515180518813_0001_34_01_000000_0_10205] reduceId: [0] jobId: [job_1515180518813_0001] dagId: [34] keepAlive: false 2018-01-05T11:29:04,154 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=34&reduce=0&map=attempt_1515180518813_0001_34_01_000000_0_10205; hash=...MzQ2QWFGCCgrc 2018-01-05T11:29:04,155 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=34&reduce=0&map=attempt_1515180518813_0001_34_01_000000_0_10205;reply=7rH35Lw7J8Y/w 2018-01-05T11:29:04,155 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=34, user='djaiswal', attemptId='attempt_1515180518813_0001_34_01_000000_0_10205'} via loader 2018-01-05T11:29:04,155 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=34, user='djaiswal', attemptId='attempt_1515180518813_0001_34_01_000000_0_10205'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:29:04,155 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_34_01_000000_0_10205 not found 2018-01-05T11:29:04,155 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_34_01_000000_0_10205,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/34/output/attempt_1515180518813_0001_34_01_000000_0_10205/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/34/output/attempt_1515180518813_0001_34_01_000000_0_10205/file.out.index 2018-01-05T11:29:04,155 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Content Length in shuffle : 139 2018-01-05T11:29:04,155 INFO [Fetcher_B {Reducer_4} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=34&reduce=0&map=attempt_1515180518813_0001_34_01_000000_0_10205 sent hash and receievd reply 0 ms 2018-01-05T11:29:04,155 INFO [Fetcher_B {Reducer_4} #0] ShuffleManager.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_34_01_000000_0_10205} to MEMORY, csize=88, dsize=84, EndTime=1515180544155, TimeTaken=0, Rate=0.00 MB/s 2018-01-05T11:29:04,155 INFO [Fetcher_B {Reducer_4} #0] impl.ShuffleManager: All inputs fetched for input vertex : Reducer 4 2018-01-05T11:29:04,155 INFO [Fetcher_B {Reducer_4} #0] impl.ShuffleManager: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:04,156 INFO [ShuffleRunner {Reducer_4}] impl.ShuffleManager: Reducer_4: Shutting down FetchScheduler, Was Interrupted: false 2018-01-05T11:29:04,156 INFO [ShuffleRunner {Reducer_4}] impl.ShuffleManager: Reducer_4: Scheduler thread completed 2018-01-05T11:29:04,156 INFO [TezTR-518813_1_34_2_0_0] readers.UnorderedKVReader: Num Records read: 1 2018-01-05T11:29:04,156 INFO [TezTR-518813_1_34_2_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:29:04,156 DEBUG [TezTR-518813_1_34_2_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:29:04,156 DEBUG [TezTR-518813_1_34_2_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:29:04,156 DEBUG [TezTR-518813_1_34_2_0_0] exec.TableScanOperator: Initializing children of 0 TS 2018-01-05T11:29:04,156 DEBUG [TezTR-518813_1_34_2_0_0] vector.VectorFilterOperator: Initializing child 34 FIL 2018-01-05T11:29:04,156 INFO [TezTR-518813_1_34_2_0_0] vector.VectorFilterOperator: Initializing operator FIL[34] 2018-01-05T11:29:04,157 DEBUG [TezTR-518813_1_34_2_0_0] vector.VectorFilterOperator: Initialization Done 34 FIL 2018-01-05T11:29:04,157 DEBUG [TezTR-518813_1_34_2_0_0] vector.VectorFilterOperator: Operator 34 FIL initialized 2018-01-05T11:29:04,157 DEBUG [TezTR-518813_1_34_2_0_0] vector.VectorFilterOperator: Initializing children of 34 FIL 2018-01-05T11:29:04,157 DEBUG [TezTR-518813_1_34_2_0_0] vector.VectorSelectOperator: Initializing child 35 SEL 2018-01-05T11:29:04,157 INFO [TezTR-518813_1_34_2_0_0] vector.VectorSelectOperator: Initializing operator SEL[35] 2018-01-05T11:29:04,157 DEBUG [TezTR-518813_1_34_2_0_0] vector.VectorSelectOperator: Initialization Done 35 SEL 2018-01-05T11:29:04,157 DEBUG [TezTR-518813_1_34_2_0_0] vector.VectorSelectOperator: Operator 35 SEL initialized 2018-01-05T11:29:04,157 DEBUG [TezTR-518813_1_34_2_0_0] vector.VectorSelectOperator: Initializing children of 35 SEL 2018-01-05T11:29:04,157 DEBUG [TezTR-518813_1_34_2_0_0] reducesink.VectorReduceSinkStringOperator: Initializing child 36 RS 2018-01-05T11:29:04,157 INFO [TezTR-518813_1_34_2_0_0] reducesink.VectorReduceSinkStringOperator: Initializing operator RS[36] 2018-01-05T11:29:04,157 INFO [TezTR-518813_1_34_2_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir/e0b5a45c-1c80-452b-acaf-ad6f51dd68d8/map.xml 2018-01-05T11:29:04,157 DEBUG [TezTR-518813_1_34_2_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:04,157 INFO [TezTR-518813_1_34_2_0_0] reducesink.VectorReduceSinkCommonOperator: Using tag = 0 2018-01-05T11:29:04,157 DEBUG [TezTR-518813_1_34_2_0_0] reducesink.VectorReduceSinkStringOperator: Initialization Done 36 RS 2018-01-05T11:29:04,157 DEBUG [TezTR-518813_1_34_2_0_0] reducesink.VectorReduceSinkStringOperator: Operator 36 RS initialized 2018-01-05T11:29:04,157 DEBUG [TezTR-518813_1_34_2_0_0] reducesink.VectorReduceSinkStringOperator: Initialization Done 36 RS done is reset. 2018-01-05T11:29:04,157 DEBUG [TezTR-518813_1_34_2_0_0] vector.VectorSelectOperator: Initialization Done 35 SEL done is reset. 2018-01-05T11:29:04,157 DEBUG [TezTR-518813_1_34_2_0_0] vector.VectorFilterOperator: Initialization Done 34 FIL done is reset. 2018-01-05T11:29:04,157 DEBUG [TezTR-518813_1_34_2_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:29:04,157 INFO [TezTR-518813_1_34_2_0_0] exec.OperatorUtils: Setting output collector: RS[36] --> Reducer 2 2018-01-05T11:29:04,157 DEBUG [TezTR-518813_1_34_2_0_0] log.PerfLogger: 2018-01-05T11:29:04,157 DEBUG [TezTR-518813_1_34_2_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:04,157 DEBUG [TezTR-518813_1_34_2_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:04,158 DEBUG [TezTR-518813_1_34_2_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:04,159 INFO [TezTR-518813_1_34_2_0_0] reducesink.VectorReduceSinkCommonOperator: RS[36]: records written - 1 2018-01-05T11:29:04,159 INFO [TezTR-518813_1_34_2_0_0] reducesink.VectorReduceSinkCommonOperator: RS[36]: records written - 10 2018-01-05T11:29:04,159 INFO [TezTR-518813_1_34_2_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:04,159 INFO [TezTR-518813_1_34_2_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, , , ]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=0, NUM_ERRORS=0, ROWS_EMITTED=500, METADATA_CACHE_HIT=0, METADATA_CACHE_MISS=0, CACHE_HIT_BYTES=0, CACHE_MISS_BYTES=5812, ALLOCATED_BYTES=0, ALLOCATED_USED_BYTES=0, TOTAL_IO_TIME_NS=47469300, DECODE_TIME_NS=621074, HDFS_TIME_NS=0, CONSUMER_TIME_NS=1615888 ] 2018-01-05T11:29:04,159 DEBUG [TezTR-518813_1_34_2_0_0] LlapIoImpl: Encoded reader is being stopped 2018-01-05T11:29:04,159 DEBUG [TezTR-518813_1_34_2_0_0] log.PerfLogger: 2018-01-05T11:29:04,159 DEBUG [TezTR-518813_1_34_2_0_0] mr.ObjectCache: djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:29:04,159 DEBUG [TezTR-518813_1_34_2_0_0] vector.VectorMapOperator: close called for operator MAP[0] 2018-01-05T11:29:04,159 INFO [TezTR-518813_1_34_2_0_0] vector.VectorMapOperator: Closing operator MAP[0] 2018-01-05T11:29:04,159 INFO [TezTR-518813_1_34_2_0_0] vector.VectorMapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:500, 2018-01-05T11:29:04,159 DEBUG [TezTR-518813_1_34_2_0_0] vector.VectorMapOperator: Closing child = TS[0] 2018-01-05T11:29:04,159 DEBUG [TezTR-518813_1_34_2_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:29:04,159 INFO [TezTR-518813_1_34_2_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:29:04,159 DEBUG [TezTR-518813_1_34_2_0_0] exec.TableScanOperator: Closing child = FIL[34] 2018-01-05T11:29:04,159 DEBUG [TezTR-518813_1_34_2_0_0] vector.VectorFilterOperator: close called for operator FIL[34] 2018-01-05T11:29:04,160 DEBUG [TezTR-518813_1_34_2_0_0] vector.VectorFilterOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:04,160 INFO [TezTR-518813_1_34_2_0_0] vector.VectorFilterOperator: Closing operator FIL[34] 2018-01-05T11:29:04,160 DEBUG [TezTR-518813_1_34_2_0_0] vector.VectorFilterOperator: Closing child = SEL[35] 2018-01-05T11:29:04,160 DEBUG [TezTR-518813_1_34_2_0_0] vector.VectorSelectOperator: close called for operator SEL[35] 2018-01-05T11:29:04,160 DEBUG [TezTR-518813_1_34_2_0_0] vector.VectorSelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:04,160 INFO [TezTR-518813_1_34_2_0_0] vector.VectorSelectOperator: Closing operator SEL[35] 2018-01-05T11:29:04,160 DEBUG [TezTR-518813_1_34_2_0_0] vector.VectorSelectOperator: Closing child = RS[36] 2018-01-05T11:29:04,160 DEBUG [TezTR-518813_1_34_2_0_0] reducesink.VectorReduceSinkStringOperator: close called for operator RS[36] 2018-01-05T11:29:04,160 DEBUG [TezTR-518813_1_34_2_0_0] reducesink.VectorReduceSinkStringOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:04,160 INFO [TezTR-518813_1_34_2_0_0] reducesink.VectorReduceSinkStringOperator: Closing operator RS[36] 2018-01-05T11:29:04,160 INFO [TezTR-518813_1_34_2_0_0] reducesink.VectorReduceSinkCommonOperator: RS[36]: records written - 37 2018-01-05T11:29:04,160 INFO [TezTR-518813_1_34_2_0_0] reducesink.VectorReduceSinkStringOperator: RECORDS_OUT_INTERMEDIATE_Map_1:37, 2018-01-05T11:29:04,160 DEBUG [TezTR-518813_1_34_2_0_0] reducesink.VectorReduceSinkStringOperator: 36 Close done 2018-01-05T11:29:04,160 DEBUG [TezTR-518813_1_34_2_0_0] vector.VectorSelectOperator: 35 Close done 2018-01-05T11:29:04,160 DEBUG [TezTR-518813_1_34_2_0_0] vector.VectorFilterOperator: 34 Close done 2018-01-05T11:29:04,160 DEBUG [TezTR-518813_1_34_2_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:29:04,160 DEBUG [TezTR-518813_1_34_2_0_0] vector.VectorMapOperator: 0 Close done 2018-01-05T11:29:04,160 INFO [TezTR-518813_1_34_2_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_34_02_000000_0 2018-01-05T11:29:04,160 INFO [TezTR-518813_1_34_2_0_0] impl.ShuffleInputEventHandlerImpl: Reducer 4: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:29:04,160 INFO [TezTR-518813_1_34_2_0_0] impl.ShuffleManager: Shutting down pending fetchers on sourceReducer_4: 0 2018-01-05T11:29:04,160 INFO [TezTR-518813_1_34_2_0_0] impl.PipelinedSorter: Reducer 2: Starting flush of map output 2018-01-05T11:29:04,160 INFO [TezTR-518813_1_34_2_0_0] impl.PipelinedSorter: Reducer 2: Span0.length = 37, perItem = 14 2018-01-05T11:29:04,160 INFO [TezTR-518813_1_34_2_0_0] impl.PipelinedSorter: Reducer 2: done sorting span=0, length=37, time=0 2018-01-05T11:29:04,160 INFO [TezTR-518813_1_34_2_0_0] impl.PipelinedSorter: Reducer 2: Heap = SpanIterator<0:36> (span=Span[5242880,549]), 2018-01-05T11:29:04,175 INFO [TezTR-518813_1_34_2_0_0] impl.PipelinedSorter: Reducer 2: Spilling to /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/34/output/attempt_1515180518813_0001_34_02_000000_0_10209_0/file.out 2018-01-05T11:29:04,189 DEBUG [TezTR-518813_1_34_2_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:04,189 INFO [TezTR-518813_1_34_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:04,189 INFO [TezTR-518813_1_34_2_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_34_02_000000_0, askedToStop=false 2018-01-05T11:29:04,189 INFO [TezTR-518813_1_34_2_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_34_02_000000_0, stopRequested=false 2018-01-05T11:29:04,189 INFO [TezTR-518813_1_34_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_34_02_000000_0: Counters: 37 [[File System Counters FILE_BYTES_READ=5872, FILE_BYTES_WRITTEN=0, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=2, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=37, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=549, OUTPUT_BYTES_WITH_OVERHEAD=502, OUTPUT_BYTES_PHYSICAL=506, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=88, SHUFFLE_BYTES_DECOMPRESSED=84, SHUFFLE_BYTES_TO_MEM=88, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=188, FIRST_EVENT_RECEIVED=185, LAST_EVENT_RECEIVED=185][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500, RECORDS_OUT_INTERMEDIATE_Map_1=37][org.apache.hadoop.hive.llap.counters.LlapIOCounters CACHE_MISS_BYTES=5812, CONSUMER_TIME_NS=1615888, DECODE_TIME_NS=621074, NUM_DECODED_BATCHES=1, NUM_VECTOR_BATCHES=1, ROWS_EMITTED=500, TOTAL_IO_TIME_NS=47469300]] 2018-01-05T11:29:04,189 DEBUG [TezTR-518813_1_34_2_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_34_2_0_0 thread id: 2066 scheme: file bytesRead: 64 bytesWritten: 538 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:04,190 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_34_02_000000_0, isTaskDone=true 2018-01-05T11:29:04,190 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000069, requestId=4, startIndex=1, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_34_02_000000_0, eventCount=4 } 2018-01-05T11:29:04,190 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_34_02_000000_0, creationTime=1515180543932, allocationTime=1515180543934, startTime=1515180543934, finishTime=1515180544190, timeTaken=256, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:04,190 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=4, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:29:04,190 INFO [Task-Executor-1] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_34_02_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:04,190 INFO [Task-Executor-1] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_34_02_000000_0 2018-01-05T11:29:04,190 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_34_02_000000_0 2018-01-05T11:29:04,190 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_34_02_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:04,190 INFO [Task-Executor-1] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000069=252 2018-01-05T11:29:04,190 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_34_02_000000_0; false 2018-01-05T11:29:04,190 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_34_02_000000_0: true 2018-01-05T11:29:04,190 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_34_02_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:04,190 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_34_02_000000_0 2018-01-05T11:29:04,190 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:04,190 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:04,190 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000069, QueryId=djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d-34, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:04,190 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_34_02_000000, startTime=1515180543934, finishTime=1515180544190, timeTaken=256, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_34_02_000000_0, diagnostics=, counters=Counters: 37, File System Counters, FILE_BYTES_READ=5936, FILE_BYTES_WRITTEN=538, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=2, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=37, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=549, OUTPUT_BYTES_WITH_OVERHEAD=502, OUTPUT_BYTES_PHYSICAL=506, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=88, SHUFFLE_BYTES_DECOMPRESSED=84, SHUFFLE_BYTES_TO_MEM=88, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=188, FIRST_EVENT_RECEIVED=185, LAST_EVENT_RECEIVED=185, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500, RECORDS_OUT_INTERMEDIATE_Map_1=37, org.apache.hadoop.hive.llap.counters.LlapIOCounters, CACHE_MISS_BYTES=5812, CONSUMER_TIME_NS=1615888, DECODE_TIME_NS=621074, NUM_DECODED_BATCHES=1, NUM_VECTOR_BATCHES=1, ROWS_EMITTED=500, TOTAL_IO_TIME_NS=47469300 2018-01-05T11:29:04,190 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_34_02_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:29:04,190 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_34_02_000000_0 2018-01-05T11:29:04,191 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_34_02 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:04,191 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_34_02 [Map 1] 2018-01-05T11:29:04,191 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_34_02, initRequestedTime=1515180543872, initedTime=1515180543928, startRequestedTime=1515180543905, startedTime=1515180543928, finishTime=1515180544191, timeTaken=263, status=SUCCEEDED, diagnostics=, counters=Counters: 37, File System Counters, FILE_BYTES_READ=5936, FILE_BYTES_WRITTEN=538, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=2, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=37, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=549, OUTPUT_BYTES_WITH_OVERHEAD=502, OUTPUT_BYTES_PHYSICAL=506, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=88, SHUFFLE_BYTES_DECOMPRESSED=84, SHUFFLE_BYTES_TO_MEM=88, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=188, FIRST_EVENT_RECEIVED=185, LAST_EVENT_RECEIVED=185, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500, RECORDS_OUT_INTERMEDIATE_Map_1=37, org.apache.hadoop.hive.llap.counters.LlapIOCounters, CACHE_MISS_BYTES=5812, CONSUMER_TIME_NS=1615888, DECODE_TIME_NS=621074, NUM_DECODED_BATCHES=1, NUM_VECTOR_BATCHES=1, ROWS_EMITTED=500, TOTAL_IO_TIME_NS=47469300, vertexStats=firstTaskStartTime=1515180543934, firstTasksToStart=[ task_1515180518813_0001_34_02_000000 ], lastTaskFinishTime=1515180544190, lastTasksToFinish=[ task_1515180518813_0001_34_02_000000 ], minTaskDuration=256, maxTaskDuration=256, avgTaskDuration=256.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_34_02_000000 ], longestDurationTasks=[ task_1515180518813_0001_34_02_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:04,191 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_34_02 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:04,191 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000069, stoppedTime=1515180544191, exitStatus=0 2018-01-05T11:29:04,191 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000069 exited with diagnostics set to Container completed. 2018-01-05T11:29:04,191 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_34_03 [Reducer 2] attempt: attempt_1515180518813_0001_34_02_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:29:04,191 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_34_02 [Map 1] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=4 2018-01-05T11:29:04,191 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=4, commitInProgress=0, terminationCause=null 2018-01-05T11:29:04,191 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000069 2018-01-05T11:29:04,191 INFO [App Shared Pool - #1] vertexmanager.ShuffleVertexManagerBase: All source tasks completed. Ramping up 1 remaining tasks for vertex: Reducer 2 2018-01-05T11:29:04,191 INFO [App Shared Pool - #1] vertexmanager.ShuffleVertexManagerBase: Scheduling 1 tasks for vertex: Reducer 2 with totalTasks: 1. 2 source tasks completed out of 2. MinSourceTaskCompletedFraction: 1.0 min: 0.05 max: 0.75 2018-01-05T11:29:04,191 INFO [App Shared Pool - #1] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Map 3, destinationVertexName=Reducer 2. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:29:04,191 INFO [App Shared Pool - #1] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Map 1, destinationVertexName=Reducer 2. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:29:04,191 INFO [App Shared Pool - #1] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_34_03 [Reducer 2] 2018-01-05T11:29:04,191 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_34_03_000000, scheduledTime=1515180544191, launchTime=1515180544191 2018-01-05T11:29:04,191 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_34_03 [Reducer 2] 2018-01-05T11:29:04,191 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_34_03_000000_0, priority=56, capability=, hosts=null 2018-01-05T11:29:04,192 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=56,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:04,192 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=56,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:04,192 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:04,192 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_34_03_000000_0 on hosts=null 2018-01-05T11:29:04,192 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_34_03_000000_0 on hosts=null 2018-01-05T11:29:04,192 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_34_03_000000_0 2018-01-05T11:29:04,192 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:04,192 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #70, task=TaskInfo{task=attempt_1515180518813_0001_34_03_000000_0, priority=56, startTime=0, containerId=null, uniqueId=69, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000070 2018-01-05T11:29:04,192 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_34_03_000000_0; false 2018-01-05T11:29:04,192 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_34_03_000000_0, priority=56, startTime=311564701, containerId=container_111101111_0001_00_000070assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=69, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:04,192 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000070 for node: 10.22.8.182:49818 2018-01-05T11:29:04,192 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000070, attempt_1515180518813_0001_34_03_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:04,192 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000070 for node: 10.22.8.182:49818 2018-01-05T11:29:04,193 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000070 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:04,193 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000070, launchTime=1515180544193 2018-01-05T11:29:04,193 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=34}, sourceName=Map 1, state=S_SUCCEEDED 2018-01-05T11:29:04,193 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_34_03_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000070] on NM: [10.22.8.182:49818] 2018-01-05T11:29:04,193 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:04,193 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:04,193 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_34_03_000000_0, startTime=1515180544193, containerId=container_111101111_0001_00_000070, nodeId=10.22.8.182:49818 2018-01-05T11:29:04,194 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_34_03_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000070, dagName=select * from src a join src1 b on a...b.key (Stage-1), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=2, numOutputs=1, numGroupedInputs=0, Inputs={{Map 3,org.apache.tez.runtime.library.input.OrderedGroupedKVInput,1}{Map 1,org.apache.tez.runtime.library.input.OrderedGroupedKVInput,1}}, Outputs={{out_Reducer 2,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=5, completedTaskCount=4, dagStartTime=1515180543833, firstAttemptStartTime=1515180544191, currentAttemptStartTime=1515180544192} 2018-01-05T11:29:04,194 INFO [IPC Server handler 3 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000070, QueryId=djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d-34, DagId=34, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, SubmitTime=1515180544194 2018-01-05T11:29:04,194 DEBUG [IPC Server handler 3 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=34} with the ShuffleHandler 2018-01-05T11:29:04,194 DEBUG [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/34] 2018-01-05T11:29:04,194 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:29:04,194 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_34_03_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:04,194 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_34_03_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180544191, dagStartTime=1515180543833, withinDagPriority=56, vertexParallelism= 1, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 4}] 2018-01-05T11:29:04,194 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_34_03_000000_0 : ACCEPTED 2018-01-05T11:29:04,194 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_34_03_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:29:04,194 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_34_03_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180544191, dagStartTime=1515180543833, withinDagPriority=56, vertexParallelism= 1, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 4} 2018-01-05T11:29:04,195 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_34_03_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:04,195 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_34_03_000000_0 to pre-emption queue 2018-01-05T11:29:04,195 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_34_03_000000_0: true 2018-01-05T11:29:04,195 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: taskOwner hashCode:797465925 2018-01-05T11:29:04,195 INFO [TaskCommunicator # 5] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_34_03_000000_0 2018-01-05T11:29:04,195 INFO [TaskCommunicator # 5] GuaranteedTasks: Received response for attempt_1515180518813_0001_34_03_000000_0, true 2018-01-05T11:29:04,195 INFO [Task-Executor-1] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : select * from src a join src1 b on a...b.key (Stage-1), VertexName: Reducer 2, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_34_03_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 3, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:29:04,195 INFO [Task-Executor-1] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=2, numOutputs=1, JVM.maxFree=1031274496, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:04,195 INFO [Task-Executor-1] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_34_03_000000_0 vertexName: Reducer 2 2018-01-05T11:29:04,195 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000070, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_34_03_000000_0, eventCount=1 } 2018-01-05T11:29:04,195 DEBUG [TezTR-518813_1_34_3_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_34 queryId: djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d instanceType: TaskRunner2Callable 2018-01-05T11:29:04,195 INFO [TezTR-518813_1_34_3_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_34_03_000000_0 2018-01-05T11:29:04,196 DEBUG [TezTR-518813_1_34_3_0_0] log.PerfLogger: 2018-01-05T11:29:04,196 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:04,196 INFO [I/O Setup 0 Initialize: {Map 3}] orderedgrouped.MergeManager: Initial Memory required for SHUFFLE_BUFFER=131404920 based on INPUT_BUFFER_FACTOR=0.7, for final merged output=0, using factor: 0.0 2018-01-05T11:29:04,196 INFO [I/O Setup 1 Initialize: {Map 1}] orderedgrouped.MergeManager: Initial Memory required for SHUFFLE_BUFFER=131404920 based on INPUT_BUFFER_FACTOR=0.7, for final merged output=0, using factor: 0.0 2018-01-05T11:29:04,197 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:04,197 INFO [IPC Server handler 0 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_34_03_000000_0 2 events [0,2) total 2 vertex_1515180518813_0001_34_03 [Reducer 2] 2018-01-05T11:29:04,197 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=2, nextPreRoutedEventId=0, eventCount=2 } 2018-01-05T11:29:04,197 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_34_03_000000_0, eventCount=2 fromEventId=0 nextFromEventId=2 2018-01-05T11:29:04,198 DEBUG [TezTR-518813_1_34_3_0_0] log.PerfLogger: 2018-01-05T11:29:04,198 INFO [TezTR-518813_1_34_3_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:04,198 INFO [TezTR-518813_1_34_3_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 3 initializers to finish 2018-01-05T11:29:04,199 INFO [TezTR-518813_1_34_3_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:04,199 INFO [TezTR-518813_1_34_3_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:04,199 INFO [I/O Setup 2 Initialize: {out_Reducer 2}] output.MROutput: out_Reducer 2: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:29:04,199 DEBUG [I/O Setup 2 Initialize: {out_Reducer 2}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:04,199 DEBUG [I/O Setup 2 Initialize: {out_Reducer 2}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:04,199 DEBUG [I/O Setup 2 Initialize: {out_Reducer 2}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:04,199 DEBUG [I/O Setup 2 Initialize: {out_Reducer 2}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:04,199 INFO [TezTR-518813_1_34_3_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:04,199 INFO [TezTR-518813_1_34_3_0_0] resources.MemoryDistributor: InitialRequests=[out_Reducer 2:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [Map 3:INPUT:131404920:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.OrderedGroupedKVInput] 2018-01-05T11:29:04,199 INFO [TezTR-518813_1_34_3_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:04,199 INFO [TezTR-518813_1_34_3_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.045, finalReserveFractionUsed=0.345 2018-01-05T11:29:04,199 INFO [TezTR-518813_1_34_3_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 3, numScaledRequests: 25, TotalRequested: 262809840, TotalRequestedScaled: 1.2614872319999999E8, TotalJVMHeap: 1031274496, TotalAvailable: 675484794, TotalRequested/TotalJVMHeap:0.25 2018-01-05T11:29:04,199 INFO [TezTR-518813_1_34_3_0_0] resources.MemoryDistributor: Allocations=[out_Reducer 2:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [Map 3:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:131404920:131404920], [Map 1:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:131404920:131404920] 2018-01-05T11:29:04,199 INFO [TezTR-518813_1_34_3_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:04,199 INFO [TezTR-518813_1_34_3_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: Map 3 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:04,199 INFO [TezTR-518813_1_34_3_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: Map 1 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:04,199 INFO [TezTR-518813_1_34_3_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 2 2018-01-05T11:29:04,199 INFO [TezTR-518813_1_34_3_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 IOs to start 2018-01-05T11:29:04,199 INFO [I/O Setup 0 Start: {Map 3}] orderedgrouped.Shuffle: Map_3: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true 2018-01-05T11:29:04,199 INFO [I/O Setup 1 Start: {Map 1}] orderedgrouped.Shuffle: Map_1: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true 2018-01-05T11:29:04,199 INFO [I/O Setup 0 Start: {Map 3}] orderedgrouped.MergeManager: Map 3: MergerManager: memoryLimit=131404920, maxSingleShuffleLimit=32851230, mergeThreshold=86727248, ioSortFactor=10, postMergeMem=0, memToMemMergeOutputsThreshold=10 2018-01-05T11:29:04,199 INFO [I/O Setup 1 Start: {Map 1}] orderedgrouped.MergeManager: Map 1: MergerManager: memoryLimit=131404920, maxSingleShuffleLimit=32851230, mergeThreshold=86727248, ioSortFactor=10, postMergeMem=0, memToMemMergeOutputsThreshold=10 2018-01-05T11:29:04,200 INFO [I/O Setup 1 Start: {Map 1}] orderedgrouped.ShuffleScheduler: ShuffleScheduler running for sourceVertex: Map 1 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true 2018-01-05T11:29:04,200 INFO [I/O Setup 0 Start: {Map 3}] orderedgrouped.ShuffleScheduler: ShuffleScheduler running for sourceVertex: Map 3 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true 2018-01-05T11:29:04,201 INFO [I/O Setup 1 Start: {Map 1}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: Map 1 2018-01-05T11:29:04,201 INFO [I/O Setup 0 Start: {Map 3}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: Map 3 2018-01-05T11:29:04,202 INFO [TezTR-518813_1_34_3_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:04,202 INFO [TezTR-518813_1_34_3_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:04,202 INFO [TezTR-518813_1_34_3_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_34_03_000000_0 2018-01-05T11:29:04,202 DEBUG [TezTR-518813_1_34_3_0_0] log.PerfLogger: 2018-01-05T11:29:04,202 DEBUG [TezTR-518813_1_34_3_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_34_03_000000_0_10210 2018-01-05T11:29:04,202 DEBUG [TezTR-518813_1_34_3_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:04,202 DEBUG [TezTR-518813_1_34_3_0_0] mr.ObjectCache: Creating djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:29:04,202 INFO [TezTR-518813_1_34_3_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir/1428a0e2-bbf8-44c9-9396-97e6939d46f1/reduce.xml 2018-01-05T11:29:04,202 DEBUG [TezTR-518813_1_34_3_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir/1428a0e2-bbf8-44c9-9396-97e6939d46f1/reduce.xml 2018-01-05T11:29:04,202 INFO [TezTaskEventRouter{attempt_1515180518813_0001_34_03_000000_0}] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 3: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:29:04,202 DEBUG [TezTR-518813_1_34_3_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/djaiswal/_tez_scratch_dir/1428a0e2-bbf8-44c9-9396-97e6939d46f1/reduce.xml 2018-01-05T11:29:04,202 DEBUG [TezTR-518813_1_34_3_0_0] log.PerfLogger: 2018-01-05T11:29:04,202 INFO [TezTR-518813_1_34_3_0_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:29:04,202 DEBUG [TezTR-518813_1_34_3_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:04,202 INFO [TezTaskEventRouter{attempt_1515180518813_0001_34_03_000000_0}] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:29:04,203 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=34&reduce=0&map=attempt_1515180518813_0001_34_00_000000_0_10202 mapId: [attempt_1515180518813_0001_34_00_000000_0_10202] reduceId: [0] jobId: [job_1515180518813_0001] dagId: [34] keepAlive: false 2018-01-05T11:29:04,203 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=34&reduce=0&map=attempt_1515180518813_0001_34_00_000000_0_10202; hash=...zN/QT/wph99cQ 2018-01-05T11:29:04,203 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=34&reduce=0&map=attempt_1515180518813_0001_34_00_000000_0_10202;reply=aHlmt6ht2cYjY 2018-01-05T11:29:04,203 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=34, user='djaiswal', attemptId='attempt_1515180518813_0001_34_00_000000_0_10202'} via loader 2018-01-05T11:29:04,203 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=34, user='djaiswal', attemptId='attempt_1515180518813_0001_34_00_000000_0_10202'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:29:04,203 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_34_00_000000_0_10202 not found 2018-01-05T11:29:04,204 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_34_00_000000_0_10202,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/34/output/attempt_1515180518813_0001_34_00_000000_0_10202/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/34/output/attempt_1515180518813_0001_34_00_000000_0_10202/file.out.index 2018-01-05T11:29:04,204 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Content Length in shuffle : 372 2018-01-05T11:29:04,204 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=34&reduce=0&map=attempt_1515180518813_0001_34_02_000000_0_10209 mapId: [attempt_1515180518813_0001_34_02_000000_0_10209] reduceId: [0] jobId: [job_1515180518813_0001] dagId: [34] keepAlive: false 2018-01-05T11:29:04,204 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=34&reduce=0&map=attempt_1515180518813_0001_34_02_000000_0_10209; hash=...lIByWduXR14jM 2018-01-05T11:29:04,204 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=34&reduce=0&map=attempt_1515180518813_0001_34_02_000000_0_10209;reply=30YVfLPTUDYlo 2018-01-05T11:29:04,204 INFO [Fetcher_O {Map_3} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=34&reduce=0&map=attempt_1515180518813_0001_34_00_000000_0_10202 sent hash and receievd reply 0 ms 2018-01-05T11:29:04,205 INFO [Fetcher_O {Map_3} #0] orderedgrouped.MergeManager: CloseInMemoryFile. Current state: inMemoryMapOutputs.size=1, commitMemory=0, usedMemory=313. Since last log: count=1, min=313, max=313, total=313, avg=313.0 2018-01-05T11:29:04,205 INFO [Fetcher_O {Map_3} #0] ShuffleScheduler.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_34_00_000000_0_10202} to MEMORY, csize=317, dsize=313, EndTime=1515180544205, TimeTaken=1, Rate=0.30 MB/s 2018-01-05T11:29:04,205 INFO [Fetcher_O {Map_3} #0] orderedgrouped.ShuffleScheduler: All inputs fetched for input vertex : Map 3 2018-01-05T11:29:04,205 INFO [Fetcher_O {Map_3} #0] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:04,205 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=34, user='djaiswal', attemptId='attempt_1515180518813_0001_34_02_000000_0_10209'} via loader 2018-01-05T11:29:04,205 INFO [ShuffleAndMergeRunner {Map_3}] orderedgrouped.ShuffleScheduler: Shutting down FetchScheduler for input: Map_3, wasInterrupted=false 2018-01-05T11:29:04,205 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=34, user='djaiswal', attemptId='attempt_1515180518813_0001_34_02_000000_0_10209'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:29:04,205 DEBUG [New I/O worker #3] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_34_02_000000_0_10209 not found 2018-01-05T11:29:04,205 INFO [ShuffleAndMergeRunner {Map_3}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:04,206 INFO [ShuffleAndMergeRunner {Map_3}] orderedgrouped.ShuffleScheduler: Shutting down fetchers for input: Map_3, shutdown timetaken: 0 ms, hasFetcherExecutorStopped: true 2018-01-05T11:29:04,206 INFO [ShuffleAndMergeRunner {Map_3}] orderedgrouped.MergeManager: TotalInMemFetchStats: count=1, totalSize=313, min=313, max=313, avg=1.0 2018-01-05T11:29:04,206 INFO [ShuffleAndMergeRunner {Map_3}] orderedgrouped.MergeManager: finalMerge with #inMemoryOutputs=1, size=313 and #onDiskOutputs=0, size=0 2018-01-05T11:29:04,206 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_34_02_000000_0_10209,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/34/output/attempt_1515180518813_0001_34_02_000000_0_10209/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/34/output/attempt_1515180518813_0001_34_02_000000_0_10209/file.out.index 2018-01-05T11:29:04,206 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Content Length in shuffle : 561 2018-01-05T11:29:04,206 INFO [Fetcher_O {Map_1} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=34&reduce=0&map=attempt_1515180518813_0001_34_02_000000_0_10209 sent hash and receievd reply 0 ms 2018-01-05T11:29:04,207 INFO [Fetcher_O {Map_1} #0] orderedgrouped.MergeManager: CloseInMemoryFile. Current state: inMemoryMapOutputs.size=1, commitMemory=0, usedMemory=502. Since last log: count=1, min=502, max=502, total=502, avg=502.0 2018-01-05T11:29:04,207 INFO [Fetcher_O {Map_1} #0] ShuffleScheduler.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_34_02_000000_0_10209} to MEMORY, csize=506, dsize=502, EndTime=1515180544207, TimeTaken=1, Rate=0.48 MB/s 2018-01-05T11:29:04,207 INFO [Fetcher_O {Map_1} #0] orderedgrouped.ShuffleScheduler: All inputs fetched for input vertex : Map 1 2018-01-05T11:29:04,207 INFO [Fetcher_O {Map_1} #0] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:04,207 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down FetchScheduler for input: Map_1, wasInterrupted=false 2018-01-05T11:29:04,208 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:04,208 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down fetchers for input: Map_1, shutdown timetaken: 0 ms, hasFetcherExecutorStopped: true 2018-01-05T11:29:04,208 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: TotalInMemFetchStats: count=1, totalSize=502, min=502, max=502, avg=1.0 2018-01-05T11:29:04,208 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: finalMerge with #inMemoryOutputs=1, size=502 and #onDiskOutputs=0, size=0 2018-01-05T11:29:04,212 DEBUG [TezTR-518813_1_34_3_0_0] log.PerfLogger: 2018-01-05T11:29:04,212 INFO [TezTR-518813_1_34_3_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 1.81KB 2018-01-05T11:29:04,212 DEBUG [TezTR-518813_1_34_3_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:04,212 DEBUG [TezTR-518813_1_34_3_0_0] log.PerfLogger: 2018-01-05T11:29:04,212 DEBUG [TezTR-518813_1_34_3_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:04,213 DEBUG [TezTR-518813_1_34_3_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:04,213 DEBUG [TezTR-518813_1_34_3_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:04,213 INFO [TezTR-518813_1_34_3_0_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:29:04,222 INFO [ShuffleAndMergeRunner {Map_3}] orderedgrouped.Shuffle: merge complete for input vertex : Map_3 2018-01-05T11:29:04,222 INFO [ShuffleAndMergeRunner {Map_3}] orderedgrouped.Shuffle: Map_3: Shuffle Runner thread complete 2018-01-05T11:29:04,223 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: merge complete for input vertex : Map_1 2018-01-05T11:29:04,223 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: Map_1: Shuffle Runner thread complete 2018-01-05T11:29:04,223 INFO [TezTR-518813_1_34_3_0_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:29:04,223 INFO [TezTR-518813_1_34_3_0_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:04,223 DEBUG [TezTR-518813_1_34_3_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:04,223 INFO [TezTR-518813_1_34_3_0_0] input.OrderedGroupedKVInput: Map 1: creating ValuesIterator with comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator, keyClass=org.apache.hadoop.hive.ql.io.HiveKey, valClass=org.apache.hadoop.io.BytesWritable 2018-01-05T11:29:04,223 DEBUG [TezTR-518813_1_34_3_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:04,223 DEBUG [TezTR-518813_1_34_3_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:04,223 DEBUG [TezTR-518813_1_34_3_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0] columnTypes=[string] 2018-01-05T11:29:04,223 DEBUG [TezTR-518813_1_34_3_0_0] log.PerfLogger: 2018-01-05T11:29:04,223 INFO [TezTR-518813_1_34_3_0_0] input.OrderedGroupedKVInput: Map 3: creating ValuesIterator with comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator, keyClass=org.apache.hadoop.hive.ql.io.HiveKey, valClass=org.apache.hadoop.io.BytesWritable 2018-01-05T11:29:04,223 DEBUG [TezTR-518813_1_34_3_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:04,223 DEBUG [TezTR-518813_1_34_3_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:04,223 DEBUG [TezTR-518813_1_34_3_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0] columnTypes=[string] 2018-01-05T11:29:04,223 DEBUG [TezTR-518813_1_34_3_0_0] log.PerfLogger: 2018-01-05T11:29:04,223 INFO [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Initializing operator MERGEJOIN[25] 2018-01-05T11:29:04,226 DEBUG [TezTR-518813_1_34_3_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:04,226 DEBUG [TezTR-518813_1_34_3_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[0_VALUE_0, 0_VALUE_1] columnTypes=[string, string] 2018-01-05T11:29:04,228 DEBUG [TezTR-518813_1_34_3_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:04,228 DEBUG [TezTR-518813_1_34_3_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[0_VALUE_0, 0_VALUE_1] columnTypes=[string, string] 2018-01-05T11:29:04,228 DEBUG [TezTR-518813_1_34_3_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:04,228 DEBUG [TezTR-518813_1_34_3_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[1_VALUE_0, 1_VALUE_1] columnTypes=[string, string] 2018-01-05T11:29:04,228 DEBUG [TezTR-518813_1_34_3_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:04,228 DEBUG [TezTR-518813_1_34_3_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[1_VALUE_0, 1_VALUE_1] columnTypes=[string, string] 2018-01-05T11:29:04,228 INFO [TezTR-518813_1_34_3_0_0] exec.CommonJoinOperator: JOIN struct<_col0:string,_col1:string,_col2:string,_col3:string> totalsz = 4 2018-01-05T11:29:04,228 DEBUG [TezTR-518813_1_34_3_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:04,228 DEBUG [TezTR-518813_1_34_3_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[0_VALUE_0, 0_VALUE_1] columnTypes=[string, string] 2018-01-05T11:29:04,229 DEBUG [TezTR-518813_1_34_3_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:04,229 DEBUG [TezTR-518813_1_34_3_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[0_VALUE_0, 0_VALUE_1] columnTypes=[string, string] 2018-01-05T11:29:04,229 DEBUG [TezTR-518813_1_34_3_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:04,229 DEBUG [TezTR-518813_1_34_3_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[0_VALUE_0, 0_VALUE_1] columnTypes=[string, string] 2018-01-05T11:29:04,229 DEBUG [TezTR-518813_1_34_3_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:04,229 DEBUG [TezTR-518813_1_34_3_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[1_VALUE_0, 1_VALUE_1] columnTypes=[string, string] 2018-01-05T11:29:04,229 DEBUG [TezTR-518813_1_34_3_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:04,229 DEBUG [TezTR-518813_1_34_3_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[1_VALUE_0, 1_VALUE_1] columnTypes=[string, string] 2018-01-05T11:29:04,229 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Initialization Done 25 MERGEJOIN 2018-01-05T11:29:04,229 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Operator 25 MERGEJOIN initialized 2018-01-05T11:29:04,229 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Initializing children of 25 MERGEJOIN 2018-01-05T11:29:04,229 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Initializing child 10 FS 2018-01-05T11:29:04,229 INFO [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Initializing operator FS[10] 2018-01-05T11:29:04,229 DEBUG [TezTR-518813_1_34_3_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:04,229 INFO [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@4bb7e20b]:[_col0, _col1, _col2, _col3]:[string, string, string, string]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@5d1ffa3e 2018-01-05T11:29:04,229 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:29:04,229 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Initialization Done 10 FS 2018-01-05T11:29:04,229 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Operator 10 FS initialized 2018-01-05T11:29:04,229 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Initialization Done 10 FS done is reset. 2018-01-05T11:29:04,229 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Initialization Done 25 MERGEJOIN done is reset. 2018-01-05T11:29:04,229 INFO [TezTR-518813_1_34_3_0_0] tez.ReduceRecordProcessor: Id =25 Id =10 <\Children> Id = 25 null<\Parent> <\FS> <\Children> <\Parent> <\MERGEJOIN> 2018-01-05T11:29:04,229 DEBUG [TezTR-518813_1_34_3_0_0] log.PerfLogger: 2018-01-05T11:29:04,229 INFO [TezTR-518813_1_34_3_0_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 2 2018-01-05T11:29:04,230 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:04,230 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:04,230 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:04,230 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:04,230 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:04,230 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:04,230 WARN [TezTR-518813_1_34_3_0_0] objectinspector.StandardStructObjectInspector: Trying to access 2 fields inside a list of 1 elements: [[128]] 2018-01-05T11:29:04,230 WARN [TezTR-518813_1_34_3_0_0] objectinspector.StandardStructObjectInspector: ignoring similar errors. 2018-01-05T11:29:04,232 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:04,232 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:04,232 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:04,232 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:04,232 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:04,232 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:04,232 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:04,232 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:04,232 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:04,232 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:04,232 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:04,232 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:04,232 INFO [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-03_453_7803144398595179850-1/_tmp.-ext-10002/000000_0 2018-01-05T11:29:04,233 INFO [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-03_453_7803144398595179850-1/_task_tmp.-ext-10002/_tmp.000000_0 2018-01-05T11:29:04,233 INFO [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-03_453_7803144398595179850-1/_tmp.-ext-10002/000000_0 2018-01-05T11:29:04,267 INFO [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: FS[10]: records written - 1 2018-01-05T11:29:04,267 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:04,267 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:04,267 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:04,267 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:04,267 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:04,267 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:04,267 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:04,267 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:04,267 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:04,268 INFO [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: FS[10]: records written - 10 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:04,268 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:04,269 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:04,270 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:04,270 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:04,270 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:04,270 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:04,270 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:04,270 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:04,270 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:04,270 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:04,270 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:04,270 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:04,270 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:04,270 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:04,270 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:04,270 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:04,270 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:04,270 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:04,270 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:04,270 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:04,270 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:04,270 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:04,270 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:04,270 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:04,270 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:04,270 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:04,270 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:04,270 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:04,270 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:04,271 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:04,271 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:04,271 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:04,271 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:04,271 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:04,271 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:04,271 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:04,271 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:04,271 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:04,271 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:04,271 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:04,271 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:04,271 DEBUG [TezTR-518813_1_34_3_0_0] log.PerfLogger: 2018-01-05T11:29:04,271 DEBUG [TezTR-518813_1_34_3_0_0] mr.ObjectCache: djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:29:04,272 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: close called for operator MERGEJOIN[25] 2018-01-05T11:29:04,272 INFO [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Closing operator MERGEJOIN[25] 2018-01-05T11:29:04,272 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Cleaning up the operator state 2018-01-05T11:29:04,272 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: Closing child = FS[10] 2018-01-05T11:29:04,272 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: close called for operator FS[10] 2018-01-05T11:29:04,272 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:04,272 INFO [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: Closing operator FS[10] 2018-01-05T11:29:04,272 INFO [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: FS[10]: records written - 37 2018-01-05T11:29:04,273 INFO [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: RECORDS_OUT_0:37, 2018-01-05T11:29:04,273 DEBUG [TezTR-518813_1_34_3_0_0] exec.FileSinkOperator: 10 Close done 2018-01-05T11:29:04,273 DEBUG [TezTR-518813_1_34_3_0_0] exec.CommonMergeJoinOperator: 25 Close done 2018-01-05T11:29:04,273 INFO [TezTR-518813_1_34_3_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_34_03_000000_0 2018-01-05T11:29:04,273 INFO [TezTR-518813_1_34_3_0_0] orderedgrouped.Shuffle: Shutting down Shuffle for source: Map_3 2018-01-05T11:29:04,273 INFO [TezTR-518813_1_34_3_0_0] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 3: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:29:04,273 INFO [TezTR-518813_1_34_3_0_0] orderedgrouped.Shuffle: Shutting down Shuffle for source: Map_1 2018-01-05T11:29:04,273 INFO [TezTR-518813_1_34_3_0_0] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:29:04,273 INFO [TezTR-518813_1_34_3_0_0] output.MROutput: out_Reducer 2 closed 2018-01-05T11:29:04,273 DEBUG [TezTR-518813_1_34_3_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:04,273 INFO [TezTR-518813_1_34_3_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:04,273 INFO [TezTR-518813_1_34_3_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_34_03_000000_0, askedToStop=false 2018-01-05T11:29:04,273 INFO [TezTR-518813_1_34_3_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_34_03_000000_0, stopRequested=false 2018-01-05T11:29:04,273 INFO [TezTR-518813_1_34_3_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_34_03_000000_0: Counters: 30 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=31, REDUCE_INPUT_RECORDS=62, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=62, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=823, ADDITIONAL_SPILLS_BYTES_READ=823, SHUFFLE_BYTES=823, SHUFFLE_BYTES_DECOMPRESSED=815, SHUFFLE_BYTES_TO_MEM=823, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=16, MERGE_PHASE_TIME=47, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][HIVE CREATED_FILES=1, RECORDS_OUT_0=37][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0]] 2018-01-05T11:29:04,273 DEBUG [TezTR-518813_1_34_3_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_34_3_0_0 thread id: 2093 scheme: file bytesRead: 0 bytesWritten: 1371 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:04,274 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_34_03_000000_0, isTaskDone=true 2018-01-05T11:29:04,274 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000070, requestId=2, startIndex=2, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_34_03_000000_0, eventCount=2 } 2018-01-05T11:29:04,275 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_34_03_000000_0, creationTime=1515180544191, allocationTime=1515180544192, startTime=1515180544193, finishTime=1515180544275, timeTaken=82, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:04,275 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_34_03_000000_0 2018-01-05T11:29:04,275 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_34_03_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:04,275 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_34_03_000000_0; false 2018-01-05T11:29:04,275 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_34_03_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:04,275 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_34_03_000000, startTime=1515180544193, finishTime=1515180544275, timeTaken=82, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_34_03_000000_0, diagnostics=, counters=Counters: 35, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=1371, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=31, REDUCE_INPUT_RECORDS=62, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=62, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=823, ADDITIONAL_SPILLS_BYTES_READ=823, SHUFFLE_BYTES=823, SHUFFLE_BYTES_DECOMPRESSED=815, SHUFFLE_BYTES_TO_MEM=823, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=16, MERGE_PHASE_TIME=47, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, HIVE, CREATED_FILES=1, RECORDS_OUT_0=37, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0 2018-01-05T11:29:04,275 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:04,275 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=2, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:04,275 INFO [Task-Executor-1] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_34_03_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:04,275 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:04,275 INFO [Task-Executor-1] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_34_03_000000_0 2018-01-05T11:29:04,275 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_34_03 [Reducer 2], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:04,275 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_34_03 [Reducer 2] 2018-01-05T11:29:04,276 INFO [Task-Executor-1] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000070=81 2018-01-05T11:29:04,276 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_34_03_000000_0: true 2018-01-05T11:29:04,276 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:VERTEX_FINISHED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_34_03, initRequestedTime=1515180543898, initedTime=1515180543904, startRequestedTime=1515180543928, startedTime=1515180543929, finishTime=1515180544275, timeTaken=346, status=SUCCEEDED, diagnostics=, counters=Counters: 35, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=1371, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=31, REDUCE_INPUT_RECORDS=62, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=62, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=823, ADDITIONAL_SPILLS_BYTES_READ=823, SHUFFLE_BYTES=823, SHUFFLE_BYTES_DECOMPRESSED=815, SHUFFLE_BYTES_TO_MEM=823, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=16, MERGE_PHASE_TIME=47, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, HIVE, CREATED_FILES=1, RECORDS_OUT_0=37, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, vertexStats=firstTaskStartTime=1515180544193, firstTasksToStart=[ task_1515180518813_0001_34_03_000000 ], lastTaskFinishTime=1515180544275, lastTasksToFinish=[ task_1515180518813_0001_34_03_000000 ], minTaskDuration=82, maxTaskDuration=82, avgTaskDuration=82.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_34_03_000000 ], longestDurationTasks=[ task_1515180518813_0001_34_03_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:04,276 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_34_03 [Reducer 2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:04,276 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000070, stoppedTime=1515180544276, exitStatus=0 2018-01-05T11:29:04,276 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000070 exited with diagnostics set to Container completed. 2018-01-05T11:29:04,276 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_34_03 [Reducer 2] completed., numCompletedVertices=4, numSuccessfulVertices=4, numFailedVertices=0, numKilledVertices=0, numVertices=4 2018-01-05T11:29:04,276 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=4, numSuccessfulVertices=4, numFailedVertices=0, numKilledVertices=0, numVertices=4, commitInProgress=0, terminationCause=null 2018-01-05T11:29:04,276 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_34_00 [Map 3] 2018-01-05T11:29:04,276 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_34_03 [Reducer 2] 2018-01-05T11:29:04,276 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_34_01 [Reducer 4] 2018-01-05T11:29:04,276 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_34_02 [Map 1] 2018-01-05T11:29:04,277 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_34, queueSize=0 2018-01-05T11:29:04,278 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_34][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_34, startTime=1515180543833, finishTime=1515180544276, timeTaken=443, status=SUCCEEDED, diagnostics=, counters=Counters: 61, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=4, TOTAL_LAUNCHED_TASKS=4, OTHER_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=1521, AM_GC_TIME_MILLIS=0, File System Counters, FILE_BYTES_READ=6232, FILE_BYTES_WRITTEN=2498, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=31, REDUCE_INPUT_RECORDS=62, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=124, NUM_SHUFFLED_INPUTS=4, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, INPUT_RECORDS_PROCESSED=4, INPUT_SPLIT_LENGTH_BYTES=6028, OUTPUT_RECORDS=64, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=992, OUTPUT_BYTES_WITH_OVERHEAD=983, OUTPUT_BYTES_PHYSICAL=1047, ADDITIONAL_SPILLS_BYTES_WRITTEN=823, ADDITIONAL_SPILLS_BYTES_READ=823, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=2, SHUFFLE_BYTES=999, SHUFFLE_BYTES_DECOMPRESSED=983, SHUFFLE_BYTES_TO_MEM=999, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=303, MERGE_PHASE_TIME=47, FIRST_EVENT_RECEIVED=287, LAST_EVENT_RECEIVED=287, HIVE, CREATED_FILES=1, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500, RECORDS_IN_Map_3=25, RECORDS_OUT_0=37, RECORDS_OUT_INTERMEDIATE_Map_1=37, RECORDS_OUT_INTERMEDIATE_Map_3=26, RECORDS_OUT_INTERMEDIATE_Reducer_4=1, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, org.apache.hadoop.hive.llap.counters.LlapIOCounters, CACHE_MISS_BYTES=6028, CONSUMER_TIME_NS=4365570, DECODE_TIME_NS=1262071, NUM_DECODED_BATCHES=2, NUM_VECTOR_BATCHES=2, ROWS_EMITTED=525, TOTAL_IO_TIME_NS=94937137 2018-01-05T11:29:04,278 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_34 finished with state: SUCCEEDED 2018-01-05T11:29:04,278 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_34 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:29:04,278 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000070 2018-01-05T11:29:04,278 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_34, dagState=SUCCEEDED 2018-01-05T11:29:04,278 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_34_post 2018-01-05T11:29:04,278 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:29:04,278 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:29:04,278 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:29:04,278 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 34 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=4, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=4,NumRequestsWithoutLocation=4, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=4} 2018-01-05T11:29:04,278 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:29:04,278 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=select * from src a join src1 b on a...b.key (Stage-1), with id=dag_1515180518813_0001_34 2018-01-05T11:29:04,278 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 34, to 10.22.8.182:49818 2018-01-05T11:29:04,279 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=4, Remaining=0 2018-01-05T11:29:04,279 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=select * from src a join src1 b on a...b.key (Stage-1), with id=dag_1515180518813_0001_34 2018-01-05T11:29:04,279 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=34} 2018-01-05T11:29:04,279 INFO [IPC Server handler 1 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=34}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:29:04,279 INFO [IPC Server handler 1 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/34 after 300 seconds 2018-01-05T11:29:04,279 ERROR [IPC Server handler 1 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d, dag_1515180518813_0001_34 2018-01-05T11:29:04,279 INFO [IPC Server handler 1 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d 2018-01-05T11:29:04,279 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=34} = 1 2018-01-05T11:29:04,279 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Issuing killFragment for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=34} attempt_1515180518813_0001_34_03_000000_0 2018-01-05T11:29:04,279 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Removing attempt_1515180518813_0001_34_03_000000_0 from preemptionQueue 2018-01-05T11:29:04,279 INFO [IPC Server handler 1 on 49818] impl.TaskRunnerCallable: Ignoring kill request for task attempt_1515180518813_0001_34_03_000000_0 since it's already complete 2018-01-05T11:29:04,279 DEBUG [IPC Server handler 1 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=34} 2018-01-05T11:29:04,279 DEBUG [IPC Server handler 1 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:29:04,279 DEBUG [IPC Server handler 1 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=34} 2018-01-05T11:29:04,281 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,281 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,281 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,281 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 Map 3: 1/1 Reducer 2: 1/1 Reducer 4: 1/1 2018-01-05T11:29:04,281 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: TaskId for 000000_0 = 000000 2018-01-05T11:29:04,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,285 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: select * from src a join src1 b on a.key = b.key 2018-01-05T11:29:04,285 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:29:04,285 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@src 2018-01-05T11:29:04,285 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@src1 2018-01-05T11:29:04,285 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/-mr-10001 2018-01-05T11:29:04,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,285 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:04,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {} 2018-01-05T11:29:04,285 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d); Time taken: 0.674 seconds 2018-01-05T11:29:04,285 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:04,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,285 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:04,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query select * from src a join src1 b on a.key = b.key 2018-01-05T11:29:04,295 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000069, QueryId=djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d-34, DagId=34, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-1, Succeeded=true, StartTime=1515180543937, EndTime=1515180544295 2018-01-05T11:29:04,295 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_34_03_000000_0 2018-01-05T11:29:04,295 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_34_03_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:29:04,295 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_34_03_000000_0 2018-01-05T11:29:04,295 INFO [ExecutionCompletionThread #0] impl.QueryTracker: Ignoring fragmentComplete message for unknown query: QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=34} 2018-01-05T11:29:04,306 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] mapred.FileInputFormat: Time taken to get FileStatuses: 20 2018-01-05T11:29:04,306 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:29:04,306 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 20 2018-01-05T11:29:04,307 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FetchOperator: Creating fetchTask with deserializer typeinfo: struct<_col0:string,_col1:string,_col2:string,_col3:string> 2018-01-05T11:29:04,307 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FetchOperator: deserializer properties: table properties: {columns=_col0,_col1,_col2,_col3, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=string:string:string:string, escape.delim=\} partition properties: {columns=_col0,_col1,_col2,_col3, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=string:string:string:string, escape.delim=\} 2018-01-05T11:29:04,308 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: close called for operator LIST_SINK[37] 2018-01-05T11:29:04,308 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: Closing operator LIST_SINK[37] 2018-01-05T11:29:04,308 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: 37 Close done 2018-01-05T11:29:04,309 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting result dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/-mr-10001 2018-01-05T11:29:04,310 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-03_453_7803144398595179850-1 2018-01-05T11:29:04,310 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-03_453_7803144398595179850-1 2018-01-05T11:29:04,314 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.832 seconds, Fetched: 37 row(s) 2018-01-05T11:29:04,314 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:04,314 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:04,314 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:04,314 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:04,314 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,314 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:04,314 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,314 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,314 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,315 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: CREATE TABLE srcbucket_mapjoin(key int, value string) partitioned by (ds string) CLUSTERED BY (key) INTO 2 BUCKETS STORED AS TEXTFILE 2018-01-05T11:29:04,315 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112904_5a6d1b8f-a53e-4884-8923-deecce44e071): CREATE TABLE srcbucket_mapjoin(key int, value string) partitioned by (ds string) CLUSTERED BY (key) INTO 2 BUCKETS STORED AS TEXTFILE 2018-01-05T11:29:04,315 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,315 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: CREATE TABLE srcbucket_mapjoin(key int, value string) partitioned by (ds string) CLUSTERED BY (key) INTO 2 BUCKETS STORED AS TEXTFILE 2018-01-05T11:29:04,315 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:04,315 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,315 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,315 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,315 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,316 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:04,316 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:29:04,316 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Creating table default.srcbucket_mapjoin position=13 2018-01-05T11:29:04,316 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,316 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_database: default 2018-01-05T11:29:04,316 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_database: default 2018-01-05T11:29:04,323 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:29:04,323 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,323 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,323 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:04,323 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: validation start 2018-01-05T11:29:04,323 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:29:04,323 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Not a partition. 2018-01-05T11:29:04,323 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,323 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:29:04,323 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,323 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:04,323 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getDatabase_(String, )=7, flushCache_()=0} 2018-01-05T11:29:04,323 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112904_5a6d1b8f-a53e-4884-8923-deecce44e071); Time taken: 0.009 seconds 2018-01-05T11:29:04,323 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:04,323 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,323 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112904_5a6d1b8f-a53e-4884-8923-deecce44e071): CREATE TABLE srcbucket_mapjoin(key int, value string) partitioned by (ds string) CLUSTERED BY (key) INTO 2 BUCKETS STORED AS TEXTFILE 2018-01-05T11:29:04,324 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,325 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: CREATE TABLE srcbucket_mapjoin(key int, value string) partitioned by (ds string) CLUSTERED BY (key) INTO 2 BUCKETS STORED AS TEXTFILE 2018-01-05T11:29:04,325 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: CREATETABLE 2018-01-05T11:29:04,325 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: database:default 2018-01-05T11:29:04,325 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: default@srcbucket_mapjoin 2018-01-05T11:29:04,325 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,325 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,325 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,325 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,325 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:29:04,325 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.DDLTask: Found class for org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe 2018-01-05T11:29:04,325 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.DDLTask: creating table default.srcbucket_mapjoin on file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin 2018-01-05T11:29:04,325 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,325 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: create_table: Table(tableName:srcbucket_mapjoin, dbName:default, owner:hive_test_user, createTime:1515180544, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:int, comment:null), FieldSchema(name:value, type:string, comment:null)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:2, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format=1}), bucketCols:[key], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[FieldSchema(name:ds, type:string, comment:null)], parameters:{}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:29:04,325 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=create_table: Table(tableName:srcbucket_mapjoin, dbName:default, owner:hive_test_user, createTime:1515180544, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:int, comment:null), FieldSchema(name:value, type:string, comment:null)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:2, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format=1}), bucketCols:[key], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[FieldSchema(name:ds, type:string, comment:null)], parameters:{}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:29:04,327 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hive-site.xml 2018-01-05T11:29:04,327 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hivemetastore-site.xml 2018-01-05T11:29:04,327 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.MetastoreConf: Unable to find config file metastore-site.xml 2018-01-05T11:29:04,327 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.MetastoreConf: Found configuration file null 2018-01-05T11:29:04,339 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:29:04,339 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: Location: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin specified for non-external table:srcbucket_mapjoin 2018-01-05T11:29:04,339 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] utils.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin 2018-01-05T11:29:04,344 DEBUG [AMReporterQueueDrainer] impl.AMReporter: Removing am 10.22.8.182:49824 with last associated dag QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=19} from heartbeat with taskCount=0, amFailed=false, isDone=true 2018-01-05T11:29:04,364 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,366 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,387 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,388 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,388 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.DDLTask: Ignoring request to add WriteEntity(default@srcbucket_mapjoin) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@srcbucket_mapjoin) Type=TABLE WriteType=DDL_NO_LOCK is present 2018-01-05T11:29:04,388 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,388 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,388 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: CREATE TABLE srcbucket_mapjoin(key int, value string) partitioned by (ds string) CLUSTERED BY (key) INTO 2 BUCKETS STORED AS TEXTFILE 2018-01-05T11:29:04,388 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: CREATETABLE 2018-01-05T11:29:04,388 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: database:default 2018-01-05T11:29:04,388 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: default@srcbucket_mapjoin 2018-01-05T11:29:04,388 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,388 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,388 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:04,388 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {createTable_(Table, )=63, isCompatibleWith_(Configuration, )=0} 2018-01-05T11:29:04,388 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112904_5a6d1b8f-a53e-4884-8923-deecce44e071); Time taken: 0.065 seconds 2018-01-05T11:29:04,388 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:04,388 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,388 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:04,388 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,388 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query CREATE TABLE srcbucket_mapjoin(key int, value string) partitioned by (ds string) CLUSTERED BY (key) INTO 2 BUCKETS STORED AS TEXTFILE 2018-01-05T11:29:04,388 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.074 seconds 2018-01-05T11:29:04,388 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:04,388 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:04,388 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:04,388 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:04,389 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,389 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:04,389 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,389 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,389 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,389 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: CREATE TABLE tab_part (key int, value string) PARTITIONED BY(ds STRING) CLUSTERED BY (key) SORTED BY (key) INTO 4 BUCKETS STORED AS ORCFILE 2018-01-05T11:29:04,389 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112904_a93ce1d8-b5d7-46d3-8ea4-fedb02e95601): CREATE TABLE tab_part (key int, value string) PARTITIONED BY(ds STRING) CLUSTERED BY (key) SORTED BY (key) INTO 4 BUCKETS STORED AS ORCFILE 2018-01-05T11:29:04,389 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,389 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: CREATE TABLE tab_part (key int, value string) PARTITIONED BY(ds STRING) CLUSTERED BY (key) SORTED BY (key) INTO 4 BUCKETS STORED AS ORCFILE 2018-01-05T11:29:04,389 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:04,389 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,389 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,390 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:04,390 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:29:04,390 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Creating table default.tab_part position=13 2018-01-05T11:29:04,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,392 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_database: default 2018-01-05T11:29:04,392 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_database: default 2018-01-05T11:29:04,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:29:04,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,393 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:04,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: validation start 2018-01-05T11:29:04,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:29:04,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Not a partition. 2018-01-05T11:29:04,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,393 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:29:04,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,393 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:04,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getDatabase_(String, )=1, flushCache_()=0} 2018-01-05T11:29:04,393 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112904_a93ce1d8-b5d7-46d3-8ea4-fedb02e95601); Time taken: 0.004 seconds 2018-01-05T11:29:04,393 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:04,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,393 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112904_a93ce1d8-b5d7-46d3-8ea4-fedb02e95601): CREATE TABLE tab_part (key int, value string) PARTITIONED BY(ds STRING) CLUSTERED BY (key) SORTED BY (key) INTO 4 BUCKETS STORED AS ORCFILE 2018-01-05T11:29:04,395 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,395 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: CREATE TABLE tab_part (key int, value string) PARTITIONED BY(ds STRING) CLUSTERED BY (key) SORTED BY (key) INTO 4 BUCKETS STORED AS ORCFILE 2018-01-05T11:29:04,395 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: CREATETABLE 2018-01-05T11:29:04,395 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: database:default 2018-01-05T11:29:04,395 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: default@tab_part 2018-01-05T11:29:04,395 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,395 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,395 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,395 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,395 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:29:04,395 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.DDLTask: Found class for org.apache.hadoop.hive.ql.io.orc.OrcSerde 2018-01-05T11:29:04,395 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.DDLTask: creating table default.tab_part on file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part 2018-01-05T11:29:04,395 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,395 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: create_table: Table(tableName:tab_part, dbName:default, owner:hive_test_user, createTime:1515180544, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:int, comment:null), FieldSchema(name:value, type:string, comment:null)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, inputFormat:org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, compressed:false, numBuckets:4, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.ql.io.orc.OrcSerde, parameters:{serialization.format=1}), bucketCols:[key], sortCols:[Order(col:key, order:1)], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[FieldSchema(name:ds, type:string, comment:null)], parameters:{SORTBUCKETCOLSPREFIX=TRUE}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:29:04,395 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=create_table: Table(tableName:tab_part, dbName:default, owner:hive_test_user, createTime:1515180544, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:int, comment:null), FieldSchema(name:value, type:string, comment:null)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, inputFormat:org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, compressed:false, numBuckets:4, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.ql.io.orc.OrcSerde, parameters:{serialization.format=1}), bucketCols:[key], sortCols:[Order(col:key, order:1)], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[FieldSchema(name:ds, type:string, comment:null)], parameters:{SORTBUCKETCOLSPREFIX=TRUE}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:29:04,397 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hive-site.xml 2018-01-05T11:29:04,397 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hivemetastore-site.xml 2018-01-05T11:29:04,398 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.MetastoreConf: Unable to find config file metastore-site.xml 2018-01-05T11:29:04,398 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.MetastoreConf: Found configuration file null 2018-01-05T11:29:04,399 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000070, QueryId=djaiswal_20180105112903_e124d557-559c-4ad0-ab5a-556217ea980d-34, DagId=34, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-1, Succeeded=true, StartTime=1515180544195, EndTime=1515180544399 2018-01-05T11:29:04,410 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:29:04,410 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: Location: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part specified for non-external table:tab_part 2018-01-05T11:29:04,410 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] utils.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part 2018-01-05T11:29:04,425 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,427 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,435 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,438 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,439 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,439 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.DDLTask: Ignoring request to add WriteEntity(default@tab_part) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@tab_part) Type=TABLE WriteType=DDL_NO_LOCK is present 2018-01-05T11:29:04,439 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,439 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,439 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: CREATE TABLE tab_part (key int, value string) PARTITIONED BY(ds STRING) CLUSTERED BY (key) SORTED BY (key) INTO 4 BUCKETS STORED AS ORCFILE 2018-01-05T11:29:04,439 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: CREATETABLE 2018-01-05T11:29:04,439 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: database:default 2018-01-05T11:29:04,439 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: default@tab_part 2018-01-05T11:29:04,439 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,439 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,439 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:04,439 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {createTable_(Table, )=44, isCompatibleWith_(Configuration, )=0} 2018-01-05T11:29:04,439 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112904_a93ce1d8-b5d7-46d3-8ea4-fedb02e95601); Time taken: 0.046 seconds 2018-01-05T11:29:04,439 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:04,439 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,439 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:04,439 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,439 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query CREATE TABLE tab_part (key int, value string) PARTITIONED BY(ds STRING) CLUSTERED BY (key) SORTED BY (key) INTO 4 BUCKETS STORED AS ORCFILE 2018-01-05T11:29:04,439 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.05 seconds 2018-01-05T11:29:04,439 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:04,439 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:04,439 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:04,439 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:04,439 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,440 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:04,440 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,440 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,440 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,440 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: CREATE TABLE srcbucket_mapjoin_part (key int, value string) partitioned by (ds string) CLUSTERED BY (key) INTO 4 BUCKETS STORED AS TEXTFILE 2018-01-05T11:29:04,440 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112904_bc6409c6-4a72-4db6-ab9b-a3682b5ddc12): CREATE TABLE srcbucket_mapjoin_part (key int, value string) partitioned by (ds string) CLUSTERED BY (key) INTO 4 BUCKETS STORED AS TEXTFILE 2018-01-05T11:29:04,440 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,440 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: CREATE TABLE srcbucket_mapjoin_part (key int, value string) partitioned by (ds string) CLUSTERED BY (key) INTO 4 BUCKETS STORED AS TEXTFILE 2018-01-05T11:29:04,440 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:04,440 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,440 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,440 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,441 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,441 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:04,441 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:29:04,441 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Creating table default.srcbucket_mapjoin_part position=13 2018-01-05T11:29:04,441 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,441 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_database: default 2018-01-05T11:29:04,441 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_database: default 2018-01-05T11:29:04,441 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:29:04,441 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,441 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,442 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:04,442 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: validation start 2018-01-05T11:29:04,442 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:29:04,442 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Not a partition. 2018-01-05T11:29:04,442 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,442 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:29:04,442 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,442 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:04,442 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getDatabase_(String, )=0, flushCache_()=1} 2018-01-05T11:29:04,442 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112904_bc6409c6-4a72-4db6-ab9b-a3682b5ddc12); Time taken: 0.002 seconds 2018-01-05T11:29:04,442 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:04,442 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,442 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112904_bc6409c6-4a72-4db6-ab9b-a3682b5ddc12): CREATE TABLE srcbucket_mapjoin_part (key int, value string) partitioned by (ds string) CLUSTERED BY (key) INTO 4 BUCKETS STORED AS TEXTFILE 2018-01-05T11:29:04,443 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,443 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: CREATE TABLE srcbucket_mapjoin_part (key int, value string) partitioned by (ds string) CLUSTERED BY (key) INTO 4 BUCKETS STORED AS TEXTFILE 2018-01-05T11:29:04,443 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: CREATETABLE 2018-01-05T11:29:04,443 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: database:default 2018-01-05T11:29:04,443 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: default@srcbucket_mapjoin_part 2018-01-05T11:29:04,443 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,443 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,443 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,443 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,444 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:29:04,444 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.DDLTask: Found class for org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe 2018-01-05T11:29:04,444 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.DDLTask: creating table default.srcbucket_mapjoin_part on file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part 2018-01-05T11:29:04,444 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,444 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: create_table: Table(tableName:srcbucket_mapjoin_part, dbName:default, owner:hive_test_user, createTime:1515180544, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:int, comment:null), FieldSchema(name:value, type:string, comment:null)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:4, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format=1}), bucketCols:[key], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[FieldSchema(name:ds, type:string, comment:null)], parameters:{}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:29:04,444 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=create_table: Table(tableName:srcbucket_mapjoin_part, dbName:default, owner:hive_test_user, createTime:1515180544, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:int, comment:null), FieldSchema(name:value, type:string, comment:null)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part, inputFormat:org.apache.hadoop.mapred.TextInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, compressed:false, numBuckets:4, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, parameters:{serialization.format=1}), bucketCols:[key], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[FieldSchema(name:ds, type:string, comment:null)], parameters:{}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:29:04,446 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hive-site.xml 2018-01-05T11:29:04,446 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hivemetastore-site.xml 2018-01-05T11:29:04,447 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.MetastoreConf: Unable to find config file metastore-site.xml 2018-01-05T11:29:04,447 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.MetastoreConf: Found configuration file null 2018-01-05T11:29:04,461 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:29:04,461 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: Location: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part specified for non-external table:srcbucket_mapjoin_part 2018-01-05T11:29:04,461 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] utils.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part 2018-01-05T11:29:04,484 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,486 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,486 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,486 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.DDLTask: Ignoring request to add WriteEntity(default@srcbucket_mapjoin_part) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@srcbucket_mapjoin_part) Type=TABLE WriteType=DDL_NO_LOCK is present 2018-01-05T11:29:04,486 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,486 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,486 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: CREATE TABLE srcbucket_mapjoin_part (key int, value string) partitioned by (ds string) CLUSTERED BY (key) INTO 4 BUCKETS STORED AS TEXTFILE 2018-01-05T11:29:04,486 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: CREATETABLE 2018-01-05T11:29:04,486 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: database:default 2018-01-05T11:29:04,486 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: default@srcbucket_mapjoin_part 2018-01-05T11:29:04,486 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,486 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,487 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:04,487 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {createTable_(Table, )=42, isCompatibleWith_(Configuration, )=0} 2018-01-05T11:29:04,487 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112904_bc6409c6-4a72-4db6-ab9b-a3682b5ddc12); Time taken: 0.044 seconds 2018-01-05T11:29:04,487 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:04,487 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,487 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:04,487 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,487 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query CREATE TABLE srcbucket_mapjoin_part (key int, value string) partitioned by (ds string) CLUSTERED BY (key) INTO 4 BUCKETS STORED AS TEXTFILE 2018-01-05T11:29:04,487 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.048 seconds 2018-01-05T11:29:04,487 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:04,487 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:04,487 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:04,487 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:04,487 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,487 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:04,487 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,487 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,487 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,487 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: load data local inpath '../../data/files/srcbucket20.txt' INTO TABLE srcbucket_mapjoin partition(ds='2008-04-08') 2018-01-05T11:29:04,487 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112904_3d9003e2-2833-4223-ac1a-475e9aa555ce): load data local inpath '../../data/files/srcbucket20.txt' INTO TABLE srcbucket_mapjoin partition(ds='2008-04-08') 2018-01-05T11:29:04,487 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,487 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: load data local inpath '../../data/files/srcbucket20.txt' INTO TABLE srcbucket_mapjoin partition(ds='2008-04-08') 2018-01-05T11:29:04,487 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:04,487 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,487 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,487 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,487 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,487 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:04,500 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.LoadSemanticAnalyzer: file@null@/Users/djaiswal/hive/hive/data/files/srcbucket20.txt 2018-01-05T11:29:04,500 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,500 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:04,500 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:04,507 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,507 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,508 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,508 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcbucket_mapjoin[2008-04-08] 2018-01-05T11:29:04,508 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcbucket_mapjoin[2008-04-08] 2018-01-05T11:29:04,510 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,516 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,517 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,518 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,518 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcbucket_mapjoin[2008-04-08] 2018-01-05T11:29:04,518 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcbucket_mapjoin[2008-04-08] 2018-01-05T11:29:04,521 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,523 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,523 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,523 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:04,523 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,523 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:29:04,523 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,523 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:04,523 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=7, flushCache_()=0} 2018-01-05T11:29:04,523 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112904_3d9003e2-2833-4223-ac1a-475e9aa555ce); Time taken: 0.036 seconds 2018-01-05T11:29:04,523 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:04,523 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,523 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112904_3d9003e2-2833-4223-ac1a-475e9aa555ce): load data local inpath '../../data/files/srcbucket20.txt' INTO TABLE srcbucket_mapjoin partition(ds='2008-04-08') 2018-01-05T11:29:04,524 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,524 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: load data local inpath '../../data/files/srcbucket20.txt' INTO TABLE srcbucket_mapjoin partition(ds='2008-04-08') 2018-01-05T11:29:04,524 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: LOAD 2018-01-05T11:29:04,525 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/srcbucket20.txt 2018-01-05T11:29:04,525 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: default@srcbucket_mapjoin 2018-01-05T11:29:04,525 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,525 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,525 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,525 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,525 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-0:MOVE] in serial mode 2018-01-05T11:29:04,525 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@2cc7e1f3, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:29:04,525 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:04,525 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,525 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:29:04,525 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:29:04,525 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:04,525 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:04,525 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,525 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,525 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:04,525 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:04,525 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,525 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:04,525 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,525 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:04,525 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:04,525 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,525 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Loading data to table default.srcbucket_mapjoin partition (ds=2008-04-08) from file:/Users/djaiswal/hive/hive/data/files/srcbucket20.txt 2018-01-05T11:29:04,526 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,526 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:29:04,526 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:29:04,526 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:29:04,526 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:29:04,526 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,526 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,526 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,527 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,527 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,527 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:29:04,528 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:29:04,528 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,528 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:29:04,528 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,528 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:04,528 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:04,531 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,532 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,532 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,532 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcbucket_mapjoin[2008-04-08] 2018-01-05T11:29:04,532 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcbucket_mapjoin[2008-04-08] 2018-01-05T11:29:04,534 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,536 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,536 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,537 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.MoveTask: Partition is: {ds=2008-04-08} 2018-01-05T11:29:04,537 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,537 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: partition_name_has_valid_characters 2018-01-05T11:29:04,537 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=partition_name_has_valid_characters 2018-01-05T11:29:04,537 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,537 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,537 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:04,537 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:04,540 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,540 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,540 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,540 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcbucket_mapjoin[2008-04-08] 2018-01-05T11:29:04,541 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcbucket_mapjoin[2008-04-08] 2018-01-05T11:29:04,543 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,544 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,545 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,545 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,545 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin/ds=2008-04-08 2018-01-05T11:29:04,557 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/data/files/srcbucket20.txt/ and the destination path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin/ds=2008-04-08/srcbucket20.txt/ 2018-01-05T11:29:04,557 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin/ds=2008-04-08/srcbucket20.txt/ and the destination path is /Users/djaiswal/hive/hive/data/files/srcbucket20.txt/ 2018-01-05T11:29:04,582 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,582 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: altering partition for table srcbucket_mapjoin with partition spec : {ds=2008-04-08} 2018-01-05T11:29:04,582 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: No new files were created, and is not a replace, or we're inserting into a partition that does not exist yet. Skipping generating INSERT event. 2018-01-05T11:29:04,583 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Adding new partition {ds=2008-04-08} 2018-01-05T11:29:04,583 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,583 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: add_partition : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:04,583 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=add_partition : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:04,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,597 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,597 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcbucket_mapjoin[2008-04-08] 2018-01-05T11:29:04,597 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcbucket_mapjoin[2008-04-08] 2018-01-05T11:29:04,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,607 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,607 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,607 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-1:STATS] in serial mode 2018-01-05T11:29:04,607 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@2ce14e20, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:29:04,607 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:04,607 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,607 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:29:04,607 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:29:04,608 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:04,608 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:04,608 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,608 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,608 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:04,608 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:04,608 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,608 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:04,608 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,608 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:04,608 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:04,608 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,608 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:29:04,608 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,609 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:29:04,609 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:29:04,609 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:29:04,609 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:29:04,609 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,609 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,609 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,610 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,610 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,610 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:29:04,610 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:29:04,610 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,610 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:29:04,610 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,610 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:04,610 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:04,613 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,613 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,613 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,613 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcbucket_mapjoin[2008-04-08] 2018-01-05T11:29:04,613 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcbucket_mapjoin[2008-04-08] 2018-01-05T11:29:04,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,638 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,639 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,639 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.BasicStatsTask: Getting file stats of all partitions. threadpool size:15 2018-01-05T11:29:04,639 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.BasicStatsTask: Finished getting file stats of all partitions! 2018-01-05T11:29:04,639 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.BasicStatsTask: Partition {ds=2008-04-08} stats: [numFiles=1, numRows=0, totalSize=1358, rawDataSize=0] 2018-01-05T11:29:04,639 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,640 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: alter_partitions : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:04,640 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_partitions : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:04,640 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: New partition values:[2008-04-08] 2018-01-05T11:29:04,642 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,650 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,653 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,653 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,653 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,653 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,653 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: load data local inpath '../../data/files/srcbucket20.txt' INTO TABLE srcbucket_mapjoin partition(ds='2008-04-08') 2018-01-05T11:29:04,653 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: LOAD 2018-01-05T11:29:04,653 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/srcbucket20.txt 2018-01-05T11:29:04,653 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: default@srcbucket_mapjoin 2018-01-05T11:29:04,653 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: default@srcbucket_mapjoin@ds=2008-04-08 2018-01-05T11:29:04,653 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,653 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,653 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:04,653 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {alter_partitions_(String, String, List, EnvironmentContext, )=14, getTable_(String, String, )=3, getPartitionWithAuthInfo_(String, String, List, String, List, )=26} 2018-01-05T11:29:04,653 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112904_3d9003e2-2833-4223-ac1a-475e9aa555ce); Time taken: 0.13 seconds 2018-01-05T11:29:04,653 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:04,653 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,653 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:04,653 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,653 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query load data local inpath '../../data/files/srcbucket20.txt' INTO TABLE srcbucket_mapjoin partition(ds='2008-04-08') 2018-01-05T11:29:04,653 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.166 seconds 2018-01-05T11:29:04,654 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:04,654 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:04,654 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:04,654 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:04,654 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,654 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:04,654 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,654 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,654 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,654 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: load data local inpath '../../data/files/srcbucket22.txt' INTO TABLE srcbucket_mapjoin partition(ds='2008-04-08') 2018-01-05T11:29:04,654 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112904_1f9335ad-8f68-4e1a-b969-2f2f3e0cf07f): load data local inpath '../../data/files/srcbucket22.txt' INTO TABLE srcbucket_mapjoin partition(ds='2008-04-08') 2018-01-05T11:29:04,654 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,654 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: load data local inpath '../../data/files/srcbucket22.txt' INTO TABLE srcbucket_mapjoin partition(ds='2008-04-08') 2018-01-05T11:29:04,654 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:04,654 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,654 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,654 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,654 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,654 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:04,654 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.LoadSemanticAnalyzer: file@null@/Users/djaiswal/hive/hive/data/files/srcbucket22.txt 2018-01-05T11:29:04,654 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,654 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:04,654 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:04,657 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,657 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,657 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,657 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcbucket_mapjoin[2008-04-08] 2018-01-05T11:29:04,657 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcbucket_mapjoin[2008-04-08] 2018-01-05T11:29:04,660 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,663 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,663 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,665 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,665 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcbucket_mapjoin[2008-04-08] 2018-01-05T11:29:04,665 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcbucket_mapjoin[2008-04-08] 2018-01-05T11:29:04,667 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,671 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,671 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,672 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:04,672 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,672 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:29:04,672 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,672 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:04,672 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=3, flushCache_()=0, getPartitionWithAuthInfo_(String, String, List, String, List, )=12} 2018-01-05T11:29:04,672 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112904_1f9335ad-8f68-4e1a-b969-2f2f3e0cf07f); Time taken: 0.018 seconds 2018-01-05T11:29:04,672 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:04,672 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,672 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112904_1f9335ad-8f68-4e1a-b969-2f2f3e0cf07f): load data local inpath '../../data/files/srcbucket22.txt' INTO TABLE srcbucket_mapjoin partition(ds='2008-04-08') 2018-01-05T11:29:04,673 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,673 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: load data local inpath '../../data/files/srcbucket22.txt' INTO TABLE srcbucket_mapjoin partition(ds='2008-04-08') 2018-01-05T11:29:04,673 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: LOAD 2018-01-05T11:29:04,673 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/srcbucket22.txt 2018-01-05T11:29:04,673 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: default@srcbucket_mapjoin@ds=2008-04-08 2018-01-05T11:29:04,673 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,673 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,673 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,673 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,673 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-0:MOVE] in serial mode 2018-01-05T11:29:04,673 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@33ca22e1, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:29:04,673 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:04,673 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,673 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:29:04,673 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:29:04,674 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:04,674 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:04,674 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,674 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,674 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:04,674 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:04,674 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,674 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:04,674 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,674 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:04,674 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:04,674 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,674 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Loading data to table default.srcbucket_mapjoin partition (ds=2008-04-08) from file:/Users/djaiswal/hive/hive/data/files/srcbucket22.txt 2018-01-05T11:29:04,674 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,675 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:29:04,675 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:29:04,675 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:29:04,675 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:29:04,675 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,675 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,675 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,676 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,676 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,676 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:29:04,676 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:29:04,676 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,676 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:29:04,676 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,676 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:04,676 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:04,679 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,679 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,679 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,679 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcbucket_mapjoin[2008-04-08] 2018-01-05T11:29:04,679 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcbucket_mapjoin[2008-04-08] 2018-01-05T11:29:04,682 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,689 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,690 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,691 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.MoveTask: Partition is: {ds=2008-04-08} 2018-01-05T11:29:04,691 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,691 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: partition_name_has_valid_characters 2018-01-05T11:29:04,691 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=partition_name_has_valid_characters 2018-01-05T11:29:04,691 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,691 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,691 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:04,691 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:04,693 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,693 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,693 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,693 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcbucket_mapjoin[2008-04-08] 2018-01-05T11:29:04,693 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcbucket_mapjoin[2008-04-08] 2018-01-05T11:29:04,695 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,700 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,700 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,700 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,701 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:29:04,701 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:29:04,728 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Moved src: file:/Users/djaiswal/hive/hive/data/files/srcbucket22.txt, to dest: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin/ds=2008-04-08/srcbucket22.txt 2018-01-05T11:29:04,728 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,728 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: altering partition for table srcbucket_mapjoin with partition spec : {ds=2008-04-08} 2018-01-05T11:29:04,728 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: No new files were created, and is not a replace, or we're inserting into a partition that does not exist yet. Skipping generating INSERT event. 2018-01-05T11:29:04,728 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Altering existing partition {ds=2008-04-08} 2018-01-05T11:29:04,728 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,728 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: alter_partition : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:04,728 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_partition : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:04,728 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: New partition values:[2008-04-08] 2018-01-05T11:29:04,732 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,737 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,740 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,741 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,741 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,741 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcbucket_mapjoin[2008-04-08] 2018-01-05T11:29:04,741 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcbucket_mapjoin[2008-04-08] 2018-01-05T11:29:04,743 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,748 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,749 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,749 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.DDLTask: Ignoring request to add WriteEntity(default@srcbucket_mapjoin@ds=2008-04-08) Type=PARTITION WriteType=INSERT because WriteEntity(default@srcbucket_mapjoin@ds=2008-04-08) Type=PARTITION WriteType=INSERT is present 2018-01-05T11:29:04,749 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-1:STATS] in serial mode 2018-01-05T11:29:04,749 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@1aebf638, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:29:04,749 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:04,749 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,749 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:29:04,749 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:29:04,749 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:04,749 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:04,749 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,749 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,749 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:04,749 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:04,749 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,752 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:04,752 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,752 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:04,752 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:04,752 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,752 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:29:04,753 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,753 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:29:04,753 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:29:04,753 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:29:04,753 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:29:04,753 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,753 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,753 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,754 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,754 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,754 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:29:04,754 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:29:04,755 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,755 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:29:04,755 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,755 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:04,755 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:04,757 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,758 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,758 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,758 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcbucket_mapjoin[2008-04-08] 2018-01-05T11:29:04,758 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcbucket_mapjoin[2008-04-08] 2018-01-05T11:29:04,760 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,765 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,765 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,765 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.BasicStatsTask: Getting file stats of all partitions. threadpool size:15 2018-01-05T11:29:04,766 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.BasicStatsTask: Finished getting file stats of all partitions! 2018-01-05T11:29:04,766 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.BasicStatsTask: Partition {ds=2008-04-08} stats: [numFiles=2, numRows=0, totalSize=2808, rawDataSize=0] 2018-01-05T11:29:04,766 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,766 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: alter_partitions : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:04,766 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_partitions : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:04,766 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: New partition values:[2008-04-08] 2018-01-05T11:29:04,769 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,779 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,782 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,782 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,782 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,782 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,783 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: load data local inpath '../../data/files/srcbucket22.txt' INTO TABLE srcbucket_mapjoin partition(ds='2008-04-08') 2018-01-05T11:29:04,783 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: LOAD 2018-01-05T11:29:04,783 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/srcbucket22.txt 2018-01-05T11:29:04,783 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: default@srcbucket_mapjoin@ds=2008-04-08 2018-01-05T11:29:04,783 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,783 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,783 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:04,783 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {alter_partitions_(String, String, List, EnvironmentContext, )=16, getTable_(String, String, )=3, getPartitionWithAuthInfo_(String, String, List, String, List, )=7} 2018-01-05T11:29:04,783 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112904_1f9335ad-8f68-4e1a-b969-2f2f3e0cf07f); Time taken: 0.111 seconds 2018-01-05T11:29:04,783 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:04,783 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,783 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:04,783 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,783 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query load data local inpath '../../data/files/srcbucket22.txt' INTO TABLE srcbucket_mapjoin partition(ds='2008-04-08') 2018-01-05T11:29:04,783 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.129 seconds 2018-01-05T11:29:04,783 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:04,783 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:04,783 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:04,783 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:04,783 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,783 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:04,783 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,783 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,783 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,783 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: load data local inpath '../../data/files/srcbucket20.txt' INTO TABLE srcbucket_mapjoin_part partition(ds='2008-04-08') 2018-01-05T11:29:04,783 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112904_ae117a5a-aa96-46eb-b31c-0fae0fdb16f8): load data local inpath '../../data/files/srcbucket20.txt' INTO TABLE srcbucket_mapjoin_part partition(ds='2008-04-08') 2018-01-05T11:29:04,783 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,783 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: load data local inpath '../../data/files/srcbucket20.txt' INTO TABLE srcbucket_mapjoin_part partition(ds='2008-04-08') 2018-01-05T11:29:04,783 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:04,783 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,783 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,784 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,784 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,784 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:04,784 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.LoadSemanticAnalyzer: file@null@/Users/djaiswal/hive/hive/data/files/srcbucket20.txt 2018-01-05T11:29:04,784 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,784 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:04,784 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:04,787 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,788 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,788 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,788 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:04,788 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:04,790 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,791 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,791 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,793 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,793 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:04,793 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:04,796 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,797 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,798 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,798 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:04,798 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,798 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:29:04,798 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,798 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:04,798 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=4, flushCache_()=0} 2018-01-05T11:29:04,798 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112904_ae117a5a-aa96-46eb-b31c-0fae0fdb16f8); Time taken: 0.015 seconds 2018-01-05T11:29:04,798 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:04,798 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,798 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112904_ae117a5a-aa96-46eb-b31c-0fae0fdb16f8): load data local inpath '../../data/files/srcbucket20.txt' INTO TABLE srcbucket_mapjoin_part partition(ds='2008-04-08') 2018-01-05T11:29:04,799 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,799 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: load data local inpath '../../data/files/srcbucket20.txt' INTO TABLE srcbucket_mapjoin_part partition(ds='2008-04-08') 2018-01-05T11:29:04,799 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: LOAD 2018-01-05T11:29:04,799 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/srcbucket20.txt 2018-01-05T11:29:04,799 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: default@srcbucket_mapjoin_part 2018-01-05T11:29:04,799 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,799 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,799 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,799 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,799 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-0:MOVE] in serial mode 2018-01-05T11:29:04,799 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@3e0af3c, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:29:04,799 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:04,799 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,799 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:29:04,799 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:29:04,800 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:04,800 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:04,800 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,800 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,800 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:04,800 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:04,800 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,800 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:04,800 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,800 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:04,800 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:04,800 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,800 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Loading data to table default.srcbucket_mapjoin_part partition (ds=2008-04-08) from file:/Users/djaiswal/hive/hive/data/files/srcbucket20.txt 2018-01-05T11:29:04,800 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,801 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:29:04,801 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:29:04,801 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:29:04,801 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:29:04,801 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,801 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,801 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,802 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,802 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,802 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:29:04,802 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:29:04,802 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,802 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:29:04,802 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,802 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:04,802 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:04,805 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,806 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,806 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,806 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:04,806 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:04,809 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,811 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,811 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,812 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.MoveTask: Partition is: {ds=2008-04-08} 2018-01-05T11:29:04,812 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,813 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: partition_name_has_valid_characters 2018-01-05T11:29:04,813 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=partition_name_has_valid_characters 2018-01-05T11:29:04,813 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,813 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,813 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:04,813 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:04,815 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,816 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,816 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,816 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:04,816 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:04,819 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,821 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,821 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,821 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,821 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08 2018-01-05T11:29:04,834 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/data/files/srcbucket20.txt/ and the destination path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket20.txt/ 2018-01-05T11:29:04,834 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket20.txt/ and the destination path is /Users/djaiswal/hive/hive/data/files/srcbucket20.txt/ 2018-01-05T11:29:04,857 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,857 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: altering partition for table srcbucket_mapjoin_part with partition spec : {ds=2008-04-08} 2018-01-05T11:29:04,857 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: No new files were created, and is not a replace, or we're inserting into a partition that does not exist yet. Skipping generating INSERT event. 2018-01-05T11:29:04,858 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Adding new partition {ds=2008-04-08} 2018-01-05T11:29:04,858 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,858 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: add_partition : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:04,858 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=add_partition : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:04,865 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,866 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,866 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,866 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:04,866 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:04,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,872 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,873 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,873 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-1:STATS] in serial mode 2018-01-05T11:29:04,873 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@13616fc3, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:29:04,873 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:04,873 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,873 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:29:04,873 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:29:04,873 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:04,873 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:04,873 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,873 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,873 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:04,873 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:04,873 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,873 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:04,873 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,873 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:04,873 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:04,873 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,873 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:29:04,874 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,874 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:29:04,874 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:29:04,874 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:29:04,874 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:29:04,874 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,874 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,875 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,875 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,875 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,875 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:29:04,875 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:29:04,875 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,876 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:29:04,876 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,876 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:04,876 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:04,878 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,878 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,878 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,878 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:04,878 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:04,881 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,885 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,886 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,886 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.BasicStatsTask: Getting file stats of all partitions. threadpool size:15 2018-01-05T11:29:04,887 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.BasicStatsTask: Finished getting file stats of all partitions! 2018-01-05T11:29:04,887 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.BasicStatsTask: Partition {ds=2008-04-08} stats: [numFiles=1, numRows=0, totalSize=1358, rawDataSize=0] 2018-01-05T11:29:04,887 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,887 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: alter_partitions : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:04,887 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_partitions : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:04,887 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: New partition values:[2008-04-08] 2018-01-05T11:29:04,891 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,898 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,901 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,902 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,902 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,902 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,902 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: load data local inpath '../../data/files/srcbucket20.txt' INTO TABLE srcbucket_mapjoin_part partition(ds='2008-04-08') 2018-01-05T11:29:04,902 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: LOAD 2018-01-05T11:29:04,902 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/srcbucket20.txt 2018-01-05T11:29:04,902 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: default@srcbucket_mapjoin_part 2018-01-05T11:29:04,902 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: default@srcbucket_mapjoin_part@ds=2008-04-08 2018-01-05T11:29:04,902 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,902 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,902 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:04,902 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {alter_partitions_(String, String, List, EnvironmentContext, )=15, getTable_(String, String, )=2, getPartitionWithAuthInfo_(String, String, List, String, List, )=8} 2018-01-05T11:29:04,902 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112904_ae117a5a-aa96-46eb-b31c-0fae0fdb16f8); Time taken: 0.104 seconds 2018-01-05T11:29:04,902 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:04,902 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,902 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:04,902 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,902 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query load data local inpath '../../data/files/srcbucket20.txt' INTO TABLE srcbucket_mapjoin_part partition(ds='2008-04-08') 2018-01-05T11:29:04,902 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.119 seconds 2018-01-05T11:29:04,902 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:04,902 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:04,902 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:04,902 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:04,902 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,902 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:04,902 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,902 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,902 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,902 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: load data local inpath '../../data/files/srcbucket21.txt' INTO TABLE srcbucket_mapjoin_part partition(ds='2008-04-08') 2018-01-05T11:29:04,902 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112904_ffb96d58-3e74-4de2-9462-c014fd6f74a5): load data local inpath '../../data/files/srcbucket21.txt' INTO TABLE srcbucket_mapjoin_part partition(ds='2008-04-08') 2018-01-05T11:29:04,902 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,902 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: load data local inpath '../../data/files/srcbucket21.txt' INTO TABLE srcbucket_mapjoin_part partition(ds='2008-04-08') 2018-01-05T11:29:04,902 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:04,903 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,903 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,903 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,903 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,903 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:04,903 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.LoadSemanticAnalyzer: file@null@/Users/djaiswal/hive/hive/data/files/srcbucket21.txt 2018-01-05T11:29:04,903 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,903 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:04,903 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:04,905 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,905 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,906 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,906 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:04,906 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:04,908 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,911 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,912 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,914 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,914 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:04,914 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:04,916 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,921 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,922 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,922 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:04,922 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,922 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:29:04,922 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,922 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:04,922 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=3, flushCache_()=0, getPartitionWithAuthInfo_(String, String, List, String, List, )=14} 2018-01-05T11:29:04,922 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112904_ffb96d58-3e74-4de2-9462-c014fd6f74a5); Time taken: 0.02 seconds 2018-01-05T11:29:04,922 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:04,922 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,922 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112904_ffb96d58-3e74-4de2-9462-c014fd6f74a5): load data local inpath '../../data/files/srcbucket21.txt' INTO TABLE srcbucket_mapjoin_part partition(ds='2008-04-08') 2018-01-05T11:29:04,924 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,924 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: load data local inpath '../../data/files/srcbucket21.txt' INTO TABLE srcbucket_mapjoin_part partition(ds='2008-04-08') 2018-01-05T11:29:04,924 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: LOAD 2018-01-05T11:29:04,924 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/srcbucket21.txt 2018-01-05T11:29:04,924 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: default@srcbucket_mapjoin_part@ds=2008-04-08 2018-01-05T11:29:04,924 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,924 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,924 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,924 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,924 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-0:MOVE] in serial mode 2018-01-05T11:29:04,924 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@5ffb815c, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:29:04,924 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:04,924 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,924 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:29:04,924 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:29:04,924 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:04,924 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:04,924 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,924 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,924 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:04,924 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:04,924 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,924 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:04,924 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,924 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:04,924 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:04,924 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,924 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Loading data to table default.srcbucket_mapjoin_part partition (ds=2008-04-08) from file:/Users/djaiswal/hive/hive/data/files/srcbucket21.txt 2018-01-05T11:29:04,925 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,925 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:29:04,925 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:29:04,925 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:29:04,925 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:29:04,925 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,925 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,925 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,926 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,926 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,926 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:29:04,926 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:29:04,926 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,926 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:29:04,926 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,926 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:04,926 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:04,930 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,930 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,931 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,931 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:04,931 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:04,933 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,937 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,938 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,939 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.MoveTask: Partition is: {ds=2008-04-08} 2018-01-05T11:29:04,939 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,939 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: partition_name_has_valid_characters 2018-01-05T11:29:04,939 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=partition_name_has_valid_characters 2018-01-05T11:29:04,939 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,939 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,939 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:04,939 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:04,941 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,942 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,942 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,942 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:04,942 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:04,945 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,949 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,949 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,949 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,949 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:29:04,949 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:29:04,975 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Moved src: file:/Users/djaiswal/hive/hive/data/files/srcbucket21.txt, to dest: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket21.txt 2018-01-05T11:29:04,975 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:04,975 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: altering partition for table srcbucket_mapjoin_part with partition spec : {ds=2008-04-08} 2018-01-05T11:29:04,975 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: No new files were created, and is not a replace, or we're inserting into a partition that does not exist yet. Skipping generating INSERT event. 2018-01-05T11:29:04,975 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Altering existing partition {ds=2008-04-08} 2018-01-05T11:29:04,975 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,975 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: alter_partition : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:04,975 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_partition : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:04,975 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: New partition values:[2008-04-08] 2018-01-05T11:29:04,978 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,982 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,985 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,986 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,986 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,986 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:04,986 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:04,988 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,991 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,992 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,992 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.DDLTask: Ignoring request to add WriteEntity(default@srcbucket_mapjoin_part@ds=2008-04-08) Type=PARTITION WriteType=INSERT because WriteEntity(default@srcbucket_mapjoin_part@ds=2008-04-08) Type=PARTITION WriteType=INSERT is present 2018-01-05T11:29:04,992 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-1:STATS] in serial mode 2018-01-05T11:29:04,992 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@1d364c1e, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:29:04,992 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:04,992 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,992 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:29:04,992 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:29:04,992 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:04,992 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:04,992 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,992 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,992 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:04,992 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:04,992 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,992 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:04,992 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,992 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:04,992 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:04,992 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,992 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:29:04,993 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,993 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:29:04,993 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:29:04,993 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:29:04,993 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:29:04,993 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,993 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,993 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,994 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,994 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,994 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:29:04,994 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:29:04,994 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,994 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:29:04,994 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,994 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:04,994 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:04,997 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:04,997 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,997 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:04,997 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:04,997 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:05,000 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,004 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,005 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,005 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.BasicStatsTask: Getting file stats of all partitions. threadpool size:15 2018-01-05T11:29:05,006 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.BasicStatsTask: Finished getting file stats of all partitions! 2018-01-05T11:29:05,006 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.BasicStatsTask: Partition {ds=2008-04-08} stats: [numFiles=2, numRows=0, totalSize=2750, rawDataSize=0] 2018-01-05T11:29:05,006 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,006 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: alter_partitions : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,006 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_partitions : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,006 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: New partition values:[2008-04-08] 2018-01-05T11:29:05,009 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,014 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,017 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,017 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,017 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,017 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,017 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: load data local inpath '../../data/files/srcbucket21.txt' INTO TABLE srcbucket_mapjoin_part partition(ds='2008-04-08') 2018-01-05T11:29:05,018 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: LOAD 2018-01-05T11:29:05,018 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/srcbucket21.txt 2018-01-05T11:29:05,018 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: default@srcbucket_mapjoin_part@ds=2008-04-08 2018-01-05T11:29:05,018 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,018 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,018 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:05,018 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {alter_partitions_(String, String, List, EnvironmentContext, )=11, getTable_(String, String, )=3, getPartitionWithAuthInfo_(String, String, List, String, List, )=8} 2018-01-05T11:29:05,018 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112904_ffb96d58-3e74-4de2-9462-c014fd6f74a5); Time taken: 0.096 seconds 2018-01-05T11:29:05,018 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:05,018 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,018 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:05,018 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,018 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query load data local inpath '../../data/files/srcbucket21.txt' INTO TABLE srcbucket_mapjoin_part partition(ds='2008-04-08') 2018-01-05T11:29:05,018 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.116 seconds 2018-01-05T11:29:05,018 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:05,018 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:05,018 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:05,018 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:05,018 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,018 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:05,018 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,018 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,018 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,018 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: load data local inpath '../../data/files/srcbucket22.txt' INTO TABLE srcbucket_mapjoin_part partition(ds='2008-04-08') 2018-01-05T11:29:05,018 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112905_85fef266-d0b7-48e5-9669-1a6a4682c5a5): load data local inpath '../../data/files/srcbucket22.txt' INTO TABLE srcbucket_mapjoin_part partition(ds='2008-04-08') 2018-01-05T11:29:05,018 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,018 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: load data local inpath '../../data/files/srcbucket22.txt' INTO TABLE srcbucket_mapjoin_part partition(ds='2008-04-08') 2018-01-05T11:29:05,018 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:05,018 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,018 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,018 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,018 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,018 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:05,019 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.LoadSemanticAnalyzer: file@null@/Users/djaiswal/hive/hive/data/files/srcbucket22.txt 2018-01-05T11:29:05,019 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,019 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,019 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,021 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,021 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,021 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,022 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:05,022 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:05,024 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,027 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,027 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,028 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,028 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:05,029 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:05,031 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,039 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,040 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,040 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:05,040 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,040 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:29:05,040 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,040 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:05,040 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=2, flushCache_()=0, getPartitionWithAuthInfo_(String, String, List, String, List, )=18} 2018-01-05T11:29:05,040 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112905_85fef266-d0b7-48e5-9669-1a6a4682c5a5); Time taken: 0.022 seconds 2018-01-05T11:29:05,040 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:05,040 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,040 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112905_85fef266-d0b7-48e5-9669-1a6a4682c5a5): load data local inpath '../../data/files/srcbucket22.txt' INTO TABLE srcbucket_mapjoin_part partition(ds='2008-04-08') 2018-01-05T11:29:05,041 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,041 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: load data local inpath '../../data/files/srcbucket22.txt' INTO TABLE srcbucket_mapjoin_part partition(ds='2008-04-08') 2018-01-05T11:29:05,041 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: LOAD 2018-01-05T11:29:05,041 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/srcbucket22.txt 2018-01-05T11:29:05,041 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: default@srcbucket_mapjoin_part@ds=2008-04-08 2018-01-05T11:29:05,041 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,041 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,041 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,041 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,041 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-0:MOVE] in serial mode 2018-01-05T11:29:05,041 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@c887298, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:29:05,041 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:05,041 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,041 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:29:05,041 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:29:05,041 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:05,041 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:05,041 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,041 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,041 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:05,041 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:05,041 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,041 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:05,042 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,042 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:05,042 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:05,042 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,042 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Loading data to table default.srcbucket_mapjoin_part partition (ds=2008-04-08) from file:/Users/djaiswal/hive/hive/data/files/srcbucket22.txt 2018-01-05T11:29:05,042 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,042 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:29:05,042 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:29:05,042 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:29:05,042 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:29:05,043 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,043 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,043 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,043 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,043 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,043 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:29:05,043 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:29:05,043 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,044 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:29:05,044 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,044 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,044 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,046 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,046 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,046 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,046 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:05,046 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:05,049 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,052 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,053 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,053 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.MoveTask: Partition is: {ds=2008-04-08} 2018-01-05T11:29:05,053 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,053 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: partition_name_has_valid_characters 2018-01-05T11:29:05,053 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=partition_name_has_valid_characters 2018-01-05T11:29:05,053 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,053 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,053 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,053 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,055 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,056 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,056 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,056 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:05,056 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:05,059 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,062 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,062 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,062 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,062 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:29:05,062 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:29:05,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Moved src: file:/Users/djaiswal/hive/hive/data/files/srcbucket22.txt, to dest: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket22.txt 2018-01-05T11:29:05,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: altering partition for table srcbucket_mapjoin_part with partition spec : {ds=2008-04-08} 2018-01-05T11:29:05,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: No new files were created, and is not a replace, or we're inserting into a partition that does not exist yet. Skipping generating INSERT event. 2018-01-05T11:29:05,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Altering existing partition {ds=2008-04-08} 2018-01-05T11:29:05,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,086 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: alter_partition : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,086 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_partition : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,086 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: New partition values:[2008-04-08] 2018-01-05T11:29:05,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,093 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,098 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,099 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,099 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,099 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:05,099 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:05,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,106 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,106 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,106 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.DDLTask: Ignoring request to add WriteEntity(default@srcbucket_mapjoin_part@ds=2008-04-08) Type=PARTITION WriteType=INSERT because WriteEntity(default@srcbucket_mapjoin_part@ds=2008-04-08) Type=PARTITION WriteType=INSERT is present 2018-01-05T11:29:05,106 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-1:STATS] in serial mode 2018-01-05T11:29:05,106 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@760420ba, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:29:05,106 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:05,106 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,107 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:29:05,107 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:29:05,107 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:05,107 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:05,107 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,107 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,107 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:05,107 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:05,107 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,107 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:05,107 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,107 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:05,107 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:05,107 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,107 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:29:05,107 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,107 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:29:05,107 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:29:05,107 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:29:05,108 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:29:05,108 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,108 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,108 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,108 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,108 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,108 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:29:05,108 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:29:05,108 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,109 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:29:05,109 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,109 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,109 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,111 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,112 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,112 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,112 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:05,112 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:05,114 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,118 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,119 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,119 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.BasicStatsTask: Getting file stats of all partitions. threadpool size:15 2018-01-05T11:29:05,119 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.BasicStatsTask: Finished getting file stats of all partitions! 2018-01-05T11:29:05,120 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.BasicStatsTask: Partition {ds=2008-04-08} stats: [numFiles=3, numRows=0, totalSize=4200, rawDataSize=0] 2018-01-05T11:29:05,120 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,120 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: alter_partitions : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,120 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_partitions : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,120 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: New partition values:[2008-04-08] 2018-01-05T11:29:05,122 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,127 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,130 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,130 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,130 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,130 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,130 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: load data local inpath '../../data/files/srcbucket22.txt' INTO TABLE srcbucket_mapjoin_part partition(ds='2008-04-08') 2018-01-05T11:29:05,130 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: LOAD 2018-01-05T11:29:05,130 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/srcbucket22.txt 2018-01-05T11:29:05,130 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: default@srcbucket_mapjoin_part@ds=2008-04-08 2018-01-05T11:29:05,130 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,130 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,131 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:05,131 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {alter_partitions_(String, String, List, EnvironmentContext, )=10, getTable_(String, String, )=3, getPartitionWithAuthInfo_(String, String, List, String, List, )=7} 2018-01-05T11:29:05,131 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112905_85fef266-d0b7-48e5-9669-1a6a4682c5a5); Time taken: 0.09 seconds 2018-01-05T11:29:05,131 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:05,131 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,131 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:05,131 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,131 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query load data local inpath '../../data/files/srcbucket22.txt' INTO TABLE srcbucket_mapjoin_part partition(ds='2008-04-08') 2018-01-05T11:29:05,131 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.113 seconds 2018-01-05T11:29:05,131 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:05,131 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:05,131 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:05,131 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:05,131 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,131 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:05,131 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,131 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,131 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,131 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: load data local inpath '../../data/files/srcbucket23.txt' INTO TABLE srcbucket_mapjoin_part partition(ds='2008-04-08') 2018-01-05T11:29:05,131 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112905_82479dc2-a46b-444e-bb96-c43e9b8e4509): load data local inpath '../../data/files/srcbucket23.txt' INTO TABLE srcbucket_mapjoin_part partition(ds='2008-04-08') 2018-01-05T11:29:05,131 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,131 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: load data local inpath '../../data/files/srcbucket23.txt' INTO TABLE srcbucket_mapjoin_part partition(ds='2008-04-08') 2018-01-05T11:29:05,131 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:05,131 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,131 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,131 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,131 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,131 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:05,131 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.LoadSemanticAnalyzer: file@null@/Users/djaiswal/hive/hive/data/files/srcbucket23.txt 2018-01-05T11:29:05,132 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,132 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,132 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,134 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,135 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,135 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,135 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:05,135 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:05,137 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,140 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,141 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,142 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,142 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:05,142 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:05,145 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,149 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,149 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,149 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:05,149 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,149 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:29:05,149 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,149 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:05,149 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=3, flushCache_()=0, getPartitionWithAuthInfo_(String, String, List, String, List, )=13} 2018-01-05T11:29:05,149 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112905_82479dc2-a46b-444e-bb96-c43e9b8e4509); Time taken: 0.018 seconds 2018-01-05T11:29:05,149 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:05,149 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,149 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112905_82479dc2-a46b-444e-bb96-c43e9b8e4509): load data local inpath '../../data/files/srcbucket23.txt' INTO TABLE srcbucket_mapjoin_part partition(ds='2008-04-08') 2018-01-05T11:29:05,150 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,150 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: load data local inpath '../../data/files/srcbucket23.txt' INTO TABLE srcbucket_mapjoin_part partition(ds='2008-04-08') 2018-01-05T11:29:05,151 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: LOAD 2018-01-05T11:29:05,151 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/srcbucket23.txt 2018-01-05T11:29:05,151 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: default@srcbucket_mapjoin_part@ds=2008-04-08 2018-01-05T11:29:05,151 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,151 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,151 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,151 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,151 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-0:MOVE] in serial mode 2018-01-05T11:29:05,151 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@50fbc617, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:29:05,151 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:05,151 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,151 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:29:05,151 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:29:05,151 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:05,151 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:05,151 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,151 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,151 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:05,151 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:05,151 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,151 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:05,151 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,151 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:05,151 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:05,151 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,151 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Loading data to table default.srcbucket_mapjoin_part partition (ds=2008-04-08) from file:/Users/djaiswal/hive/hive/data/files/srcbucket23.txt 2018-01-05T11:29:05,151 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,152 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:29:05,152 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:29:05,152 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:29:05,152 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:29:05,152 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,152 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,152 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,153 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,153 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,153 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:29:05,153 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:29:05,153 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,153 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:29:05,153 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,153 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,153 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,155 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,156 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,156 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,156 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:05,156 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:05,159 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,163 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,163 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,164 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.MoveTask: Partition is: {ds=2008-04-08} 2018-01-05T11:29:05,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,165 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: partition_name_has_valid_characters 2018-01-05T11:29:05,165 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=partition_name_has_valid_characters 2018-01-05T11:29:05,165 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,165 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,165 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,165 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,167 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,167 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,168 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,168 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:05,168 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:05,170 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,174 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:29:05,174 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:29:05,199 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Moved src: file:/Users/djaiswal/hive/hive/data/files/srcbucket23.txt, to dest: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket23.txt 2018-01-05T11:29:05,199 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,199 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: altering partition for table srcbucket_mapjoin_part with partition spec : {ds=2008-04-08} 2018-01-05T11:29:05,199 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: No new files were created, and is not a replace, or we're inserting into a partition that does not exist yet. Skipping generating INSERT event. 2018-01-05T11:29:05,199 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Altering existing partition {ds=2008-04-08} 2018-01-05T11:29:05,199 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,199 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: alter_partition : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,199 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_partition : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,199 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: New partition values:[2008-04-08] 2018-01-05T11:29:05,202 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,206 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,210 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,210 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,210 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,210 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:05,211 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:05,213 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,216 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,217 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,217 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.DDLTask: Ignoring request to add WriteEntity(default@srcbucket_mapjoin_part@ds=2008-04-08) Type=PARTITION WriteType=INSERT because WriteEntity(default@srcbucket_mapjoin_part@ds=2008-04-08) Type=PARTITION WriteType=INSERT is present 2018-01-05T11:29:05,217 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-1:STATS] in serial mode 2018-01-05T11:29:05,217 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@6249fadc, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:29:05,217 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:05,217 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,217 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:29:05,217 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:29:05,217 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:05,217 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:05,217 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,217 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,217 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:05,217 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:05,217 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,217 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:05,217 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,217 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:05,217 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:05,217 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,217 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:29:05,218 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,219 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:29:05,219 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:29:05,219 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:29:05,219 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:29:05,219 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,219 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,219 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,220 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,220 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,220 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:29:05,220 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:29:05,220 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,220 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:29:05,220 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,220 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,220 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,223 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,223 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,223 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,223 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:05,223 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=srcbucket_mapjoin_part[2008-04-08] 2018-01-05T11:29:05,225 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,229 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,229 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,229 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.BasicStatsTask: Getting file stats of all partitions. threadpool size:15 2018-01-05T11:29:05,230 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.BasicStatsTask: Finished getting file stats of all partitions! 2018-01-05T11:29:05,230 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.BasicStatsTask: Partition {ds=2008-04-08} stats: [numFiles=4, numRows=0, totalSize=5812, rawDataSize=0] 2018-01-05T11:29:05,230 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,230 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: alter_partitions : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,230 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_partitions : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,230 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: New partition values:[2008-04-08] 2018-01-05T11:29:05,233 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,237 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,241 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,242 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,242 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,242 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,242 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: load data local inpath '../../data/files/srcbucket23.txt' INTO TABLE srcbucket_mapjoin_part partition(ds='2008-04-08') 2018-01-05T11:29:05,242 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: LOAD 2018-01-05T11:29:05,242 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: file:/Users/djaiswal/hive/hive/data/files/srcbucket23.txt 2018-01-05T11:29:05,242 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: default@srcbucket_mapjoin_part@ds=2008-04-08 2018-01-05T11:29:05,242 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,242 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,242 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:05,242 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {alter_partitions_(String, String, List, EnvironmentContext, )=12, getTable_(String, String, )=3, getPartitionWithAuthInfo_(String, String, List, String, List, )=6} 2018-01-05T11:29:05,242 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112905_82479dc2-a46b-444e-bb96-c43e9b8e4509); Time taken: 0.093 seconds 2018-01-05T11:29:05,242 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:05,242 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,242 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:05,242 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,242 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query load data local inpath '../../data/files/srcbucket23.txt' INTO TABLE srcbucket_mapjoin_part partition(ds='2008-04-08') 2018-01-05T11:29:05,242 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.111 seconds 2018-01-05T11:29:05,242 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:05,242 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:05,242 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:05,242 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:05,242 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: false 2018-01-05T11:29:05,243 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:05,243 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:05,243 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:05,243 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:05,243 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,243 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:05,243 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,243 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,243 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,243 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: insert overwrite table tab_part partition (ds='2008-04-08') select key,value from srcbucket_mapjoin_part 2018-01-05T11:29:05,243 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a): insert overwrite table tab_part partition (ds='2008-04-08') select key,value from srcbucket_mapjoin_part 2018-01-05T11:29:05,243 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,243 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: insert overwrite table tab_part partition (ds='2008-04-08') select key,value from srcbucket_mapjoin_part 2018-01-05T11:29:05,243 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:05,243 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,243 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,244 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,244 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,244 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:05,244 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:29:05,244 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed phase 1 of Semantic Analysis 2018-01-05T11:29:05,244 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:05,244 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,244 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,244 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,246 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,247 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,247 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:05,247 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:05,247 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,247 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab_part 2018-01-05T11:29:05,247 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab_part 2018-01-05T11:29:05,250 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,250 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,250 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,250 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=tab_part[2008-04-08] 2018-01-05T11:29:05,250 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=tab_part[2008-04-08] 2018-01-05T11:29:05,253 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,254 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,254 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,255 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed getting MetaData in Semantic Analysis 2018-01-05T11:29:05,256 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,256 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,256 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_not_null_constraints : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,256 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_not_null_constraints : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,257 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,257 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,257 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,257 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,257 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,257 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,257 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,257 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,257 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,257 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,258 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_unique_constraints : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,258 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_unique_constraints : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,258 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,258 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,259 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:29:05,259 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,260 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.sql2rel: Plan after trimming unused fields HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.srcbucket_mapjoin_part]], table:alias=[srcbucket_mapjoin_part]) 2018-01-05T11:29:05,260 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan before removing subquery: HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.srcbucket_mapjoin_part]], table:alias=[srcbucket_mapjoin_part]) 2018-01-05T11:29:05,260 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan just after removing subquery: HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.srcbucket_mapjoin_part]], table:alias=[srcbucket_mapjoin_part]) 2018-01-05T11:29:05,260 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan after decorrelation: HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.srcbucket_mapjoin_part]], table:alias=[srcbucket_mapjoin_part]) 2018-01-05T11:29:05,260 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,260 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,260 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,260 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,260 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,261 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,261 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,261 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,261 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,261 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,261 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,261 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,261 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,261 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,261 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,262 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.sql2rel: Plan after trimming unused fields HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.srcbucket_mapjoin_part]], table:alias=[srcbucket_mapjoin_part]) 2018-01-05T11:29:05,262 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,262 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,262 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,262 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,262 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,262 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,262 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,262 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,262 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,262 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,262 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,262 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,262 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,263 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,263 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,263 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Original plan for PlanModifier HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.srcbucket_mapjoin_part]], table:alias=[srcbucket_mapjoin_part]) 2018-01-05T11:29:05,263 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after nested convertOpTree HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.srcbucket_mapjoin_part]], table:alias=[srcbucket_mapjoin_part]) 2018-01-05T11:29:05,263 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after propagating order HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.srcbucket_mapjoin_part]], table:alias=[srcbucket_mapjoin_part]) 2018-01-05T11:29:05,263 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after fixTopOBSchema HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.srcbucket_mapjoin_part]], table:alias=[srcbucket_mapjoin_part]) 2018-01-05T11:29:05,263 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Final plan after modifier HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.srcbucket_mapjoin_part]], table:alias=[srcbucket_mapjoin_part]) 2018-01-05T11:29:05,263 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:05,263 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,263 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,263 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,266 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,267 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,267 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:05,267 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:05,267 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,267 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab_part 2018-01-05T11:29:05,267 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab_part 2018-01-05T11:29:05,270 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,270 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,270 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,270 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=tab_part[2008-04-08] 2018-01-05T11:29:05,270 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=tab_part[2008-04-08] 2018-01-05T11:29:05,273 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,274 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,274 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,275 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Table Plan for srcbucket_mapjoin_part TS[0] 2018-01-05T11:29:05,275 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB srcbucket_mapjoin_part{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB srcbucket_mapjoin_part{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:05,275 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (. (tok_table_or_col srcbucket_mapjoin_part) key) key) (tok_selexpr (. (tok_table_or_col srcbucket_mapjoin_part) value) value)) 2018-01-05T11:29:05,275 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = srcbucket_mapjoin_part{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} starRr = null 2018-01-05T11:29:05,275 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(key,_col0: int)(value,_col1: string)} 2018-01-05T11:29:05,275 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:05,275 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-05_243_5530033092111214819-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:05,275 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-05_243_5530033092111214819-1 2018-01-05T11:29:05,302 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Partition: sd is org.apache.hadoop.hive.metastore.api.StorageDescriptor 2018-01-05T11:29:05,314 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/-mr-10001 2018-01-05T11:29:05,314 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 row schema: null{(key,_col0: int)(value,_col1: string)} 2018-01-05T11:29:05,315 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,315 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab_part 2018-01-05T11:29:05,315 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab_part 2018-01-05T11:29:05,318 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,318 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,318 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Generate an operator pipeline to autogather column stats for table default.tab_part in query insert overwrite table tab_part partition (ds='2008-04-08') select key,value from srcbucket_mapjoin_part 2018-01-05T11:29:05,318 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: analyze table `default`.`tab_part` compute statistics for columns 2018-01-05T11:29:05,318 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:05,319 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,319 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab_part 2018-01-05T11:29:05,319 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab_part 2018-01-05T11:29:05,325 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,325 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,325 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: select compute_stats(`key`, 'hll') , compute_stats(`value`, 'hll') , `ds` from `default`.`tab_part` group by `ds` 2018-01-05T11:29:05,326 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: select compute_stats(`key`, 'hll') , compute_stats(`value`, 'hll') , `ds` from `default`.`tab_part` group by `ds` 2018-01-05T11:29:05,326 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:05,326 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:05,327 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:05,327 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:05,327 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,327 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab_part 2018-01-05T11:29:05,327 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab_part 2018-01-05T11:29:05,330 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,331 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,331 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:05,331 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:05,346 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Table Plan for tab_part TS[0] 2018-01-05T11:29:05,346 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:05,347 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:05,347 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB tab_part{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB {((tok_table_or_col ds),_col0: string)((tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct)((tok_function compute_stats (tok_table_or_col value) 'hll'),_col2: struct)} tab_part{(ds,_col0: string)} 2018-01-05T11:29:05,347 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (tok_function compute_stats (tok_table_or_col key) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col value) 'hll')) (tok_selexpr (tok_table_or_col ds))) 2018-01-05T11:29:05,347 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = {((tok_table_or_col ds),_col0: string)((tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct)((tok_function compute_stats (tok_table_or_col value) 'hll'),_col2: struct)} tab_part{(ds,_col0: string)} starRr = tab_part{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:05,347 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(_c0,_col0: struct)(_c1,_col1: struct)(ds,_col2: string)} tab_part{(ds,_col2: string)} 2018-01-05T11:29:05,347 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:05,347 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_318_6335458483535453971-1/-mr-10000/.hive-staging_hive_2018-01-05_11-29-05_318_6335458483535453971-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_318_6335458483535453971-1/-mr-10000 2018-01-05T11:29:05,347 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_318_6335458483535453971-1/-mr-10000/.hive-staging_hive_2018-01-05_11-29-05_318_6335458483535453971-1 2018-01-05T11:29:05,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_318_6335458483535453971-1/-mr-10002 2018-01-05T11:29:05,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_318_6335458483535453971-1/-mr-10000 row schema: null{(_c0,_col0: struct)(_c1,_col1: struct)(ds,_col2: string)} tab_part{(ds,_col2: string)} 2018-01-05T11:29:05,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block null 2018-01-05T11:29:05,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:29:05,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block null 2018-01-05T11:29:05,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:29:05,372 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: CBO Succeeded; optimized logical plan. 2018-01-05T11:29:05,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Before logical optimization TS[0]-SEL[1]-RS[2]-SEL[3]-FS[4] -SEL[1] 2018-01-05T11:29:05,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] lineage.Generator: Time taken for lineage transform=1 2018-01-05T11:29:05,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,373 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FS(4) 2018-01-05T11:29:05,373 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FS(6) 2018-01-05T11:29:05,373 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(5) 2018-01-05T11:29:05,373 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for GBY(4) 2018-01-05T11:29:05,373 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(3) 2018-01-05T11:29:05,373 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for GBY(2) 2018-01-05T11:29:05,373 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(1) 2018-01-05T11:29:05,373 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(3) 2018-01-05T11:29:05,373 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(2) 2018-01-05T11:29:05,373 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(1) 2018-01-05T11:29:05,373 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:29:05,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SimplePredicatePushDown: After PPD: TS[0]-SEL[1]-RS[2]-SEL[3]-FS[4] -SEL[1] 2018-01-05T11:29:05,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 3 key:[Column[_col0]] 2018-01-05T11:29:05,373 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 3 oldColExprMap: {KEY._col0=Column[_col0], VALUE._col0=Column[_col1], VALUE._col1=Column[_col2]} 2018-01-05T11:29:05,373 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 3 newColExprMap: {KEY._col0=Column[_col0], VALUE._col0=Column[_col1], VALUE._col1=Column[_col2]} 2018-01-05T11:29:05,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 2 key:[Column[_col0]] 2018-01-05T11:29:05,373 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 2 oldColExprMap: {VALUE._col0=Column[_col1], KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:05,373 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 2 newColExprMap: {VALUE._col0=Column[_col1], KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:05,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: After logical optimization TS[0]-SEL[1]-RS[2]-SEL[3]-FS[4] -SEL[1] 2018-01-05T11:29:05,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,374 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partitions : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,374 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:05,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,381 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 3.475488ms + 0.007347ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? ] 2018-01-05T11:29:05,386 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 4.230453ms + 0.1402ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (13) order by "PART_NAME" asc] 2018-01-05T11:29:05,388 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 2.067491ms + 0.073155ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (13) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:29:05,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.806847ms + 0.045225ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (13) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:05,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 2.086908ms + 0.016181ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (40) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:29:05,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.567316ms + 0.015109ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (40) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:05,395 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.544709ms + 0.052926ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (40) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:05,397 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.985357ms + 0.017163ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (40) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:05,399 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.916523ms + 0.039783ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (27) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:05,401 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.976135ms + 0.048861ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (40) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:29:05,402 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,412 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,413 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,413 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,413 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Estimated average row size: 104 2018-01-05T11:29:05,413 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,413 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_aggr_stats_for: db=default table=srcbucket_mapjoin_part 2018-01-05T11:29:05,414 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_aggr_stats_for: db=default table=srcbucket_mapjoin_part 2018-01-05T11:29:05,416 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.AggregateStatsCache: No aggregate stats cached for database:default, table:srcbucket_mapjoin_part, column:key 2018-01-05T11:29:05,419 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 2.900849ms + 0.006495ms, the query is [select count("COLUMN_NAME") from "PART_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (?,?) and "PARTITION_NAME" in (?) group by "PARTITION_NAME"] 2018-01-05T11:29:05,420 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.719308ms + 0.005425ms, the query is [select "PARTITION_NAME", "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "PART_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (%1$s) AND "PARTITION_NAME" in (%2$s) order by "PARTITION_NAME"] 2018-01-05T11:29:05,420 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] utils.MetaStoreUtils: No stats data found for: dbName=default tblName=srcbucket_mapjoin_part partNames= [ds=2008-04-08] colNames=[key] 2018-01-05T11:29:05,420 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.AggregateStatsCache: No aggregate stats cached for database:default, table:srcbucket_mapjoin_part, column:value 2018-01-05T11:29:05,421 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.124105ms + 0.003611ms, the query is [select "PARTITION_NAME", "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "PART_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (%1$s) AND "PARTITION_NAME" in (%2$s) order by "PARTITION_NAME"] 2018-01-05T11:29:05,421 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] utils.MetaStoreUtils: No stats data found for: dbName=default tblName=srcbucket_mapjoin_part partNames= [ds=2008-04-08] colNames=[value] 2018-01-05T11:29:05,421 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: useDensityFunctionForNDVEstimation = false partsFound = 0 ColumnStatisticsObj = [] 2018-01-05T11:29:05,421 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting stats (Num rows: 55 Data size: 15964 Basic stats: COMPLETE Column stats: NONE) on TS[0] 2018-01-05T11:29:05,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (srcbucket_mapjoin_part): numRows: 55 dataSize: 15964 basicStatsState: COMPLETE colStatsState: NONE colStats: {value= colName: value colType: string countDistincts: 11 numNulls: 2 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: int countDistincts: 11 numNulls: 2 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true} 2018-01-05T11:29:05,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 55 Data size: 15964 Basic stats: COMPLETE Column stats: NONE) on SEL[1] 2018-01-05T11:29:05,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [1] STATS-SEL[1]: numRows: 55 dataSize: 15964 basicStatsState: COMPLETE colStatsState: NONE colStats: {value= colName: value colType: string countDistincts: 11 numNulls: 2 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: int countDistincts: 11 numNulls: 2 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true} 2018-01-05T11:29:05,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 55 Data size: 15964 Basic stats: COMPLETE Column stats: NONE) on RS[2] 2018-01-05T11:29:05,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[2]: numRows: 55 dataSize: 15964 basicStatsState: COMPLETE colStatsState: NONE colStats: {value= colName: value colType: string countDistincts: 11 numNulls: 2 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: int countDistincts: 11 numNulls: 2 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true} 2018-01-05T11:29:05,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 55 Data size: 15964 Basic stats: COMPLETE Column stats: NONE) on SEL[3] 2018-01-05T11:29:05,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [1] STATS-SEL[3]: numRows: 55 dataSize: 15964 basicStatsState: COMPLETE colStatsState: NONE colStats: {value= colName: value colType: string countDistincts: 11 numNulls: 2 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: int countDistincts: 11 numNulls: 2 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true} 2018-01-05T11:29:05,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-FS[4]: numRows: 55 dataSize: 15964 basicStatsState: COMPLETE colStatsState: NONE colStats: {value= colName: value colType: string countDistincts: 11 numNulls: 4 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: int countDistincts: 11 numNulls: 4 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true} 2018-01-05T11:29:05,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 55 Data size: 15964 Basic stats: COMPLETE Column stats: NONE) on SEL[1] 2018-01-05T11:29:05,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [1] STATS-SEL[1]: numRows: 55 dataSize: 15964 basicStatsState: COMPLETE colStatsState: NONE colStats: {value= colName: value colType: string countDistincts: 11 numNulls: 2 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: int countDistincts: 11 numNulls: 2 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true} 2018-01-05T11:29:05,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[2]: inputSize: 15964 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:05,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 1] STATS-GBY[2]: cardinality: 55 2018-01-05T11:29:05,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 55 Data size: 15964 Basic stats: COMPLETE Column stats: NONE) on GBY[2] 2018-01-05T11:29:05,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[2]: numRows: 55 dataSize: 15964 basicStatsState: COMPLETE colStatsState: NONE colStats: {_col2= colName: _col2 colType: struct countDistincts: 55 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 55 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, value= colName: value colType: string countDistincts: 11 numNulls: 2 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: int countDistincts: 11 numNulls: 2 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true} 2018-01-05T11:29:05,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 55 Data size: 15964 Basic stats: COMPLETE Column stats: NONE) on RS[3] 2018-01-05T11:29:05,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[3]: numRows: 55 dataSize: 15964 basicStatsState: COMPLETE colStatsState: NONE colStats: {_col2= colName: _col2 colType: struct countDistincts: 55 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 55 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, value= colName: value colType: string countDistincts: 11 numNulls: 2 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: int countDistincts: 11 numNulls: 2 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true} 2018-01-05T11:29:05,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[4]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:05,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 7] STATS-GBY[4]: cardinality: 27 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 27 Data size: 7836 Basic stats: COMPLETE Column stats: NONE) on GBY[4] 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[4]: numRows: 27 dataSize: 7836 basicStatsState: COMPLETE colStatsState: NONE colStats: {_col2= colName: _col2 colType: struct countDistincts: 55 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 55 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, value= colName: value colType: string countDistincts: 11 numNulls: 2 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: int countDistincts: 11 numNulls: 2 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true} 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 27 Data size: 7836 Basic stats: COMPLETE Column stats: NONE) on SEL[5] 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [1] STATS-SEL[5]: numRows: 27 dataSize: 7836 basicStatsState: COMPLETE colStatsState: NONE colStats: {_col2= colName: _col2 colType: struct countDistincts: 55 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 55 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, value= colName: value colType: string countDistincts: 11 numNulls: 2 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: int countDistincts: 11 numNulls: 2 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true} 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-FS[6]: numRows: 27 dataSize: 7836 basicStatsState: COMPLETE colStatsState: NONE colStats: {_col2= colName: _col2 colType: struct countDistincts: 55 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 55 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, value= colName: value colType: string countDistincts: 11 numNulls: 4 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: int countDistincts: 11 numNulls: 4 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true} 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[]]; bucket count: 4 }) on TS[0] 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[]]; bucket count: 4 }) on SEL[1] 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[KEY.reducesinkkey0]]; sort column names: [[KEY.reducesinkkey0]]; bucket count: 4 }) on RS[2] 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: 4 }) on SEL[3] 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FileSinkOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: 4 }) on FS[4] 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[key]]; bucket count: 4 }) on SEL[1] 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: -1 }) on GBY[2] 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[KEY._col0]]; sort column names: [[KEY._col0]]; bucket count: -1 }) on RS[3] 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: -1 }) on GBY[4] 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col2]]; sort column names: [[_col2]]; bucket count: -1 }) on SEL[5] 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FileSinkOperator: Setting traits ({ bucket column names: [[_col2]]; sort column names: [[_col2]]; bucket count: -1 }) on FS[6] 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,423 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 4 2018-01-05T11:29:05,423 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Set parallelism for reduce sink RS[3] to: 1 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: FS, 4 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: FS, 6 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 5 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: GBY, 4 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 3 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: GBY, 2 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 1 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 3 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 2 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 1 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:29:05,423 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Cycle free: true 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: int|{srcbucket_mapjoin_part}key,value: string|{srcbucket_mapjoin_part}value,ds: string|{srcbucket_mapjoin_part}ds,BLOCK__OFFSET__INSIDE__FILE: bigint|{srcbucket_mapjoin_part}block__offset__inside__file,INPUT__FILE__NAME: string|{srcbucket_mapjoin_part}input__file__name,ROW__ID: struct|{srcbucket_mapjoin_part}row__id) 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(_col0: int|{null}key,_col1: string|{null}value) 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value]) 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[2] with rs:(KEY.reducesinkkey0: int|{null}key,VALUE._col0: string|{null}value) 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[2] 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[3] with rs:(_col0: int|{null}key,_col1: string|{null}value) 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_2 {} 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[3] 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[KEY.reducesinkkey0] Column[VALUE._col0]) 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[4] with rs:(key: int|{},value: string|{}) 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_3 {} 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[4] 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: int|{tab_part}key,value: string|{tab_part}value,ds: string|{tab_part}ds) 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_3 {} 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0] Column[_col1] Const string 2008-04-08) 2018-01-05T11:29:05,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: string|{}(tok_table_or_col ds),_col1: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:29:05,424 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {ds: string=Const string 2008-04-08} 2018-01-05T11:29:05,424 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [_col0: string] to operator GBY[2] 2018-01-05T11:29:05,424 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Replacing column _col0: string with constant Const string 2008-04-08 in GBY[2] 2018-01-05T11:29:05,424 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(KEY._col0: string|{}(tok_table_or_col ds),VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:29:05,424 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {_col0: string=Const string 2008-04-08} 2018-01-05T11:29:05,424 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [KEY._col0: string] to operator RS[3] 2018-01-05T11:29:05,424 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Replacing column KEY._col0: string with constant Const string 2008-04-08 in RS[3] 2018-01-05T11:29:05,424 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: string|{}(tok_table_or_col ds),_col1: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:29:05,424 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {KEY._col0: string=Const string 2008-04-08} 2018-01-05T11:29:05,424 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [_col0: string] to operator GBY[4] 2018-01-05T11:29:05,424 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Replacing column _col0: string with constant Const string 2008-04-08 in GBY[4] 2018-01-05T11:29:05,424 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1,_col2: string|{null}ds) 2018-01-05T11:29:05,424 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {_col0: string=Const string 2008-04-08} 2018-01-05T11:29:05,424 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [_col2: string] to operator SEL[5] 2018-01-05T11:29:05,424 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Replacing column _col2: string with constant Const string 2008-04-08 in SEL[5] 2018-01-05T11:29:05,424 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col1] Column[_col2] Column[_col0]) 2018-01-05T11:29:05,424 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{},_col2: string|{}) 2018-01-05T11:29:05,424 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {_col2: string=Const string 2008-04-08} 2018-01-05T11:29:05,424 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:29:05,424 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,424 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: TS[0] 2018-01-05T11:29:05,424 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[2] 2018-01-05T11:29:05,424 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:29:05,424 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08 of table srcbucket_mapjoin_part 2018-01-05T11:29:05,424 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08 2018-01-05T11:29:05,424 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[2] 2018-01-05T11:29:05,424 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: SEL[3] 2018-01-05T11:29:05,424 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: FS[4] 2018-01-05T11:29:05,424 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding reduce work (Reducer 2) for SEL[3] 2018-01-05T11:29:05,425 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Setting up reduce sink: RS[2] with following reduce work: Reducer 2 2018-01-05T11:29:05,425 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[2] as parent from SEL[3] 2018-01-05T11:29:05,425 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: FS[4] 2018-01-05T11:29:05,425 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: SEL[3] 2018-01-05T11:29:05,425 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[3] 2018-01-05T11:29:05,425 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[3] 2018-01-05T11:29:05,425 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: GBY[4] 2018-01-05T11:29:05,425 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: FS[6] 2018-01-05T11:29:05,425 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding reduce work (Reducer 3) for GBY[4] 2018-01-05T11:29:05,425 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Setting up reduce sink: RS[3] with following reduce work: Reducer 3 2018-01-05T11:29:05,425 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[3] as parent from GBY[4] 2018-01-05T11:29:05,425 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: FS[6] 2018-01-05T11:29:05,425 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,425 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab_part 2018-01-05T11:29:05,425 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab_part 2018-01-05T11:29:05,429 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:29:05,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:29:05,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:05,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:05,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:29:05,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:05,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:05,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:29:05,431 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Examining input format to see if vectorization is enabled. 2018-01-05T11:29:05,431 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorization is enabled for input format(s) [org.apache.hadoop.mapred.TextInputFormat] 2018-01-05T11:29:05,431 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing MapWork... 2018-01-05T11:29:05,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[key], Vectorized Expression = col 0:int 2018-01-05T11:29:05,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[value], Vectorized Expression = col 1:string 2018-01-05T11:29:05,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator 2018-01-05T11:29:05,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.SelectDesc 2018-01-05T11:29:05,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:int 2018-01-05T11:29:05,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col1], Vectorized Expression = col 1:string 2018-01-05T11:29:05,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:int 2018-01-05T11:29:05,431 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator groupby typeName int 2018-01-05T11:29:05,432 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkObjectHashOperator 2018-01-05T11:29:05,435 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:05,435 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkObjectHashOperator: VectorReduceSinkObjectHashOperator constructor vectorReduceSinkInfo org.apache.hadoop.hive.ql.plan.VectorReduceSinkInfo@5bc76448 2018-01-05T11:29:05,435 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkObjectHashOperator 2018-01-05T11:29:05,435 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:05,435 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [key, value, ds, ROW__ID] 2018-01-05T11:29:05,435 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [int, string, string, struct] 2018-01-05T11:29:05,435 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE, NONE, NONE, NONE] 2018-01-05T11:29:05,435 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 1 2018-01-05T11:29:05,435 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 1 2018-01-05T11:29:05,435 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:05,435 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:05,435 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Using reduce tag 0 2018-01-05T11:29:05,435 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0] columnTypes=[string] 2018-01-05T11:29:05,435 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing ReduceWork... 2018-01-05T11:29:05,435 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[KEY.reducesinkkey0], Vectorized Expression = col 0:int 2018-01-05T11:29:05,435 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[VALUE._col0], Vectorized Expression = col 1:string 2018-01-05T11:29:05,435 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator 2018-01-05T11:29:05,435 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.SelectDesc 2018-01-05T11:29:05,437 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorFileSinkOperator 2018-01-05T11:29:05,437 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.FileSinkDesc 2018-01-05T11:29:05,437 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:int 2018-01-05T11:29:05,437 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col1], Vectorized Expression = col 1:string 2018-01-05T11:29:05,438 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Const string 2008-04-08, Vectorized Expression = ConstantVectorExpression(val 2008-04-08) -> 2:string 2018-01-05T11:29:05,438 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator 2018-01-05T11:29:05,438 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.SelectDesc 2018-01-05T11:29:05,439 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Cannot vectorize: Aggregation Function expression for GROUPBY operator: UDF compute_stats not supported 2018-01-05T11:29:05,439 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Using reduce tag 0 2018-01-05T11:29:05,439 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:29:05,439 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing ReduceWork... 2018-01-05T11:29:05,440 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Cannot vectorize: Aggregation Function expression for GROUPBY operator: UDF compute_stats not supported 2018-01-05T11:29:05,440 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:29:05,440 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: llap mode: only 2018-01-05T11:29:05,440 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:29:05,440 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key' 2018-01-05T11:29:05,440 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'value' 2018-01-05T11:29:05,440 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:05,440 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@12436267 can be done in LLAP 2018-01-05T11:29:05,440 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Reducer 2 2018-01-05T11:29:05,440 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'compute_stats(key, 'hll')' 2018-01-05T11:29:05,440 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key' 2018-01-05T11:29:05,440 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:29:05,440 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'compute_stats(value, 'hll')' 2018-01-05T11:29:05,440 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'value' 2018-01-05T11:29:05,440 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:29:05,440 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '_col0' 2018-01-05T11:29:05,440 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '_col1' 2018-01-05T11:29:05,440 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking ''2008-04-08'' 2018-01-05T11:29:05,440 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'KEY.reducesinkkey0' 2018-01-05T11:29:05,440 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:29:05,440 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:05,440 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@daf5692 can be done in LLAP 2018-01-05T11:29:05,440 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Reducer 3 2018-01-05T11:29:05,440 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '_col1' 2018-01-05T11:29:05,440 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '_col2' 2018-01-05T11:29:05,440 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '_col0' 2018-01-05T11:29:05,440 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'compute_stats(VALUE._col0)' 2018-01-05T11:29:05,440 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:29:05,440 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'compute_stats(VALUE._col1)' 2018-01-05T11:29:05,440 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'VALUE._col1' 2018-01-05T11:29:05,440 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:05,440 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@74b8e2c4 can be done in LLAP 2018-01-05T11:29:05,440 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,441 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed plan generation 2018-01-05T11:29:05,441 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:05,441 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: validation start 2018-01-05T11:29:05,441 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: validated ds=2008-04-08 2018-01-05T11:29:05,441 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tab_part 2018-01-05T11:29:05,441 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,441 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partitions_ps_with_auth : db=default tbl=tab_part[2008-04-08] 2018-01-05T11:29:05,441 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions_ps_with_auth : db=default tbl=tab_part[2008-04-08] 2018-01-05T11:29:05,444 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,454 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,454 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,454 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,454 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=tab_part[2008-04-08] 2018-01-05T11:29:05,454 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=tab_part[2008-04-08] 2018-01-05T11:29:05,457 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,459 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:05,459 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:05,459 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,459 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:key, type:int, comment:null), FieldSchema(name:value, type:string, comment:null)], properties:null) 2018-01-05T11:29:05,459 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,459 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:05,459 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {getNotNullConstraints_(NotNullConstraintsRequest, )=1, isCompatibleWith_(Configuration, )=1, getAggrColStatsFor_(String, String, List, List, )=9, getTable_(String, String, )=32, flushCache_()=0, getUniqueConstraints_(UniqueConstraintsRequest, )=0, getPrimaryKeys_(PrimaryKeysRequest, )=1, listPartitions_(String, String, short, )=39, getForeignKeys_(ForeignKeysRequest, )=0, listPartitionsWithAuthInfo_(String, String, List, short, String, List, )=13} 2018-01-05T11:29:05,459 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a); Time taken: 0.216 seconds 2018-01-05T11:29:05,459 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:05,459 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,459 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a): insert overwrite table tab_part partition (ds='2008-04-08') select key,value from srcbucket_mapjoin_part 2018-01-05T11:29:05,461 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,461 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: insert overwrite table tab_part partition (ds='2008-04-08') select key,value from srcbucket_mapjoin_part 2018-01-05T11:29:05,461 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:29:05,461 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@srcbucket_mapjoin_part 2018-01-05T11:29:05,461 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@srcbucket_mapjoin_part@ds=2008-04-08 2018-01-05T11:29:05,461 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: default@tab_part@ds=2008-04-08 2018-01-05T11:29:05,461 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,461 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,461 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,461 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Query ID = djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a 2018-01-05T11:29:05,461 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Total jobs = 1 2018-01-05T11:29:05,461 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,461 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:29:05,461 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-1:MAPRED] in serial mode 2018-01-05T11:29:05,475 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:29:05,500 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:29:05,500 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:29:05,500 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:29:05,500 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a 2018-01-05T11:29:05,500 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Session is already open 2018-01-05T11:29:05,500 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:29:05,500 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,500 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Dag name: insert overwrite tabl...cbucket_mapjoin_part (Stage-1) 2018-01-05T11:29:05,500 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: DagInfo: {"context":"Hive","description":"\ninsert overwrite table tab_part partition (ds='2008-04-08')\nselect key,value from srcbucket_mapjoin_part"} 2018-01-05T11:29:05,500 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:29:05,500 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,513 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,513 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:29:05,516 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,516 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 3 size: 2.66KB 2018-01-05T11:29:05,530 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_318_6335458483535453971-1/-mr-10002 2018-01-05T11:29:05,542 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_318_6335458483535453971-1/-mr-10002 2018-01-05T11:29:05,545 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,545 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,558 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,558 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:29:05,559 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,559 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 2 size: 3.36KB 2018-01-05T11:29:05,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/-mr-10001 2018-01-05T11:29:05,585 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/-mr-10001 2018-01-05T11:29:05,585 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,587 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,604 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:29:05,604 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,604 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:29:05,606 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,606 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 3.15KB 2018-01-05T11:29:05,613 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08 2018-01-05T11:29:05,613 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,618 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=insert overwrite tabl...cbucket_mapjoin_part (Stage-1), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a } 2018-01-05T11:29:05,618 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:05,619 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:05,619 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:05,621 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: insert overwrite tabl...cbucket_mapjoin_part (Stage-1) 2018-01-05T11:29:05,621 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:29:05,622 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_35, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_35.dot 2018-01-05T11:29:05,622 INFO [IPC Server handler 1 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_35 2018-01-05T11:29:05,622 WARN [IPC Server handler 1 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:29:05,622 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Running DAG: insert overwrite tabl...cbucket_mapjoin_part (Stage-1), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a } 2018-01-05T11:29:05,648 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_35, submitTime=1515180545621, queueName=null 2018-01-05T11:29:05,649 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_35_00 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_35_00 [Map 1], Default Resources= 2018-01-05T11:29:05,649 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_35_00 [Map 1] 2018-01-05T11:29:05,649 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:29:05,649 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_35_00 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:05,649 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_35_01 [Reducer 2]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_35_01 [Reducer 2], Default Resources= 2018-01-05T11:29:05,650 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 2 2018-01-05T11:29:05,650 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_35_01 [Reducer 2] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:05,650 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_35_02 [Reducer 3]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_35_02 [Reducer 3], Default Resources= 2018-01-05T11:29:05,650 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_35_02 [Reducer 3] 2018-01-05T11:29:05,650 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 3 2018-01-05T11:29:05,650 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_35_02 [Reducer 3] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:05,650 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:29:05,650 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_35, initTime=1515180545650 2018-01-05T11:29:05,650 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: dag_1515180518813_0001_35 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:29:05,651 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:29:05,651 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_35, startTime=1515180545651 2018-01-05T11:29:05,651 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_35 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:29:05,651 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {srcbucket_mapjoin_part={InputName=srcbucket_mapjoin_part}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:29:05,651 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: srcbucket_mapjoin_part, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:29:05,651 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_35_00 [Map 1] 2018-01-05T11:29:05,651 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_35, dagName=insert overwrite tabl...cbucket_mapjoin_part (Stage-1) 2018-01-05T11:29:05,651 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,651 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,651 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,656 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_35_00 [Map 1] 2018-01-05T11:29:05,656 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_35_00 [Map 1] 2018-01-05T11:29:05,656 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:05,656 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_35_00 [Map 1] 2018-01-05T11:29:05,659 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:29:05,659 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@489d5ad9 2018-01-05T11:29:05,659 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/b0856af9-75bd-4367-875b-5beea68adfa0/map.xml 2018-01-05T11:29:05,659 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/b0856af9-75bd-4367-875b-5beea68adfa0/map.xml 2018-01-05T11:29:05,659 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/b0856af9-75bd-4367-875b-5beea68adfa0/map.xml 2018-01-05T11:29:05,659 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:05,659 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:05,659 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:05,661 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:05,661 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.15KB 2018-01-05T11:29:05,661 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_35_00 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:29:05,662 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to ShuffleVertexManager for vertex_1515180518813_0001_35_01 [Reducer 2] 2018-01-05T11:29:05,661 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: srcbucket_mapjoin_part on vertex vertex_1515180518813_0001_35_00 [Map 1] 2018-01-05T11:29:05,662 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:29:05,662 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:05,662 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:29:05,662 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:05,662 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/b0856af9-75bd-4367-875b-5beea68adfa0/map.xml 2018-01-05T11:29:05,662 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:05,662 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias srcbucket_mapjoin_part 2018-01-05T11:29:05,662 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08 2018-01-05T11:29:05,662 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08 2018-01-05T11:29:05,663 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 0,1 2018-01-05T11:29:05,663 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = key,value 2018-01-05T11:29:05,663 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08 2018-01-05T11:29:05,663 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:05,663 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:05,663 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:05,663 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:05,667 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.05 maxFrac: 0.75 auto: false desiredTaskIput: 104857600 2018-01-05T11:29:05,667 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:29:05,667 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 4 tasks for vertex: vertex_1515180518813_0001_35_01 [Reducer 2] 2018-01-05T11:29:05,667 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_35_01 [Reducer 2] 2018-01-05T11:29:05,667 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_35_01, reconfigureDoneTime=1515180545667, numTasks=4, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:29:05,667 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:VERTEX_INITIALIZED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_35_01, initRequestedTime=1515180545661, initedTime=1515180545667, numTasks=4, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:05,667 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_35_01 [Reducer 2] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:29:05,667 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to ShuffleVertexManager for vertex_1515180518813_0001_35_02 [Reducer 3] 2018-01-05T11:29:05,673 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.05 maxFrac: 0.75 auto: false desiredTaskIput: 104857600 2018-01-05T11:29:05,673 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:29:05,673 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 4 tasks for vertex: vertex_1515180518813_0001_35_02 [Reducer 3] 2018-01-05T11:29:05,674 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_35_02 [Reducer 3] 2018-01-05T11:29:05,674 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_35_02, reconfigureDoneTime=1515180545674, numTasks=4, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:29:05,674 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_35_02 [Reducer 3], numAdditionalOutputs=1 2018-01-05T11:29:05,674 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:VERTEX_INITIALIZED]: vertexName=Reducer 3, vertexId=vertex_1515180518813_0001_35_02, initRequestedTime=1515180545667, initedTime=1515180545674, numTasks=4, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:05,674 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_35_02 [Reducer 3] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:29:05,726 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Time taken to get FileStatuses: 62 2018-01-05T11:29:05,726 INFO [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total input files to process : 4 2018-01-05T11:29:05,726 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total # of splits generated by getSplits: 4, TimeTaken: 62 2018-01-05T11:29:05,726 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 4 2018-01-05T11:29:05,726 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:05,726 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 4. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:29:05,726 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/b0856af9-75bd-4367-875b-5beea68adfa0/map.xml 2018-01-05T11:29:05,726 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/b0856af9-75bd-4367-875b-5beea68adfa0/map.xml 2018-01-05T11:29:05,726 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/b0856af9-75bd-4367-875b-5beea68adfa0/map.xml 2018-01-05T11:29:05,726 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:05,726 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:05,726 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:05,728 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:05,728 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.15KB 2018-01-05T11:29:05,728 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket23.txt to src new group? true 2018-01-05T11:29:05,728 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket22.txt to src new group? false 2018-01-05T11:29:05,728 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket21.txt to src new group? false 2018-01-05T11:29:05,728 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket20.txt to src new group? false 2018-01-05T11:29:05,728 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:29:05,728 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:29:05,728 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:29:05,728 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 107 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 5812 Original splits: 4 2018-01-05T11:29:05,728 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired numSplits: 1 lengthPerGroup: 5812 numLocations: 1 numSplitsPerLocation: 4 numSplitsInGroup: 4 totalLength: 5812 numOriginalSplits: 4 . Grouping by length: true count: false nodeLocalOnly: false 2018-01-05T11:29:05,728 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Number of splits desired: 1 created: 1 splitsProcessed: 4 2018-01-05T11:29:05,728 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 4 grouped split count is 1, for bucket: 1 2018-01-05T11:29:05,728 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:29:05,729 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: srcbucket_mapjoin_part on vertex vertex_1515180518813_0001_35_00 [Map 1] 2018-01-05T11:29:05,729 INFO [App Shared Pool - #1] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_35_00 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:05,729 INFO [App Shared Pool - #1] impl.VertexImpl: Got updated RootInputsSpecs: {srcbucket_mapjoin_part=forAllWorkUnits=true, update=[1]} 2018-01-05T11:29:05,729 INFO [App Shared Pool - #1] impl.VertexImpl: Vertex vertex_1515180518813_0001_35_00 [Map 1] parallelism set to 1 2018-01-05T11:29:05,729 INFO [App Shared Pool - #1] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_35_00 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:05,730 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_35_00, initRequestedTime=1515180545651, initedTime=1515180545730, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:05,730 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_35_00 [Map 1] with distanceFromRoot: 0 2018-01-05T11:29:05,730 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_35_00 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:29:05,730 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_35_00, startRequestedTime=1515180545661, startedTime=1515180545730 2018-01-05T11:29:05,730 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_35_00, reconfigureDoneTime=1515180545730, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:29:05,730 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_35_00 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:05,730 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_35_00 for vertex: vertex_1515180518813_0001_35_01 [Reducer 2] numStartedSources: 1 numSources: 1 2018-01-05T11:29:05,730 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_35_01 [Reducer 2] with distanceFromRoot: 1 2018-01-05T11:29:05,730 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_35_01, startRequestedTime=1515180545730, startedTime=1515180545730 2018-01-05T11:29:05,730 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_35_01 [Reducer 2] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:05,730 INFO [App Shared Pool - #1] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:29:05,730 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_35_01 for vertex: vertex_1515180518813_0001_35_02 [Reducer 3] numStartedSources: 1 numSources: 1 2018-01-05T11:29:05,730 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_35_02 [Reducer 3] with distanceFromRoot: 2 2018-01-05T11:29:05,730 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: OnVertexStarted vertex: Reducer 2 with 0 source tasks and 4 pending tasks 2018-01-05T11:29:05,730 INFO [App Shared Pool - #1] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:29:05,730 INFO [App Shared Pool - #1] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_35_00 [Map 1] 2018-01-05T11:29:05,730 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_35_02, startRequestedTime=1515180545730, startedTime=1515180545730 2018-01-05T11:29:05,730 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_35_02 [Reducer 3] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:05,730 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Map 1 in vertex: Reducer 2 numBipartiteSourceTasks: 1 2018-01-05T11:29:05,730 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_35_00_000000, scheduledTime=1515180545730, launchTime=1515180545730 2018-01-05T11:29:05,730 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: OnVertexStarted vertex: Reducer 3 with 0 source tasks and 4 pending tasks 2018-01-05T11:29:05,730 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_35_00 [Map 1] 2018-01-05T11:29:05,730 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Reducer 2 in vertex: Reducer 3 numBipartiteSourceTasks: 4 2018-01-05T11:29:05,730 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_35_00_000000_0, priority=8, capability=, hosts=null 2018-01-05T11:29:05,730 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=8,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:05,730 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=8,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:05,730 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:05,730 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_35_00_000000_0 on hosts=null 2018-01-05T11:29:05,730 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_35_00_000000_0 on hosts=null 2018-01-05T11:29:05,731 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_35_00_000000_0 2018-01-05T11:29:05,731 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:05,731 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #71, task=TaskInfo{task=attempt_1515180518813_0001_35_00_000000_0, priority=8, startTime=0, containerId=null, uniqueId=70, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000071 2018-01-05T11:29:05,731 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_35_00_000000_0; false 2018-01-05T11:29:05,731 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_35_00_000000_0, priority=8, startTime=311566240, containerId=container_111101111_0001_00_000071assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=70, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:05,731 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000071 for node: 10.22.8.182:49818 2018-01-05T11:29:05,734 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 35, name=insert overwrite tabl...cbucket_mapjoin_part (Stage-1), queryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a 2018-01-05T11:29:05,734 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000071, attempt_1515180518813_0001_35_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:05,734 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000071 for node: 10.22.8.182:49818 2018-01-05T11:29:05,734 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000071 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:05,734 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000071, launchTime=1515180545734 2018-01-05T11:29:05,735 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_35_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000071] on NM: [10.22.8.182:49818] 2018-01-05T11:29:05,735 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_35_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000071, dagName=insert overwrite tabl...cbucket_mapjoin_part (Stage-1), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{srcbucket_mapjoin_part,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{Reducer 2,org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput,4}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180545651, firstAttemptStartTime=1515180545730, currentAttemptStartTime=1515180545734} 2018-01-05T11:29:05,735 INFO [IPC Server handler 0 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000071, QueryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a-35, DagId=35, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180545735 2018-01-05T11:29:05,735 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:05,736 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:05,736 DEBUG [IPC Server handler 0 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=35} with the ShuffleHandler 2018-01-05T11:29:05,736 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_35_00_000000_0, startTime=1515180545734, containerId=container_111101111_0001_00_000071, nodeId=10.22.8.182:49818 2018-01-05T11:29:05,737 DEBUG [AMReporterQueueDrainer] impl.AMReporter: Removing am 10.22.8.182:49824 with last associated dag QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=20} from heartbeat with taskCount=0, amFailed=false, isDone=true 2018-01-05T11:29:05,752 INFO [IPC Server handler 0 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:29:05,752 DEBUG [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/35] 2018-01-05T11:29:05,752 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:29:05,752 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_35_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:05,752 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_35_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180545730, dagStartTime=1515180545651, withinDagPriority=8, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:29:05,752 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_35_00_000000_0 : ACCEPTED 2018-01-05T11:29:05,752 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_35_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:29:05,752 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_35_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180545730, dagStartTime=1515180545651, withinDagPriority=8, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:29:05,752 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_35_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:05,752 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_35_00_000000_0 to pre-emption queue 2018-01-05T11:29:05,752 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_35_00_000000_0: true 2018-01-05T11:29:05,752 INFO [TaskCommunicator # 7] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_35_00_000000_0 2018-01-05T11:29:05,752 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: taskOwner hashCode:1719074333 2018-01-05T11:29:05,752 INFO [TaskCommunicator # 7] GuaranteedTasks: Received response for attempt_1515180518813_0001_35_00_000000_0, true 2018-01-05T11:29:05,753 INFO [Task-Executor-1] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : insert overwrite tabl...cbucket_mapjoin_part (Stage-1), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_35_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=srcbucket_mapjoin_part, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=Reducer 2, physicalEdgeCount=4, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ] 2018-01-05T11:29:05,753 INFO [Task-Executor-1] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1028128768, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:05,754 INFO [Task-Executor-1] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_35_00_000000_0 vertexName: Map 1 2018-01-05T11:29:05,754 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000071, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_35_00_000000_0, eventCount=1 } 2018-01-05T11:29:05,754 DEBUG [TezTR-518813_1_35_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_35 queryId: djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a instanceType: TaskRunner2Callable 2018-01-05T11:29:05,754 INFO [TezTR-518813_1_35_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_35_00_000000_0 2018-01-05T11:29:05,754 DEBUG [TezTR-518813_1_35_0_0_0] log.PerfLogger: 2018-01-05T11:29:05,755 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:05,755 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:05,756 INFO [IPC Server handler 29 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_35_00_000000_0 sent events: (0-1). 2018-01-05T11:29:05,757 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:29:05,757 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_35_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:29:05,757 DEBUG [TezTR-518813_1_35_0_0_0] log.PerfLogger: 2018-01-05T11:29:05,757 INFO [TezTR-518813_1_35_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:05,757 INFO [TezTR-518813_1_35_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:05,757 INFO [TezTR-518813_1_35_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:05,758 INFO [I/O Setup 0 Initialize: {srcbucket_mapjoin_part}] input.MRInput: srcbucket_mapjoin_part using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:29:05,758 INFO [I/O Setup 0 Initialize: {srcbucket_mapjoin_part}] input.MRInputLegacy: srcbucket_mapjoin_part MRInputLegacy deferring initialization 2018-01-05T11:29:05,758 INFO [TezTR-518813_1_35_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:05,758 INFO [TezTR-518813_1_35_0_0_0] resources.MemoryDistributor: InitialRequests=[srcbucket_mapjoin_part:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [Reducer 2:OUTPUT:10485760:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput] 2018-01-05T11:29:05,758 INFO [TezTR-518813_1_35_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:05,758 INFO [TezTR-518813_1_35_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:05,758 INFO [TezTR-518813_1_35_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 10485760, TotalRequestedScaled: 9679163.076923078, TotalJVMHeap: 1028128768, TotalAvailable: 688846274, TotalRequested/TotalJVMHeap:0.01 2018-01-05T11:29:05,758 INFO [TezTR-518813_1_35_0_0_0] resources.MemoryDistributor: Allocations=[srcbucket_mapjoin_part:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:10485760:10485760] 2018-01-05T11:29:05,758 INFO [TezTR-518813_1_35_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:05,758 INFO [TezTR-518813_1_35_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: srcbucket_mapjoin_part being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:05,758 INFO [TezTR-518813_1_35_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:29:05,758 INFO [TezTR-518813_1_35_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:05,758 INFO [I/O Setup 1 Start: {srcbucket_mapjoin_part}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: srcbucket_mapjoin_part 2018-01-05T11:29:05,758 INFO [TezTR-518813_1_35_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:05,758 INFO [TezTR-518813_1_35_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_35_00_000000_0 2018-01-05T11:29:05,758 DEBUG [TezTR-518813_1_35_0_0_0] log.PerfLogger: 2018-01-05T11:29:05,758 DEBUG [TezTR-518813_1_35_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_35_00_000000_0_10214 2018-01-05T11:29:05,758 DEBUG [TezTR-518813_1_35_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:05,758 DEBUG [TezTR-518813_1_35_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_35_00_000000_0 2018-01-05T11:29:05,758 INFO [TezTR-518813_1_35_0_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a 2018-01-05T11:29:05,759 DEBUG [TezTR-518813_1_35_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:05,759 DEBUG [TezTR-518813_1_35_0_0_0] log.PerfLogger: 2018-01-05T11:29:05,759 DEBUG [TezTR-518813_1_35_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:05,759 DEBUG [TezTR-518813_1_35_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:05,759 DEBUG [TezTR-518813_1_35_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a_Map 1__MAP_PLAN__ 2018-01-05T11:29:05,759 INFO [TezTR-518813_1_35_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/b0856af9-75bd-4367-875b-5beea68adfa0/map.xml 2018-01-05T11:29:05,759 DEBUG [TezTR-518813_1_35_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/b0856af9-75bd-4367-875b-5beea68adfa0/map.xml 2018-01-05T11:29:05,759 DEBUG [TezTR-518813_1_35_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/b0856af9-75bd-4367-875b-5beea68adfa0/map.xml 2018-01-05T11:29:05,759 DEBUG [TezTR-518813_1_35_0_0_0] log.PerfLogger: 2018-01-05T11:29:05,759 INFO [TezTR-518813_1_35_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:05,759 DEBUG [TezTR-518813_1_35_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:05,760 DEBUG [TezTR-518813_1_35_0_0_0] log.PerfLogger: 2018-01-05T11:29:05,760 INFO [TezTR-518813_1_35_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.15KB 2018-01-05T11:29:05,761 DEBUG [TezTR-518813_1_35_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:05,761 INFO [TezTR-518813_1_35_0_0_0] tez.MapRecordProcessor: The input names are: [srcbucket_mapjoin_part] 2018-01-05T11:29:05,761 INFO [TezTR-518813_1_35_0_0_0] input.MRInput: Initialized MRInput: srcbucket_mapjoin_part 2018-01-05T11:29:05,761 DEBUG [TezTR-518813_1_35_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 4 2018-01-05T11:29:05,761 INFO [TezTR-518813_1_35_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/b0856af9-75bd-4367-875b-5beea68adfa0/map.xml 2018-01-05T11:29:05,761 DEBUG [TezTR-518813_1_35_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:05,761 DEBUG [TezTR-518813_1_35_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket23.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcbucket_mapjoin_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct srcbucket_mapjoin_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08, columns=key,value, partition_columns.types=string, serialization.format=1, numRows=0, numFiles=4, serialization.ddl=struct srcbucket_mapjoin_part { i32 key, string value}, transient_lastDdlTime=1515180545, rawDataSize=0, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket_mapjoin_part}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTOR_DESERIALIZE, input file format class name org.apache.hadoop.mapred.TextInputFormat, deserialize type LAZY_SIMPLE] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcbucket_mapjoin_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct srcbucket_mapjoin_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08, columns=key,value, partition_columns.types=string, serialization.format=1, numRows=0, numFiles=4, serialization.ddl=struct srcbucket_mapjoin_part { i32 key, string value}, transient_lastDdlTime=1515180545, rawDataSize=0, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket_mapjoin_part}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTOR_DESERIALIZE, input file format class name org.apache.hadoop.mapred.TextInputFormat, deserialize type LAZY_SIMPLE]} 2018-01-05T11:29:05,762 INFO [TezTR-518813_1_35_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/b0856af9-75bd-4367-875b-5beea68adfa0/map.xml 2018-01-05T11:29:05,762 DEBUG [TezTR-518813_1_35_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:05,762 DEBUG [TezTR-518813_1_35_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:29:05,762 INFO [TezTR-518813_1_35_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:29:05,762 DEBUG [TezTR-518813_1_35_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:29:05,762 INFO [TezTR-518813_1_35_0_0_0] io.HiveInputFormat: Resetting VectorMapOperatorReadType from VECTOR_DESERIALIZE for partition default.srcbucket_mapjoin_part {ds=2008-04-08} 2018-01-05T11:29:05,763 DEBUG [TezTR-518813_1_35_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:29:05,763 INFO [TezTR-518813_1_35_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/b0856af9-75bd-4367-875b-5beea68adfa0/map.xml 2018-01-05T11:29:05,763 DEBUG [TezTR-518813_1_35_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:05,764 INFO [TezTR-518813_1_35_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_35_00_000000_0 2018-01-05T11:29:05,764 INFO [TezTR-518813_1_35_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:05,764 DEBUG [TezTR-518813_1_35_0_0_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:05,764 DEBUG [TezTR-518813_1_35_0_0_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:05,764 DEBUG [TezTR-518813_1_35_0_0_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:05,764 DEBUG [TezTR-518813_1_35_0_0_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:05,764 WARN [TezTR-518813_1_35_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket23.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:29:05,764 DEBUG [TezTR-518813_1_35_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:29:05,764 INFO [TezTR-518813_1_35_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/b0856af9-75bd-4367-875b-5beea68adfa0/map.xml 2018-01-05T11:29:05,764 DEBUG [TezTR-518813_1_35_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:05,765 DEBUG [TezTR-518813_1_35_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket23.txt 2018-01-05T11:29:05,765 DEBUG [IO-Elevator-Thread-4] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_35 queryId: djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a instanceType: SerDeEncodedDataReader 2018-01-05T11:29:05,765 INFO [TezTR-518813_1_35_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket23.txt:0+1612, org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket22.txt:0+1450, org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket21.txt:0+1392, org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket20.txt:0+1358], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=5812} 2018-01-05T11:29:05,765 INFO [IO-Elevator-Thread-4] LlapIoImpl: Processing data for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket23.txt 2018-01-05T11:29:05,765 INFO [TezTR-518813_1_35_0_0_0] input.MRInput: srcbucket_mapjoin_part initialized RecordReader from event 2018-01-05T11:29:05,765 DEBUG [TezTR-518813_1_35_0_0_0] tez.MapRecordProcessor: Starting Output: Reducer 2 2018-01-05T11:29:05,765 INFO [IO-Elevator-Thread-4] LlapIoImpl: Processing one split {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket23.txt, 0, 1612} 2018-01-05T11:29:05,766 INFO [TezTR-518813_1_35_0_0_0] impl.ExternalSorter: Reducer 2 using: memoryMb=10, keySerializerClass=class org.apache.hadoop.hive.ql.io.HiveKey, valueSerializerClass=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization$TezBytesWritableSerializer@4ca9315d, comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator@2189f90c, partitioner=org.apache.tez.mapreduce.partition.MRPartitioner, serialization=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.hadoop.io.serializer.WritableSerialization, org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization, org.apache.hadoop.io.serializer.avro.AvroReflectSerialization, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:29:05,766 INFO [TezTR-518813_1_35_0_0_0] partition.MRPartitioner: Using oldApi, MRpartitionerClass=org.apache.hadoop.hive.ql.io.DefaultHivePartitioner 2018-01-05T11:29:05,766 INFO [TezTR-518813_1_35_0_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 2: , UsingHashComparator=true 2018-01-05T11:29:05,766 DEBUG [IO-Elevator-Thread-4] LlapIoImpl: Using LineRecordReader to read data 2018-01-05T11:29:05,766 INFO [IO-Elevator-Thread-4] LlapIoImpl: Creating VertorDeserializeOrcWriter for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket23.txt 2018-01-05T11:29:05,767 INFO [IO-Elevator-Thread-4] impl.WriterImpl: ORC writer created for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket23.txt with stripeSize: 9223372036854775807 blockSize: 9223372036854775807 compression: NONE bufferSize: 262144 2018-01-05T11:29:05,767 DEBUG [IO-Elevator-Thread-4] LlapIoImpl: Processing slice #0 null; has no cache data; has disk data 2018-01-05T11:29:05,767 DEBUG [IO-Elevator-Thread-4] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: int streamBuffers.length: 0 vectors: 1 columnEncoding: null present: false data: null dictionary: false lengths: false secondary: false tz: US/Pacific 2018-01-05T11:29:05,767 DEBUG [IO-Elevator-Thread-4] encoded.EncodedTreeReaderFactory: columnIndex: 2 columnType: string streamBuffers.length: 0 vectors: 1 columnEncoding: null present: false data: null dictionary: false lengths: false secondary: false tz: US/Pacific 2018-01-05T11:29:05,767 DEBUG [IO-Elevator-Thread-4] LlapIoOrc: Done with decode 2018-01-05T11:29:05,767 DEBUG [IO-Elevator-Thread-4] LlapIoImpl: setDone called; closed false, done false, err null, pending 1 2018-01-05T11:29:05,768 DEBUG [IO-Elevator-Thread-4] impl.StatsRecordingThreadPool: Updated stats: instance: SerDeEncodedDataReader thread name: IO-Elevator-Thread-4 thread id: 2194 scheme: file bytesRead: 1640 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:05,768 INFO [TezTR-518813_1_35_0_0_0] impl.PipelinedSorter: Newly allocated block size=10485760, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=10485760, total=10485760 2018-01-05T11:29:05,768 INFO [TezTR-518813_1_35_0_0_0] impl.PipelinedSorter: Pre allocating rest of memory buffers upfront 2018-01-05T11:29:05,768 INFO [TezTR-518813_1_35_0_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 2: , UsingHashComparator=true#blocks=1, maxMemUsage=10485760, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=10485760, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=10 2018-01-05T11:29:05,768 INFO [TezTR-518813_1_35_0_0_0] impl.PipelinedSorter: Reducer 2: reserved.remaining()=10485760, reserved.metasize=5242880 2018-01-05T11:29:05,768 INFO [TezTR-518813_1_35_0_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:29:05,768 INFO [TezTR-518813_1_35_0_0_0] vector.VectorMapOperator: Initializing operator MAP[0] 2018-01-05T11:29:05,768 DEBUG [TezTR-518813_1_35_0_0_0] vector.VectorMapOperator: Initialization Done 0 MAP 2018-01-05T11:29:05,768 DEBUG [TezTR-518813_1_35_0_0_0] vector.VectorMapOperator: Operator 0 MAP initialized 2018-01-05T11:29:05,768 DEBUG [TezTR-518813_1_35_0_0_0] vector.VectorMapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:29:05,768 INFO [TezTR-518813_1_35_0_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:29:05,768 INFO [TezTR-518813_1_35_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:05,768 DEBUG [TezTR-518813_1_35_0_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:29:05,768 INFO [TezTR-518813_1_35_0_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:29:05,768 DEBUG [TezTR-518813_1_35_0_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:29:05,768 DEBUG [TezTR-518813_1_35_0_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:29:05,768 DEBUG [TezTR-518813_1_35_0_0_0] exec.TableScanOperator: Initializing children of 0 TS 2018-01-05T11:29:05,768 DEBUG [TezTR-518813_1_35_0_0_0] vector.VectorSelectOperator: Initializing child 5 SEL 2018-01-05T11:29:05,768 INFO [TezTR-518813_1_35_0_0_0] vector.VectorSelectOperator: Initializing operator SEL[5] 2018-01-05T11:29:05,770 DEBUG [TezTR-518813_1_35_0_0_0] vector.VectorSelectOperator: Initialization Done 5 SEL 2018-01-05T11:29:05,770 DEBUG [TezTR-518813_1_35_0_0_0] vector.VectorSelectOperator: Operator 5 SEL initialized 2018-01-05T11:29:05,770 DEBUG [TezTR-518813_1_35_0_0_0] vector.VectorSelectOperator: Initializing children of 5 SEL 2018-01-05T11:29:05,770 DEBUG [TezTR-518813_1_35_0_0_0] reducesink.VectorReduceSinkObjectHashOperator: Initializing child 6 RS 2018-01-05T11:29:05,770 INFO [TezTR-518813_1_35_0_0_0] reducesink.VectorReduceSinkObjectHashOperator: Initializing operator RS[6] 2018-01-05T11:29:05,770 INFO [TezTR-518813_1_35_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/b0856af9-75bd-4367-875b-5beea68adfa0/map.xml 2018-01-05T11:29:05,770 DEBUG [TezTR-518813_1_35_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:05,770 INFO [TezTR-518813_1_35_0_0_0] reducesink.VectorReduceSinkCommonOperator: Using tag = -1 2018-01-05T11:29:05,770 DEBUG [TezTR-518813_1_35_0_0_0] reducesink.VectorReduceSinkObjectHashOperator: Initialization Done 6 RS 2018-01-05T11:29:05,770 DEBUG [TezTR-518813_1_35_0_0_0] reducesink.VectorReduceSinkObjectHashOperator: Operator 6 RS initialized 2018-01-05T11:29:05,770 DEBUG [TezTR-518813_1_35_0_0_0] reducesink.VectorReduceSinkObjectHashOperator: Initialization Done 6 RS done is reset. 2018-01-05T11:29:05,770 DEBUG [TezTR-518813_1_35_0_0_0] vector.VectorSelectOperator: Initialization Done 5 SEL done is reset. 2018-01-05T11:29:05,770 DEBUG [TezTR-518813_1_35_0_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:29:05,770 INFO [TezTR-518813_1_35_0_0_0] exec.OperatorUtils: Setting output collector: RS[6] --> Reducer 2 2018-01-05T11:29:05,770 DEBUG [TezTR-518813_1_35_0_0_0] log.PerfLogger: 2018-01-05T11:29:05,772 INFO [TezTR-518813_1_35_0_0_0] reducesink.VectorReduceSinkCommonOperator: RS[6]: records written - 1 2018-01-05T11:29:05,773 INFO [TezTR-518813_1_35_0_0_0] reducesink.VectorReduceSinkCommonOperator: RS[6]: records written - 10 2018-01-05T11:29:05,774 INFO [TezTR-518813_1_35_0_0_0] reducesink.VectorReduceSinkCommonOperator: RS[6]: records written - 100 2018-01-05T11:29:05,778 INFO [TezTR-518813_1_35_0_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:05,778 INFO [TezTR-518813_1_35_0_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, , , ]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=0, NUM_ERRORS=0, ROWS_EMITTED=138, METADATA_CACHE_HIT=0, METADATA_CACHE_MISS=0, CACHE_HIT_BYTES=0, CACHE_MISS_BYTES=1612, ALLOCATED_BYTES=0, ALLOCATED_USED_BYTES=0, TOTAL_IO_TIME_NS=2621815, DECODE_TIME_NS=176058, HDFS_TIME_NS=0, CONSUMER_TIME_NS=2765694 ] 2018-01-05T11:29:05,778 DEBUG [TezTR-518813_1_35_0_0_0] LlapIoImpl: Encoded reader is being stopped 2018-01-05T11:29:05,778 DEBUG [TezTR-518813_1_35_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 1 of 4 2018-01-05T11:29:05,778 DEBUG [TezTR-518813_1_35_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket22.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcbucket_mapjoin_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct srcbucket_mapjoin_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08, columns=key,value, partition_columns.types=string, serialization.format=1, numRows=0, numFiles=4, serialization.ddl=struct srcbucket_mapjoin_part { i32 key, string value}, transient_lastDdlTime=1515180545, rawDataSize=0, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket_mapjoin_part}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.mapred.TextInputFormat] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcbucket_mapjoin_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct srcbucket_mapjoin_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08, columns=key,value, partition_columns.types=string, serialization.format=1, numRows=0, numFiles=4, serialization.ddl=struct srcbucket_mapjoin_part { i32 key, string value}, transient_lastDdlTime=1515180545, rawDataSize=0, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket_mapjoin_part}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.mapred.TextInputFormat]} 2018-01-05T11:29:05,778 INFO [TezTR-518813_1_35_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/b0856af9-75bd-4367-875b-5beea68adfa0/map.xml 2018-01-05T11:29:05,778 DEBUG [TezTR-518813_1_35_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:05,778 DEBUG [TezTR-518813_1_35_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:29:05,778 INFO [TezTR-518813_1_35_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:29:05,778 DEBUG [TezTR-518813_1_35_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:29:05,780 DEBUG [TezTR-518813_1_35_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:29:05,780 INFO [TezTR-518813_1_35_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/b0856af9-75bd-4367-875b-5beea68adfa0/map.xml 2018-01-05T11:29:05,780 DEBUG [TezTR-518813_1_35_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:05,780 INFO [TezTR-518813_1_35_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_35_00_000000_0 2018-01-05T11:29:05,780 INFO [TezTR-518813_1_35_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:05,781 WARN [TezTR-518813_1_35_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket22.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:29:05,781 DEBUG [TezTR-518813_1_35_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:29:05,781 INFO [TezTR-518813_1_35_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/b0856af9-75bd-4367-875b-5beea68adfa0/map.xml 2018-01-05T11:29:05,781 DEBUG [TezTR-518813_1_35_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:05,781 DEBUG [TezTR-518813_1_35_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket22.txt 2018-01-05T11:29:05,781 DEBUG [IO-Elevator-Thread-5] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_35 queryId: djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a instanceType: SerDeEncodedDataReader 2018-01-05T11:29:05,781 INFO [IO-Elevator-Thread-5] LlapIoImpl: Processing data for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket22.txt 2018-01-05T11:29:05,781 INFO [IO-Elevator-Thread-5] LlapIoImpl: Processing one split {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket22.txt, 0, 1450} 2018-01-05T11:29:05,781 INFO [IO-Elevator-Thread-4-OrcEncode] LlapIoImpl: ORC encoder received a exit event 2018-01-05T11:29:05,782 DEBUG [IO-Elevator-Thread-4-OrcEncode] LlapIoImpl: Discarding disk data (if any wasn't cached) 2018-01-05T11:29:05,782 DEBUG [IO-Elevator-Thread-5] LlapIoImpl: Using LineRecordReader to read data 2018-01-05T11:29:05,782 INFO [IO-Elevator-Thread-5] LlapIoImpl: Creating VertorDeserializeOrcWriter for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket22.txt 2018-01-05T11:29:05,782 INFO [IO-Elevator-Thread-5] impl.WriterImpl: ORC writer created for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket22.txt with stripeSize: 9223372036854775807 blockSize: 9223372036854775807 compression: NONE bufferSize: 262144 2018-01-05T11:29:05,783 DEBUG [IO-Elevator-Thread-5] LlapIoImpl: Processing slice #0 null; has no cache data; has disk data 2018-01-05T11:29:05,783 DEBUG [IO-Elevator-Thread-5] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: int streamBuffers.length: 0 vectors: 1 columnEncoding: null present: false data: null dictionary: false lengths: false secondary: false tz: US/Pacific 2018-01-05T11:29:05,783 DEBUG [IO-Elevator-Thread-5] encoded.EncodedTreeReaderFactory: columnIndex: 2 columnType: string streamBuffers.length: 0 vectors: 1 columnEncoding: null present: false data: null dictionary: false lengths: false secondary: false tz: US/Pacific 2018-01-05T11:29:05,783 DEBUG [IO-Elevator-Thread-5] LlapIoOrc: Done with decode 2018-01-05T11:29:05,783 DEBUG [IO-Elevator-Thread-5] LlapIoImpl: setDone called; closed false, done false, err null, pending 1 2018-01-05T11:29:05,784 DEBUG [IO-Elevator-Thread-5] impl.StatsRecordingThreadPool: Updated stats: instance: SerDeEncodedDataReader thread name: IO-Elevator-Thread-5 thread id: 2196 scheme: file bytesRead: 1474 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:05,785 INFO [TezTR-518813_1_35_0_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:05,785 INFO [TezTR-518813_1_35_0_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, , , ]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=0, NUM_ERRORS=0, ROWS_EMITTED=124, METADATA_CACHE_HIT=0, METADATA_CACHE_MISS=0, CACHE_HIT_BYTES=0, CACHE_MISS_BYTES=1450, ALLOCATED_BYTES=0, ALLOCATED_USED_BYTES=0, TOTAL_IO_TIME_NS=1633125, DECODE_TIME_NS=127532, HDFS_TIME_NS=0, CONSUMER_TIME_NS=1211711 ] 2018-01-05T11:29:05,785 DEBUG [TezTR-518813_1_35_0_0_0] LlapIoImpl: Encoded reader is being stopped 2018-01-05T11:29:05,785 DEBUG [TezTR-518813_1_35_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 2 of 4 2018-01-05T11:29:05,785 DEBUG [TezTR-518813_1_35_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket21.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcbucket_mapjoin_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct srcbucket_mapjoin_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08, columns=key,value, partition_columns.types=string, serialization.format=1, numRows=0, numFiles=4, serialization.ddl=struct srcbucket_mapjoin_part { i32 key, string value}, transient_lastDdlTime=1515180545, rawDataSize=0, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket_mapjoin_part}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.mapred.TextInputFormat] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcbucket_mapjoin_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct srcbucket_mapjoin_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08, columns=key,value, partition_columns.types=string, serialization.format=1, numRows=0, numFiles=4, serialization.ddl=struct srcbucket_mapjoin_part { i32 key, string value}, transient_lastDdlTime=1515180545, rawDataSize=0, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket_mapjoin_part}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.mapred.TextInputFormat]} 2018-01-05T11:29:05,785 INFO [TezTR-518813_1_35_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/b0856af9-75bd-4367-875b-5beea68adfa0/map.xml 2018-01-05T11:29:05,785 DEBUG [TezTR-518813_1_35_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:05,785 DEBUG [TezTR-518813_1_35_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:29:05,785 INFO [TezTR-518813_1_35_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:29:05,785 DEBUG [TezTR-518813_1_35_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:29:05,787 DEBUG [TezTR-518813_1_35_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:29:05,787 INFO [TezTR-518813_1_35_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/b0856af9-75bd-4367-875b-5beea68adfa0/map.xml 2018-01-05T11:29:05,787 DEBUG [TezTR-518813_1_35_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:05,787 INFO [TezTR-518813_1_35_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_35_00_000000_0 2018-01-05T11:29:05,787 INFO [TezTR-518813_1_35_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:05,787 WARN [TezTR-518813_1_35_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket21.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:29:05,787 DEBUG [TezTR-518813_1_35_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:29:05,787 INFO [TezTR-518813_1_35_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/b0856af9-75bd-4367-875b-5beea68adfa0/map.xml 2018-01-05T11:29:05,787 DEBUG [TezTR-518813_1_35_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:05,788 DEBUG [TezTR-518813_1_35_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket21.txt 2018-01-05T11:29:05,788 DEBUG [IO-Elevator-Thread-6] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_35 queryId: djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a instanceType: SerDeEncodedDataReader 2018-01-05T11:29:05,788 INFO [IO-Elevator-Thread-6] LlapIoImpl: Processing data for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket21.txt 2018-01-05T11:29:05,788 INFO [IO-Elevator-Thread-6] LlapIoImpl: Processing one split {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket21.txt, 0, 1392} 2018-01-05T11:29:05,788 DEBUG [IO-Elevator-Thread-6] LlapIoImpl: Using LineRecordReader to read data 2018-01-05T11:29:05,788 INFO [IO-Elevator-Thread-6] LlapIoImpl: Creating VertorDeserializeOrcWriter for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket21.txt 2018-01-05T11:29:05,789 INFO [IO-Elevator-Thread-6] impl.WriterImpl: ORC writer created for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket21.txt with stripeSize: 9223372036854775807 blockSize: 9223372036854775807 compression: NONE bufferSize: 262144 2018-01-05T11:29:05,789 DEBUG [IO-Elevator-Thread-6] LlapIoImpl: Processing slice #0 null; has no cache data; has disk data 2018-01-05T11:29:05,789 DEBUG [IO-Elevator-Thread-6] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: int streamBuffers.length: 0 vectors: 1 columnEncoding: null present: false data: null dictionary: false lengths: false secondary: false tz: US/Pacific 2018-01-05T11:29:05,789 DEBUG [IO-Elevator-Thread-6] encoded.EncodedTreeReaderFactory: columnIndex: 2 columnType: string streamBuffers.length: 0 vectors: 1 columnEncoding: null present: false data: null dictionary: false lengths: false secondary: false tz: US/Pacific 2018-01-05T11:29:05,789 DEBUG [IO-Elevator-Thread-6] LlapIoOrc: Done with decode 2018-01-05T11:29:05,789 DEBUG [IO-Elevator-Thread-6] LlapIoImpl: setDone called; closed false, done false, err null, pending 1 2018-01-05T11:29:05,789 DEBUG [IO-Elevator-Thread-6] impl.StatsRecordingThreadPool: Updated stats: instance: SerDeEncodedDataReader thread name: IO-Elevator-Thread-6 thread id: 2198 scheme: file bytesRead: 1416 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:05,790 INFO [TezTR-518813_1_35_0_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:05,790 INFO [TezTR-518813_1_35_0_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, , , ]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=0, NUM_ERRORS=0, ROWS_EMITTED=120, METADATA_CACHE_HIT=0, METADATA_CACHE_MISS=0, CACHE_HIT_BYTES=0, CACHE_MISS_BYTES=1392, ALLOCATED_BYTES=0, ALLOCATED_USED_BYTES=0, TOTAL_IO_TIME_NS=1347761, DECODE_TIME_NS=75724, HDFS_TIME_NS=0, CONSUMER_TIME_NS=695819 ] 2018-01-05T11:29:05,790 DEBUG [TezTR-518813_1_35_0_0_0] LlapIoImpl: Encoded reader is being stopped 2018-01-05T11:29:05,790 DEBUG [TezTR-518813_1_35_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 3 of 4 2018-01-05T11:29:05,790 DEBUG [TezTR-518813_1_35_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket20.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcbucket_mapjoin_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct srcbucket_mapjoin_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08, columns=key,value, partition_columns.types=string, serialization.format=1, numRows=0, numFiles=4, serialization.ddl=struct srcbucket_mapjoin_part { i32 key, string value}, transient_lastDdlTime=1515180545, rawDataSize=0, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket_mapjoin_part}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.mapred.TextInputFormat] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcbucket_mapjoin_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct srcbucket_mapjoin_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08, columns=key,value, partition_columns.types=string, serialization.format=1, numRows=0, numFiles=4, serialization.ddl=struct srcbucket_mapjoin_part { i32 key, string value}, transient_lastDdlTime=1515180545, rawDataSize=0, columns.comments=, totalSize=5812, bucket_count=4, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket_mapjoin_part}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.mapred.TextInputFormat]} 2018-01-05T11:29:05,790 INFO [TezTR-518813_1_35_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/b0856af9-75bd-4367-875b-5beea68adfa0/map.xml 2018-01-05T11:29:05,790 DEBUG [TezTR-518813_1_35_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:05,790 DEBUG [TezTR-518813_1_35_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:29:05,790 INFO [TezTR-518813_1_35_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:29:05,790 DEBUG [TezTR-518813_1_35_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:29:05,792 DEBUG [TezTR-518813_1_35_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:29:05,792 INFO [TezTR-518813_1_35_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/b0856af9-75bd-4367-875b-5beea68adfa0/map.xml 2018-01-05T11:29:05,792 DEBUG [TezTR-518813_1_35_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:05,792 INFO [TezTR-518813_1_35_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_35_00_000000_0 2018-01-05T11:29:05,792 INFO [TezTR-518813_1_35_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:05,792 WARN [TezTR-518813_1_35_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket20.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:29:05,792 DEBUG [TezTR-518813_1_35_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:29:05,792 INFO [TezTR-518813_1_35_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/b0856af9-75bd-4367-875b-5beea68adfa0/map.xml 2018-01-05T11:29:05,792 DEBUG [TezTR-518813_1_35_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:05,793 DEBUG [IO-Elevator-Thread-7] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_35 queryId: djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a instanceType: SerDeEncodedDataReader 2018-01-05T11:29:05,793 INFO [IO-Elevator-Thread-7] LlapIoImpl: Processing data for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket20.txt 2018-01-05T11:29:05,793 INFO [IO-Elevator-Thread-7] LlapIoImpl: Processing one split {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket20.txt, 0, 1358} 2018-01-05T11:29:05,793 INFO [IO-Elevator-Thread-5-OrcEncode] LlapIoImpl: ORC encoder received a exit event 2018-01-05T11:29:05,793 DEBUG [IO-Elevator-Thread-5-OrcEncode] LlapIoImpl: Discarding disk data (if any wasn't cached) 2018-01-05T11:29:05,793 DEBUG [IO-Elevator-Thread-7] LlapIoImpl: Using LineRecordReader to read data 2018-01-05T11:29:05,793 DEBUG [TezTR-518813_1_35_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket20.txt 2018-01-05T11:29:05,793 INFO [IO-Elevator-Thread-7] LlapIoImpl: Creating VertorDeserializeOrcWriter for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket20.txt 2018-01-05T11:29:05,793 INFO [IO-Elevator-Thread-7] impl.WriterImpl: ORC writer created for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part/ds=2008-04-08/srcbucket20.txt with stripeSize: 9223372036854775807 blockSize: 9223372036854775807 compression: NONE bufferSize: 262144 2018-01-05T11:29:05,794 DEBUG [IO-Elevator-Thread-7] LlapIoImpl: Processing slice #0 null; has no cache data; has disk data 2018-01-05T11:29:05,794 DEBUG [IO-Elevator-Thread-7] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: int streamBuffers.length: 0 vectors: 1 columnEncoding: null present: false data: null dictionary: false lengths: false secondary: false tz: US/Pacific 2018-01-05T11:29:05,794 DEBUG [IO-Elevator-Thread-7] encoded.EncodedTreeReaderFactory: columnIndex: 2 columnType: string streamBuffers.length: 0 vectors: 1 columnEncoding: null present: false data: null dictionary: false lengths: false secondary: false tz: US/Pacific 2018-01-05T11:29:05,794 DEBUG [IO-Elevator-Thread-7] LlapIoOrc: Done with decode 2018-01-05T11:29:05,794 DEBUG [IO-Elevator-Thread-7] LlapIoImpl: setDone called; closed false, done false, err null, pending 1 2018-01-05T11:29:05,794 DEBUG [IO-Elevator-Thread-7] impl.StatsRecordingThreadPool: Updated stats: instance: SerDeEncodedDataReader thread name: IO-Elevator-Thread-7 thread id: 2200 scheme: file bytesRead: 1382 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:05,795 INFO [TezTR-518813_1_35_0_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:05,795 INFO [TezTR-518813_1_35_0_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, , , ]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=0, NUM_ERRORS=0, ROWS_EMITTED=118, METADATA_CACHE_HIT=0, METADATA_CACHE_MISS=0, CACHE_HIT_BYTES=0, CACHE_MISS_BYTES=1358, ALLOCATED_BYTES=0, ALLOCATED_USED_BYTES=0, TOTAL_IO_TIME_NS=1344427, DECODE_TIME_NS=94730, HDFS_TIME_NS=0, CONSUMER_TIME_NS=1032079 ] 2018-01-05T11:29:05,795 DEBUG [TezTR-518813_1_35_0_0_0] LlapIoImpl: Encoded reader is being stopped 2018-01-05T11:29:05,795 DEBUG [TezTR-518813_1_35_0_0_0] log.PerfLogger: 2018-01-05T11:29:05,795 DEBUG [TezTR-518813_1_35_0_0_0] mr.ObjectCache: djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:29:05,795 DEBUG [TezTR-518813_1_35_0_0_0] vector.VectorMapOperator: close called for operator MAP[0] 2018-01-05T11:29:05,795 INFO [TezTR-518813_1_35_0_0_0] vector.VectorMapOperator: Closing operator MAP[0] 2018-01-05T11:29:05,795 INFO [TezTR-518813_1_35_0_0_0] vector.VectorMapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:500, 2018-01-05T11:29:05,795 DEBUG [TezTR-518813_1_35_0_0_0] vector.VectorMapOperator: Closing child = TS[0] 2018-01-05T11:29:05,795 DEBUG [TezTR-518813_1_35_0_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:29:05,795 INFO [TezTR-518813_1_35_0_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:29:05,795 DEBUG [TezTR-518813_1_35_0_0_0] exec.TableScanOperator: Closing child = SEL[5] 2018-01-05T11:29:05,795 DEBUG [TezTR-518813_1_35_0_0_0] vector.VectorSelectOperator: close called for operator SEL[5] 2018-01-05T11:29:05,795 DEBUG [TezTR-518813_1_35_0_0_0] vector.VectorSelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:05,795 INFO [TezTR-518813_1_35_0_0_0] vector.VectorSelectOperator: Closing operator SEL[5] 2018-01-05T11:29:05,795 DEBUG [TezTR-518813_1_35_0_0_0] vector.VectorSelectOperator: Closing child = RS[6] 2018-01-05T11:29:05,795 DEBUG [TezTR-518813_1_35_0_0_0] reducesink.VectorReduceSinkObjectHashOperator: close called for operator RS[6] 2018-01-05T11:29:05,795 DEBUG [TezTR-518813_1_35_0_0_0] reducesink.VectorReduceSinkObjectHashOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:05,795 INFO [TezTR-518813_1_35_0_0_0] reducesink.VectorReduceSinkObjectHashOperator: Closing operator RS[6] 2018-01-05T11:29:05,795 INFO [TezTR-518813_1_35_0_0_0] reducesink.VectorReduceSinkCommonOperator: RS[6]: records written - 500 2018-01-05T11:29:05,795 INFO [TezTR-518813_1_35_0_0_0] reducesink.VectorReduceSinkObjectHashOperator: RECORDS_OUT_INTERMEDIATE_Map_1:500, 2018-01-05T11:29:05,795 DEBUG [TezTR-518813_1_35_0_0_0] reducesink.VectorReduceSinkObjectHashOperator: 6 Close done 2018-01-05T11:29:05,795 DEBUG [TezTR-518813_1_35_0_0_0] vector.VectorSelectOperator: 5 Close done 2018-01-05T11:29:05,795 DEBUG [TezTR-518813_1_35_0_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:29:05,795 DEBUG [TezTR-518813_1_35_0_0_0] vector.VectorMapOperator: 0 Close done 2018-01-05T11:29:05,795 INFO [TezTR-518813_1_35_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_35_00_000000_0 2018-01-05T11:29:05,795 INFO [TezTR-518813_1_35_0_0_0] impl.PipelinedSorter: Reducer 2: Starting flush of map output 2018-01-05T11:29:05,796 INFO [TezTR-518813_1_35_0_0_0] impl.PipelinedSorter: Reducer 2: Span0.length = 500, perItem = 14 2018-01-05T11:29:05,800 INFO [TezTR-518813_1_35_0_0_0] impl.PipelinedSorter: Reducer 2: done sorting span=0, length=500, time=4 2018-01-05T11:29:05,800 INFO [TezTR-518813_1_35_0_0_0] impl.PipelinedSorter: Reducer 2: Heap = SpanIterator<0:499> (span=Span[5242880,7406]), 2018-01-05T11:29:05,800 INFO [IO-Elevator-Thread-6-OrcEncode] LlapIoImpl: ORC encoder received a exit event 2018-01-05T11:29:05,800 DEBUG [IO-Elevator-Thread-6-OrcEncode] LlapIoImpl: Discarding disk data (if any wasn't cached) 2018-01-05T11:29:05,803 INFO [IO-Elevator-Thread-7-OrcEncode] LlapIoImpl: ORC encoder received a exit event 2018-01-05T11:29:05,803 DEBUG [IO-Elevator-Thread-7-OrcEncode] LlapIoImpl: Discarding disk data (if any wasn't cached) 2018-01-05T11:29:05,817 INFO [TezTR-518813_1_35_0_0_0] impl.PipelinedSorter: Reducer 2: Spilling to /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/35/output/attempt_1515180518813_0001_35_00_000000_0_10216_0/file.out 2018-01-05T11:29:05,833 DEBUG [TezTR-518813_1_35_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:05,833 INFO [TezTR-518813_1_35_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:05,833 INFO [TezTR-518813_1_35_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_35_00_000000_0, askedToStop=false 2018-01-05T11:29:05,833 INFO [TezTR-518813_1_35_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_35_00_000000_0, stopRequested=false 2018-01-05T11:29:05,833 INFO [TezTR-518813_1_35_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_35_00_000000_0: Counters: 27 [[File System Counters FILE_BYTES_READ=5912, FILE_BYTES_WRITTEN=0, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=500, INPUT_RECORDS_PROCESSED=4, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=500, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=7406, OUTPUT_BYTES_WITH_OVERHEAD=7365, OUTPUT_BYTES_PHYSICAL=7381, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500, RECORDS_OUT_INTERMEDIATE_Map_1=500][org.apache.hadoop.hive.llap.counters.LlapIOCounters CACHE_MISS_BYTES=5812, CONSUMER_TIME_NS=5705303, DECODE_TIME_NS=474044, NUM_DECODED_BATCHES=4, NUM_VECTOR_BATCHES=4, ROWS_EMITTED=500, TOTAL_IO_TIME_NS=6947128]] 2018-01-05T11:29:05,833 DEBUG [TezTR-518813_1_35_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_35_0_0_0 thread id: 2188 scheme: file bytesRead: 208 bytesWritten: 7485 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:05,833 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_35_00_000000_0, isTaskDone=true 2018-01-05T11:29:05,833 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000071, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_35_00_000000_0, eventCount=4 } 2018-01-05T11:29:05,834 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_35_00_000000_0, creationTime=1515180545730, allocationTime=1515180545731, startTime=1515180545734, finishTime=1515180545834, timeTaken=100, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:05,834 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:29:05,834 INFO [Task-Executor-1] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_35_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:05,834 INFO [Task-Executor-1] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_35_00_000000_0 2018-01-05T11:29:05,834 INFO [Task-Executor-1] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000071=81 2018-01-05T11:29:05,834 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_35_00_000000_0: true 2018-01-05T11:29:05,834 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_35_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:05,834 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_35_00_000000_0; false 2018-01-05T11:29:05,834 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_35_00_000000_0 2018-01-05T11:29:05,834 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_35_00_000000_0 2018-01-05T11:29:05,834 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000071, QueryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a-35, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:05,834 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_35_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:05,834 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_35_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:29:05,834 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_35_00_000000_0 2018-01-05T11:29:05,834 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:05,834 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:05,835 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_35_00_000000, startTime=1515180545734, finishTime=1515180545834, timeTaken=100, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_35_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=6120, FILE_BYTES_WRITTEN=7485, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=500, INPUT_RECORDS_PROCESSED=4, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=500, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=7406, OUTPUT_BYTES_WITH_OVERHEAD=7365, OUTPUT_BYTES_PHYSICAL=7381, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500, RECORDS_OUT_INTERMEDIATE_Map_1=500, org.apache.hadoop.hive.llap.counters.LlapIOCounters, CACHE_MISS_BYTES=5812, CONSUMER_TIME_NS=5705303, DECODE_TIME_NS=474044, NUM_DECODED_BATCHES=4, NUM_VECTOR_BATCHES=4, ROWS_EMITTED=500, TOTAL_IO_TIME_NS=6947128 2018-01-05T11:29:05,835 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_35_00 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:05,835 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_35_00 [Map 1] 2018-01-05T11:29:05,835 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_35_00, initRequestedTime=1515180545651, initedTime=1515180545730, startRequestedTime=1515180545661, startedTime=1515180545730, finishTime=1515180545835, timeTaken=105, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=6120, FILE_BYTES_WRITTEN=7485, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=500, INPUT_RECORDS_PROCESSED=4, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=500, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=7406, OUTPUT_BYTES_WITH_OVERHEAD=7365, OUTPUT_BYTES_PHYSICAL=7381, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500, RECORDS_OUT_INTERMEDIATE_Map_1=500, org.apache.hadoop.hive.llap.counters.LlapIOCounters, CACHE_MISS_BYTES=5812, CONSUMER_TIME_NS=5705303, DECODE_TIME_NS=474044, NUM_DECODED_BATCHES=4, NUM_VECTOR_BATCHES=4, ROWS_EMITTED=500, TOTAL_IO_TIME_NS=6947128, vertexStats=firstTaskStartTime=1515180545734, firstTasksToStart=[ task_1515180518813_0001_35_00_000000 ], lastTaskFinishTime=1515180545834, lastTasksToFinish=[ task_1515180518813_0001_35_00_000000 ], minTaskDuration=100, maxTaskDuration=100, avgTaskDuration=100.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_35_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_35_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:05,835 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_35_00 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:05,835 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000071, stoppedTime=1515180545835, exitStatus=0 2018-01-05T11:29:05,835 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000071 exited with diagnostics set to Container completed. 2018-01-05T11:29:05,836 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_35_01 [Reducer 2] attempt: attempt_1515180518813_0001_35_00_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:29:05,836 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_35_00 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3 2018-01-05T11:29:05,836 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null 2018-01-05T11:29:05,836 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000071 2018-01-05T11:29:05,836 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: All source tasks completed. Ramping up 4 remaining tasks for vertex: Reducer 2 2018-01-05T11:29:05,836 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Scheduling 4 tasks for vertex: Reducer 2 with totalTasks: 4. 1 source tasks completed out of 1. MinSourceTaskCompletedFraction: 1.0 min: 0.05 max: 0.75 2018-01-05T11:29:05,836 INFO [App Shared Pool - #0] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Map 1, destinationVertexName=Reducer 2. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:29:05,836 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_35_01 [Reducer 2] 2018-01-05T11:29:05,836 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_35_01_000000, scheduledTime=1515180545836, launchTime=1515180545836 2018-01-05T11:29:05,836 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_35_01_000001, scheduledTime=1515180545836, launchTime=1515180545836 2018-01-05T11:29:05,836 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_35_01_000002, scheduledTime=1515180545836, launchTime=1515180545836 2018-01-05T11:29:05,836 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_35_01_000003, scheduledTime=1515180545836, launchTime=1515180545836 2018-01-05T11:29:05,836 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_35_01 [Reducer 2] 2018-01-05T11:29:05,836 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_35_01_000000_0, priority=20, capability=, hosts=null 2018-01-05T11:29:05,836 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=20,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:05,836 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_35_01_000001_0, priority=20, capability=, hosts=null 2018-01-05T11:29:05,836 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=20,c=2]. totalPendingTasks=2. delayedTaskQueueSize=0 2018-01-05T11:29:05,836 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_35_01_000002_0, priority=20, capability=, hosts=null 2018-01-05T11:29:05,836 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=20,c=2]. totalPendingTasks=2. delayedTaskQueueSize=0 2018-01-05T11:29:05,836 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:05,836 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_35_01_000000_0 on hosts=null 2018-01-05T11:29:05,836 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_35_01_000000_0 on hosts=null 2018-01-05T11:29:05,836 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_35_01_000000_0 2018-01-05T11:29:05,836 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:05,836 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #72, task=TaskInfo{task=attempt_1515180518813_0001_35_01_000000_0, priority=20, startTime=0, containerId=null, uniqueId=71, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000072 2018-01-05T11:29:05,836 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_35_01_000000_0; false 2018-01-05T11:29:05,836 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_35_01_000000_0, priority=20, startTime=311566346, containerId=container_111101111_0001_00_000072assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=71, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:05,836 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_35_01_000001_0 on hosts=null 2018-01-05T11:29:05,836 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_35_01_000001_0 on hosts=null 2018-01-05T11:29:05,837 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_35_01_000001_0 2018-01-05T11:29:05,837 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:05,837 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #73, task=TaskInfo{task=attempt_1515180518813_0001_35_01_000001_0, priority=20, startTime=0, containerId=null, uniqueId=72, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, to container=container_111101111_0001_00_000073 2018-01-05T11:29:05,837 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_35_01_000001_0; false 2018-01-05T11:29:05,837 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_35_01_000001_0, priority=20, startTime=311566346, containerId=container_111101111_0001_00_000073assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=2, ac=6, commF=false, disabled=false}, uniqueId=72, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:05,837 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:05,837 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:05,837 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=20,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:05,837 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=20,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:05,837 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_35_01_000003_0, priority=20, capability=, hosts=null 2018-01-05T11:29:05,837 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:05,837 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_35_01_000002_0 on hosts=null 2018-01-05T11:29:05,837 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_35_01_000002_0 on hosts=null 2018-01-05T11:29:05,837 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_35_01_000002_0 2018-01-05T11:29:05,837 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=2, ac=6, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:05,837 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #74, task=TaskInfo{task=attempt_1515180518813_0001_35_01_000002_0, priority=20, startTime=0, containerId=null, uniqueId=73, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=2, ac=6, commF=false, disabled=false}, to container=container_111101111_0001_00_000074 2018-01-05T11:29:05,837 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_35_01_000002_0; false 2018-01-05T11:29:05,837 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_35_01_000002_0, priority=20, startTime=311566346, containerId=container_111101111_0001_00_000074assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=3, ac=5, commF=false, disabled=false}, uniqueId=73, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:05,837 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=20,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:05,837 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=20,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:05,837 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:05,837 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_35_01_000003_0 on hosts=null 2018-01-05T11:29:05,837 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_35_01_000003_0 on hosts=null 2018-01-05T11:29:05,837 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_35_01_000003_0 2018-01-05T11:29:05,837 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=3, ac=5, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:05,837 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000072 for node: 10.22.8.182:49818 2018-01-05T11:29:05,837 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #75, task=TaskInfo{task=attempt_1515180518813_0001_35_01_000003_0, priority=20, startTime=0, containerId=null, uniqueId=74, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=3, ac=5, commF=false, disabled=false}, to container=container_111101111_0001_00_000075 2018-01-05T11:29:05,837 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_35_01_000003_0; false 2018-01-05T11:29:05,837 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000072, attempt_1515180518813_0001_35_01_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:05,837 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000072 for node: 10.22.8.182:49818 2018-01-05T11:29:05,837 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_35_01_000003_0, priority=20, startTime=311566346, containerId=container_111101111_0001_00_000075assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=4, ac=4, commF=false, disabled=false}, uniqueId=74, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:05,838 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000073 for node: 10.22.8.182:49818 2018-01-05T11:29:05,838 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000073, attempt_1515180518813_0001_35_01_000001_0 for node: 10.22.8.182:49818 2018-01-05T11:29:05,838 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=35}, sourceName=Map 1, state=S_SUCCEEDED 2018-01-05T11:29:05,838 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000073 for node: 10.22.8.182:49818 2018-01-05T11:29:05,838 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000074 for node: 10.22.8.182:49818 2018-01-05T11:29:05,838 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000074, attempt_1515180518813_0001_35_01_000002_0 for node: 10.22.8.182:49818 2018-01-05T11:29:05,838 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000074 for node: 10.22.8.182:49818 2018-01-05T11:29:05,838 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000072 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:05,838 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_35_01_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000072, dagName=insert overwrite tabl...cbucket_mapjoin_part (Stage-1), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.OrderedGroupedKVInput,1}}, Outputs={{Reducer 3,org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput,4}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=5, completedTaskCount=1, dagStartTime=1515180545651, firstAttemptStartTime=1515180545836, currentAttemptStartTime=1515180545837} 2018-01-05T11:29:05,838 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000072, launchTime=1515180545838 2018-01-05T11:29:05,838 INFO [IPC Server handler 4 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000072, QueryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a-35, DagId=35, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, SubmitTime=1515180545838 2018-01-05T11:29:05,838 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000075 for node: 10.22.8.182:49818 2018-01-05T11:29:05,838 DEBUG [IPC Server handler 4 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=35} with the ShuffleHandler 2018-01-05T11:29:05,838 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000075, attempt_1515180518813_0001_35_01_000003_0 for node: 10.22.8.182:49818 2018-01-05T11:29:05,838 DEBUG [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/35] 2018-01-05T11:29:05,838 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000075 for node: 10.22.8.182:49818 2018-01-05T11:29:05,839 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:29:05,839 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_35_01_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:05,839 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_35_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180545836, dagStartTime=1515180545651, withinDagPriority=20, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1}] 2018-01-05T11:29:05,839 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_35_01_000000_0 : ACCEPTED 2018-01-05T11:29:05,839 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_35_01_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:29:05,839 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_35_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180545836, dagStartTime=1515180545651, withinDagPriority=20, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1} 2018-01-05T11:29:05,839 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_35_01_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:05,839 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_35_01_000000_0 to pre-emption queue 2018-01-05T11:29:05,839 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_35_01_000000_0: true 2018-01-05T11:29:05,839 INFO [TaskCommunicator # 9] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_35_01_000000_0 2018-01-05T11:29:05,839 INFO [TaskCommunicator # 9] GuaranteedTasks: Received response for attempt_1515180518813_0001_35_01_000000_0, true 2018-01-05T11:29:05,839 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: taskOwner hashCode:1719074333 2018-01-05T11:29:05,839 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_35_01_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000072] on NM: [10.22.8.182:49818] 2018-01-05T11:29:05,839 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:05,839 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:05,839 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_35_01_000000_0, startTime=1515180545839, containerId=container_111101111_0001_00_000072, nodeId=10.22.8.182:49818 2018-01-05T11:29:05,839 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_35_01_000001_0, am_details=10.22.8.182:49824, taskInfo= fragment 1 attempt 0, user=djaiswal, queryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000073, dagName=insert overwrite tabl...cbucket_mapjoin_part (Stage-1), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.OrderedGroupedKVInput,1}}, Outputs={{Reducer 3,org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput,4}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=5, completedTaskCount=1, dagStartTime=1515180545651, firstAttemptStartTime=1515180545836, currentAttemptStartTime=1515180545838} 2018-01-05T11:29:05,839 INFO [Task-Executor-1] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : insert overwrite tabl...cbucket_mapjoin_part (Stage-1), VertexName: Reducer 2, VertexParallelism: 4, TaskAttemptID:attempt_1515180518813_0001_35_01_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=Reducer 3, physicalEdgeCount=4, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ] 2018-01-05T11:29:05,839 INFO [IPC Server handler 3 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000073, QueryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a-35, DagId=35, VertexName=Reducer 2, TaskId=1, TaskAttemptId=0, SubmitTime=1515180545839 2018-01-05T11:29:05,839 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000073 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:05,839 INFO [Task-Executor-1] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1028128768, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:05,839 INFO [Task-Executor-1] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_35_01_000000_0 vertexName: Reducer 2 2018-01-05T11:29:05,839 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000073, launchTime=1515180545839 2018-01-05T11:29:05,839 DEBUG [IPC Server handler 3 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=35} with the ShuffleHandler 2018-01-05T11:29:05,839 DEBUG [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/35] 2018-01-05T11:29:05,840 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000072, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_35_01_000000_0, eventCount=1 } 2018-01-05T11:29:05,840 DEBUG [TezTR-518813_1_35_1_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_35 queryId: djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a instanceType: TaskRunner2Callable 2018-01-05T11:29:05,840 INFO [TezTR-518813_1_35_1_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_35_01_000000_0 2018-01-05T11:29:05,840 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=3, runningFragmentCount=1 2018-01-05T11:29:05,840 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_35_01_000001_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:05,840 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_35_01_000001_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180545836, dagStartTime=1515180545651, withinDagPriority=20, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1}] 2018-01-05T11:29:05,840 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_35_01_000001_0 : ACCEPTED 2018-01-05T11:29:05,840 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_35_01_000001_0, canFinish=true. Current state: preemptionQueueSize=1, numSlotsAvailable=3, waitQueueSize=1 2018-01-05T11:29:05,840 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_35_01_000001_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180545836, dagStartTime=1515180545651, withinDagPriority=20, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1} 2018-01-05T11:29:05,840 DEBUG [TezTR-518813_1_35_1_0_0] log.PerfLogger: 2018-01-05T11:29:05,840 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_35_01_000001_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:05,840 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_35_01_000001_0 to pre-emption queue 2018-01-05T11:29:05,840 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_35_01_000001_0: true 2018-01-05T11:29:05,840 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: taskOwner hashCode:746895039 2018-01-05T11:29:05,840 INFO [TaskCommunicator # 0] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_35_01_000001_0 2018-01-05T11:29:05,840 INFO [TaskCommunicator # 0] GuaranteedTasks: Received response for attempt_1515180518813_0001_35_01_000001_0, true 2018-01-05T11:29:05,840 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_35_01_000001_0] submitted. Is using containerId: [container_111101111_0001_00_000073] on NM: [10.22.8.182:49818] 2018-01-05T11:29:05,840 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:05,840 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:05,840 INFO [I/O Setup 0 Initialize: {Map 1}] orderedgrouped.MergeManager: Initial Memory required for SHUFFLE_BUFFER=131404920 based on INPUT_BUFFER_FACTOR=0.7, for final merged output=0, using factor: 0.0 2018-01-05T11:29:05,840 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:05,840 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_35_01_000001_0, startTime=1515180545840, containerId=container_111101111_0001_00_000073, nodeId=10.22.8.182:49818 2018-01-05T11:29:05,841 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000074 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:05,842 INFO [Task-Executor-3] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : insert overwrite tabl...cbucket_mapjoin_part (Stage-1), VertexName: Reducer 2, VertexParallelism: 4, TaskAttemptID:attempt_1515180518813_0001_35_01_000001_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=Reducer 3, physicalEdgeCount=4, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ] 2018-01-05T11:29:05,842 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000074, launchTime=1515180545842 2018-01-05T11:29:05,842 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_35_01_000002_0, am_details=10.22.8.182:49824, taskInfo= fragment 2 attempt 0, user=djaiswal, queryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000074, dagName=insert overwrite tabl...cbucket_mapjoin_part (Stage-1), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.OrderedGroupedKVInput,1}}, Outputs={{Reducer 3,org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput,4}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=5, completedTaskCount=1, dagStartTime=1515180545651, firstAttemptStartTime=1515180545836, currentAttemptStartTime=1515180545838} 2018-01-05T11:29:05,842 INFO [Task-Executor-3] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1028128768, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:05,842 INFO [IPC Server handler 1 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000074, QueryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a-35, DagId=35, VertexName=Reducer 2, TaskId=2, TaskAttemptId=0, SubmitTime=1515180545842 2018-01-05T11:29:05,842 INFO [Task-Executor-3] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_35_01_000001_0 vertexName: Reducer 2 2018-01-05T11:29:05,842 DEBUG [IPC Server handler 1 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=35} with the ShuffleHandler 2018-01-05T11:29:05,842 DEBUG [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/35] 2018-01-05T11:29:05,842 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000073, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_35_01_000001_0, eventCount=1 } 2018-01-05T11:29:05,842 DEBUG [TezTR-518813_1_35_1_1_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_35 queryId: djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a instanceType: TaskRunner2Callable 2018-01-05T11:29:05,842 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:05,842 INFO [TezTR-518813_1_35_1_1_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_35_01_000001_0 2018-01-05T11:29:05,842 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=2, runningFragmentCount=2 2018-01-05T11:29:05,842 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_35_01_000002_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:05,842 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_35_01_000002_0] submitted. Is using containerId: [container_111101111_0001_00_000074] on NM: [10.22.8.182:49818] 2018-01-05T11:29:05,842 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_35_01_000002_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180545836, dagStartTime=1515180545651, withinDagPriority=20, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1}] 2018-01-05T11:29:05,842 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_35_01_000002_0 : ACCEPTED 2018-01-05T11:29:05,842 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_35_01_000002_0, canFinish=true. Current state: preemptionQueueSize=2, numSlotsAvailable=2, waitQueueSize=1 2018-01-05T11:29:05,842 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_35_01_000002_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180545836, dagStartTime=1515180545651, withinDagPriority=20, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1} 2018-01-05T11:29:05,842 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_35_01_000002_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:05,842 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_35_01_000002_0 to pre-emption queue 2018-01-05T11:29:05,843 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_35_01_000002_0: true 2018-01-05T11:29:05,843 DEBUG [TezTR-518813_1_35_1_1_0] log.PerfLogger: 2018-01-05T11:29:05,843 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:147702357 2018-01-05T11:29:05,843 INFO [TaskCommunicator # 1] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_35_01_000002_0 2018-01-05T11:29:05,843 INFO [TaskCommunicator # 1] GuaranteedTasks: Received response for attempt_1515180518813_0001_35_01_000002_0, true 2018-01-05T11:29:05,843 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:05,843 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:05,844 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:05,844 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : insert overwrite tabl...cbucket_mapjoin_part (Stage-1), VertexName: Reducer 2, VertexParallelism: 4, TaskAttemptID:attempt_1515180518813_0001_35_01_000002_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=Reducer 3, physicalEdgeCount=4, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ] 2018-01-05T11:29:05,844 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1028128768, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:05,844 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_35_01_000002_0 vertexName: Reducer 2 2018-01-05T11:29:05,844 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_35_01_000003_0, am_details=10.22.8.182:49824, taskInfo= fragment 3 attempt 0, user=djaiswal, queryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000075, dagName=insert overwrite tabl...cbucket_mapjoin_part (Stage-1), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.OrderedGroupedKVInput,1}}, Outputs={{Reducer 3,org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput,4}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=5, completedTaskCount=1, dagStartTime=1515180545651, firstAttemptStartTime=1515180545836, currentAttemptStartTime=1515180545838} 2018-01-05T11:29:05,844 INFO [IPC Server handler 0 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000075, QueryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a-35, DagId=35, VertexName=Reducer 2, TaskId=3, TaskAttemptId=0, SubmitTime=1515180545844 2018-01-05T11:29:05,844 INFO [I/O Setup 0 Initialize: {Map 1}] orderedgrouped.MergeManager: Initial Memory required for SHUFFLE_BUFFER=131404920 based on INPUT_BUFFER_FACTOR=0.7, for final merged output=0, using factor: 0.0 2018-01-05T11:29:05,844 DEBUG [IPC Server handler 0 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=35} with the ShuffleHandler 2018-01-05T11:29:05,844 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:05,844 DEBUG [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/35] 2018-01-05T11:29:05,844 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000074, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_35_01_000002_0, eventCount=1 } 2018-01-05T11:29:05,844 DEBUG [TezTR-518813_1_35_1_2_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_35 queryId: djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a instanceType: TaskRunner2Callable 2018-01-05T11:29:05,844 INFO [IPC Server handler 24 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_35_01_000000_0 1 events [0,1) total 1 vertex_1515180518813_0001_35_01 [Reducer 2] 2018-01-05T11:29:05,844 INFO [TezTR-518813_1_35_1_2_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_35_01_000002_0 2018-01-05T11:29:05,845 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=1, runningFragmentCount=3 2018-01-05T11:29:05,845 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_35_01_000003_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:05,845 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_35_01_000003_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180545836, dagStartTime=1515180545651, withinDagPriority=20, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1}] 2018-01-05T11:29:05,845 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:29:05,845 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_35_01_000003_0, canFinish=true. Current state: preemptionQueueSize=3, numSlotsAvailable=1, waitQueueSize=1 2018-01-05T11:29:05,845 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_35_01_000003_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180545836, dagStartTime=1515180545651, withinDagPriority=20, vertexParallelism= 4, selfAndUpstreamParallelism= 5, selfAndUpstreamComplete= 1} 2018-01-05T11:29:05,845 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_35_01_000003_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:05,845 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_35_01_000003_0 to pre-emption queue 2018-01-05T11:29:05,845 DEBUG [TezTR-518813_1_35_1_2_0] log.PerfLogger: 2018-01-05T11:29:05,845 INFO [IPC Server handler 16 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_35_01_000001_0 1 events [0,1) total 1 vertex_1515180518813_0001_35_01 [Reducer 2] 2018-01-05T11:29:05,845 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_35_01_000003_0: true 2018-01-05T11:29:05,845 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: taskOwner hashCode:1205153640 2018-01-05T11:29:05,845 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_35_01_000003_0 : ACCEPTED 2018-01-05T11:29:05,845 INFO [I/O Setup 0 Initialize: {Map 1}] orderedgrouped.MergeManager: Initial Memory required for SHUFFLE_BUFFER=131404920 based on INPUT_BUFFER_FACTOR=0.7, for final merged output=0, using factor: 0.0 2018-01-05T11:29:05,845 DEBUG [TezTR-518813_1_35_1_0_0] log.PerfLogger: 2018-01-05T11:29:05,845 INFO [TezTR-518813_1_35_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:05,845 INFO [TezTR-518813_1_35_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:05,845 INFO [TezTR-518813_1_35_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:05,845 INFO [TezTR-518813_1_35_1_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:05,846 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:05,846 INFO [TezTR-518813_1_35_1_0_0] resources.MemoryDistributor: InitialRequests=[Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [Reducer 3:OUTPUT:10485760:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput] 2018-01-05T11:29:05,845 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_35_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:29:05,846 INFO [TezTR-518813_1_35_1_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:05,846 INFO [TezTR-518813_1_35_1_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:05,846 INFO [TaskCommunicator # 3] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_35_01_000003_0 2018-01-05T11:29:05,846 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:29:05,846 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_35_01_000001_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:29:05,846 INFO [TaskCommunicator # 3] GuaranteedTasks: Received response for attempt_1515180518813_0001_35_01_000003_0, true 2018-01-05T11:29:05,846 INFO [TezTR-518813_1_35_1_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 24, TotalRequested: 141890680, TotalRequestedScaled: 7.094534E7, TotalJVMHeap: 1028128768, TotalAvailable: 688846274, TotalRequested/TotalJVMHeap:0.14 2018-01-05T11:29:05,846 INFO [Task-Executor-2] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : insert overwrite tabl...cbucket_mapjoin_part (Stage-1), VertexName: Reducer 2, VertexParallelism: 4, TaskAttemptID:attempt_1515180518813_0001_35_01_000003_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=Reducer 3, physicalEdgeCount=4, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ] 2018-01-05T11:29:05,846 INFO [TezTR-518813_1_35_1_0_0] resources.MemoryDistributor: Allocations=[Map 1:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:131404920:131404920], [Reducer 3:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:10485760:10485760] 2018-01-05T11:29:05,846 INFO [TezTR-518813_1_35_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:05,846 INFO [Task-Executor-2] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1028128768, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:05,846 INFO [TezTR-518813_1_35_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: Map 1 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:05,846 INFO [Task-Executor-2] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_35_01_000003_0 vertexName: Reducer 2 2018-01-05T11:29:05,846 INFO [TezTR-518813_1_35_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:29:05,846 INFO [TezTR-518813_1_35_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:05,846 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_35_01_000002_0, startTime=1515180545842, containerId=container_111101111_0001_00_000074, nodeId=10.22.8.182:49818 2018-01-05T11:29:05,847 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000075 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:05,847 DEBUG [I/O Setup 0 Start: {Map 1}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:05,847 DEBUG [I/O Setup 0 Start: {Map 1}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:05,847 DEBUG [I/O Setup 0 Start: {Map 1}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:05,847 DEBUG [I/O Setup 0 Start: {Map 1}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:05,847 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000075, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_35_01_000003_0, eventCount=1 } 2018-01-05T11:29:05,847 DEBUG [TezTR-518813_1_35_1_3_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_35 queryId: djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a instanceType: TaskRunner2Callable 2018-01-05T11:29:05,847 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:05,847 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.Shuffle: Map_1: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true 2018-01-05T11:29:05,847 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000075, launchTime=1515180545847 2018-01-05T11:29:05,847 INFO [TezTR-518813_1_35_1_3_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_35_01_000003_0 2018-01-05T11:29:05,847 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.MergeManager: Map 1: MergerManager: memoryLimit=131404920, maxSingleShuffleLimit=32851230, mergeThreshold=86727248, ioSortFactor=10, postMergeMem=0, memToMemMergeOutputsThreshold=10 2018-01-05T11:29:05,847 DEBUG [TezTR-518813_1_35_1_3_0] log.PerfLogger: 2018-01-05T11:29:05,847 INFO [I/O Setup 0 Initialize: {Map 1}] orderedgrouped.MergeManager: Initial Memory required for SHUFFLE_BUFFER=131404920 based on INPUT_BUFFER_FACTOR=0.7, for final merged output=0, using factor: 0.0 2018-01-05T11:29:05,848 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:05,848 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.ShuffleScheduler: ShuffleScheduler running for sourceVertex: Map 1 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true 2018-01-05T11:29:05,848 DEBUG [TezTR-518813_1_35_1_1_0] log.PerfLogger: 2018-01-05T11:29:05,848 INFO [TezTR-518813_1_35_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:05,848 INFO [TezTR-518813_1_35_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:05,848 INFO [TezTR-518813_1_35_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:05,848 INFO [TezTR-518813_1_35_1_1_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:05,848 INFO [TezTR-518813_1_35_1_1_0] resources.MemoryDistributor: InitialRequests=[Reducer 3:OUTPUT:10485760:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput], [Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.OrderedGroupedKVInput] 2018-01-05T11:29:05,848 INFO [TezTR-518813_1_35_1_1_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:05,848 INFO [TezTR-518813_1_35_1_1_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:05,848 INFO [IPC Server handler 8 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_35_01_000002_0 1 events [0,1) total 1 vertex_1515180518813_0001_35_01 [Reducer 2] 2018-01-05T11:29:05,848 INFO [TezTR-518813_1_35_1_1_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 24, TotalRequested: 141890680, TotalRequestedScaled: 7.094534E7, TotalJVMHeap: 1028128768, TotalAvailable: 688846274, TotalRequested/TotalJVMHeap:0.14 2018-01-05T11:29:05,848 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:05,848 INFO [TezTR-518813_1_35_1_1_0] resources.MemoryDistributor: Allocations=[Reducer 3:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:10485760:10485760], [Map 1:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:131404920:131404920] 2018-01-05T11:29:05,848 INFO [I/O Setup 0 Start: {Map 1}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: Map 1 2018-01-05T11:29:05,849 INFO [TezTR-518813_1_35_1_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:05,849 INFO [TezTR-518813_1_35_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:05,849 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:29:05,849 INFO [TezTR-518813_1_35_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Input: Map 1 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:05,849 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_35_01_000003_0] submitted. Is using containerId: [container_111101111_0001_00_000075] on NM: [10.22.8.182:49818] 2018-01-05T11:29:05,849 INFO [TezTR-518813_1_35_1_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_35_01_000000_0 2018-01-05T11:29:05,849 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_35_01_000002_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:29:05,849 DEBUG [TezTR-518813_1_35_1_0_0] log.PerfLogger: 2018-01-05T11:29:05,850 DEBUG [TezTR-518813_1_35_1_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_35_01_000000_0_10217 2018-01-05T11:29:05,850 DEBUG [TezTR-518813_1_35_1_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:05,850 DEBUG [TezTR-518813_1_35_1_0_0] mr.ObjectCache: Creating djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:29:05,850 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:05,850 DEBUG [I/O Setup 1 Start: {Map 1}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:05,850 INFO [TezTaskEventRouter{attempt_1515180518813_0001_35_01_000000_0}] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:29:05,850 INFO [TezTR-518813_1_35_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/5ce0a5f6-4beb-4150-a5be-c4e3c7b03367/reduce.xml 2018-01-05T11:29:05,849 INFO [TezTR-518813_1_35_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:29:05,850 DEBUG [I/O Setup 1 Start: {Map 1}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:05,850 INFO [TezTR-518813_1_35_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:05,850 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:05,850 DEBUG [I/O Setup 1 Start: {Map 1}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:05,850 DEBUG [TezTR-518813_1_35_1_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/5ce0a5f6-4beb-4150-a5be-c4e3c7b03367/reduce.xml 2018-01-05T11:29:05,850 DEBUG [I/O Setup 1 Start: {Map 1}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:05,850 DEBUG [TezTR-518813_1_35_1_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/5ce0a5f6-4beb-4150-a5be-c4e3c7b03367/reduce.xml 2018-01-05T11:29:05,850 DEBUG [TezTR-518813_1_35_1_0_0] log.PerfLogger: 2018-01-05T11:29:05,850 INFO [I/O Setup 1 Start: {Map 1}] orderedgrouped.Shuffle: Map_1: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true 2018-01-05T11:29:05,850 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_35_01_000003_0, startTime=1515180545847, containerId=container_111101111_0001_00_000075, nodeId=10.22.8.182:49818 2018-01-05T11:29:05,850 INFO [TezTR-518813_1_35_1_0_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:29:05,850 INFO [I/O Setup 1 Start: {Map 1}] orderedgrouped.MergeManager: Map 1: MergerManager: memoryLimit=131404920, maxSingleShuffleLimit=32851230, mergeThreshold=86727248, ioSortFactor=10, postMergeMem=0, memToMemMergeOutputsThreshold=10 2018-01-05T11:29:05,850 INFO [IPC Server handler 21 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_35_01_000003_0 1 events [0,1) total 1 vertex_1515180518813_0001_35_01 [Reducer 2] 2018-01-05T11:29:05,850 DEBUG [TezTR-518813_1_35_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:05,850 DEBUG [TezTR-518813_1_35_1_2_0] log.PerfLogger: 2018-01-05T11:29:05,850 INFO [TezTR-518813_1_35_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:05,851 INFO [TezTR-518813_1_35_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:05,851 INFO [TezTR-518813_1_35_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:05,852 INFO [TezTR-518813_1_35_1_2_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:05,852 INFO [TezTR-518813_1_35_1_2_0] resources.MemoryDistributor: InitialRequests=[Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [Reducer 3:OUTPUT:10485760:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput] 2018-01-05T11:29:05,853 INFO [TezTR-518813_1_35_1_2_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:05,853 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:29:05,853 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_35_01_000003_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:29:05,853 INFO [TezTR-518813_1_35_1_2_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:05,853 INFO [I/O Setup 1 Start: {Map 1}] orderedgrouped.ShuffleScheduler: ShuffleScheduler running for sourceVertex: Map 1 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true 2018-01-05T11:29:05,853 INFO [TezTR-518813_1_35_1_2_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 24, TotalRequested: 141890680, TotalRequestedScaled: 7.094534E7, TotalJVMHeap: 1028128768, TotalAvailable: 688846274, TotalRequested/TotalJVMHeap:0.14 2018-01-05T11:29:05,853 INFO [TezTR-518813_1_35_1_2_0] resources.MemoryDistributor: Allocations=[Map 1:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:131404920:131404920], [Reducer 3:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:10485760:10485760] 2018-01-05T11:29:05,853 INFO [TezTR-518813_1_35_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:05,854 INFO [TezTR-518813_1_35_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Input: Map 1 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:05,854 INFO [I/O Setup 1 Start: {Map 1}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: Map 1 2018-01-05T11:29:05,855 INFO [TezTR-518813_1_35_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:29:05,855 INFO [TezTR-518813_1_35_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:05,855 DEBUG [I/O Setup 0 Start: {Map 1}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:05,855 INFO [TezTR-518813_1_35_1_1_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:05,855 DEBUG [I/O Setup 0 Start: {Map 1}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:05,855 DEBUG [I/O Setup 0 Start: {Map 1}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:05,855 DEBUG [I/O Setup 0 Start: {Map 1}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:05,855 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=35&reduce=0&map=attempt_1515180518813_0001_35_00_000000_0_10216 mapId: [attempt_1515180518813_0001_35_00_000000_0_10216] reduceId: [0] jobId: [job_1515180518813_0001] dagId: [35] keepAlive: false 2018-01-05T11:29:05,855 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=35&reduce=0&map=attempt_1515180518813_0001_35_00_000000_0_10216; hash=...MMJ1PpEYrUgJU 2018-01-05T11:29:05,855 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.Shuffle: Map_1: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true 2018-01-05T11:29:05,855 INFO [TezTR-518813_1_35_1_1_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_35_01_000001_0 2018-01-05T11:29:05,855 DEBUG [TezTR-518813_1_35_1_1_0] log.PerfLogger: 2018-01-05T11:29:05,855 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.MergeManager: Map 1: MergerManager: memoryLimit=131404920, maxSingleShuffleLimit=32851230, mergeThreshold=86727248, ioSortFactor=10, postMergeMem=0, memToMemMergeOutputsThreshold=10 2018-01-05T11:29:05,855 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=35&reduce=0&map=attempt_1515180518813_0001_35_00_000000_0_10216;reply=QXLkAs2bKfNs8 2018-01-05T11:29:05,855 INFO [TezTaskEventRouter{attempt_1515180518813_0001_35_01_000001_0}] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:29:05,855 DEBUG [TezTR-518813_1_35_1_1_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_35_01_000001_0_10220 2018-01-05T11:29:05,855 DEBUG [TezTR-518813_1_35_1_1_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:05,855 DEBUG [TezTR-518813_1_35_1_1_0] mr.ObjectCache: Creating djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:29:05,855 INFO [TezTR-518813_1_35_1_1_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/5ce0a5f6-4beb-4150-a5be-c4e3c7b03367/reduce.xml 2018-01-05T11:29:05,855 DEBUG [TezTR-518813_1_35_1_1_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/5ce0a5f6-4beb-4150-a5be-c4e3c7b03367/reduce.xml 2018-01-05T11:29:05,855 DEBUG [TezTR-518813_1_35_1_1_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/5ce0a5f6-4beb-4150-a5be-c4e3c7b03367/reduce.xml 2018-01-05T11:29:05,856 DEBUG [TezTR-518813_1_35_1_1_0] log.PerfLogger: 2018-01-05T11:29:05,856 INFO [TezTR-518813_1_35_1_1_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:29:05,856 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.ShuffleScheduler: ShuffleScheduler running for sourceVertex: Map 1 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true 2018-01-05T11:29:05,856 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=35, user='djaiswal', attemptId='attempt_1515180518813_0001_35_00_000000_0_10216'} via loader 2018-01-05T11:29:05,856 DEBUG [TezTR-518813_1_35_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:05,856 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=35, user='djaiswal', attemptId='attempt_1515180518813_0001_35_00_000000_0_10216'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:29:05,856 DEBUG [New I/O worker #3] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_35_00_000000_0_10216 not found 2018-01-05T11:29:05,856 INFO [I/O Setup 0 Start: {Map 1}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: Map 1 2018-01-05T11:29:05,856 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_35_00_000000_0_10216,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/35/output/attempt_1515180518813_0001_35_00_000000_0_10216/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/35/output/attempt_1515180518813_0001_35_00_000000_0_10216/file.out.index 2018-01-05T11:29:05,856 INFO [TezTR-518813_1_35_1_2_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:05,856 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Content Length in shuffle : 1817 2018-01-05T11:29:05,857 INFO [TezTR-518813_1_35_1_2_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_35_01_000002_0 2018-01-05T11:29:05,857 DEBUG [TezTR-518813_1_35_1_2_0] log.PerfLogger: 2018-01-05T11:29:05,857 DEBUG [TezTR-518813_1_35_1_2_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_35_01_000002_0_10223 2018-01-05T11:29:05,857 DEBUG [TezTR-518813_1_35_1_2_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:05,857 DEBUG [TezTR-518813_1_35_1_2_0] mr.ObjectCache: Creating djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:29:05,857 INFO [TezTR-518813_1_35_1_2_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/5ce0a5f6-4beb-4150-a5be-c4e3c7b03367/reduce.xml 2018-01-05T11:29:05,857 DEBUG [TezTR-518813_1_35_1_2_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/5ce0a5f6-4beb-4150-a5be-c4e3c7b03367/reduce.xml 2018-01-05T11:29:05,857 DEBUG [TezTR-518813_1_35_1_2_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/5ce0a5f6-4beb-4150-a5be-c4e3c7b03367/reduce.xml 2018-01-05T11:29:05,857 DEBUG [TezTR-518813_1_35_1_2_0] log.PerfLogger: 2018-01-05T11:29:05,857 INFO [TezTR-518813_1_35_1_2_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:29:05,857 DEBUG [TezTR-518813_1_35_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:05,857 INFO [Fetcher_O {Map_1} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=35&reduce=0&map=attempt_1515180518813_0001_35_00_000000_0_10216 sent hash and receievd reply 0 ms 2018-01-05T11:29:05,857 DEBUG [TezTR-518813_1_35_1_3_0] log.PerfLogger: 2018-01-05T11:29:05,857 INFO [TezTR-518813_1_35_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:05,857 INFO [TezTR-518813_1_35_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:05,857 INFO [TezTaskEventRouter{attempt_1515180518813_0001_35_01_000002_0}] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:29:05,857 INFO [TezTR-518813_1_35_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:05,858 INFO [TezTR-518813_1_35_1_3_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:05,858 INFO [Fetcher_O {Map_1} #0] orderedgrouped.MergeManager: CloseInMemoryFile. Current state: inMemoryMapOutputs.size=1, commitMemory=0, usedMemory=1758. Since last log: count=1, min=1758, max=1758, total=1758, avg=1758.0 2018-01-05T11:29:05,858 INFO [TezTR-518813_1_35_1_3_0] resources.MemoryDistributor: InitialRequests=[Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [Reducer 3:OUTPUT:10485760:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput] 2018-01-05T11:29:05,858 INFO [Fetcher_O {Map_1} #0] ShuffleScheduler.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_35_00_000000_0_10216} to MEMORY, csize=1762, dsize=1758, EndTime=1515180545858, TimeTaken=1, Rate=1.68 MB/s 2018-01-05T11:29:05,858 INFO [Fetcher_O {Map_1} #0] orderedgrouped.ShuffleScheduler: All inputs fetched for input vertex : Map 1 2018-01-05T11:29:05,858 INFO [TezTR-518813_1_35_1_3_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:05,858 INFO [TezTR-518813_1_35_1_3_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:05,858 INFO [Fetcher_O {Map_1} #0] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:05,858 INFO [TezTR-518813_1_35_1_3_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 24, TotalRequested: 141890680, TotalRequestedScaled: 7.094534E7, TotalJVMHeap: 1028128768, TotalAvailable: 688846274, TotalRequested/TotalJVMHeap:0.14 2018-01-05T11:29:05,858 INFO [TezTR-518813_1_35_1_3_0] resources.MemoryDistributor: Allocations=[Map 1:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:131404920:131404920], [Reducer 3:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:10485760:10485760] 2018-01-05T11:29:05,858 INFO [TezTR-518813_1_35_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:05,858 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down FetchScheduler for input: Map_1, wasInterrupted=false 2018-01-05T11:29:05,858 INFO [TezTR-518813_1_35_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Input: Map 1 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:05,858 INFO [TezTR-518813_1_35_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:29:05,858 INFO [TezTR-518813_1_35_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:05,859 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:05,860 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=35&reduce=2&map=attempt_1515180518813_0001_35_00_000000_0_10216 mapId: [attempt_1515180518813_0001_35_00_000000_0_10216] reduceId: [2] jobId: [job_1515180518813_0001] dagId: [35] keepAlive: false 2018-01-05T11:29:05,860 DEBUG [I/O Setup 0 Start: {Map 1}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:05,860 DEBUG [I/O Setup 0 Start: {Map 1}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:05,860 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=35&reduce=2&map=attempt_1515180518813_0001_35_00_000000_0_10216; hash=...WhstlJtlQILvs 2018-01-05T11:29:05,860 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down fetchers for input: Map_1, shutdown timetaken: 0 ms, hasFetcherExecutorStopped: true 2018-01-05T11:29:05,860 DEBUG [I/O Setup 0 Start: {Map 1}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:05,860 DEBUG [I/O Setup 0 Start: {Map 1}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:05,860 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: TotalInMemFetchStats: count=1, totalSize=1758, min=1758, max=1758, avg=1.0 2018-01-05T11:29:05,860 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: finalMerge with #inMemoryOutputs=1, size=1758 and #onDiskOutputs=0, size=0 2018-01-05T11:29:05,860 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.Shuffle: Map_1: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true 2018-01-05T11:29:05,860 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=35&reduce=2&map=attempt_1515180518813_0001_35_00_000000_0_10216;reply=/CwCOMGqn9x2Y 2018-01-05T11:29:05,860 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=35, user='djaiswal', attemptId='attempt_1515180518813_0001_35_00_000000_0_10216'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:29:05,860 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.MergeManager: Map 1: MergerManager: memoryLimit=131404920, maxSingleShuffleLimit=32851230, mergeThreshold=86727248, ioSortFactor=10, postMergeMem=0, memToMemMergeOutputsThreshold=10 2018-01-05T11:29:05,860 DEBUG [New I/O worker #3] shufflehandler.IndexCache: IndexCache HIT: MapId attempt_1515180518813_0001_35_00_000000_0_10216 found 2018-01-05T11:29:05,860 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_35_00_000000_0_10216,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/35/output/attempt_1515180518813_0001_35_00_000000_0_10216/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/35/output/attempt_1515180518813_0001_35_00_000000_0_10216/file.out.index 2018-01-05T11:29:05,860 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Content Length in shuffle : 1942 2018-01-05T11:29:05,861 INFO [Fetcher_O {Map_1} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=35&reduce=2&map=attempt_1515180518813_0001_35_00_000000_0_10216 sent hash and receievd reply 0 ms 2018-01-05T11:29:05,861 DEBUG [New I/O worker #3] shufflehandler.FadvisedFileRegion: shuffleBufferSize: 131072, path: /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/35/output/attempt_1515180518813_0001_35_00_000000_0_10216/file.out 2018-01-05T11:29:05,861 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=35&reduce=1&map=attempt_1515180518813_0001_35_00_000000_0_10216 mapId: [attempt_1515180518813_0001_35_00_000000_0_10216] reduceId: [1] jobId: [job_1515180518813_0001] dagId: [35] keepAlive: false 2018-01-05T11:29:05,861 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=35&reduce=1&map=attempt_1515180518813_0001_35_00_000000_0_10216; hash=...D3df3nZ1Oo/Fc 2018-01-05T11:29:05,861 INFO [Fetcher_O {Map_1} #0] orderedgrouped.MergeManager: CloseInMemoryFile. Current state: inMemoryMapOutputs.size=1, commitMemory=0, usedMemory=1883. Since last log: count=1, min=1883, max=1883, total=1883, avg=1883.0 2018-01-05T11:29:05,861 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.ShuffleScheduler: ShuffleScheduler running for sourceVertex: Map 1 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true 2018-01-05T11:29:05,861 INFO [Fetcher_O {Map_1} #0] ShuffleScheduler.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_35_00_000000_0_10216} to MEMORY, csize=1887, dsize=1883, EndTime=1515180545861, TimeTaken=0, Rate=0.00 MB/s 2018-01-05T11:29:05,861 INFO [Fetcher_O {Map_1} #0] orderedgrouped.ShuffleScheduler: All inputs fetched for input vertex : Map 1 2018-01-05T11:29:05,861 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=35&reduce=1&map=attempt_1515180518813_0001_35_00_000000_0_10216;reply=5IykrSREI09/I 2018-01-05T11:29:05,861 INFO [Fetcher_O {Map_1} #0] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:05,861 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=35, user='djaiswal', attemptId='attempt_1515180518813_0001_35_00_000000_0_10216'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:29:05,861 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache HIT: MapId attempt_1515180518813_0001_35_00_000000_0_10216 found 2018-01-05T11:29:05,861 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down FetchScheduler for input: Map_1, wasInterrupted=false 2018-01-05T11:29:05,861 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_35_00_000000_0_10216,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/35/output/attempt_1515180518813_0001_35_00_000000_0_10216/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/35/output/attempt_1515180518813_0001_35_00_000000_0_10216/file.out.index 2018-01-05T11:29:05,861 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Content Length in shuffle : 2069 2018-01-05T11:29:05,862 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:05,862 INFO [I/O Setup 0 Start: {Map 1}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: Map 1 2018-01-05T11:29:05,862 INFO [TezTR-518813_1_35_1_3_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:05,863 INFO [TezTR-518813_1_35_1_3_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_35_01_000003_0 2018-01-05T11:29:05,863 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down fetchers for input: Map_1, shutdown timetaken: 0 ms, hasFetcherExecutorStopped: true 2018-01-05T11:29:05,863 DEBUG [TezTR-518813_1_35_1_3_0] log.PerfLogger: 2018-01-05T11:29:05,863 DEBUG [TezTR-518813_1_35_1_3_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_35_01_000003_0_10226 2018-01-05T11:29:05,863 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: TotalInMemFetchStats: count=1, totalSize=1883, min=1883, max=1883, avg=1.0 2018-01-05T11:29:05,863 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: finalMerge with #inMemoryOutputs=1, size=1883 and #onDiskOutputs=0, size=0 2018-01-05T11:29:05,863 INFO [Fetcher_O {Map_1} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=35&reduce=1&map=attempt_1515180518813_0001_35_00_000000_0_10216 sent hash and receievd reply 0 ms 2018-01-05T11:29:05,863 DEBUG [New I/O worker #1] shufflehandler.FadvisedFileRegion: shuffleBufferSize: 131072, path: /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/35/output/attempt_1515180518813_0001_35_00_000000_0_10216/file.out 2018-01-05T11:29:05,863 DEBUG [TezTR-518813_1_35_1_3_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:05,863 DEBUG [TezTR-518813_1_35_1_3_0] mr.ObjectCache: Creating djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:29:05,863 INFO [TezTR-518813_1_35_1_3_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/5ce0a5f6-4beb-4150-a5be-c4e3c7b03367/reduce.xml 2018-01-05T11:29:05,863 INFO [TezTaskEventRouter{attempt_1515180518813_0001_35_01_000003_0}] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:29:05,863 INFO [Fetcher_O {Map_1} #0] orderedgrouped.MergeManager: CloseInMemoryFile. Current state: inMemoryMapOutputs.size=1, commitMemory=0, usedMemory=2010. Since last log: count=1, min=2010, max=2010, total=2010, avg=2010.0 2018-01-05T11:29:05,863 DEBUG [TezTR-518813_1_35_1_3_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/5ce0a5f6-4beb-4150-a5be-c4e3c7b03367/reduce.xml 2018-01-05T11:29:05,863 INFO [Fetcher_O {Map_1} #0] ShuffleScheduler.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_35_00_000000_0_10216} to MEMORY, csize=2014, dsize=2010, EndTime=1515180545863, TimeTaken=0, Rate=0.00 MB/s 2018-01-05T11:29:05,863 DEBUG [TezTR-518813_1_35_1_3_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/5ce0a5f6-4beb-4150-a5be-c4e3c7b03367/reduce.xml 2018-01-05T11:29:05,863 INFO [Fetcher_O {Map_1} #0] orderedgrouped.ShuffleScheduler: All inputs fetched for input vertex : Map 1 2018-01-05T11:29:05,863 DEBUG [TezTR-518813_1_35_1_3_0] log.PerfLogger: 2018-01-05T11:29:05,863 INFO [TezTR-518813_1_35_1_3_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:29:05,863 INFO [Fetcher_O {Map_1} #0] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:05,863 DEBUG [TezTR-518813_1_35_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:05,864 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down FetchScheduler for input: Map_1, wasInterrupted=false 2018-01-05T11:29:05,864 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:05,864 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down fetchers for input: Map_1, shutdown timetaken: 0 ms, hasFetcherExecutorStopped: true 2018-01-05T11:29:05,864 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: TotalInMemFetchStats: count=1, totalSize=2010, min=2010, max=2010, avg=1.0 2018-01-05T11:29:05,864 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: finalMerge with #inMemoryOutputs=1, size=2010 and #onDiskOutputs=0, size=0 2018-01-05T11:29:05,864 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=35&reduce=3&map=attempt_1515180518813_0001_35_00_000000_0_10216 mapId: [attempt_1515180518813_0001_35_00_000000_0_10216] reduceId: [3] jobId: [job_1515180518813_0001] dagId: [35] keepAlive: false 2018-01-05T11:29:05,864 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=35&reduce=3&map=attempt_1515180518813_0001_35_00_000000_0_10216; hash=...2Of8e1R5Zp2tw 2018-01-05T11:29:05,864 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=35&reduce=3&map=attempt_1515180518813_0001_35_00_000000_0_10216;reply=rCVC2+mMt0rjM 2018-01-05T11:29:05,864 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=35, user='djaiswal', attemptId='attempt_1515180518813_0001_35_00_000000_0_10216'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:29:05,865 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache HIT: MapId attempt_1515180518813_0001_35_00_000000_0_10216 found 2018-01-05T11:29:05,865 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_35_00_000000_0_10216,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/35/output/attempt_1515180518813_0001_35_00_000000_0_10216/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/35/output/attempt_1515180518813_0001_35_00_000000_0_10216/file.out.index 2018-01-05T11:29:05,865 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Content Length in shuffle : 1773 2018-01-05T11:29:05,865 INFO [Fetcher_O {Map_1} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=35&reduce=3&map=attempt_1515180518813_0001_35_00_000000_0_10216 sent hash and receievd reply 0 ms 2018-01-05T11:29:05,865 DEBUG [New I/O worker #1] shufflehandler.FadvisedFileRegion: shuffleBufferSize: 131072, path: /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/35/output/attempt_1515180518813_0001_35_00_000000_0_10216/file.out 2018-01-05T11:29:05,865 INFO [Fetcher_O {Map_1} #0] orderedgrouped.MergeManager: CloseInMemoryFile. Current state: inMemoryMapOutputs.size=1, commitMemory=0, usedMemory=1714. Since last log: count=1, min=1714, max=1714, total=1714, avg=1714.0 2018-01-05T11:29:05,865 INFO [Fetcher_O {Map_1} #0] ShuffleScheduler.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_35_00_000000_0_10216} to MEMORY, csize=1718, dsize=1714, EndTime=1515180545865, TimeTaken=0, Rate=0.00 MB/s 2018-01-05T11:29:05,865 INFO [Fetcher_O {Map_1} #0] orderedgrouped.ShuffleScheduler: All inputs fetched for input vertex : Map 1 2018-01-05T11:29:05,865 INFO [Fetcher_O {Map_1} #0] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:05,865 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down FetchScheduler for input: Map_1, wasInterrupted=false 2018-01-05T11:29:05,866 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:05,866 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down fetchers for input: Map_1, shutdown timetaken: 0 ms, hasFetcherExecutorStopped: true 2018-01-05T11:29:05,867 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: TotalInMemFetchStats: count=1, totalSize=1714, min=1714, max=1714, avg=1.0 2018-01-05T11:29:05,867 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: finalMerge with #inMemoryOutputs=1, size=1714 and #onDiskOutputs=0, size=0 2018-01-05T11:29:05,867 DEBUG [TezTR-518813_1_35_1_0_0] log.PerfLogger: 2018-01-05T11:29:05,867 INFO [TezTR-518813_1_35_1_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 3.36KB 2018-01-05T11:29:05,867 DEBUG [TezTR-518813_1_35_1_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:05,867 DEBUG [TezTR-518813_1_35_1_1_0] log.PerfLogger: 2018-01-05T11:29:05,867 DEBUG [TezTR-518813_1_35_1_0_0] log.PerfLogger: 2018-01-05T11:29:05,867 INFO [TezTR-518813_1_35_1_1_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 3.36KB 2018-01-05T11:29:05,867 DEBUG [TezTR-518813_1_35_1_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:05,867 DEBUG [TezTR-518813_1_35_1_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:05,867 DEBUG [TezTR-518813_1_35_1_1_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:05,867 DEBUG [TezTR-518813_1_35_1_1_0] log.PerfLogger: 2018-01-05T11:29:05,867 DEBUG [TezTR-518813_1_35_1_1_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:05,867 DEBUG [TezTR-518813_1_35_1_1_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:05,868 DEBUG [TezTR-518813_1_35_1_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:05,868 INFO [TezTR-518813_1_35_1_0_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:29:05,868 DEBUG [TezTR-518813_1_35_1_1_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:05,868 DEBUG [TezTR-518813_1_35_1_2_0] log.PerfLogger: 2018-01-05T11:29:05,868 INFO [TezTR-518813_1_35_1_1_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:29:05,868 INFO [TezTR-518813_1_35_1_2_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 3.36KB 2018-01-05T11:29:05,868 DEBUG [TezTR-518813_1_35_1_2_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:05,868 DEBUG [TezTR-518813_1_35_1_2_0] log.PerfLogger: 2018-01-05T11:29:05,868 DEBUG [TezTR-518813_1_35_1_2_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:05,868 DEBUG [TezTR-518813_1_35_1_2_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:05,869 DEBUG [TezTR-518813_1_35_1_3_0] log.PerfLogger: 2018-01-05T11:29:05,869 INFO [TezTR-518813_1_35_1_3_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 3.36KB 2018-01-05T11:29:05,869 DEBUG [TezTR-518813_1_35_1_3_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:05,869 DEBUG [TezTR-518813_1_35_1_3_0] log.PerfLogger: 2018-01-05T11:29:05,869 DEBUG [TezTR-518813_1_35_1_3_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:05,869 DEBUG [TezTR-518813_1_35_1_3_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:05,869 DEBUG [TezTR-518813_1_35_1_2_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:05,869 INFO [TezTR-518813_1_35_1_2_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:29:05,869 DEBUG [TezTR-518813_1_35_1_3_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:05,869 INFO [TezTR-518813_1_35_1_3_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:29:05,879 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,879 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: Status: Running (Executing on YARN cluster with App id application_1515180518813_0001) 2018-01-05T11:29:05,880 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,880 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,881 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:05,881 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 Reducer 2: 0(+4)/4 Reducer 3: 0/4 2018-01-05T11:29:05,883 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: merge complete for input vertex : Map_1 2018-01-05T11:29:05,883 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: merge complete for input vertex : Map_1 2018-01-05T11:29:05,883 INFO [TezTR-518813_1_35_1_0_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:29:05,883 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: Map_1: Shuffle Runner thread complete 2018-01-05T11:29:05,883 INFO [TezTR-518813_1_35_1_2_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:29:05,883 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: Map_1: Shuffle Runner thread complete 2018-01-05T11:29:05,883 INFO [TezTR-518813_1_35_1_0_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:05,883 INFO [TezTR-518813_1_35_1_2_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:05,883 DEBUG [TezTR-518813_1_35_1_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:05,883 DEBUG [TezTR-518813_1_35_1_2_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:05,883 INFO [TezTR-518813_1_35_1_2_0] input.OrderedGroupedKVInput: Map 1: creating ValuesIterator with comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator, keyClass=org.apache.hadoop.hive.ql.io.HiveKey, valClass=org.apache.hadoop.io.BytesWritable 2018-01-05T11:29:05,883 INFO [TezTR-518813_1_35_1_0_0] input.OrderedGroupedKVInput: Map 1: creating ValuesIterator with comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator, keyClass=org.apache.hadoop.hive.ql.io.HiveKey, valClass=org.apache.hadoop.io.BytesWritable 2018-01-05T11:29:05,883 DEBUG [TezTR-518813_1_35_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:05,883 DEBUG [TezTR-518813_1_35_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:05,883 DEBUG [TezTR-518813_1_35_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:05,883 DEBUG [TezTR-518813_1_35_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:05,883 DEBUG [TezTR-518813_1_35_1_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0] columnTypes=[string] 2018-01-05T11:29:05,883 DEBUG [TezTR-518813_1_35_1_2_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0] columnTypes=[string] 2018-01-05T11:29:05,883 DEBUG [TezTR-518813_1_35_1_0_0] log.PerfLogger: 2018-01-05T11:29:05,883 DEBUG [TezTR-518813_1_35_1_2_0] log.PerfLogger: 2018-01-05T11:29:05,883 INFO [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Initializing operator SEL[3] 2018-01-05T11:29:05,883 INFO [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Initializing operator SEL[3] 2018-01-05T11:29:05,883 INFO [TezTR-518813_1_35_1_2_0] exec.SelectOperator: SELECT struct,value:struct<_col0:string>> 2018-01-05T11:29:05,883 INFO [TezTR-518813_1_35_1_0_0] exec.SelectOperator: SELECT struct,value:struct<_col0:string>> 2018-01-05T11:29:05,883 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Initialization Done 3 SEL 2018-01-05T11:29:05,883 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Operator 3 SEL initialized 2018-01-05T11:29:05,883 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Initializing children of 3 SEL 2018-01-05T11:29:05,883 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Initializing child 4 FS 2018-01-05T11:29:05,883 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Initialization Done 3 SEL 2018-01-05T11:29:05,883 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Operator 3 SEL initialized 2018-01-05T11:29:05,883 INFO [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Initializing operator FS[4] 2018-01-05T11:29:05,883 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Initializing children of 3 SEL 2018-01-05T11:29:05,883 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Initializing child 4 FS 2018-01-05T11:29:05,883 INFO [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Initializing operator FS[4] 2018-01-05T11:29:05,884 DEBUG [TezTR-518813_1_35_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:05,884 DEBUG [TezTR-518813_1_35_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:05,884 INFO [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Using serializer : org.apache.hadoop.hive.ql.io.orc.OrcSerde@5ad925b9 and formatter : org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat@4c057cd3 2018-01-05T11:29:05,884 INFO [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Using serializer : org.apache.hadoop.hive.ql.io.orc.OrcSerde@410d1c70 and formatter : org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat@aa424e3 2018-01-05T11:29:05,884 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:29:05,884 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:29:05,884 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Initialization Done 4 FS 2018-01-05T11:29:05,884 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Operator 4 FS initialized 2018-01-05T11:29:05,884 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Initialization Done 4 FS done is reset. 2018-01-05T11:29:05,884 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Initialization Done 4 FS 2018-01-05T11:29:05,884 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Operator 4 FS initialized 2018-01-05T11:29:05,884 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Initialization Done 4 FS done is reset. 2018-01-05T11:29:05,884 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Initializing child 1 SEL 2018-01-05T11:29:05,884 INFO [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Initializing operator SEL[1] 2018-01-05T11:29:05,884 INFO [TezTR-518813_1_35_1_0_0] exec.SelectOperator: SELECT struct<_col0:int,_col1:string> 2018-01-05T11:29:05,884 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Initializing child 1 SEL 2018-01-05T11:29:05,884 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Initialization Done 1 SEL 2018-01-05T11:29:05,884 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Operator 1 SEL initialized 2018-01-05T11:29:05,884 INFO [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Initializing operator SEL[1] 2018-01-05T11:29:05,884 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Initializing children of 1 SEL 2018-01-05T11:29:05,884 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Initializing child 2 GBY 2018-01-05T11:29:05,884 INFO [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Initializing operator GBY[2] 2018-01-05T11:29:05,884 INFO [TezTR-518813_1_35_1_2_0] exec.SelectOperator: SELECT struct<_col0:int,_col1:string> 2018-01-05T11:29:05,884 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Initialization Done 1 SEL 2018-01-05T11:29:05,884 DEBUG [TezTR-518813_1_35_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:05,884 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Operator 1 SEL initialized 2018-01-05T11:29:05,884 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Initializing children of 1 SEL 2018-01-05T11:29:05,884 DEBUG [TezTR-518813_1_35_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:05,884 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Initializing child 2 GBY 2018-01-05T11:29:05,884 INFO [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Initializing operator GBY[2] 2018-01-05T11:29:05,884 DEBUG [TezTR-518813_1_35_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:05,884 DEBUG [TezTR-518813_1_35_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:05,884 INFO [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Max hash table memory: 93860656 bytes 2018-01-05T11:29:05,884 INFO [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:29:05,884 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Initialization Done 2 GBY 2018-01-05T11:29:05,884 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Operator 2 GBY initialized 2018-01-05T11:29:05,884 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Initializing children of 2 GBY 2018-01-05T11:29:05,884 INFO [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Max hash table memory: 93860656 bytes 2018-01-05T11:29:05,884 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Initializing child 3 RS 2018-01-05T11:29:05,884 INFO [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Initializing operator RS[3] 2018-01-05T11:29:05,884 INFO [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:29:05,884 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Initialization Done 2 GBY 2018-01-05T11:29:05,884 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Operator 2 GBY initialized 2018-01-05T11:29:05,884 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: keys size is 1 2018-01-05T11:29:05,884 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Initializing children of 2 GBY 2018-01-05T11:29:05,884 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Key exprNodeDesc _col0 2018-01-05T11:29:05,885 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Initializing child 3 RS 2018-01-05T11:29:05,885 INFO [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Initializing operator RS[3] 2018-01-05T11:29:05,885 INFO [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Using tag = -1 2018-01-05T11:29:05,885 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: keys size is 1 2018-01-05T11:29:05,885 DEBUG [TezTR-518813_1_35_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:05,885 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Key exprNodeDesc _col0 2018-01-05T11:29:05,885 INFO [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Using tag = -1 2018-01-05T11:29:05,885 DEBUG [TezTR-518813_1_35_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:05,885 DEBUG [TezTR-518813_1_35_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:05,885 DEBUG [TezTR-518813_1_35_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:05,885 DEBUG [TezTR-518813_1_35_1_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:29:05,885 DEBUG [TezTR-518813_1_35_1_2_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:29:05,885 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS 2018-01-05T11:29:05,885 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Operator 3 RS initialized 2018-01-05T11:29:05,885 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Initialization Done 3 RS 2018-01-05T11:29:05,885 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS done is reset. 2018-01-05T11:29:05,885 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Operator 3 RS initialized 2018-01-05T11:29:05,885 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Initialization Done 3 RS done is reset. 2018-01-05T11:29:05,885 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Initialization Done 2 GBY done is reset. 2018-01-05T11:29:05,885 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Initialization Done 2 GBY done is reset. 2018-01-05T11:29:05,885 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Initialization Done 1 SEL done is reset. 2018-01-05T11:29:05,885 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Initialization Done 3 SEL done is reset. 2018-01-05T11:29:05,885 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Initialization Done 1 SEL done is reset. 2018-01-05T11:29:05,885 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Initialization Done 3 SEL done is reset. 2018-01-05T11:29:05,885 INFO [TezTR-518813_1_35_1_0_0] tez.ReduceRecordProcessor: Id =3 Id =4 <\Children> Id = 3 null<\Parent> <\FS> Id =1 Id =2 null <\Children> Id = 1 null<\Parent> <\GBY> <\Children> Id = 3 null<\Parent> <\SEL> <\Children> <\Parent> <\SEL> 2018-01-05T11:29:05,885 INFO [TezTR-518813_1_35_1_2_0] tez.ReduceRecordProcessor: Id =3 Id =4 <\Children> Id = 3 null<\Parent> <\FS> Id =1 Id =2 null <\Children> Id = 1 null<\Parent> <\GBY> <\Children> Id = 3 null<\Parent> <\SEL> <\Children> <\Parent> <\SEL> 2018-01-05T11:29:05,885 INFO [TezTR-518813_1_35_1_0_0] exec.OperatorUtils: Setting output collector: RS[3] --> Reducer 3 2018-01-05T11:29:05,885 INFO [TezTR-518813_1_35_1_2_0] exec.OperatorUtils: Setting output collector: RS[3] --> Reducer 3 2018-01-05T11:29:05,885 DEBUG [TezTR-518813_1_35_1_0_0] log.PerfLogger: 2018-01-05T11:29:05,885 INFO [TezTR-518813_1_35_1_0_0] tez.ReduceRecordProcessor: Starting Output: Reducer 3 2018-01-05T11:29:05,885 DEBUG [TezTR-518813_1_35_1_2_0] log.PerfLogger: 2018-01-05T11:29:05,886 INFO [TezTR-518813_1_35_1_2_0] tez.ReduceRecordProcessor: Starting Output: Reducer 3 2018-01-05T11:29:05,886 INFO [TezTR-518813_1_35_1_0_0] impl.ExternalSorter: Reducer 3 using: memoryMb=10, keySerializerClass=class org.apache.hadoop.hive.ql.io.HiveKey, valueSerializerClass=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization$TezBytesWritableSerializer@710975a7, comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator@4bd4dd6e, partitioner=org.apache.tez.mapreduce.partition.MRPartitioner, serialization=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.hadoop.io.serializer.WritableSerialization, org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization, org.apache.hadoop.io.serializer.avro.AvroReflectSerialization, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:29:05,886 INFO [TezTR-518813_1_35_1_2_0] impl.ExternalSorter: Reducer 3 using: memoryMb=10, keySerializerClass=class org.apache.hadoop.hive.ql.io.HiveKey, valueSerializerClass=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization$TezBytesWritableSerializer@1b5cfc8, comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator@6f68da65, partitioner=org.apache.tez.mapreduce.partition.MRPartitioner, serialization=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.hadoop.io.serializer.WritableSerialization, org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization, org.apache.hadoop.io.serializer.avro.AvroReflectSerialization, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:29:05,886 INFO [TezTR-518813_1_35_1_0_0] partition.MRPartitioner: Using oldApi, MRpartitionerClass=org.apache.hadoop.hive.ql.io.DefaultHivePartitioner 2018-01-05T11:29:05,886 INFO [TezTR-518813_1_35_1_2_0] partition.MRPartitioner: Using oldApi, MRpartitionerClass=org.apache.hadoop.hive.ql.io.DefaultHivePartitioner 2018-01-05T11:29:05,887 INFO [TezTR-518813_1_35_1_2_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 3: , UsingHashComparator=true 2018-01-05T11:29:05,887 INFO [TezTR-518813_1_35_1_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 3: , UsingHashComparator=true 2018-01-05T11:29:05,888 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: merge complete for input vertex : Map_1 2018-01-05T11:29:05,888 INFO [TezTR-518813_1_35_1_1_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:29:05,888 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: Map_1: Shuffle Runner thread complete 2018-01-05T11:29:05,888 INFO [TezTR-518813_1_35_1_1_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:05,888 DEBUG [TezTR-518813_1_35_1_1_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:05,888 INFO [TezTR-518813_1_35_1_1_0] input.OrderedGroupedKVInput: Map 1: creating ValuesIterator with comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator, keyClass=org.apache.hadoop.hive.ql.io.HiveKey, valClass=org.apache.hadoop.io.BytesWritable 2018-01-05T11:29:05,888 DEBUG [TezTR-518813_1_35_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:05,888 INFO [TezTR-518813_1_35_1_2_0] impl.PipelinedSorter: Newly allocated block size=10485760, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=10485760, total=10485760 2018-01-05T11:29:05,888 INFO [TezTR-518813_1_35_1_2_0] impl.PipelinedSorter: Pre allocating rest of memory buffers upfront 2018-01-05T11:29:05,888 INFO [TezTR-518813_1_35_1_2_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 3: , UsingHashComparator=true#blocks=1, maxMemUsage=10485760, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=10485760, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=10 2018-01-05T11:29:05,888 INFO [TezTR-518813_1_35_1_2_0] impl.PipelinedSorter: Reducer 3: reserved.remaining()=10485760, reserved.metasize=5242880 2018-01-05T11:29:05,888 INFO [TezTR-518813_1_35_1_0_0] impl.PipelinedSorter: Newly allocated block size=10485760, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=10485760, total=10485760 2018-01-05T11:29:05,888 DEBUG [TezTR-518813_1_35_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:05,888 INFO [TezTR-518813_1_35_1_0_0] impl.PipelinedSorter: Pre allocating rest of memory buffers upfront 2018-01-05T11:29:05,888 INFO [TezTR-518813_1_35_1_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 3: , UsingHashComparator=true#blocks=1, maxMemUsage=10485760, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=10485760, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=10 2018-01-05T11:29:05,888 DEBUG [TezTR-518813_1_35_1_1_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0] columnTypes=[string] 2018-01-05T11:29:05,888 INFO [TezTR-518813_1_35_1_0_0] impl.PipelinedSorter: Reducer 3: reserved.remaining()=10485760, reserved.metasize=5242880 2018-01-05T11:29:05,888 DEBUG [TezTR-518813_1_35_1_1_0] log.PerfLogger: 2018-01-05T11:29:05,888 INFO [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Initializing operator SEL[3] 2018-01-05T11:29:05,888 INFO [TezTR-518813_1_35_1_1_0] exec.SelectOperator: SELECT struct,value:struct<_col0:string>> 2018-01-05T11:29:05,888 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Initialization Done 3 SEL 2018-01-05T11:29:05,889 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Operator 3 SEL initialized 2018-01-05T11:29:05,889 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Initializing children of 3 SEL 2018-01-05T11:29:05,889 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Initializing child 4 FS 2018-01-05T11:29:05,889 INFO [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Initializing operator FS[4] 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,890 INFO [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Using serializer : org.apache.hadoop.hive.ql.io.orc.OrcSerde@34aebc27 and formatter : org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat@7b280e30 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Initialization Done 4 FS 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Operator 4 FS initialized 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Initialization Done 4 FS done is reset. 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Initializing child 1 SEL 2018-01-05T11:29:05,890 INFO [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Initializing operator SEL[1] 2018-01-05T11:29:05,890 INFO [TezTR-518813_1_35_1_1_0] exec.SelectOperator: SELECT struct<_col0:int,_col1:string> 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Initialization Done 1 SEL 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Operator 1 SEL initialized 2018-01-05T11:29:05,890 INFO [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-05_243_5530033092111214819-1/_tmp.-ext-10000/000000_0 2018-01-05T11:29:05,890 INFO [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-05_243_5530033092111214819-1/_tmp.-ext-10000/000002_0 2018-01-05T11:29:05,890 INFO [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-05_243_5530033092111214819-1/_task_tmp.-ext-10000/_tmp.000000_0 2018-01-05T11:29:05,890 INFO [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-05_243_5530033092111214819-1/_task_tmp.-ext-10000/_tmp.000002_0 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Initializing children of 1 SEL 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Initializing child 2 GBY 2018-01-05T11:29:05,890 INFO [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Initializing operator GBY[2] 2018-01-05T11:29:05,890 INFO [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-05_243_5530033092111214819-1/_tmp.-ext-10000/000002_0 2018-01-05T11:29:05,890 INFO [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-05_243_5530033092111214819-1/_tmp.-ext-10000/000000_0 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:05,890 INFO [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Max hash table memory: 93860656 bytes 2018-01-05T11:29:05,890 INFO [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Initialization Done 2 GBY 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Operator 2 GBY initialized 2018-01-05T11:29:05,890 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Initializing children of 2 GBY 2018-01-05T11:29:05,891 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Initializing child 3 RS 2018-01-05T11:29:05,891 INFO [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Initializing operator RS[3] 2018-01-05T11:29:05,891 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: keys size is 1 2018-01-05T11:29:05,891 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Key exprNodeDesc _col0 2018-01-05T11:29:05,891 INFO [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Using tag = -1 2018-01-05T11:29:05,891 DEBUG [TezTR-518813_1_35_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:05,891 DEBUG [TezTR-518813_1_35_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:05,891 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: merge complete for input vertex : Map_1 2018-01-05T11:29:05,891 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: Map_1: Shuffle Runner thread complete 2018-01-05T11:29:05,891 INFO [TezTR-518813_1_35_1_3_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:29:05,891 DEBUG [TezTR-518813_1_35_1_1_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:29:05,891 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Initialization Done 3 RS 2018-01-05T11:29:05,891 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Operator 3 RS initialized 2018-01-05T11:29:05,891 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Initialization Done 3 RS done is reset. 2018-01-05T11:29:05,891 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Initialization Done 2 GBY done is reset. 2018-01-05T11:29:05,891 INFO [TezTR-518813_1_35_1_3_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:05,891 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Initialization Done 1 SEL done is reset. 2018-01-05T11:29:05,891 DEBUG [TezTR-518813_1_35_1_3_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:05,891 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Initialization Done 3 SEL done is reset. 2018-01-05T11:29:05,891 INFO [TezTR-518813_1_35_1_3_0] input.OrderedGroupedKVInput: Map 1: creating ValuesIterator with comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator, keyClass=org.apache.hadoop.hive.ql.io.HiveKey, valClass=org.apache.hadoop.io.BytesWritable 2018-01-05T11:29:05,891 INFO [TezTR-518813_1_35_1_1_0] tez.ReduceRecordProcessor: Id =3 Id =4 <\Children> Id = 3 null<\Parent> <\FS> Id =1 Id =2 null <\Children> Id = 1 null<\Parent> <\GBY> <\Children> Id = 3 null<\Parent> <\SEL> <\Children> <\Parent> <\SEL> 2018-01-05T11:29:05,891 INFO [TezTR-518813_1_35_1_1_0] exec.OperatorUtils: Setting output collector: RS[3] --> Reducer 3 2018-01-05T11:29:05,891 DEBUG [TezTR-518813_1_35_1_1_0] log.PerfLogger: 2018-01-05T11:29:05,891 DEBUG [TezTR-518813_1_35_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:05,891 INFO [TezTR-518813_1_35_1_1_0] tez.ReduceRecordProcessor: Starting Output: Reducer 3 2018-01-05T11:29:05,891 DEBUG [TezTR-518813_1_35_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:05,892 DEBUG [TezTR-518813_1_35_1_3_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0] columnTypes=[string] 2018-01-05T11:29:05,892 DEBUG [TezTR-518813_1_35_1_3_0] log.PerfLogger: 2018-01-05T11:29:05,892 INFO [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Initializing operator SEL[3] 2018-01-05T11:29:05,892 INFO [TezTR-518813_1_35_1_3_0] exec.SelectOperator: SELECT struct,value:struct<_col0:string>> 2018-01-05T11:29:05,892 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Initialization Done 3 SEL 2018-01-05T11:29:05,892 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Operator 3 SEL initialized 2018-01-05T11:29:05,892 INFO [TezTR-518813_1_35_1_1_0] impl.ExternalSorter: Reducer 3 using: memoryMb=10, keySerializerClass=class org.apache.hadoop.hive.ql.io.HiveKey, valueSerializerClass=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization$TezBytesWritableSerializer@329a762e, comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator@25f67fcc, partitioner=org.apache.tez.mapreduce.partition.MRPartitioner, serialization=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.hadoop.io.serializer.WritableSerialization, org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization, org.apache.hadoop.io.serializer.avro.AvroReflectSerialization, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:29:05,892 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Initializing children of 3 SEL 2018-01-05T11:29:05,892 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Initializing child 4 FS 2018-01-05T11:29:05,892 INFO [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Initializing operator FS[4] 2018-01-05T11:29:05,892 INFO [TezTR-518813_1_35_1_1_0] partition.MRPartitioner: Using oldApi, MRpartitionerClass=org.apache.hadoop.hive.ql.io.DefaultHivePartitioner 2018-01-05T11:29:05,892 DEBUG [TezTR-518813_1_35_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:05,892 INFO [TezTR-518813_1_35_1_1_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 3: , UsingHashComparator=true 2018-01-05T11:29:05,892 INFO [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Using serializer : org.apache.hadoop.hive.ql.io.orc.OrcSerde@6f998063 and formatter : org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat@1ea39a89 2018-01-05T11:29:05,892 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:29:05,892 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Initialization Done 4 FS 2018-01-05T11:29:05,892 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Operator 4 FS initialized 2018-01-05T11:29:05,892 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Initialization Done 4 FS done is reset. 2018-01-05T11:29:05,892 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Initializing child 1 SEL 2018-01-05T11:29:05,892 DEBUG [TezTR-518813_1_35_1_2_0] orc.OrcOutputFormat: ORC schema = struct 2018-01-05T11:29:05,892 DEBUG [TezTR-518813_1_35_1_0_0] orc.OrcOutputFormat: ORC schema = struct 2018-01-05T11:29:05,892 INFO [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Initializing operator SEL[1] 2018-01-05T11:29:05,892 INFO [TezTR-518813_1_35_1_3_0] exec.SelectOperator: SELECT struct<_col0:int,_col1:string> 2018-01-05T11:29:05,892 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Initialization Done 1 SEL 2018-01-05T11:29:05,892 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Operator 1 SEL initialized 2018-01-05T11:29:05,892 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Initializing children of 1 SEL 2018-01-05T11:29:05,892 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Initializing child 2 GBY 2018-01-05T11:29:05,892 INFO [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Initializing operator GBY[2] 2018-01-05T11:29:05,892 INFO [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: FS[4]: records written - 1 2018-01-05T11:29:05,892 INFO [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: FS[4]: records written - 1 2018-01-05T11:29:05,892 DEBUG [TezTR-518813_1_35_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:05,893 DEBUG [TezTR-518813_1_35_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:05,893 INFO [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Max hash table memory: 93860656 bytes 2018-01-05T11:29:05,893 INFO [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:29:05,893 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Initialization Done 2 GBY 2018-01-05T11:29:05,893 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Operator 2 GBY initialized 2018-01-05T11:29:05,893 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Initializing children of 2 GBY 2018-01-05T11:29:05,893 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Initializing child 3 RS 2018-01-05T11:29:05,893 INFO [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Initializing operator RS[3] 2018-01-05T11:29:05,893 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: keys size is 1 2018-01-05T11:29:05,893 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Key exprNodeDesc _col0 2018-01-05T11:29:05,893 INFO [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Using tag = -1 2018-01-05T11:29:05,893 DEBUG [TezTR-518813_1_35_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:05,893 DEBUG [TezTR-518813_1_35_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:05,893 DEBUG [TezTR-518813_1_35_1_3_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:29:05,893 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Initialization Done 3 RS 2018-01-05T11:29:05,893 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Operator 3 RS initialized 2018-01-05T11:29:05,893 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Initialization Done 3 RS done is reset. 2018-01-05T11:29:05,893 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Initialization Done 2 GBY done is reset. 2018-01-05T11:29:05,893 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Initialization Done 1 SEL done is reset. 2018-01-05T11:29:05,893 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Initialization Done 3 SEL done is reset. 2018-01-05T11:29:05,893 INFO [TezTR-518813_1_35_1_3_0] tez.ReduceRecordProcessor: Id =3 Id =4 <\Children> Id = 3 null<\Parent> <\FS> Id =1 Id =2 null <\Children> Id = 1 null<\Parent> <\GBY> <\Children> Id = 3 null<\Parent> <\SEL> <\Children> <\Parent> <\SEL> 2018-01-05T11:29:05,893 INFO [TezTR-518813_1_35_1_3_0] exec.OperatorUtils: Setting output collector: RS[3] --> Reducer 3 2018-01-05T11:29:05,893 DEBUG [TezTR-518813_1_35_1_3_0] log.PerfLogger: 2018-01-05T11:29:05,893 INFO [TezTR-518813_1_35_1_3_0] tez.ReduceRecordProcessor: Starting Output: Reducer 3 2018-01-05T11:29:05,894 INFO [TezTR-518813_1_35_1_1_0] impl.PipelinedSorter: Newly allocated block size=10485760, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=10485760, total=10485760 2018-01-05T11:29:05,894 INFO [TezTR-518813_1_35_1_1_0] impl.PipelinedSorter: Pre allocating rest of memory buffers upfront 2018-01-05T11:29:05,894 INFO [TezTR-518813_1_35_1_1_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 3: , UsingHashComparator=true#blocks=1, maxMemUsage=10485760, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=10485760, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=10 2018-01-05T11:29:05,894 INFO [TezTR-518813_1_35_1_1_0] impl.PipelinedSorter: Reducer 3: reserved.remaining()=10485760, reserved.metasize=5242880 2018-01-05T11:29:05,894 INFO [TezTR-518813_1_35_1_3_0] impl.ExternalSorter: Reducer 3 using: memoryMb=10, keySerializerClass=class org.apache.hadoop.hive.ql.io.HiveKey, valueSerializerClass=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization$TezBytesWritableSerializer@bcd6c88, comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator@12a516b7, partitioner=org.apache.tez.mapreduce.partition.MRPartitioner, serialization=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.hadoop.io.serializer.WritableSerialization, org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization, org.apache.hadoop.io.serializer.avro.AvroReflectSerialization, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:29:05,895 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,895 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,895 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,895 INFO [TezTR-518813_1_35_1_3_0] partition.MRPartitioner: Using oldApi, MRpartitionerClass=org.apache.hadoop.hive.ql.io.DefaultHivePartitioner 2018-01-05T11:29:05,895 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,895 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,895 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,895 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,895 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,895 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,895 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,895 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,896 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,896 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,896 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,896 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,896 INFO [TezTR-518813_1_35_1_3_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 3: , UsingHashComparator=true 2018-01-05T11:29:05,896 INFO [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-05_243_5530033092111214819-1/_tmp.-ext-10000/000001_0 2018-01-05T11:29:05,896 INFO [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-05_243_5530033092111214819-1/_task_tmp.-ext-10000/_tmp.000001_0 2018-01-05T11:29:05,896 INFO [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-05_243_5530033092111214819-1/_tmp.-ext-10000/000001_0 2018-01-05T11:29:05,896 DEBUG [TezTR-518813_1_35_1_1_0] orc.OrcOutputFormat: ORC schema = struct 2018-01-05T11:29:05,896 INFO [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: FS[4]: records written - 1 2018-01-05T11:29:05,897 INFO [TezTR-518813_1_35_1_1_0] impl.PhysicalFsWriter: ORC writer created for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-05_243_5530033092111214819-1/_task_tmp.-ext-10000/_tmp.000001_0 with stripeSize: 67108864 blockSize: 268435456 compression: ZLIB bufferSize: 262144 2018-01-05T11:29:05,897 INFO [TezTR-518813_1_35_1_2_0] impl.PhysicalFsWriter: ORC writer created for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-05_243_5530033092111214819-1/_task_tmp.-ext-10000/_tmp.000002_0 with stripeSize: 67108864 blockSize: 268435456 compression: ZLIB bufferSize: 262144 2018-01-05T11:29:05,897 INFO [TezTR-518813_1_35_1_0_0] impl.PhysicalFsWriter: ORC writer created for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-05_243_5530033092111214819-1/_task_tmp.-ext-10000/_tmp.000000_0 with stripeSize: 67108864 blockSize: 268435456 compression: ZLIB bufferSize: 262144 2018-01-05T11:29:05,897 INFO [TezTR-518813_1_35_1_3_0] impl.PipelinedSorter: Newly allocated block size=10485760, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=10485760, total=10485760 2018-01-05T11:29:05,897 INFO [TezTR-518813_1_35_1_3_0] impl.PipelinedSorter: Pre allocating rest of memory buffers upfront 2018-01-05T11:29:05,897 INFO [TezTR-518813_1_35_1_3_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 3: , UsingHashComparator=true#blocks=1, maxMemUsage=10485760, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=10485760, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=10 2018-01-05T11:29:05,898 INFO [TezTR-518813_1_35_1_3_0] impl.PipelinedSorter: Reducer 3: reserved.remaining()=10485760, reserved.metasize=5242880 2018-01-05T11:29:05,899 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,899 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,899 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,899 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,899 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,899 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,899 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,899 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,899 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,899 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,899 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,899 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,899 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,899 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,899 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,899 INFO [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-05_243_5530033092111214819-1/_tmp.-ext-10000/000003_0 2018-01-05T11:29:05,900 INFO [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-05_243_5530033092111214819-1/_task_tmp.-ext-10000/_tmp.000003_0 2018-01-05T11:29:05,900 INFO [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-05_243_5530033092111214819-1/_tmp.-ext-10000/000003_0 2018-01-05T11:29:05,900 DEBUG [TezTR-518813_1_35_1_3_0] orc.OrcOutputFormat: ORC schema = struct 2018-01-05T11:29:05,900 INFO [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: FS[4]: records written - 1 2018-01-05T11:29:05,900 INFO [TezTR-518813_1_35_1_3_0] impl.PhysicalFsWriter: ORC writer created for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-05_243_5530033092111214819-1/_task_tmp.-ext-10000/_tmp.000003_0 with stripeSize: 67108864 blockSize: 268435456 compression: ZLIB bufferSize: 262144 2018-01-05T11:29:05,936 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000071, QueryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a-35, DagId=35, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-1, Succeeded=true, StartTime=1515180545752, EndTime=1515180545936 2018-01-05T11:29:05,939 DEBUG [TezTR-518813_1_35_1_2_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:05,940 DEBUG [TezTR-518813_1_35_1_1_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:05,942 INFO [TezTR-518813_1_35_1_2_0] impl.WriterImpl: ORC writer created for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-05_243_5530033092111214819-1/_task_tmp.-ext-10000/_tmp.000002_0 with stripeSize: 67108864 blockSize: 268435456 compression: ZLIB bufferSize: 262144 2018-01-05T11:29:05,942 INFO [TezTR-518813_1_35_1_1_0] impl.WriterImpl: ORC writer created for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-05_243_5530033092111214819-1/_task_tmp.-ext-10000/_tmp.000001_0 with stripeSize: 67108864 blockSize: 268435456 compression: ZLIB bufferSize: 262144 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,943 INFO [TezTR-518813_1_35_1_3_0] impl.OrcCodecPool: Got brand-new codec ZLIB 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,943 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,943 INFO [TezTR-518813_1_35_1_3_0] impl.WriterImpl: ORC writer created for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-05_243_5530033092111214819-1/_task_tmp.-ext-10000/_tmp.000003_0 with stripeSize: 67108864 blockSize: 268435456 compression: ZLIB bufferSize: 262144 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,944 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,947 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,947 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,947 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,947 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,947 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,947 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,947 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,947 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,947 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,948 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000072, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_35_01_000000_0, eventCount=1 } 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,945 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,948 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000073, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_35_01_000001_0, eventCount=1 } 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,948 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,948 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:05,948 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,949 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,950 INFO [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: FS[4]: records written - 10 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,950 INFO [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: FS[4]: records written - 10 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,950 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,951 INFO [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: FS[4]: records written - 10 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,951 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,952 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,953 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000074, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_35_01_000002_0, eventCount=1 } 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,953 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,954 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,954 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,955 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000075, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_35_01_000003_0, eventCount=1 } 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,955 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,956 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,956 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,957 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,958 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,959 INFO [TezTR-518813_1_35_1_0_0] impl.OrcCodecPool: Got brand-new codec ZLIB 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,959 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,960 INFO [TezTR-518813_1_35_1_0_0] impl.WriterImpl: ORC writer created for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-05_243_5530033092111214819-1/_task_tmp.-ext-10000/_tmp.000000_0 with stripeSize: 67108864 blockSize: 268435456 compression: ZLIB bufferSize: 262144 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,960 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,961 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,962 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,963 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,964 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,965 INFO [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: FS[4]: records written - 10 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,965 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,966 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,967 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,968 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,969 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,970 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,971 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,972 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,973 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,974 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,975 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,976 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,977 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,978 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,979 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,980 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,981 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,982 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,983 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,984 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,986 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,987 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,985 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,989 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,988 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,990 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,991 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,992 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,993 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,994 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,995 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,996 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,997 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,999 INFO [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: FS[4]: records written - 100 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:05,998 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:05,999 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,000 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,002 INFO [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: FS[4]: records written - 100 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,001 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,003 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,005 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,002 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,004 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,006 INFO [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: FS[4]: records written - 100 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,007 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,006 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,008 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,010 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,009 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,011 DEBUG [TezTR-518813_1_35_1_1_0] log.PerfLogger: 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_1_0] mr.ObjectCache: djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: close called for operator SEL[3] 2018-01-05T11:29:06,013 INFO [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Closing operator SEL[3] 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Closing child = FS[4] 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: close called for operator FS[4] 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:06,013 INFO [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: Closing operator FS[4] 2018-01-05T11:29:06,013 INFO [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: FS[4]: records written - 138 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,012 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,013 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,014 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,015 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,016 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_2_0] log.PerfLogger: 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_2_0] mr.ObjectCache: djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: close called for operator SEL[3] 2018-01-05T11:29:06,017 INFO [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Closing operator SEL[3] 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Closing child = FS[4] 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: close called for operator FS[4] 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:06,017 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,018 INFO [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: Closing operator FS[4] 2018-01-05T11:29:06,018 INFO [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: FS[4]: records written - 128 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_3_0] log.PerfLogger: 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_3_0] mr.ObjectCache: djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: close called for operator SEL[3] 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,018 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,018 INFO [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Closing operator SEL[3] 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Closing child = FS[4] 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: close called for operator FS[4] 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:06,019 INFO [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: Closing operator FS[4] 2018-01-05T11:29:06,019 INFO [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: FS[4]: records written - 115 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,019 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,020 INFO [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: FS[4]: records written - 100 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_1_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/-mr-10001 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,020 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,021 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_2_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/-mr-10001 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_3_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/-mr-10001 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,022 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] log.PerfLogger: 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] mr.ObjectCache: djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: close called for operator SEL[3] 2018-01-05T11:29:06,023 INFO [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Closing operator SEL[3] 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Closing child = FS[4] 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: close called for operator FS[4] 2018-01-05T11:29:06,023 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:06,023 INFO [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: Closing operator FS[4] 2018-01-05T11:29:06,024 INFO [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: FS[4]: records written - 119 2018-01-05T11:29:06,027 DEBUG [TezTR-518813_1_35_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,027 DEBUG [TezTR-518813_1_35_1_0_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/-mr-10001 2018-01-05T11:29:06,052 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000072, requestId=3, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_35_01_000000_0, eventCount=1 } 2018-01-05T11:29:06,052 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000073, requestId=3, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_35_01_000001_0, eventCount=1 } 2018-01-05T11:29:06,053 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=3, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:06,053 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=3, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:06,055 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000074, requestId=3, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_35_01_000002_0, eventCount=1 } 2018-01-05T11:29:06,056 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=3, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:06,058 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000075, requestId=3, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_35_01_000003_0, eventCount=1 } 2018-01-05T11:29:06,059 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=3, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:06,063 DEBUG [TezTR-518813_1_35_1_1_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/-mr-10001/tmpstats-1 2018-01-05T11:29:06,063 DEBUG [TezTR-518813_1_35_1_1_0] fs.FSStatsPublisher: Writing stats in it : {default.tab_part/ds=2008-04-08/={rawDataSize=12972, numRows=138}} 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_2_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/-mr-10001/tmpstats-2 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_2_0] fs.FSStatsPublisher: Writing stats in it : {default.tab_part/ds=2008-04-08/={rawDataSize=12032, numRows=128}} 2018-01-05T11:29:06,064 INFO [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: RECORDS_OUT_1_default.tab_part:138, 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_1_0] exec.FileSinkOperator: 4 Close done 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Closing child = SEL[1] 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: close called for operator SEL[1] 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:06,064 INFO [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Closing operator SEL[1] 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: Closing child = GBY[2] 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: close called for operator GBY[2] 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:06,064 INFO [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Closing operator GBY[2] 2018-01-05T11:29:06,064 INFO [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Begin Hash Table flush: size = 1 2018-01-05T11:29:06,064 INFO [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: RECORDS_OUT_1_default.tab_part:128, 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_2_0] exec.FileSinkOperator: 4 Close done 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Closing child = SEL[1] 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: close called for operator SEL[1] 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:06,064 INFO [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Closing operator SEL[1] 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: Closing child = GBY[2] 2018-01-05T11:29:06,064 INFO [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: keys are [_col0] num distributions: 1 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_3_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/-mr-10001/tmpstats-3 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_3_0] fs.FSStatsPublisher: Writing stats in it : {default.tab_part/ds=2008-04-08/={rawDataSize=10810, numRows=115}} 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: close called for operator GBY[2] 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:06,064 INFO [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Closing operator GBY[2] 2018-01-05T11:29:06,064 INFO [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Begin Hash Table flush: size = 1 2018-01-05T11:29:06,064 INFO [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: keys are [_col0] num distributions: 1 2018-01-05T11:29:06,064 INFO [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: RECORDS_OUT_1_default.tab_part:115, 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_3_0] exec.FileSinkOperator: 4 Close done 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Closing child = SEL[1] 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: close called for operator SEL[1] 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:06,064 INFO [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Closing operator SEL[1] 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: Closing child = GBY[2] 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: close called for operator GBY[2] 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:06,064 INFO [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Closing operator GBY[2] 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: Closing child = RS[3] 2018-01-05T11:29:06,064 INFO [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Begin Hash Table flush: size = 1 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: close called for operator RS[3] 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:06,064 INFO [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: Closing operator RS[3] 2018-01-05T11:29:06,064 INFO [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: RECORDS_OUT_INTERMEDIATE_Reducer_2:1, 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_1_0] exec.ReduceSinkOperator: 3 Close done 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_1_0] exec.GroupByOperator: 2 Close done 2018-01-05T11:29:06,064 INFO [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: keys are [_col0] num distributions: 1 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: Closing child = RS[3] 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: 1 Close done 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_1_0] exec.SelectOperator: 3 Close done 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: Closing child = RS[3] 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: close called for operator RS[3] 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:06,064 INFO [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: Closing operator RS[3] 2018-01-05T11:29:06,064 INFO [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: RECORDS_OUT_INTERMEDIATE_Reducer_2:1, 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_3_0] exec.ReduceSinkOperator: 3 Close done 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_3_0] exec.GroupByOperator: 2 Close done 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: close called for operator RS[3] 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: 1 Close done 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:06,064 INFO [TezTR-518813_1_35_1_1_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_35_01_000001_0 2018-01-05T11:29:06,064 DEBUG [TezTR-518813_1_35_1_3_0] exec.SelectOperator: 3 Close done 2018-01-05T11:29:06,064 INFO [TezTR-518813_1_35_1_1_0] orderedgrouped.Shuffle: Shutting down Shuffle for source: Map_1 2018-01-05T11:29:06,064 INFO [TezTR-518813_1_35_1_1_0] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:29:06,064 INFO [TezTR-518813_1_35_1_3_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_35_01_000003_0 2018-01-05T11:29:06,064 INFO [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: Closing operator RS[3] 2018-01-05T11:29:06,065 INFO [TezTR-518813_1_35_1_1_0] impl.PipelinedSorter: Reducer 3: Starting flush of map output 2018-01-05T11:29:06,065 INFO [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: RECORDS_OUT_INTERMEDIATE_Reducer_2:1, 2018-01-05T11:29:06,065 DEBUG [TezTR-518813_1_35_1_2_0] exec.ReduceSinkOperator: 3 Close done 2018-01-05T11:29:06,065 DEBUG [TezTR-518813_1_35_1_2_0] exec.GroupByOperator: 2 Close done 2018-01-05T11:29:06,065 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: 1 Close done 2018-01-05T11:29:06,065 DEBUG [TezTR-518813_1_35_1_2_0] exec.SelectOperator: 3 Close done 2018-01-05T11:29:06,065 INFO [TezTR-518813_1_35_1_3_0] orderedgrouped.Shuffle: Shutting down Shuffle for source: Map_1 2018-01-05T11:29:06,065 INFO [TezTR-518813_1_35_1_3_0] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:29:06,065 INFO [TezTR-518813_1_35_1_3_0] impl.PipelinedSorter: Reducer 3: Starting flush of map output 2018-01-05T11:29:06,065 INFO [TezTR-518813_1_35_1_2_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_35_01_000002_0 2018-01-05T11:29:06,065 INFO [TezTR-518813_1_35_1_3_0] impl.PipelinedSorter: Reducer 3: Span0.length = 1, perItem = 642 2018-01-05T11:29:06,065 INFO [TezTR-518813_1_35_1_2_0] orderedgrouped.Shuffle: Shutting down Shuffle for source: Map_1 2018-01-05T11:29:06,065 INFO [TezTR-518813_1_35_1_1_0] impl.PipelinedSorter: Reducer 3: Span0.length = 1, perItem = 703 2018-01-05T11:29:06,065 INFO [TezTR-518813_1_35_1_1_0] impl.PipelinedSorter: Reducer 3: done sorting span=0, length=1, time=0 2018-01-05T11:29:06,065 INFO [TezTR-518813_1_35_1_3_0] impl.PipelinedSorter: Reducer 3: done sorting span=0, length=1, time=0 2018-01-05T11:29:06,065 DEBUG [TezTR-518813_1_35_1_1_0] util.Progress: Illegal progress value found, progress is Float.NaN. Progress will be changed to 0 2018-01-05T11:29:06,065 INFO [TezTR-518813_1_35_1_2_0] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:29:06,065 INFO [TezTR-518813_1_35_1_2_0] impl.PipelinedSorter: Reducer 3: Starting flush of map output 2018-01-05T11:29:06,065 INFO [TezTR-518813_1_35_1_1_0] impl.PipelinedSorter: Reducer 3: Heap = SpanIterator<0:0> (span=Span[5242880,703]), 2018-01-05T11:29:06,065 INFO [TezTR-518813_1_35_1_2_0] impl.PipelinedSorter: Reducer 3: Span0.length = 1, perItem = 688 2018-01-05T11:29:06,065 INFO [TezTR-518813_1_35_1_2_0] impl.PipelinedSorter: Reducer 3: done sorting span=0, length=1, time=0 2018-01-05T11:29:06,065 DEBUG [TezTR-518813_1_35_1_2_0] util.Progress: Illegal progress value found, progress is Float.NaN. Progress will be changed to 0 2018-01-05T11:29:06,065 INFO [TezTR-518813_1_35_1_2_0] impl.PipelinedSorter: Reducer 3: Heap = SpanIterator<0:0> (span=Span[5242880,688]), 2018-01-05T11:29:06,065 DEBUG [TezTR-518813_1_35_1_3_0] util.Progress: Illegal progress value found, progress is Float.NaN. Progress will be changed to 0 2018-01-05T11:29:06,065 INFO [TezTR-518813_1_35_1_3_0] impl.PipelinedSorter: Reducer 3: Heap = SpanIterator<0:0> (span=Span[5242880,642]), 2018-01-05T11:29:06,068 DEBUG [TezTR-518813_1_35_1_0_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/-mr-10001/tmpstats-0 2018-01-05T11:29:06,068 DEBUG [TezTR-518813_1_35_1_0_0] fs.FSStatsPublisher: Writing stats in it : {default.tab_part/ds=2008-04-08/={rawDataSize=11186, numRows=119}} 2018-01-05T11:29:06,069 INFO [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: RECORDS_OUT_1_default.tab_part:119, 2018-01-05T11:29:06,069 DEBUG [TezTR-518813_1_35_1_0_0] exec.FileSinkOperator: 4 Close done 2018-01-05T11:29:06,069 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Closing child = SEL[1] 2018-01-05T11:29:06,069 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: close called for operator SEL[1] 2018-01-05T11:29:06,069 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:06,069 INFO [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Closing operator SEL[1] 2018-01-05T11:29:06,069 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: Closing child = GBY[2] 2018-01-05T11:29:06,069 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: close called for operator GBY[2] 2018-01-05T11:29:06,069 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:06,069 INFO [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Closing operator GBY[2] 2018-01-05T11:29:06,069 INFO [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Begin Hash Table flush: size = 1 2018-01-05T11:29:06,069 INFO [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: keys are [_col0] num distributions: 1 2018-01-05T11:29:06,069 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: Closing child = RS[3] 2018-01-05T11:29:06,069 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: close called for operator RS[3] 2018-01-05T11:29:06,069 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:06,069 INFO [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: Closing operator RS[3] 2018-01-05T11:29:06,069 INFO [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: RECORDS_OUT_INTERMEDIATE_Reducer_2:1, 2018-01-05T11:29:06,069 DEBUG [TezTR-518813_1_35_1_0_0] exec.ReduceSinkOperator: 3 Close done 2018-01-05T11:29:06,069 DEBUG [TezTR-518813_1_35_1_0_0] exec.GroupByOperator: 2 Close done 2018-01-05T11:29:06,069 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: 1 Close done 2018-01-05T11:29:06,069 DEBUG [TezTR-518813_1_35_1_0_0] exec.SelectOperator: 3 Close done 2018-01-05T11:29:06,069 INFO [TezTR-518813_1_35_1_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_35_01_000000_0 2018-01-05T11:29:06,069 INFO [TezTR-518813_1_35_1_0_0] orderedgrouped.Shuffle: Shutting down Shuffle for source: Map_1 2018-01-05T11:29:06,069 INFO [TezTR-518813_1_35_1_0_0] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:29:06,069 INFO [TezTR-518813_1_35_1_0_0] impl.PipelinedSorter: Reducer 3: Starting flush of map output 2018-01-05T11:29:06,069 INFO [TezTR-518813_1_35_1_0_0] impl.PipelinedSorter: Reducer 3: Span0.length = 1, perItem = 652 2018-01-05T11:29:06,069 INFO [TezTR-518813_1_35_1_0_0] impl.PipelinedSorter: Reducer 3: done sorting span=0, length=1, time=0 2018-01-05T11:29:06,069 DEBUG [TezTR-518813_1_35_1_0_0] util.Progress: Illegal progress value found, progress is Float.NaN. Progress will be changed to 0 2018-01-05T11:29:06,069 INFO [TezTR-518813_1_35_1_0_0] impl.PipelinedSorter: Reducer 3: Heap = SpanIterator<0:0> (span=Span[5242880,652]), 2018-01-05T11:29:06,083 INFO [TezTR-518813_1_35_1_3_0] impl.PipelinedSorter: Reducer 3: Spilling to /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/35/output/attempt_1515180518813_0001_35_01_000003_0_10228_0/file.out 2018-01-05T11:29:06,083 INFO [TezTR-518813_1_35_1_2_0] impl.PipelinedSorter: Reducer 3: Spilling to /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/35/output/attempt_1515180518813_0001_35_01_000002_0_10225_0/file.out 2018-01-05T11:29:06,085 INFO [TezTR-518813_1_35_1_1_0] impl.PipelinedSorter: Reducer 3: Spilling to /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/35/output/attempt_1515180518813_0001_35_01_000001_0_10222_0/file.out 2018-01-05T11:29:06,086 INFO [TezTR-518813_1_35_1_0_0] impl.PipelinedSorter: Reducer 3: Spilling to /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/35/output/attempt_1515180518813_0001_35_01_000000_0_10219_0/file.out 2018-01-05T11:29:06,101 INFO [TezTR-518813_1_35_1_3_0] shuffle.ShuffleUtils: EmptyPartition bitsetSize=3, numOutputs=4, emptyPartitions=3, compressedSize=3 2018-01-05T11:29:06,102 DEBUG [TezTR-518813_1_35_1_3_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:06,102 INFO [TezTR-518813_1_35_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:06,103 INFO [TezTR-518813_1_35_1_3_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_35_01_000003_0, askedToStop=false 2018-01-05T11:29:06,103 INFO [TezTR-518813_1_35_1_3_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_35_01_000003_0, stopRequested=false 2018-01-05T11:29:06,103 INFO [TezTR-518813_1_35_1_3_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_35_01_000003_0: Counters: 37 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=73, REDUCE_INPUT_RECORDS=115, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=116, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=642, OUTPUT_BYTES_WITH_OVERHEAD=652, OUTPUT_BYTES_PHYSICAL=656, ADDITIONAL_SPILLS_BYTES_WRITTEN=1718, ADDITIONAL_SPILLS_BYTES_READ=1718, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=1718, SHUFFLE_BYTES_DECOMPRESSED=1714, SHUFFLE_BYTES_TO_MEM=1718, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=31, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][HIVE CREATED_FILES=1, RECORDS_OUT_1_default.tab_part=115, RECORDS_OUT_INTERMEDIATE_Reducer_2=1][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0]] 2018-01-05T11:29:06,103 DEBUG [TezTR-518813_1_35_1_3_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_35_1_3_0 thread id: 2224 scheme: file bytesRead: 208 bytesWritten: 1648 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:06,103 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_35_01_000003_0, isTaskDone=true 2018-01-05T11:29:06,103 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000075, requestId=4, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_35_01_000003_0, eventCount=4 } 2018-01-05T11:29:06,104 INFO [TezTR-518813_1_35_1_1_0] shuffle.ShuffleUtils: EmptyPartition bitsetSize=3, numOutputs=4, emptyPartitions=3, compressedSize=3 2018-01-05T11:29:06,104 INFO [TezTR-518813_1_35_1_0_0] shuffle.ShuffleUtils: EmptyPartition bitsetSize=3, numOutputs=4, emptyPartitions=3, compressedSize=3 2018-01-05T11:29:06,104 DEBUG [TezTR-518813_1_35_1_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:06,104 INFO [TezTR-518813_1_35_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:06,104 DEBUG [TezTR-518813_1_35_1_1_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:06,104 INFO [TezTR-518813_1_35_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:06,104 INFO [TezTR-518813_1_35_1_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_35_01_000000_0, askedToStop=false 2018-01-05T11:29:06,104 INFO [TezTR-518813_1_35_1_1_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_35_01_000001_0, askedToStop=false 2018-01-05T11:29:06,104 INFO [TezTR-518813_1_35_1_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_35_01_000000_0, stopRequested=false 2018-01-05T11:29:06,104 INFO [TezTR-518813_1_35_1_1_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_35_01_000001_0, stopRequested=false 2018-01-05T11:29:06,105 INFO [TezTR-518813_1_35_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_35_01_000001_0: Counters: 37 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=81, REDUCE_INPUT_RECORDS=138, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=139, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=703, OUTPUT_BYTES_WITH_OVERHEAD=713, OUTPUT_BYTES_PHYSICAL=717, ADDITIONAL_SPILLS_BYTES_WRITTEN=2014, ADDITIONAL_SPILLS_BYTES_READ=2014, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=2014, SHUFFLE_BYTES_DECOMPRESSED=2010, SHUFFLE_BYTES_TO_MEM=2014, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=14, MERGE_PHASE_TIME=38, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][HIVE CREATED_FILES=1, RECORDS_OUT_1_default.tab_part=138, RECORDS_OUT_INTERMEDIATE_Reducer_2=1][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0]] 2018-01-05T11:29:06,105 INFO [TezTR-518813_1_35_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_35_01_000000_0: Counters: 37 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=75, REDUCE_INPUT_RECORDS=119, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=120, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=652, OUTPUT_BYTES_WITH_OVERHEAD=662, OUTPUT_BYTES_PHYSICAL=666, ADDITIONAL_SPILLS_BYTES_WRITTEN=1762, ADDITIONAL_SPILLS_BYTES_READ=1762, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=1762, SHUFFLE_BYTES_DECOMPRESSED=1758, SHUFFLE_BYTES_TO_MEM=1762, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=36, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][HIVE CREATED_FILES=1, RECORDS_OUT_1_default.tab_part=119, RECORDS_OUT_INTERMEDIATE_Reducer_2=1][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0]] 2018-01-05T11:29:06,105 DEBUG [TezTR-518813_1_35_1_1_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_35_1_1_0 thread id: 2212 scheme: file bytesRead: 208 bytesWritten: 1766 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:06,105 INFO [TezTR-518813_1_35_1_2_0] shuffle.ShuffleUtils: EmptyPartition bitsetSize=3, numOutputs=4, emptyPartitions=3, compressedSize=3 2018-01-05T11:29:06,105 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_35_01_000003_0, creationTime=1515180545836, allocationTime=1515180545838, startTime=1515180545847, finishTime=1515180546105, timeTaken=258, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:06,105 DEBUG [TezTR-518813_1_35_1_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_35_1_0_0 thread id: 2206 scheme: file bytesRead: 208 bytesWritten: 1640 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:06,105 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=4, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:06,105 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_35_01_000001_0, isTaskDone=true 2018-01-05T11:29:06,105 INFO [Task-Executor-2] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_35_01_000003_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:06,105 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_35_01_000003_0 2018-01-05T11:29:06,105 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_35_01_000000_0, isTaskDone=true 2018-01-05T11:29:06,105 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_35_01_000003_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:06,105 INFO [Task-Executor-2] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_35_01_000003_0 2018-01-05T11:29:06,105 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000073, requestId=4, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_35_01_000001_0, eventCount=4 } 2018-01-05T11:29:06,105 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000072, requestId=4, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_35_01_000000_0, eventCount=4 } 2018-01-05T11:29:06,105 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_35_01_000003_0; false 2018-01-05T11:29:06,105 INFO [Task-Executor-2] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000075=259 2018-01-05T11:29:06,105 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_35_01_000003_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:06,105 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_35_01_000003_0: true 2018-01-05T11:29:06,105 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:06,105 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:06,105 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_35_01_000003_0 2018-01-05T11:29:06,105 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000075, QueryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a-35, Vertex=Reducer 2, FragmentNum=3, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:06,106 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_35_01_000003_0 complete. WaitQueueSize=0, numSlotsAvailable=1, preemptionQueueSize=3 2018-01-05T11:29:06,106 DEBUG [TezTR-518813_1_35_1_2_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:06,106 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_35_01_000003_0 2018-01-05T11:29:06,106 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_35_01_000003, startTime=1515180545847, finishTime=1515180546105, timeTaken=258, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_35_01_000003_0, diagnostics=, counters=Counters: 42, File System Counters, FILE_BYTES_READ=208, FILE_BYTES_WRITTEN=1648, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=73, REDUCE_INPUT_RECORDS=115, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=116, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=642, OUTPUT_BYTES_WITH_OVERHEAD=652, OUTPUT_BYTES_PHYSICAL=656, ADDITIONAL_SPILLS_BYTES_WRITTEN=1718, ADDITIONAL_SPILLS_BYTES_READ=1718, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=1718, SHUFFLE_BYTES_DECOMPRESSED=1714, SHUFFLE_BYTES_TO_MEM=1718, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=31, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, HIVE, CREATED_FILES=1, RECORDS_OUT_1_default.tab_part=115, RECORDS_OUT_INTERMEDIATE_Reducer_2=1, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0 2018-01-05T11:29:06,106 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000075, QueryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a-35, DagId=35, VertexName=Reducer 2, TaskId=3, TaskAttemptId=0, ThreadName=Task-Executor-2, Succeeded=true, StartTime=1515180545845, EndTime=1515180546106 2018-01-05T11:29:06,106 INFO [TezTR-518813_1_35_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:06,106 INFO [TezTR-518813_1_35_1_2_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_35_01_000002_0, askedToStop=false 2018-01-05T11:29:06,106 INFO [TezTR-518813_1_35_1_2_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_35_01_000002_0, stopRequested=false 2018-01-05T11:29:06,106 INFO [TezTR-518813_1_35_1_2_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_35_01_000002_0: Counters: 37 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=80, REDUCE_INPUT_RECORDS=128, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=129, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=688, OUTPUT_BYTES_WITH_OVERHEAD=698, OUTPUT_BYTES_PHYSICAL=702, ADDITIONAL_SPILLS_BYTES_WRITTEN=1887, ADDITIONAL_SPILLS_BYTES_READ=1887, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=1887, SHUFFLE_BYTES_DECOMPRESSED=1883, SHUFFLE_BYTES_TO_MEM=1887, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=28, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2][HIVE CREATED_FILES=1, RECORDS_OUT_1_default.tab_part=128, RECORDS_OUT_INTERMEDIATE_Reducer_2=1][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0]] 2018-01-05T11:29:06,106 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_35_01 [Reducer 2], tasks=4, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:06,106 DEBUG [TezTR-518813_1_35_1_2_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_35_1_2_0 thread id: 2218 scheme: file bytesRead: 208 bytesWritten: 1747 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:06,106 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000075, stoppedTime=1515180546106, exitStatus=0 2018-01-05T11:29:06,106 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000075 exited with diagnostics set to Container completed. 2018-01-05T11:29:06,106 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_35_02 [Reducer 3] attempt: attempt_1515180518813_0001_35_01_000003_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:29:06,106 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_35_01_000002_0, isTaskDone=true 2018-01-05T11:29:06,106 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000074, requestId=4, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_35_01_000002_0, eventCount=4 } 2018-01-05T11:29:06,106 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=4, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:06,106 INFO [Task-Executor-3] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_35_01_000001_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:06,106 INFO [Task-Executor-3] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_35_01_000001_0 2018-01-05T11:29:06,106 INFO [Task-Executor-3] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000073=266 2018-01-05T11:29:06,106 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_35_01_000001_0: true 2018-01-05T11:29:06,106 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Scheduling 2 tasks for vertex: Reducer 3 with totalTasks: 4. 1 source tasks completed out of 4. MinSourceTaskCompletedFraction: 0.25 min: 0.05 max: 0.75 2018-01-05T11:29:06,106 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=4, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:06,106 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_35_01_000001_0 2018-01-05T11:29:06,106 INFO [Task-Executor-1] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_35_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:06,106 INFO [Task-Executor-1] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_35_01_000000_0 2018-01-05T11:29:06,106 INFO [App Shared Pool - #0] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Reducer 2, destinationVertexName=Reducer 3. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:29:06,106 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000073, QueryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a-35, Vertex=Reducer 2, FragmentNum=1, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:06,106 INFO [Task-Executor-1] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000072=267 2018-01-05T11:29:06,106 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_35_02 [Reducer 3] 2018-01-05T11:29:06,106 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_35_01_000001_0 complete. WaitQueueSize=0, numSlotsAvailable=2, preemptionQueueSize=2 2018-01-05T11:29:06,106 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_35_01_000001_0 2018-01-05T11:29:06,106 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_35_01_000000_0: true 2018-01-05T11:29:06,106 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_35_01_000001_0, creationTime=1515180545836, allocationTime=1515180545838, startTime=1515180545840, finishTime=1515180546106, timeTaken=266, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:06,107 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000073, QueryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a-35, DagId=35, VertexName=Reducer 2, TaskId=1, TaskAttemptId=0, ThreadName=Task-Executor-3, Succeeded=true, StartTime=1515180545840, EndTime=1515180546107 2018-01-05T11:29:06,107 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_35_01_000000_0 2018-01-05T11:29:06,107 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000072, QueryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a-35, Vertex=Reducer 2, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:06,107 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000075 2018-01-05T11:29:06,107 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_35_01_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=3, preemptionQueueSize=1 2018-01-05T11:29:06,107 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_35_01_000000_0 2018-01-05T11:29:06,107 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000072, QueryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a-35, DagId=35, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-1, Succeeded=true, StartTime=1515180545839, EndTime=1515180546107 2018-01-05T11:29:06,107 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=4, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:06,107 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_35_01_000002_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:06,107 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_35_01_000002_0 2018-01-05T11:29:06,107 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_35_01_000000_0, creationTime=1515180545836, allocationTime=1515180545837, startTime=1515180545839, finishTime=1515180546107, timeTaken=268, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:06,107 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000074=264 2018-01-05T11:29:06,107 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_35_01_000002_0: true 2018-01-05T11:29:06,107 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:TASK_STARTED]: vertexName=Reducer 3, taskId=task_1515180518813_0001_35_02_000002, scheduledTime=1515180546107, launchTime=1515180546107 2018-01-05T11:29:06,107 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_35_01_000002_0 2018-01-05T11:29:06,107 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:TASK_STARTED]: vertexName=Reducer 3, taskId=task_1515180518813_0001_35_02_000000, scheduledTime=1515180546107, launchTime=1515180546107 2018-01-05T11:29:06,107 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000074, QueryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a-35, Vertex=Reducer 2, FragmentNum=2, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:06,107 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_35_01_000002_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:29:06,107 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_35_01_000002_0 2018-01-05T11:29:06,107 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_35_01_000001_0 2018-01-05T11:29:06,107 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_35_01_000001_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:06,110 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_35_01_000001_0; false 2018-01-05T11:29:06,110 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_35_01_000001_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:06,110 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:06,110 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:06,110 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_35_01_000001, startTime=1515180545840, finishTime=1515180546110, timeTaken=270, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_35_01_000001_0, diagnostics=, counters=Counters: 42, File System Counters, FILE_BYTES_READ=208, FILE_BYTES_WRITTEN=1766, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=81, REDUCE_INPUT_RECORDS=138, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=139, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=703, OUTPUT_BYTES_WITH_OVERHEAD=713, OUTPUT_BYTES_PHYSICAL=717, ADDITIONAL_SPILLS_BYTES_WRITTEN=2014, ADDITIONAL_SPILLS_BYTES_READ=2014, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=2014, SHUFFLE_BYTES_DECOMPRESSED=2010, SHUFFLE_BYTES_TO_MEM=2014, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=14, MERGE_PHASE_TIME=38, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, HIVE, CREATED_FILES=1, RECORDS_OUT_1_default.tab_part=138, RECORDS_OUT_INTERMEDIATE_Reducer_2=1, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0 2018-01-05T11:29:06,110 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_35_01_000002_0, creationTime=1515180545836, allocationTime=1515180545838, startTime=1515180545842, finishTime=1515180546110, timeTaken=268, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:06,110 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_35_01_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:06,110 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_35_01_000000_0; false 2018-01-05T11:29:06,110 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_35_01_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:06,110 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:06,110 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_35_01_000000_0 2018-01-05T11:29:06,110 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:06,111 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_35_01_000000, startTime=1515180545839, finishTime=1515180546110, timeTaken=271, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_35_01_000000_0, diagnostics=, counters=Counters: 42, File System Counters, FILE_BYTES_READ=208, FILE_BYTES_WRITTEN=1640, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=75, REDUCE_INPUT_RECORDS=119, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=120, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=652, OUTPUT_BYTES_WITH_OVERHEAD=662, OUTPUT_BYTES_PHYSICAL=666, ADDITIONAL_SPILLS_BYTES_WRITTEN=1762, ADDITIONAL_SPILLS_BYTES_READ=1762, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=1762, SHUFFLE_BYTES_DECOMPRESSED=1758, SHUFFLE_BYTES_TO_MEM=1762, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=36, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, HIVE, CREATED_FILES=1, RECORDS_OUT_1_default.tab_part=119, RECORDS_OUT_INTERMEDIATE_Reducer_2=1, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0 2018-01-05T11:29:06,111 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_35_02 [Reducer 3] 2018-01-05T11:29:06,111 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_35_01 [Reducer 2], tasks=4, failed=0, killed=0, success=2, completed=2, commits=0, err=null 2018-01-05T11:29:06,111 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000073, stoppedTime=1515180546111, exitStatus=0 2018-01-05T11:29:06,111 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000073 exited with diagnostics set to Container completed. 2018-01-05T11:29:06,111 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_35_01_000002_0 2018-01-05T11:29:06,111 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_35_01_000002_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:06,111 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_35_01_000002_0; false 2018-01-05T11:29:06,111 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_35_01_000002_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:06,111 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:06,111 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:06,111 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_35_01_000002, startTime=1515180545842, finishTime=1515180546111, timeTaken=269, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_35_01_000002_0, diagnostics=, counters=Counters: 42, File System Counters, FILE_BYTES_READ=208, FILE_BYTES_WRITTEN=1747, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=80, REDUCE_INPUT_RECORDS=128, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=129, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=688, OUTPUT_BYTES_WITH_OVERHEAD=698, OUTPUT_BYTES_PHYSICAL=702, ADDITIONAL_SPILLS_BYTES_WRITTEN=1887, ADDITIONAL_SPILLS_BYTES_READ=1887, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=1887, SHUFFLE_BYTES_DECOMPRESSED=1883, SHUFFLE_BYTES_TO_MEM=1887, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=28, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, HIVE, CREATED_FILES=1, RECORDS_OUT_1_default.tab_part=128, RECORDS_OUT_INTERMEDIATE_Reducer_2=1, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0 2018-01-05T11:29:06,111 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_35_01 [Reducer 2], tasks=4, failed=0, killed=0, success=3, completed=3, commits=0, err=null 2018-01-05T11:29:06,111 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000072, stoppedTime=1515180546111, exitStatus=0 2018-01-05T11:29:06,111 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000072 exited with diagnostics set to Container completed. 2018-01-05T11:29:06,111 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_35_02 [Reducer 3] attempt: attempt_1515180518813_0001_35_01_000001_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:29:06,112 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000073 2018-01-05T11:29:06,112 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_35_01 [Reducer 2], tasks=4, failed=0, killed=0, success=4, completed=4, commits=0, err=null 2018-01-05T11:29:06,112 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Scheduling 1 tasks for vertex: Reducer 3 with totalTasks: 4. 2 source tasks completed out of 4. MinSourceTaskCompletedFraction: 0.5 min: 0.05 max: 0.75 2018-01-05T11:29:06,112 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_35_01 [Reducer 2] 2018-01-05T11:29:06,112 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:VERTEX_FINISHED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_35_01, initRequestedTime=1515180545661, initedTime=1515180545667, startRequestedTime=1515180545730, startedTime=1515180545730, finishTime=1515180546112, timeTaken=382, status=SUCCEEDED, diagnostics=, counters=Counters: 42, File System Counters, FILE_BYTES_READ=832, FILE_BYTES_WRITTEN=6801, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=309, REDUCE_INPUT_RECORDS=500, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=504, NUM_SHUFFLED_INPUTS=4, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=4, OUTPUT_RECORDS=4, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=2685, OUTPUT_BYTES_WITH_OVERHEAD=2725, OUTPUT_BYTES_PHYSICAL=2741, ADDITIONAL_SPILLS_BYTES_WRITTEN=7381, ADDITIONAL_SPILLS_BYTES_READ=7381, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=4, SHUFFLE_BYTES=7381, SHUFFLE_BYTES_DECOMPRESSED=7365, SHUFFLE_BYTES_TO_MEM=7381, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=41, MERGE_PHASE_TIME=133, FIRST_EVENT_RECEIVED=13, LAST_EVENT_RECEIVED=13, HIVE, CREATED_FILES=4, RECORDS_OUT_1_default.tab_part=500, RECORDS_OUT_INTERMEDIATE_Reducer_2=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, vertexStats=firstTaskStartTime=1515180545839, firstTasksToStart=[ task_1515180518813_0001_35_01_000000 ], lastTaskFinishTime=1515180546110, lastTasksToFinish=[ task_1515180518813_0001_35_01_000002 ], minTaskDuration=258, maxTaskDuration=268, avgTaskDuration=265.0, numSuccessfulTasks=4, shortestDurationTasks=[ task_1515180518813_0001_35_01_000003 ], longestDurationTasks=[ task_1515180518813_0001_35_01_000000,task_1515180518813_0001_35_01_000002 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=4, numSucceededTasks=4, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:06,112 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_35_01 [Reducer 2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:06,112 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000074, stoppedTime=1515180546112, exitStatus=0 2018-01-05T11:29:06,112 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000074 exited with diagnostics set to Container completed. 2018-01-05T11:29:06,112 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_35_02 [Reducer 3] attempt: attempt_1515180518813_0001_35_01_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:29:06,112 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000072 2018-01-05T11:29:06,112 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_35_02 [Reducer 3] attempt: attempt_1515180518813_0001_35_01_000002_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:29:06,112 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Scheduling 1 tasks for vertex: Reducer 3 with totalTasks: 4. 3 source tasks completed out of 4. MinSourceTaskCompletedFraction: 0.75 min: 0.05 max: 0.75 2018-01-05T11:29:06,112 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_35_02_000002_0, priority=32, capability=, hosts=null 2018-01-05T11:29:06,112 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_35_01 [Reducer 2] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3 2018-01-05T11:29:06,112 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null 2018-01-05T11:29:06,112 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=32,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:06,112 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:TASK_STARTED]: vertexName=Reducer 3, taskId=task_1515180518813_0001_35_02_000001, scheduledTime=1515180546112, launchTime=1515180546112 2018-01-05T11:29:06,112 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=32,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:06,112 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_35_02_000000_0, priority=32, capability=, hosts=null 2018-01-05T11:29:06,113 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: All source tasks completed. Ramping up 0 remaining tasks for vertex: Reducer 3 2018-01-05T11:29:06,113 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:06,113 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000074 2018-01-05T11:29:06,113 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_35_02_000002_0 on hosts=null 2018-01-05T11:29:06,113 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_35_02_000002_0 on hosts=null 2018-01-05T11:29:06,113 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:TASK_STARTED]: vertexName=Reducer 3, taskId=task_1515180518813_0001_35_02_000003, scheduledTime=1515180546113, launchTime=1515180546113 2018-01-05T11:29:06,113 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_35_02_000002_0 2018-01-05T11:29:06,113 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:06,113 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #76, task=TaskInfo{task=attempt_1515180518813_0001_35_02_000002_0, priority=32, startTime=0, containerId=null, uniqueId=75, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000076 2018-01-05T11:29:06,113 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_35_02_000002_0; false 2018-01-05T11:29:06,113 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_35_02_000002_0, priority=32, startTime=311566623, containerId=container_111101111_0001_00_000076assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=75, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:06,113 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=32,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:06,113 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_35_02_000001_0, priority=32, capability=, hosts=null 2018-01-05T11:29:06,113 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=32,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:06,113 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:06,113 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_35_02_000000_0 on hosts=null 2018-01-05T11:29:06,113 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_35_02_000000_0 on hosts=null 2018-01-05T11:29:06,113 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_35_02_000000_0 2018-01-05T11:29:06,113 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:06,113 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #77, task=TaskInfo{task=attempt_1515180518813_0001_35_02_000000_0, priority=32, startTime=0, containerId=null, uniqueId=76, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, to container=container_111101111_0001_00_000077 2018-01-05T11:29:06,113 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_35_02_000000_0; false 2018-01-05T11:29:06,114 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_35_02_000000_0, priority=32, startTime=311566623, containerId=container_111101111_0001_00_000077assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=2, ac=6, commF=false, disabled=false}, uniqueId=76, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:06,114 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=32,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:06,114 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=32,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:06,114 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:06,114 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_35_02_000003_0, priority=32, capability=, hosts=null 2018-01-05T11:29:06,114 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_35_02_000001_0 on hosts=null 2018-01-05T11:29:06,114 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_35_02_000001_0 on hosts=null 2018-01-05T11:29:06,114 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_35_02_000001_0 2018-01-05T11:29:06,114 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=2, ac=6, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:06,114 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #78, task=TaskInfo{task=attempt_1515180518813_0001_35_02_000001_0, priority=32, startTime=0, containerId=null, uniqueId=77, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=2, ac=6, commF=false, disabled=false}, to container=container_111101111_0001_00_000078 2018-01-05T11:29:06,114 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_35_02_000001_0; false 2018-01-05T11:29:06,114 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_35_02_000001_0, priority=32, startTime=311566623, containerId=container_111101111_0001_00_000078assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=3, ac=5, commF=false, disabled=false}, uniqueId=77, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:06,114 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=32,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:06,114 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=32,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:06,114 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:06,114 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_35_02_000003_0 on hosts=null 2018-01-05T11:29:06,114 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_35_02_000003_0 on hosts=null 2018-01-05T11:29:06,114 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_35_02_000003_0 2018-01-05T11:29:06,114 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=3, ac=5, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:06,114 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000076 for node: 10.22.8.182:49818 2018-01-05T11:29:06,114 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000076, attempt_1515180518813_0001_35_02_000002_0 for node: 10.22.8.182:49818 2018-01-05T11:29:06,114 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000076 for node: 10.22.8.182:49818 2018-01-05T11:29:06,114 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #79, task=TaskInfo{task=attempt_1515180518813_0001_35_02_000003_0, priority=32, startTime=0, containerId=null, uniqueId=78, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=3, ac=5, commF=false, disabled=false}, to container=container_111101111_0001_00_000079 2018-01-05T11:29:06,114 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_35_02_000003_0; false 2018-01-05T11:29:06,114 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_35_02_000003_0, priority=32, startTime=311566624, containerId=container_111101111_0001_00_000079assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=4, ac=4, commF=false, disabled=false}, uniqueId=78, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:06,115 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000077 for node: 10.22.8.182:49818 2018-01-05T11:29:06,115 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=35}, sourceName=Reducer 2, state=S_SUCCEEDED 2018-01-05T11:29:06,115 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000077, attempt_1515180518813_0001_35_02_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:06,115 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000077 for node: 10.22.8.182:49818 2018-01-05T11:29:06,115 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000078 for node: 10.22.8.182:49818 2018-01-05T11:29:06,115 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000078, attempt_1515180518813_0001_35_02_000001_0 for node: 10.22.8.182:49818 2018-01-05T11:29:06,115 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000078 for node: 10.22.8.182:49818 2018-01-05T11:29:06,115 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000076 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:06,115 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000076, launchTime=1515180546115 2018-01-05T11:29:06,115 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000079 for node: 10.22.8.182:49818 2018-01-05T11:29:06,115 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000079, attempt_1515180518813_0001_35_02_000003_0 for node: 10.22.8.182:49818 2018-01-05T11:29:06,115 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000079 for node: 10.22.8.182:49818 2018-01-05T11:29:06,115 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_35_02_000002_0, am_details=10.22.8.182:49824, taskInfo= fragment 2 attempt 0, user=djaiswal, queryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000076, dagName=insert overwrite tabl...cbucket_mapjoin_part (Stage-1), vertexName=Reducer 3, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Reducer 2,org.apache.tez.runtime.library.input.OrderedGroupedKVInput,4}}, Outputs={{out_Reducer 3,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=9, completedTaskCount=5, dagStartTime=1515180545651, firstAttemptStartTime=1515180546111, currentAttemptStartTime=1515180546114} 2018-01-05T11:29:06,115 INFO [IPC Server handler 4 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000076, QueryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a-35, DagId=35, VertexName=Reducer 3, TaskId=2, TaskAttemptId=0, SubmitTime=1515180546115 2018-01-05T11:29:06,115 DEBUG [IPC Server handler 4 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=35} with the ShuffleHandler 2018-01-05T11:29:06,115 DEBUG [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/35] 2018-01-05T11:29:06,116 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:29:06,116 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_35_02_000002_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:06,116 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_35_02_000002_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180546111, dagStartTime=1515180545651, withinDagPriority=32, vertexParallelism= 4, selfAndUpstreamParallelism= 9, selfAndUpstreamComplete= 5}] 2018-01-05T11:29:06,116 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_35_02_000002_0 : ACCEPTED 2018-01-05T11:29:06,116 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_35_02_000002_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:29:06,116 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_35_02_000002_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180546111, dagStartTime=1515180545651, withinDagPriority=32, vertexParallelism= 4, selfAndUpstreamParallelism= 9, selfAndUpstreamComplete= 5} 2018-01-05T11:29:06,116 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_35_02_000002_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:06,116 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_35_02_000002_0 to pre-emption queue 2018-01-05T11:29:06,116 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_35_02_000002_0: true 2018-01-05T11:29:06,116 INFO [TaskCommunicator # 4] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_35_02_000002_0 2018-01-05T11:29:06,116 INFO [TaskCommunicator # 4] GuaranteedTasks: Received response for attempt_1515180518813_0001_35_02_000002_0, true 2018-01-05T11:29:06,116 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:1205153640 2018-01-05T11:29:06,116 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_35_02_000002_0] submitted. Is using containerId: [container_111101111_0001_00_000076] on NM: [10.22.8.182:49818] 2018-01-05T11:29:06,116 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:06,116 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:06,116 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 3, taskAttemptId=attempt_1515180518813_0001_35_02_000002_0, startTime=1515180546115, containerId=container_111101111_0001_00_000076, nodeId=10.22.8.182:49818 2018-01-05T11:29:06,116 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000077 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:06,116 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000077, launchTime=1515180546116 2018-01-05T11:29:06,117 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_35_02_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000077, dagName=insert overwrite tabl...cbucket_mapjoin_part (Stage-1), vertexName=Reducer 3, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Reducer 2,org.apache.tez.runtime.library.input.OrderedGroupedKVInput,4}}, Outputs={{out_Reducer 3,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=9, completedTaskCount=5, dagStartTime=1515180545651, firstAttemptStartTime=1515180546111, currentAttemptStartTime=1515180546115} 2018-01-05T11:29:06,117 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : insert overwrite tabl...cbucket_mapjoin_part (Stage-1), VertexName: Reducer 3, VertexParallelism: 4, TaskAttemptID:attempt_1515180518813_0001_35_02_000002_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Reducer 2, physicalEdgeCount=4, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:29:06,117 INFO [IPC Server handler 3 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000077, QueryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a-35, DagId=35, VertexName=Reducer 3, TaskId=0, TaskAttemptId=0, SubmitTime=1515180546117 2018-01-05T11:29:06,117 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1028128768, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:06,117 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_35_02_000002_0 vertexName: Reducer 3 2018-01-05T11:29:06,117 DEBUG [IPC Server handler 3 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=35} with the ShuffleHandler 2018-01-05T11:29:06,117 DEBUG [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/35] 2018-01-05T11:29:06,117 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000076, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_35_02_000002_0, eventCount=1 } 2018-01-05T11:29:06,117 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_35_02_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000077] on NM: [10.22.8.182:49818] 2018-01-05T11:29:06,117 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:06,117 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:06,117 DEBUG [TezTR-518813_1_35_2_2_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_35 queryId: djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a instanceType: TaskRunner2Callable 2018-01-05T11:29:06,117 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 3, taskAttemptId=attempt_1515180518813_0001_35_02_000000_0, startTime=1515180546116, containerId=container_111101111_0001_00_000077, nodeId=10.22.8.182:49818 2018-01-05T11:29:06,117 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=3, runningFragmentCount=1 2018-01-05T11:29:06,117 INFO [TezTR-518813_1_35_2_2_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_35_02_000002_0 2018-01-05T11:29:06,117 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_35_02_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:06,117 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000078 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:06,117 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000078, launchTime=1515180546117 2018-01-05T11:29:06,117 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_35_02_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180546111, dagStartTime=1515180545651, withinDagPriority=32, vertexParallelism= 4, selfAndUpstreamParallelism= 9, selfAndUpstreamComplete= 5}] 2018-01-05T11:29:06,117 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_35_02_000000_0 : ACCEPTED 2018-01-05T11:29:06,117 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_35_02_000000_0, canFinish=true. Current state: preemptionQueueSize=1, numSlotsAvailable=3, waitQueueSize=1 2018-01-05T11:29:06,117 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_35_02_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180546111, dagStartTime=1515180545651, withinDagPriority=32, vertexParallelism= 4, selfAndUpstreamParallelism= 9, selfAndUpstreamComplete= 5} 2018-01-05T11:29:06,117 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_35_02_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:06,117 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_35_02_000000_0 to pre-emption queue 2018-01-05T11:29:06,117 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_35_02_000000_0: true 2018-01-05T11:29:06,117 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: taskOwner hashCode:746895039 2018-01-05T11:29:06,117 DEBUG [TezTR-518813_1_35_2_2_0] log.PerfLogger: 2018-01-05T11:29:06,117 INFO [TaskCommunicator # 5] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_35_02_000000_0 2018-01-05T11:29:06,117 INFO [TaskCommunicator # 5] GuaranteedTasks: Received response for attempt_1515180518813_0001_35_02_000000_0, true 2018-01-05T11:29:06,118 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_35_02_000001_0] submitted. Is using containerId: [container_111101111_0001_00_000078] on NM: [10.22.8.182:49818] 2018-01-05T11:29:06,118 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:06,118 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:06,118 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:06,118 INFO [I/O Setup 0 Initialize: {Reducer 2}] orderedgrouped.MergeManager: Initial Memory required for SHUFFLE_BUFFER=131404920 based on INPUT_BUFFER_FACTOR=0.7, for final merged output=0, using factor: 0.0 2018-01-05T11:29:06,118 INFO [Task-Executor-1] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : insert overwrite tabl...cbucket_mapjoin_part (Stage-1), VertexName: Reducer 3, VertexParallelism: 4, TaskAttemptID:attempt_1515180518813_0001_35_02_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Reducer 2, physicalEdgeCount=4, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:29:06,118 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 3, taskAttemptId=attempt_1515180518813_0001_35_02_000001_0, startTime=1515180546117, containerId=container_111101111_0001_00_000078, nodeId=10.22.8.182:49818 2018-01-05T11:29:06,118 INFO [Task-Executor-1] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1028128768, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:06,118 INFO [Task-Executor-1] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_35_02_000000_0 vertexName: Reducer 3 2018-01-05T11:29:06,118 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_35_02_000001_0, am_details=10.22.8.182:49824, taskInfo= fragment 1 attempt 0, user=djaiswal, queryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000078, dagName=insert overwrite tabl...cbucket_mapjoin_part (Stage-1), vertexName=Reducer 3, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Reducer 2,org.apache.tez.runtime.library.input.OrderedGroupedKVInput,4}}, Outputs={{out_Reducer 3,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=9, completedTaskCount=5, dagStartTime=1515180545651, firstAttemptStartTime=1515180546113, currentAttemptStartTime=1515180546115} 2018-01-05T11:29:06,118 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000079 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:06,118 INFO [IPC Server handler 1 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000078, QueryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a-35, DagId=35, VertexName=Reducer 3, TaskId=1, TaskAttemptId=0, SubmitTime=1515180546118 2018-01-05T11:29:06,118 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000079, launchTime=1515180546118 2018-01-05T11:29:06,118 DEBUG [IPC Server handler 1 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=35} with the ShuffleHandler 2018-01-05T11:29:06,118 DEBUG [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/35] 2018-01-05T11:29:06,118 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000077, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_35_02_000000_0, eventCount=1 } 2018-01-05T11:29:06,118 DEBUG [TezTR-518813_1_35_2_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_35 queryId: djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a instanceType: TaskRunner2Callable 2018-01-05T11:29:06,118 INFO [TezTR-518813_1_35_2_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_35_02_000000_0 2018-01-05T11:29:06,118 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:06,118 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=2, runningFragmentCount=2 2018-01-05T11:29:06,118 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_35_02_000001_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:06,119 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_35_02_000001_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180546113, dagStartTime=1515180545651, withinDagPriority=32, vertexParallelism= 4, selfAndUpstreamParallelism= 9, selfAndUpstreamComplete= 5}] 2018-01-05T11:29:06,119 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_35_02_000001_0 : ACCEPTED 2018-01-05T11:29:06,119 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_35_02_000003_0] submitted. Is using containerId: [container_111101111_0001_00_000079] on NM: [10.22.8.182:49818] 2018-01-05T11:29:06,119 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_35_02_000001_0, canFinish=true. Current state: preemptionQueueSize=2, numSlotsAvailable=2, waitQueueSize=1 2018-01-05T11:29:06,119 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:06,119 DEBUG [TezTR-518813_1_35_2_0_0] log.PerfLogger: 2018-01-05T11:29:06,119 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_35_02_000001_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180546113, dagStartTime=1515180545651, withinDagPriority=32, vertexParallelism= 4, selfAndUpstreamParallelism= 9, selfAndUpstreamComplete= 5} 2018-01-05T11:29:06,119 INFO [TaskCommunicator # 6] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_35_02_000001_0 2018-01-05T11:29:06,119 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_35_02_000001_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:06,119 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_35_02_000001_0 to pre-emption queue 2018-01-05T11:29:06,119 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_35_02_000001_0: true 2018-01-05T11:29:06,119 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:06,119 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:06,119 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 3, taskAttemptId=attempt_1515180518813_0001_35_02_000003_0, startTime=1515180546118, containerId=container_111101111_0001_00_000079, nodeId=10.22.8.182:49818 2018-01-05T11:29:06,119 INFO [TaskCommunicator # 6] GuaranteedTasks: Received response for attempt_1515180518813_0001_35_02_000001_0, true 2018-01-05T11:29:06,119 INFO [I/O Setup 0 Initialize: {Reducer 2}] orderedgrouped.MergeManager: Initial Memory required for SHUFFLE_BUFFER=131404920 based on INPUT_BUFFER_FACTOR=0.7, for final merged output=0, using factor: 0.0 2018-01-05T11:29:06,119 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: taskOwner hashCode:1719074333 2018-01-05T11:29:06,137 INFO [Task-Executor-3] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : insert overwrite tabl...cbucket_mapjoin_part (Stage-1), VertexName: Reducer 3, VertexParallelism: 4, TaskAttemptID:attempt_1515180518813_0001_35_02_000001_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Reducer 2, physicalEdgeCount=4, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:29:06,137 INFO [Task-Executor-3] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1026555904, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:06,137 INFO [Task-Executor-3] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_35_02_000001_0 vertexName: Reducer 3 2018-01-05T11:29:06,137 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000078, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_35_02_000001_0, eventCount=1 } 2018-01-05T11:29:06,138 INFO [IPC Server handler 9 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_35_02_000002_0 4 events [0,4) total 4 vertex_1515180518813_0001_35_02 [Reducer 3] 2018-01-05T11:29:06,138 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:06,138 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_35_02_000003_0, am_details=10.22.8.182:49824, taskInfo= fragment 3 attempt 0, user=djaiswal, queryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000079, dagName=insert overwrite tabl...cbucket_mapjoin_part (Stage-1), vertexName=Reducer 3, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Reducer 2,org.apache.tez.runtime.library.input.OrderedGroupedKVInput,4}}, Outputs={{out_Reducer 3,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=9, completedTaskCount=5, dagStartTime=1515180545651, firstAttemptStartTime=1515180546113, currentAttemptStartTime=1515180546115} 2018-01-05T11:29:06,138 INFO [IPC Server handler 0 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000079, QueryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a-35, DagId=35, VertexName=Reducer 3, TaskId=3, TaskAttemptId=0, SubmitTime=1515180546138 2018-01-05T11:29:06,138 DEBUG [IPC Server handler 0 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=35} with the ShuffleHandler 2018-01-05T11:29:06,138 DEBUG [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/35] 2018-01-05T11:29:06,138 DEBUG [TezTR-518813_1_35_2_1_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_35 queryId: djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a instanceType: TaskRunner2Callable 2018-01-05T11:29:06,138 INFO [TezTR-518813_1_35_2_1_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_35_02_000001_0 2018-01-05T11:29:06,138 DEBUG [TezTR-518813_1_35_2_2_0] log.PerfLogger: 2018-01-05T11:29:06,138 INFO [TezTR-518813_1_35_2_2_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:06,138 INFO [TezTR-518813_1_35_2_2_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:06,138 INFO [TezTR-518813_1_35_2_2_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:06,138 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=1, runningFragmentCount=3 2018-01-05T11:29:06,138 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_35_02_000003_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:06,138 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=4, nextPreRoutedEventId=0, eventCount=4 } 2018-01-05T11:29:06,138 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_35_02_000002_0, eventCount=4 fromEventId=0 nextFromEventId=4 2018-01-05T11:29:06,138 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_35_02_000003_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180546113, dagStartTime=1515180545651, withinDagPriority=32, vertexParallelism= 4, selfAndUpstreamParallelism= 9, selfAndUpstreamComplete= 5}] 2018-01-05T11:29:06,138 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_35_02_000003_0 : ACCEPTED 2018-01-05T11:29:06,139 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_35_02_000003_0, canFinish=true. Current state: preemptionQueueSize=3, numSlotsAvailable=1, waitQueueSize=1 2018-01-05T11:29:06,139 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_35_02_000003_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180546113, dagStartTime=1515180545651, withinDagPriority=32, vertexParallelism= 4, selfAndUpstreamParallelism= 9, selfAndUpstreamComplete= 5} 2018-01-05T11:29:06,139 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_35_02_000003_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:06,139 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_35_02_000003_0 to pre-emption queue 2018-01-05T11:29:06,139 DEBUG [TezTR-518813_1_35_2_1_0] log.PerfLogger: 2018-01-05T11:29:06,139 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_35_02_000003_0: true 2018-01-05T11:29:06,139 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: taskOwner hashCode:147702357 2018-01-05T11:29:06,139 INFO [I/O Setup 1 Initialize: {out_Reducer 3}] output.MROutput: out_Reducer 3: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:29:06,139 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:06,139 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:06,139 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:06,139 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:06,139 INFO [IPC Server handler 10 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_35_02_000000_0 4 events [0,4) total 4 vertex_1515180518813_0001_35_02 [Reducer 3] 2018-01-05T11:29:06,139 INFO [I/O Setup 0 Initialize: {Reducer 2}] orderedgrouped.MergeManager: Initial Memory required for SHUFFLE_BUFFER=131404920 based on INPUT_BUFFER_FACTOR=0.7, for final merged output=0, using factor: 0.0 2018-01-05T11:29:06,139 INFO [TezTR-518813_1_35_2_2_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:06,139 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:06,139 INFO [TezTR-518813_1_35_2_2_0] resources.MemoryDistributor: InitialRequests=[out_Reducer 3:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [Reducer 2:INPUT:131404920:org.apache.tez.runtime.library.input.OrderedGroupedKVInput] 2018-01-05T11:29:06,139 INFO [TaskCommunicator # 7] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_35_02_000003_0 2018-01-05T11:29:06,139 INFO [TaskCommunicator # 7] GuaranteedTasks: Received response for attempt_1515180518813_0001_35_02_000003_0, true 2018-01-05T11:29:06,139 INFO [TezTR-518813_1_35_2_2_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:06,139 INFO [TezTR-518813_1_35_2_2_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:06,139 INFO [TezTR-518813_1_35_2_2_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 131404920, TotalRequestedScaled: 1.2129684923076923E8, TotalJVMHeap: 1026555904, TotalAvailable: 688846274, TotalRequested/TotalJVMHeap:0.13 2018-01-05T11:29:06,139 INFO [TezTR-518813_1_35_2_2_0] resources.MemoryDistributor: Allocations=[out_Reducer 3:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:131404920:131404920] 2018-01-05T11:29:06,139 INFO [TezTR-518813_1_35_2_2_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:06,139 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=4, nextPreRoutedEventId=0, eventCount=4 } 2018-01-05T11:29:06,140 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_35_02_000000_0, eventCount=4 fromEventId=0 nextFromEventId=4 2018-01-05T11:29:06,140 INFO [Task-Executor-2] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : insert overwrite tabl...cbucket_mapjoin_part (Stage-1), VertexName: Reducer 3, VertexParallelism: 4, TaskAttemptID:attempt_1515180518813_0001_35_02_000003_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Reducer 2, physicalEdgeCount=4, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:29:06,140 INFO [TezTR-518813_1_35_2_2_0] runtime.LogicalIOProcessorRuntimeTask: Input: Reducer 2 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:06,140 INFO [TezTR-518813_1_35_2_2_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:29:06,140 INFO [TezTR-518813_1_35_2_2_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:06,140 INFO [Task-Executor-2] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1026555904, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:06,140 INFO [Task-Executor-2] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_35_02_000003_0 vertexName: Reducer 3 2018-01-05T11:29:06,140 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:06,140 DEBUG [TezTR-518813_1_35_2_0_0] log.PerfLogger: 2018-01-05T11:29:06,140 INFO [TezTR-518813_1_35_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:06,140 INFO [TezTR-518813_1_35_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:06,140 INFO [TezTR-518813_1_35_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:06,141 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000079, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_35_02_000003_0, eventCount=1 } 2018-01-05T11:29:06,141 INFO [I/O Setup 0 Start: {Reducer 2}] orderedgrouped.Shuffle: Reducer_2: Shuffle assigned with 4 inputs, codec: None, ifileReadAhead: true 2018-01-05T11:29:06,141 INFO [I/O Setup 0 Start: {Reducer 2}] orderedgrouped.MergeManager: Reducer 2: MergerManager: memoryLimit=131404920, maxSingleShuffleLimit=32851230, mergeThreshold=86727248, ioSortFactor=10, postMergeMem=0, memToMemMergeOutputsThreshold=10 2018-01-05T11:29:06,141 INFO [I/O Setup 1 Initialize: {out_Reducer 3}] output.MROutput: out_Reducer 3: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:29:06,141 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:06,141 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:06,141 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:06,141 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:06,141 DEBUG [TezTR-518813_1_35_2_3_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_35 queryId: djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a instanceType: TaskRunner2Callable 2018-01-05T11:29:06,141 INFO [TezTR-518813_1_35_2_3_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_35_02_000003_0 2018-01-05T11:29:06,141 INFO [I/O Setup 0 Start: {Reducer 2}] orderedgrouped.ShuffleScheduler: ShuffleScheduler running for sourceVertex: Reducer 2 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=4, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=4, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true 2018-01-05T11:29:06,141 INFO [IPC Server handler 26 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_35_02_000001_0 4 events [0,4) total 4 vertex_1515180518813_0001_35_02 [Reducer 3] 2018-01-05T11:29:06,141 INFO [TezTR-518813_1_35_2_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:06,141 INFO [TezTR-518813_1_35_2_0_0] resources.MemoryDistributor: InitialRequests=[out_Reducer 3:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [Reducer 2:INPUT:131404920:org.apache.tez.runtime.library.input.OrderedGroupedKVInput] 2018-01-05T11:29:06,141 INFO [TezTR-518813_1_35_2_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:06,141 INFO [TezTR-518813_1_35_2_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:06,142 INFO [I/O Setup 0 Start: {Reducer 2}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: Reducer 2 2018-01-05T11:29:06,142 INFO [TezTR-518813_1_35_2_2_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:06,142 DEBUG [TezTR-518813_1_35_2_3_0] log.PerfLogger: 2018-01-05T11:29:06,143 INFO [I/O Setup 0 Initialize: {Reducer 2}] orderedgrouped.MergeManager: Initial Memory required for SHUFFLE_BUFFER=131404920 based on INPUT_BUFFER_FACTOR=0.7, for final merged output=0, using factor: 0.0 2018-01-05T11:29:06,143 INFO [TezTR-518813_1_35_2_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 131404920, TotalRequestedScaled: 1.2129684923076923E8, TotalJVMHeap: 1026555904, TotalAvailable: 688846274, TotalRequested/TotalJVMHeap:0.13 2018-01-05T11:29:06,143 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:06,143 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=4, nextPreRoutedEventId=0, eventCount=4 } 2018-01-05T11:29:06,143 INFO [TezTR-518813_1_35_2_0_0] resources.MemoryDistributor: Allocations=[out_Reducer 3:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:131404920:131404920] 2018-01-05T11:29:06,143 INFO [TezTR-518813_1_35_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:06,143 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_35_02_000001_0, eventCount=4 fromEventId=0 nextFromEventId=4 2018-01-05T11:29:06,143 INFO [TezTR-518813_1_35_2_2_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_35_02_000002_0 2018-01-05T11:29:06,143 DEBUG [TezTR-518813_1_35_2_2_0] log.PerfLogger: 2018-01-05T11:29:06,143 DEBUG [TezTR-518813_1_35_2_2_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_35_02_000002_0_10229 2018-01-05T11:29:06,143 DEBUG [TezTR-518813_1_35_2_2_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:06,143 INFO [TezTR-518813_1_35_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: Reducer 2 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:06,144 INFO [TezTR-518813_1_35_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:29:06,144 INFO [TezTR-518813_1_35_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:06,144 INFO [I/O Setup 0 Start: {Reducer 2}] orderedgrouped.Shuffle: Reducer_2: Shuffle assigned with 4 inputs, codec: None, ifileReadAhead: true 2018-01-05T11:29:06,144 INFO [TezTaskEventRouter{attempt_1515180518813_0001_35_02_000002_0}] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Reducer 2: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:29:06,143 DEBUG [TezTR-518813_1_35_2_2_0] mr.ObjectCache: Creating djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a_Reducer 3__REDUCE_PLAN__ 2018-01-05T11:29:06,145 INFO [TezTR-518813_1_35_2_2_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/233a4fcb-aae6-42b5-96e2-24086b11320a/reduce.xml 2018-01-05T11:29:06,145 DEBUG [TezTR-518813_1_35_2_2_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/233a4fcb-aae6-42b5-96e2-24086b11320a/reduce.xml 2018-01-05T11:29:06,145 DEBUG [TezTR-518813_1_35_2_2_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/233a4fcb-aae6-42b5-96e2-24086b11320a/reduce.xml 2018-01-05T11:29:06,145 DEBUG [TezTR-518813_1_35_2_2_0] log.PerfLogger: 2018-01-05T11:29:06,145 INFO [TezTR-518813_1_35_2_2_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:29:06,145 DEBUG [TezTR-518813_1_35_2_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,144 INFO [I/O Setup 0 Start: {Reducer 2}] orderedgrouped.MergeManager: Reducer 2: MergerManager: memoryLimit=131404920, maxSingleShuffleLimit=32851230, mergeThreshold=86727248, ioSortFactor=10, postMergeMem=0, memToMemMergeOutputsThreshold=10 2018-01-05T11:29:06,145 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:06,145 DEBUG [TezTR-518813_1_35_2_1_0] log.PerfLogger: 2018-01-05T11:29:06,145 INFO [TezTR-518813_1_35_2_1_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:06,145 INFO [TezTR-518813_1_35_2_1_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:06,145 INFO [TezTR-518813_1_35_2_1_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:06,145 INFO [I/O Setup 0 Start: {Reducer 2}] orderedgrouped.ShuffleScheduler: ShuffleScheduler running for sourceVertex: Reducer 2 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=4, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=4, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true 2018-01-05T11:29:06,145 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=35&reduce=2&map=attempt_1515180518813_0001_35_01_000003_0_10228,attempt_1515180518813_0001_35_01_000001_0_10222 mapId: [attempt_1515180518813_0001_35_01_000003_0_10228, attempt_1515180518813_0001_35_01_000001_0_10222] reduceId: [2] jobId: [job_1515180518813_0001] dagId: [35] keepAlive: false 2018-01-05T11:29:06,145 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=35&reduce=2&map=attempt_1515180518813_0001_35_01_000003_0_10228,attempt_1515180518813_0001_35_01_000001_0_10222; hash=...9zs9s/eWjbbxg 2018-01-05T11:29:06,145 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=35&reduce=2&map=attempt_1515180518813_0001_35_01_000003_0_10228,attempt_1515180518813_0001_35_01_000001_0_10222;reply=DV6jx1UZ97D5Y 2018-01-05T11:29:06,147 INFO [IPC Server handler 6 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_35_02_000003_0 4 events [0,4) total 4 vertex_1515180518813_0001_35_02 [Reducer 3] 2018-01-05T11:29:06,147 INFO [I/O Setup 0 Start: {Reducer 2}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: Reducer 2 2018-01-05T11:29:06,147 INFO [I/O Setup 1 Initialize: {out_Reducer 3}] output.MROutput: out_Reducer 3: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:29:06,148 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:06,148 INFO [TezTR-518813_1_35_2_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:06,148 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:06,148 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:06,148 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:06,148 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=35, user='djaiswal', attemptId='attempt_1515180518813_0001_35_01_000003_0_10228'} via loader 2018-01-05T11:29:06,149 INFO [TezTR-518813_1_35_2_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_35_02_000000_0 2018-01-05T11:29:06,149 DEBUG [TezTR-518813_1_35_2_0_0] log.PerfLogger: 2018-01-05T11:29:06,149 DEBUG [TezTR-518813_1_35_2_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_35_02_000000_0_10232 2018-01-05T11:29:06,149 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=35, user='djaiswal', attemptId='attempt_1515180518813_0001_35_01_000003_0_10228'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:29:06,149 DEBUG [TezTR-518813_1_35_2_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:06,149 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=4, nextPreRoutedEventId=0, eventCount=4 } 2018-01-05T11:29:06,149 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_35_01_000003_0_10228 not found 2018-01-05T11:29:06,149 INFO [TezTR-518813_1_35_2_1_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:06,150 INFO [TezTaskEventRouter{attempt_1515180518813_0001_35_02_000000_0}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 4. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:06,150 DEBUG [TezTR-518813_1_35_2_0_0] mr.ObjectCache: Creating djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a_Reducer 3__REDUCE_PLAN__ 2018-01-05T11:29:06,150 INFO [TezTR-518813_1_35_2_1_0] resources.MemoryDistributor: InitialRequests=[out_Reducer 3:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [Reducer 2:INPUT:131404920:org.apache.tez.runtime.library.input.OrderedGroupedKVInput] 2018-01-05T11:29:06,150 INFO [TezTaskEventRouter{attempt_1515180518813_0001_35_02_000000_0}] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Reducer 2: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=1, numObsoletionEventsSeen=0 2018-01-05T11:29:06,150 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_35_02_000003_0, eventCount=4 fromEventId=0 nextFromEventId=4 2018-01-05T11:29:06,150 INFO [TezTR-518813_1_35_2_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/233a4fcb-aae6-42b5-96e2-24086b11320a/reduce.xml 2018-01-05T11:29:06,150 INFO [TezTR-518813_1_35_2_1_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:06,150 DEBUG [TezTR-518813_1_35_2_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/233a4fcb-aae6-42b5-96e2-24086b11320a/reduce.xml 2018-01-05T11:29:06,150 DEBUG [TezTR-518813_1_35_2_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/233a4fcb-aae6-42b5-96e2-24086b11320a/reduce.xml 2018-01-05T11:29:06,150 INFO [TezTR-518813_1_35_2_1_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:06,150 DEBUG [TezTR-518813_1_35_2_0_0] log.PerfLogger: 2018-01-05T11:29:06,150 INFO [TezTR-518813_1_35_2_1_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 131404920, TotalRequestedScaled: 1.2129684923076923E8, TotalJVMHeap: 1026555904, TotalAvailable: 687792455, TotalRequested/TotalJVMHeap:0.13 2018-01-05T11:29:06,150 INFO [TezTaskEventRouter{attempt_1515180518813_0001_35_02_000000_0}] orderedgrouped.ShuffleScheduler: All inputs fetched for input vertex : Reducer 2 2018-01-05T11:29:06,150 INFO [TezTR-518813_1_35_2_0_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:29:06,150 INFO [TezTaskEventRouter{attempt_1515180518813_0001_35_02_000000_0}] orderedgrouped.ShuffleScheduler: copy(4 (spillsFetched=4) of 4. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:06,150 INFO [TezTR-518813_1_35_2_1_0] resources.MemoryDistributor: Allocations=[out_Reducer 3:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:131404920:131404920] 2018-01-05T11:29:06,150 DEBUG [TezTR-518813_1_35_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,150 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_35_01_000003_0_10228,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/35/output/attempt_1515180518813_0001_35_01_000003_0_10228/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/35/output/attempt_1515180518813_0001_35_01_000003_0_10228/file.out.index 2018-01-05T11:29:06,150 INFO [TezTR-518813_1_35_2_1_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:06,150 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.ShuffleScheduler: Shutting down FetchScheduler for input: Reducer_2, wasInterrupted=false 2018-01-05T11:29:06,150 INFO [TezTR-518813_1_35_2_1_0] runtime.LogicalIOProcessorRuntimeTask: Input: Reducer 2 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:06,150 INFO [TezTR-518813_1_35_2_1_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:29:06,151 INFO [TezTR-518813_1_35_2_1_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:06,151 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.ShuffleScheduler: copy(4 (spillsFetched=4) of 4. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:06,151 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.ShuffleScheduler: Shutting down fetchers for input: Reducer_2, shutdown timetaken: 0 ms, hasFetcherExecutorStopped: true 2018-01-05T11:29:06,151 INFO [I/O Setup 0 Start: {Reducer 2}] orderedgrouped.Shuffle: Reducer_2: Shuffle assigned with 4 inputs, codec: None, ifileReadAhead: true 2018-01-05T11:29:06,151 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.MergeManager: finalMerge with #inMemoryOutputs=0, size=0 and #onDiskOutputs=0, size=0 2018-01-05T11:29:06,151 INFO [I/O Setup 0 Start: {Reducer 2}] orderedgrouped.MergeManager: Reducer 2: MergerManager: memoryLimit=131404920, maxSingleShuffleLimit=32851230, mergeThreshold=86727248, ioSortFactor=10, postMergeMem=0, memToMemMergeOutputsThreshold=10 2018-01-05T11:29:06,151 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.MergeManager: . DiskSeg: 0, 0. MemSeg: 0, 0 2018-01-05T11:29:06,151 INFO [ShuffleAndMergeRunner {Reducer_2}] impl.TezMerger: Nothing to merge. Returning an empty iterator 2018-01-05T11:29:06,152 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.Shuffle: merge complete for input vertex : Reducer_2 2018-01-05T11:29:06,152 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.Shuffle: Reducer_2: Shuffle Runner thread complete 2018-01-05T11:29:06,152 DEBUG [TezTR-518813_1_35_2_2_0] log.PerfLogger: 2018-01-05T11:29:06,152 INFO [TezTR-518813_1_35_2_2_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 3 size: 2.66KB 2018-01-05T11:29:06,152 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=35, user='djaiswal', attemptId='attempt_1515180518813_0001_35_01_000001_0_10222'} via loader 2018-01-05T11:29:06,152 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=35, user='djaiswal', attemptId='attempt_1515180518813_0001_35_01_000001_0_10222'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:29:06,152 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_35_01_000001_0_10222 not found 2018-01-05T11:29:06,152 INFO [I/O Setup 0 Start: {Reducer 2}] orderedgrouped.ShuffleScheduler: ShuffleScheduler running for sourceVertex: Reducer 2 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=4, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=4, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true 2018-01-05T11:29:06,153 DEBUG [TezTR-518813_1_35_2_2_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:06,153 DEBUG [TezTR-518813_1_35_2_2_0] log.PerfLogger: 2018-01-05T11:29:06,153 DEBUG [TezTR-518813_1_35_2_2_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:06,153 DEBUG [TezTR-518813_1_35_2_2_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:06,153 DEBUG [TezTR-518813_1_35_2_0_0] log.PerfLogger: 2018-01-05T11:29:06,153 INFO [I/O Setup 0 Start: {Reducer 2}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: Reducer 2 2018-01-05T11:29:06,153 INFO [TezTR-518813_1_35_2_1_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:06,153 INFO [TezTR-518813_1_35_2_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 3 size: 2.66KB 2018-01-05T11:29:06,153 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_35_01_000001_0_10222,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/35/output/attempt_1515180518813_0001_35_01_000001_0_10222/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/35/output/attempt_1515180518813_0001_35_01_000001_0_10222/file.out.index 2018-01-05T11:29:06,153 INFO [TezTR-518813_1_35_2_1_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_35_02_000001_0 2018-01-05T11:29:06,153 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Content Length in shuffle : 1483 2018-01-05T11:29:06,153 DEBUG [TezTR-518813_1_35_2_1_0] log.PerfLogger: 2018-01-05T11:29:06,153 DEBUG [TezTR-518813_1_35_2_1_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_35_02_000001_0_10235 2018-01-05T11:29:06,153 DEBUG [TezTR-518813_1_35_2_1_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:06,153 DEBUG [TezTR-518813_1_35_2_1_0] mr.ObjectCache: Creating djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a_Reducer 3__REDUCE_PLAN__ 2018-01-05T11:29:06,154 INFO [TezTR-518813_1_35_2_1_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/233a4fcb-aae6-42b5-96e2-24086b11320a/reduce.xml 2018-01-05T11:29:06,154 DEBUG [TezTR-518813_1_35_2_1_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/233a4fcb-aae6-42b5-96e2-24086b11320a/reduce.xml 2018-01-05T11:29:06,154 DEBUG [TezTR-518813_1_35_2_1_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/233a4fcb-aae6-42b5-96e2-24086b11320a/reduce.xml 2018-01-05T11:29:06,153 DEBUG [TezTR-518813_1_35_2_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:06,154 DEBUG [TezTR-518813_1_35_2_0_0] log.PerfLogger: 2018-01-05T11:29:06,154 DEBUG [TezTR-518813_1_35_2_1_0] log.PerfLogger: 2018-01-05T11:29:06,154 INFO [TezTR-518813_1_35_2_1_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:29:06,154 DEBUG [TezTR-518813_1_35_2_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,154 DEBUG [TezTR-518813_1_35_2_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:06,154 INFO [TezTaskEventRouter{attempt_1515180518813_0001_35_02_000001_0}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 4. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:06,154 DEBUG [TezTR-518813_1_35_2_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:06,154 INFO [TezTaskEventRouter{attempt_1515180518813_0001_35_02_000001_0}] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Reducer 2: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=1, numObsoletionEventsSeen=0 2018-01-05T11:29:06,154 INFO [Fetcher_O {Reducer_2} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=35&reduce=2&map=attempt_1515180518813_0001_35_01_000003_0_10228,attempt_1515180518813_0001_35_01_000001_0_10222 sent hash and receievd reply 0 ms 2018-01-05T11:29:06,154 DEBUG [TezTR-518813_1_35_2_2_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:06,154 INFO [TezTR-518813_1_35_2_2_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:29:06,154 DEBUG [New I/O worker #1] shufflehandler.FadvisedFileRegion: shuffleBufferSize: 131072, path: /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/35/output/attempt_1515180518813_0001_35_01_000003_0_10228/file.out 2018-01-05T11:29:06,155 INFO [Fetcher_O {Reducer_2} #0] orderedgrouped.MergeManager: CloseInMemoryFile. Current state: inMemoryMapOutputs.size=1, commitMemory=0, usedMemory=652. Since last log: count=1, min=652, max=652, total=652, avg=652.0 2018-01-05T11:29:06,155 INFO [TezTaskEventRouter{attempt_1515180518813_0001_35_02_000001_0}] orderedgrouped.ShuffleScheduler: All inputs fetched for input vertex : Reducer 2 2018-01-05T11:29:06,155 INFO [TezTaskEventRouter{attempt_1515180518813_0001_35_02_000001_0}] orderedgrouped.ShuffleScheduler: copy(4 (spillsFetched=4) of 4. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:06,155 INFO [Fetcher_O {Reducer_2} #0] ShuffleScheduler.fetch: Completed fetch for attempt: {3, 0, attempt_1515180518813_0001_35_01_000003_0_10228} to MEMORY, csize=656, dsize=652, EndTime=1515180546155, TimeTaken=1, Rate=0.62 MB/s 2018-01-05T11:29:06,155 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.ShuffleScheduler: Shutting down FetchScheduler for input: Reducer_2, wasInterrupted=false 2018-01-05T11:29:06,155 INFO [Fetcher_O {Reducer_2} #0] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 4. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:06,155 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.ShuffleScheduler: copy(4 (spillsFetched=4) of 4. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:06,156 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.ShuffleScheduler: Shutting down fetchers for input: Reducer_2, shutdown timetaken: 0 ms, hasFetcherExecutorStopped: true 2018-01-05T11:29:06,156 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.MergeManager: finalMerge with #inMemoryOutputs=0, size=0 and #onDiskOutputs=0, size=0 2018-01-05T11:29:06,156 DEBUG [New I/O worker #1] shufflehandler.FadvisedFileRegion: shuffleBufferSize: 131072, path: /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/35/output/attempt_1515180518813_0001_35_01_000001_0_10222/file.out 2018-01-05T11:29:06,156 DEBUG [TezTR-518813_1_35_2_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:06,156 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.MergeManager: . DiskSeg: 0, 0. MemSeg: 0, 0 2018-01-05T11:29:06,156 INFO [TezTR-518813_1_35_2_0_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:29:06,156 INFO [ShuffleAndMergeRunner {Reducer_2}] impl.TezMerger: Nothing to merge. Returning an empty iterator 2018-01-05T11:29:06,156 DEBUG [TezTR-518813_1_35_2_3_0] log.PerfLogger: 2018-01-05T11:29:06,156 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.Shuffle: merge complete for input vertex : Reducer_2 2018-01-05T11:29:06,156 INFO [TezTR-518813_1_35_2_3_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:06,156 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.Shuffle: Reducer_2: Shuffle Runner thread complete 2018-01-05T11:29:06,156 INFO [TezTR-518813_1_35_2_3_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:06,156 INFO [TezTR-518813_1_35_2_0_0] tez.ReduceRecordProcessor: Main work is Reducer 3 2018-01-05T11:29:06,156 INFO [TezTR-518813_1_35_2_3_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:06,156 INFO [Fetcher_O {Reducer_2} #0] ShuffleScheduler.fetch: Completed fetch for attempt: {1, 0, attempt_1515180518813_0001_35_01_000001_0_10222} to MEMORY, csize=717, dsize=713, EndTime=1515180546156, TimeTaken=1, Rate=0.68 MB/s 2018-01-05T11:29:06,156 INFO [TezTR-518813_1_35_2_0_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:06,156 DEBUG [TezTR-518813_1_35_2_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:06,156 INFO [TezTR-518813_1_35_2_0_0] input.OrderedGroupedKVInput: Reducer 2: creating ValuesIterator with comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator, keyClass=org.apache.hadoop.hive.ql.io.HiveKey, valClass=org.apache.hadoop.io.BytesWritable 2018-01-05T11:29:06,156 DEBUG [TezTR-518813_1_35_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,157 DEBUG [TezTR-518813_1_35_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,157 INFO [I/O Setup 1 Initialize: {out_Reducer 3}] output.MROutput: out_Reducer 3: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:29:06,157 DEBUG [TezTR-518813_1_35_2_1_0] log.PerfLogger: 2018-01-05T11:29:06,157 INFO [TezTR-518813_1_35_2_1_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 3 size: 2.66KB 2018-01-05T11:29:06,157 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:06,157 DEBUG [TezTR-518813_1_35_2_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:29:06,157 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:06,157 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:06,157 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:06,157 DEBUG [TezTR-518813_1_35_2_0_0] log.PerfLogger: 2018-01-05T11:29:06,157 INFO [TezTR-518813_1_35_2_0_0] exec.GroupByOperator: Initializing operator GBY[4] 2018-01-05T11:29:06,157 DEBUG [TezTR-518813_1_35_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,157 DEBUG [TezTR-518813_1_35_2_1_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:06,157 DEBUG [TezTR-518813_1_35_2_1_0] log.PerfLogger: 2018-01-05T11:29:06,157 INFO [TezTR-518813_1_35_2_3_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:06,157 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=35&reduce=2&map=attempt_1515180518813_0001_35_01_000000_0_10219,attempt_1515180518813_0001_35_01_000002_0_10225 mapId: [attempt_1515180518813_0001_35_01_000000_0_10219, attempt_1515180518813_0001_35_01_000002_0_10225] reduceId: [2] jobId: [job_1515180518813_0001] dagId: [35] keepAlive: false 2018-01-05T11:29:06,157 INFO [TezTR-518813_1_35_2_3_0] resources.MemoryDistributor: InitialRequests=[Reducer 2:INPUT:131404920:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [out_Reducer 3:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput] 2018-01-05T11:29:06,157 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=35&reduce=2&map=attempt_1515180518813_0001_35_01_000000_0_10219,attempt_1515180518813_0001_35_01_000002_0_10225; hash=...ZUVflGaFSiwZs 2018-01-05T11:29:06,157 DEBUG [TezTR-518813_1_35_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,157 DEBUG [TezTR-518813_1_35_2_1_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:06,157 INFO [TezTR-518813_1_35_2_3_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:06,157 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=35&reduce=2&map=attempt_1515180518813_0001_35_01_000000_0_10219,attempt_1515180518813_0001_35_01_000002_0_10225;reply=wXFkrj7aQa5p0 2018-01-05T11:29:06,157 DEBUG [TezTR-518813_1_35_2_1_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:06,157 INFO [TezTR-518813_1_35_2_3_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:06,157 INFO [TezTR-518813_1_35_2_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:29:06,157 DEBUG [TezTR-518813_1_35_2_0_0] exec.GroupByOperator: Initialization Done 4 GBY 2018-01-05T11:29:06,157 DEBUG [TezTR-518813_1_35_2_0_0] exec.GroupByOperator: Operator 4 GBY initialized 2018-01-05T11:29:06,157 DEBUG [TezTR-518813_1_35_2_0_0] exec.GroupByOperator: Initializing children of 4 GBY 2018-01-05T11:29:06,157 INFO [TezTR-518813_1_35_2_3_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 131404920, TotalRequestedScaled: 1.2129684923076923E8, TotalJVMHeap: 1026555904, TotalAvailable: 687792455, TotalRequested/TotalJVMHeap:0.13 2018-01-05T11:29:06,157 DEBUG [TezTR-518813_1_35_2_0_0] exec.SelectOperator: Initializing child 5 SEL 2018-01-05T11:29:06,157 INFO [TezTR-518813_1_35_2_0_0] exec.SelectOperator: Initializing operator SEL[5] 2018-01-05T11:29:06,157 INFO [TezTR-518813_1_35_2_3_0] resources.MemoryDistributor: Allocations=[Reducer 2:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:131404920:131404920], [out_Reducer 3:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0] 2018-01-05T11:29:06,157 INFO [TezTR-518813_1_35_2_3_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:06,157 INFO [TezTR-518813_1_35_2_0_0] exec.SelectOperator: SELECT struct<_col0:string,_col1:struct,_col2:struct> 2018-01-05T11:29:06,157 INFO [TezTR-518813_1_35_2_3_0] runtime.LogicalIOProcessorRuntimeTask: Input: Reducer 2 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:06,157 DEBUG [TezTR-518813_1_35_2_0_0] exec.SelectOperator: Initialization Done 5 SEL 2018-01-05T11:29:06,157 INFO [TezTR-518813_1_35_2_3_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:29:06,157 INFO [TezTR-518813_1_35_2_3_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:06,157 DEBUG [TezTR-518813_1_35_2_0_0] exec.SelectOperator: Operator 5 SEL initialized 2018-01-05T11:29:06,158 INFO [I/O Setup 0 Start: {Reducer 2}] orderedgrouped.Shuffle: Reducer_2: Shuffle assigned with 4 inputs, codec: None, ifileReadAhead: true 2018-01-05T11:29:06,158 DEBUG [TezTR-518813_1_35_2_0_0] exec.SelectOperator: Initializing children of 5 SEL 2018-01-05T11:29:06,158 DEBUG [TezTR-518813_1_35_2_0_0] exec.FileSinkOperator: Initializing child 6 FS 2018-01-05T11:29:06,158 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=35, user='djaiswal', attemptId='attempt_1515180518813_0001_35_01_000000_0_10219'} via loader 2018-01-05T11:29:06,158 INFO [I/O Setup 0 Start: {Reducer 2}] orderedgrouped.MergeManager: Reducer 2: MergerManager: memoryLimit=131404920, maxSingleShuffleLimit=32851230, mergeThreshold=86727248, ioSortFactor=10, postMergeMem=0, memToMemMergeOutputsThreshold=10 2018-01-05T11:29:06,158 INFO [TezTR-518813_1_35_2_0_0] exec.FileSinkOperator: Initializing operator FS[6] 2018-01-05T11:29:06,158 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=35, user='djaiswal', attemptId='attempt_1515180518813_0001_35_01_000000_0_10219'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:29:06,158 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_35_01_000000_0_10219 not found 2018-01-05T11:29:06,158 DEBUG [TezTR-518813_1_35_2_1_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:06,158 INFO [TezTR-518813_1_35_2_1_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:29:06,158 INFO [TezTR-518813_1_35_2_1_0] tez.ReduceRecordProcessor: Main work is Reducer 3 2018-01-05T11:29:06,158 INFO [TezTR-518813_1_35_2_1_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:06,158 DEBUG [TezTR-518813_1_35_2_1_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:06,158 DEBUG [TezTR-518813_1_35_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,158 INFO [TezTR-518813_1_35_2_1_0] input.OrderedGroupedKVInput: Reducer 2: creating ValuesIterator with comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator, keyClass=org.apache.hadoop.hive.ql.io.HiveKey, valClass=org.apache.hadoop.io.BytesWritable 2018-01-05T11:29:06,158 DEBUG [TezTR-518813_1_35_2_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,158 INFO [I/O Setup 0 Start: {Reducer 2}] orderedgrouped.ShuffleScheduler: ShuffleScheduler running for sourceVertex: Reducer 2 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=4, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=4, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true 2018-01-05T11:29:06,158 DEBUG [TezTR-518813_1_35_2_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,158 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_35_01_000000_0_10219,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/35/output/attempt_1515180518813_0001_35_01_000000_0_10219/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/35/output/attempt_1515180518813_0001_35_01_000000_0_10219/file.out.index 2018-01-05T11:29:06,158 INFO [TezTR-518813_1_35_2_0_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@7845448d]:[_col0, _col1, _col2]:[struct, struct, string]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@2e4c1024 2018-01-05T11:29:06,158 DEBUG [TezTR-518813_1_35_2_1_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:29:06,158 DEBUG [TezTR-518813_1_35_2_1_0] log.PerfLogger: 2018-01-05T11:29:06,158 INFO [TezTR-518813_1_35_2_1_0] exec.GroupByOperator: Initializing operator GBY[4] 2018-01-05T11:29:06,158 DEBUG [TezTR-518813_1_35_2_0_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:29:06,158 DEBUG [TezTR-518813_1_35_2_0_0] exec.FileSinkOperator: Initialization Done 6 FS 2018-01-05T11:29:06,158 DEBUG [TezTR-518813_1_35_2_0_0] exec.FileSinkOperator: Operator 6 FS initialized 2018-01-05T11:29:06,158 DEBUG [TezTR-518813_1_35_2_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,158 DEBUG [TezTR-518813_1_35_2_0_0] exec.FileSinkOperator: Initialization Done 6 FS done is reset. 2018-01-05T11:29:06,158 DEBUG [TezTR-518813_1_35_2_0_0] exec.SelectOperator: Initialization Done 5 SEL done is reset. 2018-01-05T11:29:06,158 DEBUG [TezTR-518813_1_35_2_0_0] exec.GroupByOperator: Initialization Done 4 GBY done is reset. 2018-01-05T11:29:06,158 DEBUG [TezTR-518813_1_35_2_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,159 INFO [TezTR-518813_1_35_2_0_0] tez.ReduceRecordProcessor: Id =4 Id =5 Id =6 <\Children> Id = 5 null<\Parent> <\FS> <\Children> Id = 4 null<\Parent> <\SEL> <\Children> <\Parent> <\GBY> 2018-01-05T11:29:06,159 INFO [TezTR-518813_1_35_2_1_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:29:06,160 DEBUG [TezTR-518813_1_35_2_1_0] exec.GroupByOperator: Initialization Done 4 GBY 2018-01-05T11:29:06,159 INFO [I/O Setup 0 Start: {Reducer 2}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: Reducer 2 2018-01-05T11:29:06,160 DEBUG [TezTR-518813_1_35_2_1_0] exec.GroupByOperator: Operator 4 GBY initialized 2018-01-05T11:29:06,160 DEBUG [TezTR-518813_1_35_2_0_0] log.PerfLogger: 2018-01-05T11:29:06,160 DEBUG [TezTR-518813_1_35_2_1_0] exec.GroupByOperator: Initializing children of 4 GBY 2018-01-05T11:29:06,160 DEBUG [TezTR-518813_1_35_2_1_0] exec.SelectOperator: Initializing child 5 SEL 2018-01-05T11:29:06,160 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=35, user='djaiswal', attemptId='attempt_1515180518813_0001_35_01_000002_0_10225'} via loader 2018-01-05T11:29:06,160 INFO [TezTR-518813_1_35_2_1_0] exec.SelectOperator: Initializing operator SEL[5] 2018-01-05T11:29:06,160 INFO [TezTR-518813_1_35_2_3_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:06,160 INFO [TezTR-518813_1_35_2_0_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 3 2018-01-05T11:29:06,160 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=35, user='djaiswal', attemptId='attempt_1515180518813_0001_35_01_000002_0_10225'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:29:06,160 DEBUG [TezTR-518813_1_35_2_0_0] log.PerfLogger: 2018-01-05T11:29:06,160 INFO [TezTR-518813_1_35_2_1_0] exec.SelectOperator: SELECT struct<_col0:string,_col1:struct,_col2:struct> 2018-01-05T11:29:06,160 INFO [TezTR-518813_1_35_2_3_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_35_02_000003_0 2018-01-05T11:29:06,160 DEBUG [TezTR-518813_1_35_2_1_0] exec.SelectOperator: Initialization Done 5 SEL 2018-01-05T11:29:06,160 DEBUG [TezTR-518813_1_35_2_3_0] log.PerfLogger: 2018-01-05T11:29:06,160 DEBUG [TezTR-518813_1_35_2_0_0] mr.ObjectCache: djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a_Reducer 3__REDUCE_PLAN__ no longer needed 2018-01-05T11:29:06,160 DEBUG [TezTR-518813_1_35_2_3_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_35_02_000003_0_10238 2018-01-05T11:29:06,160 DEBUG [TezTR-518813_1_35_2_1_0] exec.SelectOperator: Operator 5 SEL initialized 2018-01-05T11:29:06,160 DEBUG [TezTR-518813_1_35_2_3_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:06,160 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_35_01_000002_0_10225 not found 2018-01-05T11:29:06,160 DEBUG [TezTR-518813_1_35_2_3_0] mr.ObjectCache: Creating djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a_Reducer 3__REDUCE_PLAN__ 2018-01-05T11:29:06,160 DEBUG [TezTR-518813_1_35_2_1_0] exec.SelectOperator: Initializing children of 5 SEL 2018-01-05T11:29:06,160 INFO [TezTaskEventRouter{attempt_1515180518813_0001_35_02_000003_0}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 4. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:06,160 DEBUG [TezTR-518813_1_35_2_0_0] exec.GroupByOperator: close called for operator GBY[4] 2018-01-05T11:29:06,160 INFO [TezTR-518813_1_35_2_3_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/233a4fcb-aae6-42b5-96e2-24086b11320a/reduce.xml 2018-01-05T11:29:06,160 INFO [TezTaskEventRouter{attempt_1515180518813_0001_35_02_000003_0}] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Reducer 2: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=1, numObsoletionEventsSeen=0 2018-01-05T11:29:06,160 DEBUG [TezTR-518813_1_35_2_1_0] exec.FileSinkOperator: Initializing child 6 FS 2018-01-05T11:29:06,160 DEBUG [TezTR-518813_1_35_2_3_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/233a4fcb-aae6-42b5-96e2-24086b11320a/reduce.xml 2018-01-05T11:29:06,160 INFO [TezTR-518813_1_35_2_1_0] exec.FileSinkOperator: Initializing operator FS[6] 2018-01-05T11:29:06,160 INFO [TezTR-518813_1_35_2_0_0] exec.GroupByOperator: Closing operator GBY[4] 2018-01-05T11:29:06,160 DEBUG [TezTR-518813_1_35_2_3_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/djaiswal/_tez_scratch_dir/233a4fcb-aae6-42b5-96e2-24086b11320a/reduce.xml 2018-01-05T11:29:06,160 INFO [TezTaskEventRouter{attempt_1515180518813_0001_35_02_000003_0}] orderedgrouped.ShuffleScheduler: All inputs fetched for input vertex : Reducer 2 2018-01-05T11:29:06,160 DEBUG [TezTR-518813_1_35_2_3_0] log.PerfLogger: 2018-01-05T11:29:06,160 INFO [TezTR-518813_1_35_2_3_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:29:06,160 DEBUG [TezTR-518813_1_35_2_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,160 DEBUG [TezTR-518813_1_35_2_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,160 INFO [TezTaskEventRouter{attempt_1515180518813_0001_35_02_000003_0}] orderedgrouped.ShuffleScheduler: copy(4 (spillsFetched=4) of 4. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:06,160 DEBUG [TezTR-518813_1_35_2_0_0] exec.GroupByOperator: Closing child = SEL[5] 2018-01-05T11:29:06,160 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_35_01_000002_0_10225,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/35/output/attempt_1515180518813_0001_35_01_000002_0_10225/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/35/output/attempt_1515180518813_0001_35_01_000002_0_10225/file.out.index 2018-01-05T11:29:06,160 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Content Length in shuffle : 1478 2018-01-05T11:29:06,160 DEBUG [TezTR-518813_1_35_2_0_0] exec.SelectOperator: close called for operator SEL[5] 2018-01-05T11:29:06,161 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.ShuffleScheduler: Shutting down FetchScheduler for input: Reducer_2, wasInterrupted=false 2018-01-05T11:29:06,161 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.ShuffleScheduler: copy(4 (spillsFetched=4) of 4. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:06,161 INFO [TezTR-518813_1_35_2_1_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@24ccba30]:[_col0, _col1, _col2]:[struct, struct, string]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@44134311 2018-01-05T11:29:06,161 DEBUG [TezTR-518813_1_35_2_1_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:29:06,162 DEBUG [TezTR-518813_1_35_2_1_0] exec.FileSinkOperator: Initialization Done 6 FS 2018-01-05T11:29:06,162 DEBUG [TezTR-518813_1_35_2_1_0] exec.FileSinkOperator: Operator 6 FS initialized 2018-01-05T11:29:06,162 DEBUG [TezTR-518813_1_35_2_1_0] exec.FileSinkOperator: Initialization Done 6 FS done is reset. 2018-01-05T11:29:06,162 DEBUG [TezTR-518813_1_35_2_1_0] exec.SelectOperator: Initialization Done 5 SEL done is reset. 2018-01-05T11:29:06,162 DEBUG [TezTR-518813_1_35_2_1_0] exec.GroupByOperator: Initialization Done 4 GBY done is reset. 2018-01-05T11:29:06,162 DEBUG [TezTR-518813_1_35_2_0_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:06,162 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.ShuffleScheduler: Shutting down fetchers for input: Reducer_2, shutdown timetaken: 0 ms, hasFetcherExecutorStopped: true 2018-01-05T11:29:06,162 INFO [TezTR-518813_1_35_2_1_0] tez.ReduceRecordProcessor: Id =4 Id =5 Id =6 <\Children> Id = 5 null<\Parent> <\FS> <\Children> Id = 4 null<\Parent> <\SEL> <\Children> <\Parent> <\GBY> 2018-01-05T11:29:06,162 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.MergeManager: finalMerge with #inMemoryOutputs=0, size=0 and #onDiskOutputs=0, size=0 2018-01-05T11:29:06,162 DEBUG [TezTR-518813_1_35_2_1_0] log.PerfLogger: 2018-01-05T11:29:06,162 INFO [TezTR-518813_1_35_2_1_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 3 2018-01-05T11:29:06,162 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.MergeManager: . DiskSeg: 0, 0. MemSeg: 0, 0 2018-01-05T11:29:06,162 INFO [ShuffleAndMergeRunner {Reducer_2}] impl.TezMerger: Nothing to merge. Returning an empty iterator 2018-01-05T11:29:06,162 INFO [TezTR-518813_1_35_2_0_0] exec.SelectOperator: Closing operator SEL[5] 2018-01-05T11:29:06,162 INFO [Fetcher_O {Reducer_2} #1] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=35&reduce=2&map=attempt_1515180518813_0001_35_01_000000_0_10219,attempt_1515180518813_0001_35_01_000002_0_10225 sent hash and receievd reply 0 ms 2018-01-05T11:29:06,162 DEBUG [New I/O worker #1] shufflehandler.FadvisedFileRegion: shuffleBufferSize: 131072, path: /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/35/output/attempt_1515180518813_0001_35_01_000000_0_10219/file.out 2018-01-05T11:29:06,162 DEBUG [TezTR-518813_1_35_2_1_0] log.PerfLogger: 2018-01-05T11:29:06,162 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.Shuffle: merge complete for input vertex : Reducer_2 2018-01-05T11:29:06,162 DEBUG [TezTR-518813_1_35_2_0_0] exec.SelectOperator: Closing child = FS[6] 2018-01-05T11:29:06,162 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.Shuffle: Reducer_2: Shuffle Runner thread complete 2018-01-05T11:29:06,162 DEBUG [TezTR-518813_1_35_2_1_0] mr.ObjectCache: djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a_Reducer 3__REDUCE_PLAN__ no longer needed 2018-01-05T11:29:06,162 INFO [Fetcher_O {Reducer_2} #1] ShuffleScheduler.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_35_01_000000_0_10219} to MEMORY, csize=666, dsize=662, EndTime=1515180546162, TimeTaken=0, Rate=0.00 MB/s 2018-01-05T11:29:06,162 DEBUG [TezTR-518813_1_35_2_0_0] exec.FileSinkOperator: close called for operator FS[6] 2018-01-05T11:29:06,162 DEBUG [TezTR-518813_1_35_2_0_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:06,162 INFO [TezTR-518813_1_35_2_0_0] exec.FileSinkOperator: Closing operator FS[6] 2018-01-05T11:29:06,162 DEBUG [TezTR-518813_1_35_2_1_0] exec.GroupByOperator: close called for operator GBY[4] 2018-01-05T11:29:06,162 INFO [TezTR-518813_1_35_2_1_0] exec.GroupByOperator: Closing operator GBY[4] 2018-01-05T11:29:06,162 INFO [TezTR-518813_1_35_2_0_0] exec.FileSinkOperator: FS[6]: records written - 0 2018-01-05T11:29:06,162 DEBUG [TezTR-518813_1_35_2_1_0] exec.GroupByOperator: Closing child = SEL[5] 2018-01-05T11:29:06,162 DEBUG [TezTR-518813_1_35_2_1_0] exec.SelectOperator: close called for operator SEL[5] 2018-01-05T11:29:06,162 DEBUG [TezTR-518813_1_35_2_1_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:06,162 INFO [TezTR-518813_1_35_2_1_0] exec.SelectOperator: Closing operator SEL[5] 2018-01-05T11:29:06,162 DEBUG [TezTR-518813_1_35_2_1_0] exec.SelectOperator: Closing child = FS[6] 2018-01-05T11:29:06,162 DEBUG [TezTR-518813_1_35_2_1_0] exec.FileSinkOperator: close called for operator FS[6] 2018-01-05T11:29:06,162 DEBUG [TezTR-518813_1_35_2_1_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:06,162 INFO [TezTR-518813_1_35_2_1_0] exec.FileSinkOperator: Closing operator FS[6] 2018-01-05T11:29:06,162 INFO [TezTR-518813_1_35_2_1_0] exec.FileSinkOperator: FS[6]: records written - 0 2018-01-05T11:29:06,162 DEBUG [New I/O worker #1] shufflehandler.FadvisedFileRegion: shuffleBufferSize: 131072, path: /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/35/output/attempt_1515180518813_0001_35_01_000002_0_10225/file.out 2018-01-05T11:29:06,163 INFO [Fetcher_O {Reducer_2} #1] ShuffleScheduler.fetch: Completed fetch for attempt: {2, 0, attempt_1515180518813_0001_35_01_000002_0_10225} to MEMORY, csize=702, dsize=698, EndTime=1515180546163, TimeTaken=0, Rate=0.00 MB/s 2018-01-05T11:29:06,163 INFO [Fetcher_O {Reducer_2} #1] orderedgrouped.ShuffleScheduler: All inputs fetched for input vertex : Reducer 2 2018-01-05T11:29:06,163 INFO [Fetcher_O {Reducer_2} #1] orderedgrouped.ShuffleScheduler: copy(4 (spillsFetched=4) of 4. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:06,163 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.ShuffleScheduler: Shutting down FetchScheduler for input: Reducer_2, wasInterrupted=false 2018-01-05T11:29:06,163 INFO [TezTR-518813_1_35_2_0_0] exec.FileSinkOperator: RECORDS_OUT_0:0, 2018-01-05T11:29:06,163 INFO [TezTR-518813_1_35_2_1_0] exec.FileSinkOperator: RECORDS_OUT_0:0, 2018-01-05T11:29:06,163 DEBUG [TezTR-518813_1_35_2_3_0] log.PerfLogger: 2018-01-05T11:29:06,163 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.ShuffleScheduler: copy(4 (spillsFetched=4) of 4. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:06,163 DEBUG [TezTR-518813_1_35_2_0_0] exec.FileSinkOperator: 6 Close done 2018-01-05T11:29:06,163 INFO [TezTR-518813_1_35_2_3_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 3 size: 2.66KB 2018-01-05T11:29:06,163 DEBUG [TezTR-518813_1_35_2_0_0] exec.SelectOperator: 5 Close done 2018-01-05T11:29:06,163 DEBUG [TezTR-518813_1_35_2_1_0] exec.FileSinkOperator: 6 Close done 2018-01-05T11:29:06,163 DEBUG [TezTR-518813_1_35_2_0_0] exec.GroupByOperator: 4 Close done 2018-01-05T11:29:06,163 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.ShuffleScheduler: Shutting down fetchers for input: Reducer_2, shutdown timetaken: 0 ms, hasFetcherExecutorStopped: true 2018-01-05T11:29:06,163 DEBUG [TezTR-518813_1_35_2_1_0] exec.SelectOperator: 5 Close done 2018-01-05T11:29:06,163 DEBUG [TezTR-518813_1_35_2_1_0] exec.GroupByOperator: 4 Close done 2018-01-05T11:29:06,164 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.MergeManager: TotalInMemFetchStats: count=4, totalSize=2725, min=652, max=713, avg=1.0 2018-01-05T11:29:06,164 INFO [TezTR-518813_1_35_2_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_35_02_000000_0 2018-01-05T11:29:06,164 INFO [TezTR-518813_1_35_2_1_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_35_02_000001_0 2018-01-05T11:29:06,164 INFO [TezTR-518813_1_35_2_0_0] orderedgrouped.Shuffle: Shutting down Shuffle for source: Reducer_2 2018-01-05T11:29:06,164 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.MergeManager: finalMerge with #inMemoryOutputs=4, size=2725 and #onDiskOutputs=0, size=0 2018-01-05T11:29:06,164 INFO [TezTR-518813_1_35_2_1_0] orderedgrouped.Shuffle: Shutting down Shuffle for source: Reducer_2 2018-01-05T11:29:06,164 INFO [TezTR-518813_1_35_2_0_0] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Reducer 2: numDmeEventsSeen=4, numDmeEventsSeenWithNoData=4, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:29:06,164 INFO [TezTR-518813_1_35_2_1_0] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Reducer 2: numDmeEventsSeen=4, numDmeEventsSeenWithNoData=4, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:29:06,164 INFO [TezTR-518813_1_35_2_0_0] output.MROutput: out_Reducer 3 closed 2018-01-05T11:29:06,164 DEBUG [TezTR-518813_1_35_2_3_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:06,164 INFO [TezTR-518813_1_35_2_1_0] output.MROutput: out_Reducer 3 closed 2018-01-05T11:29:06,164 DEBUG [TezTR-518813_1_35_2_3_0] log.PerfLogger: 2018-01-05T11:29:06,164 DEBUG [TezTR-518813_1_35_2_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:06,164 INFO [TezTR-518813_1_35_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:06,164 DEBUG [TezTR-518813_1_35_2_3_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:06,164 DEBUG [TezTR-518813_1_35_2_3_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:06,164 DEBUG [TezTR-518813_1_35_2_1_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:06,164 INFO [TezTR-518813_1_35_2_1_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:06,164 INFO [TezTR-518813_1_35_2_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_35_02_000000_0, askedToStop=false 2018-01-05T11:29:06,164 INFO [TezTR-518813_1_35_2_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_35_02_000000_0, stopRequested=false 2018-01-05T11:29:06,164 INFO [TezTR-518813_1_35_2_1_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_35_02_000001_0, askedToStop=false 2018-01-05T11:29:06,164 INFO [TezTR-518813_1_35_2_1_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_35_02_000001_0, stopRequested=false 2018-01-05T11:29:06,164 INFO [TezTR-518813_1_35_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_35_02_000000_0: Counters: 29 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=0, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=0, NUM_SKIPPED_INPUTS=4, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=0, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, SHUFFLE_BYTES=0, SHUFFLE_BYTES_DECOMPRESSED=0, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][HIVE RECORDS_OUT_0=0][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0]] 2018-01-05T11:29:06,164 INFO [TezTR-518813_1_35_2_1_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_35_02_000001_0: Counters: 29 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=0, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=0, NUM_SKIPPED_INPUTS=4, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=0, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, SHUFFLE_BYTES=0, SHUFFLE_BYTES_DECOMPRESSED=0, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=5, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=4][HIVE RECORDS_OUT_0=0][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0]] 2018-01-05T11:29:06,164 DEBUG [TezTR-518813_1_35_2_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_35_2_0_0 thread id: 2293 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:06,164 DEBUG [TezTR-518813_1_35_2_1_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_35_2_1_0 thread id: 2299 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:06,164 DEBUG [TezTR-518813_1_35_2_3_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:06,164 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_35_02_000001_0, isTaskDone=true 2018-01-05T11:29:06,164 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_35_02_000000_0, isTaskDone=true 2018-01-05T11:29:06,164 INFO [TezTR-518813_1_35_2_3_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:29:06,164 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000077, requestId=2, startIndex=4, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_35_02_000000_0, eventCount=2 } 2018-01-05T11:29:06,164 INFO [TezTR-518813_1_35_2_3_0] tez.ReduceRecordProcessor: Main work is Reducer 3 2018-01-05T11:29:06,164 INFO [TezTR-518813_1_35_2_3_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:06,164 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000078, requestId=2, startIndex=4, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_35_02_000001_0, eventCount=2 } 2018-01-05T11:29:06,164 DEBUG [TezTR-518813_1_35_2_3_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:06,164 INFO [TezTR-518813_1_35_2_3_0] input.OrderedGroupedKVInput: Reducer 2: creating ValuesIterator with comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator, keyClass=org.apache.hadoop.hive.ql.io.HiveKey, valClass=org.apache.hadoop.io.BytesWritable 2018-01-05T11:29:06,164 DEBUG [TezTR-518813_1_35_2_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,165 DEBUG [TezTR-518813_1_35_2_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,165 DEBUG [TezTR-518813_1_35_2_3_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:29:06,165 DEBUG [TezTR-518813_1_35_2_3_0] log.PerfLogger: 2018-01-05T11:29:06,165 INFO [TezTR-518813_1_35_2_3_0] exec.GroupByOperator: Initializing operator GBY[4] 2018-01-05T11:29:06,165 DEBUG [TezTR-518813_1_35_2_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,165 DEBUG [TezTR-518813_1_35_2_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,165 INFO [TezTR-518813_1_35_2_3_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:29:06,165 DEBUG [TezTR-518813_1_35_2_3_0] exec.GroupByOperator: Initialization Done 4 GBY 2018-01-05T11:29:06,165 DEBUG [TezTR-518813_1_35_2_3_0] exec.GroupByOperator: Operator 4 GBY initialized 2018-01-05T11:29:06,165 DEBUG [TezTR-518813_1_35_2_3_0] exec.GroupByOperator: Initializing children of 4 GBY 2018-01-05T11:29:06,165 DEBUG [TezTR-518813_1_35_2_3_0] exec.SelectOperator: Initializing child 5 SEL 2018-01-05T11:29:06,165 INFO [TezTR-518813_1_35_2_3_0] exec.SelectOperator: Initializing operator SEL[5] 2018-01-05T11:29:06,165 INFO [TezTR-518813_1_35_2_3_0] exec.SelectOperator: SELECT struct<_col0:string,_col1:struct,_col2:struct> 2018-01-05T11:29:06,165 DEBUG [TezTR-518813_1_35_2_3_0] exec.SelectOperator: Initialization Done 5 SEL 2018-01-05T11:29:06,165 DEBUG [TezTR-518813_1_35_2_3_0] exec.SelectOperator: Operator 5 SEL initialized 2018-01-05T11:29:06,165 DEBUG [TezTR-518813_1_35_2_3_0] exec.SelectOperator: Initializing children of 5 SEL 2018-01-05T11:29:06,165 DEBUG [TezTR-518813_1_35_2_3_0] exec.FileSinkOperator: Initializing child 6 FS 2018-01-05T11:29:06,165 INFO [TezTR-518813_1_35_2_3_0] exec.FileSinkOperator: Initializing operator FS[6] 2018-01-05T11:29:06,165 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 3, taskAttemptId=attempt_1515180518813_0001_35_02_000000_0, creationTime=1515180546107, allocationTime=1515180546115, startTime=1515180546116, finishTime=1515180546165, timeTaken=49, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:06,165 DEBUG [TezTR-518813_1_35_2_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,165 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 3, taskAttemptId=attempt_1515180518813_0001_35_02_000001_0, creationTime=1515180546113, allocationTime=1515180546115, startTime=1515180546117, finishTime=1515180546165, timeTaken=48, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:06,165 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=4, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:06,165 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=4, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:06,165 INFO [Task-Executor-1] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_35_02_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:06,165 INFO [Task-Executor-1] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_35_02_000000_0 2018-01-05T11:29:06,165 INFO [Task-Executor-3] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_35_02_000001_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:06,165 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_35_02_000000_0 2018-01-05T11:29:06,165 INFO [Task-Executor-3] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_35_02_000001_0 2018-01-05T11:29:06,165 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_35_02_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:06,165 INFO [TezTR-518813_1_35_2_3_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@1ebca550]:[_col0, _col1, _col2]:[struct, struct, string]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@764fe3b7 2018-01-05T11:29:06,165 INFO [Task-Executor-1] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000077=48 2018-01-05T11:29:06,165 INFO [Task-Executor-3] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000078=46 2018-01-05T11:29:06,165 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_35_02_000000_0: true 2018-01-05T11:29:06,165 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_35_02_000000_0; false 2018-01-05T11:29:06,165 DEBUG [TezTR-518813_1_35_2_3_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:29:06,165 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_35_02_000001_0: true 2018-01-05T11:29:06,165 DEBUG [TezTR-518813_1_35_2_3_0] exec.FileSinkOperator: Initialization Done 6 FS 2018-01-05T11:29:06,165 DEBUG [TezTR-518813_1_35_2_3_0] exec.FileSinkOperator: Operator 6 FS initialized 2018-01-05T11:29:06,165 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_35_02_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:06,165 DEBUG [TezTR-518813_1_35_2_3_0] exec.FileSinkOperator: Initialization Done 6 FS done is reset. 2018-01-05T11:29:06,165 DEBUG [TezTR-518813_1_35_2_3_0] exec.SelectOperator: Initialization Done 5 SEL done is reset. 2018-01-05T11:29:06,165 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:TASK_FINISHED]: vertexName=Reducer 3, taskId=task_1515180518813_0001_35_02_000000, startTime=1515180546116, finishTime=1515180546165, timeTaken=49, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_35_02_000000_0, diagnostics=, counters=Counters: 34, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=0, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=0, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=0, NUM_SKIPPED_INPUTS=4, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=0, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, SHUFFLE_BYTES=0, SHUFFLE_BYTES_DECOMPRESSED=0, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, HIVE, RECORDS_OUT_0=0, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0 2018-01-05T11:29:06,165 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:06,165 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:06,165 DEBUG [TezTR-518813_1_35_2_3_0] exec.GroupByOperator: Initialization Done 4 GBY done is reset. 2018-01-05T11:29:06,166 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_35_02_000001_0 2018-01-05T11:29:06,166 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_35_02_000001_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:06,166 INFO [TezTR-518813_1_35_2_3_0] tez.ReduceRecordProcessor: Id =4 Id =5 Id =6 <\Children> Id = 5 null<\Parent> <\FS> <\Children> Id = 4 null<\Parent> <\SEL> <\Children> <\Parent> <\GBY> 2018-01-05T11:29:06,166 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_35_02_000001_0; false 2018-01-05T11:29:06,166 DEBUG [TezTR-518813_1_35_2_3_0] log.PerfLogger: 2018-01-05T11:29:06,166 INFO [TezTR-518813_1_35_2_3_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 3 2018-01-05T11:29:06,166 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_35_02_000001_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:06,166 DEBUG [TezTR-518813_1_35_2_3_0] log.PerfLogger: 2018-01-05T11:29:06,166 DEBUG [TezTR-518813_1_35_2_3_0] mr.ObjectCache: djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a_Reducer 3__REDUCE_PLAN__ no longer needed 2018-01-05T11:29:06,166 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:06,166 DEBUG [TezTR-518813_1_35_2_3_0] exec.GroupByOperator: close called for operator GBY[4] 2018-01-05T11:29:06,166 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:TASK_FINISHED]: vertexName=Reducer 3, taskId=task_1515180518813_0001_35_02_000001, startTime=1515180546117, finishTime=1515180546166, timeTaken=49, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_35_02_000001_0, diagnostics=, counters=Counters: 34, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=0, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=0, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=0, NUM_SKIPPED_INPUTS=4, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=0, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, SHUFFLE_BYTES=0, SHUFFLE_BYTES_DECOMPRESSED=0, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=5, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=4, HIVE, RECORDS_OUT_0=0, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0 2018-01-05T11:29:06,166 INFO [TezTR-518813_1_35_2_3_0] exec.GroupByOperator: Closing operator GBY[4] 2018-01-05T11:29:06,166 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:06,166 DEBUG [TezTR-518813_1_35_2_3_0] exec.GroupByOperator: Closing child = SEL[5] 2018-01-05T11:29:06,166 DEBUG [TezTR-518813_1_35_2_3_0] exec.SelectOperator: close called for operator SEL[5] 2018-01-05T11:29:06,166 DEBUG [TezTR-518813_1_35_2_3_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:06,166 INFO [TezTR-518813_1_35_2_3_0] exec.SelectOperator: Closing operator SEL[5] 2018-01-05T11:29:06,166 DEBUG [TezTR-518813_1_35_2_3_0] exec.SelectOperator: Closing child = FS[6] 2018-01-05T11:29:06,166 DEBUG [TezTR-518813_1_35_2_3_0] exec.FileSinkOperator: close called for operator FS[6] 2018-01-05T11:29:06,166 DEBUG [TezTR-518813_1_35_2_3_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:06,166 INFO [TezTR-518813_1_35_2_3_0] exec.FileSinkOperator: Closing operator FS[6] 2018-01-05T11:29:06,166 INFO [TezTR-518813_1_35_2_3_0] exec.FileSinkOperator: FS[6]: records written - 0 2018-01-05T11:29:06,166 INFO [TezTR-518813_1_35_2_3_0] exec.FileSinkOperator: RECORDS_OUT_0:0, 2018-01-05T11:29:06,166 DEBUG [TezTR-518813_1_35_2_3_0] exec.FileSinkOperator: 6 Close done 2018-01-05T11:29:06,166 DEBUG [TezTR-518813_1_35_2_3_0] exec.SelectOperator: 5 Close done 2018-01-05T11:29:06,166 DEBUG [TezTR-518813_1_35_2_3_0] exec.GroupByOperator: 4 Close done 2018-01-05T11:29:06,166 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_35_02 [Reducer 3], tasks=4, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:06,166 INFO [TezTR-518813_1_35_2_3_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_35_02_000003_0 2018-01-05T11:29:06,166 INFO [TezTR-518813_1_35_2_3_0] orderedgrouped.Shuffle: Shutting down Shuffle for source: Reducer_2 2018-01-05T11:29:06,166 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000077, stoppedTime=1515180546166, exitStatus=0 2018-01-05T11:29:06,166 INFO [TezTR-518813_1_35_2_3_0] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Reducer 2: numDmeEventsSeen=4, numDmeEventsSeenWithNoData=4, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:29:06,166 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000077 exited with diagnostics set to Container completed. 2018-01-05T11:29:06,166 INFO [TezTR-518813_1_35_2_3_0] output.MROutput: out_Reducer 3 closed 2018-01-05T11:29:06,166 DEBUG [TezTR-518813_1_35_2_3_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:06,166 INFO [TezTR-518813_1_35_2_3_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:06,166 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_35_02 [Reducer 3], tasks=4, failed=0, killed=0, success=2, completed=2, commits=0, err=null 2018-01-05T11:29:06,166 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000078, stoppedTime=1515180546166, exitStatus=0 2018-01-05T11:29:06,166 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000078 exited with diagnostics set to Container completed. 2018-01-05T11:29:06,166 INFO [TezTR-518813_1_35_2_3_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_35_02_000003_0, askedToStop=false 2018-01-05T11:29:06,166 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000077 2018-01-05T11:29:06,166 INFO [TezTR-518813_1_35_2_3_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_35_02_000003_0, stopRequested=false 2018-01-05T11:29:06,166 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000078 2018-01-05T11:29:06,166 INFO [TezTR-518813_1_35_2_3_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_35_02_000003_0: Counters: 29 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=0, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=0, NUM_SKIPPED_INPUTS=4, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=0, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, SHUFFLE_BYTES=0, SHUFFLE_BYTES_DECOMPRESSED=0, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=4, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2][HIVE RECORDS_OUT_0=0][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0]] 2018-01-05T11:29:06,166 DEBUG [TezTR-518813_1_35_2_3_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_35_2_3_0 thread id: 2305 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:06,166 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_35_02_000003_0, isTaskDone=true 2018-01-05T11:29:06,166 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000079, requestId=2, startIndex=4, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_35_02_000003_0, eventCount=2 } 2018-01-05T11:29:06,167 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 3, taskAttemptId=attempt_1515180518813_0001_35_02_000003_0, creationTime=1515180546113, allocationTime=1515180546115, startTime=1515180546118, finishTime=1515180546167, timeTaken=49, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:06,167 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_35_02_000003_0 2018-01-05T11:29:06,167 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_35_02_000003_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:06,167 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=4, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:06,167 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_35_02_000003_0; false 2018-01-05T11:29:06,167 INFO [Task-Executor-2] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_35_02_000003_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:06,167 INFO [Task-Executor-2] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_35_02_000003_0 2018-01-05T11:29:06,167 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_35_02_000003_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:06,167 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:06,167 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:06,167 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:TASK_FINISHED]: vertexName=Reducer 3, taskId=task_1515180518813_0001_35_02_000003, startTime=1515180546118, finishTime=1515180546167, timeTaken=49, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_35_02_000003_0, diagnostics=, counters=Counters: 34, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=0, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=0, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=0, NUM_SKIPPED_INPUTS=4, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=0, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, SHUFFLE_BYTES=0, SHUFFLE_BYTES_DECOMPRESSED=0, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=4, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, HIVE, RECORDS_OUT_0=0, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0 2018-01-05T11:29:06,167 INFO [Task-Executor-2] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000079=28 2018-01-05T11:29:06,167 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_35_02_000003_0: true 2018-01-05T11:29:06,167 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_35_02 [Reducer 3], tasks=4, failed=0, killed=0, success=3, completed=3, commits=0, err=null 2018-01-05T11:29:06,167 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000079, stoppedTime=1515180546167, exitStatus=0 2018-01-05T11:29:06,167 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000079 exited with diagnostics set to Container completed. 2018-01-05T11:29:06,167 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000079 2018-01-05T11:29:06,180 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.Shuffle: merge complete for input vertex : Reducer_2 2018-01-05T11:29:06,180 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.Shuffle: Reducer_2: Shuffle Runner thread complete 2018-01-05T11:29:06,180 INFO [TezTR-518813_1_35_2_2_0] tez.ReduceRecordProcessor: Main work is Reducer 3 2018-01-05T11:29:06,180 INFO [TezTR-518813_1_35_2_2_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:06,180 DEBUG [TezTR-518813_1_35_2_2_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:06,180 INFO [TezTR-518813_1_35_2_2_0] input.OrderedGroupedKVInput: Reducer 2: creating ValuesIterator with comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator, keyClass=org.apache.hadoop.hive.ql.io.HiveKey, valClass=org.apache.hadoop.io.BytesWritable 2018-01-05T11:29:06,180 DEBUG [TezTR-518813_1_35_2_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,180 DEBUG [TezTR-518813_1_35_2_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,180 DEBUG [TezTR-518813_1_35_2_2_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:29:06,180 DEBUG [TezTR-518813_1_35_2_2_0] log.PerfLogger: 2018-01-05T11:29:06,180 INFO [TezTR-518813_1_35_2_2_0] exec.GroupByOperator: Initializing operator GBY[4] 2018-01-05T11:29:06,180 DEBUG [TezTR-518813_1_35_2_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,180 DEBUG [TezTR-518813_1_35_2_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,180 INFO [TezTR-518813_1_35_2_2_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:29:06,180 DEBUG [TezTR-518813_1_35_2_2_0] exec.GroupByOperator: Initialization Done 4 GBY 2018-01-05T11:29:06,180 DEBUG [TezTR-518813_1_35_2_2_0] exec.GroupByOperator: Operator 4 GBY initialized 2018-01-05T11:29:06,180 DEBUG [TezTR-518813_1_35_2_2_0] exec.GroupByOperator: Initializing children of 4 GBY 2018-01-05T11:29:06,180 DEBUG [TezTR-518813_1_35_2_2_0] exec.SelectOperator: Initializing child 5 SEL 2018-01-05T11:29:06,180 INFO [TezTR-518813_1_35_2_2_0] exec.SelectOperator: Initializing operator SEL[5] 2018-01-05T11:29:06,180 INFO [TezTR-518813_1_35_2_2_0] exec.SelectOperator: SELECT struct<_col0:string,_col1:struct,_col2:struct> 2018-01-05T11:29:06,180 DEBUG [TezTR-518813_1_35_2_2_0] exec.SelectOperator: Initialization Done 5 SEL 2018-01-05T11:29:06,180 DEBUG [TezTR-518813_1_35_2_2_0] exec.SelectOperator: Operator 5 SEL initialized 2018-01-05T11:29:06,180 DEBUG [TezTR-518813_1_35_2_2_0] exec.SelectOperator: Initializing children of 5 SEL 2018-01-05T11:29:06,180 DEBUG [TezTR-518813_1_35_2_2_0] exec.FileSinkOperator: Initializing child 6 FS 2018-01-05T11:29:06,180 INFO [TezTR-518813_1_35_2_2_0] exec.FileSinkOperator: Initializing operator FS[6] 2018-01-05T11:29:06,181 DEBUG [TezTR-518813_1_35_2_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,181 INFO [TezTR-518813_1_35_2_2_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@540688fb]:[_col0, _col1, _col2]:[struct, struct, string]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@237dd013 2018-01-05T11:29:06,181 DEBUG [TezTR-518813_1_35_2_2_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:29:06,181 DEBUG [TezTR-518813_1_35_2_2_0] exec.FileSinkOperator: Initialization Done 6 FS 2018-01-05T11:29:06,181 DEBUG [TezTR-518813_1_35_2_2_0] exec.FileSinkOperator: Operator 6 FS initialized 2018-01-05T11:29:06,181 DEBUG [TezTR-518813_1_35_2_2_0] exec.FileSinkOperator: Initialization Done 6 FS done is reset. 2018-01-05T11:29:06,181 DEBUG [TezTR-518813_1_35_2_2_0] exec.SelectOperator: Initialization Done 5 SEL done is reset. 2018-01-05T11:29:06,181 DEBUG [TezTR-518813_1_35_2_2_0] exec.GroupByOperator: Initialization Done 4 GBY done is reset. 2018-01-05T11:29:06,181 INFO [TezTR-518813_1_35_2_2_0] tez.ReduceRecordProcessor: Id =4 Id =5 Id =6 <\Children> Id = 5 null<\Parent> <\FS> <\Children> Id = 4 null<\Parent> <\SEL> <\Children> <\Parent> <\GBY> 2018-01-05T11:29:06,181 DEBUG [TezTR-518813_1_35_2_2_0] log.PerfLogger: 2018-01-05T11:29:06,181 INFO [TezTR-518813_1_35_2_2_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 3 2018-01-05T11:29:06,182 DEBUG [TezTR-518813_1_35_2_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,182 DEBUG [TezTR-518813_1_35_2_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,182 DEBUG [TezTR-518813_1_35_2_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,182 DEBUG [TezTR-518813_1_35_2_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,182 DEBUG [TezTR-518813_1_35_2_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,182 DEBUG [TezTR-518813_1_35_2_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,182 DEBUG [TezTR-518813_1_35_2_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,182 DEBUG [TezTR-518813_1_35_2_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,182 DEBUG [TezTR-518813_1_35_2_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,183 DEBUG [TezTR-518813_1_35_2_2_0] log.PerfLogger: 2018-01-05T11:29:06,183 DEBUG [TezTR-518813_1_35_2_2_0] mr.ObjectCache: djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a_Reducer 3__REDUCE_PLAN__ no longer needed 2018-01-05T11:29:06,183 DEBUG [TezTR-518813_1_35_2_2_0] exec.GroupByOperator: close called for operator GBY[4] 2018-01-05T11:29:06,183 INFO [TezTR-518813_1_35_2_2_0] exec.GroupByOperator: Closing operator GBY[4] 2018-01-05T11:29:06,183 INFO [TezTR-518813_1_35_2_2_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_318_6335458483535453971-1/-mr-10000/.hive-staging_hive_2018-01-05_11-29-05_318_6335458483535453971-1/_tmp.-ext-10001/000002_0 2018-01-05T11:29:06,183 INFO [TezTR-518813_1_35_2_2_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_318_6335458483535453971-1/-mr-10000/.hive-staging_hive_2018-01-05_11-29-05_318_6335458483535453971-1/_task_tmp.-ext-10001/_tmp.000002_0 2018-01-05T11:29:06,183 INFO [TezTR-518813_1_35_2_2_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_318_6335458483535453971-1/-mr-10000/.hive-staging_hive_2018-01-05_11-29-05_318_6335458483535453971-1/_tmp.-ext-10001/000002_0 2018-01-05T11:29:06,212 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000074, QueryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a-35, DagId=35, VertexName=Reducer 2, TaskId=2, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180545843, EndTime=1515180546212 2018-01-05T11:29:06,212 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_35_02_000000_0 2018-01-05T11:29:06,212 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000077, QueryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a-35, Vertex=Reducer 3, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:06,212 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_35_02_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=1, preemptionQueueSize=3 2018-01-05T11:29:06,212 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_35_02_000000_0 2018-01-05T11:29:06,212 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000077, QueryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a-35, DagId=35, VertexName=Reducer 3, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-1, Succeeded=true, StartTime=1515180546117, EndTime=1515180546212 2018-01-05T11:29:06,212 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_35_02_000001_0 2018-01-05T11:29:06,212 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000078, QueryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a-35, Vertex=Reducer 3, FragmentNum=1, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:06,212 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_35_02_000001_0 complete. WaitQueueSize=0, numSlotsAvailable=2, preemptionQueueSize=2 2018-01-05T11:29:06,212 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_35_02_000001_0 2018-01-05T11:29:06,212 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000078, QueryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a-35, DagId=35, VertexName=Reducer 3, TaskId=1, TaskAttemptId=0, ThreadName=Task-Executor-3, Succeeded=true, StartTime=1515180546119, EndTime=1515180546212 2018-01-05T11:29:06,212 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_35_02_000003_0 2018-01-05T11:29:06,212 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000079, QueryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a-35, Vertex=Reducer 3, FragmentNum=3, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:06,212 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_35_02_000003_0 complete. WaitQueueSize=0, numSlotsAvailable=3, preemptionQueueSize=1 2018-01-05T11:29:06,212 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_35_02_000003_0 2018-01-05T11:29:06,212 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000079, QueryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a-35, DagId=35, VertexName=Reducer 3, TaskId=3, TaskAttemptId=0, ThreadName=Task-Executor-2, Succeeded=true, StartTime=1515180546139, EndTime=1515180546212 2018-01-05T11:29:06,217 INFO [TezTR-518813_1_35_2_2_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:29:06,217 DEBUG [TezTR-518813_1_35_2_2_0] exec.GroupByOperator: Closing child = SEL[5] 2018-01-05T11:29:06,217 DEBUG [TezTR-518813_1_35_2_2_0] exec.SelectOperator: close called for operator SEL[5] 2018-01-05T11:29:06,217 DEBUG [TezTR-518813_1_35_2_2_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:06,217 INFO [TezTR-518813_1_35_2_2_0] exec.SelectOperator: Closing operator SEL[5] 2018-01-05T11:29:06,217 DEBUG [TezTR-518813_1_35_2_2_0] exec.SelectOperator: Closing child = FS[6] 2018-01-05T11:29:06,217 DEBUG [TezTR-518813_1_35_2_2_0] exec.FileSinkOperator: close called for operator FS[6] 2018-01-05T11:29:06,217 DEBUG [TezTR-518813_1_35_2_2_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:06,217 INFO [TezTR-518813_1_35_2_2_0] exec.FileSinkOperator: Closing operator FS[6] 2018-01-05T11:29:06,217 INFO [TezTR-518813_1_35_2_2_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:29:06,217 INFO [TezTR-518813_1_35_2_2_0] exec.FileSinkOperator: RECORDS_OUT_0:1, 2018-01-05T11:29:06,217 DEBUG [TezTR-518813_1_35_2_2_0] exec.FileSinkOperator: 6 Close done 2018-01-05T11:29:06,217 DEBUG [TezTR-518813_1_35_2_2_0] exec.SelectOperator: 5 Close done 2018-01-05T11:29:06,217 DEBUG [TezTR-518813_1_35_2_2_0] exec.GroupByOperator: 4 Close done 2018-01-05T11:29:06,217 INFO [TezTR-518813_1_35_2_2_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_35_02_000002_0 2018-01-05T11:29:06,217 INFO [TezTR-518813_1_35_2_2_0] orderedgrouped.Shuffle: Shutting down Shuffle for source: Reducer_2 2018-01-05T11:29:06,217 INFO [TezTR-518813_1_35_2_2_0] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Reducer 2: numDmeEventsSeen=4, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:29:06,217 INFO [TezTR-518813_1_35_2_2_0] output.MROutput: out_Reducer 3 closed 2018-01-05T11:29:06,218 DEBUG [TezTR-518813_1_35_2_2_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:06,218 INFO [TezTR-518813_1_35_2_2_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:06,218 INFO [TezTR-518813_1_35_2_2_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_35_02_000002_0, askedToStop=false 2018-01-05T11:29:06,218 INFO [TezTR-518813_1_35_2_2_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_35_02_000002_0, stopRequested=false 2018-01-05T11:29:06,218 INFO [TezTR-518813_1_35_2_2_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_35_02_000002_0: Counters: 30 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=4, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=4, NUM_SHUFFLED_INPUTS=4, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=4, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=2671, ADDITIONAL_SPILLS_BYTES_READ=2671, SHUFFLE_BYTES=2741, SHUFFLE_BYTES_DECOMPRESSED=2725, SHUFFLE_BYTES_TO_MEM=2741, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=22, MERGE_PHASE_TIME=39, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=4][HIVE CREATED_FILES=1, RECORDS_OUT_0=1][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0]] 2018-01-05T11:29:06,218 DEBUG [TezTR-518813_1_35_2_2_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_35_2_2_0 thread id: 2287 scheme: file bytesRead: 0 bytesWritten: 3284 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:06,218 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_35_02_000002_0, isTaskDone=true 2018-01-05T11:29:06,218 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000076, requestId=2, startIndex=4, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_35_02_000002_0, eventCount=2 } 2018-01-05T11:29:06,218 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 3, taskAttemptId=attempt_1515180518813_0001_35_02_000002_0, creationTime=1515180546107, allocationTime=1515180546114, startTime=1515180546115, finishTime=1515180546218, timeTaken=103, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:06,219 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=4, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:06,219 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_35_02_000002_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:06,219 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_35_02_000002_0 2018-01-05T11:29:06,219 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_35_02_000002_0 2018-01-05T11:29:06,219 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_35_02_000002_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:06,219 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_35_02_000002_0; false 2018-01-05T11:29:06,219 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000076=103 2018-01-05T11:29:06,219 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_35_02_000002_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:06,219 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_35_02_000002_0: true 2018-01-05T11:29:06,219 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:06,219 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_35_02_000002_0 2018-01-05T11:29:06,219 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:TASK_FINISHED]: vertexName=Reducer 3, taskId=task_1515180518813_0001_35_02_000002, startTime=1515180546115, finishTime=1515180546219, timeTaken=104, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_35_02_000002_0, diagnostics=, counters=Counters: 35, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=3284, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=4, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=4, NUM_SHUFFLED_INPUTS=4, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=4, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=2671, ADDITIONAL_SPILLS_BYTES_READ=2671, SHUFFLE_BYTES=2741, SHUFFLE_BYTES_DECOMPRESSED=2725, SHUFFLE_BYTES_TO_MEM=2741, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=22, MERGE_PHASE_TIME=39, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=4, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0 2018-01-05T11:29:06,219 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:06,219 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000076, QueryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a-35, Vertex=Reducer 3, FragmentNum=2, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:06,219 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_35_02_000002_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:29:06,219 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_35_02_000002_0 2018-01-05T11:29:06,219 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_35_02 [Reducer 3], tasks=4, failed=0, killed=0, success=4, completed=4, commits=0, err=null 2018-01-05T11:29:06,219 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_35_02 [Reducer 3] 2018-01-05T11:29:06,220 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:VERTEX_FINISHED]: vertexName=Reducer 3, vertexId=vertex_1515180518813_0001_35_02, initRequestedTime=1515180545667, initedTime=1515180545674, startRequestedTime=1515180545730, startedTime=1515180545730, finishTime=1515180546219, timeTaken=489, status=SUCCEEDED, diagnostics=, counters=Counters: 35, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=3284, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=4, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=4, NUM_SHUFFLED_INPUTS=4, NUM_SKIPPED_INPUTS=12, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=4, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=2671, ADDITIONAL_SPILLS_BYTES_READ=2671, SHUFFLE_BYTES=2741, SHUFFLE_BYTES_DECOMPRESSED=2725, SHUFFLE_BYTES_TO_MEM=2741, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=38, MERGE_PHASE_TIME=55, FIRST_EVENT_RECEIVED=14, LAST_EVENT_RECEIVED=16, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, vertexStats=firstTaskStartTime=1515180546115, firstTasksToStart=[ task_1515180518813_0001_35_02_000002 ], lastTaskFinishTime=1515180546218, lastTasksToFinish=[ task_1515180518813_0001_35_02_000002 ], minTaskDuration=48, maxTaskDuration=103, avgTaskDuration=62.25, numSuccessfulTasks=4, shortestDurationTasks=[ task_1515180518813_0001_35_02_000001 ], longestDurationTasks=[ task_1515180518813_0001_35_02_000002 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=4, numSucceededTasks=4, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:06,220 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_35_02 [Reducer 3] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:06,220 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000076, stoppedTime=1515180546220, exitStatus=0 2018-01-05T11:29:06,220 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000076 exited with diagnostics set to Container completed. 2018-01-05T11:29:06,220 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_35_02 [Reducer 3] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3 2018-01-05T11:29:06,220 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null 2018-01-05T11:29:06,220 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_35_02 [Reducer 3] 2018-01-05T11:29:06,220 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_35_01 [Reducer 2] 2018-01-05T11:29:06,220 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_35_00 [Map 1] 2018-01-05T11:29:06,221 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_35, queueSize=0 2018-01-05T11:29:06,221 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_35][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_35, startTime=1515180545651, finishTime=1515180546220, timeTaken=569, status=SUCCEEDED, diagnostics=, counters=Counters: 60, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=9, TOTAL_LAUNCHED_TASKS=9, OTHER_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=2017, AM_GC_TIME_MILLIS=16, File System Counters, FILE_BYTES_READ=6952, FILE_BYTES_WRITTEN=17570, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=310, REDUCE_INPUT_RECORDS=504, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=1008, NUM_SHUFFLED_INPUTS=8, NUM_SKIPPED_INPUTS=12, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=8, INPUT_RECORDS_PROCESSED=4, INPUT_SPLIT_LENGTH_BYTES=5812, OUTPUT_RECORDS=504, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=10091, OUTPUT_BYTES_WITH_OVERHEAD=10090, OUTPUT_BYTES_PHYSICAL=10122, ADDITIONAL_SPILLS_BYTES_WRITTEN=10052, ADDITIONAL_SPILLS_BYTES_READ=10052, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=5, SHUFFLE_BYTES=10122, SHUFFLE_BYTES_DECOMPRESSED=10090, SHUFFLE_BYTES_TO_MEM=10122, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=79, MERGE_PHASE_TIME=188, FIRST_EVENT_RECEIVED=27, LAST_EVENT_RECEIVED=29, HIVE, CREATED_FILES=5, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=500, RECORDS_OUT_0=1, RECORDS_OUT_1_default.tab_part=500, RECORDS_OUT_INTERMEDIATE_Map_1=500, RECORDS_OUT_INTERMEDIATE_Reducer_2=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, org.apache.hadoop.hive.llap.counters.LlapIOCounters, CACHE_MISS_BYTES=5812, CONSUMER_TIME_NS=5705303, DECODE_TIME_NS=474044, NUM_DECODED_BATCHES=4, NUM_VECTOR_BATCHES=4, ROWS_EMITTED=500, TOTAL_IO_TIME_NS=6947128 2018-01-05T11:29:06,221 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_35 finished with state: SUCCEEDED 2018-01-05T11:29:06,221 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_35 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:29:06,221 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000076 2018-01-05T11:29:06,221 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_35, dagState=SUCCEEDED 2018-01-05T11:29:06,221 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_35_post 2018-01-05T11:29:06,221 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:29:06,221 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:29:06,221 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:29:06,221 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 35 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=9, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=9,NumRequestsWithoutLocation=9, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=9} 2018-01-05T11:29:06,221 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:29:06,221 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=insert overwrite tabl...cbucket_mapjoin_part (Stage-1), with id=dag_1515180518813_0001_35 2018-01-05T11:29:06,221 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 35, to 10.22.8.182:49818 2018-01-05T11:29:06,221 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=9, Remaining=0 2018-01-05T11:29:06,221 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=insert overwrite tabl...cbucket_mapjoin_part (Stage-1), with id=dag_1515180518813_0001_35 2018-01-05T11:29:06,222 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=35} 2018-01-05T11:29:06,222 INFO [IPC Server handler 2 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=35}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:29:06,222 INFO [IPC Server handler 2 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/35 after 300 seconds 2018-01-05T11:29:06,222 ERROR [IPC Server handler 2 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a, dag_1515180518813_0001_35 2018-01-05T11:29:06,222 INFO [IPC Server handler 2 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a 2018-01-05T11:29:06,222 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=35} = 0 2018-01-05T11:29:06,222 DEBUG [IPC Server handler 2 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=35} 2018-01-05T11:29:06,222 DEBUG [IPC Server handler 2 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:29:06,222 DEBUG [IPC Server handler 2 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=35} 2018-01-05T11:29:06,223 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,223 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,223 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,223 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 Reducer 2: 4/4 Reducer 3: 4/4 2018-01-05T11:29:06,223 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,224 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,224 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: TaskId for 000000_0 = 000000 2018-01-05T11:29:06,224 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: TaskId for 000001_0 = 000001 2018-01-05T11:29:06,224 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: TaskId for 000002_0 = 000002 2018-01-05T11:29:06,224 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: TaskId for 000003_0 = 000003 2018-01-05T11:29:06,224 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,224 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,225 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,225 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,225 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: TaskId for 000002_0 = 000002 2018-01-05T11:29:06,225 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,225 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,226 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,226 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-2:DEPENDENCY_COLLECTION] in serial mode 2018-01-05T11:29:06,226 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-0:MOVE] in serial mode 2018-01-05T11:29:06,226 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@72fe4fe8, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:29:06,226 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:06,226 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,226 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:29:06,226 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:29:06,226 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:06,226 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:06,226 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,226 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,226 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:06,226 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:06,226 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,226 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:06,226 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,226 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:06,226 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:06,226 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,226 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Loading data to table default.tab_part partition (ds=2008-04-08) from file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-05_243_5530033092111214819-1/-ext-10000 2018-01-05T11:29:06,227 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,227 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:29:06,227 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:29:06,227 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:29:06,227 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:29:06,227 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,227 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,227 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,229 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,229 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,229 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:29:06,229 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:29:06,229 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,229 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:29:06,230 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,230 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab_part 2018-01-05T11:29:06,230 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab_part 2018-01-05T11:29:06,232 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,233 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,233 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.MoveTask: Partition is: {ds=2008-04-08} 2018-01-05T11:29:06,233 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,233 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: partition_name_has_valid_characters 2018-01-05T11:29:06,233 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=partition_name_has_valid_characters 2018-01-05T11:29:06,233 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,233 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,233 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab_part 2018-01-05T11:29:06,233 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab_part 2018-01-05T11:29:06,235 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,235 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,235 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,235 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=tab_part[2008-04-08] 2018-01-05T11:29:06,235 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=tab_part[2008-04-08] 2018-01-05T11:29:06,238 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,239 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,239 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,239 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,240 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:06,240 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-05_243_5530033092111214819-1/-ext-10000/ and the destination path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/ 2018-01-05T11:29:06,240 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/ and the destination path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-05_243_5530033092111214819-1/-ext-10000/ 2018-01-05T11:29:06,240 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:29:06,240 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:29:06,260 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,260 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: altering partition for table tab_part with partition spec : {ds=2008-04-08} 2018-01-05T11:29:06,260 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: No new files were created, and is not a replace, or we're inserting into a partition that does not exist yet. Skipping generating INSERT event. 2018-01-05T11:29:06,261 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Adding new partition {ds=2008-04-08} 2018-01-05T11:29:06,261 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,261 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: add_partition : db=default tbl=tab_part 2018-01-05T11:29:06,261 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=add_partition : db=default tbl=tab_part 2018-01-05T11:29:06,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,265 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,268 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,269 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,269 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,269 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=tab_part[2008-04-08] 2018-01-05T11:29:06,269 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=tab_part[2008-04-08] 2018-01-05T11:29:06,271 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,274 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,275 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,276 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-3:STATS] in serial mode 2018-01-05T11:29:06,276 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@40ed3939, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:29:06,276 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:06,276 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,276 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:29:06,276 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:29:06,276 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:06,276 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:06,276 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,276 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,276 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:06,276 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:06,276 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,276 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:06,276 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,276 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:06,276 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:06,276 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,276 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:29:06,276 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/-mr-10001 2018-01-05T11:29:06,276 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/-mr-10001 2018-01-05T11:29:06,277 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,278 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:29:06,278 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:29:06,278 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:29:06,278 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:29:06,278 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,278 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,278 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,279 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,279 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,279 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:29:06,279 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:29:06,279 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,279 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:29:06,279 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,279 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab_part 2018-01-05T11:29:06,279 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab_part 2018-01-05T11:29:06,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,282 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=tab_part[2008-04-08] 2018-01-05T11:29:06,282 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=tab_part[2008-04-08] 2018-01-05T11:29:06,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,288 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,288 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,288 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.BasicStatsTask: Getting file stats of all partitions. threadpool size:15 2018-01-05T11:29:06,289 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.BasicStatsTask: Finished getting file stats of all partitions! 2018-01-05T11:29:06,289 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] fs.FSStatsAggregator: Part ID: default.tab_part/ds=2008-04-08/ numRows 2018-01-05T11:29:06,289 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] fs.FSStatsAggregator: Read stats for : default.tab_part/ds=2008-04-08/ numRows 500 2018-01-05T11:29:06,289 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] fs.FSStatsAggregator: Part ID: default.tab_part/ds=2008-04-08/ rawDataSize 2018-01-05T11:29:06,289 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] fs.FSStatsAggregator: Read stats for : default.tab_part/ds=2008-04-08/ rawDataSize 47000 2018-01-05T11:29:06,289 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.BasicStatsTask: Partition {ds=2008-04-08} stats: [numFiles=4, numRows=500, totalSize=3172, rawDataSize=47000] 2018-01-05T11:29:06,290 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,290 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: alter_partitions : db=default tbl=tab_part 2018-01-05T11:29:06,290 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_partitions : db=default tbl=tab_part 2018-01-05T11:29:06,290 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: New partition values:[2008-04-08] 2018-01-05T11:29:06,292 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,299 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,302 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,303 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,303 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] fs.FSStatsAggregator: About to delete stats tmp dir :file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1/-mr-10001 2018-01-05T11:29:06,318 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] mapred.FileInputFormat: Time taken to get FileStatuses: 14 2018-01-05T11:29:06,318 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:29:06,318 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 14 2018-01-05T11:29:06,318 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FetchOperator: Creating fetchTask with deserializer typeinfo: struct<_col0:struct,_col1:struct,_col2:string> 2018-01-05T11:29:06,318 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FetchOperator: deserializer properties: table properties: {columns=_col0,_col1,_col2, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct:struct:string, escape.delim=\} partition properties: {columns=_col0,_col1,_col2, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct:struct:string, escape.delim=\} 2018-01-05T11:29:06,319 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,321 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000076, QueryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a-35, DagId=35, VertexName=Reducer 3, TaskId=2, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180546116, EndTime=1515180546321 2018-01-05T11:29:06,322 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,322 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: write_partition_column_statistics: db=default table=tab_part part=ds=2008-04-08 2018-01-05T11:29:06,322 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=write_partition_column_statistics: db=default table=tab_part part=ds=2008-04-08 2018-01-05T11:29:06,328 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: Updating partition level column statistics for db=default tableName=tab_part partName=ds=2008-04-08 colName=key 2018-01-05T11:29:06,330 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: Updating partition level column statistics for db=default tableName=tab_part partName=ds=2008-04-08 colName=value 2018-01-05T11:29:06,332 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,332 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,332 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,333 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,333 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: insert overwrite table tab_part partition (ds='2008-04-08') select key,value from srcbucket_mapjoin_part 2018-01-05T11:29:06,333 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:29:06,333 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@srcbucket_mapjoin_part 2018-01-05T11:29:06,333 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@srcbucket_mapjoin_part@ds=2008-04-08 2018-01-05T11:29:06,333 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: default@tab_part@ds=2008-04-08 2018-01-05T11:29:06,333 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Lineage: tab_part PARTITION(ds=2008-04-08).key SIMPLE [(srcbucket_mapjoin_part)srcbucket_mapjoin_part.FieldSchema(name:key, type:int, comment:null), ] 2018-01-05T11:29:06,333 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Lineage: tab_part PARTITION(ds=2008-04-08).value SIMPLE [(srcbucket_mapjoin_part)srcbucket_mapjoin_part.FieldSchema(name:value, type:string, comment:null), ] 2018-01-05T11:29:06,333 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,333 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,333 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:06,333 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {alter_partitions_(String, String, List, EnvironmentContext, )=13, getTable_(String, String, )=3, getPartitionWithAuthInfo_(String, String, List, String, List, )=6, setPartitionColumnStatistics_(SetPartitionsStatsRequest, )=13} 2018-01-05T11:29:06,333 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112905_498552b4-f8b4-4615-a1e9-c164fea9519a); Time taken: 0.874 seconds 2018-01-05T11:29:06,333 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:06,333 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,333 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:06,333 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,333 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query insert overwrite table tab_part partition (ds='2008-04-08') select key,value from srcbucket_mapjoin_part 2018-01-05T11:29:06,333 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-05_243_5530033092111214819-1 2018-01-05T11:29:06,333 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-05_243_5530033092111214819-1 2018-01-05T11:29:06,334 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 1.09 seconds 2018-01-05T11:29:06,334 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:06,334 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:06,334 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:06,334 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:06,334 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,334 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:06,334 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,334 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,334 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,334 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: CREATE TABLE tab(key int, value string) PARTITIONED BY(ds STRING) CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS STORED AS ORCFILE 2018-01-05T11:29:06,334 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112906_42aae7c0-807f-4334-90f7-6b23bff556a2): CREATE TABLE tab(key int, value string) PARTITIONED BY(ds STRING) CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS STORED AS ORCFILE 2018-01-05T11:29:06,334 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,334 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: CREATE TABLE tab(key int, value string) PARTITIONED BY(ds STRING) CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS STORED AS ORCFILE 2018-01-05T11:29:06,335 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:06,335 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,335 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,335 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,335 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,335 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:06,335 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:29:06,335 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Creating table default.tab position=13 2018-01-05T11:29:06,335 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,335 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_database: default 2018-01-05T11:29:06,335 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_database: default 2018-01-05T11:29:06,335 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:29:06,335 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,335 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,335 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:06,336 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: validation start 2018-01-05T11:29:06,336 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:29:06,336 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Not a partition. 2018-01-05T11:29:06,336 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,336 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:29:06,336 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,336 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:06,336 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getDatabase_(String, )=0, flushCache_()=0} 2018-01-05T11:29:06,336 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112906_42aae7c0-807f-4334-90f7-6b23bff556a2); Time taken: 0.002 seconds 2018-01-05T11:29:06,336 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:06,336 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,336 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112906_42aae7c0-807f-4334-90f7-6b23bff556a2): CREATE TABLE tab(key int, value string) PARTITIONED BY(ds STRING) CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS STORED AS ORCFILE 2018-01-05T11:29:06,337 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,337 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: CREATE TABLE tab(key int, value string) PARTITIONED BY(ds STRING) CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS STORED AS ORCFILE 2018-01-05T11:29:06,337 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: CREATETABLE 2018-01-05T11:29:06,337 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: database:default 2018-01-05T11:29:06,337 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: default@tab 2018-01-05T11:29:06,337 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,337 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,337 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,337 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,337 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:29:06,337 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.DDLTask: Found class for org.apache.hadoop.hive.ql.io.orc.OrcSerde 2018-01-05T11:29:06,337 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.DDLTask: creating table default.tab on file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab 2018-01-05T11:29:06,337 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,337 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: create_table: Table(tableName:tab, dbName:default, owner:hive_test_user, createTime:1515180546, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:int, comment:null), FieldSchema(name:value, type:string, comment:null)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab, inputFormat:org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, compressed:false, numBuckets:2, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.ql.io.orc.OrcSerde, parameters:{serialization.format=1}), bucketCols:[key], sortCols:[Order(col:key, order:1)], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[FieldSchema(name:ds, type:string, comment:null)], parameters:{SORTBUCKETCOLSPREFIX=TRUE}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:29:06,337 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=create_table: Table(tableName:tab, dbName:default, owner:hive_test_user, createTime:1515180546, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:key, type:int, comment:null), FieldSchema(name:value, type:string, comment:null)], location:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab, inputFormat:org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, compressed:false, numBuckets:2, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.ql.io.orc.OrcSerde, parameters:{serialization.format=1}), bucketCols:[key], sortCols:[Order(col:key, order:1)], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[FieldSchema(name:ds, type:string, comment:null)], parameters:{SORTBUCKETCOLSPREFIX=TRUE}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hive_test_user=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hive_test_user, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false) 2018-01-05T11:29:06,338 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hive-site.xml 2018-01-05T11:29:06,339 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.MetastoreConf: Found configuration file file:/Users/djaiswal/hive/hive/itests/qtest/target/testconf/hivemetastore-site.xml 2018-01-05T11:29:06,339 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.MetastoreConf: Unable to find config file metastore-site.xml 2018-01-05T11:29:06,339 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.MetastoreConf: Found configuration file null 2018-01-05T11:29:06,348 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: getDatabase: directsql returning db default locn[file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse] desc [Default Hive database] owner [public] ownertype [ROLE] 2018-01-05T11:29:06,348 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: Location: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab specified for non-external table:tab 2018-01-05T11:29:06,349 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] utils.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab 2018-01-05T11:29:06,366 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,368 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,369 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,369 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.DDLTask: Ignoring request to add WriteEntity(default@tab) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@tab) Type=TABLE WriteType=DDL_NO_LOCK is present 2018-01-05T11:29:06,369 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,369 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,369 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: CREATE TABLE tab(key int, value string) PARTITIONED BY(ds STRING) CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS STORED AS ORCFILE 2018-01-05T11:29:06,369 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: CREATETABLE 2018-01-05T11:29:06,369 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: database:default 2018-01-05T11:29:06,369 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: default@tab 2018-01-05T11:29:06,369 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,369 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,369 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:06,369 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {createTable_(Table, )=32, isCompatibleWith_(Configuration, )=0} 2018-01-05T11:29:06,369 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112906_42aae7c0-807f-4334-90f7-6b23bff556a2); Time taken: 0.033 seconds 2018-01-05T11:29:06,369 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:06,369 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,369 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:06,369 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,369 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query CREATE TABLE tab(key int, value string) PARTITIONED BY(ds STRING) CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS STORED AS ORCFILE 2018-01-05T11:29:06,369 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.035 seconds 2018-01-05T11:29:06,369 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:06,369 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:06,369 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:06,369 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:06,369 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,369 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:06,369 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,369 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,369 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,369 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: insert overwrite table tab partition (ds='2008-04-08') select key,value from srcbucket_mapjoin 2018-01-05T11:29:06,369 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd): insert overwrite table tab partition (ds='2008-04-08') select key,value from srcbucket_mapjoin 2018-01-05T11:29:06,370 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,370 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: insert overwrite table tab partition (ds='2008-04-08') select key,value from srcbucket_mapjoin 2018-01-05T11:29:06,370 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:06,370 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,370 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,370 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,370 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,370 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:06,370 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:29:06,370 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed phase 1 of Semantic Analysis 2018-01-05T11:29:06,370 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:06,370 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,370 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:06,370 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:06,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,373 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:06,373 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:06,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,373 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab 2018-01-05T11:29:06,373 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab 2018-01-05T11:29:06,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,377 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=tab[2008-04-08] 2018-01-05T11:29:06,377 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=tab[2008-04-08] 2018-01-05T11:29:06,380 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,381 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,382 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,382 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed getting MetaData in Semantic Analysis 2018-01-05T11:29:06,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,384 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_not_null_constraints : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:06,384 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_not_null_constraints : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:06,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,384 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:06,384 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:06,385 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,385 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,385 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,385 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:06,385 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:06,385 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,385 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,385 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,385 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_unique_constraints : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:06,385 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_unique_constraints : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:06,385 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,386 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,386 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,386 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=srcbucket_mapjoin 2018-01-05T11:29:06,386 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=srcbucket_mapjoin 2018-01-05T11:29:06,386 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,386 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,387 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:29:06,387 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,409 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.sql2rel: Plan after trimming unused fields HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.srcbucket_mapjoin]], table:alias=[srcbucket_mapjoin]) 2018-01-05T11:29:06,409 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan before removing subquery: HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.srcbucket_mapjoin]], table:alias=[srcbucket_mapjoin]) 2018-01-05T11:29:06,409 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan just after removing subquery: HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.srcbucket_mapjoin]], table:alias=[srcbucket_mapjoin]) 2018-01-05T11:29:06,409 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan after decorrelation: HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.srcbucket_mapjoin]], table:alias=[srcbucket_mapjoin]) 2018-01-05T11:29:06,410 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,410 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,410 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,410 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,410 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,410 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,410 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,410 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,410 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,410 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,410 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,420 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,420 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,420 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,420 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,421 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.sql2rel: Plan after trimming unused fields HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.srcbucket_mapjoin]], table:alias=[srcbucket_mapjoin]) 2018-01-05T11:29:06,421 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,421 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,421 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,421 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,421 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,421 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,421 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,421 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Original plan for PlanModifier HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.srcbucket_mapjoin]], table:alias=[srcbucket_mapjoin]) 2018-01-05T11:29:06,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after nested convertOpTree HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.srcbucket_mapjoin]], table:alias=[srcbucket_mapjoin]) 2018-01-05T11:29:06,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after propagating order HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.srcbucket_mapjoin]], table:alias=[srcbucket_mapjoin]) 2018-01-05T11:29:06,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after fixTopOBSchema HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.srcbucket_mapjoin]], table:alias=[srcbucket_mapjoin]) 2018-01-05T11:29:06,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Final plan after modifier HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.srcbucket_mapjoin]], table:alias=[srcbucket_mapjoin]) 2018-01-05T11:29:06,422 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:06,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,423 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:06,423 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:06,426 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,426 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,427 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:06,427 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:06,427 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,427 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab 2018-01-05T11:29:06,427 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab 2018-01-05T11:29:06,429 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,430 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=tab[2008-04-08] 2018-01-05T11:29:06,430 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=tab[2008-04-08] 2018-01-05T11:29:06,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,435 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,435 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,435 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Table Plan for srcbucket_mapjoin TS[0] 2018-01-05T11:29:06,435 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB srcbucket_mapjoin{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB srcbucket_mapjoin{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:06,435 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (. (tok_table_or_col srcbucket_mapjoin) key) key) (tok_selexpr (. (tok_table_or_col srcbucket_mapjoin) value) value)) 2018-01-05T11:29:06,435 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = srcbucket_mapjoin{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} starRr = null 2018-01-05T11:29:06,436 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(key,_col0: int)(value,_col1: string)} 2018-01-05T11:29:06,436 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:06,436 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-06_370_8944885619275660846-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:06,436 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-06_370_8944885619275660846-1 2018-01-05T11:29:06,461 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Partition: sd is org.apache.hadoop.hive.metastore.api.StorageDescriptor 2018-01-05T11:29:06,472 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/-mr-10001 2018-01-05T11:29:06,472 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 row schema: null{(key,_col0: int)(value,_col1: string)} 2018-01-05T11:29:06,472 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,472 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab 2018-01-05T11:29:06,472 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab 2018-01-05T11:29:06,475 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,475 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,475 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Generate an operator pipeline to autogather column stats for table default.tab in query insert overwrite table tab partition (ds='2008-04-08') select key,value from srcbucket_mapjoin 2018-01-05T11:29:06,475 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: analyze table `default`.`tab` compute statistics for columns 2018-01-05T11:29:06,475 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:06,476 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,476 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab 2018-01-05T11:29:06,476 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab 2018-01-05T11:29:06,478 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,478 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,478 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: select compute_stats(`key`, 'hll') , compute_stats(`value`, 'hll') , `ds` from `default`.`tab` group by `ds` 2018-01-05T11:29:06,479 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: select compute_stats(`key`, 'hll') , compute_stats(`value`, 'hll') , `ds` from `default`.`tab` group by `ds` 2018-01-05T11:29:06,480 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:06,480 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:06,481 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:06,481 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:06,481 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,481 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab 2018-01-05T11:29:06,481 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab 2018-01-05T11:29:06,483 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,483 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,483 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:06,483 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:06,497 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Table Plan for tab TS[0] 2018-01-05T11:29:06,497 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:06,498 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: compute_stats 2018-01-05T11:29:06,498 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB tab{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB {((tok_table_or_col ds),_col0: string)((tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct)((tok_function compute_stats (tok_table_or_col value) 'hll'),_col2: struct)} tab{(ds,_col0: string)} 2018-01-05T11:29:06,498 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (tok_function compute_stats (tok_table_or_col key) 'hll')) (tok_selexpr (tok_function compute_stats (tok_table_or_col value) 'hll')) (tok_selexpr (tok_table_or_col ds))) 2018-01-05T11:29:06,498 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = {((tok_table_or_col ds),_col0: string)((tok_function compute_stats (tok_table_or_col key) 'hll'),_col1: struct)((tok_function compute_stats (tok_table_or_col value) 'hll'),_col2: struct)} tab{(ds,_col0: string)} starRr = tab{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:06,498 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(_c0,_col0: struct)(_c1,_col1: struct)(ds,_col2: string)} tab{(ds,_col2: string)} 2018-01-05T11:29:06,498 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:06,498 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_475_8915019180860607944-1/-mr-10000/.hive-staging_hive_2018-01-05_11-29-06_475_8915019180860607944-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_475_8915019180860607944-1/-mr-10000 2018-01-05T11:29:06,498 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_475_8915019180860607944-1/-mr-10000/.hive-staging_hive_2018-01-05_11-29-06_475_8915019180860607944-1 2018-01-05T11:29:06,528 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_475_8915019180860607944-1/-mr-10002 2018-01-05T11:29:06,528 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_475_8915019180860607944-1/-mr-10000 row schema: null{(_c0,_col0: struct)(_c1,_col1: struct)(ds,_col2: string)} tab{(ds,_col2: string)} 2018-01-05T11:29:06,528 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block null 2018-01-05T11:29:06,528 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:29:06,528 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block null 2018-01-05T11:29:06,528 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:29:06,528 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: CBO Succeeded; optimized logical plan. 2018-01-05T11:29:06,528 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Before logical optimization TS[0]-SEL[1]-RS[2]-SEL[3]-FS[4] -SEL[1] 2018-01-05T11:29:06,528 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,528 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,528 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,529 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] lineage.Generator: Time taken for lineage transform=1 2018-01-05T11:29:06,529 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,529 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,529 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,529 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,529 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,529 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,529 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FS(4) 2018-01-05T11:29:06,529 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FS(6) 2018-01-05T11:29:06,529 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(5) 2018-01-05T11:29:06,529 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for GBY(4) 2018-01-05T11:29:06,529 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(3) 2018-01-05T11:29:06,529 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for GBY(2) 2018-01-05T11:29:06,529 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(1) 2018-01-05T11:29:06,529 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(3) 2018-01-05T11:29:06,529 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(2) 2018-01-05T11:29:06,529 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(1) 2018-01-05T11:29:06,529 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:29:06,529 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SimplePredicatePushDown: After PPD: TS[0]-SEL[1]-RS[2]-SEL[3]-FS[4] -SEL[1] 2018-01-05T11:29:06,529 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,529 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,529 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,529 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,529 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,529 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,529 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,529 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,529 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,529 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,529 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,529 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,529 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 3 key:[Column[_col0]] 2018-01-05T11:29:06,529 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 3 oldColExprMap: {KEY._col0=Column[_col0], VALUE._col0=Column[_col1], VALUE._col1=Column[_col2]} 2018-01-05T11:29:06,529 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 3 newColExprMap: {KEY._col0=Column[_col0], VALUE._col0=Column[_col1], VALUE._col1=Column[_col2]} 2018-01-05T11:29:06,530 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 2 key:[Column[_col0]] 2018-01-05T11:29:06,530 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 2 oldColExprMap: {VALUE._col0=Column[_col1], KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:06,530 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 2 newColExprMap: {VALUE._col0=Column[_col1], KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:06,530 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,530 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,530 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,530 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,530 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,530 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,530 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,530 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,530 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,530 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,530 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,530 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,530 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,530 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,530 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,530 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,530 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,530 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,530 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,530 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,530 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,530 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,530 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,530 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: After logical optimization TS[0]-SEL[1]-RS[2]-SEL[3]-FS[4] -SEL[1] 2018-01-05T11:29:06,530 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,530 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,530 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,530 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,530 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,531 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partitions : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:06,531 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:06,535 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,536 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.28066ms + 0.008117ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? ] 2018-01-05T11:29:06,540 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 4.103673ms + 0.06913ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (12) order by "PART_NAME" asc] 2018-01-05T11:29:06,542 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 2.107137ms + 0.090312ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (12) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:29:06,544 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 2.030166ms + 0.045439ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (12) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:06,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 3.744524ms + 0.018435ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (39) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:29:06,550 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.683519ms + 0.016451ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (39) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:06,551 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.52819ms + 0.041953ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (39) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:06,553 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.613989ms + 0.014891ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (39) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:06,555 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.691043ms + 0.040005ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (25) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:06,557 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.719346ms + 0.059952ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (39) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:29:06,557 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,560 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,560 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,560 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,560 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Estimated average row size: 104 2018-01-05T11:29:06,560 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,560 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_aggr_stats_for: db=default table=srcbucket_mapjoin 2018-01-05T11:29:06,560 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_aggr_stats_for: db=default table=srcbucket_mapjoin 2018-01-05T11:29:06,562 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.AggregateStatsCache: No aggregate stats cached for database:default, table:srcbucket_mapjoin, column:key 2018-01-05T11:29:06,563 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.176685ms + 0.004881ms, the query is [select count("COLUMN_NAME") from "PART_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (?,?) and "PARTITION_NAME" in (?) group by "PARTITION_NAME"] 2018-01-05T11:29:06,563 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.113137ms + 0.003297ms, the query is [select "PARTITION_NAME", "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "PART_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (%1$s) AND "PARTITION_NAME" in (%2$s) order by "PARTITION_NAME"] 2018-01-05T11:29:06,563 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] utils.MetaStoreUtils: No stats data found for: dbName=default tblName=srcbucket_mapjoin partNames= [ds=2008-04-08] colNames=[key] 2018-01-05T11:29:06,563 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.AggregateStatsCache: No aggregate stats cached for database:default, table:srcbucket_mapjoin, column:value 2018-01-05T11:29:06,563 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.098224ms + 0.003185ms, the query is [select "PARTITION_NAME", "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "PART_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (%1$s) AND "PARTITION_NAME" in (%2$s) order by "PARTITION_NAME"] 2018-01-05T11:29:06,563 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] utils.MetaStoreUtils: No stats data found for: dbName=default tblName=srcbucket_mapjoin partNames= [ds=2008-04-08] colNames=[value] 2018-01-05T11:29:06,563 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: useDensityFunctionForNDVEstimation = false partsFound = 0 ColumnStatisticsObj = [] 2018-01-05T11:29:06,564 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,564 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting stats (Num rows: 27 Data size: 7884 Basic stats: COMPLETE Column stats: NONE) on TS[0] 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (srcbucket_mapjoin): numRows: 27 dataSize: 7884 basicStatsState: COMPLETE colStatsState: NONE colStats: {value= colName: value colType: string countDistincts: 5 numNulls: 1 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: int countDistincts: 5 numNulls: 1 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true} 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 27 Data size: 7884 Basic stats: COMPLETE Column stats: NONE) on SEL[1] 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [1] STATS-SEL[1]: numRows: 27 dataSize: 7884 basicStatsState: COMPLETE colStatsState: NONE colStats: {value= colName: value colType: string countDistincts: 5 numNulls: 1 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: int countDistincts: 5 numNulls: 1 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true} 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 27 Data size: 7884 Basic stats: COMPLETE Column stats: NONE) on RS[2] 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[2]: numRows: 27 dataSize: 7884 basicStatsState: COMPLETE colStatsState: NONE colStats: {value= colName: value colType: string countDistincts: 5 numNulls: 1 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: int countDistincts: 5 numNulls: 1 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true} 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 27 Data size: 7884 Basic stats: COMPLETE Column stats: NONE) on SEL[3] 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [1] STATS-SEL[3]: numRows: 27 dataSize: 7884 basicStatsState: COMPLETE colStatsState: NONE colStats: {value= colName: value colType: string countDistincts: 5 numNulls: 1 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: int countDistincts: 5 numNulls: 1 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true} 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-FS[4]: numRows: 27 dataSize: 7884 basicStatsState: COMPLETE colStatsState: NONE colStats: {value= colName: value colType: string countDistincts: 5 numNulls: 2 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: int countDistincts: 5 numNulls: 2 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true} 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 27 Data size: 7884 Basic stats: COMPLETE Column stats: NONE) on SEL[1] 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [1] STATS-SEL[1]: numRows: 27 dataSize: 7884 basicStatsState: COMPLETE colStatsState: NONE colStats: {value= colName: value colType: string countDistincts: 5 numNulls: 1 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: int countDistincts: 5 numNulls: 1 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true} 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[2]: inputSize: 7884 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 1] STATS-GBY[2]: cardinality: 27 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 27 Data size: 7884 Basic stats: COMPLETE Column stats: NONE) on GBY[2] 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[2]: numRows: 27 dataSize: 7884 basicStatsState: COMPLETE colStatsState: NONE colStats: {_col2= colName: _col2 colType: struct countDistincts: 27 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 27 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, value= colName: value colType: string countDistincts: 5 numNulls: 1 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: int countDistincts: 5 numNulls: 1 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true} 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 27 Data size: 7884 Basic stats: COMPLETE Column stats: NONE) on RS[3] 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[3]: numRows: 27 dataSize: 7884 basicStatsState: COMPLETE colStatsState: NONE colStats: {_col2= colName: _col2 colType: struct countDistincts: 27 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 27 numNulls: 0 avgColLen: 424.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, value= colName: value colType: string countDistincts: 5 numNulls: 1 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: int countDistincts: 5 numNulls: 1 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true} 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[4]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 7] STATS-GBY[4]: cardinality: 13 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 13 Data size: 3796 Basic stats: COMPLETE Column stats: NONE) on GBY[4] 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[4]: numRows: 13 dataSize: 3796 basicStatsState: COMPLETE colStatsState: NONE colStats: {_col2= colName: _col2 colType: struct countDistincts: 27 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 27 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, value= colName: value colType: string countDistincts: 5 numNulls: 1 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: int countDistincts: 5 numNulls: 1 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true} 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 13 Data size: 3796 Basic stats: COMPLETE Column stats: NONE) on SEL[5] 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [1] STATS-SEL[5]: numRows: 13 dataSize: 3796 basicStatsState: COMPLETE colStatsState: NONE colStats: {_col2= colName: _col2 colType: struct countDistincts: 27 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 27 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, value= colName: value colType: string countDistincts: 5 numNulls: 1 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: int countDistincts: 5 numNulls: 1 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true} 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-FS[6]: numRows: 13 dataSize: 3796 basicStatsState: COMPLETE colStatsState: NONE colStats: {_col2= colName: _col2 colType: struct countDistincts: 27 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: struct countDistincts: 27 numNulls: 0 avgColLen: 440.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, value= colName: value colType: string countDistincts: 5 numNulls: 2 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: true, key= colName: key colType: int countDistincts: 5 numNulls: 2 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: -9223372036854775808 max: 9223372036854775807 ] isPrimaryKey: false isEstimated: true} 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[]]; bucket count: 2 }) on TS[0] 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[]]; bucket count: 2 }) on SEL[1] 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[KEY.reducesinkkey0]]; sort column names: [[KEY.reducesinkkey0]]; bucket count: 2 }) on RS[2] 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: 2 }) on SEL[3] 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FileSinkOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: 2 }) on FS[4] 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[key]]; bucket count: 2 }) on SEL[1] 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: -1 }) on GBY[2] 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[KEY._col0]]; sort column names: [[KEY._col0]]; bucket count: -1 }) on RS[3] 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: -1 }) on GBY[4] 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col2]]; sort column names: [[_col2]]; bucket count: -1 }) on SEL[5] 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FileSinkOperator: Setting traits ({ bucket column names: [[_col2]]; sort column names: [[_col2]]; bucket count: -1 }) on FS[6] 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,565 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 2 2018-01-05T11:29:06,565 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Set parallelism for reduce sink RS[3] to: 1 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: FS, 4 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: FS, 6 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 5 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: GBY, 4 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 3 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: GBY, 2 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 1 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 3 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 2 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 1 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:29:06,566 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Cycle free: true 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: int|{srcbucket_mapjoin}key,value: string|{srcbucket_mapjoin}value,ds: string|{srcbucket_mapjoin}ds,BLOCK__OFFSET__INSIDE__FILE: bigint|{srcbucket_mapjoin}block__offset__inside__file,INPUT__FILE__NAME: string|{srcbucket_mapjoin}input__file__name,ROW__ID: struct|{srcbucket_mapjoin}row__id) 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(_col0: int|{null}key,_col1: string|{null}value) 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value]) 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[2] with rs:(KEY.reducesinkkey0: int|{null}key,VALUE._col0: string|{null}value) 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[2] 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[3] with rs:(_col0: int|{null}key,_col1: string|{null}value) 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_2 {} 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[3] 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[KEY.reducesinkkey0] Column[VALUE._col0]) 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[4] with rs:(key: int|{},value: string|{}) 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_3 {} 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[4] 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(key: int|{tab}key,value: string|{tab}value,ds: string|{tab}ds) 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_3 {} 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0] Column[_col1] Const string 2008-04-08) 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[2] with rs:(_col0: string|{}(tok_table_or_col ds),_col1: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {ds: string=Const string 2008-04-08} 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [_col0: string] to operator GBY[2] 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Replacing column _col0: string with constant Const string 2008-04-08 in GBY[2] 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[3] with rs:(KEY._col0: string|{}(tok_table_or_col ds),VALUE._col0: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),VALUE._col1: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_2 {_col0: string=Const string 2008-04-08} 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [KEY._col0: string] to operator RS[3] 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Replacing column KEY._col0: string with constant Const string 2008-04-08 in RS[3] 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[4] with rs:(_col0: string|{}(tok_table_or_col ds),_col1: struct|{}(tok_function compute_stats (tok_table_or_col key) 'hll'),_col2: struct|{}(tok_function compute_stats (tok_table_or_col value) 'hll')) 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_3 {KEY._col0: string=Const string 2008-04-08} 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [_col0: string] to operator GBY[4] 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Replacing column _col0: string with constant Const string 2008-04-08 in GBY[4] 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: struct|{null}_c0,_col1: struct|{null}_c1,_col2: string|{null}ds) 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_4 {_col0: string=Const string 2008-04-08} 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [_col2: string] to operator SEL[5] 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Replacing column _col2: string with constant Const string 2008-04-08 in SEL[5] 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col1] Column[_col2] Column[_col0]) 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[6] with rs:(_col0: struct|{},_col1: struct|{},_col2: string|{}) 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {_col2: string=Const string 2008-04-08} 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[6] 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: TS[0] 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[2] 2018-01-05T11:29:06,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:29:06,567 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin/ds=2008-04-08 of table srcbucket_mapjoin 2018-01-05T11:29:06,567 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin/ds=2008-04-08 2018-01-05T11:29:06,567 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[2] 2018-01-05T11:29:06,567 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: SEL[3] 2018-01-05T11:29:06,567 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: FS[4] 2018-01-05T11:29:06,567 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding reduce work (Reducer 2) for SEL[3] 2018-01-05T11:29:06,567 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Setting up reduce sink: RS[2] with following reduce work: Reducer 2 2018-01-05T11:29:06,567 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[2] as parent from SEL[3] 2018-01-05T11:29:06,567 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: FS[4] 2018-01-05T11:29:06,567 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: SEL[3] 2018-01-05T11:29:06,567 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[3] 2018-01-05T11:29:06,567 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[3] 2018-01-05T11:29:06,567 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: GBY[4] 2018-01-05T11:29:06,567 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: FS[6] 2018-01-05T11:29:06,567 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding reduce work (Reducer 3) for GBY[4] 2018-01-05T11:29:06,567 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Setting up reduce sink: RS[3] with following reduce work: Reducer 3 2018-01-05T11:29:06,567 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[3] as parent from GBY[4] 2018-01-05T11:29:06,567 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: FS[6] 2018-01-05T11:29:06,567 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,567 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab 2018-01-05T11:29:06,567 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab 2018-01-05T11:29:06,570 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,570 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,570 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:29:06,570 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,570 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,570 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:29:06,570 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:06,570 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:06,570 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:29:06,570 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:06,570 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:06,570 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:29:06,571 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Examining input format to see if vectorization is enabled. 2018-01-05T11:29:06,571 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorization is enabled for input format(s) [org.apache.hadoop.mapred.TextInputFormat] 2018-01-05T11:29:06,571 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing MapWork... 2018-01-05T11:29:06,571 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[key], Vectorized Expression = col 0:int 2018-01-05T11:29:06,571 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[value], Vectorized Expression = col 1:string 2018-01-05T11:29:06,571 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator 2018-01-05T11:29:06,571 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.SelectDesc 2018-01-05T11:29:06,571 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:int 2018-01-05T11:29:06,571 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col1], Vectorized Expression = col 1:string 2018-01-05T11:29:06,571 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:int 2018-01-05T11:29:06,571 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator groupby typeName int 2018-01-05T11:29:06,571 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkObjectHashOperator 2018-01-05T11:29:06,571 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:06,571 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkObjectHashOperator: VectorReduceSinkObjectHashOperator constructor vectorReduceSinkInfo org.apache.hadoop.hive.ql.plan.VectorReduceSinkInfo@1955a811 2018-01-05T11:29:06,571 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkObjectHashOperator 2018-01-05T11:29:06,571 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:06,571 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [key, value, ds, ROW__ID] 2018-01-05T11:29:06,571 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [int, string, string, struct] 2018-01-05T11:29:06,571 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE, NONE, NONE, NONE] 2018-01-05T11:29:06,571 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 1 2018-01-05T11:29:06,571 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 1 2018-01-05T11:29:06,571 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:06,571 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:06,571 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Using reduce tag 0 2018-01-05T11:29:06,571 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0] columnTypes=[string] 2018-01-05T11:29:06,571 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing ReduceWork... 2018-01-05T11:29:06,571 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[KEY.reducesinkkey0], Vectorized Expression = col 0:int 2018-01-05T11:29:06,571 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[VALUE._col0], Vectorized Expression = col 1:string 2018-01-05T11:29:06,571 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator 2018-01-05T11:29:06,571 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.SelectDesc 2018-01-05T11:29:06,571 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorFileSinkOperator 2018-01-05T11:29:06,571 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.FileSinkDesc 2018-01-05T11:29:06,571 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:int 2018-01-05T11:29:06,571 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col1], Vectorized Expression = col 1:string 2018-01-05T11:29:06,571 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Const string 2008-04-08, Vectorized Expression = ConstantVectorExpression(val 2008-04-08) -> 2:string 2018-01-05T11:29:06,571 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator 2018-01-05T11:29:06,571 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.SelectDesc 2018-01-05T11:29:06,571 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Cannot vectorize: Aggregation Function expression for GROUPBY operator: UDF compute_stats not supported 2018-01-05T11:29:06,571 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Using reduce tag 0 2018-01-05T11:29:06,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:29:06,572 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing ReduceWork... 2018-01-05T11:29:06,572 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Cannot vectorize: Aggregation Function expression for GROUPBY operator: UDF compute_stats not supported 2018-01-05T11:29:06,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:29:06,572 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: llap mode: only 2018-01-05T11:29:06,572 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:29:06,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key' 2018-01-05T11:29:06,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'value' 2018-01-05T11:29:06,572 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:06,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@299bafe9 can be done in LLAP 2018-01-05T11:29:06,572 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Reducer 2 2018-01-05T11:29:06,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'compute_stats(key, 'hll')' 2018-01-05T11:29:06,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key' 2018-01-05T11:29:06,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:29:06,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'compute_stats(value, 'hll')' 2018-01-05T11:29:06,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'value' 2018-01-05T11:29:06,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking ''hll'' 2018-01-05T11:29:06,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '_col0' 2018-01-05T11:29:06,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '_col1' 2018-01-05T11:29:06,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking ''2008-04-08'' 2018-01-05T11:29:06,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'KEY.reducesinkkey0' 2018-01-05T11:29:06,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:29:06,572 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:06,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@476e510 can be done in LLAP 2018-01-05T11:29:06,572 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Reducer 3 2018-01-05T11:29:06,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '_col1' 2018-01-05T11:29:06,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '_col2' 2018-01-05T11:29:06,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '_col0' 2018-01-05T11:29:06,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'compute_stats(VALUE._col0)' 2018-01-05T11:29:06,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:29:06,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'compute_stats(VALUE._col1)' 2018-01-05T11:29:06,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'VALUE._col1' 2018-01-05T11:29:06,572 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:06,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@7ff39597 can be done in LLAP 2018-01-05T11:29:06,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,572 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed plan generation 2018-01-05T11:29:06,572 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:06,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: validation start 2018-01-05T11:29:06,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: validated ds=2008-04-08 2018-01-05T11:29:06,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tab 2018-01-05T11:29:06,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,572 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partitions_ps_with_auth : db=default tbl=tab[2008-04-08] 2018-01-05T11:29:06,572 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions_ps_with_auth : db=default tbl=tab[2008-04-08] 2018-01-05T11:29:06,575 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,578 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,578 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,578 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,578 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=tab[2008-04-08] 2018-01-05T11:29:06,578 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=tab[2008-04-08] 2018-01-05T11:29:06,580 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,582 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:06,582 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:06,582 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,582 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:key, type:int, comment:null), FieldSchema(name:value, type:string, comment:null)], properties:null) 2018-01-05T11:29:06,582 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,582 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:06,582 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {getNotNullConstraints_(NotNullConstraintsRequest, )=0, isCompatibleWith_(Configuration, )=1, getAggrColStatsFor_(String, String, List, List, )=4, getTable_(String, String, )=24, flushCache_()=0, getUniqueConstraints_(UniqueConstraintsRequest, )=1, getPrimaryKeys_(PrimaryKeysRequest, )=1, listPartitions_(String, String, short, )=30, getForeignKeys_(ForeignKeysRequest, )=0, listPartitionsWithAuthInfo_(String, String, List, short, String, List, )=6} 2018-01-05T11:29:06,582 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd); Time taken: 0.213 seconds 2018-01-05T11:29:06,582 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:06,582 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,582 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd): insert overwrite table tab partition (ds='2008-04-08') select key,value from srcbucket_mapjoin 2018-01-05T11:29:06,583 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,583 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: insert overwrite table tab partition (ds='2008-04-08') select key,value from srcbucket_mapjoin 2018-01-05T11:29:06,583 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:29:06,583 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@srcbucket_mapjoin 2018-01-05T11:29:06,583 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@srcbucket_mapjoin@ds=2008-04-08 2018-01-05T11:29:06,583 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: default@tab@ds=2008-04-08 2018-01-05T11:29:06,583 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,583 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,583 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,583 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Query ID = djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd 2018-01-05T11:29:06,583 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Total jobs = 1 2018-01-05T11:29:06,583 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,583 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:29:06,583 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-1:MAPRED] in serial mode 2018-01-05T11:29:06,594 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:29:06,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:29:06,618 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:29:06,618 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:29:06,618 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd 2018-01-05T11:29:06,618 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Session is already open 2018-01-05T11:29:06,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:29:06,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,618 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Dag name: insert overwrite table t...srcbucket_mapjoin (Stage-1) 2018-01-05T11:29:06,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: DagInfo: {"context":"Hive","description":"\ninsert overwrite table tab partition (ds='2008-04-08')\nselect key,value from srcbucket_mapjoin"} 2018-01-05T11:29:06,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:29:06,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,631 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,631 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:29:06,632 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,632 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 3 size: 2.66KB 2018-01-05T11:29:06,646 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_475_8915019180860607944-1/-mr-10002 2018-01-05T11:29:06,657 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_475_8915019180860607944-1/-mr-10002 2018-01-05T11:29:06,660 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,660 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,672 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,673 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:29:06,673 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,673 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 2 size: 3.36KB 2018-01-05T11:29:06,688 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/-mr-10001 2018-01-05T11:29:06,701 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/-mr-10001 2018-01-05T11:29:06,702 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,704 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,717 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:29:06,717 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,717 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:29:06,718 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,718 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 3.13KB 2018-01-05T11:29:06,723 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin/ds=2008-04-08 2018-01-05T11:29:06,723 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,724 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,724 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,724 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=insert overwrite table t...srcbucket_mapjoin (Stage-1), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd } 2018-01-05T11:29:06,725 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:06,725 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:06,725 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:06,727 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: insert overwrite table t...srcbucket_mapjoin (Stage-1) 2018-01-05T11:29:06,727 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:29:06,727 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_36, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_36.dot 2018-01-05T11:29:06,727 INFO [IPC Server handler 0 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_36 2018-01-05T11:29:06,727 WARN [IPC Server handler 0 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:29:06,727 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Running DAG: insert overwrite table t...srcbucket_mapjoin (Stage-1), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd } 2018-01-05T11:29:06,752 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_36, submitTime=1515180546727, queueName=null 2018-01-05T11:29:06,753 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_36_00 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_36_00 [Map 1], Default Resources= 2018-01-05T11:29:06,753 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_36_00 [Map 1] 2018-01-05T11:29:06,753 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:29:06,753 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_36_00 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:06,753 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_36_01 [Reducer 2]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_36_01 [Reducer 2], Default Resources= 2018-01-05T11:29:06,753 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 2 2018-01-05T11:29:06,753 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_36_01 [Reducer 2] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:06,754 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_36_02 [Reducer 3]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_36_02 [Reducer 3], Default Resources= 2018-01-05T11:29:06,754 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_36_02 [Reducer 3] 2018-01-05T11:29:06,754 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 3 2018-01-05T11:29:06,754 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_36_02 [Reducer 3] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:06,754 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:29:06,754 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_36, initTime=1515180546754 2018-01-05T11:29:06,754 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: dag_1515180518813_0001_36 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:29:06,755 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:29:06,755 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_36, startTime=1515180546755 2018-01-05T11:29:06,755 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_36 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:29:06,755 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {srcbucket_mapjoin={InputName=srcbucket_mapjoin}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:29:06,755 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: srcbucket_mapjoin, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:29:06,755 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_36_00 [Map 1] 2018-01-05T11:29:06,755 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_36, dagName=insert overwrite table t...srcbucket_mapjoin (Stage-1) 2018-01-05T11:29:06,755 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,755 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,755 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,759 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_36_00 [Map 1] 2018-01-05T11:29:06,759 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_36_00 [Map 1] 2018-01-05T11:29:06,759 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:06,759 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_36_00 [Map 1] 2018-01-05T11:29:06,761 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:29:06,761 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@1986afd6 2018-01-05T11:29:06,761 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/djaiswal/_tez_scratch_dir/a833d035-e05c-4b95-8547-d8c5c20298ee/map.xml 2018-01-05T11:29:06,761 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/djaiswal/_tez_scratch_dir/a833d035-e05c-4b95-8547-d8c5c20298ee/map.xml 2018-01-05T11:29:06,761 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/djaiswal/_tez_scratch_dir/a833d035-e05c-4b95-8547-d8c5c20298ee/map.xml 2018-01-05T11:29:06,761 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:06,761 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:06,761 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,763 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:06,763 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.13KB 2018-01-05T11:29:06,763 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_36_00 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:29:06,764 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to ShuffleVertexManager for vertex_1515180518813_0001_36_01 [Reducer 2] 2018-01-05T11:29:06,764 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: srcbucket_mapjoin on vertex vertex_1515180518813_0001_36_00 [Map 1] 2018-01-05T11:29:06,764 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:29:06,764 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:06,764 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:29:06,764 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:06,764 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/djaiswal/_tez_scratch_dir/a833d035-e05c-4b95-8547-d8c5c20298ee/map.xml 2018-01-05T11:29:06,764 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:06,764 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias srcbucket_mapjoin 2018-01-05T11:29:06,764 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin/ds=2008-04-08 2018-01-05T11:29:06,764 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin/ds=2008-04-08 2018-01-05T11:29:06,764 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 0,1 2018-01-05T11:29:06,764 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = key,value 2018-01-05T11:29:06,764 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin/ds=2008-04-08 2018-01-05T11:29:06,765 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:06,765 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:06,765 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:06,765 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:06,769 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.05 maxFrac: 0.75 auto: false desiredTaskIput: 104857600 2018-01-05T11:29:06,769 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:29:06,769 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 2 tasks for vertex: vertex_1515180518813_0001_36_01 [Reducer 2] 2018-01-05T11:29:06,769 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_36_01 [Reducer 2] 2018-01-05T11:29:06,769 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_36_01, reconfigureDoneTime=1515180546769, numTasks=2, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:29:06,769 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:VERTEX_INITIALIZED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_36_01, initRequestedTime=1515180546763, initedTime=1515180546769, numTasks=2, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:06,769 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_36_01 [Reducer 2] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:29:06,769 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to ShuffleVertexManager for vertex_1515180518813_0001_36_02 [Reducer 3] 2018-01-05T11:29:06,773 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.05 maxFrac: 0.75 auto: false desiredTaskIput: 104857600 2018-01-05T11:29:06,773 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:29:06,773 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 4 tasks for vertex: vertex_1515180518813_0001_36_02 [Reducer 3] 2018-01-05T11:29:06,773 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_36_02 [Reducer 3] 2018-01-05T11:29:06,773 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_36_02, reconfigureDoneTime=1515180546773, numTasks=4, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:29:06,773 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_36_02 [Reducer 3], numAdditionalOutputs=1 2018-01-05T11:29:06,773 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:VERTEX_INITIALIZED]: vertexName=Reducer 3, vertexId=vertex_1515180518813_0001_36_02, initRequestedTime=1515180546769, initedTime=1515180546773, numTasks=4, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:06,773 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_36_02 [Reducer 3] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:29:06,790 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Time taken to get FileStatuses: 25 2018-01-05T11:29:06,790 INFO [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total input files to process : 2 2018-01-05T11:29:06,791 DEBUG [InputInitializer {Map 1} #0] mapred.FileInputFormat: Total # of splits generated by getSplits: 2, TimeTaken: 26 2018-01-05T11:29:06,791 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 2 2018-01-05T11:29:06,791 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:06,791 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 2. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:29:06,791 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/djaiswal/_tez_scratch_dir/a833d035-e05c-4b95-8547-d8c5c20298ee/map.xml 2018-01-05T11:29:06,791 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/djaiswal/_tez_scratch_dir/a833d035-e05c-4b95-8547-d8c5c20298ee/map.xml 2018-01-05T11:29:06,791 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/djaiswal/_tez_scratch_dir/a833d035-e05c-4b95-8547-d8c5c20298ee/map.xml 2018-01-05T11:29:06,791 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:06,791 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:06,791 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,792 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:06,792 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.13KB 2018-01-05T11:29:06,792 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin/ds=2008-04-08/srcbucket22.txt to src new group? true 2018-01-05T11:29:06,792 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin/ds=2008-04-08/srcbucket20.txt to src new group? false 2018-01-05T11:29:06,792 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:29:06,792 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:29:06,792 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:29:06,792 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 52 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 2808 Original splits: 2 2018-01-05T11:29:06,792 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired numSplits: 1 lengthPerGroup: 2808 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 2808 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false 2018-01-05T11:29:06,792 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Number of splits desired: 1 created: 1 splitsProcessed: 2 2018-01-05T11:29:06,792 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 2 grouped split count is 1, for bucket: 1 2018-01-05T11:29:06,792 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:29:06,792 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: srcbucket_mapjoin on vertex vertex_1515180518813_0001_36_00 [Map 1] 2018-01-05T11:29:06,792 INFO [App Shared Pool - #2] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_36_00 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:06,792 INFO [App Shared Pool - #2] impl.VertexImpl: Got updated RootInputsSpecs: {srcbucket_mapjoin=forAllWorkUnits=true, update=[1]} 2018-01-05T11:29:06,793 INFO [App Shared Pool - #2] impl.VertexImpl: Vertex vertex_1515180518813_0001_36_00 [Map 1] parallelism set to 1 2018-01-05T11:29:06,793 INFO [App Shared Pool - #2] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_36_00 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:06,793 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_36_00, initRequestedTime=1515180546755, initedTime=1515180546793, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:06,793 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_36_00 [Map 1] with distanceFromRoot: 0 2018-01-05T11:29:06,793 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_36_00 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:29:06,793 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_36_00, startRequestedTime=1515180546763, startedTime=1515180546793 2018-01-05T11:29:06,793 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_36_00, reconfigureDoneTime=1515180546793, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:29:06,793 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_36_00 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:06,793 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_36_00 for vertex: vertex_1515180518813_0001_36_01 [Reducer 2] numStartedSources: 1 numSources: 1 2018-01-05T11:29:06,793 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_36_01 [Reducer 2] with distanceFromRoot: 1 2018-01-05T11:29:06,793 INFO [App Shared Pool - #2] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:29:06,793 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_36_01, startRequestedTime=1515180546793, startedTime=1515180546793 2018-01-05T11:29:06,793 INFO [App Shared Pool - #2] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:29:06,793 INFO [App Shared Pool - #2] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_36_00 [Map 1] 2018-01-05T11:29:06,793 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_36_01 [Reducer 2] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:06,793 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_36_01 for vertex: vertex_1515180518813_0001_36_02 [Reducer 3] numStartedSources: 1 numSources: 1 2018-01-05T11:29:06,793 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_36_02 [Reducer 3] with distanceFromRoot: 2 2018-01-05T11:29:06,793 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_36_02, startRequestedTime=1515180546793, startedTime=1515180546793 2018-01-05T11:29:06,793 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: OnVertexStarted vertex: Reducer 2 with 0 source tasks and 2 pending tasks 2018-01-05T11:29:06,793 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_36_02 [Reducer 3] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:06,794 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Map 1 in vertex: Reducer 2 numBipartiteSourceTasks: 1 2018-01-05T11:29:06,794 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_36_00_000000, scheduledTime=1515180546793, launchTime=1515180546793 2018-01-05T11:29:06,794 INFO [App Shared Pool - #1] vertexmanager.ShuffleVertexManagerBase: OnVertexStarted vertex: Reducer 3 with 0 source tasks and 4 pending tasks 2018-01-05T11:29:06,794 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_36_00 [Map 1] 2018-01-05T11:29:06,794 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Reducer 2 in vertex: Reducer 3 numBipartiteSourceTasks: 2 2018-01-05T11:29:06,794 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_36_00_000000_0, priority=8, capability=, hosts=null 2018-01-05T11:29:06,794 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=8,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:06,794 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=8,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:06,794 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:06,794 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_36_00_000000_0 on hosts=null 2018-01-05T11:29:06,794 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_36_00_000000_0 on hosts=null 2018-01-05T11:29:06,794 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_36_00_000000_0 2018-01-05T11:29:06,795 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:06,795 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #80, task=TaskInfo{task=attempt_1515180518813_0001_36_00_000000_0, priority=8, startTime=0, containerId=null, uniqueId=79, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000080 2018-01-05T11:29:06,795 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_36_00_000000_0; false 2018-01-05T11:29:06,795 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_36_00_000000_0, priority=8, startTime=311567304, containerId=container_111101111_0001_00_000080assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=79, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:06,795 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000080 for node: 10.22.8.182:49818 2018-01-05T11:29:06,797 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 36, name=insert overwrite table t...srcbucket_mapjoin (Stage-1), queryId=djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd 2018-01-05T11:29:06,797 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000080, attempt_1515180518813_0001_36_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:06,797 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000080 for node: 10.22.8.182:49818 2018-01-05T11:29:06,797 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000080 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:06,797 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000080, launchTime=1515180546797 2018-01-05T11:29:06,797 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_36_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000080, dagName=insert overwrite table t...srcbucket_mapjoin (Stage-1), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{srcbucket_mapjoin,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{Reducer 2,org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput,2}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180546755, firstAttemptStartTime=1515180546794, currentAttemptStartTime=1515180546797} 2018-01-05T11:29:06,798 INFO [IPC Server handler 4 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000080, QueryId=djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd-36, DagId=36, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180546798 2018-01-05T11:29:06,798 DEBUG [IPC Server handler 4 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=36} with the ShuffleHandler 2018-01-05T11:29:06,798 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_36_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000080] on NM: [10.22.8.182:49818] 2018-01-05T11:29:06,798 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:06,798 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:06,798 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_36_00_000000_0, startTime=1515180546797, containerId=container_111101111_0001_00_000080, nodeId=10.22.8.182:49818 2018-01-05T11:29:06,811 INFO [IPC Server handler 4 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:29:06,811 DEBUG [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/36] 2018-01-05T11:29:06,811 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:29:06,811 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_36_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:06,811 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_36_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180546794, dagStartTime=1515180546755, withinDagPriority=8, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:29:06,811 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_36_00_000000_0 : ACCEPTED 2018-01-05T11:29:06,811 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_36_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:29:06,811 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_36_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180546794, dagStartTime=1515180546755, withinDagPriority=8, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:29:06,811 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_36_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:06,811 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_36_00_000000_0 to pre-emption queue 2018-01-05T11:29:06,811 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_36_00_000000_0: true 2018-01-05T11:29:06,811 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:1942334159 2018-01-05T11:29:06,811 INFO [TaskCommunicator # 9] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_36_00_000000_0 2018-01-05T11:29:06,811 INFO [TaskCommunicator # 9] GuaranteedTasks: Received response for attempt_1515180518813_0001_36_00_000000_0, true 2018-01-05T11:29:06,812 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : insert overwrite table t...srcbucket_mapjoin (Stage-1), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_36_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=srcbucket_mapjoin, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=Reducer 2, physicalEdgeCount=2, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ] 2018-01-05T11:29:06,812 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1026555904, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:06,812 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_36_00_000000_0 vertexName: Map 1 2018-01-05T11:29:06,813 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000080, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_36_00_000000_0, eventCount=1 } 2018-01-05T11:29:06,813 DEBUG [TezTR-518813_1_36_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_36 queryId: djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd instanceType: TaskRunner2Callable 2018-01-05T11:29:06,813 INFO [TezTR-518813_1_36_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_36_00_000000_0 2018-01-05T11:29:06,813 DEBUG [TezTR-518813_1_36_0_0_0] log.PerfLogger: 2018-01-05T11:29:06,813 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:06,814 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:06,815 INFO [IPC Server handler 18 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_36_00_000000_0 sent events: (0-1). 2018-01-05T11:29:06,815 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:29:06,815 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_36_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:29:06,815 DEBUG [TezTR-518813_1_36_0_0_0] log.PerfLogger: 2018-01-05T11:29:06,815 INFO [TezTR-518813_1_36_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:06,815 INFO [TezTR-518813_1_36_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:06,815 INFO [TezTR-518813_1_36_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:06,816 INFO [I/O Setup 0 Initialize: {srcbucket_mapjoin}] input.MRInput: srcbucket_mapjoin using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:29:06,816 INFO [I/O Setup 0 Initialize: {srcbucket_mapjoin}] input.MRInputLegacy: srcbucket_mapjoin MRInputLegacy deferring initialization 2018-01-05T11:29:06,816 INFO [TezTR-518813_1_36_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:06,816 INFO [TezTR-518813_1_36_0_0_0] resources.MemoryDistributor: InitialRequests=[srcbucket_mapjoin:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [Reducer 2:OUTPUT:10485760:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput] 2018-01-05T11:29:06,816 INFO [TezTR-518813_1_36_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:06,816 INFO [TezTR-518813_1_36_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:06,816 INFO [TezTR-518813_1_36_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 10485760, TotalRequestedScaled: 9679163.076923078, TotalJVMHeap: 1026555904, TotalAvailable: 687792455, TotalRequested/TotalJVMHeap:0.01 2018-01-05T11:29:06,816 INFO [TezTR-518813_1_36_0_0_0] resources.MemoryDistributor: Allocations=[srcbucket_mapjoin:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:10485760:10485760] 2018-01-05T11:29:06,816 INFO [TezTR-518813_1_36_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:06,816 INFO [TezTR-518813_1_36_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: srcbucket_mapjoin being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:06,816 INFO [TezTR-518813_1_36_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:29:06,816 INFO [TezTR-518813_1_36_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:06,816 INFO [I/O Setup 1 Start: {srcbucket_mapjoin}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: srcbucket_mapjoin 2018-01-05T11:29:06,816 INFO [TezTR-518813_1_36_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:06,816 INFO [TezTR-518813_1_36_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_36_00_000000_0 2018-01-05T11:29:06,816 DEBUG [TezTR-518813_1_36_0_0_0] log.PerfLogger: 2018-01-05T11:29:06,816 DEBUG [TezTR-518813_1_36_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_36_00_000000_0_10241 2018-01-05T11:29:06,816 DEBUG [TezTR-518813_1_36_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:06,816 DEBUG [TezTR-518813_1_36_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_36_00_000000_0 2018-01-05T11:29:06,816 INFO [TezTR-518813_1_36_0_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd 2018-01-05T11:29:06,816 DEBUG [TezTR-518813_1_36_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:06,816 DEBUG [TezTR-518813_1_36_0_0_0] log.PerfLogger: 2018-01-05T11:29:06,816 DEBUG [TezTR-518813_1_36_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:06,816 DEBUG [TezTR-518813_1_36_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:06,816 DEBUG [TezTR-518813_1_36_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd_Map 1__MAP_PLAN__ 2018-01-05T11:29:06,816 INFO [TezTR-518813_1_36_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/djaiswal/_tez_scratch_dir/a833d035-e05c-4b95-8547-d8c5c20298ee/map.xml 2018-01-05T11:29:06,816 DEBUG [TezTR-518813_1_36_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/djaiswal/_tez_scratch_dir/a833d035-e05c-4b95-8547-d8c5c20298ee/map.xml 2018-01-05T11:29:06,816 DEBUG [TezTR-518813_1_36_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/djaiswal/_tez_scratch_dir/a833d035-e05c-4b95-8547-d8c5c20298ee/map.xml 2018-01-05T11:29:06,816 DEBUG [TezTR-518813_1_36_0_0_0] log.PerfLogger: 2018-01-05T11:29:06,816 INFO [TezTR-518813_1_36_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:06,816 DEBUG [TezTR-518813_1_36_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,817 DEBUG [TezTR-518813_1_36_0_0_0] log.PerfLogger: 2018-01-05T11:29:06,817 INFO [TezTR-518813_1_36_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.13KB 2018-01-05T11:29:06,817 DEBUG [TezTR-518813_1_36_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:06,817 INFO [TezTR-518813_1_36_0_0_0] tez.MapRecordProcessor: The input names are: [srcbucket_mapjoin] 2018-01-05T11:29:06,817 INFO [TezTR-518813_1_36_0_0_0] input.MRInput: Initialized MRInput: srcbucket_mapjoin 2018-01-05T11:29:06,818 DEBUG [TezTR-518813_1_36_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 2 2018-01-05T11:29:06,818 INFO [TezTR-518813_1_36_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/djaiswal/_tez_scratch_dir/a833d035-e05c-4b95-8547-d8c5c20298ee/map.xml 2018-01-05T11:29:06,818 DEBUG [TezTR-518813_1_36_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:06,818 DEBUG [TezTR-518813_1_36_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin/ds=2008-04-08/srcbucket22.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcbucket_mapjoin, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct srcbucket_mapjoin { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, bucket_count=2, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin/ds=2008-04-08, columns=key,value, partition_columns.types=string, serialization.format=1, numRows=0, numFiles=2, serialization.ddl=struct srcbucket_mapjoin { i32 key, string value}, transient_lastDdlTime=1515180544, rawDataSize=0, columns.comments=, totalSize=2808, bucket_count=2, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket_mapjoin}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTOR_DESERIALIZE, input file format class name org.apache.hadoop.mapred.TextInputFormat, deserialize type LAZY_SIMPLE] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin/ds=2008-04-08=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcbucket_mapjoin, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct srcbucket_mapjoin { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, bucket_count=2, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin/ds=2008-04-08, columns=key,value, partition_columns.types=string, serialization.format=1, numRows=0, numFiles=2, serialization.ddl=struct srcbucket_mapjoin { i32 key, string value}, transient_lastDdlTime=1515180544, rawDataSize=0, columns.comments=, totalSize=2808, bucket_count=2, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket_mapjoin}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTOR_DESERIALIZE, input file format class name org.apache.hadoop.mapred.TextInputFormat, deserialize type LAZY_SIMPLE]} 2018-01-05T11:29:06,818 INFO [TezTR-518813_1_36_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/djaiswal/_tez_scratch_dir/a833d035-e05c-4b95-8547-d8c5c20298ee/map.xml 2018-01-05T11:29:06,818 DEBUG [TezTR-518813_1_36_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:06,818 DEBUG [TezTR-518813_1_36_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:29:06,818 INFO [TezTR-518813_1_36_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:29:06,818 DEBUG [TezTR-518813_1_36_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:29:06,818 INFO [TezTR-518813_1_36_0_0_0] io.HiveInputFormat: Resetting VectorMapOperatorReadType from VECTOR_DESERIALIZE for partition default.srcbucket_mapjoin {ds=2008-04-08} 2018-01-05T11:29:06,819 DEBUG [TezTR-518813_1_36_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:29:06,819 INFO [TezTR-518813_1_36_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/djaiswal/_tez_scratch_dir/a833d035-e05c-4b95-8547-d8c5c20298ee/map.xml 2018-01-05T11:29:06,819 DEBUG [TezTR-518813_1_36_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:06,819 INFO [TezTR-518813_1_36_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_36_00_000000_0 2018-01-05T11:29:06,819 INFO [TezTR-518813_1_36_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:06,819 DEBUG [TezTR-518813_1_36_0_0_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:06,819 DEBUG [TezTR-518813_1_36_0_0_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:06,819 DEBUG [TezTR-518813_1_36_0_0_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:06,819 DEBUG [TezTR-518813_1_36_0_0_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:06,819 WARN [TezTR-518813_1_36_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin/ds=2008-04-08/srcbucket22.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:29:06,820 DEBUG [TezTR-518813_1_36_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:29:06,820 INFO [TezTR-518813_1_36_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/djaiswal/_tez_scratch_dir/a833d035-e05c-4b95-8547-d8c5c20298ee/map.xml 2018-01-05T11:29:06,820 DEBUG [TezTR-518813_1_36_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:06,820 DEBUG [TezTR-518813_1_36_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin/ds=2008-04-08/srcbucket22.txt 2018-01-05T11:29:06,820 INFO [TezTR-518813_1_36_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin/ds=2008-04-08/srcbucket22.txt:0+1450, org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin/ds=2008-04-08/srcbucket20.txt:0+1358], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=2808} 2018-01-05T11:29:06,820 INFO [TezTR-518813_1_36_0_0_0] input.MRInput: srcbucket_mapjoin initialized RecordReader from event 2018-01-05T11:29:06,820 DEBUG [TezTR-518813_1_36_0_0_0] tez.MapRecordProcessor: Starting Output: Reducer 2 2018-01-05T11:29:06,820 DEBUG [IO-Elevator-Thread-8] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_36 queryId: djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd instanceType: SerDeEncodedDataReader 2018-01-05T11:29:06,820 INFO [IO-Elevator-Thread-8] LlapIoImpl: Processing data for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin/ds=2008-04-08/srcbucket22.txt 2018-01-05T11:29:06,820 INFO [IO-Elevator-Thread-8] LlapIoImpl: Processing one split {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin/ds=2008-04-08/srcbucket22.txt, 0, 1450} 2018-01-05T11:29:06,820 INFO [TezTR-518813_1_36_0_0_0] impl.ExternalSorter: Reducer 2 using: memoryMb=10, keySerializerClass=class org.apache.hadoop.hive.ql.io.HiveKey, valueSerializerClass=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization$TezBytesWritableSerializer@67bab7c, comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator@4b1e102a, partitioner=org.apache.tez.mapreduce.partition.MRPartitioner, serialization=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.hadoop.io.serializer.WritableSerialization, org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization, org.apache.hadoop.io.serializer.avro.AvroReflectSerialization, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:29:06,820 INFO [TezTR-518813_1_36_0_0_0] partition.MRPartitioner: Using oldApi, MRpartitionerClass=org.apache.hadoop.hive.ql.io.DefaultHivePartitioner 2018-01-05T11:29:06,820 INFO [TezTR-518813_1_36_0_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 2: , UsingHashComparator=true 2018-01-05T11:29:06,821 DEBUG [IO-Elevator-Thread-8] LlapIoImpl: Using LineRecordReader to read data 2018-01-05T11:29:06,821 INFO [IO-Elevator-Thread-8] LlapIoImpl: Creating VertorDeserializeOrcWriter for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin/ds=2008-04-08/srcbucket22.txt 2018-01-05T11:29:06,821 INFO [IO-Elevator-Thread-8] impl.WriterImpl: ORC writer created for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin/ds=2008-04-08/srcbucket22.txt with stripeSize: 9223372036854775807 blockSize: 9223372036854775807 compression: NONE bufferSize: 262144 2018-01-05T11:29:06,821 DEBUG [IO-Elevator-Thread-8] LlapIoImpl: Processing slice #0 null; has no cache data; has disk data 2018-01-05T11:29:06,821 DEBUG [IO-Elevator-Thread-8] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: int streamBuffers.length: 0 vectors: 1 columnEncoding: null present: false data: null dictionary: false lengths: false secondary: false tz: US/Pacific 2018-01-05T11:29:06,821 DEBUG [IO-Elevator-Thread-8] encoded.EncodedTreeReaderFactory: columnIndex: 2 columnType: string streamBuffers.length: 0 vectors: 1 columnEncoding: null present: false data: null dictionary: false lengths: false secondary: false tz: US/Pacific 2018-01-05T11:29:06,821 DEBUG [IO-Elevator-Thread-8] LlapIoOrc: Done with decode 2018-01-05T11:29:06,821 DEBUG [IO-Elevator-Thread-8] LlapIoImpl: setDone called; closed false, done false, err null, pending 1 2018-01-05T11:29:06,821 DEBUG [IO-Elevator-Thread-8] impl.StatsRecordingThreadPool: Updated stats: instance: SerDeEncodedDataReader thread name: IO-Elevator-Thread-8 thread id: 2387 scheme: file bytesRead: 1474 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:06,821 INFO [TezTR-518813_1_36_0_0_0] impl.PipelinedSorter: Newly allocated block size=10485760, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=10485760, total=10485760 2018-01-05T11:29:06,821 INFO [TezTR-518813_1_36_0_0_0] impl.PipelinedSorter: Pre allocating rest of memory buffers upfront 2018-01-05T11:29:06,821 INFO [TezTR-518813_1_36_0_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 2: , UsingHashComparator=true#blocks=1, maxMemUsage=10485760, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=10485760, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=10 2018-01-05T11:29:06,821 INFO [TezTR-518813_1_36_0_0_0] impl.PipelinedSorter: Reducer 2: reserved.remaining()=10485760, reserved.metasize=5242880 2018-01-05T11:29:06,822 INFO [TezTR-518813_1_36_0_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:29:06,822 INFO [TezTR-518813_1_36_0_0_0] vector.VectorMapOperator: Initializing operator MAP[0] 2018-01-05T11:29:06,822 DEBUG [TezTR-518813_1_36_0_0_0] vector.VectorMapOperator: Initialization Done 0 MAP 2018-01-05T11:29:06,822 DEBUG [TezTR-518813_1_36_0_0_0] vector.VectorMapOperator: Operator 0 MAP initialized 2018-01-05T11:29:06,822 DEBUG [TezTR-518813_1_36_0_0_0] vector.VectorMapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:29:06,822 INFO [TezTR-518813_1_36_0_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:29:06,822 INFO [TezTR-518813_1_36_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:06,822 DEBUG [TezTR-518813_1_36_0_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:29:06,822 INFO [TezTR-518813_1_36_0_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:29:06,822 DEBUG [TezTR-518813_1_36_0_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:29:06,822 DEBUG [TezTR-518813_1_36_0_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:29:06,822 DEBUG [TezTR-518813_1_36_0_0_0] exec.TableScanOperator: Initializing children of 0 TS 2018-01-05T11:29:06,822 DEBUG [TezTR-518813_1_36_0_0_0] vector.VectorSelectOperator: Initializing child 5 SEL 2018-01-05T11:29:06,822 INFO [TezTR-518813_1_36_0_0_0] vector.VectorSelectOperator: Initializing operator SEL[5] 2018-01-05T11:29:06,822 DEBUG [TezTR-518813_1_36_0_0_0] vector.VectorSelectOperator: Initialization Done 5 SEL 2018-01-05T11:29:06,822 DEBUG [TezTR-518813_1_36_0_0_0] vector.VectorSelectOperator: Operator 5 SEL initialized 2018-01-05T11:29:06,822 DEBUG [TezTR-518813_1_36_0_0_0] vector.VectorSelectOperator: Initializing children of 5 SEL 2018-01-05T11:29:06,822 DEBUG [TezTR-518813_1_36_0_0_0] reducesink.VectorReduceSinkObjectHashOperator: Initializing child 6 RS 2018-01-05T11:29:06,822 INFO [TezTR-518813_1_36_0_0_0] reducesink.VectorReduceSinkObjectHashOperator: Initializing operator RS[6] 2018-01-05T11:29:06,822 INFO [TezTR-518813_1_36_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/djaiswal/_tez_scratch_dir/a833d035-e05c-4b95-8547-d8c5c20298ee/map.xml 2018-01-05T11:29:06,822 DEBUG [TezTR-518813_1_36_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:06,822 INFO [TezTR-518813_1_36_0_0_0] reducesink.VectorReduceSinkCommonOperator: Using tag = -1 2018-01-05T11:29:06,822 DEBUG [TezTR-518813_1_36_0_0_0] reducesink.VectorReduceSinkObjectHashOperator: Initialization Done 6 RS 2018-01-05T11:29:06,822 DEBUG [TezTR-518813_1_36_0_0_0] reducesink.VectorReduceSinkObjectHashOperator: Operator 6 RS initialized 2018-01-05T11:29:06,822 DEBUG [TezTR-518813_1_36_0_0_0] reducesink.VectorReduceSinkObjectHashOperator: Initialization Done 6 RS done is reset. 2018-01-05T11:29:06,822 DEBUG [TezTR-518813_1_36_0_0_0] vector.VectorSelectOperator: Initialization Done 5 SEL done is reset. 2018-01-05T11:29:06,822 DEBUG [TezTR-518813_1_36_0_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:29:06,822 INFO [TezTR-518813_1_36_0_0_0] exec.OperatorUtils: Setting output collector: RS[6] --> Reducer 2 2018-01-05T11:29:06,822 DEBUG [TezTR-518813_1_36_0_0_0] log.PerfLogger: 2018-01-05T11:29:06,822 INFO [TezTR-518813_1_36_0_0_0] reducesink.VectorReduceSinkCommonOperator: RS[6]: records written - 1 2018-01-05T11:29:06,822 INFO [TezTR-518813_1_36_0_0_0] reducesink.VectorReduceSinkCommonOperator: RS[6]: records written - 10 2018-01-05T11:29:06,823 INFO [TezTR-518813_1_36_0_0_0] reducesink.VectorReduceSinkCommonOperator: RS[6]: records written - 100 2018-01-05T11:29:06,823 INFO [TezTR-518813_1_36_0_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:06,823 INFO [TezTR-518813_1_36_0_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, , , ]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=0, NUM_ERRORS=0, ROWS_EMITTED=124, METADATA_CACHE_HIT=0, METADATA_CACHE_MISS=0, CACHE_HIT_BYTES=0, CACHE_MISS_BYTES=1450, ALLOCATED_BYTES=0, ALLOCATED_USED_BYTES=0, TOTAL_IO_TIME_NS=1100742, DECODE_TIME_NS=82803, HDFS_TIME_NS=0, CONSUMER_TIME_NS=948432 ] 2018-01-05T11:29:06,824 DEBUG [TezTR-518813_1_36_0_0_0] LlapIoImpl: Encoded reader is being stopped 2018-01-05T11:29:06,824 DEBUG [TezTR-518813_1_36_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 1 of 2 2018-01-05T11:29:06,824 DEBUG [TezTR-518813_1_36_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin/ds=2008-04-08/srcbucket20.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcbucket_mapjoin, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct srcbucket_mapjoin { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, bucket_count=2, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin/ds=2008-04-08, columns=key,value, partition_columns.types=string, serialization.format=1, numRows=0, numFiles=2, serialization.ddl=struct srcbucket_mapjoin { i32 key, string value}, transient_lastDdlTime=1515180544, rawDataSize=0, columns.comments=, totalSize=2808, bucket_count=2, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket_mapjoin}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.mapred.TextInputFormat] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin/ds=2008-04-08=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={name=default.srcbucket_mapjoin, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct srcbucket_mapjoin { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, bucket_count=2, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=int:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin/ds=2008-04-08, columns=key,value, partition_columns.types=string, serialization.format=1, numRows=0, numFiles=2, serialization.ddl=struct srcbucket_mapjoin { i32 key, string value}, transient_lastDdlTime=1515180544, rawDataSize=0, columns.comments=, totalSize=2808, bucket_count=2, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, partition_columns=ds, bucket_field_name=key, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.srcbucket_mapjoin}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.mapred.TextInputFormat]} 2018-01-05T11:29:06,824 INFO [TezTR-518813_1_36_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/djaiswal/_tez_scratch_dir/a833d035-e05c-4b95-8547-d8c5c20298ee/map.xml 2018-01-05T11:29:06,824 DEBUG [TezTR-518813_1_36_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:06,824 DEBUG [TezTR-518813_1_36_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:29:06,824 INFO [TezTR-518813_1_36_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:29:06,824 DEBUG [TezTR-518813_1_36_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:29:06,825 DEBUG [TezTR-518813_1_36_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:29:06,825 INFO [TezTR-518813_1_36_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/djaiswal/_tez_scratch_dir/a833d035-e05c-4b95-8547-d8c5c20298ee/map.xml 2018-01-05T11:29:06,825 DEBUG [TezTR-518813_1_36_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:06,825 INFO [TezTR-518813_1_36_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_36_00_000000_0 2018-01-05T11:29:06,825 INFO [TezTR-518813_1_36_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:06,825 WARN [TezTR-518813_1_36_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin/ds=2008-04-08/srcbucket20.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:29:06,825 DEBUG [TezTR-518813_1_36_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:29:06,825 INFO [TezTR-518813_1_36_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/djaiswal/_tez_scratch_dir/a833d035-e05c-4b95-8547-d8c5c20298ee/map.xml 2018-01-05T11:29:06,825 DEBUG [TezTR-518813_1_36_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:06,825 DEBUG [TezTR-518813_1_36_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin/ds=2008-04-08/srcbucket20.txt 2018-01-05T11:29:06,825 DEBUG [IO-Elevator-Thread-9] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_36 queryId: djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd instanceType: SerDeEncodedDataReader 2018-01-05T11:29:06,825 INFO [IO-Elevator-Thread-9] LlapIoImpl: Processing data for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin/ds=2008-04-08/srcbucket20.txt 2018-01-05T11:29:06,825 INFO [IO-Elevator-Thread-9] LlapIoImpl: Processing one split {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin/ds=2008-04-08/srcbucket20.txt, 0, 1358} 2018-01-05T11:29:06,826 DEBUG [IO-Elevator-Thread-9] LlapIoImpl: Using LineRecordReader to read data 2018-01-05T11:29:06,826 INFO [IO-Elevator-Thread-9] LlapIoImpl: Creating VertorDeserializeOrcWriter for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin/ds=2008-04-08/srcbucket20.txt 2018-01-05T11:29:06,826 INFO [IO-Elevator-Thread-9] impl.WriterImpl: ORC writer created for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin/ds=2008-04-08/srcbucket20.txt with stripeSize: 9223372036854775807 blockSize: 9223372036854775807 compression: NONE bufferSize: 262144 2018-01-05T11:29:06,826 DEBUG [IO-Elevator-Thread-9] LlapIoImpl: Processing slice #0 null; has no cache data; has disk data 2018-01-05T11:29:06,826 DEBUG [IO-Elevator-Thread-9] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: int streamBuffers.length: 0 vectors: 1 columnEncoding: null present: false data: null dictionary: false lengths: false secondary: false tz: US/Pacific 2018-01-05T11:29:06,826 DEBUG [IO-Elevator-Thread-9] encoded.EncodedTreeReaderFactory: columnIndex: 2 columnType: string streamBuffers.length: 0 vectors: 1 columnEncoding: null present: false data: null dictionary: false lengths: false secondary: false tz: US/Pacific 2018-01-05T11:29:06,826 DEBUG [IO-Elevator-Thread-9] LlapIoOrc: Done with decode 2018-01-05T11:29:06,826 DEBUG [IO-Elevator-Thread-9] LlapIoImpl: setDone called; closed false, done false, err null, pending 0 2018-01-05T11:29:06,827 DEBUG [IO-Elevator-Thread-9] impl.StatsRecordingThreadPool: Updated stats: instance: SerDeEncodedDataReader thread name: IO-Elevator-Thread-9 thread id: 2389 scheme: file bytesRead: 1382 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:06,827 INFO [TezTR-518813_1_36_0_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:06,827 INFO [TezTR-518813_1_36_0_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, , , ]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=0, NUM_ERRORS=0, ROWS_EMITTED=118, METADATA_CACHE_HIT=0, METADATA_CACHE_MISS=0, CACHE_HIT_BYTES=0, CACHE_MISS_BYTES=1358, ALLOCATED_BYTES=0, ALLOCATED_USED_BYTES=0, TOTAL_IO_TIME_NS=850023, DECODE_TIME_NS=74006, HDFS_TIME_NS=0, CONSUMER_TIME_NS=834730 ] 2018-01-05T11:29:06,827 DEBUG [TezTR-518813_1_36_0_0_0] LlapIoImpl: Encoded reader is being stopped 2018-01-05T11:29:06,827 DEBUG [TezTR-518813_1_36_0_0_0] log.PerfLogger: 2018-01-05T11:29:06,827 DEBUG [TezTR-518813_1_36_0_0_0] mr.ObjectCache: djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:29:06,827 DEBUG [TezTR-518813_1_36_0_0_0] vector.VectorMapOperator: close called for operator MAP[0] 2018-01-05T11:29:06,827 INFO [TezTR-518813_1_36_0_0_0] vector.VectorMapOperator: Closing operator MAP[0] 2018-01-05T11:29:06,827 INFO [TezTR-518813_1_36_0_0_0] vector.VectorMapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:242, 2018-01-05T11:29:06,827 DEBUG [TezTR-518813_1_36_0_0_0] vector.VectorMapOperator: Closing child = TS[0] 2018-01-05T11:29:06,827 DEBUG [TezTR-518813_1_36_0_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:29:06,827 INFO [TezTR-518813_1_36_0_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:29:06,827 DEBUG [TezTR-518813_1_36_0_0_0] exec.TableScanOperator: Closing child = SEL[5] 2018-01-05T11:29:06,827 DEBUG [TezTR-518813_1_36_0_0_0] vector.VectorSelectOperator: close called for operator SEL[5] 2018-01-05T11:29:06,827 DEBUG [TezTR-518813_1_36_0_0_0] vector.VectorSelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:06,827 INFO [TezTR-518813_1_36_0_0_0] vector.VectorSelectOperator: Closing operator SEL[5] 2018-01-05T11:29:06,827 DEBUG [TezTR-518813_1_36_0_0_0] vector.VectorSelectOperator: Closing child = RS[6] 2018-01-05T11:29:06,827 DEBUG [TezTR-518813_1_36_0_0_0] reducesink.VectorReduceSinkObjectHashOperator: close called for operator RS[6] 2018-01-05T11:29:06,827 DEBUG [TezTR-518813_1_36_0_0_0] reducesink.VectorReduceSinkObjectHashOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:06,827 INFO [TezTR-518813_1_36_0_0_0] reducesink.VectorReduceSinkObjectHashOperator: Closing operator RS[6] 2018-01-05T11:29:06,827 INFO [TezTR-518813_1_36_0_0_0] reducesink.VectorReduceSinkCommonOperator: RS[6]: records written - 242 2018-01-05T11:29:06,827 INFO [TezTR-518813_1_36_0_0_0] reducesink.VectorReduceSinkObjectHashOperator: RECORDS_OUT_INTERMEDIATE_Map_1:242, 2018-01-05T11:29:06,827 DEBUG [TezTR-518813_1_36_0_0_0] reducesink.VectorReduceSinkObjectHashOperator: 6 Close done 2018-01-05T11:29:06,827 DEBUG [TezTR-518813_1_36_0_0_0] vector.VectorSelectOperator: 5 Close done 2018-01-05T11:29:06,827 DEBUG [TezTR-518813_1_36_0_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:29:06,827 DEBUG [TezTR-518813_1_36_0_0_0] vector.VectorMapOperator: 0 Close done 2018-01-05T11:29:06,827 INFO [TezTR-518813_1_36_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_36_00_000000_0 2018-01-05T11:29:06,827 INFO [TezTR-518813_1_36_0_0_0] impl.PipelinedSorter: Reducer 2: Starting flush of map output 2018-01-05T11:29:06,827 INFO [TezTR-518813_1_36_0_0_0] impl.PipelinedSorter: Reducer 2: Span0.length = 242, perItem = 14 2018-01-05T11:29:06,828 INFO [TezTR-518813_1_36_0_0_0] impl.PipelinedSorter: Reducer 2: done sorting span=0, length=242, time=1 2018-01-05T11:29:06,828 INFO [TezTR-518813_1_36_0_0_0] impl.PipelinedSorter: Reducer 2: Heap = SpanIterator<0:241> (span=Span[5242880,3582]), 2018-01-05T11:29:06,831 INFO [IO-Elevator-Thread-8-OrcEncode] LlapIoImpl: ORC encoder received a exit event 2018-01-05T11:29:06,831 DEBUG [IO-Elevator-Thread-8-OrcEncode] LlapIoImpl: Discarding disk data (if any wasn't cached) 2018-01-05T11:29:06,837 INFO [IO-Elevator-Thread-9-OrcEncode] LlapIoImpl: ORC encoder received a exit event 2018-01-05T11:29:06,837 DEBUG [IO-Elevator-Thread-9-OrcEncode] LlapIoImpl: Discarding disk data (if any wasn't cached) 2018-01-05T11:29:06,844 INFO [TezTR-518813_1_36_0_0_0] impl.PipelinedSorter: Reducer 2: Spilling to /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/36/output/attempt_1515180518813_0001_36_00_000000_0_10243_0/file.out 2018-01-05T11:29:06,859 DEBUG [TezTR-518813_1_36_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:06,859 INFO [TezTR-518813_1_36_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:06,859 INFO [TezTR-518813_1_36_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_36_00_000000_0, askedToStop=false 2018-01-05T11:29:06,859 INFO [TezTR-518813_1_36_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_36_00_000000_0, stopRequested=false 2018-01-05T11:29:06,859 INFO [TezTR-518813_1_36_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_36_00_000000_0: Counters: 27 [[File System Counters FILE_BYTES_READ=2856, FILE_BYTES_WRITTEN=0, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=242, INPUT_RECORDS_PROCESSED=2, INPUT_SPLIT_LENGTH_BYTES=2808, OUTPUT_RECORDS=242, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=3582, OUTPUT_BYTES_WITH_OVERHEAD=3583, OUTPUT_BYTES_PHYSICAL=3591, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=242, RECORDS_OUT_INTERMEDIATE_Map_1=242][org.apache.hadoop.hive.llap.counters.LlapIOCounters CACHE_MISS_BYTES=2808, CONSUMER_TIME_NS=1783162, DECODE_TIME_NS=156809, NUM_DECODED_BATCHES=2, NUM_VECTOR_BATCHES=2, ROWS_EMITTED=242, TOTAL_IO_TIME_NS=1950765]] 2018-01-05T11:29:06,859 DEBUG [TezTR-518813_1_36_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_36_0_0_0 thread id: 2381 scheme: file bytesRead: 112 bytesWritten: 3647 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:06,859 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_36_00_000000_0, isTaskDone=true 2018-01-05T11:29:06,859 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000080, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_36_00_000000_0, eventCount=4 } 2018-01-05T11:29:06,860 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_36_00_000000_0, creationTime=1515180546794, allocationTime=1515180546795, startTime=1515180546797, finishTime=1515180546860, timeTaken=63, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:06,860 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_36_00_000000_0 2018-01-05T11:29:06,860 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:29:06,860 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_36_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:06,860 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_36_00_000000_0; false 2018-01-05T11:29:06,860 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_36_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:06,860 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_36_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:06,860 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_36_00_000000_0 2018-01-05T11:29:06,860 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_36_00_000000, startTime=1515180546797, finishTime=1515180546860, timeTaken=63, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_36_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2968, FILE_BYTES_WRITTEN=3647, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=242, INPUT_RECORDS_PROCESSED=2, INPUT_SPLIT_LENGTH_BYTES=2808, OUTPUT_RECORDS=242, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=3582, OUTPUT_BYTES_WITH_OVERHEAD=3583, OUTPUT_BYTES_PHYSICAL=3591, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=242, RECORDS_OUT_INTERMEDIATE_Map_1=242, org.apache.hadoop.hive.llap.counters.LlapIOCounters, CACHE_MISS_BYTES=2808, CONSUMER_TIME_NS=1783162, DECODE_TIME_NS=156809, NUM_DECODED_BATCHES=2, NUM_VECTOR_BATCHES=2, ROWS_EMITTED=242, TOTAL_IO_TIME_NS=1950765 2018-01-05T11:29:06,860 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000080=48 2018-01-05T11:29:06,860 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:06,860 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_36_00_000000_0: true 2018-01-05T11:29:06,860 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:06,860 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_36_00_000000_0 2018-01-05T11:29:06,860 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000080, QueryId=djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd-36, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:06,860 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_36_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:29:06,860 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_36_00_000000_0 2018-01-05T11:29:06,860 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_36_00 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:06,860 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_36_00 [Map 1] 2018-01-05T11:29:06,860 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_36_00, initRequestedTime=1515180546755, initedTime=1515180546793, startRequestedTime=1515180546763, startedTime=1515180546793, finishTime=1515180546860, timeTaken=67, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2968, FILE_BYTES_WRITTEN=3647, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=242, INPUT_RECORDS_PROCESSED=2, INPUT_SPLIT_LENGTH_BYTES=2808, OUTPUT_RECORDS=242, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=3582, OUTPUT_BYTES_WITH_OVERHEAD=3583, OUTPUT_BYTES_PHYSICAL=3591, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=242, RECORDS_OUT_INTERMEDIATE_Map_1=242, org.apache.hadoop.hive.llap.counters.LlapIOCounters, CACHE_MISS_BYTES=2808, CONSUMER_TIME_NS=1783162, DECODE_TIME_NS=156809, NUM_DECODED_BATCHES=2, NUM_VECTOR_BATCHES=2, ROWS_EMITTED=242, TOTAL_IO_TIME_NS=1950765, vertexStats=firstTaskStartTime=1515180546797, firstTasksToStart=[ task_1515180518813_0001_36_00_000000 ], lastTaskFinishTime=1515180546860, lastTasksToFinish=[ task_1515180518813_0001_36_00_000000 ], minTaskDuration=63, maxTaskDuration=63, avgTaskDuration=63.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_36_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_36_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:06,861 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_36_00 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:06,861 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000080, stoppedTime=1515180546861, exitStatus=0 2018-01-05T11:29:06,861 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000080 exited with diagnostics set to Container completed. 2018-01-05T11:29:06,861 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_36_01 [Reducer 2] attempt: attempt_1515180518813_0001_36_00_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:29:06,861 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_36_00 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3 2018-01-05T11:29:06,861 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null 2018-01-05T11:29:06,861 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000080 2018-01-05T11:29:06,861 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: All source tasks completed. Ramping up 2 remaining tasks for vertex: Reducer 2 2018-01-05T11:29:06,861 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: Scheduling 2 tasks for vertex: Reducer 2 with totalTasks: 2. 1 source tasks completed out of 1. MinSourceTaskCompletedFraction: 1.0 min: 0.05 max: 0.75 2018-01-05T11:29:06,861 INFO [App Shared Pool - #2] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Map 1, destinationVertexName=Reducer 2. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:29:06,861 INFO [App Shared Pool - #2] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_36_01 [Reducer 2] 2018-01-05T11:29:06,861 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_36_01_000000, scheduledTime=1515180546861, launchTime=1515180546861 2018-01-05T11:29:06,861 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_36_01_000001, scheduledTime=1515180546861, launchTime=1515180546861 2018-01-05T11:29:06,861 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_36_01 [Reducer 2] 2018-01-05T11:29:06,861 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_36_01_000000_0, priority=20, capability=, hosts=null 2018-01-05T11:29:06,861 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=20,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:06,861 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=20,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:06,861 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_36_01_000001_0, priority=20, capability=, hosts=null 2018-01-05T11:29:06,861 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:06,861 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_36_01_000000_0 on hosts=null 2018-01-05T11:29:06,861 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_36_01_000000_0 on hosts=null 2018-01-05T11:29:06,861 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_36_01_000000_0 2018-01-05T11:29:06,861 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:06,861 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #81, task=TaskInfo{task=attempt_1515180518813_0001_36_01_000000_0, priority=20, startTime=0, containerId=null, uniqueId=80, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000081 2018-01-05T11:29:06,861 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_36_01_000000_0; false 2018-01-05T11:29:06,861 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_36_01_000000_0, priority=20, startTime=311567371, containerId=container_111101111_0001_00_000081assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=80, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:06,861 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=20,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:06,862 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=20,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:06,862 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:06,862 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_36_01_000001_0 on hosts=null 2018-01-05T11:29:06,862 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_36_01_000001_0 on hosts=null 2018-01-05T11:29:06,862 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_36_01_000001_0 2018-01-05T11:29:06,862 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:06,862 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #82, task=TaskInfo{task=attempt_1515180518813_0001_36_01_000001_0, priority=20, startTime=0, containerId=null, uniqueId=81, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, to container=container_111101111_0001_00_000082 2018-01-05T11:29:06,862 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_36_01_000001_0; false 2018-01-05T11:29:06,862 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_36_01_000001_0, priority=20, startTime=311567371, containerId=container_111101111_0001_00_000082assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=2, ac=6, commF=false, disabled=false}, uniqueId=81, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:06,862 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000081 for node: 10.22.8.182:49818 2018-01-05T11:29:06,862 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000081, attempt_1515180518813_0001_36_01_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:06,862 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000081 for node: 10.22.8.182:49818 2018-01-05T11:29:06,862 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000082 for node: 10.22.8.182:49818 2018-01-05T11:29:06,862 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000082, attempt_1515180518813_0001_36_01_000001_0 for node: 10.22.8.182:49818 2018-01-05T11:29:06,862 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000082 for node: 10.22.8.182:49818 2018-01-05T11:29:06,862 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=36}, sourceName=Map 1, state=S_SUCCEEDED 2018-01-05T11:29:06,862 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000081 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:06,862 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000081, launchTime=1515180546862 2018-01-05T11:29:06,863 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_36_01_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000081, dagName=insert overwrite table t...srcbucket_mapjoin (Stage-1), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.OrderedGroupedKVInput,1}}, Outputs={{Reducer 3,org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput,4}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=3, completedTaskCount=1, dagStartTime=1515180546755, firstAttemptStartTime=1515180546861, currentAttemptStartTime=1515180546862} 2018-01-05T11:29:06,863 INFO [IPC Server handler 1 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000081, QueryId=djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd-36, DagId=36, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, SubmitTime=1515180546863 2018-01-05T11:29:06,863 DEBUG [IPC Server handler 1 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=36} with the ShuffleHandler 2018-01-05T11:29:06,863 DEBUG [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/36] 2018-01-05T11:29:06,863 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_36_01_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000081] on NM: [10.22.8.182:49818] 2018-01-05T11:29:06,863 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:06,863 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:06,863 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_36_01_000000_0, startTime=1515180546862, containerId=container_111101111_0001_00_000081, nodeId=10.22.8.182:49818 2018-01-05T11:29:06,863 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000082 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:06,863 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000082, launchTime=1515180546863 2018-01-05T11:29:06,863 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:29:06,863 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_36_01_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:06,863 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_36_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180546861, dagStartTime=1515180546755, withinDagPriority=20, vertexParallelism= 2, selfAndUpstreamParallelism= 3, selfAndUpstreamComplete= 1}] 2018-01-05T11:29:06,863 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_36_01_000000_0 : ACCEPTED 2018-01-05T11:29:06,863 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_36_01_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:29:06,863 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_36_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180546861, dagStartTime=1515180546755, withinDagPriority=20, vertexParallelism= 2, selfAndUpstreamParallelism= 3, selfAndUpstreamComplete= 1} 2018-01-05T11:29:06,863 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_36_01_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:06,863 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_36_01_000000_0 to pre-emption queue 2018-01-05T11:29:06,863 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_36_01_000000_0: true 2018-01-05T11:29:06,864 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:1942334159 2018-01-05T11:29:06,864 INFO [TaskCommunicator # 1] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_36_01_000000_0 2018-01-05T11:29:06,864 INFO [TaskCommunicator # 1] GuaranteedTasks: Received response for attempt_1515180518813_0001_36_01_000000_0, true 2018-01-05T11:29:06,864 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_36_01_000001_0] submitted. Is using containerId: [container_111101111_0001_00_000082] on NM: [10.22.8.182:49818] 2018-01-05T11:29:06,864 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:06,864 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:06,864 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_36_01_000001_0, startTime=1515180546863, containerId=container_111101111_0001_00_000082, nodeId=10.22.8.182:49818 2018-01-05T11:29:06,864 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : insert overwrite table t...srcbucket_mapjoin (Stage-1), VertexName: Reducer 2, VertexParallelism: 2, TaskAttemptID:attempt_1515180518813_0001_36_01_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=Reducer 3, physicalEdgeCount=4, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ] 2018-01-05T11:29:06,864 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1026555904, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:06,864 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_36_01_000000_0 vertexName: Reducer 2 2018-01-05T11:29:06,864 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_36_01_000001_0, am_details=10.22.8.182:49824, taskInfo= fragment 1 attempt 0, user=djaiswal, queryId=djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000082, dagName=insert overwrite table t...srcbucket_mapjoin (Stage-1), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.OrderedGroupedKVInput,1}}, Outputs={{Reducer 3,org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput,4}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=3, completedTaskCount=1, dagStartTime=1515180546755, firstAttemptStartTime=1515180546861, currentAttemptStartTime=1515180546862} 2018-01-05T11:29:06,864 INFO [IPC Server handler 0 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000082, QueryId=djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd-36, DagId=36, VertexName=Reducer 2, TaskId=1, TaskAttemptId=0, SubmitTime=1515180546864 2018-01-05T11:29:06,864 DEBUG [IPC Server handler 0 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=36} with the ShuffleHandler 2018-01-05T11:29:06,864 DEBUG [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/36] 2018-01-05T11:29:06,864 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000081, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_36_01_000000_0, eventCount=1 } 2018-01-05T11:29:06,864 DEBUG [TezTR-518813_1_36_1_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_36 queryId: djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd instanceType: TaskRunner2Callable 2018-01-05T11:29:06,864 INFO [TezTR-518813_1_36_1_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_36_01_000000_0 2018-01-05T11:29:06,865 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=3, runningFragmentCount=1 2018-01-05T11:29:06,865 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_36_01_000001_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:06,865 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_36_01_000001_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180546861, dagStartTime=1515180546755, withinDagPriority=20, vertexParallelism= 2, selfAndUpstreamParallelism= 3, selfAndUpstreamComplete= 1}] 2018-01-05T11:29:06,865 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_36_01_000001_0 : ACCEPTED 2018-01-05T11:29:06,865 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_36_01_000001_0, canFinish=true. Current state: preemptionQueueSize=1, numSlotsAvailable=3, waitQueueSize=1 2018-01-05T11:29:06,865 DEBUG [TezTR-518813_1_36_1_0_0] log.PerfLogger: 2018-01-05T11:29:06,865 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_36_01_000001_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180546861, dagStartTime=1515180546755, withinDagPriority=20, vertexParallelism= 2, selfAndUpstreamParallelism= 3, selfAndUpstreamComplete= 1} 2018-01-05T11:29:06,865 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_36_01_000001_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:06,865 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_36_01_000001_0 to pre-emption queue 2018-01-05T11:29:06,865 INFO [I/O Setup 0 Initialize: {Map 1}] orderedgrouped.MergeManager: Initial Memory required for SHUFFLE_BUFFER=131404920 based on INPUT_BUFFER_FACTOR=0.7, for final merged output=0, using factor: 0.0 2018-01-05T11:29:06,865 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_36_01_000001_0: true 2018-01-05T11:29:06,865 INFO [TaskCommunicator # 3] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_36_01_000001_0 2018-01-05T11:29:06,865 INFO [TaskCommunicator # 3] GuaranteedTasks: Received response for attempt_1515180518813_0001_36_01_000001_0, true 2018-01-05T11:29:06,865 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: taskOwner hashCode:140744346 2018-01-05T11:29:06,866 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:06,866 INFO [Task-Executor-2] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : insert overwrite table t...srcbucket_mapjoin (Stage-1), VertexName: Reducer 2, VertexParallelism: 2, TaskAttemptID:attempt_1515180518813_0001_36_01_000001_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=Reducer 3, physicalEdgeCount=4, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ] 2018-01-05T11:29:06,866 INFO [Task-Executor-2] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1026555904, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:06,866 INFO [Task-Executor-2] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_36_01_000001_0 vertexName: Reducer 2 2018-01-05T11:29:06,867 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:06,867 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000082, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_36_01_000001_0, eventCount=1 } 2018-01-05T11:29:06,867 DEBUG [TezTR-518813_1_36_1_1_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_36 queryId: djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd instanceType: TaskRunner2Callable 2018-01-05T11:29:06,867 INFO [TezTR-518813_1_36_1_1_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_36_01_000001_0 2018-01-05T11:29:06,868 INFO [IPC Server handler 15 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_36_01_000000_0 1 events [0,1) total 1 vertex_1515180518813_0001_36_01 [Reducer 2] 2018-01-05T11:29:06,868 INFO [I/O Setup 0 Initialize: {Map 1}] orderedgrouped.MergeManager: Initial Memory required for SHUFFLE_BUFFER=131404920 based on INPUT_BUFFER_FACTOR=0.7, for final merged output=0, using factor: 0.0 2018-01-05T11:29:06,868 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:06,868 DEBUG [TezTR-518813_1_36_1_1_0] log.PerfLogger: 2018-01-05T11:29:06,868 DEBUG [TezTR-518813_1_36_1_0_0] log.PerfLogger: 2018-01-05T11:29:06,869 INFO [TezTR-518813_1_36_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:06,869 INFO [TezTR-518813_1_36_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:06,869 INFO [TezTR-518813_1_36_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:06,869 INFO [TezTR-518813_1_36_1_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:06,869 INFO [TezTR-518813_1_36_1_0_0] resources.MemoryDistributor: InitialRequests=[Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [Reducer 3:OUTPUT:10485760:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput] 2018-01-05T11:29:06,869 INFO [TezTR-518813_1_36_1_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:06,869 INFO [TezTR-518813_1_36_1_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:06,869 INFO [TezTR-518813_1_36_1_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 24, TotalRequested: 141890680, TotalRequestedScaled: 7.094534E7, TotalJVMHeap: 1026555904, TotalAvailable: 687792455, TotalRequested/TotalJVMHeap:0.14 2018-01-05T11:29:06,869 INFO [TezTR-518813_1_36_1_0_0] resources.MemoryDistributor: Allocations=[Map 1:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:131404920:131404920], [Reducer 3:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:10485760:10485760] 2018-01-05T11:29:06,869 INFO [TezTR-518813_1_36_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:06,869 INFO [TezTR-518813_1_36_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: Map 1 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:06,869 INFO [TezTR-518813_1_36_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:29:06,869 INFO [TezTR-518813_1_36_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:06,869 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:06,869 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:29:06,869 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_36_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:29:06,870 DEBUG [I/O Setup 0 Start: {Map 1}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:06,870 DEBUG [I/O Setup 0 Start: {Map 1}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:06,870 DEBUG [I/O Setup 0 Start: {Map 1}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:06,870 DEBUG [I/O Setup 0 Start: {Map 1}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:06,870 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.Shuffle: Map_1: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true 2018-01-05T11:29:06,870 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.MergeManager: Map 1: MergerManager: memoryLimit=131404920, maxSingleShuffleLimit=32851230, mergeThreshold=86727248, ioSortFactor=10, postMergeMem=0, memToMemMergeOutputsThreshold=10 2018-01-05T11:29:06,870 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.ShuffleScheduler: ShuffleScheduler running for sourceVertex: Map 1 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true 2018-01-05T11:29:06,871 INFO [IPC Server handler 17 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_36_01_000001_0 1 events [0,1) total 1 vertex_1515180518813_0001_36_01 [Reducer 2] 2018-01-05T11:29:06,871 INFO [I/O Setup 0 Start: {Map 1}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: Map 1 2018-01-05T11:29:06,872 INFO [TezTR-518813_1_36_1_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:06,872 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:29:06,872 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_36_01_000001_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:29:06,872 INFO [TezTR-518813_1_36_1_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_36_01_000000_0 2018-01-05T11:29:06,872 DEBUG [TezTR-518813_1_36_1_0_0] log.PerfLogger: 2018-01-05T11:29:06,872 DEBUG [TezTR-518813_1_36_1_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_36_01_000000_0_10244 2018-01-05T11:29:06,872 DEBUG [TezTR-518813_1_36_1_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:06,872 DEBUG [TezTR-518813_1_36_1_0_0] mr.ObjectCache: Creating djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:29:06,872 INFO [TezTR-518813_1_36_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/djaiswal/_tez_scratch_dir/bbe41d34-2ea1-432a-aa2b-305b2dc82600/reduce.xml 2018-01-05T11:29:06,872 DEBUG [TezTR-518813_1_36_1_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/djaiswal/_tez_scratch_dir/bbe41d34-2ea1-432a-aa2b-305b2dc82600/reduce.xml 2018-01-05T11:29:06,872 DEBUG [TezTR-518813_1_36_1_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/djaiswal/_tez_scratch_dir/bbe41d34-2ea1-432a-aa2b-305b2dc82600/reduce.xml 2018-01-05T11:29:06,872 INFO [TezTaskEventRouter{attempt_1515180518813_0001_36_01_000000_0}] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:29:06,872 DEBUG [TezTR-518813_1_36_1_0_0] log.PerfLogger: 2018-01-05T11:29:06,872 INFO [TezTR-518813_1_36_1_0_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:29:06,872 DEBUG [TezTR-518813_1_36_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,873 DEBUG [TezTR-518813_1_36_1_1_0] log.PerfLogger: 2018-01-05T11:29:06,873 INFO [TezTR-518813_1_36_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:06,873 INFO [TezTR-518813_1_36_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:06,873 INFO [TezTR-518813_1_36_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:06,873 INFO [TezTR-518813_1_36_1_1_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:06,873 INFO [TezTR-518813_1_36_1_1_0] resources.MemoryDistributor: InitialRequests=[Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [Reducer 3:OUTPUT:10485760:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput] 2018-01-05T11:29:06,873 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=36&reduce=0&map=attempt_1515180518813_0001_36_00_000000_0_10243 mapId: [attempt_1515180518813_0001_36_00_000000_0_10243] reduceId: [0] jobId: [job_1515180518813_0001] dagId: [36] keepAlive: false 2018-01-05T11:29:06,873 INFO [TezTR-518813_1_36_1_1_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:06,873 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=36&reduce=0&map=attempt_1515180518813_0001_36_00_000000_0_10243; hash=...7BbeD9Bjmg/SA 2018-01-05T11:29:06,873 INFO [TezTR-518813_1_36_1_1_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:06,873 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=36&reduce=0&map=attempt_1515180518813_0001_36_00_000000_0_10243;reply=TPH9StiJ58aKA 2018-01-05T11:29:06,873 INFO [TezTR-518813_1_36_1_1_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 24, TotalRequested: 141890680, TotalRequestedScaled: 7.094534E7, TotalJVMHeap: 1026555904, TotalAvailable: 687792455, TotalRequested/TotalJVMHeap:0.14 2018-01-05T11:29:06,873 INFO [TezTR-518813_1_36_1_1_0] resources.MemoryDistributor: Allocations=[Map 1:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:131404920:131404920], [Reducer 3:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:10485760:10485760] 2018-01-05T11:29:06,873 INFO [TezTR-518813_1_36_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:06,873 INFO [TezTR-518813_1_36_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Input: Map 1 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:06,873 INFO [TezTR-518813_1_36_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:29:06,873 INFO [TezTR-518813_1_36_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:06,874 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=36, user='djaiswal', attemptId='attempt_1515180518813_0001_36_00_000000_0_10243'} via loader 2018-01-05T11:29:06,874 DEBUG [I/O Setup 0 Start: {Map 1}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:06,874 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=36, user='djaiswal', attemptId='attempt_1515180518813_0001_36_00_000000_0_10243'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:29:06,874 DEBUG [I/O Setup 0 Start: {Map 1}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:06,874 DEBUG [I/O Setup 0 Start: {Map 1}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:06,874 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_36_00_000000_0_10243 not found 2018-01-05T11:29:06,874 DEBUG [I/O Setup 0 Start: {Map 1}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:06,874 DEBUG [TezTR-518813_1_36_1_0_0] log.PerfLogger: 2018-01-05T11:29:06,874 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.Shuffle: Map_1: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true 2018-01-05T11:29:06,874 INFO [TezTR-518813_1_36_1_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 3.36KB 2018-01-05T11:29:06,874 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.MergeManager: Map 1: MergerManager: memoryLimit=131404920, maxSingleShuffleLimit=32851230, mergeThreshold=86727248, ioSortFactor=10, postMergeMem=0, memToMemMergeOutputsThreshold=10 2018-01-05T11:29:06,874 DEBUG [TezTR-518813_1_36_1_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:06,874 DEBUG [TezTR-518813_1_36_1_0_0] log.PerfLogger: 2018-01-05T11:29:06,874 DEBUG [TezTR-518813_1_36_1_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:06,874 DEBUG [TezTR-518813_1_36_1_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:06,874 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_36_00_000000_0_10243,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/36/output/attempt_1515180518813_0001_36_00_000000_0_10243/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/36/output/attempt_1515180518813_0001_36_00_000000_0_10243/file.out.index 2018-01-05T11:29:06,875 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Content Length in shuffle : 1875 2018-01-05T11:29:06,875 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.ShuffleScheduler: ShuffleScheduler running for sourceVertex: Map 1 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true 2018-01-05T11:29:06,875 DEBUG [TezTR-518813_1_36_1_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:06,875 INFO [TezTR-518813_1_36_1_0_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:29:06,875 INFO [Fetcher_O {Map_1} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=36&reduce=0&map=attempt_1515180518813_0001_36_00_000000_0_10243 sent hash and receievd reply 0 ms 2018-01-05T11:29:06,876 INFO [I/O Setup 0 Start: {Map 1}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: Map 1 2018-01-05T11:29:06,876 INFO [TezTR-518813_1_36_1_1_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:06,876 INFO [Fetcher_O {Map_1} #0] orderedgrouped.MergeManager: CloseInMemoryFile. Current state: inMemoryMapOutputs.size=1, commitMemory=0, usedMemory=1816. Since last log: count=1, min=1816, max=1816, total=1816, avg=1816.0 2018-01-05T11:29:06,876 INFO [Fetcher_O {Map_1} #0] ShuffleScheduler.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_36_00_000000_0_10243} to MEMORY, csize=1820, dsize=1816, EndTime=1515180546876, TimeTaken=1, Rate=1.73 MB/s 2018-01-05T11:29:06,876 INFO [Fetcher_O {Map_1} #0] orderedgrouped.ShuffleScheduler: All inputs fetched for input vertex : Map 1 2018-01-05T11:29:06,876 INFO [Fetcher_O {Map_1} #0] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:06,876 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down FetchScheduler for input: Map_1, wasInterrupted=false 2018-01-05T11:29:06,876 INFO [TezTR-518813_1_36_1_1_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_36_01_000001_0 2018-01-05T11:29:06,876 DEBUG [TezTR-518813_1_36_1_1_0] log.PerfLogger: 2018-01-05T11:29:06,876 DEBUG [TezTR-518813_1_36_1_1_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_36_01_000001_0_10247 2018-01-05T11:29:06,877 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:06,877 DEBUG [TezTR-518813_1_36_1_1_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:06,877 INFO [TezTaskEventRouter{attempt_1515180518813_0001_36_01_000001_0}] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:29:06,877 DEBUG [TezTR-518813_1_36_1_1_0] mr.ObjectCache: Creating djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:29:06,877 INFO [TezTR-518813_1_36_1_1_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/djaiswal/_tez_scratch_dir/bbe41d34-2ea1-432a-aa2b-305b2dc82600/reduce.xml 2018-01-05T11:29:06,877 DEBUG [TezTR-518813_1_36_1_1_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/djaiswal/_tez_scratch_dir/bbe41d34-2ea1-432a-aa2b-305b2dc82600/reduce.xml 2018-01-05T11:29:06,877 DEBUG [TezTR-518813_1_36_1_1_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/djaiswal/_tez_scratch_dir/bbe41d34-2ea1-432a-aa2b-305b2dc82600/reduce.xml 2018-01-05T11:29:06,878 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down fetchers for input: Map_1, shutdown timetaken: 0 ms, hasFetcherExecutorStopped: true 2018-01-05T11:29:06,878 DEBUG [TezTR-518813_1_36_1_1_0] log.PerfLogger: 2018-01-05T11:29:06,878 INFO [TezTR-518813_1_36_1_1_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:29:06,878 DEBUG [TezTR-518813_1_36_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,879 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: TotalInMemFetchStats: count=1, totalSize=1816, min=1816, max=1816, avg=1.0 2018-01-05T11:29:06,879 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: finalMerge with #inMemoryOutputs=1, size=1816 and #onDiskOutputs=0, size=0 2018-01-05T11:29:06,879 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=36&reduce=1&map=attempt_1515180518813_0001_36_00_000000_0_10243 mapId: [attempt_1515180518813_0001_36_00_000000_0_10243] reduceId: [1] jobId: [job_1515180518813_0001] dagId: [36] keepAlive: false 2018-01-05T11:29:06,879 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=36&reduce=1&map=attempt_1515180518813_0001_36_00_000000_0_10243; hash=...aBdkgh3r66Rvg 2018-01-05T11:29:06,879 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=36&reduce=1&map=attempt_1515180518813_0001_36_00_000000_0_10243;reply=uEms+9/S3hLZY 2018-01-05T11:29:06,879 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=36, user='djaiswal', attemptId='attempt_1515180518813_0001_36_00_000000_0_10243'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:29:06,879 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache HIT: MapId attempt_1515180518813_0001_36_00_000000_0_10243 found 2018-01-05T11:29:06,879 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_36_00_000000_0_10243,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/36/output/attempt_1515180518813_0001_36_00_000000_0_10243/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/36/output/attempt_1515180518813_0001_36_00_000000_0_10243/file.out.index 2018-01-05T11:29:06,879 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Content Length in shuffle : 1826 2018-01-05T11:29:06,879 DEBUG [TezTR-518813_1_36_1_1_0] log.PerfLogger: 2018-01-05T11:29:06,879 INFO [TezTR-518813_1_36_1_1_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 3.36KB 2018-01-05T11:29:06,880 INFO [Fetcher_O {Map_1} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=36&reduce=1&map=attempt_1515180518813_0001_36_00_000000_0_10243 sent hash and receievd reply 0 ms 2018-01-05T11:29:06,880 DEBUG [New I/O worker #1] shufflehandler.FadvisedFileRegion: shuffleBufferSize: 131072, path: /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/36/output/attempt_1515180518813_0001_36_00_000000_0_10243/file.out 2018-01-05T11:29:06,880 DEBUG [TezTR-518813_1_36_1_1_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:06,880 DEBUG [TezTR-518813_1_36_1_1_0] log.PerfLogger: 2018-01-05T11:29:06,880 DEBUG [TezTR-518813_1_36_1_1_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:06,880 INFO [Fetcher_O {Map_1} #0] orderedgrouped.MergeManager: CloseInMemoryFile. Current state: inMemoryMapOutputs.size=1, commitMemory=0, usedMemory=1767. Since last log: count=1, min=1767, max=1767, total=1767, avg=1767.0 2018-01-05T11:29:06,880 DEBUG [TezTR-518813_1_36_1_1_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:06,880 INFO [Fetcher_O {Map_1} #0] ShuffleScheduler.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_36_00_000000_0_10243} to MEMORY, csize=1771, dsize=1767, EndTime=1515180546880, TimeTaken=0, Rate=0.00 MB/s 2018-01-05T11:29:06,881 INFO [Fetcher_O {Map_1} #0] orderedgrouped.ShuffleScheduler: All inputs fetched for input vertex : Map 1 2018-01-05T11:29:06,881 INFO [Fetcher_O {Map_1} #0] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:06,881 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down FetchScheduler for input: Map_1, wasInterrupted=false 2018-01-05T11:29:06,881 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:06,882 DEBUG [TezTR-518813_1_36_1_1_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:06,882 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down fetchers for input: Map_1, shutdown timetaken: 0 ms, hasFetcherExecutorStopped: true 2018-01-05T11:29:06,882 INFO [TezTR-518813_1_36_1_1_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:29:06,882 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: TotalInMemFetchStats: count=1, totalSize=1767, min=1767, max=1767, avg=1.0 2018-01-05T11:29:06,882 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: finalMerge with #inMemoryOutputs=1, size=1767 and #onDiskOutputs=0, size=0 2018-01-05T11:29:06,897 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: merge complete for input vertex : Map_1 2018-01-05T11:29:06,897 INFO [TezTR-518813_1_36_1_0_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:29:06,897 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: Map_1: Shuffle Runner thread complete 2018-01-05T11:29:06,897 INFO [TezTR-518813_1_36_1_0_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:06,897 DEBUG [TezTR-518813_1_36_1_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:06,897 INFO [TezTR-518813_1_36_1_0_0] input.OrderedGroupedKVInput: Map 1: creating ValuesIterator with comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator, keyClass=org.apache.hadoop.hive.ql.io.HiveKey, valClass=org.apache.hadoop.io.BytesWritable 2018-01-05T11:29:06,897 DEBUG [TezTR-518813_1_36_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,897 DEBUG [TezTR-518813_1_36_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,897 DEBUG [TezTR-518813_1_36_1_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0] columnTypes=[string] 2018-01-05T11:29:06,897 DEBUG [TezTR-518813_1_36_1_0_0] log.PerfLogger: 2018-01-05T11:29:06,897 INFO [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Initializing operator SEL[3] 2018-01-05T11:29:06,897 INFO [TezTR-518813_1_36_1_0_0] exec.SelectOperator: SELECT struct,value:struct<_col0:string>> 2018-01-05T11:29:06,897 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Initialization Done 3 SEL 2018-01-05T11:29:06,897 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Operator 3 SEL initialized 2018-01-05T11:29:06,897 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Initializing children of 3 SEL 2018-01-05T11:29:06,897 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Initializing child 4 FS 2018-01-05T11:29:06,897 INFO [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Initializing operator FS[4] 2018-01-05T11:29:06,898 DEBUG [TezTR-518813_1_36_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,898 INFO [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Using serializer : org.apache.hadoop.hive.ql.io.orc.OrcSerde@4f472670 and formatter : org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat@450e95dd 2018-01-05T11:29:06,898 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:29:06,898 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Initialization Done 4 FS 2018-01-05T11:29:06,898 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Operator 4 FS initialized 2018-01-05T11:29:06,898 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Initialization Done 4 FS done is reset. 2018-01-05T11:29:06,898 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Initializing child 1 SEL 2018-01-05T11:29:06,898 INFO [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Initializing operator SEL[1] 2018-01-05T11:29:06,898 INFO [TezTR-518813_1_36_1_0_0] exec.SelectOperator: SELECT struct<_col0:int,_col1:string> 2018-01-05T11:29:06,898 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Initialization Done 1 SEL 2018-01-05T11:29:06,898 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Operator 1 SEL initialized 2018-01-05T11:29:06,898 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Initializing children of 1 SEL 2018-01-05T11:29:06,898 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Initializing child 2 GBY 2018-01-05T11:29:06,898 INFO [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Initializing operator GBY[2] 2018-01-05T11:29:06,898 DEBUG [TezTR-518813_1_36_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,898 DEBUG [TezTR-518813_1_36_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,898 INFO [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Max hash table memory: 93860656 bytes 2018-01-05T11:29:06,898 INFO [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:29:06,898 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Initialization Done 2 GBY 2018-01-05T11:29:06,898 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Operator 2 GBY initialized 2018-01-05T11:29:06,898 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Initializing children of 2 GBY 2018-01-05T11:29:06,898 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Initializing child 3 RS 2018-01-05T11:29:06,898 INFO [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Initializing operator RS[3] 2018-01-05T11:29:06,898 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: keys size is 1 2018-01-05T11:29:06,898 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Key exprNodeDesc _col0 2018-01-05T11:29:06,898 INFO [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Using tag = -1 2018-01-05T11:29:06,898 DEBUG [TezTR-518813_1_36_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,898 DEBUG [TezTR-518813_1_36_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,898 DEBUG [TezTR-518813_1_36_1_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:29:06,898 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS 2018-01-05T11:29:06,898 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Operator 3 RS initialized 2018-01-05T11:29:06,898 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Initialization Done 3 RS done is reset. 2018-01-05T11:29:06,898 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Initialization Done 2 GBY done is reset. 2018-01-05T11:29:06,898 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Initialization Done 1 SEL done is reset. 2018-01-05T11:29:06,898 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Initialization Done 3 SEL done is reset. 2018-01-05T11:29:06,898 INFO [TezTR-518813_1_36_1_0_0] tez.ReduceRecordProcessor: Id =3 Id =4 <\Children> Id = 3 null<\Parent> <\FS> Id =1 Id =2 null <\Children> Id = 1 null<\Parent> <\GBY> <\Children> Id = 3 null<\Parent> <\SEL> <\Children> <\Parent> <\SEL> 2018-01-05T11:29:06,898 INFO [TezTR-518813_1_36_1_0_0] exec.OperatorUtils: Setting output collector: RS[3] --> Reducer 3 2018-01-05T11:29:06,898 DEBUG [TezTR-518813_1_36_1_0_0] log.PerfLogger: 2018-01-05T11:29:06,898 INFO [TezTR-518813_1_36_1_0_0] tez.ReduceRecordProcessor: Starting Output: Reducer 3 2018-01-05T11:29:06,899 INFO [TezTR-518813_1_36_1_0_0] impl.ExternalSorter: Reducer 3 using: memoryMb=10, keySerializerClass=class org.apache.hadoop.hive.ql.io.HiveKey, valueSerializerClass=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization$TezBytesWritableSerializer@691eb5c8, comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator@46f53a21, partitioner=org.apache.tez.mapreduce.partition.MRPartitioner, serialization=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.hadoop.io.serializer.WritableSerialization, org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization, org.apache.hadoop.io.serializer.avro.AvroReflectSerialization, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:29:06,899 INFO [TezTR-518813_1_36_1_0_0] partition.MRPartitioner: Using oldApi, MRpartitionerClass=org.apache.hadoop.hive.ql.io.DefaultHivePartitioner 2018-01-05T11:29:06,899 INFO [TezTR-518813_1_36_1_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 3: , UsingHashComparator=true 2018-01-05T11:29:06,900 INFO [TezTR-518813_1_36_1_0_0] impl.PipelinedSorter: Newly allocated block size=10485760, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=10485760, total=10485760 2018-01-05T11:29:06,900 INFO [TezTR-518813_1_36_1_0_0] impl.PipelinedSorter: Pre allocating rest of memory buffers upfront 2018-01-05T11:29:06,900 INFO [TezTR-518813_1_36_1_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 3: , UsingHashComparator=true#blocks=1, maxMemUsage=10485760, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=10485760, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=10 2018-01-05T11:29:06,900 INFO [TezTR-518813_1_36_1_0_0] impl.PipelinedSorter: Reducer 3: reserved.remaining()=10485760, reserved.metasize=5242880 2018-01-05T11:29:06,901 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,901 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,901 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,901 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,901 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,901 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,902 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,902 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,902 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,902 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,902 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,902 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,902 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,902 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,902 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,902 INFO [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-06_370_8944885619275660846-1/_tmp.-ext-10000/000000_0 2018-01-05T11:29:06,902 INFO [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-06_370_8944885619275660846-1/_task_tmp.-ext-10000/_tmp.000000_0 2018-01-05T11:29:06,902 INFO [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-06_370_8944885619275660846-1/_tmp.-ext-10000/000000_0 2018-01-05T11:29:06,902 DEBUG [TezTR-518813_1_36_1_0_0] orc.OrcOutputFormat: ORC schema = struct 2018-01-05T11:29:06,902 INFO [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: FS[4]: records written - 1 2018-01-05T11:29:06,902 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: merge complete for input vertex : Map_1 2018-01-05T11:29:06,902 INFO [TezTR-518813_1_36_1_1_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:29:06,902 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: Map_1: Shuffle Runner thread complete 2018-01-05T11:29:06,902 INFO [TezTR-518813_1_36_1_1_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:06,902 DEBUG [TezTR-518813_1_36_1_1_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:06,902 INFO [TezTR-518813_1_36_1_1_0] input.OrderedGroupedKVInput: Map 1: creating ValuesIterator with comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator, keyClass=org.apache.hadoop.hive.ql.io.HiveKey, valClass=org.apache.hadoop.io.BytesWritable 2018-01-05T11:29:06,902 INFO [TezTR-518813_1_36_1_0_0] impl.PhysicalFsWriter: ORC writer created for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-06_370_8944885619275660846-1/_task_tmp.-ext-10000/_tmp.000000_0 with stripeSize: 67108864 blockSize: 268435456 compression: ZLIB bufferSize: 262144 2018-01-05T11:29:06,902 DEBUG [TezTR-518813_1_36_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,903 DEBUG [TezTR-518813_1_36_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,903 DEBUG [TezTR-518813_1_36_1_1_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0] columnTypes=[string] 2018-01-05T11:29:06,903 DEBUG [TezTR-518813_1_36_1_1_0] log.PerfLogger: 2018-01-05T11:29:06,903 INFO [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Initializing operator SEL[3] 2018-01-05T11:29:06,903 INFO [TezTR-518813_1_36_1_1_0] exec.SelectOperator: SELECT struct,value:struct<_col0:string>> 2018-01-05T11:29:06,903 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Initialization Done 3 SEL 2018-01-05T11:29:06,903 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Operator 3 SEL initialized 2018-01-05T11:29:06,903 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Initializing children of 3 SEL 2018-01-05T11:29:06,903 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Initializing child 4 FS 2018-01-05T11:29:06,903 INFO [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Initializing operator FS[4] 2018-01-05T11:29:06,903 DEBUG [TezTR-518813_1_36_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,903 INFO [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Using serializer : org.apache.hadoop.hive.ql.io.orc.OrcSerde@629219 and formatter : org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat@5117c89a 2018-01-05T11:29:06,903 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:29:06,903 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Initialization Done 4 FS 2018-01-05T11:29:06,903 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Operator 4 FS initialized 2018-01-05T11:29:06,903 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Initialization Done 4 FS done is reset. 2018-01-05T11:29:06,903 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Initializing child 1 SEL 2018-01-05T11:29:06,903 INFO [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Initializing operator SEL[1] 2018-01-05T11:29:06,903 INFO [TezTR-518813_1_36_1_1_0] exec.SelectOperator: SELECT struct<_col0:int,_col1:string> 2018-01-05T11:29:06,903 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Initialization Done 1 SEL 2018-01-05T11:29:06,903 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Operator 1 SEL initialized 2018-01-05T11:29:06,903 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Initializing children of 1 SEL 2018-01-05T11:29:06,903 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Initializing child 2 GBY 2018-01-05T11:29:06,903 INFO [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Initializing operator GBY[2] 2018-01-05T11:29:06,903 DEBUG [TezTR-518813_1_36_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,903 DEBUG [TezTR-518813_1_36_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,903 INFO [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Max hash table memory: 93860656 bytes 2018-01-05T11:29:06,903 INFO [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:29:06,903 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Initialization Done 2 GBY 2018-01-05T11:29:06,903 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Operator 2 GBY initialized 2018-01-05T11:29:06,903 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Initializing children of 2 GBY 2018-01-05T11:29:06,903 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Initializing child 3 RS 2018-01-05T11:29:06,903 INFO [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Initializing operator RS[3] 2018-01-05T11:29:06,903 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: keys size is 1 2018-01-05T11:29:06,903 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Key exprNodeDesc _col0 2018-01-05T11:29:06,903 INFO [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Using tag = -1 2018-01-05T11:29:06,903 DEBUG [TezTR-518813_1_36_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,904 DEBUG [TezTR-518813_1_36_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,904 DEBUG [TezTR-518813_1_36_1_1_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:29:06,904 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Initialization Done 3 RS 2018-01-05T11:29:06,904 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Operator 3 RS initialized 2018-01-05T11:29:06,904 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Initialization Done 3 RS done is reset. 2018-01-05T11:29:06,904 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Initialization Done 2 GBY done is reset. 2018-01-05T11:29:06,904 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Initialization Done 1 SEL done is reset. 2018-01-05T11:29:06,904 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Initialization Done 3 SEL done is reset. 2018-01-05T11:29:06,904 INFO [TezTR-518813_1_36_1_1_0] tez.ReduceRecordProcessor: Id =3 Id =4 <\Children> Id = 3 null<\Parent> <\FS> Id =1 Id =2 null <\Children> Id = 1 null<\Parent> <\GBY> <\Children> Id = 3 null<\Parent> <\SEL> <\Children> <\Parent> <\SEL> 2018-01-05T11:29:06,904 INFO [TezTR-518813_1_36_1_1_0] exec.OperatorUtils: Setting output collector: RS[3] --> Reducer 3 2018-01-05T11:29:06,904 DEBUG [TezTR-518813_1_36_1_1_0] log.PerfLogger: 2018-01-05T11:29:06,904 INFO [TezTR-518813_1_36_1_1_0] tez.ReduceRecordProcessor: Starting Output: Reducer 3 2018-01-05T11:29:06,904 INFO [TezTR-518813_1_36_1_1_0] impl.ExternalSorter: Reducer 3 using: memoryMb=10, keySerializerClass=class org.apache.hadoop.hive.ql.io.HiveKey, valueSerializerClass=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization$TezBytesWritableSerializer@2d6e99a8, comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator@772296e9, partitioner=org.apache.tez.mapreduce.partition.MRPartitioner, serialization=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.hadoop.io.serializer.WritableSerialization, org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization, org.apache.hadoop.io.serializer.avro.AvroReflectSerialization, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:29:06,904 INFO [TezTR-518813_1_36_1_1_0] partition.MRPartitioner: Using oldApi, MRpartitionerClass=org.apache.hadoop.hive.ql.io.DefaultHivePartitioner 2018-01-05T11:29:06,905 INFO [TezTR-518813_1_36_1_1_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 3: , UsingHashComparator=true 2018-01-05T11:29:06,906 INFO [TezTR-518813_1_36_1_1_0] impl.PipelinedSorter: Newly allocated block size=10485760, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=10485760, total=10485760 2018-01-05T11:29:06,906 INFO [TezTR-518813_1_36_1_1_0] impl.PipelinedSorter: Pre allocating rest of memory buffers upfront 2018-01-05T11:29:06,906 INFO [TezTR-518813_1_36_1_1_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 3: , UsingHashComparator=true#blocks=1, maxMemUsage=10485760, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=10485760, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=10 2018-01-05T11:29:06,906 INFO [TezTR-518813_1_36_1_1_0] impl.PipelinedSorter: Reducer 3: reserved.remaining()=10485760, reserved.metasize=5242880 2018-01-05T11:29:06,907 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,907 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,907 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,907 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,907 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,907 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,907 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,907 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,907 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,907 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,907 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,907 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,907 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,907 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,907 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,907 INFO [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-06_370_8944885619275660846-1/_tmp.-ext-10000/000001_0 2018-01-05T11:29:06,907 INFO [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-06_370_8944885619275660846-1/_task_tmp.-ext-10000/_tmp.000001_0 2018-01-05T11:29:06,907 INFO [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-06_370_8944885619275660846-1/_tmp.-ext-10000/000001_0 2018-01-05T11:29:06,907 DEBUG [TezTR-518813_1_36_1_1_0] orc.OrcOutputFormat: ORC schema = struct 2018-01-05T11:29:06,908 INFO [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: FS[4]: records written - 1 2018-01-05T11:29:06,908 INFO [TezTR-518813_1_36_1_1_0] impl.PhysicalFsWriter: ORC writer created for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-06_370_8944885619275660846-1/_task_tmp.-ext-10000/_tmp.000001_0 with stripeSize: 67108864 blockSize: 268435456 compression: ZLIB bufferSize: 262144 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:06,946 INFO [TezTR-518813_1_36_1_1_0] impl.WriterImpl: ORC writer created for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-06_370_8944885619275660846-1/_task_tmp.-ext-10000/_tmp.000001_0 with stripeSize: 67108864 blockSize: 268435456 compression: ZLIB bufferSize: 262144 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,946 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,947 INFO [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: FS[4]: records written - 10 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,947 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,948 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,949 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,950 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,951 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,952 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,953 INFO [TezTR-518813_1_36_1_0_0] impl.WriterImpl: ORC writer created for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-06_370_8944885619275660846-1/_task_tmp.-ext-10000/_tmp.000000_0 with stripeSize: 67108864 blockSize: 268435456 compression: ZLIB bufferSize: 262144 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,953 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,954 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,955 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,956 INFO [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: FS[4]: records written - 10 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,956 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,957 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,958 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,959 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,960 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,961 INFO [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: FS[4]: records written - 100 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,961 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000080, QueryId=djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd-36, DagId=36, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180546811, EndTime=1515180546961 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,961 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,962 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,963 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,964 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,965 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,966 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_1_0] log.PerfLogger: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_1_0] mr.ObjectCache: djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: close called for operator SEL[3] 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,967 INFO [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Closing operator SEL[3] 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Closing child = FS[4] 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: close called for operator FS[4] 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:06,967 INFO [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: Closing operator FS[4] 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,967 INFO [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: FS[4]: records written - 120 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,967 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,968 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,969 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_1_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/-mr-10001 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,970 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,971 INFO [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: FS[4]: records written - 100 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,971 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Ending group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: End group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] log.PerfLogger: 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] mr.ObjectCache: djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: close called for operator SEL[3] 2018-01-05T11:29:06,972 INFO [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Closing operator SEL[3] 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Closing child = FS[4] 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: close called for operator FS[4] 2018-01-05T11:29:06,972 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:06,972 INFO [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: Closing operator FS[4] 2018-01-05T11:29:06,972 INFO [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: FS[4]: records written - 122 2018-01-05T11:29:06,973 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000081, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_36_01_000000_0, eventCount=1 } 2018-01-05T11:29:06,974 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:06,975 DEBUG [TezTR-518813_1_36_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:06,975 DEBUG [TezTR-518813_1_36_1_0_0] fs.FSStatsPublisher: Connecting to : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/-mr-10001 2018-01-05T11:29:06,976 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000082, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_36_01_000001_0, eventCount=1 } 2018-01-05T11:29:06,977 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:06,979 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,979 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: Status: Running (Executing on YARN cluster with App id application_1515180518813_0001) 2018-01-05T11:29:06,981 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,981 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,981 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:06,981 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 Reducer 2: 0(+2)/2 Reducer 3: 0/4 2018-01-05T11:29:06,993 DEBUG [TezTR-518813_1_36_1_1_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/-mr-10001/tmpstats-1 2018-01-05T11:29:06,993 DEBUG [TezTR-518813_1_36_1_1_0] fs.FSStatsPublisher: Writing stats in it : {default.tab/ds=2008-04-08/={rawDataSize=11280, numRows=120}} 2018-01-05T11:29:06,993 INFO [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: RECORDS_OUT_1_default.tab:120, 2018-01-05T11:29:06,993 DEBUG [TezTR-518813_1_36_1_1_0] exec.FileSinkOperator: 4 Close done 2018-01-05T11:29:06,993 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Closing child = SEL[1] 2018-01-05T11:29:06,993 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: close called for operator SEL[1] 2018-01-05T11:29:06,993 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:06,993 INFO [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Closing operator SEL[1] 2018-01-05T11:29:06,993 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: Closing child = GBY[2] 2018-01-05T11:29:06,993 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: close called for operator GBY[2] 2018-01-05T11:29:06,993 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:06,993 INFO [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Closing operator GBY[2] 2018-01-05T11:29:06,993 INFO [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Begin Hash Table flush: size = 1 2018-01-05T11:29:06,993 INFO [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: keys are [_col0] num distributions: 1 2018-01-05T11:29:06,993 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: Closing child = RS[3] 2018-01-05T11:29:06,993 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: close called for operator RS[3] 2018-01-05T11:29:06,993 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:06,993 INFO [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: Closing operator RS[3] 2018-01-05T11:29:06,993 INFO [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: RECORDS_OUT_INTERMEDIATE_Reducer_2:1, 2018-01-05T11:29:06,993 DEBUG [TezTR-518813_1_36_1_1_0] exec.ReduceSinkOperator: 3 Close done 2018-01-05T11:29:06,993 DEBUG [TezTR-518813_1_36_1_1_0] exec.GroupByOperator: 2 Close done 2018-01-05T11:29:06,993 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: 1 Close done 2018-01-05T11:29:06,993 DEBUG [TezTR-518813_1_36_1_1_0] exec.SelectOperator: 3 Close done 2018-01-05T11:29:06,993 INFO [TezTR-518813_1_36_1_1_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_36_01_000001_0 2018-01-05T11:29:06,993 INFO [TezTR-518813_1_36_1_1_0] orderedgrouped.Shuffle: Shutting down Shuffle for source: Map_1 2018-01-05T11:29:06,993 INFO [TezTR-518813_1_36_1_1_0] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:29:06,993 INFO [TezTR-518813_1_36_1_1_0] impl.PipelinedSorter: Reducer 3: Starting flush of map output 2018-01-05T11:29:06,993 INFO [TezTR-518813_1_36_1_1_0] impl.PipelinedSorter: Reducer 3: Span0.length = 1, perItem = 642 2018-01-05T11:29:06,993 INFO [TezTR-518813_1_36_1_1_0] impl.PipelinedSorter: Reducer 3: done sorting span=0, length=1, time=0 2018-01-05T11:29:06,993 DEBUG [TezTR-518813_1_36_1_1_0] util.Progress: Illegal progress value found, progress is Float.NaN. Progress will be changed to 0 2018-01-05T11:29:06,993 INFO [TezTR-518813_1_36_1_1_0] impl.PipelinedSorter: Reducer 3: Heap = SpanIterator<0:0> (span=Span[5242880,642]), 2018-01-05T11:29:06,998 DEBUG [TezTR-518813_1_36_1_0_0] fs.FSStatsPublisher: Created file : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/-mr-10001/tmpstats-0 2018-01-05T11:29:06,998 DEBUG [TezTR-518813_1_36_1_0_0] fs.FSStatsPublisher: Writing stats in it : {default.tab/ds=2008-04-08/={rawDataSize=11468, numRows=122}} 2018-01-05T11:29:06,998 INFO [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: RECORDS_OUT_1_default.tab:122, 2018-01-05T11:29:06,998 DEBUG [TezTR-518813_1_36_1_0_0] exec.FileSinkOperator: 4 Close done 2018-01-05T11:29:06,998 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Closing child = SEL[1] 2018-01-05T11:29:06,998 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: close called for operator SEL[1] 2018-01-05T11:29:06,998 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:06,998 INFO [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Closing operator SEL[1] 2018-01-05T11:29:06,998 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: Closing child = GBY[2] 2018-01-05T11:29:06,998 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: close called for operator GBY[2] 2018-01-05T11:29:06,998 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:06,998 INFO [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Closing operator GBY[2] 2018-01-05T11:29:06,998 INFO [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Begin Hash Table flush: size = 1 2018-01-05T11:29:06,998 INFO [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: keys are [_col0] num distributions: 1 2018-01-05T11:29:06,998 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: Closing child = RS[3] 2018-01-05T11:29:06,998 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: close called for operator RS[3] 2018-01-05T11:29:06,998 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:06,998 INFO [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: Closing operator RS[3] 2018-01-05T11:29:06,998 INFO [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: RECORDS_OUT_INTERMEDIATE_Reducer_2:1, 2018-01-05T11:29:06,998 DEBUG [TezTR-518813_1_36_1_0_0] exec.ReduceSinkOperator: 3 Close done 2018-01-05T11:29:06,998 DEBUG [TezTR-518813_1_36_1_0_0] exec.GroupByOperator: 2 Close done 2018-01-05T11:29:06,998 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: 1 Close done 2018-01-05T11:29:06,998 DEBUG [TezTR-518813_1_36_1_0_0] exec.SelectOperator: 3 Close done 2018-01-05T11:29:06,998 INFO [TezTR-518813_1_36_1_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_36_01_000000_0 2018-01-05T11:29:06,998 INFO [TezTR-518813_1_36_1_0_0] orderedgrouped.Shuffle: Shutting down Shuffle for source: Map_1 2018-01-05T11:29:06,998 INFO [TezTR-518813_1_36_1_0_0] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:29:06,998 INFO [TezTR-518813_1_36_1_0_0] impl.PipelinedSorter: Reducer 3: Starting flush of map output 2018-01-05T11:29:06,998 INFO [TezTR-518813_1_36_1_0_0] impl.PipelinedSorter: Reducer 3: Span0.length = 1, perItem = 691 2018-01-05T11:29:06,998 INFO [TezTR-518813_1_36_1_0_0] impl.PipelinedSorter: Reducer 3: done sorting span=0, length=1, time=0 2018-01-05T11:29:06,998 DEBUG [TezTR-518813_1_36_1_0_0] util.Progress: Illegal progress value found, progress is Float.NaN. Progress will be changed to 0 2018-01-05T11:29:06,999 INFO [TezTR-518813_1_36_1_0_0] impl.PipelinedSorter: Reducer 3: Heap = SpanIterator<0:0> (span=Span[5242880,691]), 2018-01-05T11:29:07,006 INFO [TezTR-518813_1_36_1_1_0] impl.PipelinedSorter: Reducer 3: Spilling to /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/36/output/attempt_1515180518813_0001_36_01_000001_0_10249_0/file.out 2018-01-05T11:29:07,011 INFO [TezTR-518813_1_36_1_0_0] impl.PipelinedSorter: Reducer 3: Spilling to /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/36/output/attempt_1515180518813_0001_36_01_000000_0_10246_0/file.out 2018-01-05T11:29:07,020 INFO [TezTR-518813_1_36_1_1_0] shuffle.ShuffleUtils: EmptyPartition bitsetSize=3, numOutputs=4, emptyPartitions=3, compressedSize=3 2018-01-05T11:29:07,021 DEBUG [TezTR-518813_1_36_1_1_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:07,021 INFO [TezTR-518813_1_36_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:07,021 INFO [TezTR-518813_1_36_1_1_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_36_01_000001_0, askedToStop=false 2018-01-05T11:29:07,021 INFO [TezTR-518813_1_36_1_1_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_36_01_000001_0, stopRequested=false 2018-01-05T11:29:07,021 INFO [TezTR-518813_1_36_1_1_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_36_01_000001_0: Counters: 37 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=73, REDUCE_INPUT_RECORDS=120, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=121, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=642, OUTPUT_BYTES_WITH_OVERHEAD=652, OUTPUT_BYTES_PHYSICAL=656, ADDITIONAL_SPILLS_BYTES_WRITTEN=1771, ADDITIONAL_SPILLS_BYTES_READ=1771, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=1771, SHUFFLE_BYTES_DECOMPRESSED=1767, SHUFFLE_BYTES_TO_MEM=1771, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=28, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][HIVE CREATED_FILES=1, RECORDS_OUT_1_default.tab=120, RECORDS_OUT_INTERMEDIATE_Reducer_2=1][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0]] 2018-01-05T11:29:07,021 DEBUG [TezTR-518813_1_36_1_1_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_36_1_1_0 thread id: 2401 scheme: file bytesRead: 208 bytesWritten: 1640 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:07,021 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_36_01_000001_0, isTaskDone=true 2018-01-05T11:29:07,021 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000082, requestId=3, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_36_01_000001_0, eventCount=4 } 2018-01-05T11:29:07,022 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_36_01_000001_0, creationTime=1515180546861, allocationTime=1515180546862, startTime=1515180546863, finishTime=1515180547022, timeTaken=159, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:07,022 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_36_01_000001_0 2018-01-05T11:29:07,022 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=3, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:07,022 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_36_01_000001, startTime=1515180546863, finishTime=1515180547022, timeTaken=159, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_36_01_000001_0, diagnostics=, counters=Counters: 42, File System Counters, FILE_BYTES_READ=208, FILE_BYTES_WRITTEN=1640, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=73, REDUCE_INPUT_RECORDS=120, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=121, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=642, OUTPUT_BYTES_WITH_OVERHEAD=652, OUTPUT_BYTES_PHYSICAL=656, ADDITIONAL_SPILLS_BYTES_WRITTEN=1771, ADDITIONAL_SPILLS_BYTES_READ=1771, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=1771, SHUFFLE_BYTES_DECOMPRESSED=1767, SHUFFLE_BYTES_TO_MEM=1771, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=28, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, HIVE, CREATED_FILES=1, RECORDS_OUT_1_default.tab=120, RECORDS_OUT_INTERMEDIATE_Reducer_2=1, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0 2018-01-05T11:29:07,022 INFO [Task-Executor-2] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_36_01_000001_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:07,022 INFO [Task-Executor-2] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_36_01_000001_0 2018-01-05T11:29:07,022 INFO [Task-Executor-2] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000082=156 2018-01-05T11:29:07,022 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_36_01_000001_0: true 2018-01-05T11:29:07,022 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_36_01 [Reducer 2], tasks=2, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:07,022 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_36_02 [Reducer 3] attempt: attempt_1515180518813_0001_36_01_000001_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:29:07,022 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_36_01_000001_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:07,022 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_36_01_000001_0; false 2018-01-05T11:29:07,022 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_36_01_000001_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:07,022 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_36_01_000001_0 2018-01-05T11:29:07,023 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000082, QueryId=djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd-36, Vertex=Reducer 2, FragmentNum=1, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:07,023 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_36_01_000001_0 complete. WaitQueueSize=0, numSlotsAvailable=3, preemptionQueueSize=1 2018-01-05T11:29:07,023 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_36_01_000001_0 2018-01-05T11:29:07,023 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:07,023 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:07,023 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000082, stoppedTime=1515180547023, exitStatus=0 2018-01-05T11:29:07,023 INFO [App Shared Pool - #1] vertexmanager.ShuffleVertexManagerBase: Scheduling 3 tasks for vertex: Reducer 3 with totalTasks: 4. 1 source tasks completed out of 2. MinSourceTaskCompletedFraction: 0.5 min: 0.05 max: 0.75 2018-01-05T11:29:07,023 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000082 exited with diagnostics set to Container completed. 2018-01-05T11:29:07,023 INFO [App Shared Pool - #1] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Reducer 2, destinationVertexName=Reducer 3. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:29:07,023 INFO [App Shared Pool - #1] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_36_02 [Reducer 3] 2018-01-05T11:29:07,023 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000082 2018-01-05T11:29:07,023 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000082, QueryId=djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd-36, DagId=36, VertexName=Reducer 2, TaskId=1, TaskAttemptId=0, ThreadName=Task-Executor-2, Succeeded=true, StartTime=1515180546865, EndTime=1515180547023 2018-01-05T11:29:07,023 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:TASK_STARTED]: vertexName=Reducer 3, taskId=task_1515180518813_0001_36_02_000002, scheduledTime=1515180547023, launchTime=1515180547023 2018-01-05T11:29:07,023 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:TASK_STARTED]: vertexName=Reducer 3, taskId=task_1515180518813_0001_36_02_000000, scheduledTime=1515180547023, launchTime=1515180547023 2018-01-05T11:29:07,024 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:TASK_STARTED]: vertexName=Reducer 3, taskId=task_1515180518813_0001_36_02_000001, scheduledTime=1515180547024, launchTime=1515180547024 2018-01-05T11:29:07,024 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_36_02 [Reducer 3] 2018-01-05T11:29:07,024 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_36_02_000002_0, priority=32, capability=, hosts=null 2018-01-05T11:29:07,024 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=32,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:07,024 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_36_02_000000_0, priority=32, capability=, hosts=null 2018-01-05T11:29:07,024 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=32,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:07,024 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:07,024 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_36_02_000002_0 on hosts=null 2018-01-05T11:29:07,024 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_36_02_000002_0 on hosts=null 2018-01-05T11:29:07,024 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_36_02_000002_0 2018-01-05T11:29:07,024 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:07,024 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #83, task=TaskInfo{task=attempt_1515180518813_0001_36_02_000002_0, priority=32, startTime=0, containerId=null, uniqueId=82, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, to container=container_111101111_0001_00_000083 2018-01-05T11:29:07,024 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_36_02_000002_0; false 2018-01-05T11:29:07,024 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_36_02_000002_0, priority=32, startTime=311567534, containerId=container_111101111_0001_00_000083assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=2, ac=6, commF=false, disabled=false}, uniqueId=82, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:07,024 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=32,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:07,024 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=32,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:07,024 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:07,024 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_36_02_000000_0 on hosts=null 2018-01-05T11:29:07,024 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_36_02_000000_0 on hosts=null 2018-01-05T11:29:07,024 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_36_02_000001_0, priority=32, capability=, hosts=null 2018-01-05T11:29:07,024 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_36_02_000000_0 2018-01-05T11:29:07,024 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=2, ac=6, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:07,024 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000083 for node: 10.22.8.182:49818 2018-01-05T11:29:07,024 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #84, task=TaskInfo{task=attempt_1515180518813_0001_36_02_000000_0, priority=32, startTime=0, containerId=null, uniqueId=83, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=2, ac=6, commF=false, disabled=false}, to container=container_111101111_0001_00_000084 2018-01-05T11:29:07,025 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_36_02_000000_0; false 2018-01-05T11:29:07,025 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000083, attempt_1515180518813_0001_36_02_000002_0 for node: 10.22.8.182:49818 2018-01-05T11:29:07,025 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_36_02_000000_0, priority=32, startTime=311567534, containerId=container_111101111_0001_00_000084assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=3, ac=5, commF=false, disabled=false}, uniqueId=83, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:07,025 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000083 for node: 10.22.8.182:49818 2018-01-05T11:29:07,025 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=32,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:07,025 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=32,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:07,025 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:07,025 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_36_02_000001_0 on hosts=null 2018-01-05T11:29:07,025 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_36_02_000001_0 on hosts=null 2018-01-05T11:29:07,025 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_36_02_000001_0 2018-01-05T11:29:07,025 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=3, ac=5, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:07,025 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #85, task=TaskInfo{task=attempt_1515180518813_0001_36_02_000001_0, priority=32, startTime=0, containerId=null, uniqueId=84, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=3, ac=5, commF=false, disabled=false}, to container=container_111101111_0001_00_000085 2018-01-05T11:29:07,025 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_36_02_000001_0; false 2018-01-05T11:29:07,025 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_36_02_000001_0, priority=32, startTime=311567534, containerId=container_111101111_0001_00_000085assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=4, ac=4, commF=false, disabled=false}, uniqueId=84, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:07,025 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000083 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:07,025 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000083, launchTime=1515180547025 2018-01-05T11:29:07,025 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000084 for node: 10.22.8.182:49818 2018-01-05T11:29:07,025 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000084, attempt_1515180518813_0001_36_02_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:07,025 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000084 for node: 10.22.8.182:49818 2018-01-05T11:29:07,026 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000085 for node: 10.22.8.182:49818 2018-01-05T11:29:07,026 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_36_02_000002_0, am_details=10.22.8.182:49824, taskInfo= fragment 2 attempt 0, user=djaiswal, queryId=djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000083, dagName=insert overwrite table t...srcbucket_mapjoin (Stage-1), vertexName=Reducer 3, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Reducer 2,org.apache.tez.runtime.library.input.OrderedGroupedKVInput,2}}, Outputs={{out_Reducer 3,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=7, completedTaskCount=2, dagStartTime=1515180546755, firstAttemptStartTime=1515180547024, currentAttemptStartTime=1515180547025} 2018-01-05T11:29:07,026 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000085, attempt_1515180518813_0001_36_02_000001_0 for node: 10.22.8.182:49818 2018-01-05T11:29:07,026 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000085 for node: 10.22.8.182:49818 2018-01-05T11:29:07,026 INFO [IPC Server handler 2 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000083, QueryId=djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd-36, DagId=36, VertexName=Reducer 3, TaskId=2, TaskAttemptId=0, SubmitTime=1515180547026 2018-01-05T11:29:07,026 DEBUG [IPC Server handler 2 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=36} with the ShuffleHandler 2018-01-05T11:29:07,026 DEBUG [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/36] 2018-01-05T11:29:07,026 INFO [TezTR-518813_1_36_1_0_0] shuffle.ShuffleUtils: EmptyPartition bitsetSize=3, numOutputs=4, emptyPartitions=3, compressedSize=3 2018-01-05T11:29:07,027 DEBUG [TezTR-518813_1_36_1_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:07,027 INFO [TezTR-518813_1_36_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:07,027 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_36_02_000002_0] submitted. Is using containerId: [container_111101111_0001_00_000083] on NM: [10.22.8.182:49818] 2018-01-05T11:29:07,027 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:07,027 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:07,027 INFO [TezTR-518813_1_36_1_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_36_01_000000_0, askedToStop=false 2018-01-05T11:29:07,027 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 3, taskAttemptId=attempt_1515180518813_0001_36_02_000002_0, startTime=1515180547026, containerId=container_111101111_0001_00_000083, nodeId=10.22.8.182:49818 2018-01-05T11:29:07,027 INFO [TezTR-518813_1_36_1_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_36_01_000000_0, stopRequested=false 2018-01-05T11:29:07,027 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=3, runningFragmentCount=1 2018-01-05T11:29:07,027 DEBUG [IPC Server handler 2 on 49818] impl.QueryFragmentInfo: Cannot finish due to source: Reducer 2 2018-01-05T11:29:07,027 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000084 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:07,027 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_36_02_000002_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:07,027 INFO [TezTR-518813_1_36_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_36_01_000000_0: Counters: 37 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=80, REDUCE_INPUT_RECORDS=122, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=123, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=691, OUTPUT_BYTES_WITH_OVERHEAD=701, OUTPUT_BYTES_PHYSICAL=705, ADDITIONAL_SPILLS_BYTES_WRITTEN=1820, ADDITIONAL_SPILLS_BYTES_READ=1820, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=1820, SHUFFLE_BYTES_DECOMPRESSED=1816, SHUFFLE_BYTES_TO_MEM=1820, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=27, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2][HIVE CREATED_FILES=1, RECORDS_OUT_1_default.tab=122, RECORDS_OUT_INTERMEDIATE_Reducer_2=1][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0]] 2018-01-05T11:29:07,027 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000084, launchTime=1515180547027 2018-01-05T11:29:07,027 DEBUG [IPC Server handler 2 on 49818] impl.QueryFragmentInfo: Cannot finish due to source: Reducer 2 2018-01-05T11:29:07,027 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_36_02_000002_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=false, canFinish(in queue)=false, isGuaranteed=false, firstAttemptStartTime=1515180547024, dagStartTime=1515180546755, withinDagPriority=32, vertexParallelism= 4, selfAndUpstreamParallelism= 7, selfAndUpstreamComplete= 2}] 2018-01-05T11:29:07,027 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_36_02_000002_0 : ACCEPTED 2018-01-05T11:29:07,027 DEBUG [TezTR-518813_1_36_1_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_36_1_0_0 thread id: 2395 scheme: file bytesRead: 208 bytesWritten: 1666 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:07,027 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_36_02_000002_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=false, canFinish(in queue)=false, isGuaranteed=false, firstAttemptStartTime=1515180547024, dagStartTime=1515180546755, withinDagPriority=32, vertexParallelism= 4, selfAndUpstreamParallelism= 7, selfAndUpstreamComplete= 2} 2018-01-05T11:29:07,027 DEBUG [Wait-Queue-Scheduler-0] impl.QueryFragmentInfo: Cannot finish due to source: Reducer 2 2018-01-05T11:29:07,027 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_36_02_000002_0 scheduled for execution. canFinish=false, isGuaranteed=false 2018-01-05T11:29:07,027 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_36_02_000002_0 to pre-emption queue 2018-01-05T11:29:07,027 DEBUG [Task-Executor-2] impl.QueryFragmentInfo: Cannot finish due to source: Reducer 2 2018-01-05T11:29:07,027 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_36_02_000002_0: false 2018-01-05T11:29:07,027 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_36_01_000000_0, isTaskDone=true 2018-01-05T11:29:07,027 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000081, requestId=3, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_36_01_000000_0, eventCount=4 } 2018-01-05T11:29:07,027 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: taskOwner hashCode:140744346 2018-01-05T11:29:07,027 INFO [TaskCommunicator # 2] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_36_02_000002_0 2018-01-05T11:29:07,027 INFO [TaskCommunicator # 2] GuaranteedTasks: Received response for attempt_1515180518813_0001_36_02_000002_0, true 2018-01-05T11:29:07,027 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_36_02_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000084] on NM: [10.22.8.182:49818] 2018-01-05T11:29:07,028 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:07,028 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:07,028 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 3, taskAttemptId=attempt_1515180518813_0001_36_02_000000_0, startTime=1515180547027, containerId=container_111101111_0001_00_000084, nodeId=10.22.8.182:49818 2018-01-05T11:29:07,028 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000085 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:07,028 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000085, launchTime=1515180547028 2018-01-05T11:29:07,028 INFO [Task-Executor-2] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : insert overwrite table t...srcbucket_mapjoin (Stage-1), VertexName: Reducer 3, VertexParallelism: 4, TaskAttemptID:attempt_1515180518813_0001_36_02_000002_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Reducer 2, physicalEdgeCount=2, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:29:07,028 INFO [Task-Executor-2] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1026555904, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:07,028 INFO [Task-Executor-2] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_36_02_000002_0 vertexName: Reducer 3 2018-01-05T11:29:07,028 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_36_02_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000084, dagName=insert overwrite table t...srcbucket_mapjoin (Stage-1), vertexName=Reducer 3, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Reducer 2,org.apache.tez.runtime.library.input.OrderedGroupedKVInput,2}}, Outputs={{out_Reducer 3,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=7, completedTaskCount=2, dagStartTime=1515180546755, firstAttemptStartTime=1515180547024, currentAttemptStartTime=1515180547025} 2018-01-05T11:29:07,028 INFO [IPC Server handler 4 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000084, QueryId=djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd-36, DagId=36, VertexName=Reducer 3, TaskId=0, TaskAttemptId=0, SubmitTime=1515180547028 2018-01-05T11:29:07,028 DEBUG [IPC Server handler 4 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=36} with the ShuffleHandler 2018-01-05T11:29:07,028 DEBUG [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/36] 2018-01-05T11:29:07,028 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=3, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:07,028 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_36_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:07,028 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_36_01_000000_0 2018-01-05T11:29:07,028 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=2, runningFragmentCount=1 2018-01-05T11:29:07,028 DEBUG [IPC Server handler 4 on 49818] impl.QueryFragmentInfo: Cannot finish due to source: Reducer 2 2018-01-05T11:29:07,028 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000083, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_36_02_000002_0, eventCount=1 } 2018-01-05T11:29:07,028 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000081=164 2018-01-05T11:29:07,028 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_36_01_000000_0: true 2018-01-05T11:29:07,029 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_36_02_000001_0] submitted. Is using containerId: [container_111101111_0001_00_000085] on NM: [10.22.8.182:49818] 2018-01-05T11:29:07,029 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:07,029 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:07,029 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 3, taskAttemptId=attempt_1515180518813_0001_36_02_000001_0, startTime=1515180547028, containerId=container_111101111_0001_00_000085, nodeId=10.22.8.182:49818 2018-01-05T11:29:07,028 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_36_02_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:07,029 DEBUG [IPC Server handler 4 on 49818] impl.QueryFragmentInfo: Cannot finish due to source: Reducer 2 2018-01-05T11:29:07,029 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_36_02_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=false, canFinish(in queue)=false, isGuaranteed=false, firstAttemptStartTime=1515180547024, dagStartTime=1515180546755, withinDagPriority=32, vertexParallelism= 4, selfAndUpstreamParallelism= 7, selfAndUpstreamComplete= 2}] 2018-01-05T11:29:07,029 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_36_01_000000_0 2018-01-05T11:29:07,029 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_36_01_000000_0, creationTime=1515180546861, allocationTime=1515180546862, startTime=1515180546862, finishTime=1515180547029, timeTaken=167, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:07,029 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000081, QueryId=djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd-36, Vertex=Reducer 2, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:07,029 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_36_01_000000_0 complete. WaitQueueSize=1, numSlotsAvailable=3, preemptionQueueSize=1 2018-01-05T11:29:07,029 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_36_01_000000_0 2018-01-05T11:29:07,029 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_36_01_000000_0 2018-01-05T11:29:07,029 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000081, QueryId=djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd-36, DagId=36, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180546863, EndTime=1515180547029 2018-01-05T11:29:07,029 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_36_01_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:07,029 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_36_01_000000_0; false 2018-01-05T11:29:07,029 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_36_01_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:07,029 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_36_02_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=false, canFinish(in queue)=false, isGuaranteed=false, firstAttemptStartTime=1515180547024, dagStartTime=1515180546755, withinDagPriority=32, vertexParallelism= 4, selfAndUpstreamParallelism= 7, selfAndUpstreamComplete= 2} 2018-01-05T11:29:07,029 DEBUG [Wait-Queue-Scheduler-0] impl.QueryFragmentInfo: Cannot finish due to source: Reducer 2 2018-01-05T11:29:07,029 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_36_01_000000, startTime=1515180546862, finishTime=1515180547029, timeTaken=167, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_36_01_000000_0, diagnostics=, counters=Counters: 42, File System Counters, FILE_BYTES_READ=208, FILE_BYTES_WRITTEN=1666, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=80, REDUCE_INPUT_RECORDS=122, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=123, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=691, OUTPUT_BYTES_WITH_OVERHEAD=701, OUTPUT_BYTES_PHYSICAL=705, ADDITIONAL_SPILLS_BYTES_WRITTEN=1820, ADDITIONAL_SPILLS_BYTES_READ=1820, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, SHUFFLE_BYTES=1820, SHUFFLE_BYTES_DECOMPRESSED=1816, SHUFFLE_BYTES_TO_MEM=1820, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=27, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, HIVE, CREATED_FILES=1, RECORDS_OUT_1_default.tab=122, RECORDS_OUT_INTERMEDIATE_Reducer_2=1, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0 2018-01-05T11:29:07,029 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:07,029 DEBUG [TezTR-518813_1_36_2_2_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_36 queryId: djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd instanceType: TaskRunner2Callable 2018-01-05T11:29:07,029 INFO [TezTR-518813_1_36_2_2_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_36_02_000002_0 2018-01-05T11:29:07,029 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_36_02_000000_0 scheduled for execution. canFinish=false, isGuaranteed=false 2018-01-05T11:29:07,029 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_36_02_000000_0 to pre-emption queue 2018-01-05T11:29:07,029 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_36_01 [Reducer 2], tasks=2, failed=0, killed=0, success=2, completed=2, commits=0, err=null 2018-01-05T11:29:07,029 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_36_01 [Reducer 2] 2018-01-05T11:29:07,029 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:07,029 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_36_02_000000_0 : ACCEPTED 2018-01-05T11:29:07,029 DEBUG [Task-Executor-0] impl.QueryFragmentInfo: Cannot finish due to source: Reducer 2 2018-01-05T11:29:07,029 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_36_02_000000_0: false 2018-01-05T11:29:07,029 INFO [IPC Server handler 5 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_36_02_000002_0 2 events [0,2) total 2 vertex_1515180518813_0001_36_02 [Reducer 3] 2018-01-05T11:29:07,030 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: taskOwner hashCode:1942334159 2018-01-05T11:29:07,032 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=2, nextPreRoutedEventId=0, eventCount=2 } 2018-01-05T11:29:07,032 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_36_02_000002_0, eventCount=2 fromEventId=0 nextFromEventId=2 2018-01-05T11:29:07,032 INFO [TaskCommunicator # 4] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_36_02_000000_0 2018-01-05T11:29:07,032 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:VERTEX_FINISHED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_36_01, initRequestedTime=1515180546763, initedTime=1515180546769, startRequestedTime=1515180546793, startedTime=1515180546793, finishTime=1515180547029, timeTaken=236, status=SUCCEEDED, diagnostics=, counters=Counters: 42, File System Counters, FILE_BYTES_READ=416, FILE_BYTES_WRITTEN=3306, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=153, REDUCE_INPUT_RECORDS=242, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=244, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=2, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=1333, OUTPUT_BYTES_WITH_OVERHEAD=1353, OUTPUT_BYTES_PHYSICAL=1361, ADDITIONAL_SPILLS_BYTES_WRITTEN=3591, ADDITIONAL_SPILLS_BYTES_READ=3591, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=2, SHUFFLE_BYTES=3591, SHUFFLE_BYTES_DECOMPRESSED=3583, SHUFFLE_BYTES_TO_MEM=3591, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=16, MERGE_PHASE_TIME=55, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, HIVE, CREATED_FILES=2, RECORDS_OUT_1_default.tab=242, RECORDS_OUT_INTERMEDIATE_Reducer_2=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, vertexStats=firstTaskStartTime=1515180546862, firstTasksToStart=[ task_1515180518813_0001_36_01_000000 ], lastTaskFinishTime=1515180547029, lastTasksToFinish=[ task_1515180518813_0001_36_01_000000 ], minTaskDuration=159, maxTaskDuration=167, avgTaskDuration=163.0, numSuccessfulTasks=2, shortestDurationTasks=[ task_1515180518813_0001_36_01_000001 ], longestDurationTasks=[ task_1515180518813_0001_36_01_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=2, numSucceededTasks=2, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:07,032 INFO [TaskCommunicator # 4] GuaranteedTasks: Received response for attempt_1515180518813_0001_36_02_000000_0, true 2018-01-05T11:29:07,032 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_36_01 [Reducer 2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:07,032 INFO [Task-Executor-0] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : insert overwrite table t...srcbucket_mapjoin (Stage-1), VertexName: Reducer 3, VertexParallelism: 4, TaskAttemptID:attempt_1515180518813_0001_36_02_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Reducer 2, physicalEdgeCount=2, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:29:07,032 INFO [Task-Executor-0] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1026555904, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:07,032 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000081, stoppedTime=1515180547032, exitStatus=0 2018-01-05T11:29:07,032 INFO [Task-Executor-0] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_36_02_000000_0 vertexName: Reducer 3 2018-01-05T11:29:07,032 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000081 exited with diagnostics set to Container completed. 2018-01-05T11:29:07,032 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_36_02 [Reducer 3] attempt: attempt_1515180518813_0001_36_01_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:29:07,033 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_36_01 [Reducer 2] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3 2018-01-05T11:29:07,033 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null 2018-01-05T11:29:07,033 INFO [App Shared Pool - #1] vertexmanager.ShuffleVertexManagerBase: All source tasks completed. Ramping up 1 remaining tasks for vertex: Reducer 3 2018-01-05T11:29:07,033 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000081 2018-01-05T11:29:07,033 INFO [App Shared Pool - #1] vertexmanager.ShuffleVertexManagerBase: Scheduling 1 tasks for vertex: Reducer 3 with totalTasks: 4. 2 source tasks completed out of 2. MinSourceTaskCompletedFraction: 1.0 min: 0.05 max: 0.75 2018-01-05T11:29:07,033 DEBUG [TezTR-518813_1_36_2_2_0] log.PerfLogger: 2018-01-05T11:29:07,033 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:TASK_STARTED]: vertexName=Reducer 3, taskId=task_1515180518813_0001_36_02_000003, scheduledTime=1515180547033, launchTime=1515180547033 2018-01-05T11:29:07,033 INFO [I/O Setup 0 Initialize: {Reducer 2}] orderedgrouped.MergeManager: Initial Memory required for SHUFFLE_BUFFER=131404920 based on INPUT_BUFFER_FACTOR=0.7, for final merged output=0, using factor: 0.0 2018-01-05T11:29:07,033 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_36_02_000003_0, priority=32, capability=, hosts=null 2018-01-05T11:29:07,033 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_36_02_000001_0, am_details=10.22.8.182:49824, taskInfo= fragment 1 attempt 0, user=djaiswal, queryId=djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000085, dagName=insert overwrite table t...srcbucket_mapjoin (Stage-1), vertexName=Reducer 3, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Reducer 2,org.apache.tez.runtime.library.input.OrderedGroupedKVInput,2}}, Outputs={{out_Reducer 3,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=7, completedTaskCount=2, dagStartTime=1515180546755, firstAttemptStartTime=1515180547024, currentAttemptStartTime=1515180547026} 2018-01-05T11:29:07,033 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=32,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:07,033 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000084, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_36_02_000000_0, eventCount=1 } 2018-01-05T11:29:07,033 INFO [IPC Server handler 3 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000085, QueryId=djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd-36, DagId=36, VertexName=Reducer 3, TaskId=1, TaskAttemptId=0, SubmitTime=1515180547033 2018-01-05T11:29:07,033 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=32,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:07,033 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:07,033 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_36_02_000003_0 on hosts=null 2018-01-05T11:29:07,033 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_36_02_000003_0 on hosts=null 2018-01-05T11:29:07,033 DEBUG [IPC Server handler 3 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=36} with the ShuffleHandler 2018-01-05T11:29:07,033 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_36_02_000003_0 2018-01-05T11:29:07,033 DEBUG [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/36] 2018-01-05T11:29:07,033 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=3, ac=5, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:07,033 DEBUG [TezTR-518813_1_36_2_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_36 queryId: djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd instanceType: TaskRunner2Callable 2018-01-05T11:29:07,033 INFO [TezTR-518813_1_36_2_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_36_02_000000_0 2018-01-05T11:29:07,033 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #86, task=TaskInfo{task=attempt_1515180518813_0001_36_02_000003_0, priority=32, startTime=0, containerId=null, uniqueId=85, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=3, ac=5, commF=false, disabled=false}, to container=container_111101111_0001_00_000086 2018-01-05T11:29:07,033 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_36_02_000003_0; false 2018-01-05T11:29:07,033 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_36_02_000003_0, priority=32, startTime=311567543, containerId=container_111101111_0001_00_000086assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=4, ac=4, commF=false, disabled=false}, uniqueId=85, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:07,034 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=2, runningFragmentCount=2 2018-01-05T11:29:07,034 DEBUG [IPC Server handler 3 on 49818] impl.QueryFragmentInfo: Cannot finish due to source: Reducer 2 2018-01-05T11:29:07,034 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_36_02_000001_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:07,034 DEBUG [IPC Server handler 3 on 49818] impl.QueryFragmentInfo: Cannot finish due to source: Reducer 2 2018-01-05T11:29:07,034 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_36_02_000001_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=false, canFinish(in queue)=false, isGuaranteed=false, firstAttemptStartTime=1515180547024, dagStartTime=1515180546755, withinDagPriority=32, vertexParallelism= 4, selfAndUpstreamParallelism= 7, selfAndUpstreamComplete= 2}] 2018-01-05T11:29:07,034 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_36_02_000001_0 : ACCEPTED 2018-01-05T11:29:07,034 INFO [IPC Server handler 3 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_36_02_000000_0 2 events [0,2) total 2 vertex_1515180518813_0001_36_02 [Reducer 3] 2018-01-05T11:29:07,034 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_36_02_000001_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=false, canFinish(in queue)=false, isGuaranteed=false, firstAttemptStartTime=1515180547024, dagStartTime=1515180546755, withinDagPriority=32, vertexParallelism= 4, selfAndUpstreamParallelism= 7, selfAndUpstreamComplete= 2} 2018-01-05T11:29:07,034 DEBUG [Wait-Queue-Scheduler-0] impl.QueryFragmentInfo: Cannot finish due to source: Reducer 2 2018-01-05T11:29:07,034 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_36_02_000001_0 scheduled for execution. canFinish=false, isGuaranteed=false 2018-01-05T11:29:07,034 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_36_02_000001_0 to pre-emption queue 2018-01-05T11:29:07,034 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000086 for node: 10.22.8.182:49818 2018-01-05T11:29:07,034 DEBUG [Task-Executor-3] impl.QueryFragmentInfo: Cannot finish due to source: Reducer 2 2018-01-05T11:29:07,034 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_36_02_000001_0: false 2018-01-05T11:29:07,034 INFO [TaskCommunicator # 5] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_36_02_000001_0 2018-01-05T11:29:07,034 INFO [TaskCommunicator # 5] GuaranteedTasks: Received response for attempt_1515180518813_0001_36_02_000001_0, true 2018-01-05T11:29:07,034 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: taskOwner hashCode:1756576102 2018-01-05T11:29:07,034 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=2, nextPreRoutedEventId=0, eventCount=2 } 2018-01-05T11:29:07,034 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_36_02_000000_0, eventCount=2 fromEventId=0 nextFromEventId=2 2018-01-05T11:29:07,034 DEBUG [TezTR-518813_1_36_2_0_0] log.PerfLogger: 2018-01-05T11:29:07,035 INFO [I/O Setup 0 Initialize: {Reducer 2}] orderedgrouped.MergeManager: Initial Memory required for SHUFFLE_BUFFER=131404920 based on INPUT_BUFFER_FACTOR=0.7, for final merged output=0, using factor: 0.0 2018-01-05T11:29:07,034 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000086, attempt_1515180518813_0001_36_02_000003_0 for node: 10.22.8.182:49818 2018-01-05T11:29:07,035 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000086 for node: 10.22.8.182:49818 2018-01-05T11:29:07,035 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000086 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:07,035 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000086, launchTime=1515180547035 2018-01-05T11:29:07,036 INFO [Task-Executor-3] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : insert overwrite table t...srcbucket_mapjoin (Stage-1), VertexName: Reducer 3, VertexParallelism: 4, TaskAttemptID:attempt_1515180518813_0001_36_02_000001_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Reducer 2, physicalEdgeCount=2, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:29:07,036 INFO [Task-Executor-3] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1026555904, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:07,036 INFO [Task-Executor-3] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_36_02_000001_0 vertexName: Reducer 3 2018-01-05T11:29:07,036 DEBUG [TezTR-518813_1_36_2_2_0] log.PerfLogger: 2018-01-05T11:29:07,036 INFO [TezTR-518813_1_36_2_2_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:07,036 INFO [TezTR-518813_1_36_2_2_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:07,036 INFO [TezTR-518813_1_36_2_2_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:07,036 INFO [I/O Setup 1 Initialize: {out_Reducer 3}] output.MROutput: out_Reducer 3: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:29:07,036 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=36}, sourceName=Reducer 2, state=S_SUCCEEDED 2018-01-05T11:29:07,036 INFO [IPC Server handler 1 on 49818] impl.TaskExecutorService: Received finishable state update for attempt_1515180518813_0001_36_02_000002_0, state=true 2018-01-05T11:29:07,036 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:07,036 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:07,036 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:07,036 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:07,036 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Fragment attempt_1515180518813_0001_36_02_000002_0 guaranteed state changed to false; finishable true, in wait queue false, in preemption queue true 2018-01-05T11:29:07,036 INFO [IPC Server handler 1 on 49818] impl.TaskExecutorService: Received finishable state update for attempt_1515180518813_0001_36_02_000000_0, state=true 2018-01-05T11:29:07,036 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Fragment attempt_1515180518813_0001_36_02_000000_0 guaranteed state changed to false; finishable true, in wait queue false, in preemption queue true 2018-01-05T11:29:07,036 INFO [IPC Server handler 1 on 49818] impl.TaskExecutorService: Received finishable state update for attempt_1515180518813_0001_36_02_000001_0, state=true 2018-01-05T11:29:07,037 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Fragment attempt_1515180518813_0001_36_02_000001_0 guaranteed state changed to false; finishable true, in wait queue false, in preemption queue true 2018-01-05T11:29:07,037 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000085, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_36_02_000001_0, eventCount=1 } 2018-01-05T11:29:07,037 INFO [TezTR-518813_1_36_2_2_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:07,037 INFO [TezTR-518813_1_36_2_2_0] resources.MemoryDistributor: InitialRequests=[out_Reducer 3:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [Reducer 2:INPUT:131404920:org.apache.tez.runtime.library.input.OrderedGroupedKVInput] 2018-01-05T11:29:07,037 DEBUG [TezTR-518813_1_36_2_1_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_36 queryId: djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd instanceType: TaskRunner2Callable 2018-01-05T11:29:07,037 INFO [TezTR-518813_1_36_2_2_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:07,037 INFO [TezTR-518813_1_36_2_1_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_36_02_000001_0 2018-01-05T11:29:07,037 INFO [TezTR-518813_1_36_2_2_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:07,037 INFO [TezTR-518813_1_36_2_2_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 131404920, TotalRequestedScaled: 1.2129684923076923E8, TotalJVMHeap: 1026555904, TotalAvailable: 687792455, TotalRequested/TotalJVMHeap:0.13 2018-01-05T11:29:07,037 INFO [TezTR-518813_1_36_2_2_0] resources.MemoryDistributor: Allocations=[out_Reducer 3:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:131404920:131404920] 2018-01-05T11:29:07,037 INFO [TezTR-518813_1_36_2_2_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:07,038 DEBUG [TezTR-518813_1_36_2_0_0] log.PerfLogger: 2018-01-05T11:29:07,038 INFO [TezTR-518813_1_36_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:07,038 INFO [TezTR-518813_1_36_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:07,038 INFO [TezTR-518813_1_36_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:07,038 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:07,038 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_36_02_000003_0] submitted. Is using containerId: [container_111101111_0001_00_000086] on NM: [10.22.8.182:49818] 2018-01-05T11:29:07,038 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:07,038 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:07,038 INFO [I/O Setup 0 Initialize: {Reducer 2}] orderedgrouped.MergeManager: Initial Memory required for SHUFFLE_BUFFER=131404920 based on INPUT_BUFFER_FACTOR=0.7, for final merged output=0, using factor: 0.0 2018-01-05T11:29:07,038 DEBUG [TezTR-518813_1_36_2_1_0] log.PerfLogger: 2018-01-05T11:29:07,038 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 3, taskAttemptId=attempt_1515180518813_0001_36_02_000003_0, startTime=1515180547035, containerId=container_111101111_0001_00_000086, nodeId=10.22.8.182:49818 2018-01-05T11:29:07,038 INFO [TezTR-518813_1_36_2_2_0] runtime.LogicalIOProcessorRuntimeTask: Input: Reducer 2 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:07,038 INFO [TezTR-518813_1_36_2_2_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:29:07,038 INFO [TezTR-518813_1_36_2_2_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:07,038 INFO [I/O Setup 1 Initialize: {out_Reducer 3}] output.MROutput: out_Reducer 3: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:29:07,039 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:07,039 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:07,039 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:07,039 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:07,038 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_36_02_000003_0, am_details=10.22.8.182:49824, taskInfo= fragment 3 attempt 0, user=djaiswal, queryId=djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000086, dagName=insert overwrite table t...srcbucket_mapjoin (Stage-1), vertexName=Reducer 3, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Reducer 2,org.apache.tez.runtime.library.input.OrderedGroupedKVInput,2}}, Outputs={{out_Reducer 3,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=7, completedTaskCount=3, dagStartTime=1515180546755, firstAttemptStartTime=1515180547033, currentAttemptStartTime=1515180547035} 2018-01-05T11:29:07,039 INFO [IPC Server handler 0 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000086, QueryId=djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd-36, DagId=36, VertexName=Reducer 3, TaskId=3, TaskAttemptId=0, SubmitTime=1515180547039 2018-01-05T11:29:07,039 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:07,039 DEBUG [IPC Server handler 0 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=36} with the ShuffleHandler 2018-01-05T11:29:07,039 DEBUG [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/36] 2018-01-05T11:29:07,039 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=1, runningFragmentCount=3 2018-01-05T11:29:07,040 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_36_02_000003_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:07,040 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_36_02_000003_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180547033, dagStartTime=1515180546755, withinDagPriority=32, vertexParallelism= 4, selfAndUpstreamParallelism= 7, selfAndUpstreamComplete= 3}] 2018-01-05T11:29:07,040 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_36_02_000003_0 : ACCEPTED 2018-01-05T11:29:07,040 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_36_02_000003_0, canFinish=true. Current state: preemptionQueueSize=3, numSlotsAvailable=1, waitQueueSize=1 2018-01-05T11:29:07,040 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_36_02_000003_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180547033, dagStartTime=1515180546755, withinDagPriority=32, vertexParallelism= 4, selfAndUpstreamParallelism= 7, selfAndUpstreamComplete= 3} 2018-01-05T11:29:07,040 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_36_02_000003_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:07,040 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_36_02_000003_0 to pre-emption queue 2018-01-05T11:29:07,040 INFO [TezTR-518813_1_36_2_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:07,040 INFO [TezTR-518813_1_36_2_0_0] resources.MemoryDistributor: InitialRequests=[Reducer 2:INPUT:131404920:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [out_Reducer 3:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput] 2018-01-05T11:29:07,040 INFO [TezTR-518813_1_36_2_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:07,040 INFO [TezTR-518813_1_36_2_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:07,040 INFO [TezTR-518813_1_36_2_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 131404920, TotalRequestedScaled: 1.2129684923076923E8, TotalJVMHeap: 1026555904, TotalAvailable: 687792455, TotalRequested/TotalJVMHeap:0.13 2018-01-05T11:29:07,040 INFO [TezTR-518813_1_36_2_0_0] resources.MemoryDistributor: Allocations=[Reducer 2:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:131404920:131404920], [out_Reducer 3:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0] 2018-01-05T11:29:07,040 INFO [TezTR-518813_1_36_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:07,040 INFO [TezTR-518813_1_36_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: Reducer 2 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:07,040 INFO [TezTR-518813_1_36_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:29:07,040 INFO [I/O Setup 0 Start: {Reducer 2}] orderedgrouped.Shuffle: Reducer_2: Shuffle assigned with 2 inputs, codec: None, ifileReadAhead: true 2018-01-05T11:29:07,040 INFO [TezTR-518813_1_36_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:07,040 INFO [I/O Setup 0 Start: {Reducer 2}] orderedgrouped.MergeManager: Reducer 2: MergerManager: memoryLimit=131404920, maxSingleShuffleLimit=32851230, mergeThreshold=86727248, ioSortFactor=10, postMergeMem=0, memToMemMergeOutputsThreshold=10 2018-01-05T11:29:07,040 INFO [TaskCommunicator # 7] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_36_02_000003_0 2018-01-05T11:29:07,040 INFO [TaskCommunicator # 7] GuaranteedTasks: Received response for attempt_1515180518813_0001_36_02_000003_0, true 2018-01-05T11:29:07,040 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_36_02_000003_0: true 2018-01-05T11:29:07,041 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: taskOwner hashCode:1752428668 2018-01-05T11:29:07,041 INFO [I/O Setup 0 Start: {Reducer 2}] orderedgrouped.ShuffleScheduler: ShuffleScheduler running for sourceVertex: Reducer 2 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=2, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=2, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true 2018-01-05T11:29:07,041 INFO [IPC Server handler 2 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_36_02_000001_0 2 events [0,2) total 2 vertex_1515180518813_0001_36_02 [Reducer 3] 2018-01-05T11:29:07,041 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=2, nextPreRoutedEventId=0, eventCount=2 } 2018-01-05T11:29:07,041 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_36_02_000001_0, eventCount=2 fromEventId=0 nextFromEventId=2 2018-01-05T11:29:07,042 INFO [I/O Setup 0 Start: {Reducer 2}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: Reducer 2 2018-01-05T11:29:07,042 INFO [I/O Setup 0 Start: {Reducer 2}] orderedgrouped.Shuffle: Reducer_2: Shuffle assigned with 2 inputs, codec: None, ifileReadAhead: true 2018-01-05T11:29:07,042 INFO [I/O Setup 0 Start: {Reducer 2}] orderedgrouped.MergeManager: Reducer 2: MergerManager: memoryLimit=131404920, maxSingleShuffleLimit=32851230, mergeThreshold=86727248, ioSortFactor=10, postMergeMem=0, memToMemMergeOutputsThreshold=10 2018-01-05T11:29:07,043 INFO [TezTR-518813_1_36_2_2_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:07,043 DEBUG [TezTR-518813_1_36_2_1_0] log.PerfLogger: 2018-01-05T11:29:07,043 INFO [TezTR-518813_1_36_2_1_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:07,043 INFO [TezTR-518813_1_36_2_1_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:07,043 INFO [TezTR-518813_1_36_2_1_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:07,043 INFO [Task-Executor-1] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : insert overwrite table t...srcbucket_mapjoin (Stage-1), VertexName: Reducer 3, VertexParallelism: 4, TaskAttemptID:attempt_1515180518813_0001_36_02_000003_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Reducer 2, physicalEdgeCount=2, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:29:07,043 INFO [TezTR-518813_1_36_2_2_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_36_02_000002_0 2018-01-05T11:29:07,043 DEBUG [TezTR-518813_1_36_2_2_0] log.PerfLogger: 2018-01-05T11:29:07,043 DEBUG [TezTR-518813_1_36_2_2_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_36_02_000002_0_10250 2018-01-05T11:29:07,043 INFO [Task-Executor-1] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1026555904, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:07,043 DEBUG [TezTR-518813_1_36_2_2_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:07,043 INFO [Task-Executor-1] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_36_02_000003_0 vertexName: Reducer 3 2018-01-05T11:29:07,043 DEBUG [TezTR-518813_1_36_2_2_0] mr.ObjectCache: Creating djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd_Reducer 3__REDUCE_PLAN__ 2018-01-05T11:29:07,044 INFO [I/O Setup 0 Start: {Reducer 2}] orderedgrouped.ShuffleScheduler: ShuffleScheduler running for sourceVertex: Reducer 2 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=2, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=2, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true 2018-01-05T11:29:07,044 INFO [TezTR-518813_1_36_2_2_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/djaiswal/_tez_scratch_dir/e826609d-5b54-47c6-a628-3fb7d0722b70/reduce.xml 2018-01-05T11:29:07,044 DEBUG [TezTR-518813_1_36_2_2_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/djaiswal/_tez_scratch_dir/e826609d-5b54-47c6-a628-3fb7d0722b70/reduce.xml 2018-01-05T11:29:07,044 DEBUG [TezTR-518813_1_36_2_2_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/djaiswal/_tez_scratch_dir/e826609d-5b54-47c6-a628-3fb7d0722b70/reduce.xml 2018-01-05T11:29:07,044 INFO [TezTaskEventRouter{attempt_1515180518813_0001_36_02_000002_0}] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Reducer 2: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:29:07,044 DEBUG [TezTR-518813_1_36_2_2_0] log.PerfLogger: 2018-01-05T11:29:07,044 INFO [TezTR-518813_1_36_2_2_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:29:07,044 DEBUG [TezTR-518813_1_36_2_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,044 INFO [I/O Setup 1 Initialize: {out_Reducer 3}] output.MROutput: out_Reducer 3: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:29:07,044 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000086, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_36_02_000003_0, eventCount=1 } 2018-01-05T11:29:07,044 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:07,044 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:07,044 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:07,044 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:07,044 INFO [I/O Setup 0 Start: {Reducer 2}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: Reducer 2 2018-01-05T11:29:07,044 INFO [TezTR-518813_1_36_2_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:07,044 INFO [TezTR-518813_1_36_2_1_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:07,044 INFO [TezTR-518813_1_36_2_1_0] resources.MemoryDistributor: InitialRequests=[Reducer 2:INPUT:131404920:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [out_Reducer 3:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput] 2018-01-05T11:29:07,044 INFO [TezTR-518813_1_36_2_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_36_02_000000_0 2018-01-05T11:29:07,044 INFO [TezTR-518813_1_36_2_1_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:07,044 DEBUG [TezTR-518813_1_36_2_0_0] log.PerfLogger: 2018-01-05T11:29:07,045 DEBUG [TezTR-518813_1_36_2_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_36_02_000000_0_10253 2018-01-05T11:29:07,045 INFO [TezTR-518813_1_36_2_1_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:07,045 DEBUG [TezTR-518813_1_36_2_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:07,045 DEBUG [TezTR-518813_1_36_2_0_0] mr.ObjectCache: Creating djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd_Reducer 3__REDUCE_PLAN__ 2018-01-05T11:29:07,045 INFO [TezTR-518813_1_36_2_1_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 131404920, TotalRequestedScaled: 1.2129684923076923E8, TotalJVMHeap: 1026555904, TotalAvailable: 687792455, TotalRequested/TotalJVMHeap:0.13 2018-01-05T11:29:07,045 INFO [TezTR-518813_1_36_2_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/djaiswal/_tez_scratch_dir/e826609d-5b54-47c6-a628-3fb7d0722b70/reduce.xml 2018-01-05T11:29:07,045 DEBUG [TezTR-518813_1_36_2_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/djaiswal/_tez_scratch_dir/e826609d-5b54-47c6-a628-3fb7d0722b70/reduce.xml 2018-01-05T11:29:07,045 DEBUG [TezTR-518813_1_36_2_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/djaiswal/_tez_scratch_dir/e826609d-5b54-47c6-a628-3fb7d0722b70/reduce.xml 2018-01-05T11:29:07,045 INFO [TezTR-518813_1_36_2_1_0] resources.MemoryDistributor: Allocations=[Reducer 2:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:131404920:131404920], [out_Reducer 3:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0] 2018-01-05T11:29:07,045 DEBUG [TezTR-518813_1_36_2_0_0] log.PerfLogger: 2018-01-05T11:29:07,045 INFO [TezTR-518813_1_36_2_0_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:29:07,045 DEBUG [TezTR-518813_1_36_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,045 INFO [TezTR-518813_1_36_2_1_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:07,045 INFO [TezTR-518813_1_36_2_1_0] runtime.LogicalIOProcessorRuntimeTask: Input: Reducer 2 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:07,045 INFO [TezTR-518813_1_36_2_1_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:29:07,045 INFO [TezTR-518813_1_36_2_1_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:07,046 DEBUG [TezTR-518813_1_36_2_2_0] log.PerfLogger: 2018-01-05T11:29:07,046 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=36&reduce=2&map=attempt_1515180518813_0001_36_01_000001_0_10249,attempt_1515180518813_0001_36_01_000000_0_10246 mapId: [attempt_1515180518813_0001_36_01_000001_0_10249, attempt_1515180518813_0001_36_01_000000_0_10246] reduceId: [2] jobId: [job_1515180518813_0001] dagId: [36] keepAlive: false 2018-01-05T11:29:07,046 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:07,046 INFO [TezTR-518813_1_36_2_2_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 3 size: 2.66KB 2018-01-05T11:29:07,046 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=36&reduce=2&map=attempt_1515180518813_0001_36_01_000001_0_10249,attempt_1515180518813_0001_36_01_000000_0_10246; hash=...k/6vyE4+EC6hI 2018-01-05T11:29:07,046 DEBUG [TezTR-518813_1_36_2_3_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_36 queryId: djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd instanceType: TaskRunner2Callable 2018-01-05T11:29:07,046 DEBUG [TezTR-518813_1_36_2_0_0] log.PerfLogger: 2018-01-05T11:29:07,046 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=36&reduce=2&map=attempt_1515180518813_0001_36_01_000001_0_10249,attempt_1515180518813_0001_36_01_000000_0_10246;reply=nFlRjKOV8sWLo 2018-01-05T11:29:07,046 INFO [TezTR-518813_1_36_2_3_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_36_02_000003_0 2018-01-05T11:29:07,046 INFO [TezTR-518813_1_36_2_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 3 size: 2.66KB 2018-01-05T11:29:07,046 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=36, user='djaiswal', attemptId='attempt_1515180518813_0001_36_01_000001_0_10249'} via loader 2018-01-05T11:29:07,046 DEBUG [TezTR-518813_1_36_2_2_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:07,046 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:07,046 DEBUG [TezTR-518813_1_36_2_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:07,046 DEBUG [TezTR-518813_1_36_2_2_0] log.PerfLogger: 2018-01-05T11:29:07,046 DEBUG [TezTR-518813_1_36_2_0_0] log.PerfLogger: 2018-01-05T11:29:07,046 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=36, user='djaiswal', attemptId='attempt_1515180518813_0001_36_01_000001_0_10249'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:29:07,047 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_36_01_000001_0_10249 not found 2018-01-05T11:29:07,047 INFO [TezTaskEventRouter{attempt_1515180518813_0001_36_02_000000_0}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 2. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:07,047 INFO [TezTaskEventRouter{attempt_1515180518813_0001_36_02_000000_0}] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Reducer 2: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=1, numObsoletionEventsSeen=0 2018-01-05T11:29:07,047 INFO [TezTaskEventRouter{attempt_1515180518813_0001_36_02_000000_0}] orderedgrouped.ShuffleScheduler: All inputs fetched for input vertex : Reducer 2 2018-01-05T11:29:07,047 INFO [I/O Setup 0 Start: {Reducer 2}] orderedgrouped.Shuffle: Reducer_2: Shuffle assigned with 2 inputs, codec: None, ifileReadAhead: true 2018-01-05T11:29:07,047 INFO [TezTaskEventRouter{attempt_1515180518813_0001_36_02_000000_0}] orderedgrouped.ShuffleScheduler: copy(2 (spillsFetched=2) of 2. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:07,047 DEBUG [TezTR-518813_1_36_2_2_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:07,047 INFO [I/O Setup 0 Start: {Reducer 2}] orderedgrouped.MergeManager: Reducer 2: MergerManager: memoryLimit=131404920, maxSingleShuffleLimit=32851230, mergeThreshold=86727248, ioSortFactor=10, postMergeMem=0, memToMemMergeOutputsThreshold=10 2018-01-05T11:29:07,047 DEBUG [TezTR-518813_1_36_2_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:07,047 DEBUG [TezTR-518813_1_36_2_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:07,047 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_36_01_000001_0_10249,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/36/output/attempt_1515180518813_0001_36_01_000001_0_10249/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/36/output/attempt_1515180518813_0001_36_01_000001_0_10249/file.out.index 2018-01-05T11:29:07,047 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.ShuffleScheduler: Shutting down FetchScheduler for input: Reducer_2, wasInterrupted=false 2018-01-05T11:29:07,047 DEBUG [TezTR-518813_1_36_2_2_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:07,047 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=36, user='djaiswal', attemptId='attempt_1515180518813_0001_36_01_000000_0_10246'} via loader 2018-01-05T11:29:07,047 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=36, user='djaiswal', attemptId='attempt_1515180518813_0001_36_01_000000_0_10246'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:29:07,047 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_36_01_000000_0_10246 not found 2018-01-05T11:29:07,047 DEBUG [TezTR-518813_1_36_2_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:07,048 INFO [TezTR-518813_1_36_2_0_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:29:07,048 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_36_01_000000_0_10246,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/36/output/attempt_1515180518813_0001_36_01_000000_0_10246/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/36/output/attempt_1515180518813_0001_36_01_000000_0_10246/file.out.index 2018-01-05T11:29:07,048 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Content Length in shuffle : 1471 2018-01-05T11:29:07,048 INFO [IPC Server handler 4 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_36_02_000003_0 2 events [0,2) total 2 vertex_1515180518813_0001_36_02 [Reducer 3] 2018-01-05T11:29:07,048 DEBUG [TezTR-518813_1_36_2_3_0] log.PerfLogger: 2018-01-05T11:29:07,048 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.ShuffleScheduler: copy(2 (spillsFetched=2) of 2. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:07,048 DEBUG [TezTR-518813_1_36_2_2_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:07,049 INFO [TezTR-518813_1_36_2_2_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:29:07,049 INFO [Fetcher_O {Reducer_2} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=36&reduce=2&map=attempt_1515180518813_0001_36_01_000001_0_10249,attempt_1515180518813_0001_36_01_000000_0_10246 sent hash and receievd reply 0 ms 2018-01-05T11:29:07,048 INFO [I/O Setup 0 Start: {Reducer 2}] orderedgrouped.ShuffleScheduler: ShuffleScheduler running for sourceVertex: Reducer 2 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=2, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=2, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true 2018-01-05T11:29:07,049 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.ShuffleScheduler: Shutting down fetchers for input: Reducer_2, shutdown timetaken: 0 ms, hasFetcherExecutorStopped: true 2018-01-05T11:29:07,049 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=2, nextPreRoutedEventId=0, eventCount=2 } 2018-01-05T11:29:07,049 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_36_02_000003_0, eventCount=2 fromEventId=0 nextFromEventId=2 2018-01-05T11:29:07,050 DEBUG [New I/O worker #1] shufflehandler.FadvisedFileRegion: shuffleBufferSize: 131072, path: /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/36/output/attempt_1515180518813_0001_36_01_000001_0_10249/file.out 2018-01-05T11:29:07,050 INFO [I/O Setup 0 Initialize: {Reducer 2}] orderedgrouped.MergeManager: Initial Memory required for SHUFFLE_BUFFER=131404920 based on INPUT_BUFFER_FACTOR=0.7, for final merged output=0, using factor: 0.0 2018-01-05T11:29:07,050 DEBUG [New I/O worker #1] shufflehandler.FadvisedFileRegion: shuffleBufferSize: 131072, path: /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/36/output/attempt_1515180518813_0001_36_01_000000_0_10246/file.out 2018-01-05T11:29:07,050 INFO [I/O Setup 0 Start: {Reducer 2}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: Reducer 2 2018-01-05T11:29:07,050 INFO [TezTR-518813_1_36_2_1_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:07,051 INFO [Fetcher_O {Reducer_2} #0] orderedgrouped.MergeManager: CloseInMemoryFile. Current state: inMemoryMapOutputs.size=1, commitMemory=0, usedMemory=652. Since last log: count=1, min=652, max=652, total=652, avg=652.0 2018-01-05T11:29:07,051 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.MergeManager: finalMerge with #inMemoryOutputs=0, size=0 and #onDiskOutputs=0, size=0 2018-01-05T11:29:07,051 INFO [Fetcher_O {Reducer_2} #0] ShuffleScheduler.fetch: Completed fetch for attempt: {1, 0, attempt_1515180518813_0001_36_01_000001_0_10249} to MEMORY, csize=656, dsize=652, EndTime=1515180547051, TimeTaken=2, Rate=0.31 MB/s 2018-01-05T11:29:07,051 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.MergeManager: . DiskSeg: 0, 0. MemSeg: 0, 0 2018-01-05T11:29:07,051 INFO [ShuffleAndMergeRunner {Reducer_2}] impl.TezMerger: Nothing to merge. Returning an empty iterator 2018-01-05T11:29:07,051 INFO [TezTR-518813_1_36_2_1_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_36_02_000001_0 2018-01-05T11:29:07,051 INFO [Fetcher_O {Reducer_2} #0] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 2. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:07,051 INFO [Fetcher_O {Reducer_2} #0] ShuffleScheduler.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_36_01_000000_0_10246} to MEMORY, csize=705, dsize=701, EndTime=1515180547051, TimeTaken=0, Rate=0.00 MB/s 2018-01-05T11:29:07,051 INFO [Fetcher_O {Reducer_2} #0] orderedgrouped.ShuffleScheduler: All inputs fetched for input vertex : Reducer 2 2018-01-05T11:29:07,051 INFO [TezTR-518813_1_36_2_0_0] tez.ReduceRecordProcessor: Main work is Reducer 3 2018-01-05T11:29:07,051 INFO [Fetcher_O {Reducer_2} #0] orderedgrouped.ShuffleScheduler: copy(2 (spillsFetched=2) of 2. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:07,051 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.Shuffle: merge complete for input vertex : Reducer_2 2018-01-05T11:29:07,051 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.Shuffle: Reducer_2: Shuffle Runner thread complete 2018-01-05T11:29:07,051 DEBUG [TezTR-518813_1_36_2_1_0] log.PerfLogger: 2018-01-05T11:29:07,052 DEBUG [TezTR-518813_1_36_2_1_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_36_02_000001_0_10256 2018-01-05T11:29:07,052 DEBUG [TezTR-518813_1_36_2_1_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:07,052 DEBUG [TezTR-518813_1_36_2_1_0] mr.ObjectCache: Creating djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd_Reducer 3__REDUCE_PLAN__ 2018-01-05T11:29:07,052 INFO [TezTR-518813_1_36_2_1_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/djaiswal/_tez_scratch_dir/e826609d-5b54-47c6-a628-3fb7d0722b70/reduce.xml 2018-01-05T11:29:07,052 DEBUG [TezTR-518813_1_36_2_1_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/djaiswal/_tez_scratch_dir/e826609d-5b54-47c6-a628-3fb7d0722b70/reduce.xml 2018-01-05T11:29:07,052 DEBUG [TezTR-518813_1_36_2_1_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/djaiswal/_tez_scratch_dir/e826609d-5b54-47c6-a628-3fb7d0722b70/reduce.xml 2018-01-05T11:29:07,052 DEBUG [TezTR-518813_1_36_2_1_0] log.PerfLogger: 2018-01-05T11:29:07,052 INFO [TezTaskEventRouter{attempt_1515180518813_0001_36_02_000001_0}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 2. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:07,052 INFO [TezTaskEventRouter{attempt_1515180518813_0001_36_02_000001_0}] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Reducer 2: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=1, numObsoletionEventsSeen=0 2018-01-05T11:29:07,051 INFO [TezTR-518813_1_36_2_0_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:07,052 DEBUG [TezTR-518813_1_36_2_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:07,052 INFO [TezTaskEventRouter{attempt_1515180518813_0001_36_02_000001_0}] orderedgrouped.ShuffleScheduler: All inputs fetched for input vertex : Reducer 2 2018-01-05T11:29:07,052 INFO [TezTaskEventRouter{attempt_1515180518813_0001_36_02_000001_0}] orderedgrouped.ShuffleScheduler: copy(2 (spillsFetched=2) of 2. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:07,051 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.ShuffleScheduler: Shutting down FetchScheduler for input: Reducer_2, wasInterrupted=false 2018-01-05T11:29:07,052 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.ShuffleScheduler: Shutting down FetchScheduler for input: Reducer_2, wasInterrupted=false 2018-01-05T11:29:07,052 INFO [TezTR-518813_1_36_2_1_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:29:07,052 DEBUG [TezTR-518813_1_36_2_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,052 INFO [TezTR-518813_1_36_2_0_0] input.OrderedGroupedKVInput: Reducer 2: creating ValuesIterator with comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator, keyClass=org.apache.hadoop.hive.ql.io.HiveKey, valClass=org.apache.hadoop.io.BytesWritable 2018-01-05T11:29:07,053 DEBUG [TezTR-518813_1_36_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,053 DEBUG [TezTR-518813_1_36_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,053 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.ShuffleScheduler: copy(2 (spillsFetched=2) of 2. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:07,053 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.ShuffleScheduler: copy(2 (spillsFetched=2) of 2. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:07,053 DEBUG [TezTR-518813_1_36_2_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:29:07,053 DEBUG [TezTR-518813_1_36_2_0_0] log.PerfLogger: 2018-01-05T11:29:07,054 INFO [TezTR-518813_1_36_2_0_0] exec.GroupByOperator: Initializing operator GBY[4] 2018-01-05T11:29:07,054 DEBUG [TezTR-518813_1_36_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,054 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.ShuffleScheduler: Shutting down fetchers for input: Reducer_2, shutdown timetaken: 0 ms, hasFetcherExecutorStopped: true 2018-01-05T11:29:07,054 DEBUG [TezTR-518813_1_36_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,054 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.MergeManager: finalMerge with #inMemoryOutputs=0, size=0 and #onDiskOutputs=0, size=0 2018-01-05T11:29:07,054 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.MergeManager: . DiskSeg: 0, 0. MemSeg: 0, 0 2018-01-05T11:29:07,054 INFO [ShuffleAndMergeRunner {Reducer_2}] impl.TezMerger: Nothing to merge. Returning an empty iterator 2018-01-05T11:29:07,054 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.Shuffle: merge complete for input vertex : Reducer_2 2018-01-05T11:29:07,054 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.Shuffle: Reducer_2: Shuffle Runner thread complete 2018-01-05T11:29:07,054 DEBUG [TezTR-518813_1_36_2_1_0] log.PerfLogger: 2018-01-05T11:29:07,054 INFO [TezTR-518813_1_36_2_1_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 3 size: 2.66KB 2018-01-05T11:29:07,055 INFO [TezTR-518813_1_36_2_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:29:07,055 DEBUG [TezTR-518813_1_36_2_0_0] exec.GroupByOperator: Initialization Done 4 GBY 2018-01-05T11:29:07,055 DEBUG [TezTR-518813_1_36_2_0_0] exec.GroupByOperator: Operator 4 GBY initialized 2018-01-05T11:29:07,055 DEBUG [TezTR-518813_1_36_2_0_0] exec.GroupByOperator: Initializing children of 4 GBY 2018-01-05T11:29:07,055 DEBUG [TezTR-518813_1_36_2_0_0] exec.SelectOperator: Initializing child 5 SEL 2018-01-05T11:29:07,055 INFO [TezTR-518813_1_36_2_0_0] exec.SelectOperator: Initializing operator SEL[5] 2018-01-05T11:29:07,055 INFO [TezTR-518813_1_36_2_0_0] exec.SelectOperator: SELECT struct<_col0:string,_col1:struct,_col2:struct> 2018-01-05T11:29:07,055 DEBUG [TezTR-518813_1_36_2_0_0] exec.SelectOperator: Initialization Done 5 SEL 2018-01-05T11:29:07,055 DEBUG [TezTR-518813_1_36_2_0_0] exec.SelectOperator: Operator 5 SEL initialized 2018-01-05T11:29:07,055 DEBUG [TezTR-518813_1_36_2_0_0] exec.SelectOperator: Initializing children of 5 SEL 2018-01-05T11:29:07,055 DEBUG [TezTR-518813_1_36_2_0_0] exec.FileSinkOperator: Initializing child 6 FS 2018-01-05T11:29:07,055 INFO [TezTR-518813_1_36_2_0_0] exec.FileSinkOperator: Initializing operator FS[6] 2018-01-05T11:29:07,055 INFO [I/O Setup 1 Initialize: {out_Reducer 3}] output.MROutput: out_Reducer 3: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:29:07,055 DEBUG [TezTR-518813_1_36_2_1_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:07,055 DEBUG [TezTR-518813_1_36_2_1_0] log.PerfLogger: 2018-01-05T11:29:07,055 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:07,055 DEBUG [TezTR-518813_1_36_2_1_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:07,055 DEBUG [TezTR-518813_1_36_2_1_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:07,055 DEBUG [TezTR-518813_1_36_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,055 DEBUG [TezTR-518813_1_36_2_3_0] log.PerfLogger: 2018-01-05T11:29:07,055 INFO [TezTR-518813_1_36_2_3_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:07,055 INFO [TezTR-518813_1_36_2_3_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:07,055 INFO [TezTR-518813_1_36_2_3_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:07,055 INFO [TezTR-518813_1_36_2_0_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@e84c9d8]:[_col0, _col1, _col2]:[struct, struct, string]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@28e6a86c 2018-01-05T11:29:07,055 DEBUG [TezTR-518813_1_36_2_0_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:29:07,055 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:07,055 DEBUG [TezTR-518813_1_36_2_0_0] exec.FileSinkOperator: Initialization Done 6 FS 2018-01-05T11:29:07,055 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.ShuffleScheduler: Shutting down fetchers for input: Reducer_2, shutdown timetaken: 0 ms, hasFetcherExecutorStopped: true 2018-01-05T11:29:07,056 DEBUG [TezTR-518813_1_36_2_1_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:07,056 INFO [TezTR-518813_1_36_2_1_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:29:07,056 INFO [TezTR-518813_1_36_2_1_0] tez.ReduceRecordProcessor: Main work is Reducer 3 2018-01-05T11:29:07,056 INFO [TezTR-518813_1_36_2_1_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:07,056 DEBUG [TezTR-518813_1_36_2_1_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:07,056 DEBUG [TezTR-518813_1_36_2_0_0] exec.FileSinkOperator: Operator 6 FS initialized 2018-01-05T11:29:07,056 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.MergeManager: TotalInMemFetchStats: count=2, totalSize=1353, min=652, max=701, avg=1.0 2018-01-05T11:29:07,056 DEBUG [TezTR-518813_1_36_2_0_0] exec.FileSinkOperator: Initialization Done 6 FS done is reset. 2018-01-05T11:29:07,055 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:07,056 INFO [TezTR-518813_1_36_2_1_0] input.OrderedGroupedKVInput: Reducer 2: creating ValuesIterator with comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator, keyClass=org.apache.hadoop.hive.ql.io.HiveKey, valClass=org.apache.hadoop.io.BytesWritable 2018-01-05T11:29:07,056 DEBUG [TezTR-518813_1_36_2_0_0] exec.SelectOperator: Initialization Done 5 SEL done is reset. 2018-01-05T11:29:07,056 DEBUG [TezTR-518813_1_36_2_0_0] exec.GroupByOperator: Initialization Done 4 GBY done is reset. 2018-01-05T11:29:07,056 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.MergeManager: finalMerge with #inMemoryOutputs=2, size=1353 and #onDiskOutputs=0, size=0 2018-01-05T11:29:07,056 DEBUG [TezTR-518813_1_36_2_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,056 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:07,056 DEBUG [TezTR-518813_1_36_2_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,056 DEBUG [TezTR-518813_1_36_2_1_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:29:07,056 DEBUG [TezTR-518813_1_36_2_1_0] log.PerfLogger: 2018-01-05T11:29:07,056 INFO [TezTR-518813_1_36_2_3_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:07,056 INFO [TezTR-518813_1_36_2_0_0] tez.ReduceRecordProcessor: Id =4 Id =5 Id =6 <\Children> Id = 5 null<\Parent> <\FS> <\Children> Id = 4 null<\Parent> <\SEL> <\Children> <\Parent> <\GBY> 2018-01-05T11:29:07,056 INFO [TezTR-518813_1_36_2_3_0] resources.MemoryDistributor: InitialRequests=[out_Reducer 3:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [Reducer 2:INPUT:131404920:org.apache.tez.runtime.library.input.OrderedGroupedKVInput] 2018-01-05T11:29:07,056 DEBUG [TezTR-518813_1_36_2_0_0] log.PerfLogger: 2018-01-05T11:29:07,056 INFO [TezTR-518813_1_36_2_0_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 3 2018-01-05T11:29:07,056 INFO [TezTR-518813_1_36_2_3_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:07,056 INFO [TezTR-518813_1_36_2_1_0] exec.GroupByOperator: Initializing operator GBY[4] 2018-01-05T11:29:07,056 INFO [TezTR-518813_1_36_2_3_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:07,056 DEBUG [TezTR-518813_1_36_2_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,056 INFO [TezTR-518813_1_36_2_3_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 131404920, TotalRequestedScaled: 1.2129684923076923E8, TotalJVMHeap: 1026555904, TotalAvailable: 687792455, TotalRequested/TotalJVMHeap:0.13 2018-01-05T11:29:07,056 DEBUG [TezTR-518813_1_36_2_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,056 INFO [TezTR-518813_1_36_2_3_0] resources.MemoryDistributor: Allocations=[out_Reducer 3:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:131404920:131404920] 2018-01-05T11:29:07,056 INFO [TezTR-518813_1_36_2_3_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:07,056 INFO [TezTR-518813_1_36_2_3_0] runtime.LogicalIOProcessorRuntimeTask: Input: Reducer 2 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:07,056 DEBUG [TezTR-518813_1_36_2_0_0] log.PerfLogger: 2018-01-05T11:29:07,056 DEBUG [TezTR-518813_1_36_2_0_0] mr.ObjectCache: djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd_Reducer 3__REDUCE_PLAN__ no longer needed 2018-01-05T11:29:07,056 DEBUG [TezTR-518813_1_36_2_0_0] exec.GroupByOperator: close called for operator GBY[4] 2018-01-05T11:29:07,056 INFO [TezTR-518813_1_36_2_0_0] exec.GroupByOperator: Closing operator GBY[4] 2018-01-05T11:29:07,056 DEBUG [TezTR-518813_1_36_2_0_0] exec.GroupByOperator: Closing child = SEL[5] 2018-01-05T11:29:07,056 INFO [TezTR-518813_1_36_2_1_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:29:07,057 DEBUG [TezTR-518813_1_36_2_0_0] exec.SelectOperator: close called for operator SEL[5] 2018-01-05T11:29:07,057 DEBUG [TezTR-518813_1_36_2_0_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:07,057 INFO [TezTR-518813_1_36_2_0_0] exec.SelectOperator: Closing operator SEL[5] 2018-01-05T11:29:07,056 INFO [TezTR-518813_1_36_2_3_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:29:07,057 INFO [I/O Setup 0 Start: {Reducer 2}] orderedgrouped.Shuffle: Reducer_2: Shuffle assigned with 2 inputs, codec: None, ifileReadAhead: true 2018-01-05T11:29:07,057 DEBUG [TezTR-518813_1_36_2_1_0] exec.GroupByOperator: Initialization Done 4 GBY 2018-01-05T11:29:07,057 DEBUG [TezTR-518813_1_36_2_1_0] exec.GroupByOperator: Operator 4 GBY initialized 2018-01-05T11:29:07,057 DEBUG [TezTR-518813_1_36_2_1_0] exec.GroupByOperator: Initializing children of 4 GBY 2018-01-05T11:29:07,057 DEBUG [TezTR-518813_1_36_2_1_0] exec.SelectOperator: Initializing child 5 SEL 2018-01-05T11:29:07,057 INFO [TezTR-518813_1_36_2_1_0] exec.SelectOperator: Initializing operator SEL[5] 2018-01-05T11:29:07,057 INFO [TezTR-518813_1_36_2_3_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:07,057 DEBUG [TezTR-518813_1_36_2_0_0] exec.SelectOperator: Closing child = FS[6] 2018-01-05T11:29:07,057 DEBUG [TezTR-518813_1_36_2_0_0] exec.FileSinkOperator: close called for operator FS[6] 2018-01-05T11:29:07,057 DEBUG [TezTR-518813_1_36_2_0_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:07,057 INFO [I/O Setup 0 Start: {Reducer 2}] orderedgrouped.MergeManager: Reducer 2: MergerManager: memoryLimit=131404920, maxSingleShuffleLimit=32851230, mergeThreshold=86727248, ioSortFactor=10, postMergeMem=0, memToMemMergeOutputsThreshold=10 2018-01-05T11:29:07,057 INFO [TezTR-518813_1_36_2_0_0] exec.FileSinkOperator: Closing operator FS[6] 2018-01-05T11:29:07,057 INFO [TezTR-518813_1_36_2_1_0] exec.SelectOperator: SELECT struct<_col0:string,_col1:struct,_col2:struct> 2018-01-05T11:29:07,057 DEBUG [TezTR-518813_1_36_2_1_0] exec.SelectOperator: Initialization Done 5 SEL 2018-01-05T11:29:07,057 DEBUG [TezTR-518813_1_36_2_1_0] exec.SelectOperator: Operator 5 SEL initialized 2018-01-05T11:29:07,057 DEBUG [TezTR-518813_1_36_2_1_0] exec.SelectOperator: Initializing children of 5 SEL 2018-01-05T11:29:07,057 DEBUG [TezTR-518813_1_36_2_1_0] exec.FileSinkOperator: Initializing child 6 FS 2018-01-05T11:29:07,057 INFO [TezTR-518813_1_36_2_1_0] exec.FileSinkOperator: Initializing operator FS[6] 2018-01-05T11:29:07,057 DEBUG [TezTR-518813_1_36_2_1_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,057 INFO [TezTR-518813_1_36_2_0_0] exec.FileSinkOperator: FS[6]: records written - 0 2018-01-05T11:29:07,057 INFO [TezTR-518813_1_36_2_0_0] exec.FileSinkOperator: RECORDS_OUT_0:0, 2018-01-05T11:29:07,057 DEBUG [TezTR-518813_1_36_2_0_0] exec.FileSinkOperator: 6 Close done 2018-01-05T11:29:07,057 DEBUG [TezTR-518813_1_36_2_0_0] exec.SelectOperator: 5 Close done 2018-01-05T11:29:07,057 DEBUG [TezTR-518813_1_36_2_0_0] exec.GroupByOperator: 4 Close done 2018-01-05T11:29:07,057 INFO [TezTR-518813_1_36_2_1_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@1b5be189]:[_col0, _col1, _col2]:[struct, struct, string]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@4b835c10 2018-01-05T11:29:07,057 INFO [I/O Setup 0 Start: {Reducer 2}] orderedgrouped.ShuffleScheduler: ShuffleScheduler running for sourceVertex: Reducer 2 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=2, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=2, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true 2018-01-05T11:29:07,057 DEBUG [TezTR-518813_1_36_2_1_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:29:07,057 DEBUG [TezTR-518813_1_36_2_1_0] exec.FileSinkOperator: Initialization Done 6 FS 2018-01-05T11:29:07,057 DEBUG [TezTR-518813_1_36_2_1_0] exec.FileSinkOperator: Operator 6 FS initialized 2018-01-05T11:29:07,057 DEBUG [TezTR-518813_1_36_2_1_0] exec.FileSinkOperator: Initialization Done 6 FS done is reset. 2018-01-05T11:29:07,057 DEBUG [TezTR-518813_1_36_2_1_0] exec.SelectOperator: Initialization Done 5 SEL done is reset. 2018-01-05T11:29:07,057 DEBUG [TezTR-518813_1_36_2_1_0] exec.GroupByOperator: Initialization Done 4 GBY done is reset. 2018-01-05T11:29:07,057 INFO [TezTR-518813_1_36_2_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_36_02_000000_0 2018-01-05T11:29:07,057 INFO [TezTR-518813_1_36_2_1_0] tez.ReduceRecordProcessor: Id =4 Id =5 Id =6 <\Children> Id = 5 null<\Parent> <\FS> <\Children> Id = 4 null<\Parent> <\SEL> <\Children> <\Parent> <\GBY> 2018-01-05T11:29:07,057 INFO [TezTR-518813_1_36_2_0_0] orderedgrouped.Shuffle: Shutting down Shuffle for source: Reducer_2 2018-01-05T11:29:07,057 INFO [TezTR-518813_1_36_2_0_0] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Reducer 2: numDmeEventsSeen=2, numDmeEventsSeenWithNoData=2, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:29:07,057 INFO [TezTR-518813_1_36_2_0_0] output.MROutput: out_Reducer 3 closed 2018-01-05T11:29:07,057 DEBUG [TezTR-518813_1_36_2_1_0] log.PerfLogger: 2018-01-05T11:29:07,058 INFO [TezTR-518813_1_36_2_1_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 3 2018-01-05T11:29:07,058 DEBUG [TezTR-518813_1_36_2_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:07,058 INFO [TezTR-518813_1_36_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:07,058 INFO [I/O Setup 0 Start: {Reducer 2}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: Reducer 2 2018-01-05T11:29:07,058 DEBUG [TezTR-518813_1_36_2_1_0] log.PerfLogger: 2018-01-05T11:29:07,058 DEBUG [TezTR-518813_1_36_2_1_0] mr.ObjectCache: djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd_Reducer 3__REDUCE_PLAN__ no longer needed 2018-01-05T11:29:07,058 DEBUG [TezTR-518813_1_36_2_1_0] exec.GroupByOperator: close called for operator GBY[4] 2018-01-05T11:29:07,058 INFO [TezTR-518813_1_36_2_3_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:07,058 INFO [TezTR-518813_1_36_2_1_0] exec.GroupByOperator: Closing operator GBY[4] 2018-01-05T11:29:07,058 DEBUG [TezTR-518813_1_36_2_1_0] exec.GroupByOperator: Closing child = SEL[5] 2018-01-05T11:29:07,058 DEBUG [TezTR-518813_1_36_2_1_0] exec.SelectOperator: close called for operator SEL[5] 2018-01-05T11:29:07,058 DEBUG [TezTR-518813_1_36_2_1_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:07,058 INFO [TezTR-518813_1_36_2_1_0] exec.SelectOperator: Closing operator SEL[5] 2018-01-05T11:29:07,058 INFO [TezTR-518813_1_36_2_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_36_02_000000_0, askedToStop=false 2018-01-05T11:29:07,058 INFO [TezTR-518813_1_36_2_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_36_02_000000_0, stopRequested=false 2018-01-05T11:29:07,058 DEBUG [TezTR-518813_1_36_2_1_0] exec.SelectOperator: Closing child = FS[6] 2018-01-05T11:29:07,058 DEBUG [TezTR-518813_1_36_2_1_0] exec.FileSinkOperator: close called for operator FS[6] 2018-01-05T11:29:07,058 DEBUG [TezTR-518813_1_36_2_1_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:07,058 INFO [TezTR-518813_1_36_2_1_0] exec.FileSinkOperator: Closing operator FS[6] 2018-01-05T11:29:07,058 INFO [TezTR-518813_1_36_2_1_0] exec.FileSinkOperator: FS[6]: records written - 0 2018-01-05T11:29:07,058 INFO [TezTR-518813_1_36_2_1_0] exec.FileSinkOperator: RECORDS_OUT_0:0, 2018-01-05T11:29:07,058 DEBUG [TezTR-518813_1_36_2_1_0] exec.FileSinkOperator: 6 Close done 2018-01-05T11:29:07,059 DEBUG [TezTR-518813_1_36_2_1_0] exec.SelectOperator: 5 Close done 2018-01-05T11:29:07,059 DEBUG [TezTR-518813_1_36_2_1_0] exec.GroupByOperator: 4 Close done 2018-01-05T11:29:07,059 INFO [TezTR-518813_1_36_2_3_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_36_02_000003_0 2018-01-05T11:29:07,059 DEBUG [TezTR-518813_1_36_2_3_0] log.PerfLogger: 2018-01-05T11:29:07,059 DEBUG [TezTR-518813_1_36_2_3_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_36_02_000003_0_10259 2018-01-05T11:29:07,059 DEBUG [TezTR-518813_1_36_2_3_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:07,059 DEBUG [TezTR-518813_1_36_2_3_0] mr.ObjectCache: Creating djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd_Reducer 3__REDUCE_PLAN__ 2018-01-05T11:29:07,059 INFO [TezTR-518813_1_36_2_3_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/djaiswal/_tez_scratch_dir/e826609d-5b54-47c6-a628-3fb7d0722b70/reduce.xml 2018-01-05T11:29:07,059 INFO [TezTR-518813_1_36_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_36_02_000000_0: Counters: 29 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=0, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=0, NUM_SKIPPED_INPUTS=2, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=0, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, SHUFFLE_BYTES=0, SHUFFLE_BYTES_DECOMPRESSED=0, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][HIVE RECORDS_OUT_0=0][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0]] 2018-01-05T11:29:07,059 INFO [TezTR-518813_1_36_2_1_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_36_02_000001_0 2018-01-05T11:29:07,059 INFO [TezTaskEventRouter{attempt_1515180518813_0001_36_02_000003_0}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 2. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:07,059 DEBUG [TezTR-518813_1_36_2_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_36_2_0_0 thread id: 2441 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:07,059 DEBUG [TezTR-518813_1_36_2_3_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/djaiswal/_tez_scratch_dir/e826609d-5b54-47c6-a628-3fb7d0722b70/reduce.xml 2018-01-05T11:29:07,059 DEBUG [TezTR-518813_1_36_2_3_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/djaiswal/_tez_scratch_dir/e826609d-5b54-47c6-a628-3fb7d0722b70/reduce.xml 2018-01-05T11:29:07,059 DEBUG [TezTR-518813_1_36_2_3_0] log.PerfLogger: 2018-01-05T11:29:07,059 INFO [TezTR-518813_1_36_2_3_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:29:07,059 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_36_02_000000_0, isTaskDone=true 2018-01-05T11:29:07,059 INFO [TezTaskEventRouter{attempt_1515180518813_0001_36_02_000003_0}] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Reducer 2: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=1, numObsoletionEventsSeen=0 2018-01-05T11:29:07,059 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000084, requestId=2, startIndex=2, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_36_02_000000_0, eventCount=2 } 2018-01-05T11:29:07,060 INFO [TezTaskEventRouter{attempt_1515180518813_0001_36_02_000003_0}] orderedgrouped.ShuffleScheduler: All inputs fetched for input vertex : Reducer 2 2018-01-05T11:29:07,060 INFO [TezTaskEventRouter{attempt_1515180518813_0001_36_02_000003_0}] orderedgrouped.ShuffleScheduler: copy(2 (spillsFetched=2) of 2. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:07,059 INFO [TezTR-518813_1_36_2_1_0] orderedgrouped.Shuffle: Shutting down Shuffle for source: Reducer_2 2018-01-05T11:29:07,060 INFO [TezTR-518813_1_36_2_1_0] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Reducer 2: numDmeEventsSeen=2, numDmeEventsSeenWithNoData=2, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:29:07,060 INFO [TezTR-518813_1_36_2_1_0] output.MROutput: out_Reducer 3 closed 2018-01-05T11:29:07,060 DEBUG [TezTR-518813_1_36_2_1_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:07,060 INFO [TezTR-518813_1_36_2_1_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:07,060 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.ShuffleScheduler: Shutting down FetchScheduler for input: Reducer_2, wasInterrupted=false 2018-01-05T11:29:07,059 DEBUG [TezTR-518813_1_36_2_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,060 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.ShuffleScheduler: copy(2 (spillsFetched=2) of 2. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:07,061 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.ShuffleScheduler: Shutting down fetchers for input: Reducer_2, shutdown timetaken: 0 ms, hasFetcherExecutorStopped: true 2018-01-05T11:29:07,061 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.MergeManager: finalMerge with #inMemoryOutputs=0, size=0 and #onDiskOutputs=0, size=0 2018-01-05T11:29:07,061 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.MergeManager: . DiskSeg: 0, 0. MemSeg: 0, 0 2018-01-05T11:29:07,061 INFO [TezTR-518813_1_36_2_1_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_36_02_000001_0, askedToStop=false 2018-01-05T11:29:07,061 INFO [ShuffleAndMergeRunner {Reducer_2}] impl.TezMerger: Nothing to merge. Returning an empty iterator 2018-01-05T11:29:07,061 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.Shuffle: merge complete for input vertex : Reducer_2 2018-01-05T11:29:07,061 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.Shuffle: Reducer_2: Shuffle Runner thread complete 2018-01-05T11:29:07,061 INFO [TezTR-518813_1_36_2_1_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_36_02_000001_0, stopRequested=false 2018-01-05T11:29:07,061 INFO [TezTR-518813_1_36_2_1_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_36_02_000001_0: Counters: 29 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=0, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=0, NUM_SKIPPED_INPUTS=2, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=0, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, SHUFFLE_BYTES=0, SHUFFLE_BYTES_DECOMPRESSED=0, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][HIVE RECORDS_OUT_0=0][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0]] 2018-01-05T11:29:07,061 DEBUG [TezTR-518813_1_36_2_1_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_36_2_1_0 thread id: 2446 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:07,061 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_36_02_000001_0, isTaskDone=true 2018-01-05T11:29:07,061 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000085, requestId=2, startIndex=2, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_36_02_000001_0, eventCount=2 } 2018-01-05T11:29:07,062 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 3, taskAttemptId=attempt_1515180518813_0001_36_02_000000_0, creationTime=1515180547024, allocationTime=1515180547025, startTime=1515180547027, finishTime=1515180547062, timeTaken=35, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:07,062 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_36_02_000000_0 2018-01-05T11:29:07,062 DEBUG [TezTR-518813_1_36_2_3_0] log.PerfLogger: 2018-01-05T11:29:07,062 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:TASK_FINISHED]: vertexName=Reducer 3, taskId=task_1515180518813_0001_36_02_000000, startTime=1515180547027, finishTime=1515180547062, timeTaken=35, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_36_02_000000_0, diagnostics=, counters=Counters: 34, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=0, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=0, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=0, NUM_SKIPPED_INPUTS=2, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=0, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, SHUFFLE_BYTES=0, SHUFFLE_BYTES_DECOMPRESSED=0, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, HIVE, RECORDS_OUT_0=0, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0 2018-01-05T11:29:07,062 INFO [TezTR-518813_1_36_2_3_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 3 size: 2.66KB 2018-01-05T11:29:07,062 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_36_02_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:07,062 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_36_02_000000_0; false 2018-01-05T11:29:07,063 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_36_02_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:07,063 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_36_02 [Reducer 3], tasks=4, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:07,063 DEBUG [Task-Executor-0] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=2, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:07,063 INFO [Task-Executor-0] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_36_02_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:07,063 INFO [Task-Executor-0] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_36_02_000000_0 2018-01-05T11:29:07,063 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:07,063 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:07,063 DEBUG [TezTR-518813_1_36_2_3_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:07,063 DEBUG [TezTR-518813_1_36_2_3_0] log.PerfLogger: 2018-01-05T11:29:07,063 DEBUG [TezTR-518813_1_36_2_3_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:07,063 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000084, stoppedTime=1515180547063, exitStatus=0 2018-01-05T11:29:07,063 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000084 exited with diagnostics set to Container completed. 2018-01-05T11:29:07,063 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 3, taskAttemptId=attempt_1515180518813_0001_36_02_000001_0, creationTime=1515180547024, allocationTime=1515180547026, startTime=1515180547028, finishTime=1515180547063, timeTaken=35, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:07,063 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000084 2018-01-05T11:29:07,063 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_36_02_000001_0 2018-01-05T11:29:07,063 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:TASK_FINISHED]: vertexName=Reducer 3, taskId=task_1515180518813_0001_36_02_000001, startTime=1515180547028, finishTime=1515180547063, timeTaken=35, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_36_02_000001_0, diagnostics=, counters=Counters: 34, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=0, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=0, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=0, NUM_SKIPPED_INPUTS=2, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=0, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, SHUFFLE_BYTES=0, SHUFFLE_BYTES_DECOMPRESSED=0, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, HIVE, RECORDS_OUT_0=0, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0 2018-01-05T11:29:07,063 DEBUG [TezTR-518813_1_36_2_3_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:07,063 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_36_02 [Reducer 3], tasks=4, failed=0, killed=0, success=2, completed=2, commits=0, err=null 2018-01-05T11:29:07,064 DEBUG [TezTR-518813_1_36_2_3_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:07,064 INFO [TezTR-518813_1_36_2_3_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:29:07,064 INFO [TezTR-518813_1_36_2_3_0] tez.ReduceRecordProcessor: Main work is Reducer 3 2018-01-05T11:29:07,064 INFO [TezTR-518813_1_36_2_3_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:07,064 DEBUG [TezTR-518813_1_36_2_3_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:07,064 INFO [TezTR-518813_1_36_2_3_0] input.OrderedGroupedKVInput: Reducer 2: creating ValuesIterator with comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator, keyClass=org.apache.hadoop.hive.ql.io.HiveKey, valClass=org.apache.hadoop.io.BytesWritable 2018-01-05T11:29:07,064 DEBUG [TezTR-518813_1_36_2_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,064 DEBUG [TezTR-518813_1_36_2_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,064 DEBUG [TezTR-518813_1_36_2_3_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:29:07,064 DEBUG [TezTR-518813_1_36_2_3_0] log.PerfLogger: 2018-01-05T11:29:07,064 INFO [TezTR-518813_1_36_2_3_0] exec.GroupByOperator: Initializing operator GBY[4] 2018-01-05T11:29:07,064 DEBUG [TezTR-518813_1_36_2_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,064 DEBUG [TezTR-518813_1_36_2_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,064 INFO [TezTR-518813_1_36_2_3_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:29:07,064 DEBUG [TezTR-518813_1_36_2_3_0] exec.GroupByOperator: Initialization Done 4 GBY 2018-01-05T11:29:07,064 DEBUG [TezTR-518813_1_36_2_3_0] exec.GroupByOperator: Operator 4 GBY initialized 2018-01-05T11:29:07,064 DEBUG [TezTR-518813_1_36_2_3_0] exec.GroupByOperator: Initializing children of 4 GBY 2018-01-05T11:29:07,064 DEBUG [TezTR-518813_1_36_2_3_0] exec.SelectOperator: Initializing child 5 SEL 2018-01-05T11:29:07,064 INFO [TezTR-518813_1_36_2_3_0] exec.SelectOperator: Initializing operator SEL[5] 2018-01-05T11:29:07,064 INFO [TezTR-518813_1_36_2_3_0] exec.SelectOperator: SELECT struct<_col0:string,_col1:struct,_col2:struct> 2018-01-05T11:29:07,064 DEBUG [TezTR-518813_1_36_2_3_0] exec.SelectOperator: Initialization Done 5 SEL 2018-01-05T11:29:07,064 DEBUG [TezTR-518813_1_36_2_3_0] exec.SelectOperator: Operator 5 SEL initialized 2018-01-05T11:29:07,064 DEBUG [TezTR-518813_1_36_2_3_0] exec.SelectOperator: Initializing children of 5 SEL 2018-01-05T11:29:07,064 DEBUG [TezTR-518813_1_36_2_3_0] exec.FileSinkOperator: Initializing child 6 FS 2018-01-05T11:29:07,064 INFO [TezTR-518813_1_36_2_3_0] exec.FileSinkOperator: Initializing operator FS[6] 2018-01-05T11:29:07,065 DEBUG [TezTR-518813_1_36_2_3_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,065 INFO [TezTR-518813_1_36_2_3_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@5e6e627e]:[_col0, _col1, _col2]:[struct, struct, string]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@3ceccf9 2018-01-05T11:29:07,065 DEBUG [TezTR-518813_1_36_2_3_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:29:07,065 DEBUG [TezTR-518813_1_36_2_3_0] exec.FileSinkOperator: Initialization Done 6 FS 2018-01-05T11:29:07,065 DEBUG [TezTR-518813_1_36_2_3_0] exec.FileSinkOperator: Operator 6 FS initialized 2018-01-05T11:29:07,065 DEBUG [TezTR-518813_1_36_2_3_0] exec.FileSinkOperator: Initialization Done 6 FS done is reset. 2018-01-05T11:29:07,065 DEBUG [TezTR-518813_1_36_2_3_0] exec.SelectOperator: Initialization Done 5 SEL done is reset. 2018-01-05T11:29:07,065 DEBUG [TezTR-518813_1_36_2_3_0] exec.GroupByOperator: Initialization Done 4 GBY done is reset. 2018-01-05T11:29:07,065 INFO [TezTR-518813_1_36_2_3_0] tez.ReduceRecordProcessor: Id =4 Id =5 Id =6 <\Children> Id = 5 null<\Parent> <\FS> <\Children> Id = 4 null<\Parent> <\SEL> <\Children> <\Parent> <\GBY> 2018-01-05T11:29:07,065 DEBUG [TezTR-518813_1_36_2_3_0] log.PerfLogger: 2018-01-05T11:29:07,065 INFO [TezTR-518813_1_36_2_3_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 3 2018-01-05T11:29:07,065 DEBUG [TezTR-518813_1_36_2_3_0] log.PerfLogger: 2018-01-05T11:29:07,065 DEBUG [TezTR-518813_1_36_2_3_0] mr.ObjectCache: djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd_Reducer 3__REDUCE_PLAN__ no longer needed 2018-01-05T11:29:07,065 DEBUG [TezTR-518813_1_36_2_3_0] exec.GroupByOperator: close called for operator GBY[4] 2018-01-05T11:29:07,065 INFO [TezTR-518813_1_36_2_3_0] exec.GroupByOperator: Closing operator GBY[4] 2018-01-05T11:29:07,065 DEBUG [TezTR-518813_1_36_2_3_0] exec.GroupByOperator: Closing child = SEL[5] 2018-01-05T11:29:07,065 DEBUG [TezTR-518813_1_36_2_3_0] exec.SelectOperator: close called for operator SEL[5] 2018-01-05T11:29:07,065 DEBUG [TezTR-518813_1_36_2_3_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:07,065 INFO [TezTR-518813_1_36_2_3_0] exec.SelectOperator: Closing operator SEL[5] 2018-01-05T11:29:07,065 DEBUG [TezTR-518813_1_36_2_3_0] exec.SelectOperator: Closing child = FS[6] 2018-01-05T11:29:07,065 DEBUG [TezTR-518813_1_36_2_3_0] exec.FileSinkOperator: close called for operator FS[6] 2018-01-05T11:29:07,065 DEBUG [TezTR-518813_1_36_2_3_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:07,065 INFO [TezTR-518813_1_36_2_3_0] exec.FileSinkOperator: Closing operator FS[6] 2018-01-05T11:29:07,065 INFO [TezTR-518813_1_36_2_3_0] exec.FileSinkOperator: FS[6]: records written - 0 2018-01-05T11:29:07,065 INFO [TezTR-518813_1_36_2_3_0] exec.FileSinkOperator: RECORDS_OUT_0:0, 2018-01-05T11:29:07,065 DEBUG [TezTR-518813_1_36_2_3_0] exec.FileSinkOperator: 6 Close done 2018-01-05T11:29:07,065 DEBUG [TezTR-518813_1_36_2_3_0] exec.SelectOperator: 5 Close done 2018-01-05T11:29:07,065 DEBUG [TezTR-518813_1_36_2_3_0] exec.GroupByOperator: 4 Close done 2018-01-05T11:29:07,065 INFO [TezTR-518813_1_36_2_3_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_36_02_000003_0 2018-01-05T11:29:07,065 INFO [TezTR-518813_1_36_2_3_0] orderedgrouped.Shuffle: Shutting down Shuffle for source: Reducer_2 2018-01-05T11:29:07,065 INFO [TezTR-518813_1_36_2_3_0] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Reducer 2: numDmeEventsSeen=2, numDmeEventsSeenWithNoData=2, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:29:07,065 INFO [TezTR-518813_1_36_2_3_0] output.MROutput: out_Reducer 3 closed 2018-01-05T11:29:07,065 DEBUG [TezTR-518813_1_36_2_3_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:07,065 INFO [TezTR-518813_1_36_2_3_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:07,065 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_36_02_000001_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:07,065 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_36_02_000001_0; false 2018-01-05T11:29:07,065 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_36_02_000001_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:07,063 INFO [Task-Executor-0] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000084=33 2018-01-05T11:29:07,066 DEBUG [Task-Executor-0] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_36_02_000000_0: true 2018-01-05T11:29:07,066 DEBUG [Task-Executor-3] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=2, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:07,066 INFO [Task-Executor-3] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_36_02_000001_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:07,066 INFO [Task-Executor-3] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_36_02_000001_0 2018-01-05T11:29:07,066 INFO [Task-Executor-3] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000085=31 2018-01-05T11:29:07,066 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_36_02_000001_0: true 2018-01-05T11:29:07,066 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:07,066 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:07,066 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_36_02_000000_0 2018-01-05T11:29:07,066 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000084, QueryId=djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd-36, Vertex=Reducer 3, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:07,066 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_36_02_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=1, preemptionQueueSize=3 2018-01-05T11:29:07,066 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_36_02_000000_0 2018-01-05T11:29:07,066 INFO [TezTR-518813_1_36_2_3_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_36_02_000003_0, askedToStop=false 2018-01-05T11:29:07,066 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000085, stoppedTime=1515180547066, exitStatus=0 2018-01-05T11:29:07,066 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000085 exited with diagnostics set to Container completed. 2018-01-05T11:29:07,066 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000085 2018-01-05T11:29:07,066 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000084, QueryId=djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd-36, DagId=36, VertexName=Reducer 3, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-0, Succeeded=true, StartTime=1515180547029, EndTime=1515180547066 2018-01-05T11:29:07,066 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_36_02_000001_0 2018-01-05T11:29:07,066 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000085, QueryId=djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd-36, Vertex=Reducer 3, FragmentNum=1, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:07,066 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_36_02_000001_0 complete. WaitQueueSize=0, numSlotsAvailable=2, preemptionQueueSize=2 2018-01-05T11:29:07,066 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_36_02_000001_0 2018-01-05T11:29:07,066 INFO [TezTR-518813_1_36_2_3_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_36_02_000003_0, stopRequested=false 2018-01-05T11:29:07,066 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000085, QueryId=djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd-36, DagId=36, VertexName=Reducer 3, TaskId=1, TaskAttemptId=0, ThreadName=Task-Executor-3, Succeeded=true, StartTime=1515180547034, EndTime=1515180547066 2018-01-05T11:29:07,066 INFO [TezTR-518813_1_36_2_3_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_36_02_000003_0: Counters: 29 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=0, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=0, NUM_SKIPPED_INPUTS=2, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=0, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, SHUFFLE_BYTES=0, SHUFFLE_BYTES_DECOMPRESSED=0, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=4, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=3][HIVE RECORDS_OUT_0=0][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0]] 2018-01-05T11:29:07,066 DEBUG [TezTR-518813_1_36_2_3_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_36_2_3_0 thread id: 2464 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:07,067 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_36_02_000003_0, isTaskDone=true 2018-01-05T11:29:07,067 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000086, requestId=2, startIndex=2, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_36_02_000003_0, eventCount=2 } 2018-01-05T11:29:07,067 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 3, taskAttemptId=attempt_1515180518813_0001_36_02_000003_0, creationTime=1515180547033, allocationTime=1515180547034, startTime=1515180547035, finishTime=1515180547067, timeTaken=32, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:07,068 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_36_02_000003_0 2018-01-05T11:29:07,068 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=2, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:07,068 INFO [Task-Executor-1] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_36_02_000003_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:07,068 INFO [Task-Executor-1] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_36_02_000003_0 2018-01-05T11:29:07,068 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_36_02_000003_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:07,068 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_36_02_000003_0; false 2018-01-05T11:29:07,068 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_36_02_000003_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:07,068 INFO [Task-Executor-1] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000086=27 2018-01-05T11:29:07,068 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_36_02_000003_0: true 2018-01-05T11:29:07,068 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:07,068 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:07,068 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:TASK_FINISHED]: vertexName=Reducer 3, taskId=task_1515180518813_0001_36_02_000003, startTime=1515180547035, finishTime=1515180547068, timeTaken=33, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_36_02_000003_0, diagnostics=, counters=Counters: 34, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=0, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=0, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=0, NUM_SKIPPED_INPUTS=2, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=0, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, SHUFFLE_BYTES=0, SHUFFLE_BYTES_DECOMPRESSED=0, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=4, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=3, HIVE, RECORDS_OUT_0=0, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0 2018-01-05T11:29:07,068 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_36_02_000003_0 2018-01-05T11:29:07,068 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000086, QueryId=djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd-36, Vertex=Reducer 3, FragmentNum=3, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:07,068 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_36_02_000003_0 complete. WaitQueueSize=0, numSlotsAvailable=3, preemptionQueueSize=1 2018-01-05T11:29:07,068 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_36_02_000003_0 2018-01-05T11:29:07,068 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_36_02 [Reducer 3], tasks=4, failed=0, killed=0, success=3, completed=3, commits=0, err=null 2018-01-05T11:29:07,068 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000086, QueryId=djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd-36, DagId=36, VertexName=Reducer 3, TaskId=3, TaskAttemptId=0, ThreadName=Task-Executor-1, Succeeded=true, StartTime=1515180547040, EndTime=1515180547068 2018-01-05T11:29:07,068 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000086, stoppedTime=1515180547068, exitStatus=0 2018-01-05T11:29:07,068 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000086 exited with diagnostics set to Container completed. 2018-01-05T11:29:07,068 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000086 2018-01-05T11:29:07,073 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.Shuffle: merge complete for input vertex : Reducer_2 2018-01-05T11:29:07,073 INFO [TezTR-518813_1_36_2_2_0] tez.ReduceRecordProcessor: Main work is Reducer 3 2018-01-05T11:29:07,073 INFO [ShuffleAndMergeRunner {Reducer_2}] orderedgrouped.Shuffle: Reducer_2: Shuffle Runner thread complete 2018-01-05T11:29:07,073 INFO [TezTR-518813_1_36_2_2_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:07,073 DEBUG [TezTR-518813_1_36_2_2_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:07,073 INFO [TezTR-518813_1_36_2_2_0] input.OrderedGroupedKVInput: Reducer 2: creating ValuesIterator with comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator, keyClass=org.apache.hadoop.hive.ql.io.HiveKey, valClass=org.apache.hadoop.io.BytesWritable 2018-01-05T11:29:07,073 DEBUG [TezTR-518813_1_36_2_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,073 DEBUG [TezTR-518813_1_36_2_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,073 DEBUG [TezTR-518813_1_36_2_2_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[struct, struct] 2018-01-05T11:29:07,074 DEBUG [TezTR-518813_1_36_2_2_0] log.PerfLogger: 2018-01-05T11:29:07,074 INFO [TezTR-518813_1_36_2_2_0] exec.GroupByOperator: Initializing operator GBY[4] 2018-01-05T11:29:07,074 DEBUG [TezTR-518813_1_36_2_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,074 DEBUG [TezTR-518813_1_36_2_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,074 INFO [TezTR-518813_1_36_2_2_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:29:07,074 DEBUG [TezTR-518813_1_36_2_2_0] exec.GroupByOperator: Initialization Done 4 GBY 2018-01-05T11:29:07,074 DEBUG [TezTR-518813_1_36_2_2_0] exec.GroupByOperator: Operator 4 GBY initialized 2018-01-05T11:29:07,074 DEBUG [TezTR-518813_1_36_2_2_0] exec.GroupByOperator: Initializing children of 4 GBY 2018-01-05T11:29:07,074 DEBUG [TezTR-518813_1_36_2_2_0] exec.SelectOperator: Initializing child 5 SEL 2018-01-05T11:29:07,074 INFO [TezTR-518813_1_36_2_2_0] exec.SelectOperator: Initializing operator SEL[5] 2018-01-05T11:29:07,074 INFO [TezTR-518813_1_36_2_2_0] exec.SelectOperator: SELECT struct<_col0:string,_col1:struct,_col2:struct> 2018-01-05T11:29:07,074 DEBUG [TezTR-518813_1_36_2_2_0] exec.SelectOperator: Initialization Done 5 SEL 2018-01-05T11:29:07,074 DEBUG [TezTR-518813_1_36_2_2_0] exec.SelectOperator: Operator 5 SEL initialized 2018-01-05T11:29:07,074 DEBUG [TezTR-518813_1_36_2_2_0] exec.SelectOperator: Initializing children of 5 SEL 2018-01-05T11:29:07,074 DEBUG [TezTR-518813_1_36_2_2_0] exec.FileSinkOperator: Initializing child 6 FS 2018-01-05T11:29:07,074 INFO [TezTR-518813_1_36_2_2_0] exec.FileSinkOperator: Initializing operator FS[6] 2018-01-05T11:29:07,074 DEBUG [TezTR-518813_1_36_2_2_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,074 INFO [TezTR-518813_1_36_2_2_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@12476273]:[_col0, _col1, _col2]:[struct, struct, string]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@4995faae 2018-01-05T11:29:07,074 DEBUG [TezTR-518813_1_36_2_2_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:29:07,074 DEBUG [TezTR-518813_1_36_2_2_0] exec.FileSinkOperator: Initialization Done 6 FS 2018-01-05T11:29:07,074 DEBUG [TezTR-518813_1_36_2_2_0] exec.FileSinkOperator: Operator 6 FS initialized 2018-01-05T11:29:07,074 DEBUG [TezTR-518813_1_36_2_2_0] exec.FileSinkOperator: Initialization Done 6 FS done is reset. 2018-01-05T11:29:07,074 DEBUG [TezTR-518813_1_36_2_2_0] exec.SelectOperator: Initialization Done 5 SEL done is reset. 2018-01-05T11:29:07,074 DEBUG [TezTR-518813_1_36_2_2_0] exec.GroupByOperator: Initialization Done 4 GBY done is reset. 2018-01-05T11:29:07,074 INFO [TezTR-518813_1_36_2_2_0] tez.ReduceRecordProcessor: Id =4 Id =5 Id =6 <\Children> Id = 5 null<\Parent> <\FS> <\Children> Id = 4 null<\Parent> <\SEL> <\Children> <\Parent> <\GBY> 2018-01-05T11:29:07,074 DEBUG [TezTR-518813_1_36_2_2_0] log.PerfLogger: 2018-01-05T11:29:07,074 INFO [TezTR-518813_1_36_2_2_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 3 2018-01-05T11:29:07,075 DEBUG [TezTR-518813_1_36_2_2_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:07,075 DEBUG [TezTR-518813_1_36_2_2_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:07,075 DEBUG [TezTR-518813_1_36_2_2_0] exec.SelectOperator: Starting group 2018-01-05T11:29:07,075 DEBUG [TezTR-518813_1_36_2_2_0] exec.SelectOperator: Starting group for children: 2018-01-05T11:29:07,075 DEBUG [TezTR-518813_1_36_2_2_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,075 DEBUG [TezTR-518813_1_36_2_2_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,075 DEBUG [TezTR-518813_1_36_2_2_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,075 DEBUG [TezTR-518813_1_36_2_2_0] exec.SelectOperator: Start group Done 2018-01-05T11:29:07,075 DEBUG [TezTR-518813_1_36_2_2_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:07,075 DEBUG [TezTR-518813_1_36_2_2_0] log.PerfLogger: 2018-01-05T11:29:07,075 DEBUG [TezTR-518813_1_36_2_2_0] mr.ObjectCache: djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd_Reducer 3__REDUCE_PLAN__ no longer needed 2018-01-05T11:29:07,075 DEBUG [TezTR-518813_1_36_2_2_0] exec.GroupByOperator: close called for operator GBY[4] 2018-01-05T11:29:07,075 INFO [TezTR-518813_1_36_2_2_0] exec.GroupByOperator: Closing operator GBY[4] 2018-01-05T11:29:07,076 INFO [TezTR-518813_1_36_2_2_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_475_8915019180860607944-1/-mr-10000/.hive-staging_hive_2018-01-05_11-29-06_475_8915019180860607944-1/_tmp.-ext-10001/000002_0 2018-01-05T11:29:07,076 INFO [TezTR-518813_1_36_2_2_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_475_8915019180860607944-1/-mr-10000/.hive-staging_hive_2018-01-05_11-29-06_475_8915019180860607944-1/_task_tmp.-ext-10001/_tmp.000002_0 2018-01-05T11:29:07,076 INFO [TezTR-518813_1_36_2_2_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_475_8915019180860607944-1/-mr-10000/.hive-staging_hive_2018-01-05_11-29-06_475_8915019180860607944-1/_tmp.-ext-10001/000002_0 2018-01-05T11:29:07,114 INFO [TezTR-518813_1_36_2_2_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:29:07,114 DEBUG [TezTR-518813_1_36_2_2_0] exec.GroupByOperator: Closing child = SEL[5] 2018-01-05T11:29:07,114 DEBUG [TezTR-518813_1_36_2_2_0] exec.SelectOperator: close called for operator SEL[5] 2018-01-05T11:29:07,114 DEBUG [TezTR-518813_1_36_2_2_0] exec.SelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:07,114 INFO [TezTR-518813_1_36_2_2_0] exec.SelectOperator: Closing operator SEL[5] 2018-01-05T11:29:07,114 DEBUG [TezTR-518813_1_36_2_2_0] exec.SelectOperator: Closing child = FS[6] 2018-01-05T11:29:07,114 DEBUG [TezTR-518813_1_36_2_2_0] exec.FileSinkOperator: close called for operator FS[6] 2018-01-05T11:29:07,114 DEBUG [TezTR-518813_1_36_2_2_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:07,114 INFO [TezTR-518813_1_36_2_2_0] exec.FileSinkOperator: Closing operator FS[6] 2018-01-05T11:29:07,114 INFO [TezTR-518813_1_36_2_2_0] exec.FileSinkOperator: FS[6]: records written - 1 2018-01-05T11:29:07,114 INFO [TezTR-518813_1_36_2_2_0] exec.FileSinkOperator: RECORDS_OUT_0:1, 2018-01-05T11:29:07,114 DEBUG [TezTR-518813_1_36_2_2_0] exec.FileSinkOperator: 6 Close done 2018-01-05T11:29:07,114 DEBUG [TezTR-518813_1_36_2_2_0] exec.SelectOperator: 5 Close done 2018-01-05T11:29:07,114 DEBUG [TezTR-518813_1_36_2_2_0] exec.GroupByOperator: 4 Close done 2018-01-05T11:29:07,114 INFO [TezTR-518813_1_36_2_2_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_36_02_000002_0 2018-01-05T11:29:07,114 INFO [TezTR-518813_1_36_2_2_0] orderedgrouped.Shuffle: Shutting down Shuffle for source: Reducer_2 2018-01-05T11:29:07,115 INFO [TezTR-518813_1_36_2_2_0] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Reducer 2: numDmeEventsSeen=2, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:29:07,115 INFO [TezTR-518813_1_36_2_2_0] output.MROutput: out_Reducer 3 closed 2018-01-05T11:29:07,115 DEBUG [TezTR-518813_1_36_2_2_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:07,115 INFO [TezTR-518813_1_36_2_2_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:07,115 INFO [TezTR-518813_1_36_2_2_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_36_02_000002_0, askedToStop=false 2018-01-05T11:29:07,115 INFO [TezTR-518813_1_36_2_2_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_36_02_000002_0, stopRequested=false 2018-01-05T11:29:07,115 INFO [TezTR-518813_1_36_2_2_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_36_02_000002_0: Counters: 30 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=1339, ADDITIONAL_SPILLS_BYTES_READ=1339, SHUFFLE_BYTES=1361, SHUFFLE_BYTES_DECOMPRESSED=1353, SHUFFLE_BYTES_TO_MEM=1361, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=16, MERGE_PHASE_TIME=33, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=5][HIVE CREATED_FILES=1, RECORDS_OUT_0=1][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0]] 2018-01-05T11:29:07,115 DEBUG [TezTR-518813_1_36_2_2_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_36_2_2_0 thread id: 2436 scheme: file bytesRead: 0 bytesWritten: 1784 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:07,115 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_36_02_000002_0, isTaskDone=true 2018-01-05T11:29:07,115 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000083, requestId=2, startIndex=2, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_36_02_000002_0, eventCount=2 } 2018-01-05T11:29:07,116 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 3, taskAttemptId=attempt_1515180518813_0001_36_02_000002_0, creationTime=1515180547023, allocationTime=1515180547025, startTime=1515180547026, finishTime=1515180547116, timeTaken=90, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:07,116 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_36_02_000002_0 2018-01-05T11:29:07,116 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=2, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:07,116 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_36_02_000002_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:07,116 INFO [Task-Executor-2] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_36_02_000002_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:07,116 INFO [Task-Executor-2] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_36_02_000002_0 2018-01-05T11:29:07,116 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_36_02_000002_0; false 2018-01-05T11:29:07,116 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_36_02_000002_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:07,116 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:TASK_FINISHED]: vertexName=Reducer 3, taskId=task_1515180518813_0001_36_02_000002, startTime=1515180547026, finishTime=1515180547116, timeTaken=90, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_36_02_000002_0, diagnostics=, counters=Counters: 35, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=1784, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=1339, ADDITIONAL_SPILLS_BYTES_READ=1339, SHUFFLE_BYTES=1361, SHUFFLE_BYTES_DECOMPRESSED=1353, SHUFFLE_BYTES_TO_MEM=1361, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=16, MERGE_PHASE_TIME=33, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=5, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0 2018-01-05T11:29:07,116 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:07,116 INFO [Task-Executor-2] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000083=89 2018-01-05T11:29:07,116 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:07,116 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_36_02_000002_0: true 2018-01-05T11:29:07,116 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_36_02_000002_0 2018-01-05T11:29:07,116 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000083, QueryId=djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd-36, Vertex=Reducer 3, FragmentNum=2, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:07,116 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_36_02_000002_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:29:07,116 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_36_02 [Reducer 3], tasks=4, failed=0, killed=0, success=4, completed=4, commits=0, err=null 2018-01-05T11:29:07,116 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_36_02 [Reducer 3] 2018-01-05T11:29:07,116 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_36_02_000002_0 2018-01-05T11:29:07,117 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:VERTEX_FINISHED]: vertexName=Reducer 3, vertexId=vertex_1515180518813_0001_36_02, initRequestedTime=1515180546769, initedTime=1515180546773, startRequestedTime=1515180546793, startedTime=1515180546793, finishTime=1515180547116, timeTaken=323, status=SUCCEEDED, diagnostics=, counters=Counters: 35, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=1784, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=6, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=1339, ADDITIONAL_SPILLS_BYTES_READ=1339, SHUFFLE_BYTES=1361, SHUFFLE_BYTES_DECOMPRESSED=1353, SHUFFLE_BYTES_TO_MEM=1361, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=34, MERGE_PHASE_TIME=53, FIRST_EVENT_RECEIVED=16, LAST_EVENT_RECEIVED=18, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, vertexStats=firstTaskStartTime=1515180547026, firstTasksToStart=[ task_1515180518813_0001_36_02_000002 ], lastTaskFinishTime=1515180547116, lastTasksToFinish=[ task_1515180518813_0001_36_02_000002 ], minTaskDuration=32, maxTaskDuration=90, avgTaskDuration=48.0, numSuccessfulTasks=4, shortestDurationTasks=[ task_1515180518813_0001_36_02_000003 ], longestDurationTasks=[ task_1515180518813_0001_36_02_000002 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=4, numSucceededTasks=4, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:07,117 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_36_02 [Reducer 3] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:07,117 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000083, stoppedTime=1515180547117, exitStatus=0 2018-01-05T11:29:07,117 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000083 exited with diagnostics set to Container completed. 2018-01-05T11:29:07,117 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_36_02 [Reducer 3] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3 2018-01-05T11:29:07,117 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null 2018-01-05T11:29:07,117 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_36_02 [Reducer 3] 2018-01-05T11:29:07,117 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_36_01 [Reducer 2] 2018-01-05T11:29:07,117 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_36_00 [Map 1] 2018-01-05T11:29:07,118 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_36, queueSize=0 2018-01-05T11:29:07,118 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_36][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_36, startTime=1515180546755, finishTime=1515180547117, timeTaken=362, status=SUCCEEDED, diagnostics=, counters=Counters: 60, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=7, TOTAL_LAUNCHED_TASKS=7, OTHER_LOCAL_TASKS=1, AM_CPU_MILLISECONDS=1202, AM_GC_TIME_MILLIS=0, File System Counters, FILE_BYTES_READ=3384, FILE_BYTES_WRITTEN=8737, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=154, REDUCE_INPUT_RECORDS=244, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=488, NUM_SHUFFLED_INPUTS=4, NUM_SKIPPED_INPUTS=6, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=4, INPUT_RECORDS_PROCESSED=2, INPUT_SPLIT_LENGTH_BYTES=2808, OUTPUT_RECORDS=244, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=4915, OUTPUT_BYTES_WITH_OVERHEAD=4936, OUTPUT_BYTES_PHYSICAL=4952, ADDITIONAL_SPILLS_BYTES_WRITTEN=4930, ADDITIONAL_SPILLS_BYTES_READ=4930, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=3, SHUFFLE_BYTES=4952, SHUFFLE_BYTES_DECOMPRESSED=4936, SHUFFLE_BYTES_TO_MEM=4952, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=50, MERGE_PHASE_TIME=108, FIRST_EVENT_RECEIVED=21, LAST_EVENT_RECEIVED=23, HIVE, CREATED_FILES=3, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=242, RECORDS_OUT_0=1, RECORDS_OUT_1_default.tab=242, RECORDS_OUT_INTERMEDIATE_Map_1=242, RECORDS_OUT_INTERMEDIATE_Reducer_2=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, org.apache.hadoop.hive.llap.counters.LlapIOCounters, CACHE_MISS_BYTES=2808, CONSUMER_TIME_NS=1783162, DECODE_TIME_NS=156809, NUM_DECODED_BATCHES=2, NUM_VECTOR_BATCHES=2, ROWS_EMITTED=242, TOTAL_IO_TIME_NS=1950765 2018-01-05T11:29:07,119 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_36 finished with state: SUCCEEDED 2018-01-05T11:29:07,119 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_36 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:29:07,119 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000083 2018-01-05T11:29:07,119 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_36, dagState=SUCCEEDED 2018-01-05T11:29:07,119 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_36_post 2018-01-05T11:29:07,119 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:29:07,119 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:29:07,119 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:29:07,119 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 36 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=7, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=7,NumRequestsWithoutLocation=7, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=7} 2018-01-05T11:29:07,119 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:29:07,119 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=insert overwrite table t...srcbucket_mapjoin (Stage-1), with id=dag_1515180518813_0001_36 2018-01-05T11:29:07,119 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 36, to 10.22.8.182:49818 2018-01-05T11:29:07,119 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=7, Remaining=0 2018-01-05T11:29:07,119 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=insert overwrite table t...srcbucket_mapjoin (Stage-1), with id=dag_1515180518813_0001_36 2018-01-05T11:29:07,119 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=36} 2018-01-05T11:29:07,119 INFO [IPC Server handler 2 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=36}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:29:07,119 INFO [IPC Server handler 2 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/36 after 300 seconds 2018-01-05T11:29:07,119 ERROR [IPC Server handler 2 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd, dag_1515180518813_0001_36 2018-01-05T11:29:07,119 INFO [IPC Server handler 2 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd 2018-01-05T11:29:07,119 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=36} = 0 2018-01-05T11:29:07,119 DEBUG [IPC Server handler 2 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=36} 2018-01-05T11:29:07,119 DEBUG [IPC Server handler 2 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:29:07,119 DEBUG [IPC Server handler 2 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=36} 2018-01-05T11:29:07,121 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,121 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,121 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,121 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 Reducer 2: 2/2 Reducer 3: 4/4 2018-01-05T11:29:07,121 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,122 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,122 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: TaskId for 000000_0 = 000000 2018-01-05T11:29:07,122 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: TaskId for 000001_0 = 000001 2018-01-05T11:29:07,122 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,122 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,122 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,123 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,123 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: TaskId for 000002_0 = 000002 2018-01-05T11:29:07,123 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,123 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,124 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,124 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-2:DEPENDENCY_COLLECTION] in serial mode 2018-01-05T11:29:07,124 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-0:MOVE] in serial mode 2018-01-05T11:29:07,124 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@717c8e2d, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:29:07,124 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:07,124 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,124 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:29:07,124 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:29:07,124 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:07,124 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:07,124 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,124 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,124 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:07,124 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:07,124 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,124 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:07,124 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,124 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:07,124 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:07,124 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,124 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Loading data to table default.tab partition (ds=2008-04-08) from file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-06_370_8944885619275660846-1/-ext-10000 2018-01-05T11:29:07,125 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,125 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:29:07,125 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:29:07,125 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:29:07,125 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:29:07,126 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,126 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,126 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,126 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,127 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,127 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:29:07,127 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:29:07,127 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,127 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:29:07,127 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,127 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab 2018-01-05T11:29:07,127 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab 2018-01-05T11:29:07,129 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,130 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,130 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.MoveTask: Partition is: {ds=2008-04-08} 2018-01-05T11:29:07,130 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,130 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: partition_name_has_valid_characters 2018-01-05T11:29:07,130 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=partition_name_has_valid_characters 2018-01-05T11:29:07,130 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,130 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,130 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab 2018-01-05T11:29:07,130 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab 2018-01-05T11:29:07,132 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,133 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,133 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,133 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=tab[2008-04-08] 2018-01-05T11:29:07,133 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=tab[2008-04-08] 2018-01-05T11:29:07,135 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,137 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,137 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,137 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,138 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:07,138 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-06_370_8944885619275660846-1/-ext-10000/ and the destination path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/ 2018-01-05T11:29:07,138 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: The source path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/ and the destination path is /Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-06_370_8944885619275660846-1/-ext-10000/ 2018-01-05T11:29:07,138 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:29:07,138 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Could not get hdfsEncryptionShim, it is only applicable to hdfs filesystem. 2018-01-05T11:29:07,159 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,159 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: altering partition for table tab with partition spec : {ds=2008-04-08} 2018-01-05T11:29:07,159 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: No new files were created, and is not a replace, or we're inserting into a partition that does not exist yet. Skipping generating INSERT event. 2018-01-05T11:29:07,160 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Adding new partition {ds=2008-04-08} 2018-01-05T11:29:07,160 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,160 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: add_partition : db=default tbl=tab 2018-01-05T11:29:07,160 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=add_partition : db=default tbl=tab 2018-01-05T11:29:07,169 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,170 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,170 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,170 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=tab[2008-04-08] 2018-01-05T11:29:07,170 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=tab[2008-04-08] 2018-01-05T11:29:07,172 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,176 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,178 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-3:STATS] in serial mode 2018-01-05T11:29:07,178 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@768112e5, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:29:07,178 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:07,178 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,178 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:29:07,178 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:29:07,178 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:07,178 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:07,178 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,178 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,178 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:07,178 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:07,178 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,178 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:07,178 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,178 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:07,178 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:07,178 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,178 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.BasicStatsTask: Executing stats task 2018-01-05T11:29:07,178 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] fs.FSStatsPublisher: Initing FSStatsPublisher with : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/-mr-10001 2018-01-05T11:29:07,178 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] fs.FSStatsPublisher: created : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/-mr-10001 2018-01-05T11:29:07,180 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,181 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:29:07,181 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:29:07,181 WARN [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:29:07,181 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:29:07,182 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,182 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,182 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,183 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,184 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,184 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:29:07,184 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:29:07,184 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,184 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:29:07,184 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,184 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab 2018-01-05T11:29:07,184 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab 2018-01-05T11:29:07,191 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,191 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,192 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,192 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partition_with_auth : db=default tbl=tab[2008-04-08] 2018-01-05T11:29:07,192 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partition_with_auth : db=default tbl=tab[2008-04-08] 2018-01-05T11:29:07,195 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,200 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,201 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,201 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.BasicStatsTask: Getting file stats of all partitions. threadpool size:15 2018-01-05T11:29:07,202 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.BasicStatsTask: Finished getting file stats of all partitions! 2018-01-05T11:29:07,202 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] fs.FSStatsAggregator: Part ID: default.tab/ds=2008-04-08/ numRows 2018-01-05T11:29:07,202 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] fs.FSStatsAggregator: Read stats for : default.tab/ds=2008-04-08/ numRows 242 2018-01-05T11:29:07,202 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] fs.FSStatsAggregator: Part ID: default.tab/ds=2008-04-08/ rawDataSize 2018-01-05T11:29:07,202 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] fs.FSStatsAggregator: Read stats for : default.tab/ds=2008-04-08/ rawDataSize 22748 2018-01-05T11:29:07,202 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.BasicStatsTask: Partition {ds=2008-04-08} stats: [numFiles=2, numRows=242, totalSize=1511, rawDataSize=22748] 2018-01-05T11:29:07,202 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,202 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: alter_partitions : db=default tbl=tab 2018-01-05T11:29:07,202 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=alter_partitions : db=default tbl=tab 2018-01-05T11:29:07,202 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: New partition values:[2008-04-08] 2018-01-05T11:29:07,205 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,212 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,216 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,216 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,217 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] fs.FSStatsAggregator: About to delete stats tmp dir :file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1/-mr-10001 2018-01-05T11:29:07,220 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000083, QueryId=djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd-36, DagId=36, VertexName=Reducer 3, TaskId=2, TaskAttemptId=0, ThreadName=Task-Executor-2, Succeeded=true, StartTime=1515180547027, EndTime=1515180547220 2018-01-05T11:29:07,233 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] mapred.FileInputFormat: Time taken to get FileStatuses: 14 2018-01-05T11:29:07,233 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:29:07,233 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 14 2018-01-05T11:29:07,233 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FetchOperator: Creating fetchTask with deserializer typeinfo: struct<_col0:struct,_col1:struct,_col2:string> 2018-01-05T11:29:07,233 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FetchOperator: deserializer properties: table properties: {columns=_col0,_col1,_col2, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct:struct:string, escape.delim=\} partition properties: {columns=_col0,_col1,_col2, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=struct:struct:string, escape.delim=\} 2018-01-05T11:29:07,233 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,236 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,236 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: write_partition_column_statistics: db=default table=tab part=ds=2008-04-08 2018-01-05T11:29:07,237 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=write_partition_column_statistics: db=default table=tab part=ds=2008-04-08 2018-01-05T11:29:07,240 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: Updating partition level column statistics for db=default tableName=tab partName=ds=2008-04-08 colName=key 2018-01-05T11:29:07,241 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,241 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.ObjectStore: Updating partition level column statistics for db=default tableName=tab partName=ds=2008-04-08 colName=value 2018-01-05T11:29:07,243 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,244 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,244 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,244 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,244 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: insert overwrite table tab partition (ds='2008-04-08') select key,value from srcbucket_mapjoin 2018-01-05T11:29:07,244 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:29:07,244 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@srcbucket_mapjoin 2018-01-05T11:29:07,244 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@srcbucket_mapjoin@ds=2008-04-08 2018-01-05T11:29:07,244 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: default@tab@ds=2008-04-08 2018-01-05T11:29:07,244 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Lineage: tab PARTITION(ds=2008-04-08).key SIMPLE [(srcbucket_mapjoin)srcbucket_mapjoin.FieldSchema(name:key, type:int, comment:null), ] 2018-01-05T11:29:07,244 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Lineage: tab PARTITION(ds=2008-04-08).value SIMPLE [(srcbucket_mapjoin)srcbucket_mapjoin.FieldSchema(name:value, type:string, comment:null), ] 2018-01-05T11:29:07,244 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,244 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,244 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:07,244 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {alter_partitions_(String, String, List, EnvironmentContext, )=14, getTable_(String, String, )=8, getPartitionWithAuthInfo_(String, String, List, String, List, )=9, setPartitionColumnStatistics_(SetPartitionsStatsRequest, )=11} 2018-01-05T11:29:07,244 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112906_bfd6f91a-0962-4158-ade2-2a7d1515d5bd); Time taken: 0.662 seconds 2018-01-05T11:29:07,244 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:07,244 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,244 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:07,244 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,244 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query insert overwrite table tab partition (ds='2008-04-08') select key,value from srcbucket_mapjoin 2018-01-05T11:29:07,244 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/.hive-staging_hive_2018-01-05_11-29-06_370_8944885619275660846-1 2018-01-05T11:29:07,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-06_370_8944885619275660846-1 2018-01-05T11:29:07,245 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.875 seconds 2018-01-05T11:29:07,245 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:07,245 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:07,246 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:07,246 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:07,246 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,246 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:07,246 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,246 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,246 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,246 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: explain vectorization detail select count(*) from tab a join tab_part b on a.key = b.key 2018-01-05T11:29:07,246 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112907_92f8e280-bcde-4186-b44c-5cae84e32628): explain vectorization detail select count(*) from tab a join tab_part b on a.key = b.key 2018-01-05T11:29:07,246 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,246 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: explain vectorization detail select count(*) from tab a join tab_part b on a.key = b.key 2018-01-05T11:29:07,247 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:07,247 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,247 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,247 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,247 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,247 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:07,247 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:29:07,247 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:07,248 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed phase 1 of Semantic Analysis 2018-01-05T11:29:07,248 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:07,248 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,248 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab 2018-01-05T11:29:07,248 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab 2018-01-05T11:29:07,250 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,251 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,251 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,251 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab_part 2018-01-05T11:29:07,251 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab_part 2018-01-05T11:29:07,253 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,254 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,254 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:07,254 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:07,254 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed getting MetaData in Semantic Analysis 2018-01-05T11:29:07,255 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,255 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,255 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_not_null_constraints : db=default tbl=tab 2018-01-05T11:29:07,255 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_not_null_constraints : db=default tbl=tab 2018-01-05T11:29:07,256 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,256 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,256 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,256 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab 2018-01-05T11:29:07,256 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab 2018-01-05T11:29:07,256 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,256 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,256 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,256 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab 2018-01-05T11:29:07,256 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab 2018-01-05T11:29:07,257 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,257 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,257 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,257 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_unique_constraints : db=default tbl=tab 2018-01-05T11:29:07,257 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_unique_constraints : db=default tbl=tab 2018-01-05T11:29:07,257 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,257 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,257 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,257 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab 2018-01-05T11:29:07,257 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab 2018-01-05T11:29:07,257 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,257 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,258 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_not_null_constraints : db=default tbl=tab_part 2018-01-05T11:29:07,258 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_not_null_constraints : db=default tbl=tab_part 2018-01-05T11:29:07,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,258 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:07,258 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:07,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,258 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,258 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:07,258 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:07,259 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,259 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,259 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,259 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_unique_constraints : db=default tbl=tab_part 2018-01-05T11:29:07,259 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_unique_constraints : db=default tbl=tab_part 2018-01-05T11:29:07,259 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,259 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,259 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,259 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab_part 2018-01-05T11:29:07,259 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab_part 2018-01-05T11:29:07,260 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,260 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,260 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:07,260 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:07,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:07,265 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:29:07,265 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,287 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.sql2rel: Plan after trimming unused fields HiveAggregate(group=[{}], agg#0=[count()]) HiveProject(DUMMY=[0]) HiveJoin(condition=[=($0, $1)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:07,287 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan before removing subquery: HiveProject(_o__c0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveProject($f0=[$0]) HiveJoin(condition=[=($0, $6)], joinType=[inner], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:07,288 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan just after removing subquery: HiveProject(_o__c0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveProject($f0=[$0]) HiveJoin(condition=[=($0, $6)], joinType=[inner], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:07,288 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan after decorrelation: HiveProject(_o__c0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveProject($f0=[$0]) HiveJoin(condition=[=($0, $6)], joinType=[inner], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:07,288 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,288 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,288 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,289 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,289 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,289 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,289 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,290 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,290 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,290 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,290 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,310 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,310 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,310 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,310 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,310 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partitions : db=default tbl=tab 2018-01-05T11:29:07,310 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions : db=default tbl=tab 2018-01-05T11:29:07,313 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,314 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.170628ms + 0.004994ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? ] 2018-01-05T11:29:07,318 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 3.710267ms + 0.067041ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (15) order by "PART_NAME" asc] 2018-01-05T11:29:07,320 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.924668ms + 0.082152ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (15) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:29:07,322 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.861246ms + 0.053871ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (15) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:07,323 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.671086ms + 0.015228ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (43) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:29:07,325 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.571133ms + 0.061125ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (43) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:07,326 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.417729ms + 0.033837ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (43) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:07,328 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.347209ms + 0.013322ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (43) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:07,329 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.472603ms + 0.034224ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (28) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:07,331 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.523082ms + 0.040631ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (43) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:29:07,331 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,334 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,334 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,334 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,335 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,335 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,335 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partitions : db=default tbl=tab_part 2018-01-05T11:29:07,335 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions : db=default tbl=tab_part 2018-01-05T11:29:07,337 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,338 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.171018ms + 0.00484ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? ] 2018-01-05T11:29:07,341 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 3.116698ms + 0.069141ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (14) order by "PART_NAME" asc] 2018-01-05T11:29:07,343 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.931029ms + 0.100572ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (14) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:29:07,345 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.60787ms + 0.036573ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (14) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:07,346 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.606847ms + 0.014533ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (41) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:29:07,348 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.849894ms + 0.045608ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (41) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:07,350 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.531991ms + 0.035088ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (41) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:07,352 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.726333ms + 0.014365ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (41) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:07,353 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.696566ms + 0.031739ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (26) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:07,355 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.458382ms + 0.034363ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (41) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:29:07,355 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,358 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,358 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,358 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,358 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,358 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.sql2rel: Plan after trimming unused fields HiveAggregate(group=[{}], agg#0=[count()]) HiveProject(DUMMY=[0]) HiveJoin(condition=[=($0, $1)], joinType=[inner], algorithm=[none], cost=[not available]) HiveFilter(condition=[IS NOT NULL($0)]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveFilter(condition=[IS NOT NULL($0)]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:07,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,378 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,378 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,378 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,378 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Original plan for PlanModifier HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:07,378 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after top-level introduceDerivedTable HiveProject($f0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:07,379 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after nested convertOpTree HiveProject($f0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:07,379 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after propagating order HiveProject($f0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:07,379 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after fixTopOBSchema HiveProject($f0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:07,379 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Final plan after modifier HiveProject(_c0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:07,380 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:07,380 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:07,380 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:07,380 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:07,380 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,380 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab 2018-01-05T11:29:07,380 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab 2018-01-05T11:29:07,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,383 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:07,383 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:07,383 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:07,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,383 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab_part 2018-01-05T11:29:07,383 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab_part 2018-01-05T11:29:07,389 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,389 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,389 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:07,389 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:07,389 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:07,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Table Plan for a TS[0] 2018-01-05T11:29:07,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Filter Plan for $hdt$_0 row schema: a{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:07,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB a{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB a{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:07,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (. (tok_table_or_col a) key) key)) 2018-01-05T11:29:07,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = a{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} starRr = null 2018-01-05T11:29:07,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(key,_col0: int)} 2018-01-05T11:29:07,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:07,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block $hdt$_0 2018-01-05T11:29:07,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block $hdt$_0 2018-01-05T11:29:07,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Table Plan for b TS[3] 2018-01-05T11:29:07,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Filter Plan for $hdt$_1 row schema: b{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:07,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB b{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB b{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:07,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (. (tok_table_or_col b) key) key)) 2018-01-05T11:29:07,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = b{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} starRr = null 2018-01-05T11:29:07,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(key,_col0: int)} 2018-01-05T11:29:07,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:07,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block $hdt$_1 2018-01-05T11:29:07,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block $hdt$_1 2018-01-05T11:29:07,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:07,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB $hdt$_0{(key,_col0: int)} $hdt$_1{(key,_col1: int)} after GB {((tok_functionstar count),_col0: bigint)} 2018-01-05T11:29:07,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (tok_functionstar count) _c0)) 2018-01-05T11:29:07,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = {((tok_functionstar count),_col0: bigint)} starRr = $hdt$_0{(key,_col0: int)} $hdt$_1{(key,_col1: int)} 2018-01-05T11:29:07,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(_c0,_col0: bigint)} 2018-01-05T11:29:07,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:07,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_246_7640625295093051097-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-07_246_7640625295093051097-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_246_7640625295093051097-1/-mr-10001 2018-01-05T11:29:07,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_246_7640625295093051097-1/-mr-10003 2018-01-05T11:29:07,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_246_7640625295093051097-1/-mr-10001 row schema: null{(_c0,_col0: bigint)} 2018-01-05T11:29:07,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block null 2018-01-05T11:29:07,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:29:07,392 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: CBO Succeeded; optimized logical plan. 2018-01-05T11:29:07,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Before logical optimization TS[0]-FIL[1]-SEL[2]-RS[6]-JOIN[8]-SEL[9]-GBY[10]-RS[11]-GBY[12]-SEL[13]-FS[14] TS[3]-FIL[4]-SEL[5]-RS[7]-JOIN[8] 2018-01-05T11:29:07,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] lineage.Generator: Time taken for lineage transform=1 2018-01-05T11:29:07,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.PartitionColumnsSeparator: Partition columns not separated for null, is not IN operator : 2018-01-05T11:29:07,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.PartitionColumnsSeparator: Partition columns not separated for null, is not IN operator : 2018-01-05T11:29:07,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SyntheticJoinPredicate: Synthetic predicate: 0 --> 1 2018-01-05T11:29:07,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SyntheticJoinPredicate: Synthetic predicate: 1 --> 0 2018-01-05T11:29:07,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,393 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FS(14) 2018-01-05T11:29:07,393 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(13) 2018-01-05T11:29:07,393 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for GBY(12) 2018-01-05T11:29:07,393 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(11) 2018-01-05T11:29:07,393 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for GBY(10) 2018-01-05T11:29:07,393 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(9) 2018-01-05T11:29:07,393 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for JOIN(8) 2018-01-05T11:29:07,393 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(6) 2018-01-05T11:29:07,393 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(15) 2018-01-05T11:29:07,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias $hdt$_0: (_col0) IN (RS[7]) 2018-01-05T11:29:07,393 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(2) 2018-01-05T11:29:07,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of SEL for alias $hdt$_0: (key) IN (RS[7]) 2018-01-05T11:29:07,393 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(1) 2018-01-05T11:29:07,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias a: key is not null 2018-01-05T11:29:07,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias a: (key) IN (RS[7]) 2018-01-05T11:29:07,394 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:29:07,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of TS for alias a: key is not null; (key) IN (RS[7]) 2018-01-05T11:29:07,394 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(7) 2018-01-05T11:29:07,394 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(16) 2018-01-05T11:29:07,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias $hdt$_1: (_col0) IN (RS[6]) 2018-01-05T11:29:07,394 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(5) 2018-01-05T11:29:07,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of SEL for alias $hdt$_1: (key) IN (RS[6]) 2018-01-05T11:29:07,394 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(4) 2018-01-05T11:29:07,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias b: key is not null 2018-01-05T11:29:07,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias b: (key) IN (RS[6]) 2018-01-05T11:29:07,394 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for TS(3) 2018-01-05T11:29:07,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of TS for alias b: key is not null; (key) IN (RS[6]) 2018-01-05T11:29:07,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SimplePredicatePushDown: After PPD: TS[0]-FIL[17]-SEL[2]-RS[6]-JOIN[8]-SEL[9]-GBY[10]-RS[11]-GBY[12]-SEL[13]-FS[14] TS[3]-FIL[18]-SEL[5]-RS[7]-JOIN[8] 2018-01-05T11:29:07,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pcr.PcrExprProcFactory: Processing key is not null org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNotNull outputs {UNKNOWN, Column[key]}, 2018-01-05T11:29:07,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pcr.PcrExprProcFactory: Processing (key) IN (RS[7]) org.apache.hadoop.hive.ql.udf.generic.GenericUDFIn outputs {UNKNOWN, Column[key]}, {UNKNOWN, RS[7]}, 2018-01-05T11:29:07,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pcr.PcrExprProcFactory: Processing (key is not null and (key) IN (RS[7])) org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPAnd outputs {UNKNOWN, GenericUDFOPNotNull(Column[key])}, {UNKNOWN, GenericUDFIn(Column[key], RS[7])}, 2018-01-05T11:29:07,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pcr.PcrExprProcFactory: Processing key is not null org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNotNull outputs {UNKNOWN, Column[key]}, 2018-01-05T11:29:07,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pcr.PcrExprProcFactory: Processing (key) IN (RS[6]) org.apache.hadoop.hive.ql.udf.generic.GenericUDFIn outputs {UNKNOWN, Column[key]}, {UNKNOWN, RS[6]}, 2018-01-05T11:29:07,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pcr.PcrExprProcFactory: Processing (key is not null and (key) IN (RS[6])) org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPAnd outputs {UNKNOWN, GenericUDFOPNotNull(Column[key])}, {UNKNOWN, GenericUDFIn(Column[key], RS[6])}, 2018-01-05T11:29:07,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 11 key:[] 2018-01-05T11:29:07,400 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 11 oldColExprMap: {VALUE._col0=Column[_col0]} 2018-01-05T11:29:07,400 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 11 newColExprMap: {VALUE._col0=Column[_col0]} 2018-01-05T11:29:07,400 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: JOIN 8 oldExprs: {0=[Column[KEY.reducesinkkey0]], 1=[Column[KEY.reducesinkkey0]]} 2018-01-05T11:29:07,400 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: JOIN 8 newExprs: {0=[], 1=[]} 2018-01-05T11:29:07,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 6 key:[Column[_col0]] 2018-01-05T11:29:07,400 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 6 oldColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:07,400 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 6 newColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:07,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 7 key:[Column[_col0]] 2018-01-05T11:29:07,400 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 7 oldColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:07,400 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 7 newColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:07,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,401 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,401 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,401 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,401 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,401 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.IdentityProjectRemover: Identity project remover optimization removed : SEL[13] 2018-01-05T11:29:07,401 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.IdentityProjectRemover: Identity project remover optimization removed : SEL[9] 2018-01-05T11:29:07,401 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,401 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,401 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,401 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,401 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,401 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,401 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,401 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: After logical optimization TS[0]-FIL[17]-SEL[2]-RS[6]-JOIN[8]-GBY[10]-RS[11]-GBY[12]-FS[14] TS[3]-FIL[18]-SEL[5]-RS[7]-JOIN[8] 2018-01-05T11:29:07,401 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,401 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Parent: TS[0] 2018-01-05T11:29:07,401 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Filter: (key is not null and (key) IN (RS[7])) (type: boolean) 2018-01-05T11:29:07,401 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: TableScan: TS[0] 2018-01-05T11:29:07,401 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Column key is not a partition column 2018-01-05T11:29:07,401 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Initiate semijoin reduction for key ((key is not null and (key) IN (RS[7])) 2018-01-05T11:29:07,401 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:07,401 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:07,401 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:07,401 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:07,401 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:07,401 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:07,401 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: DynamicSemiJoinPushdown: Saving RS to TS mapping: RS[23]: TS[0] 2018-01-05T11:29:07,401 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Parent: TS[3] 2018-01-05T11:29:07,401 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Filter: (key is not null and (key) IN (RS[6])) (type: boolean) 2018-01-05T11:29:07,401 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: TableScan: TS[3] 2018-01-05T11:29:07,401 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Column key is not a partition column 2018-01-05T11:29:07,402 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Initiate semijoin reduction for key ((key is not null and (key) IN (RS[6])) 2018-01-05T11:29:07,402 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:07,402 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:07,402 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:07,402 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:07,402 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:07,402 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:07,402 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: DynamicSemiJoinPushdown: Saving RS to TS mapping: RS[28]: TS[3] 2018-01-05T11:29:07,402 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,402 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,402 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Estimated average row size: 4 2018-01-05T11:29:07,402 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,402 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_aggr_stats_for: db=default table=tab 2018-01-05T11:29:07,402 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_aggr_stats_for: db=default table=tab 2018-01-05T11:29:07,404 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.AggregateStatsCache: No aggregate stats cached for database:default, table:tab, column:key 2018-01-05T11:29:07,406 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 2.182943ms + 0.005371ms, the query is [select count("COLUMN_NAME") from "PART_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (?) and "PARTITION_NAME" in (?) group by "PARTITION_NAME"] 2018-01-05T11:29:07,406 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.133149ms + 0.003434ms, the query is [select "PARTITION_NAME", "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "PART_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (%1$s) AND "PARTITION_NAME" in (%2$s) order by "PARTITION_NAME"] 2018-01-05T11:29:07,421 DEBUG [aggr-col-stats-0] aggr.LongColumnStatsAggregator: doAllPartitionContainStats for key is true 2018-01-05T11:29:07,421 DEBUG [aggr-col-stats-0] aggr.LongColumnStatsAggregator: all of the bit vectors can merge for key is true 2018-01-05T11:29:07,421 DEBUG [aggr-col-stats-0] aggr.LongColumnStatsAggregator: Ndv estimatation for key is 153 # of partitions requested: 1 # of partitions found: 1 2018-01-05T11:29:07,421 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] utils.MetaStoreUtils: Time for aggr col stats in seconds: 0.014 Threads used: 1 2018-01-05T11:29:07,422 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: useDensityFunctionForNDVEstimation = false partsFound = 1 ColumnStatisticsObj = [ColumnStatisticsObj(colName:key, colType:int, statsData:)] 2018-01-05T11:29:07,423 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,424 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,424 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting stats (Num rows: 242 Data size: 968 Basic stats: COMPLETE Column stats: COMPLETE) on TS[0] 2018-01-05T11:29:07,424 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (tab): numRows: 242 dataSize: 968 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {key= colName: key colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:07,424 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Estimated average row size: 4 2018-01-05T11:29:07,424 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,424 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_aggr_stats_for: db=default table=tab_part 2018-01-05T11:29:07,424 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_aggr_stats_for: db=default table=tab_part 2018-01-05T11:29:07,426 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.AggregateStatsCache: No aggregate stats cached for database:default, table:tab_part, column:key 2018-01-05T11:29:07,426 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.185009ms + 0.004796ms, the query is [select count("COLUMN_NAME") from "PART_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (?) and "PARTITION_NAME" in (?) group by "PARTITION_NAME"] 2018-01-05T11:29:07,426 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.100874ms + 0.003037ms, the query is [select "PARTITION_NAME", "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "PART_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (%1$s) AND "PARTITION_NAME" in (%2$s) order by "PARTITION_NAME"] 2018-01-05T11:29:07,426 DEBUG [aggr-col-stats-0] aggr.LongColumnStatsAggregator: doAllPartitionContainStats for key is true 2018-01-05T11:29:07,426 DEBUG [aggr-col-stats-0] aggr.LongColumnStatsAggregator: all of the bit vectors can merge for key is true 2018-01-05T11:29:07,427 DEBUG [aggr-col-stats-0] aggr.LongColumnStatsAggregator: Ndv estimatation for key is 308 # of partitions requested: 1 # of partitions found: 1 2018-01-05T11:29:07,427 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] utils.MetaStoreUtils: Time for aggr col stats in seconds: 0.001 Threads used: 1 2018-01-05T11:29:07,427 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: useDensityFunctionForNDVEstimation = false partsFound = 1 ColumnStatisticsObj = [ColumnStatisticsObj(colName:key, colType:int, statsData:)] 2018-01-05T11:29:07,428 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,428 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,428 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on TS[3] 2018-01-05T11:29:07,428 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-TS[3] (tab_part): numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {key= colName: key colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:07,428 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_7_b_key_min), DynamicValue(RS_7_b_key_max)) Original num rows: 242 New num rows: 242 2018-01-05T11:29:07,428 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_7_b_key_min), DynamicValue(RS_7_b_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_7_b_key_bloom_filter))) Original num rows: 242 New num rows: 242 2018-01-05T11:29:07,428 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_7_b_key_min), DynamicValue(RS_7_b_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_7_b_key_bloom_filter)))) Original num rows: 242 New num rows: 242 2018-01-05T11:29:07,428 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-FIL[17]: numRows: 242 dataSize: 968 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {key= colName: key colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:07,428 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting stats (Num rows: 242 Data size: 968 Basic stats: COMPLETE Column stats: COMPLETE) on FIL[17] 2018-01-05T11:29:07,428 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_6_a_key_min), DynamicValue(RS_6_a_key_max)) Original num rows: 500 New num rows: 500 2018-01-05T11:29:07,428 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_6_a_key_min), DynamicValue(RS_6_a_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_6_a_key_bloom_filter))) Original num rows: 500 New num rows: 500 2018-01-05T11:29:07,428 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_6_a_key_min), DynamicValue(RS_6_a_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_6_a_key_bloom_filter)))) Original num rows: 500 New num rows: 500 2018-01-05T11:29:07,429 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-FIL[18]: numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {key= colName: key colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:07,429 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on FIL[18] 2018-01-05T11:29:07,429 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 242 Data size: 968 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[2] 2018-01-05T11:29:07,429 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[2]: numRows: 242 dataSize: 968 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:07,429 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[5] 2018-01-05T11:29:07,429 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[5]: numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:07,429 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 242 Data size: 968 Basic stats: COMPLETE Column stats: COMPLETE) on RS[6] 2018-01-05T11:29:07,429 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[6]: numRows: 242 dataSize: 968 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {KEY.reducesinkkey0= colName: KEY.reducesinkkey0 colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:07,429 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 242 Data size: 968 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[24] 2018-01-05T11:29:07,429 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[24]: numRows: 242 dataSize: 968 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:07,429 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on RS[7] 2018-01-05T11:29:07,429 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[7]: numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {KEY.reducesinkkey0= colName: KEY.reducesinkkey0 colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:07,429 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[19] 2018-01-05T11:29:07,429 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[19]: numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:07,429 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-JOIN[8]: detects none/multiple PK parents. 2018-01-05T11:29:07,429 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.JoinOperator: Setting stats (Num rows: 392 Data size: 3136 Basic stats: COMPLETE Column stats: COMPLETE) on JOIN[8] 2018-01-05T11:29:07,429 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-JOIN[8]: numRows: 392 dataSize: 3136 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:07,429 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[25]: inputSize: 968 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:07,429 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[25] hashAgg: true 2018-01-05T11:29:07,429 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 3] STATS-GBY[25]: cardinality: 1 2018-01-05T11:29:07,429 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[25] 2018-01-05T11:29:07,429 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[25]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:07,429 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[20]: inputSize: 2000 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:07,429 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[20] hashAgg: true 2018-01-05T11:29:07,429 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 3] STATS-GBY[20]: cardinality: 1 2018-01-05T11:29:07,429 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[20] 2018-01-05T11:29:07,429 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[20]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:07,429 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[10]: inputSize: 3136 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:07,429 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[10] hashAgg: true 2018-01-05T11:29:07,429 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 3] STATS-GBY[10]: cardinality: 1 2018-01-05T11:29:07,429 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[10] 2018-01-05T11:29:07,429 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[10]: numRows: 1 dataSize: 8 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:07,429 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on RS[26] 2018-01-05T11:29:07,429 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[26]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on RS[21] 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[21]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE) on RS[11] 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[11]: numRows: 1 dataSize: 8 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {VALUE._col0= colName: VALUE._col0 colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[27]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[27] hashAgg: false 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 9] STATS-GBY[27]: cardinality: 1 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[27] 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[27]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[22]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[22] hashAgg: false 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 9] STATS-GBY[22]: cardinality: 1 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[22] 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[22]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[12]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[12] hashAgg: false 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 9] STATS-GBY[12]: cardinality: 1 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[12] 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[12]: numRows: 1 dataSize: 8 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on RS[28] 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[28]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on RS[23] 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[23]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-FS[14]: numRows: 1 dataSize: 8 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[key]]; bucket count: 2 }) on TS[0] 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[key]]; bucket count: 4 }) on TS[3] 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[key]]; bucket count: 2 }) on FIL[17] 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[key]]; bucket count: 4 }) on FIL[18] 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: 2 }) on SEL[2] 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: 4 }) on SEL[5] 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[KEY.reducesinkkey0]]; sort column names: [[KEY.reducesinkkey0]]; bucket count: 2 }) on RS[6] 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: 2 }) on SEL[24] 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[KEY.reducesinkkey0]]; sort column names: [[KEY.reducesinkkey0]]; bucket count: 4 }) on RS[7] 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: 4 }) on SEL[19] 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.JoinOperator: Setting traits ({ bucket column names: [[], []]; sort column names: [[], []]; bucket count: -1 }) on JOIN[8] 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[25] 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[20] 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[10] 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[26] 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[21] 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[11] 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[27] 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[22] 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[12] 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[28] 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[23] 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FileSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on FS[14] 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,430 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Set parallelism for reduce sink RS[6] to: 1 2018-01-05T11:29:07,430 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Set parallelism for reduce sink RS[7] to: 1 2018-01-05T11:29:07,430 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConvertJoinMapJoin: Memory monitor info set to : { isLlap: true executorsPerNode: 4 maxExecutorsOverSubscribeMemory: 3 memoryOverSubscriptionFactor: 0.20000000298023224 memoryCheckInterval: 100000 noConditionalTaskSize: 10000000 adjustedNoConditionalTaskSize: 16000000 hashTableInflationFactor: 2.0 threshold: 32000000 } 2018-01-05T11:29:07,430 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConvertJoinMapJoin: Fallback to common merge join operator 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.CommonMergeJoinOperator: Setting traits ({ bucket column names: [[], []]; sort column names: [[], []]; bucket count: 0 }) on MERGEJOIN[29] 2018-01-05T11:29:07,430 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.CommonMergeJoinOperator: Setting stats (Num rows: 392 Data size: 3136 Basic stats: COMPLETE Column stats: COMPLETE) on MERGEJOIN[29] 2018-01-05T11:29:07,430 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:07,430 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:07,430 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:07,430 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:07,430 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing BloomFilter cost/benefit for RS[23] - TS[0] (a) (Column[key]) 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing key domain cardinality, keyDomainCardinality=461, semiJoinKeyIsPK=false, selColStat= colName: _col0 colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false, selColSourceStat= colName: key colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false, tsColStat= colName: key colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit for Column[_col0] to Column[key], selKeyCardinality=308, tsKeyCardinality=153, tsRows=242, keyDomainCardinality=461 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: SemiJoin key selectivity=0.6681127982646421, benefit=80.31670281995662 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit=80.31670281995662, cost=500.0, tsDataSize=242, netBenefit=-419.6832971800434 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: netBenefit=-1.7342285007439808 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing BloomFilter cost/benefit for RS[28] - TS[3] (b) (Column[key]) 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing key domain cardinality, keyDomainCardinality=461, semiJoinKeyIsPK=false, selColStat= colName: _col0 colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false, selColSourceStat= colName: key colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false, tsColStat= colName: key colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit for Column[_col0] to Column[key], selKeyCardinality=153, tsKeyCardinality=308, tsRows=500, keyDomainCardinality=461 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: SemiJoin key selectivity=0.3318872017353579, benefit=334.056399132321 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit=334.056399132321, cost=242.0, tsDataSize=500, netBenefit=92.05639913232102 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: netBenefit=0.18411279826464202 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Reduction factor not satisfied for RS[23]-TS[0] (a). Removing semijoin optimization. 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Removing ReduceSink RS[23] and TableScan TS[0] 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Reduction factor not satisfied for RS[28]-TS[3] (b). Removing semijoin optimization. 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Removing ReduceSink RS[28] and TableScan TS[3] 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: FS, 14 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: GBY, 12 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 11 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: GBY, 10 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:07,431 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: MERGEJOIN, 29 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 6 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 2 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: FIL, 17 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 7 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 5 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: FIL, 18 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: TS, 3 2018-01-05T11:29:07,432 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Cycle free: true 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: Before SharedWorkOptimizer: TS[0]-FIL[17]-SEL[2]-RS[6]-MERGEJOIN[29]-GBY[10]-RS[11]-GBY[12]-FS[14] TS[3]-FIL[18]-SEL[5]-RS[7]-MERGEJOIN[29] 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: DPP information stored in the cache: {} 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: Sorted tables by size: [default.tab_part=2000, default.tab=968] 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: After SharedWorkOptimizer: TS[0]-FIL[17]-SEL[2]-RS[6]-MERGEJOIN[29]-GBY[10]-RS[11]-GBY[12]-FS[14] TS[3]-FIL[18]-SEL[5]-RS[7]-MERGEJOIN[29] 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: int|{a}key,value: string|{a}value,ds: string|{a}ds,BLOCK__OFFSET__INSIDE__FILE: bigint|{a}block__offset__inside__file,INPUT__FILE__NAME: string|{a}input__file__name,ROW__ID: struct|{a}row__id) 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(Const boolean true, Const boolean true) -> Const boolean true 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPAnd(Const boolean true, Const boolean true)) -> GenericUDFOPNotNull(Column[key]) 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[3] with rs:(key: int|{b}key,value: string|{b}value,ds: string|{b}ds,BLOCK__OFFSET__INSIDE__FILE: bigint|{b}block__offset__inside__file,INPUT__FILE__NAME: string|{b}input__file__name,ROW__ID: struct|{b}row__id) 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(Const boolean true, Const boolean true) -> Const boolean true 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPAnd(Const boolean true, Const boolean true)) -> GenericUDFOPNotNull(Column[key]) 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FIL[17] with rs:(key: int|{a}key) 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FIL[17] 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Old filter FIL[17] conditions:(key is not null and (true and true)) 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(Const boolean true, Const boolean true) -> Const boolean true 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPAnd(Const boolean true, Const boolean true)) -> GenericUDFOPNotNull(Column[key]) 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New filter FIL[17] conditions:key is not null 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FIL[18] with rs:(key: int|{b}key) 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_3 {} 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FIL[18] 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Old filter FIL[18] conditions:(key is not null and (true and true)) 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(Const boolean true, Const boolean true) -> Const boolean true 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPAnd(Const boolean true, Const boolean true)) -> GenericUDFOPNotNull(Column[key]) 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New filter FIL[18] conditions:key is not null 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[2] with rs:(_col0: int|{$hdt$_0}key) 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op FIL_17 {} 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[2] 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key]) 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: int|{$hdt$_1}key) 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op FIL_18 {} 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key]) 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[6] with rs:(KEY.reducesinkkey0: int|{$hdt$_0}key) 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_2 {} 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[6] 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[7] with rs:(KEY.reducesinkkey0: int|{$hdt$_1}key) 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[7] 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:MERGEJOIN[29] with rs:() 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_6 {} 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_7 {} 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator MERGEJOIN[29] 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[10] with rs:(_col0: bigint|{}(tok_functionstar count)) 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op MERGEJOIN_29 {} 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[10] 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[11] with rs:(VALUE._col0: bigint|{}(tok_functionstar count)) 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_10 {} 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[11] 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[12] with rs:(_col0: bigint|{}(tok_functionstar count)) 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_11 {} 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[12] 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[14] with rs:(_col0: bigint|{}) 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_12 {} 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[14] 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: TS[0] 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[6] 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:29:07,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 of table a 2018-01-05T11:29:07,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:07,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[6] 2018-01-05T11:29:07,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: MERGEJOIN[29] 2018-01-05T11:29:07,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[11] 2018-01-05T11:29:07,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding reduce work (Reducer 2) for MERGEJOIN[29] 2018-01-05T11:29:07,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Setting up reduce sink: RS[6] with following reduce work: Reducer 2 2018-01-05T11:29:07,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[6] as parent from MERGEJOIN[29] 2018-01-05T11:29:07,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[7] as parent from MERGEJOIN[29] 2018-01-05T11:29:07,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[11] 2018-01-05T11:29:07,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: GBY[12] 2018-01-05T11:29:07,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: FS[14] 2018-01-05T11:29:07,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding reduce work (Reducer 3) for GBY[12] 2018-01-05T11:29:07,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Setting up reduce sink: RS[11] with following reduce work: Reducer 3 2018-01-05T11:29:07,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[11] as parent from GBY[12] 2018-01-05T11:29:07,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: FS[14] 2018-01-05T11:29:07,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: TS[3] 2018-01-05T11:29:07,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[7] 2018-01-05T11:29:07,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding map work (Map 4) for TS[3] 2018-01-05T11:29:07,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 of table b 2018-01-05T11:29:07,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:07,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Second pass. Leaf operator: RS[7] has common downstream work: org.apache.hadoop.hive.ql.plan.MergeJoinWork@1ca8a60d 2018-01-05T11:29:07,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:29:07,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:29:07,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:07,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:07,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 4 2018-01-05T11:29:07,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:07,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:07,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:29:07,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:07,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:07,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 4 2018-01-05T11:29:07,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:07,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:07,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:29:07,434 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Examining input format to see if vectorization is enabled. 2018-01-05T11:29:07,434 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorization is enabled for input format(s) [org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] 2018-01-05T11:29:07,434 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing MapWork... 2018-01-05T11:29:07,434 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFOPNotNull(Column[key]), Vectorized Expression = SelectColumnIsNotNull(col 0:int) 2018-01-05T11:29:07,434 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator 2018-01-05T11:29:07,434 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.FilterDesc 2018-01-05T11:29:07,434 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[key], Vectorized Expression = col 0:int 2018-01-05T11:29:07,434 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator 2018-01-05T11:29:07,434 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.SelectDesc 2018-01-05T11:29:07,434 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:int 2018-01-05T11:29:07,434 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator groupby typeName int 2018-01-05T11:29:07,435 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkLongOperator 2018-01-05T11:29:07,442 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:07,442 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkLongOperator 2018-01-05T11:29:07,442 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:07,442 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [key, value, ds, ROW__ID] 2018-01-05T11:29:07,442 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [int, string, string, struct] 2018-01-05T11:29:07,442 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE, NONE, NONE, NONE] 2018-01-05T11:29:07,442 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 1 2018-01-05T11:29:07,442 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 1 2018-01-05T11:29:07,442 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:07,442 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:07,442 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Examining input format to see if vectorization is enabled. 2018-01-05T11:29:07,442 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorization is enabled for input format(s) [org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] 2018-01-05T11:29:07,442 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing MapWork... 2018-01-05T11:29:07,442 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFOPNotNull(Column[key]), Vectorized Expression = SelectColumnIsNotNull(col 0:int) 2018-01-05T11:29:07,442 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator 2018-01-05T11:29:07,442 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.FilterDesc 2018-01-05T11:29:07,442 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[key], Vectorized Expression = col 0:int 2018-01-05T11:29:07,442 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator 2018-01-05T11:29:07,442 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.SelectDesc 2018-01-05T11:29:07,442 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:int 2018-01-05T11:29:07,442 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator groupby typeName int 2018-01-05T11:29:07,442 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkLongOperator 2018-01-05T11:29:07,442 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:07,442 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkLongOperator 2018-01-05T11:29:07,442 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:07,442 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [key, value, ds, ROW__ID] 2018-01-05T11:29:07,442 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [int, string, string, struct] 2018-01-05T11:29:07,442 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE, NONE, NONE, NONE] 2018-01-05T11:29:07,442 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 1 2018-01-05T11:29:07,442 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 1 2018-01-05T11:29:07,442 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:07,442 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:07,442 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Using reduce tag 0 2018-01-05T11:29:07,443 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0] columnTypes=[bigint] 2018-01-05T11:29:07,443 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing ReduceWork... 2018-01-05T11:29:07,443 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vector GROUP BY operator will use processing mode GLOBAL 2018-01-05T11:29:07,444 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[VALUE._col0], Vectorized Expression = col 0:bigint 2018-01-05T11:29:07,444 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator 2018-01-05T11:29:07,444 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.GroupByDesc 2018-01-05T11:29:07,444 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorFileSinkOperator 2018-01-05T11:29:07,444 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.FileSinkDesc 2018-01-05T11:29:07,444 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [VALUE._col0] 2018-01-05T11:29:07,444 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [bigint] 2018-01-05T11:29:07,444 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE] 2018-01-05T11:29:07,444 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 0 2018-01-05T11:29:07,444 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 0 2018-01-05T11:29:07,444 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:07,444 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:07,444 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:29:07,445 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: llap mode: only 2018-01-05T11:29:07,445 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:29:07,445 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key' 2018-01-05T11:29:07,445 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key is not null' 2018-01-05T11:29:07,445 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key is not null' 2018-01-05T11:29:07,445 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:07,445 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@116ae9ac can be done in LLAP 2018-01-05T11:29:07,445 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Map 4 2018-01-05T11:29:07,445 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key' 2018-01-05T11:29:07,445 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key is not null' 2018-01-05T11:29:07,445 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key is not null' 2018-01-05T11:29:07,445 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:07,445 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@6f13b21a can be done in LLAP 2018-01-05T11:29:07,445 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Reducer 2 2018-01-05T11:29:07,445 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'count()' 2018-01-05T11:29:07,445 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:07,445 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MergeJoinWork@1ca8a60d can be done in LLAP 2018-01-05T11:29:07,445 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Reducer 3 2018-01-05T11:29:07,445 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'count(VALUE._col0)' 2018-01-05T11:29:07,445 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:29:07,445 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:07,445 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@29f21901 can be done in LLAP 2018-01-05T11:29:07,445 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.SerializeFilter: Serializing: key is not null 2018-01-05T11:29:07,445 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.SerializeFilter: Serializing: key is not null 2018-01-05T11:29:07,445 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,445 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed plan generation 2018-01-05T11:29:07,445 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: validation start 2018-01-05T11:29:07,445 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:29:07,445 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:07,445 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,445 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:Explain, type:string, comment:null)], properties:null) 2018-01-05T11:29:07,445 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,445 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:07,445 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {getNotNullConstraints_(NotNullConstraintsRequest, )=1, isCompatibleWith_(Configuration, )=0, getAggrColStatsFor_(String, String, List, List, )=26, getTable_(String, String, )=15, flushCache_()=0, getUniqueConstraints_(UniqueConstraintsRequest, )=0, getPrimaryKeys_(PrimaryKeysRequest, )=2, listPartitions_(String, String, short, )=47, getForeignKeys_(ForeignKeysRequest, )=1} 2018-01-05T11:29:07,445 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112907_92f8e280-bcde-4186-b44c-5cae84e32628); Time taken: 0.199 seconds 2018-01-05T11:29:07,445 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:07,445 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,445 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112907_92f8e280-bcde-4186-b44c-5cae84e32628): explain vectorization detail select count(*) from tab a join tab_part b on a.key = b.key 2018-01-05T11:29:07,447 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,447 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: explain vectorization detail select count(*) from tab a join tab_part b on a.key = b.key 2018-01-05T11:29:07,447 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:29:07,447 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,447 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,447 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,447 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,447 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-3:EXPLAIN] in serial mode 2018-01-05T11:29:07,488 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,488 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,488 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: explain vectorization detail select count(*) from tab a join tab_part b on a.key = b.key 2018-01-05T11:29:07,488 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:29:07,488 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,488 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,488 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:07,488 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {} 2018-01-05T11:29:07,488 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112907_92f8e280-bcde-4186-b44c-5cae84e32628); Time taken: 0.043 seconds 2018-01-05T11:29:07,488 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:07,488 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,488 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:07,488 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,488 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query explain vectorization detail select count(*) from tab a join tab_part b on a.key = b.key 2018-01-05T11:29:07,490 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting result file: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_246_7640625295093051097-1/-local-10004 2018-01-05T11:29:07,490 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_246_7640625295093051097-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-07_246_7640625295093051097-1 2018-01-05T11:29:07,490 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_246_7640625295093051097-1 2018-01-05T11:29:07,490 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.242 seconds, Fetched: 187 row(s) 2018-01-05T11:29:07,490 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:07,490 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:07,490 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:07,490 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:07,490 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,490 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:07,490 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,490 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,490 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,490 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: select * from tab a join tab_part b on a.key = b.key 2018-01-05T11:29:07,490 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949): select * from tab a join tab_part b on a.key = b.key 2018-01-05T11:29:07,490 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,490 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: select * from tab a join tab_part b on a.key = b.key 2018-01-05T11:29:07,492 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:07,492 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,492 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,492 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,492 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,492 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:07,492 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:29:07,492 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed phase 1 of Semantic Analysis 2018-01-05T11:29:07,493 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:07,493 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,493 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab 2018-01-05T11:29:07,493 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab 2018-01-05T11:29:07,495 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,496 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,496 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,496 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab_part 2018-01-05T11:29:07,496 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab_part 2018-01-05T11:29:07,498 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,499 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,499 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:07,499 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:07,511 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed getting MetaData in Semantic Analysis 2018-01-05T11:29:07,512 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,512 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,512 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_not_null_constraints : db=default tbl=tab 2018-01-05T11:29:07,512 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_not_null_constraints : db=default tbl=tab 2018-01-05T11:29:07,512 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,512 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,512 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,512 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab 2018-01-05T11:29:07,512 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab 2018-01-05T11:29:07,513 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,513 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,513 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,513 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab 2018-01-05T11:29:07,513 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab 2018-01-05T11:29:07,513 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,513 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,513 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,513 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_unique_constraints : db=default tbl=tab 2018-01-05T11:29:07,513 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_unique_constraints : db=default tbl=tab 2018-01-05T11:29:07,513 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,513 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,513 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,513 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab 2018-01-05T11:29:07,513 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab 2018-01-05T11:29:07,514 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,514 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,514 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,514 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_not_null_constraints : db=default tbl=tab_part 2018-01-05T11:29:07,514 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_not_null_constraints : db=default tbl=tab_part 2018-01-05T11:29:07,514 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,514 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,514 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,514 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:07,514 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:07,515 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,515 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,515 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,515 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:07,515 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:07,515 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,515 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,515 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,515 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_unique_constraints : db=default tbl=tab_part 2018-01-05T11:29:07,515 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_unique_constraints : db=default tbl=tab_part 2018-01-05T11:29:07,515 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,515 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,515 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,515 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab_part 2018-01-05T11:29:07,515 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab_part 2018-01-05T11:29:07,515 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,515 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,516 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:29:07,516 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,517 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.sql2rel: Plan after trimming unused fields HiveJoin(condition=[=($0, $3)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1], ds=[$2]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0], value=[$1], ds=[$2]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:07,518 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan before removing subquery: HiveProject(key=[$0], value=[$1], ds=[$2], key1=[$6], value1=[$7], ds1=[$8]) HiveJoin(condition=[=($0, $6)], joinType=[inner], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:07,518 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan just after removing subquery: HiveProject(key=[$0], value=[$1], ds=[$2], key1=[$6], value1=[$7], ds1=[$8]) HiveJoin(condition=[=($0, $6)], joinType=[inner], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:07,518 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan after decorrelation: HiveProject(key=[$0], value=[$1], ds=[$2], key1=[$6], value1=[$7], ds1=[$8]) HiveJoin(condition=[=($0, $6)], joinType=[inner], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:07,518 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,518 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,518 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,519 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,519 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,519 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,519 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,519 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,519 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,520 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,520 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,531 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,531 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,531 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,531 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,531 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partitions : db=default tbl=tab 2018-01-05T11:29:07,531 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions : db=default tbl=tab 2018-01-05T11:29:07,533 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,534 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.161194ms + 0.005329ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? ] 2018-01-05T11:29:07,534 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.129729ms + 0.046564ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (15) order by "PART_NAME" asc] 2018-01-05T11:29:07,535 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.079955ms + 0.058066ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (15) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:29:07,535 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.067966ms + 0.01926ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (15) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:07,535 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.061437ms + 0.007139ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (43) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:29:07,535 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.059428ms + 0.020439ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (43) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:07,535 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.058101ms + 0.017097ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (43) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:07,535 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.054389ms + 0.00631ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (43) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:07,535 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.082024ms + 0.021902ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (28) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:07,535 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.066244ms + 0.024602ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (43) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:29:07,535 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,538 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,538 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,538 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,538 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,538 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,538 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partitions : db=default tbl=tab_part 2018-01-05T11:29:07,538 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions : db=default tbl=tab_part 2018-01-05T11:29:07,541 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,541 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.15ms + 0.004553ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? ] 2018-01-05T11:29:07,541 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.141281ms + 0.048331ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (14) order by "PART_NAME" asc] 2018-01-05T11:29:07,542 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.108329ms + 0.08817ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (14) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:29:07,542 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.086021ms + 0.021434ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (14) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:07,542 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.065484ms + 0.007649ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (41) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:29:07,542 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.060841ms + 0.021152ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (41) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:07,542 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.059098ms + 0.01689ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (41) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:07,542 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.053986ms + 0.006292ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (41) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:07,542 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.086436ms + 0.020428ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (26) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:07,542 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.067667ms + 0.019946ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (41) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:29:07,542 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,545 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,545 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,545 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,545 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,545 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,562 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.sql2rel: Plan after trimming unused fields HiveJoin(condition=[=($0, $3)], joinType=[inner], algorithm=[none], cost=[not available]) HiveFilter(condition=[IS NOT NULL($0)]) HiveProject(key=[$0], value=[$1], ds=[$2]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveFilter(condition=[IS NOT NULL($0)]) HiveProject(key=[$0], value=[$1], ds=[$2]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:07,562 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,562 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,562 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,562 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,563 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,563 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,563 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,563 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,564 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,564 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,564 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,564 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,564 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,564 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,564 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Original plan for PlanModifier HiveJoin(condition=[=($0, $3)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1], ds=[$2]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0], value=[$1], ds=[$2]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:07,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after top-level introduceDerivedTable HiveProject(key=[$0], value=[$1], ds=[$2], key0=[$3], value0=[$4], ds0=[$5]) HiveJoin(condition=[=($0, $3)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1], ds=[$2]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0], value=[$1], ds=[$2]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:07,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after nested convertOpTree HiveProject(key=[$0], value=[$1], ds=[$2], key0=[$3], value0=[$4], ds0=[$5]) HiveJoin(condition=[=($0, $3)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1], ds=[$2]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0], value=[$1], ds=[$2]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:07,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after propagating order HiveProject(key=[$0], value=[$1], ds=[$2], key0=[$3], value0=[$4], ds0=[$5]) HiveJoin(condition=[=($0, $3)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1], ds=[$2]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0], value=[$1], ds=[$2]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:07,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after fixTopOBSchema HiveProject(key=[$0], value=[$1], ds=[$2], key0=[$3], value0=[$4], ds0=[$5]) HiveJoin(condition=[=($0, $3)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1], ds=[$2]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0], value=[$1], ds=[$2]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:07,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Final plan after modifier HiveProject(a.key=[$0], a.value=[$1], a.ds=[$2], b.key=[$3], b.value=[$4], b.ds=[$5]) HiveJoin(condition=[=($0, $3)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1], ds=[$2]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0], value=[$1], ds=[$2]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:07,566 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:07,566 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:07,566 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:07,566 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,566 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab 2018-01-05T11:29:07,566 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab 2018-01-05T11:29:07,569 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,569 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,569 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:07,569 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:07,569 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:07,569 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,569 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab_part 2018-01-05T11:29:07,569 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab_part 2018-01-05T11:29:07,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,572 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:07,572 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:07,572 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:07,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Table Plan for a TS[0] 2018-01-05T11:29:07,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Filter Plan for $hdt$_0 row schema: a{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:07,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB a{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB a{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:07,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (. (tok_table_or_col a) key) key) (tok_selexpr (. (tok_table_or_col a) value) value) (tok_selexpr (. (tok_table_or_col a) ds) ds)) 2018-01-05T11:29:07,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = a{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} starRr = null 2018-01-05T11:29:07,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(key,_col0: int)(value,_col1: string)(ds,_col2: string)} 2018-01-05T11:29:07,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:07,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block $hdt$_0 2018-01-05T11:29:07,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block $hdt$_0 2018-01-05T11:29:07,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Table Plan for b TS[3] 2018-01-05T11:29:07,574 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Filter Plan for $hdt$_1 row schema: b{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:07,574 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB b{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB b{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:07,574 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (. (tok_table_or_col b) key) key) (tok_selexpr (. (tok_table_or_col b) value) value) (tok_selexpr (. (tok_table_or_col b) ds) ds)) 2018-01-05T11:29:07,574 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = b{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} starRr = null 2018-01-05T11:29:07,574 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(key,_col0: int)(value,_col1: string)(ds,_col2: string)} 2018-01-05T11:29:07,574 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:07,574 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block $hdt$_1 2018-01-05T11:29:07,574 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block $hdt$_1 2018-01-05T11:29:07,574 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB $hdt$_0{(key,_col0: int)(value,_col1: string)(ds,_col2: string)} $hdt$_1{(key,_col3: int)(value,_col4: string)(ds,_col5: string)} after GB $hdt$_0{(key,_col0: int)(value,_col1: string)(ds,_col2: string)} $hdt$_1{(key,_col3: int)(value,_col4: string)(ds,_col5: string)} 2018-01-05T11:29:07,574 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (. (tok_table_or_col $hdt$_0) key) a.key) (tok_selexpr (. (tok_table_or_col $hdt$_0) value) a.value) (tok_selexpr (. (tok_table_or_col $hdt$_0) ds) a.ds) (tok_selexpr (. (tok_table_or_col $hdt$_1) key) b.key) (tok_selexpr (. (tok_table_or_col $hdt$_1) value) b.value) (tok_selexpr (. (tok_table_or_col $hdt$_1) ds) b.ds)) 2018-01-05T11:29:07,574 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = $hdt$_0{(key,_col0: int)(value,_col1: string)(ds,_col2: string)} $hdt$_1{(key,_col3: int)(value,_col4: string)(ds,_col5: string)} starRr = null 2018-01-05T11:29:07,575 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(a.key,_col0: int)(a.value,_col1: string)(a.ds,_col2: string)(b.key,_col3: int)(b.value,_col4: string)(b.ds,_col5: string)} 2018-01-05T11:29:07,575 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:07,575 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-07_490_5077619032005156997-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/-mr-10001 2018-01-05T11:29:07,575 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-07_490_5077619032005156997-1 2018-01-05T11:29:07,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/-mr-10003 2018-01-05T11:29:07,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/-mr-10001 row schema: null{(a.key,_col0: int)(a.value,_col1: string)(a.ds,_col2: string)(b.key,_col3: int)(b.value,_col4: string)(b.ds,_col5: string)} 2018-01-05T11:29:07,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block null 2018-01-05T11:29:07,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:29:07,598 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: CBO Succeeded; optimized logical plan. 2018-01-05T11:29:07,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Before logical optimization TS[0]-FIL[1]-SEL[2]-RS[6]-JOIN[8]-SEL[9]-FS[10] TS[3]-FIL[4]-SEL[5]-RS[7]-JOIN[8] 2018-01-05T11:29:07,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,599 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] lineage.Generator: Time taken for lineage transform=1 2018-01-05T11:29:07,599 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,599 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,599 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.PartitionColumnsSeparator: Partition columns not separated for null, is not IN operator : 2018-01-05T11:29:07,599 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.PartitionColumnsSeparator: Partition columns not separated for null, is not IN operator : 2018-01-05T11:29:07,599 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,599 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,599 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SyntheticJoinPredicate: Synthetic predicate: 0 --> 1 2018-01-05T11:29:07,599 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SyntheticJoinPredicate: Synthetic predicate: 1 --> 0 2018-01-05T11:29:07,599 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,599 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,599 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FS(10) 2018-01-05T11:29:07,599 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(9) 2018-01-05T11:29:07,599 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for JOIN(8) 2018-01-05T11:29:07,599 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(6) 2018-01-05T11:29:07,599 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(11) 2018-01-05T11:29:07,599 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias $hdt$_0: (_col0) IN (RS[7]) 2018-01-05T11:29:07,599 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(2) 2018-01-05T11:29:07,599 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of SEL for alias $hdt$_0: (key) IN (RS[7]) 2018-01-05T11:29:07,599 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(1) 2018-01-05T11:29:07,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias a: key is not null 2018-01-05T11:29:07,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias a: (key) IN (RS[7]) 2018-01-05T11:29:07,600 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:29:07,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of TS for alias a: key is not null; (key) IN (RS[7]) 2018-01-05T11:29:07,600 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(7) 2018-01-05T11:29:07,600 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(12) 2018-01-05T11:29:07,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias $hdt$_1: (_col0) IN (RS[6]) 2018-01-05T11:29:07,600 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(5) 2018-01-05T11:29:07,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of SEL for alias $hdt$_1: (key) IN (RS[6]) 2018-01-05T11:29:07,600 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(4) 2018-01-05T11:29:07,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias b: key is not null 2018-01-05T11:29:07,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias b: (key) IN (RS[6]) 2018-01-05T11:29:07,600 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for TS(3) 2018-01-05T11:29:07,600 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of TS for alias b: key is not null; (key) IN (RS[6]) 2018-01-05T11:29:07,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SimplePredicatePushDown: After PPD: TS[0]-FIL[13]-SEL[2]-RS[6]-JOIN[8]-SEL[9]-FS[10] TS[3]-FIL[14]-SEL[5]-RS[7]-JOIN[8] 2018-01-05T11:29:07,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pcr.PcrExprProcFactory: Processing key is not null org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNotNull outputs {UNKNOWN, Column[key]}, 2018-01-05T11:29:07,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pcr.PcrExprProcFactory: Processing (key) IN (RS[7]) org.apache.hadoop.hive.ql.udf.generic.GenericUDFIn outputs {UNKNOWN, Column[key]}, {UNKNOWN, RS[7]}, 2018-01-05T11:29:07,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pcr.PcrExprProcFactory: Processing (key is not null and (key) IN (RS[7])) org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPAnd outputs {UNKNOWN, GenericUDFOPNotNull(Column[key])}, {UNKNOWN, GenericUDFIn(Column[key], RS[7])}, 2018-01-05T11:29:07,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pcr.PcrExprProcFactory: Processing key is not null org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNotNull outputs {UNKNOWN, Column[key]}, 2018-01-05T11:29:07,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pcr.PcrExprProcFactory: Processing (key) IN (RS[6]) org.apache.hadoop.hive.ql.udf.generic.GenericUDFIn outputs {UNKNOWN, Column[key]}, {UNKNOWN, RS[6]}, 2018-01-05T11:29:07,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pcr.PcrExprProcFactory: Processing (key is not null and (key) IN (RS[6])) org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPAnd outputs {UNKNOWN, GenericUDFOPNotNull(Column[key])}, {UNKNOWN, GenericUDFIn(Column[key], RS[6])}, 2018-01-05T11:29:07,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,601 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: JOIN 8 oldExprs: {0=[Column[KEY.reducesinkkey0], Column[VALUE._col0], Column[VALUE._col1]], 1=[Column[KEY.reducesinkkey0], Column[VALUE._col0], Column[VALUE._col1]]} 2018-01-05T11:29:07,601 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: JOIN 8 newExprs: {0=[Column[KEY.reducesinkkey0], Column[VALUE._col0], Column[VALUE._col1]], 1=[Column[KEY.reducesinkkey0], Column[VALUE._col0], Column[VALUE._col1]]} 2018-01-05T11:29:07,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 6 key:[Column[_col0]] 2018-01-05T11:29:07,601 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 6 oldColExprMap: {VALUE._col0=Column[_col1], VALUE._col1=Column[_col2], KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:07,601 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 6 newColExprMap: {VALUE._col0=Column[_col1], VALUE._col1=Column[_col2], KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:07,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 7 key:[Column[_col0]] 2018-01-05T11:29:07,601 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 7 oldColExprMap: {VALUE._col0=Column[_col1], VALUE._col1=Column[_col2], KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:07,601 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 7 newColExprMap: {VALUE._col0=Column[_col1], VALUE._col1=Column[_col2], KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:07,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,601 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.IdentityProjectRemover: Identity project remover optimization removed : SEL[9] 2018-01-05T11:29:07,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: After logical optimization TS[0]-FIL[13]-SEL[2]-RS[6]-JOIN[8]-FS[10] TS[3]-FIL[14]-SEL[5]-RS[7]-JOIN[8] 2018-01-05T11:29:07,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Parent: TS[0] 2018-01-05T11:29:07,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Filter: (key is not null and (key) IN (RS[7])) (type: boolean) 2018-01-05T11:29:07,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: TableScan: TS[0] 2018-01-05T11:29:07,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Column key is not a partition column 2018-01-05T11:29:07,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Initiate semijoin reduction for key ((key is not null and (key) IN (RS[7])) 2018-01-05T11:29:07,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:07,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:07,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:07,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:07,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:07,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:07,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: DynamicSemiJoinPushdown: Saving RS to TS mapping: RS[19]: TS[0] 2018-01-05T11:29:07,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Parent: TS[3] 2018-01-05T11:29:07,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Filter: (key is not null and (key) IN (RS[6])) (type: boolean) 2018-01-05T11:29:07,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: TableScan: TS[3] 2018-01-05T11:29:07,603 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Column key is not a partition column 2018-01-05T11:29:07,603 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Initiate semijoin reduction for key ((key is not null and (key) IN (RS[6])) 2018-01-05T11:29:07,603 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:07,603 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:07,603 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:07,603 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:07,603 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:07,603 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:07,603 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: DynamicSemiJoinPushdown: Saving RS to TS mapping: RS[24]: TS[3] 2018-01-05T11:29:07,603 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,603 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,603 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Estimated average row size: 104 2018-01-05T11:29:07,603 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,603 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_aggr_stats_for: db=default table=tab 2018-01-05T11:29:07,603 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_aggr_stats_for: db=default table=tab 2018-01-05T11:29:07,606 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.AggregateStatsCache: Returning aggregate stats from the cache; total hits: 1, total misses: 0, hit ratio: 1.0 2018-01-05T11:29:07,606 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.AggregateStatsCache: No aggregate stats cached for database:default, table:tab, column:value 2018-01-05T11:29:07,607 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.414399ms + 0.012915ms, the query is [select count("COLUMN_NAME") from "PART_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (?,?) and "PARTITION_NAME" in (?) group by "PARTITION_NAME"] 2018-01-05T11:29:07,607 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.118706ms + 0.003111ms, the query is [select "PARTITION_NAME", "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "PART_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (%1$s) AND "PARTITION_NAME" in (%2$s) order by "PARTITION_NAME"] 2018-01-05T11:29:07,607 DEBUG [aggr-col-stats-0] aggr.LongColumnStatsAggregator: doAllPartitionContainStats for value is true 2018-01-05T11:29:07,608 DEBUG [aggr-col-stats-0] aggr.LongColumnStatsAggregator: all of the bit vectors can merge for value is true 2018-01-05T11:29:07,608 DEBUG [aggr-col-stats-0] aggr.LongColumnStatsAggregator: Ndv estimatation for value is 153 # of partitions requested: 1 # of partitions found: 1 2018-01-05T11:29:07,608 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] utils.MetaStoreUtils: Time for aggr col stats in seconds: 0.001 Threads used: 1 2018-01-05T11:29:07,608 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: useDensityFunctionForNDVEstimation = false partsFound = 1 ColumnStatisticsObj = [ColumnStatisticsObj(colName:key, colType:int, statsData:), ColumnStatisticsObj(colName:value, colType:string, statsData:)] 2018-01-05T11:29:07,609 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,610 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,610 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting stats (Num rows: 242 Data size: 67518 Basic stats: COMPLETE Column stats: COMPLETE) on TS[0] 2018-01-05T11:29:07,610 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (tab): numRows: 242 dataSize: 67518 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {value= colName: value colType: string countDistincts: 153 numNulls: 0 avgColLen: 6.801652892561983 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, key= colName: key colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false, ds= colName: ds colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:07,610 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Estimated average row size: 104 2018-01-05T11:29:07,610 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,610 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_aggr_stats_for: db=default table=tab_part 2018-01-05T11:29:07,610 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_aggr_stats_for: db=default table=tab_part 2018-01-05T11:29:07,612 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.AggregateStatsCache: Returning aggregate stats from the cache; total hits: 2, total misses: 0, hit ratio: 1.0 2018-01-05T11:29:07,612 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.AggregateStatsCache: No aggregate stats cached for database:default, table:tab_part, column:value 2018-01-05T11:29:07,612 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.186181ms + 0.003975ms, the query is [select count("COLUMN_NAME") from "PART_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (?,?) and "PARTITION_NAME" in (?) group by "PARTITION_NAME"] 2018-01-05T11:29:07,612 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.096235ms + 0.003041ms, the query is [select "PARTITION_NAME", "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "PART_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (%1$s) AND "PARTITION_NAME" in (%2$s) order by "PARTITION_NAME"] 2018-01-05T11:29:07,613 DEBUG [aggr-col-stats-0] aggr.LongColumnStatsAggregator: doAllPartitionContainStats for value is true 2018-01-05T11:29:07,613 DEBUG [aggr-col-stats-0] aggr.LongColumnStatsAggregator: all of the bit vectors can merge for value is true 2018-01-05T11:29:07,613 DEBUG [aggr-col-stats-0] aggr.LongColumnStatsAggregator: Ndv estimatation for value is 309 # of partitions requested: 1 # of partitions found: 1 2018-01-05T11:29:07,613 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] utils.MetaStoreUtils: Time for aggr col stats in seconds: 0.001 Threads used: 1 2018-01-05T11:29:07,613 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: useDensityFunctionForNDVEstimation = false partsFound = 1 ColumnStatisticsObj = [ColumnStatisticsObj(colName:key, colType:int, statsData:), ColumnStatisticsObj(colName:value, colType:string, statsData:)] 2018-01-05T11:29:07,614 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:07,614 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:07,614 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting stats (Num rows: 500 Data size: 139500 Basic stats: COMPLETE Column stats: COMPLETE) on TS[3] 2018-01-05T11:29:07,614 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-TS[3] (tab_part): numRows: 500 dataSize: 139500 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {value= colName: value colType: string countDistincts: 309 numNulls: 0 avgColLen: 6.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, key= colName: key colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false, ds= colName: ds colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:07,614 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_7_b_key_min), DynamicValue(RS_7_b_key_max)) Original num rows: 242 New num rows: 242 2018-01-05T11:29:07,614 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_7_b_key_min), DynamicValue(RS_7_b_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_7_b_key_bloom_filter))) Original num rows: 242 New num rows: 242 2018-01-05T11:29:07,614 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_7_b_key_min), DynamicValue(RS_7_b_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_7_b_key_bloom_filter)))) Original num rows: 242 New num rows: 242 2018-01-05T11:29:07,614 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-FIL[13]: numRows: 242 dataSize: 67518 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {value= colName: value colType: string countDistincts: 153 numNulls: 0 avgColLen: 6.801652892561983 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, key= colName: key colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false, ds= colName: ds colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:07,614 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting stats (Num rows: 242 Data size: 67518 Basic stats: COMPLETE Column stats: COMPLETE) on FIL[13] 2018-01-05T11:29:07,614 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_6_a_key_min), DynamicValue(RS_6_a_key_max)) Original num rows: 500 New num rows: 500 2018-01-05T11:29:07,614 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_6_a_key_min), DynamicValue(RS_6_a_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_6_a_key_bloom_filter))) Original num rows: 500 New num rows: 500 2018-01-05T11:29:07,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_6_a_key_min), DynamicValue(RS_6_a_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_6_a_key_bloom_filter)))) Original num rows: 500 New num rows: 500 2018-01-05T11:29:07,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-FIL[14]: numRows: 500 dataSize: 139500 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {value= colName: value colType: string countDistincts: 309 numNulls: 0 avgColLen: 6.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, key= colName: key colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false, ds= colName: ds colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:07,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting stats (Num rows: 500 Data size: 139500 Basic stats: COMPLETE Column stats: COMPLETE) on FIL[14] 2018-01-05T11:29:07,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 242 Data size: 67518 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[2] 2018-01-05T11:29:07,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[2]: numRows: 242 dataSize: 67518 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: string countDistincts: 153 numNulls: 0 avgColLen: 6.801652892561983 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:07,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 500 Data size: 139500 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[5] 2018-01-05T11:29:07,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[5]: numRows: 500 dataSize: 139500 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: string countDistincts: 309 numNulls: 0 avgColLen: 6.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:07,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 242 Data size: 67518 Basic stats: COMPLETE Column stats: COMPLETE) on RS[6] 2018-01-05T11:29:07,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[6]: numRows: 242 dataSize: 67518 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {VALUE._col0= colName: VALUE._col0 colType: string countDistincts: 153 numNulls: 0 avgColLen: 6.801652892561983 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col1= colName: VALUE._col1 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, KEY.reducesinkkey0= colName: KEY.reducesinkkey0 colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:07,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 242 Data size: 968 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[20] 2018-01-05T11:29:07,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[20]: numRows: 242 dataSize: 968 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:07,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 500 Data size: 139500 Basic stats: COMPLETE Column stats: COMPLETE) on RS[7] 2018-01-05T11:29:07,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[7]: numRows: 500 dataSize: 139500 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {VALUE._col0= colName: VALUE._col0 colType: string countDistincts: 309 numNulls: 0 avgColLen: 6.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col1= colName: VALUE._col1 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, KEY.reducesinkkey0= colName: KEY.reducesinkkey0 colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:07,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[15] 2018-01-05T11:29:07,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[15]: numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:07,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-JOIN[8]: detects none/multiple PK parents. 2018-01-05T11:29:07,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.JoinOperator: Setting stats (Num rows: 392 Data size: 218736 Basic stats: COMPLETE Column stats: COMPLETE) on JOIN[8] 2018-01-05T11:29:07,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-JOIN[8]: numRows: 392 dataSize: 218736 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: string countDistincts: 153 numNulls: 0 avgColLen: 6.801652892561983 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col4= colName: _col4 colType: string countDistincts: 243 numNulls: 0 avgColLen: 6.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: int countDistincts: 120 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false, _col5= colName: _col5 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:07,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[21]: inputSize: 67518 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:07,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[21] hashAgg: true 2018-01-05T11:29:07,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 3] STATS-GBY[21]: cardinality: 1 2018-01-05T11:29:07,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[21] 2018-01-05T11:29:07,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[21]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:07,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[16]: inputSize: 139500 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:07,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[16] hashAgg: true 2018-01-05T11:29:07,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 3] STATS-GBY[16]: cardinality: 1 2018-01-05T11:29:07,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[16] 2018-01-05T11:29:07,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[16]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:07,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-FS[10]: numRows: 392 dataSize: 218736 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: string countDistincts: 153 numNulls: 0 avgColLen: 6.801652892561983 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col4= colName: _col4 colType: string countDistincts: 243 numNulls: 0 avgColLen: 6.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: int countDistincts: 120 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false, _col5= colName: _col5 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:07,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on RS[22] 2018-01-05T11:29:07,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[22]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:07,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on RS[17] 2018-01-05T11:29:07,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[17]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:07,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[23]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:07,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[23] hashAgg: false 2018-01-05T11:29:07,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 9] STATS-GBY[23]: cardinality: 1 2018-01-05T11:29:07,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[23] 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[23]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[18]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[18] hashAgg: false 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 9] STATS-GBY[18]: cardinality: 1 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[18] 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[18]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on RS[24] 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[24]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on RS[19] 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[19]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[key]]; bucket count: 2 }) on TS[0] 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[key]]; bucket count: 4 }) on TS[3] 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[key]]; bucket count: 2 }) on FIL[13] 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[key]]; bucket count: 4 }) on FIL[14] 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: 2 }) on SEL[2] 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: 4 }) on SEL[5] 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[KEY.reducesinkkey0]]; sort column names: [[KEY.reducesinkkey0]]; bucket count: 2 }) on RS[6] 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: 2 }) on SEL[20] 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[KEY.reducesinkkey0]]; sort column names: [[KEY.reducesinkkey0]]; bucket count: 4 }) on RS[7] 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: 4 }) on SEL[15] 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.JoinOperator: Setting traits ({ bucket column names: [[_col0], [_col3]]; sort column names: [[_col0], [_col3]]; bucket count: -1 }) on JOIN[8] 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[21] 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[16] 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FileSinkOperator: Setting traits ({ bucket column names: [[_col0], [_col3]]; sort column names: [[_col0], [_col3]]; bucket count: -1 }) on FS[10] 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[22] 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[17] 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[23] 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[18] 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[24] 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[19] 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,616 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Set parallelism for reduce sink RS[6] to: 1 2018-01-05T11:29:07,616 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Set parallelism for reduce sink RS[7] to: 1 2018-01-05T11:29:07,616 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConvertJoinMapJoin: Memory monitor info set to : { isLlap: true executorsPerNode: 4 maxExecutorsOverSubscribeMemory: 3 memoryOverSubscriptionFactor: 0.20000000298023224 memoryCheckInterval: 100000 noConditionalTaskSize: 10000000 adjustedNoConditionalTaskSize: 16000000 hashTableInflationFactor: 2.0 threshold: 32000000 } 2018-01-05T11:29:07,616 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConvertJoinMapJoin: Fallback to common merge join operator 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.CommonMergeJoinOperator: Setting traits ({ bucket column names: [[_col0], [_col3]]; sort column names: [[_col0], [_col3]]; bucket count: 0 }) on MERGEJOIN[25] 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.CommonMergeJoinOperator: Setting stats (Num rows: 392 Data size: 218736 Basic stats: COMPLETE Column stats: COMPLETE) on MERGEJOIN[25] 2018-01-05T11:29:07,616 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:07,616 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:07,616 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:07,616 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing BloomFilter cost/benefit for RS[19] - TS[0] (a) (Column[key]) 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing key domain cardinality, keyDomainCardinality=461, semiJoinKeyIsPK=false, selColStat= colName: _col0 colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false, selColSourceStat= colName: key colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false, tsColStat= colName: key colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit for Column[_col0] to Column[key], selKeyCardinality=308, tsKeyCardinality=153, tsRows=242, keyDomainCardinality=461 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: SemiJoin key selectivity=0.6681127982646421, benefit=80.31670281995662 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit=80.31670281995662, cost=500.0, tsDataSize=242, netBenefit=-419.6832971800434 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: netBenefit=-1.7342285007439808 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing BloomFilter cost/benefit for RS[24] - TS[3] (b) (Column[key]) 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing key domain cardinality, keyDomainCardinality=461, semiJoinKeyIsPK=false, selColStat= colName: _col0 colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false, selColSourceStat= colName: key colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false, tsColStat= colName: key colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit for Column[_col0] to Column[key], selKeyCardinality=153, tsKeyCardinality=308, tsRows=500, keyDomainCardinality=461 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: SemiJoin key selectivity=0.3318872017353579, benefit=334.056399132321 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit=334.056399132321, cost=242.0, tsDataSize=500, netBenefit=92.05639913232102 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: netBenefit=0.18411279826464202 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Reduction factor not satisfied for RS[19]-TS[0] (a). Removing semijoin optimization. 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Removing ReduceSink RS[19] and TableScan TS[0] 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Reduction factor not satisfied for RS[24]-TS[3] (b). Removing semijoin optimization. 2018-01-05T11:29:07,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Removing ReduceSink RS[24] and TableScan TS[3] 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: FS, 10 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: MERGEJOIN, 25 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 6 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 2 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: FIL, 13 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 7 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 5 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: FIL, 14 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: TS, 3 2018-01-05T11:29:07,617 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Cycle free: true 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: Before SharedWorkOptimizer: TS[0]-FIL[13]-SEL[2]-RS[6]-MERGEJOIN[25]-FS[10] TS[3]-FIL[14]-SEL[5]-RS[7]-MERGEJOIN[25] 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: DPP information stored in the cache: {} 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: Sorted tables by size: [default.tab_part=139500, default.tab=67518] 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: After SharedWorkOptimizer: TS[0]-FIL[13]-SEL[2]-RS[6]-MERGEJOIN[25]-FS[10] TS[3]-FIL[14]-SEL[5]-RS[7]-MERGEJOIN[25] 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: int|{a}key,value: string|{a}value,ds: string|{a}ds,BLOCK__OFFSET__INSIDE__FILE: bigint|{a}block__offset__inside__file,INPUT__FILE__NAME: string|{a}input__file__name,ROW__ID: struct|{a}row__id) 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(Const boolean true, Const boolean true) -> Const boolean true 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPAnd(Const boolean true, Const boolean true)) -> GenericUDFOPNotNull(Column[key]) 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[3] with rs:(key: int|{b}key,value: string|{b}value,ds: string|{b}ds,BLOCK__OFFSET__INSIDE__FILE: bigint|{b}block__offset__inside__file,INPUT__FILE__NAME: string|{b}input__file__name,ROW__ID: struct|{b}row__id) 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(Const boolean true, Const boolean true) -> Const boolean true 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPAnd(Const boolean true, Const boolean true)) -> GenericUDFOPNotNull(Column[key]) 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FIL[13] with rs:(key: int|{a}key,value: string|{a}value,ds: string|{a}ds) 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FIL[13] 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Old filter FIL[13] conditions:(key is not null and (true and true)) 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(Const boolean true, Const boolean true) -> Const boolean true 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPAnd(Const boolean true, Const boolean true)) -> GenericUDFOPNotNull(Column[key]) 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New filter FIL[13] conditions:key is not null 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FIL[14] with rs:(key: int|{b}key,value: string|{b}value,ds: string|{b}ds) 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_3 {} 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FIL[14] 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Old filter FIL[14] conditions:(key is not null and (true and true)) 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(Const boolean true, Const boolean true) -> Const boolean true 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPAnd(Const boolean true, Const boolean true)) -> GenericUDFOPNotNull(Column[key]) 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New filter FIL[14] conditions:key is not null 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[2] with rs:(_col0: int|{$hdt$_0}key,_col1: string|{$hdt$_0}value,_col2: string|{$hdt$_0}ds) 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op FIL_13 {} 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[2] 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value] Column[ds]) 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: int|{$hdt$_1}key,_col1: string|{$hdt$_1}value,_col2: string|{$hdt$_1}ds) 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op FIL_14 {} 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value] Column[ds]) 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[6] with rs:(KEY.reducesinkkey0: int|{$hdt$_0}key,VALUE._col0: string|{$hdt$_0}value,VALUE._col1: string|{$hdt$_0}ds) 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_2 {} 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[6] 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[7] with rs:(KEY.reducesinkkey0: int|{$hdt$_1}key,VALUE._col0: string|{$hdt$_1}value,VALUE._col1: string|{$hdt$_1}ds) 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[7] 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:MERGEJOIN[25] with rs:(_col0: int|{$hdt$_0}key,_col1: string|{$hdt$_0}value,_col2: string|{$hdt$_0}ds,_col3: int|{$hdt$_1}key,_col4: string|{$hdt$_1}value,_col5: string|{$hdt$_1}ds) 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_6 {} 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_7 {} 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator MERGEJOIN[25] 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[10] with rs:(_col0: int|{},_col1: string|{},_col2: string|{},_col3: int|{},_col4: string|{},_col5: string|{}) 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op MERGEJOIN_25 {} 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[10] 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: TS[0] 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[6] 2018-01-05T11:29:07,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:29:07,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 of table a 2018-01-05T11:29:07,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:07,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[6] 2018-01-05T11:29:07,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: MERGEJOIN[25] 2018-01-05T11:29:07,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: FS[10] 2018-01-05T11:29:07,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding reduce work (Reducer 2) for MERGEJOIN[25] 2018-01-05T11:29:07,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Setting up reduce sink: RS[6] with following reduce work: Reducer 2 2018-01-05T11:29:07,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[6] as parent from MERGEJOIN[25] 2018-01-05T11:29:07,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[7] as parent from MERGEJOIN[25] 2018-01-05T11:29:07,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: FS[10] 2018-01-05T11:29:07,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: TS[3] 2018-01-05T11:29:07,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[7] 2018-01-05T11:29:07,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding map work (Map 3) for TS[3] 2018-01-05T11:29:07,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 of table b 2018-01-05T11:29:07,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:07,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Second pass. Leaf operator: RS[7] has common downstream work: org.apache.hadoop.hive.ql.plan.MergeJoinWork@6abf2254 2018-01-05T11:29:07,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:29:07,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:29:07,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:07,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:07,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 3 2018-01-05T11:29:07,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:07,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:07,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:29:07,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:07,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:07,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 3 2018-01-05T11:29:07,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:07,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:07,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:29:07,619 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Examining input format to see if vectorization is enabled. 2018-01-05T11:29:07,619 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorization is enabled for input format(s) [org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] 2018-01-05T11:29:07,619 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing MapWork... 2018-01-05T11:29:07,619 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFOPNotNull(Column[key]), Vectorized Expression = SelectColumnIsNotNull(col 0:int) 2018-01-05T11:29:07,619 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator 2018-01-05T11:29:07,619 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.FilterDesc 2018-01-05T11:29:07,619 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[key], Vectorized Expression = col 0:int 2018-01-05T11:29:07,619 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[value], Vectorized Expression = col 1:string 2018-01-05T11:29:07,619 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[ds], Vectorized Expression = col 2:string 2018-01-05T11:29:07,619 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator 2018-01-05T11:29:07,619 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.SelectDesc 2018-01-05T11:29:07,619 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:int 2018-01-05T11:29:07,619 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col1], Vectorized Expression = col 1:string 2018-01-05T11:29:07,619 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col2], Vectorized Expression = col 2:string 2018-01-05T11:29:07,619 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator groupby typeName int 2018-01-05T11:29:07,619 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkLongOperator 2018-01-05T11:29:07,619 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:07,619 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkLongOperator 2018-01-05T11:29:07,619 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:07,619 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [key, value, ds, ROW__ID] 2018-01-05T11:29:07,619 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [int, string, string, struct] 2018-01-05T11:29:07,619 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE, NONE, NONE, NONE] 2018-01-05T11:29:07,619 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 1 2018-01-05T11:29:07,619 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 1 2018-01-05T11:29:07,619 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:07,619 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:07,619 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Examining input format to see if vectorization is enabled. 2018-01-05T11:29:07,619 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorization is enabled for input format(s) [org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] 2018-01-05T11:29:07,619 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing MapWork... 2018-01-05T11:29:07,619 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFOPNotNull(Column[key]), Vectorized Expression = SelectColumnIsNotNull(col 0:int) 2018-01-05T11:29:07,619 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator 2018-01-05T11:29:07,619 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.FilterDesc 2018-01-05T11:29:07,619 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[key], Vectorized Expression = col 0:int 2018-01-05T11:29:07,619 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[value], Vectorized Expression = col 1:string 2018-01-05T11:29:07,619 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[ds], Vectorized Expression = col 2:string 2018-01-05T11:29:07,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator 2018-01-05T11:29:07,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.SelectDesc 2018-01-05T11:29:07,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:int 2018-01-05T11:29:07,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col1], Vectorized Expression = col 1:string 2018-01-05T11:29:07,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col2], Vectorized Expression = col 2:string 2018-01-05T11:29:07,620 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator groupby typeName int 2018-01-05T11:29:07,620 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkLongOperator 2018-01-05T11:29:07,620 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:07,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkLongOperator 2018-01-05T11:29:07,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:07,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [key, value, ds, ROW__ID] 2018-01-05T11:29:07,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [int, string, string, struct] 2018-01-05T11:29:07,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE, NONE, NONE, NONE] 2018-01-05T11:29:07,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 1 2018-01-05T11:29:07,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 1 2018-01-05T11:29:07,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:07,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:07,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:29:07,620 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: llap mode: only 2018-01-05T11:29:07,620 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:29:07,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key' 2018-01-05T11:29:07,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'value' 2018-01-05T11:29:07,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'ds' 2018-01-05T11:29:07,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key is not null' 2018-01-05T11:29:07,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key is not null' 2018-01-05T11:29:07,620 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:07,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@344d05f1 can be done in LLAP 2018-01-05T11:29:07,620 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Map 3 2018-01-05T11:29:07,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key' 2018-01-05T11:29:07,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'value' 2018-01-05T11:29:07,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'ds' 2018-01-05T11:29:07,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key is not null' 2018-01-05T11:29:07,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key is not null' 2018-01-05T11:29:07,620 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:07,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@3637d146 can be done in LLAP 2018-01-05T11:29:07,620 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Reducer 2 2018-01-05T11:29:07,620 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:07,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MergeJoinWork@6abf2254 can be done in LLAP 2018-01-05T11:29:07,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.SerializeFilter: Serializing: key is not null 2018-01-05T11:29:07,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.SerializeFilter: Serializing: key is not null 2018-01-05T11:29:07,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,620 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed plan generation 2018-01-05T11:29:07,620 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:07,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: validation start 2018-01-05T11:29:07,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:29:07,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,620 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:a.key, type:int, comment:null), FieldSchema(name:a.value, type:string, comment:null), FieldSchema(name:a.ds, type:string, comment:null), FieldSchema(name:b.key, type:int, comment:null), FieldSchema(name:b.value, type:string, comment:null), FieldSchema(name:b.ds, type:string, comment:null)], properties:null) 2018-01-05T11:29:07,621 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: Initializing operator LIST_SINK[32] 2018-01-05T11:29:07,621 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: Initialization Done 32 LIST_SINK 2018-01-05T11:29:07,621 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: Operator 32 LIST_SINK initialized 2018-01-05T11:29:07,621 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: Initialization Done 32 LIST_SINK done is reset. 2018-01-05T11:29:07,621 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,621 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:07,621 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {getNotNullConstraints_(NotNullConstraintsRequest, )=0, isCompatibleWith_(Configuration, )=0, getAggrColStatsFor_(String, String, List, List, )=11, getTable_(String, String, )=12, flushCache_()=0, getUniqueConstraints_(UniqueConstraintsRequest, )=0, getPrimaryKeys_(PrimaryKeysRequest, )=2, listPartitions_(String, String, short, )=14, getForeignKeys_(ForeignKeysRequest, )=1} 2018-01-05T11:29:07,621 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949); Time taken: 0.131 seconds 2018-01-05T11:29:07,621 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:07,621 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,621 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949): select * from tab a join tab_part b on a.key = b.key 2018-01-05T11:29:07,622 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,622 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: select * from tab a join tab_part b on a.key = b.key 2018-01-05T11:29:07,622 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:29:07,622 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@tab 2018-01-05T11:29:07,622 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@tab@ds=2008-04-08 2018-01-05T11:29:07,622 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@tab_part 2018-01-05T11:29:07,622 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@tab_part@ds=2008-04-08 2018-01-05T11:29:07,622 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/-mr-10001 2018-01-05T11:29:07,622 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,622 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,622 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,622 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Query ID = djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949 2018-01-05T11:29:07,622 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Total jobs = 1 2018-01-05T11:29:07,622 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,623 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:29:07,623 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-1:MAPRED] in serial mode 2018-01-05T11:29:07,633 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:29:07,656 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:29:07,656 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:29:07,656 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:29:07,656 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949 2018-01-05T11:29:07,656 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Session is already open 2018-01-05T11:29:07,656 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:29:07,656 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,657 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Dag name: select * from tab a join tab_part b ...b.key (Stage-1) 2018-01-05T11:29:07,657 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: DagInfo: {"context":"Hive","description":"\n\nselect * from tab a join tab_part b on a.key = b.key"} 2018-01-05T11:29:07,657 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949 with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:29:07,657 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,670 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,670 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:29:07,671 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,672 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 2 size: 2.00KB 2018-01-05T11:29:07,689 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,689 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,701 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:29:07,701 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,701 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:29:07,702 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,703 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: Serialized plan (via RPC) - name: Map 3 size: 3.97KB 2018-01-05T11:29:07,708 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:07,708 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,731 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,744 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:29:07,744 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,744 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:29:07,745 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,746 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 3.96KB 2018-01-05T11:29:07,751 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:07,751 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,753 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,753 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,753 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=select * from tab a join tab_part b ...b.key (Stage-1), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949 } 2018-01-05T11:29:07,753 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:07,754 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:07,754 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:07,756 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: select * from tab a join tab_part b ...b.key (Stage-1) 2018-01-05T11:29:07,756 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:29:07,756 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_37, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_37.dot 2018-01-05T11:29:07,758 INFO [IPC Server handler 1 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_37 2018-01-05T11:29:07,758 WARN [IPC Server handler 1 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:29:07,758 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Running DAG: select * from tab a join tab_part b ...b.key (Stage-1), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949 } 2018-01-05T11:29:07,782 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_37][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_37, submitTime=1515180547756, queueName=null 2018-01-05T11:29:07,783 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_37_00 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_37_00 [Map 1], Default Resources= 2018-01-05T11:29:07,783 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_37_00 [Map 1] 2018-01-05T11:29:07,783 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:29:07,783 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_37_00 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:07,783 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_37_01 [Map 3]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_37_01 [Map 3], Default Resources= 2018-01-05T11:29:07,783 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_37_01 [Map 3] 2018-01-05T11:29:07,784 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 3 2018-01-05T11:29:07,784 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_37_01 [Map 3] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:07,784 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_37_02 [Reducer 2]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_37_02 [Reducer 2], Default Resources= 2018-01-05T11:29:07,784 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_37_02 [Reducer 2] 2018-01-05T11:29:07,784 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 2 2018-01-05T11:29:07,784 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_37_02 [Reducer 2] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:07,784 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:29:07,784 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_37][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_37, initTime=1515180547784 2018-01-05T11:29:07,784 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: dag_1515180518813_0001_37 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:29:07,785 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:29:07,785 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_37][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_37, startTime=1515180547784 2018-01-05T11:29:07,785 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_37 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:29:07,785 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 3 : {b={InputName=b}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:29:07,785 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: b, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:29:07,785 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_37_01 [Map 3] 2018-01-05T11:29:07,785 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_37, dagName=select * from tab a join tab_part b ...b.key (Stage-1) 2018-01-05T11:29:07,785 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,785 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,785 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:07,789 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_37_01 [Map 3] 2018-01-05T11:29:07,789 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_37_01 [Map 3] 2018-01-05T11:29:07,789 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:07,789 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_37_01 [Map 3] 2018-01-05T11:29:07,791 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:29:07,791 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@1649782e 2018-01-05T11:29:07,791 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/698bf3ff-2334-401f-bbb9-449cc735a1c6/map.xml 2018-01-05T11:29:07,791 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/698bf3ff-2334-401f-bbb9-449cc735a1c6/map.xml 2018-01-05T11:29:07,791 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/698bf3ff-2334-401f-bbb9-449cc735a1c6/map.xml 2018-01-05T11:29:07,791 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:07,791 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:07,791 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,793 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:07,793 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 3 size: 3.97KB 2018-01-05T11:29:07,793 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_37_01 [Map 3] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:29:07,793 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {a={InputName=a}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:29:07,793 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: a, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:29:07,793 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_37_00 [Map 1] 2018-01-05T11:29:07,793 INFO [InputInitializer {Map 3} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: b on vertex vertex_1515180518813_0001_37_01 [Map 3] 2018-01-05T11:29:07,793 INFO [InputInitializer {Map 3} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:29:07,793 DEBUG [InputInitializer {Map 3} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:07,793 INFO [InputInitializer {Map 3} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:29:07,793 DEBUG [InputInitializer {Map 3} #0] log.PerfLogger: 2018-01-05T11:29:07,793 INFO [InputInitializer {Map 3} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/698bf3ff-2334-401f-bbb9-449cc735a1c6/map.xml 2018-01-05T11:29:07,793 DEBUG [InputInitializer {Map 3} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:07,793 INFO [InputInitializer {Map 3} #0] exec.Utilities: Processing alias b 2018-01-05T11:29:07,793 DEBUG [InputInitializer {Map 3} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:07,793 INFO [InputInitializer {Map 3} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:07,794 DEBUG [InputInitializer {Map 3} #0] io.HiveInputFormat: Pushdown initiated with filterText = key is not null, filterExpr = GenericUDFOPNotNull(Column[key]), serializedFilterExpr = AQEAamF2YS51dGlsLkFycmF5TGlz9AEBAQFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVDb2x1bW5EZXPjAQFrZfkAAAGCYgECb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5zZXJkZTIudHlwZWluZm8uUHJpbWl0aXZlVHlwZUluZu8BAWlu9AEDb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGT1BOb3ROdWzsAQEEb3JnLmFwYWNoZS5oYWRvb3AuaW8uQm9vbGVhbldyaXRhYmzlAQAAAQIBAWJvb2xlYe4= (desc) 2018-01-05T11:29:07,794 INFO [InputInitializer {Map 3} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 0,1 2018-01-05T11:29:07,794 INFO [InputInitializer {Map 3} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = key,value 2018-01-05T11:29:07,794 INFO [InputInitializer {Map 3} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:07,794 DEBUG [InputInitializer {Map 3} #0] io.HiveInputFormat: Pushdown initiated with filterText = key is not null, filterExpr = GenericUDFOPNotNull(Column[key]), serializedFilterExpr = AQEAamF2YS51dGlsLkFycmF5TGlz9AEBAQFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVDb2x1bW5EZXPjAQFrZfkAAAGCYgECb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5zZXJkZTIudHlwZWluZm8uUHJpbWl0aXZlVHlwZUluZu8BAWlu9AEDb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGT1BOb3ROdWzsAQEEb3JnLmFwYWNoZS5oYWRvb3AuaW8uQm9vbGVhbldyaXRhYmzlAQAAAQIBAWJvb2xlYe4= (desc) 2018-01-05T11:29:07,794 DEBUG [InputInitializer {Map 3} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:07,794 DEBUG [InputInitializer {Map 3} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:07,794 DEBUG [InputInitializer {Map 3} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:07,794 DEBUG [InputInitializer {Map 3} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:07,794 DEBUG [InputInitializer {Map 3} #0] orc.OrcInputFormat: getSplits started 2018-01-05T11:29:07,794 DEBUG [InputInitializer {Map 3} #0] orc.OrcInputFormat: Number of buckets specified by conf file is 4 2018-01-05T11:29:07,794 INFO [InputInitializer {Map 3} #0] orc.OrcInputFormat: ORC pushdown predicate: leaf-0 = (IS_NULL key), expr = (not leaf-0) 2018-01-05T11:29:07,795 INFO [InputInitializer {Map 3} #0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:07,795 DEBUG [InputInitializer {Map 3} #0] orc.OrcInputFormat: Generate splits schema evolution property true reader schema struct transactional scan property false 2018-01-05T11:29:07,795 ERROR [ORC_GET_SPLITS #6] io.AcidUtils: Failed to get files with ID; using regular API: Only supported for DFS; got class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:07,797 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_37_00 [Map 1] 2018-01-05T11:29:07,797 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_37_00 [Map 1] 2018-01-05T11:29:07,797 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:07,797 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_37_00 [Map 1] 2018-01-05T11:29:07,799 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:29:07,799 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@11e9d713 2018-01-05T11:29:07,799 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/04343b50-c7d4-4ff5-9f68-f7ee98dcd09d/map.xml 2018-01-05T11:29:07,799 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/04343b50-c7d4-4ff5-9f68-f7ee98dcd09d/map.xml 2018-01-05T11:29:07,799 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/04343b50-c7d4-4ff5-9f68-f7ee98dcd09d/map.xml 2018-01-05T11:29:07,799 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:07,799 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:07,799 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,800 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:07,800 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.96KB 2018-01-05T11:29:07,800 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_37_00 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:29:07,800 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: a on vertex vertex_1515180518813_0001_37_00 [Map 1] 2018-01-05T11:29:07,801 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to ShuffleVertexManager for vertex_1515180518813_0001_37_02 [Reducer 2] 2018-01-05T11:29:07,801 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:29:07,801 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:07,801 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:29:07,801 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:07,801 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/04343b50-c7d4-4ff5-9f68-f7ee98dcd09d/map.xml 2018-01-05T11:29:07,801 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:07,801 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias a 2018-01-05T11:29:07,801 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:07,801 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:07,802 DEBUG [InputInitializer {Map 1} #0] io.HiveInputFormat: Pushdown initiated with filterText = key is not null, filterExpr = GenericUDFOPNotNull(Column[key]), serializedFilterExpr = AQEAamF2YS51dGlsLkFycmF5TGlz9AEBAQFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVDb2x1bW5EZXPjAQFrZfkAAAGCYQECb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5zZXJkZTIudHlwZWluZm8uUHJpbWl0aXZlVHlwZUluZu8BAWlu9AEDb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGT1BOb3ROdWzsAQEEb3JnLmFwYWNoZS5oYWRvb3AuaW8uQm9vbGVhbldyaXRhYmzlAQAAAQIBAWJvb2xlYe4= (desc) 2018-01-05T11:29:07,802 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 0,1 2018-01-05T11:29:07,802 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = key,value 2018-01-05T11:29:07,802 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:07,802 DEBUG [InputInitializer {Map 1} #0] io.HiveInputFormat: Pushdown initiated with filterText = key is not null, filterExpr = GenericUDFOPNotNull(Column[key]), serializedFilterExpr = AQEAamF2YS51dGlsLkFycmF5TGlz9AEBAQFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVDb2x1bW5EZXPjAQFrZfkAAAGCYQECb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5zZXJkZTIudHlwZWluZm8uUHJpbWl0aXZlVHlwZUluZu8BAWlu9AEDb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGT1BOb3ROdWzsAQEEb3JnLmFwYWNoZS5oYWRvb3AuaW8uQm9vbGVhbldyaXRhYmzlAQAAAQIBAWJvb2xlYe4= (desc) 2018-01-05T11:29:07,802 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: getSplits started 2018-01-05T11:29:07,802 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: Number of buckets specified by conf file is 2 2018-01-05T11:29:07,802 INFO [InputInitializer {Map 1} #0] orc.OrcInputFormat: ORC pushdown predicate: leaf-0 = (IS_NULL key), expr = (not leaf-0) 2018-01-05T11:29:07,802 INFO [InputInitializer {Map 1} #0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:07,803 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: Generate splits schema evolution property true reader schema struct transactional scan property false 2018-01-05T11:29:07,803 ERROR [ORC_GET_SPLITS #7] io.AcidUtils: Failed to get files with ID; using regular API: Only supported for DFS; got class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:07,806 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.05 maxFrac: 0.75 auto: false desiredTaskIput: 104857600 2018-01-05T11:29:07,806 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:29:07,806 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 1 tasks for vertex: vertex_1515180518813_0001_37_02 [Reducer 2] 2018-01-05T11:29:07,806 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_37_02 [Reducer 2] 2018-01-05T11:29:07,806 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_37][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_37_02, reconfigureDoneTime=1515180547806, numTasks=1, vertexLocationHint=null, edgeManagersCount=2, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:29:07,806 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_37_02 [Reducer 2], numAdditionalOutputs=1 2018-01-05T11:29:07,806 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_37][Event:VERTEX_INITIALIZED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_37_02, initRequestedTime=1515180547800, initedTime=1515180547806, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:07,806 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_37_02 [Reducer 2] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:29:07,826 DEBUG [AMReporterQueueDrainer] impl.AMReporter: Removing am 10.22.8.182:49824 with last associated dag QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=21} from heartbeat with taskCount=0, amFailed=false, isDone=true 2018-01-05T11:29:07,829 DEBUG [ORC_GET_SPLITS #7] io.AcidUtils: in directory file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 base = null deltas = 0 2018-01-05T11:29:07,829 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: Split strategy: ETLSplitStrategy strategy for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:07,830 DEBUG [ORC_GET_SPLITS #8] orc.LocalCache: Serialized tail not cached for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0 2018-01-05T11:29:07,830 DEBUG [ORC_GET_SPLITS #8] orc.LocalCache: Serialized tail not cached for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0 2018-01-05T11:29:07,830 DEBUG [ORC_GET_SPLITS #8] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,830 DEBUG [ORC_GET_SPLITS #9] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,831 DEBUG [ORC_GET_SPLITS #9] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,831 DEBUG [ORC_GET_SPLITS #8] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,831 DEBUG [ORC_GET_SPLITS #9] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,831 DEBUG [ORC_GET_SPLITS #8] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,831 DEBUG [ORC_GET_SPLITS #9] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:29:07,831 DEBUG [ORC_GET_SPLITS #8] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:29:07,832 INFO [InputInitializer {Map 1} #0] orc.OrcInputFormat: FooterCacheHitRatio: 0/2 2018-01-05T11:29:07,832 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0, start=3, length=566, isOriginal=true, fileLength=744, hasFooter=false, hasBase=true, deltas=0] projected_columns_uncompressed_size: 8724 2018-01-05T11:29:07,832 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0, start=3, length=583, isOriginal=true, fileLength=767, hasFooter=false, hasBase=true, deltas=0] projected_columns_uncompressed_size: 8573 2018-01-05T11:29:07,832 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: getSplits finished 2018-01-05T11:29:07,832 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 2 2018-01-05T11:29:07,833 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:07,833 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 2. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:29:07,833 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/04343b50-c7d4-4ff5-9f68-f7ee98dcd09d/map.xml 2018-01-05T11:29:07,833 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/04343b50-c7d4-4ff5-9f68-f7ee98dcd09d/map.xml 2018-01-05T11:29:07,833 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/04343b50-c7d4-4ff5-9f68-f7ee98dcd09d/map.xml 2018-01-05T11:29:07,833 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:07,833 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:07,833 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,833 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:07,833 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.96KB 2018-01-05T11:29:07,833 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0 to src new group? true 2018-01-05T11:29:07,833 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0 to src new group? false 2018-01-05T11:29:07,833 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:29:07,833 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:29:07,833 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:29:07,834 DEBUG [InputInitializer {Map 1} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 8573 2018-01-05T11:29:07,834 DEBUG [InputInitializer {Map 1} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 8724 2018-01-05T11:29:07,834 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 320 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 17297 Original splits: 2 2018-01-05T11:29:07,834 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired numSplits: 1 lengthPerGroup: 17297 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 17297 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false 2018-01-05T11:29:07,834 DEBUG [InputInitializer {Map 1} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 8573 2018-01-05T11:29:07,834 DEBUG [InputInitializer {Map 1} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 8724 2018-01-05T11:29:07,834 DEBUG [InputInitializer {Map 1} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 8724 2018-01-05T11:29:07,834 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Number of splits desired: 1 created: 1 splitsProcessed: 2 2018-01-05T11:29:07,834 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 2 grouped split count is 1, for bucket: 1 2018-01-05T11:29:07,834 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:29:07,834 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: a on vertex vertex_1515180518813_0001_37_00 [Map 1] 2018-01-05T11:29:07,834 INFO [App Shared Pool - #1] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_37_00 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:07,834 INFO [App Shared Pool - #1] impl.VertexImpl: Got updated RootInputsSpecs: {a=forAllWorkUnits=true, update=[1]} 2018-01-05T11:29:07,835 INFO [App Shared Pool - #1] impl.VertexImpl: Vertex vertex_1515180518813_0001_37_00 [Map 1] parallelism set to 1 2018-01-05T11:29:07,835 INFO [App Shared Pool - #1] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_37_00 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:07,835 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_37][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_37_00, initRequestedTime=1515180547793, initedTime=1515180547835, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:07,835 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_37_00 [Map 1] with distanceFromRoot: 0 2018-01-05T11:29:07,835 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_37_00 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:29:07,835 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_37][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_37_00, startRequestedTime=1515180547800, startedTime=1515180547835 2018-01-05T11:29:07,835 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_37][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_37_00, reconfigureDoneTime=1515180547835, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:29:07,835 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_37_00 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:07,835 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_37_00 for vertex: vertex_1515180518813_0001_37_02 [Reducer 2] numStartedSources: 1 numSources: 2 2018-01-05T11:29:07,835 INFO [Dispatcher thread {Central}] impl.VertexImpl: Cannot start vertex: vertex_1515180518813_0001_37_02 [Reducer 2] numStartedSources: 1 numSources: 2 2018-01-05T11:29:07,835 INFO [App Shared Pool - #1] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:29:07,835 INFO [App Shared Pool - #1] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:29:07,835 INFO [App Shared Pool - #1] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_37_00 [Map 1] 2018-01-05T11:29:07,835 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_37][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_37_00_000000, scheduledTime=1515180547835, launchTime=1515180547835 2018-01-05T11:29:07,835 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_37_00 [Map 1] 2018-01-05T11:29:07,835 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_37_00_000000_0, priority=8, capability=, hosts=null 2018-01-05T11:29:07,835 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=8,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:07,835 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=8,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:07,835 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:07,835 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_37_00_000000_0 on hosts=null 2018-01-05T11:29:07,835 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_37_00_000000_0 on hosts=null 2018-01-05T11:29:07,835 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_37_00_000000_0 2018-01-05T11:29:07,835 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:07,835 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #87, task=TaskInfo{task=attempt_1515180518813_0001_37_00_000000_0, priority=8, startTime=0, containerId=null, uniqueId=86, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000087 2018-01-05T11:29:07,835 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_37_00_000000_0; false 2018-01-05T11:29:07,835 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_37_00_000000_0, priority=8, startTime=311568345, containerId=container_111101111_0001_00_000087assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=86, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:07,836 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000087 for node: 10.22.8.182:49818 2018-01-05T11:29:07,838 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 37, name=select * from tab a join tab_part b ...b.key (Stage-1), queryId=djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949 2018-01-05T11:29:07,838 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000087, attempt_1515180518813_0001_37_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:07,838 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000087 for node: 10.22.8.182:49818 2018-01-05T11:29:07,838 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000087 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:07,838 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000087, launchTime=1515180547838 2018-01-05T11:29:07,839 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_37_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000087, dagName=select * from tab a join tab_part b ...b.key (Stage-1), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{a,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{Reducer 2,org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput,1}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180547784, firstAttemptStartTime=1515180547835, currentAttemptStartTime=1515180547838} 2018-01-05T11:29:07,839 INFO [IPC Server handler 4 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000087, QueryId=djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949-37, DagId=37, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180547839 2018-01-05T11:29:07,839 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_37_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000087] on NM: [10.22.8.182:49818] 2018-01-05T11:29:07,839 DEBUG [IPC Server handler 4 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=37} with the ShuffleHandler 2018-01-05T11:29:07,839 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:07,839 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:07,839 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_37][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_37_00_000000_0, startTime=1515180547838, containerId=container_111101111_0001_00_000087, nodeId=10.22.8.182:49818 2018-01-05T11:29:07,849 INFO [IPC Server handler 4 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:29:07,850 DEBUG [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/37] 2018-01-05T11:29:07,850 DEBUG [ORC_GET_SPLITS #6] io.AcidUtils: in directory file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 base = null deltas = 0 2018-01-05T11:29:07,850 DEBUG [InputInitializer {Map 3} #0] orc.OrcInputFormat: Split strategy: ETLSplitStrategy strategy for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:07,850 DEBUG [ORC_GET_SPLITS #1] orc.LocalCache: Serialized tail not cached for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0 2018-01-05T11:29:07,850 DEBUG [ORC_GET_SPLITS #1] orc.LocalCache: Serialized tail not cached for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0 2018-01-05T11:29:07,850 DEBUG [ORC_GET_SPLITS #1] orc.LocalCache: Serialized tail not cached for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0 2018-01-05T11:29:07,850 DEBUG [ORC_GET_SPLITS #1] orc.LocalCache: Serialized tail not cached for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0 2018-01-05T11:29:07,850 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:29:07,850 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_37_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:07,850 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_37_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180547835, dagStartTime=1515180547784, withinDagPriority=8, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:29:07,850 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_37_00_000000_0 : ACCEPTED 2018-01-05T11:29:07,850 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_37_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:29:07,850 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_37_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180547835, dagStartTime=1515180547784, withinDagPriority=8, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:29:07,850 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_37_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:07,850 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_37_00_000000_0 to pre-emption queue 2018-01-05T11:29:07,850 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_37_00_000000_0: true 2018-01-05T11:29:07,850 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: taskOwner hashCode:1599253873 2018-01-05T11:29:07,850 INFO [TaskCommunicator # 9] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_37_00_000000_0 2018-01-05T11:29:07,850 INFO [TaskCommunicator # 9] GuaranteedTasks: Received response for attempt_1515180518813_0001_37_00_000000_0, true 2018-01-05T11:29:07,851 INFO [Task-Executor-2] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : select * from tab a join tab_part b ...b.key (Stage-1), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_37_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=a, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=Reducer 2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ] 2018-01-05T11:29:07,851 INFO [Task-Executor-2] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1021837312, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:07,851 INFO [Task-Executor-2] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_37_00_000000_0 vertexName: Map 1 2018-01-05T11:29:07,851 DEBUG [ORC_GET_SPLITS #2] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,851 DEBUG [ORC_GET_SPLITS #1] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,851 DEBUG [ORC_GET_SPLITS #4] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,851 DEBUG [ORC_GET_SPLITS #3] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,851 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000087, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_37_00_000000_0, eventCount=1 } 2018-01-05T11:29:07,851 DEBUG [ORC_GET_SPLITS #2] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,851 DEBUG [ORC_GET_SPLITS #1] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,851 DEBUG [ORC_GET_SPLITS #3] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,851 DEBUG [TezTR-518813_1_37_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_37 queryId: djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949 instanceType: TaskRunner2Callable 2018-01-05T11:29:07,851 DEBUG [ORC_GET_SPLITS #4] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,852 DEBUG [ORC_GET_SPLITS #2] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,852 INFO [TezTR-518813_1_37_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_37_00_000000_0 2018-01-05T11:29:07,852 DEBUG [ORC_GET_SPLITS #1] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,852 DEBUG [ORC_GET_SPLITS #3] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,852 DEBUG [ORC_GET_SPLITS #2] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:29:07,852 DEBUG [ORC_GET_SPLITS #4] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,852 DEBUG [ORC_GET_SPLITS #1] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:29:07,852 DEBUG [ORC_GET_SPLITS #3] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:29:07,852 DEBUG [TezTR-518813_1_37_0_0_0] log.PerfLogger: 2018-01-05T11:29:07,852 DEBUG [ORC_GET_SPLITS #4] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:29:07,852 INFO [InputInitializer {Map 3} #0] orc.OrcInputFormat: FooterCacheHitRatio: 0/4 2018-01-05T11:29:07,852 DEBUG [InputInitializer {Map 3} #0] orc.OrcInputFormat: OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0, start=3, length=574, isOriginal=true, fileLength=752, hasFooter=false, hasBase=true, deltas=0] projected_columns_uncompressed_size: 8538 2018-01-05T11:29:07,852 DEBUG [InputInitializer {Map 3} #0] orc.OrcInputFormat: OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0, start=3, length=641, isOriginal=true, fileLength=827, hasFooter=false, hasBase=true, deltas=0] projected_columns_uncompressed_size: 10054 2018-01-05T11:29:07,852 DEBUG [InputInitializer {Map 3} #0] orc.OrcInputFormat: OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0, start=3, length=641, isOriginal=true, fileLength=823, hasFooter=false, hasBase=true, deltas=0] projected_columns_uncompressed_size: 9371 2018-01-05T11:29:07,852 DEBUG [InputInitializer {Map 3} #0] orc.OrcInputFormat: OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0, start=3, length=586, isOriginal=true, fileLength=770, hasFooter=false, hasBase=true, deltas=0] projected_columns_uncompressed_size: 8226 2018-01-05T11:29:07,852 DEBUG [InputInitializer {Map 3} #0] orc.OrcInputFormat: getSplits finished 2018-01-05T11:29:07,852 INFO [InputInitializer {Map 3} #0] io.HiveInputFormat: number of splits 4 2018-01-05T11:29:07,852 DEBUG [InputInitializer {Map 3} #0] log.PerfLogger: 2018-01-05T11:29:07,852 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:07,852 INFO [InputInitializer {Map 3} #0] tez.HiveSplitGenerator: Number of input splits: 4. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:29:07,852 INFO [InputInitializer {Map 3} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/698bf3ff-2334-401f-bbb9-449cc735a1c6/map.xml 2018-01-05T11:29:07,852 DEBUG [InputInitializer {Map 3} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/698bf3ff-2334-401f-bbb9-449cc735a1c6/map.xml 2018-01-05T11:29:07,852 DEBUG [InputInitializer {Map 3} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/698bf3ff-2334-401f-bbb9-449cc735a1c6/map.xml 2018-01-05T11:29:07,852 DEBUG [InputInitializer {Map 3} #0] log.PerfLogger: 2018-01-05T11:29:07,852 INFO [InputInitializer {Map 3} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:07,852 DEBUG [InputInitializer {Map 3} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,853 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:07,853 DEBUG [InputInitializer {Map 3} #0] log.PerfLogger: 2018-01-05T11:29:07,853 INFO [InputInitializer {Map 3} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 3 size: 3.97KB 2018-01-05T11:29:07,853 DEBUG [InputInitializer {Map 3} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0 to src new group? true 2018-01-05T11:29:07,853 DEBUG [InputInitializer {Map 3} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0 to src new group? false 2018-01-05T11:29:07,853 DEBUG [InputInitializer {Map 3} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0 to src new group? false 2018-01-05T11:29:07,853 DEBUG [InputInitializer {Map 3} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0 to src new group? false 2018-01-05T11:29:07,853 INFO [InputInitializer {Map 3} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:29:07,853 INFO [InputInitializer {Map 3} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:29:07,854 INFO [InputInitializer {Map 3} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:29:07,854 DEBUG [InputInitializer {Map 3} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 10054 2018-01-05T11:29:07,854 DEBUG [InputInitializer {Map 3} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 9371 2018-01-05T11:29:07,854 DEBUG [InputInitializer {Map 3} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 8226 2018-01-05T11:29:07,854 INFO [IPC Server handler 8 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_37_00_000000_0 sent events: (0-1). 2018-01-05T11:29:07,854 DEBUG [InputInitializer {Map 3} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 8538 2018-01-05T11:29:07,854 INFO [InputInitializer {Map 3} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 670 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 36189 Original splits: 4 2018-01-05T11:29:07,854 INFO [InputInitializer {Map 3} #0] grouper.TezSplitGrouper: Desired numSplits: 1 lengthPerGroup: 36189 numLocations: 1 numSplitsPerLocation: 4 numSplitsInGroup: 4 totalLength: 36189 numOriginalSplits: 4 . Grouping by length: true count: false nodeLocalOnly: false 2018-01-05T11:29:07,854 DEBUG [InputInitializer {Map 3} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 10054 2018-01-05T11:29:07,854 DEBUG [InputInitializer {Map 3} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 9371 2018-01-05T11:29:07,854 DEBUG [InputInitializer {Map 3} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 9371 2018-01-05T11:29:07,854 DEBUG [InputInitializer {Map 3} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 8226 2018-01-05T11:29:07,854 DEBUG [InputInitializer {Map 3} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 8226 2018-01-05T11:29:07,854 DEBUG [InputInitializer {Map 3} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 8538 2018-01-05T11:29:07,854 DEBUG [InputInitializer {Map 3} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 8538 2018-01-05T11:29:07,854 INFO [InputInitializer {Map 3} #0] grouper.TezSplitGrouper: Number of splits desired: 1 created: 1 splitsProcessed: 4 2018-01-05T11:29:07,854 INFO [InputInitializer {Map 3} #0] tez.SplitGrouper: Original split count is 4 grouped split count is 1, for bucket: 1 2018-01-05T11:29:07,854 INFO [InputInitializer {Map 3} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:29:07,854 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:29:07,854 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_37_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:29:07,854 INFO [InputInitializer {Map 3} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: b on vertex vertex_1515180518813_0001_37_01 [Map 3] 2018-01-05T11:29:07,854 INFO [App Shared Pool - #1] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_37_01 [Map 3] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:07,854 INFO [App Shared Pool - #1] impl.VertexImpl: Got updated RootInputsSpecs: {b=forAllWorkUnits=true, update=[1]} 2018-01-05T11:29:07,854 INFO [App Shared Pool - #1] impl.VertexImpl: Vertex vertex_1515180518813_0001_37_01 [Map 3] parallelism set to 1 2018-01-05T11:29:07,854 INFO [App Shared Pool - #1] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_37_01 [Map 3] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:07,854 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_37][Event:VERTEX_INITIALIZED]: vertexName=Map 3, vertexId=vertex_1515180518813_0001_37_01, initRequestedTime=1515180547785, initedTime=1515180547854, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:07,854 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_37_01 [Map 3] with distanceFromRoot: 0 2018-01-05T11:29:07,854 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_37_01 [Map 3] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:29:07,854 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_37][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_37_01, startRequestedTime=1515180547800, startedTime=1515180547854 2018-01-05T11:29:07,855 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_37][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_37_01, reconfigureDoneTime=1515180547855, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:29:07,855 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_37_01 [Map 3] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:07,855 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_37_01 for vertex: vertex_1515180518813_0001_37_02 [Reducer 2] numStartedSources: 2 numSources: 2 2018-01-05T11:29:07,855 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_37_02 [Reducer 2] with distanceFromRoot: 1 2018-01-05T11:29:07,855 INFO [App Shared Pool - #1] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 3 2018-01-05T11:29:07,855 INFO [App Shared Pool - #1] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 3 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:29:07,855 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_37][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_37_02, startRequestedTime=1515180547855, startedTime=1515180547855 2018-01-05T11:29:07,855 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_37_02 [Reducer 2] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:07,855 INFO [App Shared Pool - #1] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_37_01 [Map 3] 2018-01-05T11:29:07,855 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: OnVertexStarted vertex: Reducer 2 with 0 source tasks and 1 pending tasks 2018-01-05T11:29:07,855 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_37][Event:TASK_STARTED]: vertexName=Map 3, taskId=task_1515180518813_0001_37_01_000000, scheduledTime=1515180547855, launchTime=1515180547855 2018-01-05T11:29:07,855 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Map 3 in vertex: Reducer 2 numBipartiteSourceTasks: 1 2018-01-05T11:29:07,855 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_37_01 [Map 3] 2018-01-05T11:29:07,855 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Map 1 in vertex: Reducer 2 numBipartiteSourceTasks: 2 2018-01-05T11:29:07,855 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_37_01_000000_0, priority=11, capability=, hosts=null 2018-01-05T11:29:07,855 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=11,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:07,855 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=11,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:07,855 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:07,855 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_37_01_000000_0 on hosts=null 2018-01-05T11:29:07,856 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_37_01_000000_0 on hosts=null 2018-01-05T11:29:07,856 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_37_01_000000_0 2018-01-05T11:29:07,856 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:07,856 INFO [I/O Setup 0 Initialize: {a}] input.MRInput: a using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:29:07,856 INFO [I/O Setup 0 Initialize: {a}] input.MRInputLegacy: a MRInputLegacy deferring initialization 2018-01-05T11:29:07,856 DEBUG [TezTR-518813_1_37_0_0_0] log.PerfLogger: 2018-01-05T11:29:07,856 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #88, task=TaskInfo{task=attempt_1515180518813_0001_37_01_000000_0, priority=11, startTime=0, containerId=null, uniqueId=87, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, to container=container_111101111_0001_00_000088 2018-01-05T11:29:07,856 INFO [TezTR-518813_1_37_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:07,856 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_37_01_000000_0; false 2018-01-05T11:29:07,856 INFO [TezTR-518813_1_37_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:07,856 INFO [TezTR-518813_1_37_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:07,856 INFO [TezTR-518813_1_37_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:07,856 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_37_01_000000_0, priority=11, startTime=311568365, containerId=container_111101111_0001_00_000088assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=2, ac=6, commF=false, disabled=false}, uniqueId=87, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:07,856 INFO [TezTR-518813_1_37_0_0_0] resources.MemoryDistributor: InitialRequests=[a:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [Reducer 2:OUTPUT:10485760:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput] 2018-01-05T11:29:07,856 INFO [TezTR-518813_1_37_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:07,856 INFO [TezTR-518813_1_37_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:07,856 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000088 for node: 10.22.8.182:49818 2018-01-05T11:29:07,856 INFO [TezTR-518813_1_37_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 10485760, TotalRequestedScaled: 9679163.076923078, TotalJVMHeap: 1021837312, TotalAvailable: 684630999, TotalRequested/TotalJVMHeap:0.01 2018-01-05T11:29:07,856 INFO [TezTR-518813_1_37_0_0_0] resources.MemoryDistributor: Allocations=[a:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:10485760:10485760] 2018-01-05T11:29:07,856 INFO [TezTR-518813_1_37_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:07,856 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000088, attempt_1515180518813_0001_37_01_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:07,856 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000088 for node: 10.22.8.182:49818 2018-01-05T11:29:07,856 INFO [TezTR-518813_1_37_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: a being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:07,856 INFO [I/O Setup 1 Start: {a}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: a 2018-01-05T11:29:07,856 INFO [TezTR-518813_1_37_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:29:07,856 INFO [TezTR-518813_1_37_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:07,856 INFO [TezTR-518813_1_37_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:07,856 INFO [TezTR-518813_1_37_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_37_00_000000_0 2018-01-05T11:29:07,856 DEBUG [TezTR-518813_1_37_0_0_0] log.PerfLogger: 2018-01-05T11:29:07,856 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000088 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:07,856 DEBUG [TezTR-518813_1_37_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_37_00_000000_0_10262 2018-01-05T11:29:07,856 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000088, launchTime=1515180547856 2018-01-05T11:29:07,856 DEBUG [TezTR-518813_1_37_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:07,856 DEBUG [TezTR-518813_1_37_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_37_00_000000_0 2018-01-05T11:29:07,856 INFO [TezTR-518813_1_37_0_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949 2018-01-05T11:29:07,856 DEBUG [TezTR-518813_1_37_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:07,856 DEBUG [TezTR-518813_1_37_0_0_0] log.PerfLogger: 2018-01-05T11:29:07,857 DEBUG [TezTR-518813_1_37_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:07,857 DEBUG [TezTR-518813_1_37_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:07,857 DEBUG [TezTR-518813_1_37_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949_Map 1__MAP_PLAN__ 2018-01-05T11:29:07,857 INFO [TezTR-518813_1_37_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/04343b50-c7d4-4ff5-9f68-f7ee98dcd09d/map.xml 2018-01-05T11:29:07,857 DEBUG [TezTR-518813_1_37_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/04343b50-c7d4-4ff5-9f68-f7ee98dcd09d/map.xml 2018-01-05T11:29:07,857 DEBUG [TezTR-518813_1_37_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/04343b50-c7d4-4ff5-9f68-f7ee98dcd09d/map.xml 2018-01-05T11:29:07,857 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_37_01_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000088, dagName=select * from tab a join tab_part b ...b.key (Stage-1), vertexName=Map 3, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{b,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{Reducer 2,org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput,1}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180547784, firstAttemptStartTime=1515180547855, currentAttemptStartTime=1515180547856} 2018-01-05T11:29:07,857 INFO [IPC Server handler 3 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000088, QueryId=djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949-37, DagId=37, VertexName=Map 3, TaskId=0, TaskAttemptId=0, SubmitTime=1515180547857 2018-01-05T11:29:07,857 DEBUG [TezTR-518813_1_37_0_0_0] log.PerfLogger: 2018-01-05T11:29:07,857 INFO [TezTR-518813_1_37_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:07,857 DEBUG [TezTR-518813_1_37_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,857 DEBUG [IPC Server handler 3 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=37} with the ShuffleHandler 2018-01-05T11:29:07,857 DEBUG [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/37] 2018-01-05T11:29:07,857 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_37_01_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000088] on NM: [10.22.8.182:49818] 2018-01-05T11:29:07,857 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:07,857 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:07,857 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_37][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 3, taskAttemptId=attempt_1515180518813_0001_37_01_000000_0, startTime=1515180547856, containerId=container_111101111_0001_00_000088, nodeId=10.22.8.182:49818 2018-01-05T11:29:07,857 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=3, runningFragmentCount=1 2018-01-05T11:29:07,857 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_37_01_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:07,857 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_37_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180547855, dagStartTime=1515180547784, withinDagPriority=11, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:29:07,857 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_37_01_000000_0 : ACCEPTED 2018-01-05T11:29:07,857 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_37_01_000000_0, canFinish=true. Current state: preemptionQueueSize=1, numSlotsAvailable=3, waitQueueSize=1 2018-01-05T11:29:07,857 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_37_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180547855, dagStartTime=1515180547784, withinDagPriority=11, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:29:07,857 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_37_01_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:07,857 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_37_01_000000_0 to pre-emption queue 2018-01-05T11:29:07,857 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_37_01_000000_0: true 2018-01-05T11:29:07,857 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: taskOwner hashCode:89612976 2018-01-05T11:29:07,857 INFO [TaskCommunicator # 0] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_37_01_000000_0 2018-01-05T11:29:07,857 INFO [TaskCommunicator # 0] GuaranteedTasks: Received response for attempt_1515180518813_0001_37_01_000000_0, true 2018-01-05T11:29:07,857 DEBUG [TezTR-518813_1_37_0_0_0] log.PerfLogger: 2018-01-05T11:29:07,858 INFO [TezTR-518813_1_37_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.96KB 2018-01-05T11:29:07,860 INFO [Task-Executor-1] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : select * from tab a join tab_part b ...b.key (Stage-1), VertexName: Map 3, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_37_01_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=b, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=Reducer 2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ] 2018-01-05T11:29:07,860 INFO [Task-Executor-1] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1021837312, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:07,860 INFO [Task-Executor-1] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_37_01_000000_0 vertexName: Map 3 2018-01-05T11:29:07,860 DEBUG [TezTR-518813_1_37_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:07,860 INFO [TezTR-518813_1_37_0_0_0] tez.MapRecordProcessor: The input names are: [a] 2018-01-05T11:29:07,860 INFO [TezTR-518813_1_37_0_0_0] input.MRInput: Initialized MRInput: a 2018-01-05T11:29:07,860 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000088, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_37_01_000000_0, eventCount=1 } 2018-01-05T11:29:07,860 DEBUG [TezTR-518813_1_37_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 2 2018-01-05T11:29:07,860 DEBUG [TezTR-518813_1_37_1_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_37 queryId: djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949 instanceType: TaskRunner2Callable 2018-01-05T11:29:07,860 INFO [TezTR-518813_1_37_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/04343b50-c7d4-4ff5-9f68-f7ee98dcd09d/map.xml 2018-01-05T11:29:07,860 DEBUG [TezTR-518813_1_37_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:07,860 INFO [TezTR-518813_1_37_1_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_37_01_000000_0 2018-01-05T11:29:07,861 DEBUG [TezTR-518813_1_37_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180546}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, numFiles=2, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, rawDataSize=22748, partition_columns.types=string, numRows=242, partition_columns=ds, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=1511, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08, transient_lastDdlTime=1515180547}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180546}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, numFiles=2, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, rawDataSize=22748, partition_columns.types=string, numRows=242, partition_columns=ds, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=1511, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08, transient_lastDdlTime=1515180547}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat]} 2018-01-05T11:29:07,861 DEBUG [TezTR-518813_1_37_1_0_0] log.PerfLogger: 2018-01-05T11:29:07,861 DEBUG [TezTR-518813_1_37_0_0_0] io.HiveInputFormat: Pushdown initiated with filterText = key is not null, filterExpr = GenericUDFOPNotNull(Column[key]), serializedFilterExpr = AQEAamF2YS51dGlsLkFycmF5TGlz9AEBAQFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVDb2x1bW5EZXPjAQFrZfkAAAGCYQECb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5zZXJkZTIudHlwZWluZm8uUHJpbWl0aXZlVHlwZUluZu8BAWlu9AEDb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGT1BOb3ROdWzsAQEEb3JnLmFwYWNoZS5oYWRvb3AuaW8uQm9vbGVhbldyaXRhYmzlAQAAAQIBAWJvb2xlYe4= (desc) 2018-01-05T11:29:07,861 INFO [TezTR-518813_1_37_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/04343b50-c7d4-4ff5-9f68-f7ee98dcd09d/map.xml 2018-01-05T11:29:07,861 DEBUG [TezTR-518813_1_37_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:07,861 DEBUG [TezTR-518813_1_37_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.hive.ql.io.orc.OrcInputFormat 2018-01-05T11:29:07,861 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:07,862 DEBUG [TezTR-518813_1_37_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:29:07,862 INFO [TezTR-518813_1_37_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/04343b50-c7d4-4ff5-9f68-f7ee98dcd09d/map.xml 2018-01-05T11:29:07,862 DEBUG [TezTR-518813_1_37_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:07,862 INFO [TezTR-518813_1_37_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_37_00_000000_0 2018-01-05T11:29:07,862 INFO [TezTR-518813_1_37_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:07,862 DEBUG [TezTR-518813_1_37_0_0_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:07,862 DEBUG [TezTR-518813_1_37_0_0_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:07,862 DEBUG [TezTR-518813_1_37_0_0_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:07,862 DEBUG [TezTR-518813_1_37_0_0_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:07,862 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:07,863 DEBUG [TezTR-518813_1_37_0_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,863 INFO [IPC Server handler 7 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_37_01_000000_0 sent events: (0-1). 2018-01-05T11:29:07,863 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:29:07,863 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_37_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:29:07,863 DEBUG [TezTR-518813_1_37_0_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,864 INFO [TezTR-518813_1_37_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/04343b50-c7d4-4ff5-9f68-f7ee98dcd09d/map.xml 2018-01-05T11:29:07,864 DEBUG [TezTR-518813_1_37_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:07,864 DEBUG [TezTR-518813_1_37_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0 2018-01-05T11:29:07,864 DEBUG [IO-Elevator-Thread-0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_37 queryId: djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949 instanceType: OrcEncodedDataReader 2018-01-05T11:29:07,864 INFO [TezTR-518813_1_37_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0, start=3, length=583, isOriginal=true, fileLength=767, hasFooter=false, hasBase=true, deltas=0], org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0, start=3, length=566, isOriginal=true, fileLength=744, hasFooter=false, hasBase=true, deltas=0]], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=1149} 2018-01-05T11:29:07,864 INFO [TezTR-518813_1_37_0_0_0] input.MRInput: a initialized RecordReader from event 2018-01-05T11:29:07,864 INFO [IO-Elevator-Thread-0] LlapIoImpl: Processing data for file [-5122461581150239451, 1515180546000, 767]: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0 2018-01-05T11:29:07,864 DEBUG [TezTR-518813_1_37_0_0_0] tez.MapRecordProcessor: Starting Output: Reducer 2 2018-01-05T11:29:07,864 DEBUG [IO-Elevator-Thread-0] LlapIoOrc: FileSplit {3, 583}; stripes {3, 583}, 2018-01-05T11:29:07,865 DEBUG [IO-Elevator-Thread-0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,865 INFO [TezTR-518813_1_37_0_0_0] impl.ExternalSorter: Reducer 2 using: memoryMb=10, keySerializerClass=class org.apache.hadoop.hive.ql.io.HiveKey, valueSerializerClass=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization$TezBytesWritableSerializer@3261a57f, comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator@35629269, partitioner=org.apache.tez.mapreduce.partition.MRPartitioner, serialization=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.hadoop.io.serializer.WritableSerialization, org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization, org.apache.hadoop.io.serializer.avro.AvroReflectSerialization, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:29:07,865 INFO [TezTR-518813_1_37_0_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 2: , UsingHashComparator=true 2018-01-05T11:29:07,865 INFO [I/O Setup 0 Initialize: {b}] input.MRInput: b using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:29:07,865 INFO [I/O Setup 0 Initialize: {b}] input.MRInputLegacy: b MRInputLegacy deferring initialization 2018-01-05T11:29:07,865 DEBUG [TezTR-518813_1_37_1_0_0] log.PerfLogger: 2018-01-05T11:29:07,865 INFO [TezTR-518813_1_37_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:07,865 INFO [TezTR-518813_1_37_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:07,865 INFO [TezTR-518813_1_37_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:07,865 INFO [TezTR-518813_1_37_1_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:07,865 INFO [TezTR-518813_1_37_1_0_0] resources.MemoryDistributor: InitialRequests=[b:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [Reducer 2:OUTPUT:10485760:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput] 2018-01-05T11:29:07,865 INFO [TezTR-518813_1_37_1_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:07,865 INFO [TezTR-518813_1_37_1_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:07,865 DEBUG [IO-Elevator-Thread-0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,865 INFO [TezTR-518813_1_37_1_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 10485760, TotalRequestedScaled: 9679163.076923078, TotalJVMHeap: 1021837312, TotalAvailable: 684630999, TotalRequested/TotalJVMHeap:0.01 2018-01-05T11:29:07,865 INFO [TezTR-518813_1_37_1_0_0] resources.MemoryDistributor: Allocations=[b:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:10485760:10485760] 2018-01-05T11:29:07,865 INFO [TezTR-518813_1_37_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:07,865 DEBUG [IO-Elevator-Thread-0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,865 INFO [TezTR-518813_1_37_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: b being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:07,865 INFO [IO-Elevator-Thread-0] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [-5122461581150239451, 1515180546000, 767]): [{range start: 0 end: 76}] 2018-01-05T11:29:07,865 INFO [TezTR-518813_1_37_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:29:07,865 INFO [I/O Setup 1 Start: {b}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: b 2018-01-05T11:29:07,865 INFO [TezTR-518813_1_37_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:07,866 INFO [TezTR-518813_1_37_1_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:07,866 INFO [TezTR-518813_1_37_0_0_0] impl.PipelinedSorter: Newly allocated block size=10485760, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=10485760, total=10485760 2018-01-05T11:29:07,866 INFO [TezTR-518813_1_37_0_0_0] impl.PipelinedSorter: Pre allocating rest of memory buffers upfront 2018-01-05T11:29:07,866 INFO [IO-Elevator-Thread-0] encoded.EncodedReaderImpl: Disk ranges after cache (found everything false; file [-5122461581150239451, 1515180546000, 767], base offset 3): [{range start: 0 end: 76}] 2018-01-05T11:29:07,866 INFO [TezTR-518813_1_37_0_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 2: , UsingHashComparator=true#blocks=1, maxMemUsage=10485760, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=10485760, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=10 2018-01-05T11:29:07,866 INFO [TezTR-518813_1_37_0_0_0] impl.PipelinedSorter: Reducer 2: reserved.remaining()=10485760, reserved.metasize=5242880 2018-01-05T11:29:07,866 INFO [TezTR-518813_1_37_1_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_37_01_000000_0 2018-01-05T11:29:07,866 DEBUG [TezTR-518813_1_37_1_0_0] log.PerfLogger: 2018-01-05T11:29:07,866 DEBUG [TezTR-518813_1_37_1_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_37_01_000000_0_10265 2018-01-05T11:29:07,866 DEBUG [TezTR-518813_1_37_1_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:07,866 DEBUG [TezTR-518813_1_37_1_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_37_01_000000_0 2018-01-05T11:29:07,867 INFO [TezTR-518813_1_37_0_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:29:07,867 INFO [TezTR-518813_1_37_0_0_0] vector.VectorMapOperator: Initializing operator MAP[0] 2018-01-05T11:29:07,867 DEBUG [TezTR-518813_1_37_0_0_0] vector.VectorMapOperator: Initialization Done 0 MAP 2018-01-05T11:29:07,867 DEBUG [TezTR-518813_1_37_0_0_0] vector.VectorMapOperator: Operator 0 MAP initialized 2018-01-05T11:29:07,867 DEBUG [TezTR-518813_1_37_0_0_0] vector.VectorMapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:29:07,867 DEBUG [TezTR-518813_1_37_1_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:07,867 DEBUG [TezTR-518813_1_37_1_0_0] log.PerfLogger: 2018-01-05T11:29:07,867 DEBUG [TezTR-518813_1_37_1_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:07,867 DEBUG [TezTR-518813_1_37_1_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:07,867 DEBUG [TezTR-518813_1_37_1_0_0] mr.ObjectCache: Creating djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949_Map 3__MAP_PLAN__ 2018-01-05T11:29:07,867 INFO [TezTR-518813_1_37_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/698bf3ff-2334-401f-bbb9-449cc735a1c6/map.xml 2018-01-05T11:29:07,867 INFO [TezTR-518813_1_37_0_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:29:07,867 DEBUG [TezTR-518813_1_37_1_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/698bf3ff-2334-401f-bbb9-449cc735a1c6/map.xml 2018-01-05T11:29:07,867 DEBUG [TezTR-518813_1_37_1_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/698bf3ff-2334-401f-bbb9-449cc735a1c6/map.xml 2018-01-05T11:29:07,867 INFO [TezTR-518813_1_37_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:07,867 DEBUG [TezTR-518813_1_37_1_0_0] log.PerfLogger: 2018-01-05T11:29:07,867 DEBUG [TezTR-518813_1_37_0_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:29:07,867 INFO [TezTR-518813_1_37_1_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:07,867 DEBUG [TezTR-518813_1_37_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,867 INFO [TezTR-518813_1_37_0_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:29:07,867 DEBUG [TezTR-518813_1_37_0_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:29:07,867 DEBUG [TezTR-518813_1_37_0_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:29:07,867 DEBUG [TezTR-518813_1_37_0_0_0] exec.TableScanOperator: Initializing children of 0 TS 2018-01-05T11:29:07,867 DEBUG [TezTR-518813_1_37_0_0_0] vector.VectorFilterOperator: Initializing child 26 FIL 2018-01-05T11:29:07,867 INFO [TezTR-518813_1_37_0_0_0] vector.VectorFilterOperator: Initializing operator FIL[26] 2018-01-05T11:29:07,867 DEBUG [TezTR-518813_1_37_0_0_0] vector.VectorFilterOperator: Initialization Done 26 FIL 2018-01-05T11:29:07,867 DEBUG [TezTR-518813_1_37_0_0_0] vector.VectorFilterOperator: Operator 26 FIL initialized 2018-01-05T11:29:07,867 DEBUG [TezTR-518813_1_37_0_0_0] vector.VectorFilterOperator: Initializing children of 26 FIL 2018-01-05T11:29:07,867 DEBUG [TezTR-518813_1_37_0_0_0] vector.VectorSelectOperator: Initializing child 27 SEL 2018-01-05T11:29:07,867 INFO [TezTR-518813_1_37_0_0_0] vector.VectorSelectOperator: Initializing operator SEL[27] 2018-01-05T11:29:07,867 DEBUG [TezTR-518813_1_37_0_0_0] vector.VectorSelectOperator: Initialization Done 27 SEL 2018-01-05T11:29:07,867 DEBUG [TezTR-518813_1_37_0_0_0] vector.VectorSelectOperator: Operator 27 SEL initialized 2018-01-05T11:29:07,867 DEBUG [TezTR-518813_1_37_0_0_0] vector.VectorSelectOperator: Initializing children of 27 SEL 2018-01-05T11:29:07,867 DEBUG [TezTR-518813_1_37_0_0_0] reducesink.VectorReduceSinkLongOperator: Initializing child 28 RS 2018-01-05T11:29:07,867 INFO [TezTR-518813_1_37_0_0_0] reducesink.VectorReduceSinkLongOperator: Initializing operator RS[28] 2018-01-05T11:29:07,867 INFO [TezTR-518813_1_37_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/04343b50-c7d4-4ff5-9f68-f7ee98dcd09d/map.xml 2018-01-05T11:29:07,867 DEBUG [TezTR-518813_1_37_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:07,867 INFO [TezTR-518813_1_37_0_0_0] reducesink.VectorReduceSinkCommonOperator: Using tag = 0 2018-01-05T11:29:07,868 DEBUG [TezTR-518813_1_37_1_0_0] log.PerfLogger: 2018-01-05T11:29:07,868 DEBUG [IO-Elevator-Thread-0] impl.RecordReaderImpl: Row group 0 to 9999 is included. 2018-01-05T11:29:07,868 INFO [TezTR-518813_1_37_1_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 3 size: 3.97KB 2018-01-05T11:29:07,868 DEBUG [IO-Elevator-Thread-0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,868 DEBUG [TezTR-518813_1_37_0_0_0] reducesink.VectorReduceSinkLongOperator: Initialization Done 28 RS 2018-01-05T11:29:07,868 DEBUG [TezTR-518813_1_37_0_0_0] reducesink.VectorReduceSinkLongOperator: Operator 28 RS initialized 2018-01-05T11:29:07,868 INFO [IO-Elevator-Thread-0] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [-5122461581150239451, 1515180546000, 767]): [{range start: 76 end: 523}] 2018-01-05T11:29:07,868 DEBUG [TezTR-518813_1_37_0_0_0] reducesink.VectorReduceSinkLongOperator: Initialization Done 28 RS done is reset. 2018-01-05T11:29:07,868 INFO [IO-Elevator-Thread-0] encoded.EncodedReaderImpl: Disk ranges after cache (found everything false; file [-5122461581150239451, 1515180546000, 767], base offset 3): [{range start: 76 end: 523}] 2018-01-05T11:29:07,868 DEBUG [TezTR-518813_1_37_0_0_0] vector.VectorSelectOperator: Initialization Done 27 SEL done is reset. 2018-01-05T11:29:07,868 DEBUG [TezTR-518813_1_37_0_0_0] vector.VectorFilterOperator: Initialization Done 26 FIL done is reset. 2018-01-05T11:29:07,868 DEBUG [TezTR-518813_1_37_0_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:29:07,868 INFO [TezTR-518813_1_37_0_0_0] exec.OperatorUtils: Setting output collector: RS[28] --> Reducer 2 2018-01-05T11:29:07,868 DEBUG [TezTR-518813_1_37_0_0_0] log.PerfLogger: 2018-01-05T11:29:07,868 DEBUG [TezTR-518813_1_37_1_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:07,868 INFO [TezTR-518813_1_37_1_0_0] tez.MapRecordProcessor: The input names are: [b] 2018-01-05T11:29:07,868 INFO [TezTR-518813_1_37_1_0_0] input.MRInput: Initialized MRInput: b 2018-01-05T11:29:07,869 DEBUG [TezTR-518813_1_37_1_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 4 2018-01-05T11:29:07,869 INFO [TezTR-518813_1_37_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/698bf3ff-2334-401f-bbb9-449cc735a1c6/map.xml 2018-01-05T11:29:07,869 DEBUG [TezTR-518813_1_37_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:07,869 DEBUG [TezTR-518813_1_37_1_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat]} 2018-01-05T11:29:07,869 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: int streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 164 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:29:07,869 DEBUG [TezTR-518813_1_37_1_0_0] io.HiveInputFormat: Pushdown initiated with filterText = key is not null, filterExpr = GenericUDFOPNotNull(Column[key]), serializedFilterExpr = AQEAamF2YS51dGlsLkFycmF5TGlz9AEBAQFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVDb2x1bW5EZXPjAQFrZfkAAAGCYgECb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5zZXJkZTIudHlwZWluZm8uUHJpbWl0aXZlVHlwZUluZu8BAWlu9AEDb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGT1BOb3ROdWzsAQEEb3JnLmFwYWNoZS5oYWRvb3AuaW8uQm9vbGVhbldyaXRhYmzlAQAAAQIBAWJvb2xlYe4= (desc) 2018-01-05T11:29:07,869 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 2 columnType: string streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DICTIONARY_V2 dictionarySize: 73 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 97 bytes, ], indexBaseOffset=0] dictionary: true lengths: true secondary: false tz: 2018-01-05T11:29:07,869 INFO [TezTR-518813_1_37_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/698bf3ff-2334-401f-bbb9-449cc735a1c6/map.xml 2018-01-05T11:29:07,869 DEBUG [TezTR-518813_1_37_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:07,869 DEBUG [TezTR-518813_1_37_1_0_0] io.HiveInputFormat: Processing org.apache.hadoop.hive.ql.io.orc.OrcInputFormat 2018-01-05T11:29:07,869 DEBUG [IO-Elevator-Thread-0] LlapIoOrc: Done with decode 2018-01-05T11:29:07,869 DEBUG [IO-Elevator-Thread-0] LlapIoImpl: setDone called; closed false, done false, err null, pending 0 2018-01-05T11:29:07,869 DEBUG [IO-Elevator-Thread-0] impl.StatsRecordingThreadPool: Updated stats: instance: OrcEncodedDataReader thread name: IO-Elevator-Thread-0 thread id: 1252 scheme: file bytesRead: 1885 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:07,870 INFO [TezTR-518813_1_37_0_0_0] reducesink.VectorReduceSinkCommonOperator: RS[28]: records written - 1 2018-01-05T11:29:07,870 DEBUG [TezTR-518813_1_37_1_0_0] impl.LlapRecordReader: Initializing for input Map 3 2018-01-05T11:29:07,870 INFO [TezTR-518813_1_37_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/698bf3ff-2334-401f-bbb9-449cc735a1c6/map.xml 2018-01-05T11:29:07,870 DEBUG [TezTR-518813_1_37_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:07,870 INFO [TezTR-518813_1_37_0_0_0] reducesink.VectorReduceSinkCommonOperator: RS[28]: records written - 10 2018-01-05T11:29:07,870 INFO [TezTR-518813_1_37_1_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_37_01_000000_0 2018-01-05T11:29:07,870 INFO [TezTR-518813_1_37_1_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:07,870 DEBUG [TezTR-518813_1_37_1_0_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:07,870 DEBUG [TezTR-518813_1_37_1_0_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:07,870 DEBUG [TezTR-518813_1_37_1_0_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:07,870 DEBUG [TezTR-518813_1_37_1_0_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:07,870 INFO [TezTR-518813_1_37_0_0_0] reducesink.VectorReduceSinkCommonOperator: RS[28]: records written - 100 2018-01-05T11:29:07,870 INFO [TezTR-518813_1_37_0_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:07,870 INFO [TezTR-518813_1_37_0_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, warehouse.tab, file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0 ([-5122461581150239451, 1515180546000, 767]), 0,1]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=1, NUM_ERRORS=0, ROWS_EMITTED=120, METADATA_CACHE_HIT=0, METADATA_CACHE_MISS=2, CACHE_HIT_BYTES=0, CACHE_MISS_BYTES=523, ALLOCATED_BYTES=1835008, ALLOCATED_USED_BYTES=864, TOTAL_IO_TIME_NS=5170707, DECODE_TIME_NS=348283, HDFS_TIME_NS=2494158, CONSUMER_TIME_NS=1047837 ] 2018-01-05T11:29:07,870 DEBUG [TezTR-518813_1_37_0_0_0] encoded.OrcEncodedDataReader: Encoded reader is being stopped 2018-01-05T11:29:07,870 DEBUG [TezTR-518813_1_37_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 1 of 2 2018-01-05T11:29:07,870 DEBUG [TezTR-518813_1_37_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180546}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, numFiles=2, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, rawDataSize=22748, partition_columns.types=string, numRows=242, partition_columns=ds, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=1511, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08, transient_lastDdlTime=1515180547}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180546}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, numFiles=2, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, rawDataSize=22748, partition_columns.types=string, numRows=242, partition_columns=ds, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=1511, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08, transient_lastDdlTime=1515180547}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat]} 2018-01-05T11:29:07,870 DEBUG [TezTR-518813_1_37_0_0_0] io.HiveInputFormat: Pushdown initiated with filterText = key is not null, filterExpr = GenericUDFOPNotNull(Column[key]), serializedFilterExpr = AQEAamF2YS51dGlsLkFycmF5TGlz9AEBAQFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVDb2x1bW5EZXPjAQFrZfkAAAGCYQECb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5zZXJkZTIudHlwZWluZm8uUHJpbWl0aXZlVHlwZUluZu8BAWlu9AEDb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGT1BOb3ROdWzsAQEEb3JnLmFwYWNoZS5oYWRvb3AuaW8uQm9vbGVhbldyaXRhYmzlAQAAAQIBAWJvb2xlYe4= (desc) 2018-01-05T11:29:07,870 DEBUG [TezTR-518813_1_37_1_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,870 INFO [TezTR-518813_1_37_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/04343b50-c7d4-4ff5-9f68-f7ee98dcd09d/map.xml 2018-01-05T11:29:07,871 DEBUG [TezTR-518813_1_37_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:07,871 DEBUG [TezTR-518813_1_37_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.hive.ql.io.orc.OrcInputFormat 2018-01-05T11:29:07,871 DEBUG [TezTR-518813_1_37_1_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,871 INFO [TezTR-518813_1_37_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/698bf3ff-2334-401f-bbb9-449cc735a1c6/map.xml 2018-01-05T11:29:07,871 DEBUG [TezTR-518813_1_37_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:07,871 DEBUG [TezTR-518813_1_37_1_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0 2018-01-05T11:29:07,871 DEBUG [IO-Elevator-Thread-1] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_37 queryId: djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949 instanceType: OrcEncodedDataReader 2018-01-05T11:29:07,871 INFO [TezTR-518813_1_37_1_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0, start=3, length=641, isOriginal=true, fileLength=827, hasFooter=false, hasBase=true, deltas=0], org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0, start=3, length=641, isOriginal=true, fileLength=823, hasFooter=false, hasBase=true, deltas=0], org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0, start=3, length=586, isOriginal=true, fileLength=770, hasFooter=false, hasBase=true, deltas=0], org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0, start=3, length=574, isOriginal=true, fileLength=752, hasFooter=false, hasBase=true, deltas=0]], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=2442} 2018-01-05T11:29:07,871 INFO [IO-Elevator-Thread-1] LlapIoImpl: Processing data for file [6462268725460710401, 1515180546000, 827]: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0 2018-01-05T11:29:07,871 INFO [TezTR-518813_1_37_1_0_0] input.MRInput: b initialized RecordReader from event 2018-01-05T11:29:07,871 DEBUG [IO-Elevator-Thread-1] LlapIoOrc: FileSplit {3, 641}; stripes {3, 641}, 2018-01-05T11:29:07,871 DEBUG [TezTR-518813_1_37_1_0_0] tez.MapRecordProcessor: Starting Output: Reducer 2 2018-01-05T11:29:07,871 DEBUG [TezTR-518813_1_37_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:29:07,871 DEBUG [IO-Elevator-Thread-1] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,871 INFO [TezTR-518813_1_37_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/04343b50-c7d4-4ff5-9f68-f7ee98dcd09d/map.xml 2018-01-05T11:29:07,872 DEBUG [TezTR-518813_1_37_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:07,872 INFO [TezTR-518813_1_37_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_37_00_000000_0 2018-01-05T11:29:07,872 INFO [TezTR-518813_1_37_1_0_0] impl.ExternalSorter: Reducer 2 using: memoryMb=10, keySerializerClass=class org.apache.hadoop.hive.ql.io.HiveKey, valueSerializerClass=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization$TezBytesWritableSerializer@74c5b709, comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator@7a801fcb, partitioner=org.apache.tez.mapreduce.partition.MRPartitioner, serialization=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.hadoop.io.serializer.WritableSerialization, org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization, org.apache.hadoop.io.serializer.avro.AvroReflectSerialization, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:29:07,872 INFO [TezTR-518813_1_37_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:07,872 INFO [TezTR-518813_1_37_1_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 2: , UsingHashComparator=true 2018-01-05T11:29:07,872 DEBUG [IO-Elevator-Thread-1] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,872 DEBUG [IO-Elevator-Thread-1] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,872 INFO [IO-Elevator-Thread-1] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725460710401, 1515180546000, 827]): [{range start: 0 end: 79}] 2018-01-05T11:29:07,874 INFO [IO-Elevator-Thread-1] encoded.EncodedReaderImpl: Disk ranges after cache (found everything false; file [6462268725460710401, 1515180546000, 827], base offset 3): [{range start: 0 end: 79}] 2018-01-05T11:29:07,874 INFO [TezTR-518813_1_37_1_0_0] impl.PipelinedSorter: Newly allocated block size=10485760, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=10485760, total=10485760 2018-01-05T11:29:07,874 INFO [TezTR-518813_1_37_1_0_0] impl.PipelinedSorter: Pre allocating rest of memory buffers upfront 2018-01-05T11:29:07,874 INFO [TezTR-518813_1_37_1_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 2: , UsingHashComparator=true#blocks=1, maxMemUsage=10485760, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=10485760, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=10 2018-01-05T11:29:07,874 INFO [TezTR-518813_1_37_1_0_0] impl.PipelinedSorter: Reducer 2: reserved.remaining()=10485760, reserved.metasize=5242880 2018-01-05T11:29:07,874 INFO [TezTR-518813_1_37_1_0_0] tez.MapRecordProcessor: Main input name is Map 3 2018-01-05T11:29:07,874 INFO [TezTR-518813_1_37_1_0_0] vector.VectorMapOperator: Initializing operator MAP[0] 2018-01-05T11:29:07,874 DEBUG [TezTR-518813_1_37_1_0_0] vector.VectorMapOperator: Initialization Done 0 MAP 2018-01-05T11:29:07,874 DEBUG [TezTR-518813_1_37_1_0_0] vector.VectorMapOperator: Operator 0 MAP initialized 2018-01-05T11:29:07,874 DEBUG [TezTR-518813_1_37_1_0_0] vector.VectorMapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:29:07,874 INFO [TezTR-518813_1_37_1_0_0] tez.MapRecordProcessor: Id =0 Id =3 Id =29 Id =30 Id =31 <\Children> Id = 30 null<\Parent> <\RS> <\Children> Id = 29 null<\Parent> <\SEL> <\Children> Id = 3 null<\Parent> <\FIL> <\Children> <\Parent> <\TS> <\Children> <\Parent> <\MAP> 2018-01-05T11:29:07,874 INFO [TezTR-518813_1_37_1_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:07,874 DEBUG [TezTR-518813_1_37_1_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:07,874 INFO [TezTR-518813_1_37_1_0_0] exec.TableScanOperator: Initializing operator TS[3] 2018-01-05T11:29:07,874 DEBUG [TezTR-518813_1_37_1_0_0] exec.TableScanOperator: Initialization Done 3 TS 2018-01-05T11:29:07,874 DEBUG [TezTR-518813_1_37_1_0_0] exec.TableScanOperator: Operator 3 TS initialized 2018-01-05T11:29:07,874 DEBUG [TezTR-518813_1_37_1_0_0] exec.TableScanOperator: Initializing children of 3 TS 2018-01-05T11:29:07,874 DEBUG [TezTR-518813_1_37_1_0_0] vector.VectorFilterOperator: Initializing child 29 FIL 2018-01-05T11:29:07,874 INFO [TezTR-518813_1_37_1_0_0] vector.VectorFilterOperator: Initializing operator FIL[29] 2018-01-05T11:29:07,874 DEBUG [TezTR-518813_1_37_0_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,874 DEBUG [TezTR-518813_1_37_1_0_0] vector.VectorFilterOperator: Initialization Done 29 FIL 2018-01-05T11:29:07,874 DEBUG [TezTR-518813_1_37_1_0_0] vector.VectorFilterOperator: Operator 29 FIL initialized 2018-01-05T11:29:07,874 DEBUG [TezTR-518813_1_37_1_0_0] vector.VectorFilterOperator: Initializing children of 29 FIL 2018-01-05T11:29:07,874 DEBUG [TezTR-518813_1_37_1_0_0] vector.VectorSelectOperator: Initializing child 30 SEL 2018-01-05T11:29:07,874 INFO [TezTR-518813_1_37_1_0_0] vector.VectorSelectOperator: Initializing operator SEL[30] 2018-01-05T11:29:07,874 DEBUG [TezTR-518813_1_37_1_0_0] vector.VectorSelectOperator: Initialization Done 30 SEL 2018-01-05T11:29:07,874 DEBUG [TezTR-518813_1_37_1_0_0] vector.VectorSelectOperator: Operator 30 SEL initialized 2018-01-05T11:29:07,874 DEBUG [TezTR-518813_1_37_1_0_0] vector.VectorSelectOperator: Initializing children of 30 SEL 2018-01-05T11:29:07,874 DEBUG [TezTR-518813_1_37_1_0_0] reducesink.VectorReduceSinkLongOperator: Initializing child 31 RS 2018-01-05T11:29:07,874 INFO [TezTR-518813_1_37_1_0_0] reducesink.VectorReduceSinkLongOperator: Initializing operator RS[31] 2018-01-05T11:29:07,874 DEBUG [TezTR-518813_1_37_0_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,874 INFO [TezTR-518813_1_37_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/698bf3ff-2334-401f-bbb9-449cc735a1c6/map.xml 2018-01-05T11:29:07,874 DEBUG [TezTR-518813_1_37_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:07,874 DEBUG [IO-Elevator-Thread-1] impl.RecordReaderImpl: Row group 0 to 9999 is included. 2018-01-05T11:29:07,875 INFO [TezTR-518813_1_37_1_0_0] reducesink.VectorReduceSinkCommonOperator: Using tag = 1 2018-01-05T11:29:07,875 DEBUG [IO-Elevator-Thread-1] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,875 DEBUG [TezTR-518813_1_37_1_0_0] reducesink.VectorReduceSinkLongOperator: Initialization Done 31 RS 2018-01-05T11:29:07,875 DEBUG [TezTR-518813_1_37_1_0_0] reducesink.VectorReduceSinkLongOperator: Operator 31 RS initialized 2018-01-05T11:29:07,875 DEBUG [TezTR-518813_1_37_1_0_0] reducesink.VectorReduceSinkLongOperator: Initialization Done 31 RS done is reset. 2018-01-05T11:29:07,875 INFO [IO-Elevator-Thread-1] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725460710401, 1515180546000, 827]): [{range start: 79 end: 581}] 2018-01-05T11:29:07,875 DEBUG [TezTR-518813_1_37_1_0_0] vector.VectorSelectOperator: Initialization Done 30 SEL done is reset. 2018-01-05T11:29:07,875 DEBUG [TezTR-518813_1_37_1_0_0] vector.VectorFilterOperator: Initialization Done 29 FIL done is reset. 2018-01-05T11:29:07,875 DEBUG [TezTR-518813_1_37_1_0_0] exec.TableScanOperator: Initialization Done 3 TS done is reset. 2018-01-05T11:29:07,875 INFO [IO-Elevator-Thread-1] encoded.EncodedReaderImpl: Disk ranges after cache (found everything false; file [6462268725460710401, 1515180546000, 827], base offset 3): [{range start: 79 end: 581}] 2018-01-05T11:29:07,875 INFO [TezTR-518813_1_37_1_0_0] exec.OperatorUtils: Setting output collector: RS[31] --> Reducer 2 2018-01-05T11:29:07,875 INFO [TezTR-518813_1_37_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/04343b50-c7d4-4ff5-9f68-f7ee98dcd09d/map.xml 2018-01-05T11:29:07,875 DEBUG [TezTR-518813_1_37_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:07,875 DEBUG [TezTR-518813_1_37_1_0_0] log.PerfLogger: 2018-01-05T11:29:07,875 DEBUG [IO-Elevator-Thread-2] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_37 queryId: djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949 instanceType: OrcEncodedDataReader 2018-01-05T11:29:07,875 DEBUG [TezTR-518813_1_37_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0 2018-01-05T11:29:07,875 INFO [IO-Elevator-Thread-2] LlapIoImpl: Processing data for file [-5122461581151735180, 1515180546000, 744]: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0 2018-01-05T11:29:07,875 DEBUG [IO-Elevator-Thread-2] LlapIoOrc: FileSplit {3, 566}; stripes {3, 566}, 2018-01-05T11:29:07,875 DEBUG [IO-Elevator-Thread-2] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,875 INFO [IO-Elevator-Thread-2] impl.OrcCodecPool: Got brand-new codec ZLIB 2018-01-05T11:29:07,875 DEBUG [IO-Elevator-Thread-2] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,875 INFO [IO-Elevator-Thread-2] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [-5122461581151735180, 1515180546000, 744]): [{range start: 0 end: 73}] 2018-01-05T11:29:07,875 INFO [IO-Elevator-Thread-2] encoded.EncodedReaderImpl: Disk ranges after cache (found everything false; file [-5122461581151735180, 1515180546000, 744], base offset 3): [{range start: 0 end: 73}] 2018-01-05T11:29:07,875 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: int streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 200 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:29:07,875 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 2 columnType: string streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DICTIONARY_V2 dictionarySize: 81 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 131 bytes, ], indexBaseOffset=0] dictionary: true lengths: true secondary: false tz: 2018-01-05T11:29:07,876 DEBUG [IO-Elevator-Thread-1] LlapIoOrc: Done with decode 2018-01-05T11:29:07,876 DEBUG [IO-Elevator-Thread-1] LlapIoImpl: setDone called; closed false, done false, err null, pending 0 2018-01-05T11:29:07,876 DEBUG [IO-Elevator-Thread-1] impl.StatsRecordingThreadPool: Updated stats: instance: OrcEncodedDataReader thread name: IO-Elevator-Thread-1 thread id: 1309 scheme: file bytesRead: 2065 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:07,876 DEBUG [IO-Elevator-Thread-2] impl.RecordReaderImpl: Row group 0 to 9999 is included. 2018-01-05T11:29:07,876 DEBUG [IO-Elevator-Thread-2] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,876 INFO [IO-Elevator-Thread-2] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [-5122461581151735180, 1515180546000, 744]): [{range start: 73 end: 506}] 2018-01-05T11:29:07,876 INFO [IO-Elevator-Thread-2] encoded.EncodedReaderImpl: Disk ranges after cache (found everything false; file [-5122461581151735180, 1515180546000, 744], base offset 3): [{range start: 73 end: 506}] 2018-01-05T11:29:07,876 INFO [TezTR-518813_1_37_1_0_0] reducesink.VectorReduceSinkCommonOperator: RS[31]: records written - 1 2018-01-05T11:29:07,876 INFO [TezTR-518813_1_37_1_0_0] reducesink.VectorReduceSinkCommonOperator: RS[31]: records written - 10 2018-01-05T11:29:07,876 INFO [TezTR-518813_1_37_1_0_0] reducesink.VectorReduceSinkCommonOperator: RS[31]: records written - 100 2018-01-05T11:29:07,876 INFO [TezTR-518813_1_37_1_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:07,876 INFO [TezTR-518813_1_37_1_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, warehouse.tab_part, file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0 ([6462268725460710401, 1515180546000, 827]), 0,1]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=1, NUM_ERRORS=0, ROWS_EMITTED=138, METADATA_CACHE_HIT=0, METADATA_CACHE_MISS=2, CACHE_HIT_BYTES=0, CACHE_MISS_BYTES=581, ALLOCATED_BYTES=1835008, ALLOCATED_USED_BYTES=989, TOTAL_IO_TIME_NS=4805104, DECODE_TIME_NS=433281, HDFS_TIME_NS=2732398, CONSUMER_TIME_NS=549911 ] 2018-01-05T11:29:07,876 DEBUG [TezTR-518813_1_37_1_0_0] encoded.OrcEncodedDataReader: Encoded reader is being stopped 2018-01-05T11:29:07,876 DEBUG [TezTR-518813_1_37_1_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 1 of 4 2018-01-05T11:29:07,876 DEBUG [TezTR-518813_1_37_1_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat]} 2018-01-05T11:29:07,876 DEBUG [TezTR-518813_1_37_1_0_0] io.HiveInputFormat: Pushdown initiated with filterText = key is not null, filterExpr = GenericUDFOPNotNull(Column[key]), serializedFilterExpr = AQEAamF2YS51dGlsLkFycmF5TGlz9AEBAQFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVDb2x1bW5EZXPjAQFrZfkAAAGCYgECb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5zZXJkZTIudHlwZWluZm8uUHJpbWl0aXZlVHlwZUluZu8BAWlu9AEDb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGT1BOb3ROdWzsAQEEb3JnLmFwYWNoZS5oYWRvb3AuaW8uQm9vbGVhbldyaXRhYmzlAQAAAQIBAWJvb2xlYe4= (desc) 2018-01-05T11:29:07,877 INFO [TezTR-518813_1_37_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/698bf3ff-2334-401f-bbb9-449cc735a1c6/map.xml 2018-01-05T11:29:07,877 DEBUG [TezTR-518813_1_37_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:07,877 DEBUG [IO-Elevator-Thread-2] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: int streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 136 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:29:07,877 DEBUG [TezTR-518813_1_37_1_0_0] io.HiveInputFormat: Processing org.apache.hadoop.hive.ql.io.orc.OrcInputFormat 2018-01-05T11:29:07,877 DEBUG [IO-Elevator-Thread-2] encoded.EncodedTreeReaderFactory: columnIndex: 2 columnType: string streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DICTIONARY_V2 dictionarySize: 80 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 89 bytes, ], indexBaseOffset=0] dictionary: true lengths: true secondary: false tz: 2018-01-05T11:29:07,877 DEBUG [IO-Elevator-Thread-2] LlapIoOrc: Done with decode 2018-01-05T11:29:07,877 DEBUG [IO-Elevator-Thread-2] LlapIoImpl: setDone called; closed false, done false, err null, pending 0 2018-01-05T11:29:07,877 DEBUG [IO-Elevator-Thread-2] impl.StatsRecordingThreadPool: Updated stats: instance: OrcEncodedDataReader thread name: IO-Elevator-Thread-2 thread id: 2076 scheme: file bytesRead: 2328 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:07,878 DEBUG [TezTR-518813_1_37_1_0_0] impl.LlapRecordReader: Initializing for input Map 3 2018-01-05T11:29:07,878 INFO [TezTR-518813_1_37_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/698bf3ff-2334-401f-bbb9-449cc735a1c6/map.xml 2018-01-05T11:29:07,878 DEBUG [TezTR-518813_1_37_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:07,878 INFO [TezTR-518813_1_37_1_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_37_01_000000_0 2018-01-05T11:29:07,878 INFO [TezTR-518813_1_37_1_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:07,878 INFO [TezTR-518813_1_37_0_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:07,878 INFO [TezTR-518813_1_37_0_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, warehouse.tab, file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0 ([-5122461581151735180, 1515180546000, 744]), 0,1]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=1, NUM_ERRORS=0, ROWS_EMITTED=122, METADATA_CACHE_HIT=0, METADATA_CACHE_MISS=2, CACHE_HIT_BYTES=0, CACHE_MISS_BYTES=506, ALLOCATED_BYTES=1835008, ALLOCATED_USED_BYTES=876, TOTAL_IO_TIME_NS=2108345, DECODE_TIME_NS=326506, HDFS_TIME_NS=3539719, CONSUMER_TIME_NS=922921 ] 2018-01-05T11:29:07,878 DEBUG [TezTR-518813_1_37_0_0_0] encoded.OrcEncodedDataReader: Encoded reader is being stopped 2018-01-05T11:29:07,878 DEBUG [TezTR-518813_1_37_0_0_0] log.PerfLogger: 2018-01-05T11:29:07,878 DEBUG [TezTR-518813_1_37_0_0_0] mr.ObjectCache: djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:29:07,878 DEBUG [TezTR-518813_1_37_0_0_0] vector.VectorMapOperator: close called for operator MAP[0] 2018-01-05T11:29:07,878 INFO [TezTR-518813_1_37_0_0_0] vector.VectorMapOperator: Closing operator MAP[0] 2018-01-05T11:29:07,878 INFO [TezTR-518813_1_37_0_0_0] vector.VectorMapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:242, 2018-01-05T11:29:07,878 DEBUG [TezTR-518813_1_37_0_0_0] vector.VectorMapOperator: Closing child = TS[0] 2018-01-05T11:29:07,878 DEBUG [TezTR-518813_1_37_0_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:29:07,878 INFO [TezTR-518813_1_37_0_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:29:07,878 DEBUG [TezTR-518813_1_37_0_0_0] exec.TableScanOperator: Closing child = FIL[26] 2018-01-05T11:29:07,878 DEBUG [TezTR-518813_1_37_0_0_0] vector.VectorFilterOperator: close called for operator FIL[26] 2018-01-05T11:29:07,878 DEBUG [TezTR-518813_1_37_0_0_0] vector.VectorFilterOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:07,878 INFO [TezTR-518813_1_37_0_0_0] vector.VectorFilterOperator: Closing operator FIL[26] 2018-01-05T11:29:07,878 DEBUG [TezTR-518813_1_37_0_0_0] vector.VectorFilterOperator: Closing child = SEL[27] 2018-01-05T11:29:07,878 DEBUG [TezTR-518813_1_37_0_0_0] vector.VectorSelectOperator: close called for operator SEL[27] 2018-01-05T11:29:07,878 DEBUG [TezTR-518813_1_37_0_0_0] vector.VectorSelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:07,878 INFO [TezTR-518813_1_37_0_0_0] vector.VectorSelectOperator: Closing operator SEL[27] 2018-01-05T11:29:07,878 DEBUG [TezTR-518813_1_37_0_0_0] vector.VectorSelectOperator: Closing child = RS[28] 2018-01-05T11:29:07,878 DEBUG [TezTR-518813_1_37_0_0_0] reducesink.VectorReduceSinkLongOperator: close called for operator RS[28] 2018-01-05T11:29:07,878 DEBUG [TezTR-518813_1_37_0_0_0] reducesink.VectorReduceSinkLongOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:07,878 INFO [TezTR-518813_1_37_0_0_0] reducesink.VectorReduceSinkLongOperator: Closing operator RS[28] 2018-01-05T11:29:07,878 INFO [TezTR-518813_1_37_0_0_0] reducesink.VectorReduceSinkCommonOperator: RS[28]: records written - 242 2018-01-05T11:29:07,878 INFO [TezTR-518813_1_37_0_0_0] reducesink.VectorReduceSinkLongOperator: RECORDS_OUT_INTERMEDIATE_Map_1:242, 2018-01-05T11:29:07,878 DEBUG [TezTR-518813_1_37_0_0_0] reducesink.VectorReduceSinkLongOperator: 28 Close done 2018-01-05T11:29:07,878 DEBUG [TezTR-518813_1_37_0_0_0] vector.VectorSelectOperator: 27 Close done 2018-01-05T11:29:07,878 DEBUG [TezTR-518813_1_37_0_0_0] vector.VectorFilterOperator: 26 Close done 2018-01-05T11:29:07,878 DEBUG [TezTR-518813_1_37_0_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:29:07,878 DEBUG [TezTR-518813_1_37_0_0_0] vector.VectorMapOperator: 0 Close done 2018-01-05T11:29:07,878 INFO [TezTR-518813_1_37_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_37_00_000000_0 2018-01-05T11:29:07,878 INFO [TezTR-518813_1_37_0_0_0] impl.PipelinedSorter: Reducer 2: Starting flush of map output 2018-01-05T11:29:07,878 INFO [TezTR-518813_1_37_0_0_0] impl.PipelinedSorter: Reducer 2: Span0.length = 242, perItem = 25 2018-01-05T11:29:07,878 DEBUG [TezTR-518813_1_37_1_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,878 INFO [TezTR-518813_1_37_0_0_0] impl.PipelinedSorter: Reducer 2: done sorting span=0, length=242, time=0 2018-01-05T11:29:07,878 INFO [TezTR-518813_1_37_0_0_0] impl.PipelinedSorter: Reducer 2: Heap = SpanIterator<0:241> (span=Span[5242880,6244]), 2018-01-05T11:29:07,878 DEBUG [TezTR-518813_1_37_1_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,879 INFO [TezTR-518813_1_37_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/698bf3ff-2334-401f-bbb9-449cc735a1c6/map.xml 2018-01-05T11:29:07,879 DEBUG [TezTR-518813_1_37_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:07,879 DEBUG [TezTR-518813_1_37_1_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0 2018-01-05T11:29:07,879 DEBUG [IO-Elevator-Thread-3] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_37 queryId: djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949 instanceType: OrcEncodedDataReader 2018-01-05T11:29:07,879 INFO [IO-Elevator-Thread-3] LlapIoImpl: Processing data for file [6462268725462206130, 1515180546000, 823]: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0 2018-01-05T11:29:07,879 DEBUG [IO-Elevator-Thread-3] LlapIoOrc: FileSplit {3, 641}; stripes {3, 641}, 2018-01-05T11:29:07,879 DEBUG [IO-Elevator-Thread-3] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,879 DEBUG [IO-Elevator-Thread-3] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,879 DEBUG [IO-Elevator-Thread-3] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,879 INFO [IO-Elevator-Thread-3] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725462206130, 1515180546000, 823]): [{range start: 0 end: 76}] 2018-01-05T11:29:07,879 INFO [IO-Elevator-Thread-3] encoded.EncodedReaderImpl: Disk ranges after cache (found everything false; file [6462268725462206130, 1515180546000, 823], base offset 3): [{range start: 0 end: 76}] 2018-01-05T11:29:07,880 DEBUG [IO-Elevator-Thread-3] impl.RecordReaderImpl: Row group 0 to 9999 is included. 2018-01-05T11:29:07,880 DEBUG [IO-Elevator-Thread-3] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,880 INFO [IO-Elevator-Thread-3] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725462206130, 1515180546000, 823]): [{range start: 76 end: 581}] 2018-01-05T11:29:07,880 INFO [IO-Elevator-Thread-3] encoded.EncodedReaderImpl: Disk ranges after cache (found everything false; file [6462268725462206130, 1515180546000, 823], base offset 3): [{range start: 76 end: 581}] 2018-01-05T11:29:07,880 DEBUG [IO-Elevator-Thread-3] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: int streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 200 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:29:07,881 DEBUG [IO-Elevator-Thread-3] encoded.EncodedTreeReaderFactory: columnIndex: 2 columnType: string streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DICTIONARY_V2 dictionarySize: 80 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 129 bytes, ], indexBaseOffset=0] dictionary: true lengths: true secondary: false tz: 2018-01-05T11:29:07,881 DEBUG [IO-Elevator-Thread-3] LlapIoOrc: Done with decode 2018-01-05T11:29:07,881 DEBUG [IO-Elevator-Thread-3] LlapIoImpl: setDone called; closed false, done false, err null, pending 0 2018-01-05T11:29:07,881 DEBUG [IO-Elevator-Thread-3] impl.StatsRecordingThreadPool: Updated stats: instance: OrcEncodedDataReader thread name: IO-Elevator-Thread-3 thread id: 2077 scheme: file bytesRead: 2053 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:07,882 INFO [TezTR-518813_1_37_1_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:07,882 INFO [TezTR-518813_1_37_1_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, warehouse.tab_part, file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0 ([6462268725462206130, 1515180546000, 823]), 0,1]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=1, NUM_ERRORS=0, ROWS_EMITTED=128, METADATA_CACHE_HIT=0, METADATA_CACHE_MISS=2, CACHE_HIT_BYTES=0, CACHE_MISS_BYTES=581, ALLOCATED_BYTES=1835008, ALLOCATED_USED_BYTES=983, TOTAL_IO_TIME_NS=1995317, DECODE_TIME_NS=221049, HDFS_TIME_NS=1826410, CONSUMER_TIME_NS=833648 ] 2018-01-05T11:29:07,882 DEBUG [TezTR-518813_1_37_1_0_0] encoded.OrcEncodedDataReader: Encoded reader is being stopped 2018-01-05T11:29:07,882 DEBUG [TezTR-518813_1_37_1_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 2 of 4 2018-01-05T11:29:07,882 DEBUG [TezTR-518813_1_37_1_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat]} 2018-01-05T11:29:07,882 DEBUG [TezTR-518813_1_37_1_0_0] io.HiveInputFormat: Pushdown initiated with filterText = key is not null, filterExpr = GenericUDFOPNotNull(Column[key]), serializedFilterExpr = AQEAamF2YS51dGlsLkFycmF5TGlz9AEBAQFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVDb2x1bW5EZXPjAQFrZfkAAAGCYgECb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5zZXJkZTIudHlwZWluZm8uUHJpbWl0aXZlVHlwZUluZu8BAWlu9AEDb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGT1BOb3ROdWzsAQEEb3JnLmFwYWNoZS5oYWRvb3AuaW8uQm9vbGVhbldyaXRhYmzlAQAAAQIBAWJvb2xlYe4= (desc) 2018-01-05T11:29:07,882 INFO [TezTR-518813_1_37_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/698bf3ff-2334-401f-bbb9-449cc735a1c6/map.xml 2018-01-05T11:29:07,882 DEBUG [TezTR-518813_1_37_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:07,882 DEBUG [TezTR-518813_1_37_1_0_0] io.HiveInputFormat: Processing org.apache.hadoop.hive.ql.io.orc.OrcInputFormat 2018-01-05T11:29:07,883 DEBUG [TezTR-518813_1_37_1_0_0] impl.LlapRecordReader: Initializing for input Map 3 2018-01-05T11:29:07,883 INFO [TezTR-518813_1_37_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/698bf3ff-2334-401f-bbb9-449cc735a1c6/map.xml 2018-01-05T11:29:07,883 DEBUG [TezTR-518813_1_37_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:07,883 INFO [TezTR-518813_1_37_1_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_37_01_000000_0 2018-01-05T11:29:07,883 INFO [TezTR-518813_1_37_1_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:07,884 DEBUG [TezTR-518813_1_37_1_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,884 DEBUG [TezTR-518813_1_37_1_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,884 INFO [TezTR-518813_1_37_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/698bf3ff-2334-401f-bbb9-449cc735a1c6/map.xml 2018-01-05T11:29:07,884 DEBUG [TezTR-518813_1_37_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:07,884 DEBUG [TezTR-518813_1_37_1_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0 2018-01-05T11:29:07,884 DEBUG [IO-Elevator-Thread-4] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_37 queryId: djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949 instanceType: OrcEncodedDataReader 2018-01-05T11:29:07,884 INFO [IO-Elevator-Thread-4] LlapIoImpl: Processing data for file [6462268725463701859, 1515180546000, 770]: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0 2018-01-05T11:29:07,884 DEBUG [IO-Elevator-Thread-4] LlapIoOrc: FileSplit {3, 586}; stripes {3, 586}, 2018-01-05T11:29:07,884 DEBUG [IO-Elevator-Thread-4] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,885 DEBUG [IO-Elevator-Thread-4] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,885 DEBUG [IO-Elevator-Thread-4] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,885 INFO [IO-Elevator-Thread-4] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725463701859, 1515180546000, 770]): [{range start: 0 end: 76}] 2018-01-05T11:29:07,885 INFO [IO-Elevator-Thread-4] encoded.EncodedReaderImpl: Disk ranges after cache (found everything false; file [6462268725463701859, 1515180546000, 770], base offset 3): [{range start: 0 end: 76}] 2018-01-05T11:29:07,885 DEBUG [IO-Elevator-Thread-4] impl.RecordReaderImpl: Row group 0 to 9999 is included. 2018-01-05T11:29:07,885 DEBUG [IO-Elevator-Thread-4] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,886 INFO [IO-Elevator-Thread-4] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725463701859, 1515180546000, 770]): [{range start: 76 end: 526}] 2018-01-05T11:29:07,886 INFO [IO-Elevator-Thread-4] encoded.EncodedReaderImpl: Disk ranges after cache (found everything false; file [6462268725463701859, 1515180546000, 770], base offset 3): [{range start: 76 end: 526}] 2018-01-05T11:29:07,886 DEBUG [IO-Elevator-Thread-4] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: int streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 171 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:29:07,886 DEBUG [IO-Elevator-Thread-4] encoded.EncodedTreeReaderFactory: columnIndex: 2 columnType: string streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DICTIONARY_V2 dictionarySize: 73 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 110 bytes, ], indexBaseOffset=0] dictionary: true lengths: true secondary: false tz: 2018-01-05T11:29:07,886 DEBUG [IO-Elevator-Thread-4] LlapIoOrc: Done with decode 2018-01-05T11:29:07,886 DEBUG [IO-Elevator-Thread-4] LlapIoImpl: setDone called; closed false, done false, err null, pending 0 2018-01-05T11:29:07,886 DEBUG [IO-Elevator-Thread-4] impl.StatsRecordingThreadPool: Updated stats: instance: OrcEncodedDataReader thread name: IO-Elevator-Thread-4 thread id: 2194 scheme: file bytesRead: 1894 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:07,887 INFO [TezTR-518813_1_37_1_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:07,887 INFO [TezTR-518813_1_37_1_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, warehouse.tab_part, file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0 ([6462268725463701859, 1515180546000, 770]), 0,1]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=1, NUM_ERRORS=0, ROWS_EMITTED=115, METADATA_CACHE_HIT=0, METADATA_CACHE_MISS=2, CACHE_HIT_BYTES=0, CACHE_MISS_BYTES=526, ALLOCATED_BYTES=1835008, ALLOCATED_USED_BYTES=890, TOTAL_IO_TIME_NS=2090693, DECODE_TIME_NS=311170, HDFS_TIME_NS=1899621, CONSUMER_TIME_NS=501639 ] 2018-01-05T11:29:07,887 DEBUG [TezTR-518813_1_37_1_0_0] encoded.OrcEncodedDataReader: Encoded reader is being stopped 2018-01-05T11:29:07,887 DEBUG [TezTR-518813_1_37_1_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 3 of 4 2018-01-05T11:29:07,887 DEBUG [TezTR-518813_1_37_1_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat]} 2018-01-05T11:29:07,887 DEBUG [TezTR-518813_1_37_1_0_0] io.HiveInputFormat: Pushdown initiated with filterText = key is not null, filterExpr = GenericUDFOPNotNull(Column[key]), serializedFilterExpr = AQEAamF2YS51dGlsLkFycmF5TGlz9AEBAQFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVDb2x1bW5EZXPjAQFrZfkAAAGCYgECb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5zZXJkZTIudHlwZWluZm8uUHJpbWl0aXZlVHlwZUluZu8BAWlu9AEDb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGT1BOb3ROdWzsAQEEb3JnLmFwYWNoZS5oYWRvb3AuaW8uQm9vbGVhbldyaXRhYmzlAQAAAQIBAWJvb2xlYe4= (desc) 2018-01-05T11:29:07,887 INFO [TezTR-518813_1_37_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/698bf3ff-2334-401f-bbb9-449cc735a1c6/map.xml 2018-01-05T11:29:07,887 DEBUG [TezTR-518813_1_37_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:07,887 DEBUG [TezTR-518813_1_37_1_0_0] io.HiveInputFormat: Processing org.apache.hadoop.hive.ql.io.orc.OrcInputFormat 2018-01-05T11:29:07,888 DEBUG [TezTR-518813_1_37_1_0_0] impl.LlapRecordReader: Initializing for input Map 3 2018-01-05T11:29:07,888 INFO [TezTR-518813_1_37_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/698bf3ff-2334-401f-bbb9-449cc735a1c6/map.xml 2018-01-05T11:29:07,888 DEBUG [TezTR-518813_1_37_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:07,888 INFO [TezTR-518813_1_37_1_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_37_01_000000_0 2018-01-05T11:29:07,888 INFO [TezTR-518813_1_37_1_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:07,889 DEBUG [TezTR-518813_1_37_1_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,889 DEBUG [TezTR-518813_1_37_1_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,889 INFO [TezTR-518813_1_37_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/698bf3ff-2334-401f-bbb9-449cc735a1c6/map.xml 2018-01-05T11:29:07,889 DEBUG [TezTR-518813_1_37_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:07,889 DEBUG [TezTR-518813_1_37_1_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0 2018-01-05T11:29:07,889 DEBUG [IO-Elevator-Thread-5] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_37 queryId: djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949 instanceType: OrcEncodedDataReader 2018-01-05T11:29:07,889 INFO [IO-Elevator-Thread-5] LlapIoImpl: Processing data for file [6462268725459214672, 1515180546000, 752]: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0 2018-01-05T11:29:07,889 DEBUG [IO-Elevator-Thread-5] LlapIoOrc: FileSplit {3, 574}; stripes {3, 574}, 2018-01-05T11:29:07,890 DEBUG [IO-Elevator-Thread-5] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,890 DEBUG [IO-Elevator-Thread-5] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,890 DEBUG [IO-Elevator-Thread-5] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,890 INFO [IO-Elevator-Thread-5] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725459214672, 1515180546000, 752]): [{range start: 0 end: 73}] 2018-01-05T11:29:07,890 INFO [IO-Elevator-Thread-5] encoded.EncodedReaderImpl: Disk ranges after cache (found everything false; file [6462268725459214672, 1515180546000, 752], base offset 3): [{range start: 0 end: 73}] 2018-01-05T11:29:07,891 DEBUG [IO-Elevator-Thread-5] impl.RecordReaderImpl: Row group 0 to 9999 is included. 2018-01-05T11:29:07,891 DEBUG [IO-Elevator-Thread-5] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:07,891 INFO [IO-Elevator-Thread-5] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725459214672, 1515180546000, 752]): [{range start: 73 end: 514}] 2018-01-05T11:29:07,891 INFO [IO-Elevator-Thread-5] encoded.EncodedReaderImpl: Disk ranges after cache (found everything false; file [6462268725459214672, 1515180546000, 752], base offset 3): [{range start: 73 end: 514}] 2018-01-05T11:29:07,891 DEBUG [IO-Elevator-Thread-5] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: int streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 150 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:29:07,891 DEBUG [IO-Elevator-Thread-5] encoded.EncodedTreeReaderFactory: columnIndex: 2 columnType: string streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DICTIONARY_V2 dictionarySize: 75 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 104 bytes, ], indexBaseOffset=0] dictionary: true lengths: true secondary: false tz: 2018-01-05T11:29:07,892 DEBUG [IO-Elevator-Thread-5] LlapIoOrc: Done with decode 2018-01-05T11:29:07,892 DEBUG [IO-Elevator-Thread-5] LlapIoImpl: setDone called; closed false, done false, err null, pending 0 2018-01-05T11:29:07,892 DEBUG [IO-Elevator-Thread-5] impl.StatsRecordingThreadPool: Updated stats: instance: OrcEncodedDataReader thread name: IO-Elevator-Thread-5 thread id: 2196 scheme: file bytesRead: 1840 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:07,892 INFO [TezTR-518813_1_37_1_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:07,892 INFO [TezTR-518813_1_37_1_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, warehouse.tab_part, file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0 ([6462268725459214672, 1515180546000, 752]), 0,1]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=1, NUM_ERRORS=0, ROWS_EMITTED=119, METADATA_CACHE_HIT=0, METADATA_CACHE_MISS=2, CACHE_HIT_BYTES=0, CACHE_MISS_BYTES=514, ALLOCATED_BYTES=1835008, ALLOCATED_USED_BYTES=872, TOTAL_IO_TIME_NS=2336681, DECODE_TIME_NS=545599, HDFS_TIME_NS=1837889, CONSUMER_TIME_NS=395181 ] 2018-01-05T11:29:07,892 DEBUG [TezTR-518813_1_37_1_0_0] encoded.OrcEncodedDataReader: Encoded reader is being stopped 2018-01-05T11:29:07,892 DEBUG [TezTR-518813_1_37_1_0_0] log.PerfLogger: 2018-01-05T11:29:07,892 DEBUG [TezTR-518813_1_37_1_0_0] mr.ObjectCache: djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949_Map 3__MAP_PLAN__ no longer needed 2018-01-05T11:29:07,892 DEBUG [TezTR-518813_1_37_1_0_0] vector.VectorMapOperator: close called for operator MAP[0] 2018-01-05T11:29:07,892 INFO [TezTR-518813_1_37_1_0_0] vector.VectorMapOperator: Closing operator MAP[0] 2018-01-05T11:29:07,892 INFO [TezTR-518813_1_37_1_0_0] vector.VectorMapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_3:500, 2018-01-05T11:29:07,892 DEBUG [TezTR-518813_1_37_1_0_0] vector.VectorMapOperator: Closing child = TS[3] 2018-01-05T11:29:07,892 DEBUG [TezTR-518813_1_37_1_0_0] exec.TableScanOperator: close called for operator TS[3] 2018-01-05T11:29:07,892 INFO [TezTR-518813_1_37_1_0_0] exec.TableScanOperator: Closing operator TS[3] 2018-01-05T11:29:07,892 DEBUG [TezTR-518813_1_37_1_0_0] exec.TableScanOperator: Closing child = FIL[29] 2018-01-05T11:29:07,892 DEBUG [TezTR-518813_1_37_1_0_0] vector.VectorFilterOperator: close called for operator FIL[29] 2018-01-05T11:29:07,892 DEBUG [TezTR-518813_1_37_1_0_0] vector.VectorFilterOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:07,892 INFO [TezTR-518813_1_37_1_0_0] vector.VectorFilterOperator: Closing operator FIL[29] 2018-01-05T11:29:07,892 DEBUG [TezTR-518813_1_37_1_0_0] vector.VectorFilterOperator: Closing child = SEL[30] 2018-01-05T11:29:07,892 DEBUG [TezTR-518813_1_37_1_0_0] vector.VectorSelectOperator: close called for operator SEL[30] 2018-01-05T11:29:07,892 DEBUG [TezTR-518813_1_37_1_0_0] vector.VectorSelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:07,892 INFO [TezTR-518813_1_37_1_0_0] vector.VectorSelectOperator: Closing operator SEL[30] 2018-01-05T11:29:07,892 DEBUG [TezTR-518813_1_37_1_0_0] vector.VectorSelectOperator: Closing child = RS[31] 2018-01-05T11:29:07,892 DEBUG [TezTR-518813_1_37_1_0_0] reducesink.VectorReduceSinkLongOperator: close called for operator RS[31] 2018-01-05T11:29:07,892 DEBUG [TezTR-518813_1_37_1_0_0] reducesink.VectorReduceSinkLongOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:07,892 INFO [TezTR-518813_1_37_1_0_0] reducesink.VectorReduceSinkLongOperator: Closing operator RS[31] 2018-01-05T11:29:07,892 INFO [TezTR-518813_1_37_1_0_0] reducesink.VectorReduceSinkCommonOperator: RS[31]: records written - 500 2018-01-05T11:29:07,892 INFO [TezTR-518813_1_37_1_0_0] reducesink.VectorReduceSinkLongOperator: RECORDS_OUT_INTERMEDIATE_Map_3:500, 2018-01-05T11:29:07,892 DEBUG [TezTR-518813_1_37_1_0_0] reducesink.VectorReduceSinkLongOperator: 31 Close done 2018-01-05T11:29:07,892 DEBUG [TezTR-518813_1_37_1_0_0] vector.VectorSelectOperator: 30 Close done 2018-01-05T11:29:07,892 DEBUG [TezTR-518813_1_37_1_0_0] vector.VectorFilterOperator: 29 Close done 2018-01-05T11:29:07,892 DEBUG [TezTR-518813_1_37_1_0_0] exec.TableScanOperator: 3 Close done 2018-01-05T11:29:07,892 DEBUG [TezTR-518813_1_37_1_0_0] vector.VectorMapOperator: 0 Close done 2018-01-05T11:29:07,892 INFO [TezTR-518813_1_37_1_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_37_01_000000_0 2018-01-05T11:29:07,892 INFO [TezTR-518813_1_37_1_0_0] impl.PipelinedSorter: Reducer 2: Starting flush of map output 2018-01-05T11:29:07,892 INFO [TezTR-518813_1_37_1_0_0] impl.PipelinedSorter: Reducer 2: Span0.length = 500, perItem = 25 2018-01-05T11:29:07,893 INFO [TezTR-518813_1_37_1_0_0] impl.PipelinedSorter: Reducer 2: done sorting span=0, length=500, time=1 2018-01-05T11:29:07,893 INFO [TezTR-518813_1_37_1_0_0] impl.PipelinedSorter: Reducer 2: Heap = SpanIterator<0:499> (span=Span[5242880,12906]), 2018-01-05T11:29:07,895 INFO [TezTR-518813_1_37_0_0_0] impl.PipelinedSorter: Reducer 2: Spilling to /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/37/output/attempt_1515180518813_0001_37_00_000000_0_10264_0/file.out 2018-01-05T11:29:07,904 INFO [TezTR-518813_1_37_1_0_0] impl.PipelinedSorter: Reducer 2: Spilling to /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/37/output/attempt_1515180518813_0001_37_01_000000_0_10267_0/file.out 2018-01-05T11:29:07,908 DEBUG [TezTR-518813_1_37_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:07,908 INFO [TezTR-518813_1_37_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:07,908 INFO [TezTR-518813_1_37_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_37_00_000000_0, askedToStop=false 2018-01-05T11:29:07,908 INFO [TezTR-518813_1_37_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_37_00_000000_0, stopRequested=false 2018-01-05T11:29:07,908 INFO [TezTR-518813_1_37_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_37_00_000000_0: Counters: 33 [[File System Counters FILE_BYTES_READ=4213, FILE_BYTES_WRITTEN=0, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=242, INPUT_RECORDS_PROCESSED=2, INPUT_SPLIT_LENGTH_BYTES=1149, OUTPUT_RECORDS=242, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=6244, OUTPUT_BYTES_WITH_OVERHEAD=6239, OUTPUT_BYTES_PHYSICAL=6243, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=242, RECORDS_OUT_INTERMEDIATE_Map_1=242][org.apache.hadoop.hive.llap.counters.LlapIOCounters ALLOCATED_BYTES=3670016, ALLOCATED_USED_BYTES=1740, CACHE_HIT_BYTES=0, CACHE_MISS_BYTES=1029, CONSUMER_TIME_NS=1970758, DECODE_TIME_NS=674789, HDFS_TIME_NS=6033877, METADATA_CACHE_MISS=4, NUM_DECODED_BATCHES=2, NUM_VECTOR_BATCHES=2, ROWS_EMITTED=242, SELECTED_ROWGROUPS=2, TOTAL_IO_TIME_NS=7279052]] 2018-01-05T11:29:07,909 DEBUG [TezTR-518813_1_37_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_37_0_0_0 thread id: 2532 scheme: file bytesRead: 1639 bytesWritten: 6275 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:07,909 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_37_00_000000_0, isTaskDone=true 2018-01-05T11:29:07,909 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000087, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_37_00_000000_0, eventCount=4 } 2018-01-05T11:29:07,909 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_37][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_37_00_000000_0, creationTime=1515180547835, allocationTime=1515180547836, startTime=1515180547838, finishTime=1515180547909, timeTaken=71, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:07,909 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:29:07,909 INFO [Task-Executor-2] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_37_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:07,909 INFO [Task-Executor-2] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_37_00_000000_0 2018-01-05T11:29:07,909 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_37_00_000000_0 2018-01-05T11:29:07,909 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_37_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:07,910 INFO [Task-Executor-2] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000087=59 2018-01-05T11:29:07,910 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_37_00_000000_0: true 2018-01-05T11:29:07,910 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_37_00_000000_0; false 2018-01-05T11:29:07,910 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_37_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:07,910 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_37_00_000000_0 2018-01-05T11:29:07,910 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:07,910 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000087, QueryId=djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949-37, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:07,910 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_37][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_37_00_000000, startTime=1515180547838, finishTime=1515180547909, timeTaken=71, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_37_00_000000_0, diagnostics=, counters=Counters: 34, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=5852, FILE_BYTES_WRITTEN=6275, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=242, INPUT_RECORDS_PROCESSED=2, INPUT_SPLIT_LENGTH_BYTES=1149, OUTPUT_RECORDS=242, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=6244, OUTPUT_BYTES_WITH_OVERHEAD=6239, OUTPUT_BYTES_PHYSICAL=6243, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=242, RECORDS_OUT_INTERMEDIATE_Map_1=242, org.apache.hadoop.hive.llap.counters.LlapIOCounters, ALLOCATED_BYTES=3670016, ALLOCATED_USED_BYTES=1740, CACHE_HIT_BYTES=0, CACHE_MISS_BYTES=1029, CONSUMER_TIME_NS=1970758, DECODE_TIME_NS=674789, HDFS_TIME_NS=6033877, METADATA_CACHE_MISS=4, NUM_DECODED_BATCHES=2, NUM_VECTOR_BATCHES=2, ROWS_EMITTED=242, SELECTED_ROWGROUPS=2, TOTAL_IO_TIME_NS=7279052 2018-01-05T11:29:07,910 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:07,910 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_37_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=3, preemptionQueueSize=1 2018-01-05T11:29:07,910 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_37_00_000000_0 2018-01-05T11:29:07,910 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000087, QueryId=djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949-37, DagId=37, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-2, Succeeded=true, StartTime=1515180547850, EndTime=1515180547910 2018-01-05T11:29:07,910 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_37_00 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:07,910 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_37_00 [Map 1] 2018-01-05T11:29:07,911 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_37][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_37_00, initRequestedTime=1515180547793, initedTime=1515180547835, startRequestedTime=1515180547800, startedTime=1515180547835, finishTime=1515180547910, timeTaken=75, status=SUCCEEDED, diagnostics=, counters=Counters: 34, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=5852, FILE_BYTES_WRITTEN=6275, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=242, INPUT_RECORDS_PROCESSED=2, INPUT_SPLIT_LENGTH_BYTES=1149, OUTPUT_RECORDS=242, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=6244, OUTPUT_BYTES_WITH_OVERHEAD=6239, OUTPUT_BYTES_PHYSICAL=6243, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=242, RECORDS_OUT_INTERMEDIATE_Map_1=242, org.apache.hadoop.hive.llap.counters.LlapIOCounters, ALLOCATED_BYTES=3670016, ALLOCATED_USED_BYTES=1740, CACHE_HIT_BYTES=0, CACHE_MISS_BYTES=1029, CONSUMER_TIME_NS=1970758, DECODE_TIME_NS=674789, HDFS_TIME_NS=6033877, METADATA_CACHE_MISS=4, NUM_DECODED_BATCHES=2, NUM_VECTOR_BATCHES=2, ROWS_EMITTED=242, SELECTED_ROWGROUPS=2, TOTAL_IO_TIME_NS=7279052, vertexStats=firstTaskStartTime=1515180547838, firstTasksToStart=[ task_1515180518813_0001_37_00_000000 ], lastTaskFinishTime=1515180547909, lastTasksToFinish=[ task_1515180518813_0001_37_00_000000 ], minTaskDuration=71, maxTaskDuration=71, avgTaskDuration=71.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_37_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_37_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:07,911 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_37_00 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:07,911 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_37][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000087, stoppedTime=1515180547911, exitStatus=0 2018-01-05T11:29:07,911 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000087 exited with diagnostics set to Container completed. 2018-01-05T11:29:07,911 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_37_02 [Reducer 2] attempt: attempt_1515180518813_0001_37_00_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:29:07,911 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_37_00 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3 2018-01-05T11:29:07,911 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null 2018-01-05T11:29:07,911 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000087 2018-01-05T11:29:07,917 DEBUG [TezTR-518813_1_37_1_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:07,917 INFO [TezTR-518813_1_37_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:07,917 INFO [TezTR-518813_1_37_1_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_37_01_000000_0, askedToStop=false 2018-01-05T11:29:07,917 INFO [TezTR-518813_1_37_1_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_37_01_000000_0, stopRequested=false 2018-01-05T11:29:07,917 INFO [TezTR-518813_1_37_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_37_01_000000_0: Counters: 33 [[File System Counters FILE_BYTES_READ=7852, FILE_BYTES_WRITTEN=0, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=500, INPUT_RECORDS_PROCESSED=4, INPUT_SPLIT_LENGTH_BYTES=2442, OUTPUT_RECORDS=500, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=12906, OUTPUT_BYTES_WITH_OVERHEAD=12847, OUTPUT_BYTES_PHYSICAL=12851, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_3=500, RECORDS_OUT_INTERMEDIATE_Map_3=500][org.apache.hadoop.hive.llap.counters.LlapIOCounters ALLOCATED_BYTES=7340032, ALLOCATED_USED_BYTES=3734, CACHE_HIT_BYTES=0, CACHE_MISS_BYTES=2202, CONSUMER_TIME_NS=2280379, DECODE_TIME_NS=1511099, HDFS_TIME_NS=8296318, METADATA_CACHE_MISS=8, NUM_DECODED_BATCHES=4, NUM_VECTOR_BATCHES=4, ROWS_EMITTED=500, SELECTED_ROWGROUPS=4, TOTAL_IO_TIME_NS=11227795]] 2018-01-05T11:29:07,917 DEBUG [TezTR-518813_1_37_1_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_37_1_0_0 thread id: 2540 scheme: file bytesRead: 3364 bytesWritten: 12883 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:07,917 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_37_01_000000_0, isTaskDone=true 2018-01-05T11:29:07,917 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000088, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_37_01_000000_0, eventCount=4 } 2018-01-05T11:29:07,918 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_37][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 3, taskAttemptId=attempt_1515180518813_0001_37_01_000000_0, creationTime=1515180547855, allocationTime=1515180547856, startTime=1515180547856, finishTime=1515180547918, timeTaken=62, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:07,918 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_37_01_000000_0 2018-01-05T11:29:07,918 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:29:07,918 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_37_01_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:07,918 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_37_01_000000_0; false 2018-01-05T11:29:07,918 INFO [Task-Executor-1] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_37_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:07,918 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_37_01_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:07,918 INFO [Task-Executor-1] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_37_01_000000_0 2018-01-05T11:29:07,918 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:07,918 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_37][Event:TASK_FINISHED]: vertexName=Map 3, taskId=task_1515180518813_0001_37_01_000000, startTime=1515180547856, finishTime=1515180547918, timeTaken=62, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_37_01_000000_0, diagnostics=, counters=Counters: 34, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=11216, FILE_BYTES_WRITTEN=12883, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=500, INPUT_RECORDS_PROCESSED=4, INPUT_SPLIT_LENGTH_BYTES=2442, OUTPUT_RECORDS=500, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=12906, OUTPUT_BYTES_WITH_OVERHEAD=12847, OUTPUT_BYTES_PHYSICAL=12851, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_3=500, RECORDS_OUT_INTERMEDIATE_Map_3=500, org.apache.hadoop.hive.llap.counters.LlapIOCounters, ALLOCATED_BYTES=7340032, ALLOCATED_USED_BYTES=3734, CACHE_HIT_BYTES=0, CACHE_MISS_BYTES=2202, CONSUMER_TIME_NS=2280379, DECODE_TIME_NS=1511099, HDFS_TIME_NS=8296318, METADATA_CACHE_MISS=8, NUM_DECODED_BATCHES=4, NUM_VECTOR_BATCHES=4, ROWS_EMITTED=500, SELECTED_ROWGROUPS=4, TOTAL_IO_TIME_NS=11227795 2018-01-05T11:29:07,918 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:07,918 INFO [Task-Executor-1] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000088=60 2018-01-05T11:29:07,918 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_37_01_000000_0: true 2018-01-05T11:29:07,918 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_37_01_000000_0 2018-01-05T11:29:07,918 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000088, QueryId=djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949-37, Vertex=Map 3, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:07,918 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_37_01_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:29:07,918 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_37_01_000000_0 2018-01-05T11:29:07,918 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_37_01 [Map 3], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:07,918 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_37_01 [Map 3] 2018-01-05T11:29:07,919 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_37][Event:VERTEX_FINISHED]: vertexName=Map 3, vertexId=vertex_1515180518813_0001_37_01, initRequestedTime=1515180547785, initedTime=1515180547854, startRequestedTime=1515180547800, startedTime=1515180547854, finishTime=1515180547918, timeTaken=64, status=SUCCEEDED, diagnostics=, counters=Counters: 34, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=11216, FILE_BYTES_WRITTEN=12883, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=500, INPUT_RECORDS_PROCESSED=4, INPUT_SPLIT_LENGTH_BYTES=2442, OUTPUT_RECORDS=500, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=12906, OUTPUT_BYTES_WITH_OVERHEAD=12847, OUTPUT_BYTES_PHYSICAL=12851, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_3=500, RECORDS_OUT_INTERMEDIATE_Map_3=500, org.apache.hadoop.hive.llap.counters.LlapIOCounters, ALLOCATED_BYTES=7340032, ALLOCATED_USED_BYTES=3734, CACHE_HIT_BYTES=0, CACHE_MISS_BYTES=2202, CONSUMER_TIME_NS=2280379, DECODE_TIME_NS=1511099, HDFS_TIME_NS=8296318, METADATA_CACHE_MISS=8, NUM_DECODED_BATCHES=4, NUM_VECTOR_BATCHES=4, ROWS_EMITTED=500, SELECTED_ROWGROUPS=4, TOTAL_IO_TIME_NS=11227795, vertexStats=firstTaskStartTime=1515180547856, firstTasksToStart=[ task_1515180518813_0001_37_01_000000 ], lastTaskFinishTime=1515180547918, lastTasksToFinish=[ task_1515180518813_0001_37_01_000000 ], minTaskDuration=62, maxTaskDuration=62, avgTaskDuration=62.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_37_01_000000 ], longestDurationTasks=[ task_1515180518813_0001_37_01_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:07,919 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_37_01 [Map 3] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:07,919 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_37][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000088, stoppedTime=1515180547919, exitStatus=0 2018-01-05T11:29:07,919 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000088 exited with diagnostics set to Container completed. 2018-01-05T11:29:07,919 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_37_02 [Reducer 2] attempt: attempt_1515180518813_0001_37_01_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:29:07,919 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_37_01 [Map 3] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3 2018-01-05T11:29:07,919 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null 2018-01-05T11:29:07,919 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000088 2018-01-05T11:29:07,919 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: All source tasks completed. Ramping up 1 remaining tasks for vertex: Reducer 2 2018-01-05T11:29:07,919 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: Scheduling 1 tasks for vertex: Reducer 2 with totalTasks: 1. 2 source tasks completed out of 2. MinSourceTaskCompletedFraction: 1.0 min: 0.05 max: 0.75 2018-01-05T11:29:07,919 INFO [App Shared Pool - #2] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Map 1, destinationVertexName=Reducer 2. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:29:07,919 INFO [App Shared Pool - #2] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Map 3, destinationVertexName=Reducer 2. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:29:07,919 INFO [App Shared Pool - #2] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_37_02 [Reducer 2] 2018-01-05T11:29:07,919 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_37][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_37_02_000000, scheduledTime=1515180547919, launchTime=1515180547919 2018-01-05T11:29:07,919 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_37_02 [Reducer 2] 2018-01-05T11:29:07,919 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_37_02_000000_0, priority=23, capability=, hosts=null 2018-01-05T11:29:07,919 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=23,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:07,919 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=23,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:07,919 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:07,919 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_37_02_000000_0 on hosts=null 2018-01-05T11:29:07,919 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_37_02_000000_0 on hosts=null 2018-01-05T11:29:07,919 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_37_02_000000_0 2018-01-05T11:29:07,919 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:07,919 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #89, task=TaskInfo{task=attempt_1515180518813_0001_37_02_000000_0, priority=23, startTime=0, containerId=null, uniqueId=88, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000089 2018-01-05T11:29:07,919 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_37_02_000000_0; false 2018-01-05T11:29:07,919 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_37_02_000000_0, priority=23, startTime=311568429, containerId=container_111101111_0001_00_000089assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=88, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:07,920 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000089 for node: 10.22.8.182:49818 2018-01-05T11:29:07,920 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000089, attempt_1515180518813_0001_37_02_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:07,920 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000089 for node: 10.22.8.182:49818 2018-01-05T11:29:07,920 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000089 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:07,920 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000089, launchTime=1515180547920 2018-01-05T11:29:07,920 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=37}, sourceName=Map 1, state=S_SUCCEEDED 2018-01-05T11:29:07,920 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=37}, sourceName=Map 3, state=S_SUCCEEDED 2018-01-05T11:29:07,920 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_37_02_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000089] on NM: [10.22.8.182:49818] 2018-01-05T11:29:07,920 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:07,920 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:07,920 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_37][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_37_02_000000_0, startTime=1515180547920, containerId=container_111101111_0001_00_000089, nodeId=10.22.8.182:49818 2018-01-05T11:29:07,921 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_37_02_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000089, dagName=select * from tab a join tab_part b ...b.key (Stage-1), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=2, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.OrderedGroupedKVInput,1}{Map 3,org.apache.tez.runtime.library.input.OrderedGroupedKVInput,1}}, Outputs={{out_Reducer 2,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=3, completedTaskCount=2, dagStartTime=1515180547784, firstAttemptStartTime=1515180547919, currentAttemptStartTime=1515180547920} 2018-01-05T11:29:07,921 INFO [IPC Server handler 2 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000089, QueryId=djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949-37, DagId=37, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, SubmitTime=1515180547921 2018-01-05T11:29:07,921 DEBUG [IPC Server handler 2 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=37} with the ShuffleHandler 2018-01-05T11:29:07,921 DEBUG [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/37] 2018-01-05T11:29:07,921 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:29:07,921 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_37_02_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:07,921 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_37_02_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180547919, dagStartTime=1515180547784, withinDagPriority=23, vertexParallelism= 1, selfAndUpstreamParallelism= 3, selfAndUpstreamComplete= 2}] 2018-01-05T11:29:07,921 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_37_02_000000_0 : ACCEPTED 2018-01-05T11:29:07,921 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_37_02_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:29:07,921 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_37_02_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180547919, dagStartTime=1515180547784, withinDagPriority=23, vertexParallelism= 1, selfAndUpstreamParallelism= 3, selfAndUpstreamComplete= 2} 2018-01-05T11:29:07,921 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_37_02_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:07,921 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_37_02_000000_0 to pre-emption queue 2018-01-05T11:29:07,922 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_37_02_000000_0: true 2018-01-05T11:29:07,922 INFO [TaskCommunicator # 2] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_37_02_000000_0 2018-01-05T11:29:07,922 INFO [TaskCommunicator # 2] GuaranteedTasks: Received response for attempt_1515180518813_0001_37_02_000000_0, true 2018-01-05T11:29:07,922 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: taskOwner hashCode:1599253873 2018-01-05T11:29:07,922 INFO [Task-Executor-1] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : select * from tab a join tab_part b ...b.key (Stage-1), VertexName: Reducer 2, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_37_02_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=Map 3, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:29:07,922 INFO [Task-Executor-1] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=2, numOutputs=1, JVM.maxFree=1021837312, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:07,922 INFO [Task-Executor-1] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_37_02_000000_0 vertexName: Reducer 2 2018-01-05T11:29:07,922 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000089, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_37_02_000000_0, eventCount=1 } 2018-01-05T11:29:07,922 DEBUG [TezTR-518813_1_37_2_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_37 queryId: djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949 instanceType: TaskRunner2Callable 2018-01-05T11:29:07,922 INFO [TezTR-518813_1_37_2_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_37_02_000000_0 2018-01-05T11:29:07,923 DEBUG [TezTR-518813_1_37_2_0_0] log.PerfLogger: 2018-01-05T11:29:07,923 INFO [I/O Setup 0 Initialize: {Map 1}] orderedgrouped.MergeManager: Initial Memory required for SHUFFLE_BUFFER=131404920 based on INPUT_BUFFER_FACTOR=0.7, for final merged output=0, using factor: 0.0 2018-01-05T11:29:07,923 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:07,923 INFO [I/O Setup 1 Initialize: {Map 3}] orderedgrouped.MergeManager: Initial Memory required for SHUFFLE_BUFFER=131404920 based on INPUT_BUFFER_FACTOR=0.7, for final merged output=0, using factor: 0.0 2018-01-05T11:29:07,923 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:07,924 INFO [IPC Server handler 13 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_37_02_000000_0 2 events [0,2) total 2 vertex_1515180518813_0001_37_02 [Reducer 2] 2018-01-05T11:29:07,924 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=2, nextPreRoutedEventId=0, eventCount=2 } 2018-01-05T11:29:07,924 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_37_02_000000_0, eventCount=2 fromEventId=0 nextFromEventId=2 2018-01-05T11:29:07,925 DEBUG [TezTR-518813_1_37_2_0_0] log.PerfLogger: 2018-01-05T11:29:07,925 INFO [TezTR-518813_1_37_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:07,925 INFO [TezTR-518813_1_37_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 3 initializers to finish 2018-01-05T11:29:07,925 INFO [TezTR-518813_1_37_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:07,925 INFO [TezTR-518813_1_37_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:07,925 INFO [I/O Setup 2 Initialize: {out_Reducer 2}] output.MROutput: out_Reducer 2: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:29:07,925 DEBUG [I/O Setup 2 Initialize: {out_Reducer 2}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:07,925 DEBUG [I/O Setup 2 Initialize: {out_Reducer 2}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:07,925 DEBUG [I/O Setup 2 Initialize: {out_Reducer 2}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:07,925 DEBUG [I/O Setup 2 Initialize: {out_Reducer 2}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:07,925 INFO [TezTR-518813_1_37_2_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:07,925 INFO [TezTR-518813_1_37_2_0_0] resources.MemoryDistributor: InitialRequests=[out_Reducer 2:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [Map 3:INPUT:131404920:org.apache.tez.runtime.library.input.OrderedGroupedKVInput] 2018-01-05T11:29:07,925 INFO [TezTR-518813_1_37_2_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:07,925 INFO [TezTR-518813_1_37_2_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.045, finalReserveFractionUsed=0.345 2018-01-05T11:29:07,925 INFO [TezTR-518813_1_37_2_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 3, numScaledRequests: 25, TotalRequested: 262809840, TotalRequestedScaled: 1.2614872319999999E8, TotalJVMHeap: 1021837312, TotalAvailable: 669303439, TotalRequested/TotalJVMHeap:0.26 2018-01-05T11:29:07,925 INFO [TezTR-518813_1_37_2_0_0] resources.MemoryDistributor: Allocations=[out_Reducer 2:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [Map 1:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:131404920:131404920], [Map 3:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:131404920:131404920] 2018-01-05T11:29:07,925 INFO [TezTR-518813_1_37_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:07,925 INFO [TezTR-518813_1_37_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: Map 1 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:07,925 INFO [TezTR-518813_1_37_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: Map 3 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:07,925 INFO [TezTR-518813_1_37_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 2 2018-01-05T11:29:07,926 INFO [TezTR-518813_1_37_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 IOs to start 2018-01-05T11:29:07,926 INFO [I/O Setup 1 Start: {Map 3}] orderedgrouped.Shuffle: Map_3: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true 2018-01-05T11:29:07,926 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.Shuffle: Map_1: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true 2018-01-05T11:29:07,926 INFO [I/O Setup 1 Start: {Map 3}] orderedgrouped.MergeManager: Map 3: MergerManager: memoryLimit=131404920, maxSingleShuffleLimit=32851230, mergeThreshold=86727248, ioSortFactor=10, postMergeMem=0, memToMemMergeOutputsThreshold=10 2018-01-05T11:29:07,926 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.MergeManager: Map 1: MergerManager: memoryLimit=131404920, maxSingleShuffleLimit=32851230, mergeThreshold=86727248, ioSortFactor=10, postMergeMem=0, memToMemMergeOutputsThreshold=10 2018-01-05T11:29:07,927 INFO [I/O Setup 1 Start: {Map 3}] orderedgrouped.ShuffleScheduler: ShuffleScheduler running for sourceVertex: Map 3 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true 2018-01-05T11:29:07,927 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.ShuffleScheduler: ShuffleScheduler running for sourceVertex: Map 1 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true 2018-01-05T11:29:07,927 INFO [I/O Setup 1 Start: {Map 3}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: Map 3 2018-01-05T11:29:07,928 INFO [TezTR-518813_1_37_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:07,928 INFO [I/O Setup 0 Start: {Map 1}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: Map 1 2018-01-05T11:29:07,928 INFO [TezTR-518813_1_37_2_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:07,928 INFO [TezTR-518813_1_37_2_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_37_02_000000_0 2018-01-05T11:29:07,928 DEBUG [TezTR-518813_1_37_2_0_0] log.PerfLogger: 2018-01-05T11:29:07,928 DEBUG [TezTR-518813_1_37_2_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_37_02_000000_0_10268 2018-01-05T11:29:07,928 DEBUG [TezTR-518813_1_37_2_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:07,928 DEBUG [TezTR-518813_1_37_2_0_0] mr.ObjectCache: Creating djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:29:07,928 INFO [TezTR-518813_1_37_2_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/22d173e8-c3de-43c1-95e6-c21647689fe8/reduce.xml 2018-01-05T11:29:07,928 DEBUG [TezTR-518813_1_37_2_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/22d173e8-c3de-43c1-95e6-c21647689fe8/reduce.xml 2018-01-05T11:29:07,928 DEBUG [TezTR-518813_1_37_2_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/djaiswal/_tez_scratch_dir/22d173e8-c3de-43c1-95e6-c21647689fe8/reduce.xml 2018-01-05T11:29:07,928 INFO [TezTaskEventRouter{attempt_1515180518813_0001_37_02_000000_0}] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:29:07,928 DEBUG [TezTR-518813_1_37_2_0_0] log.PerfLogger: 2018-01-05T11:29:07,928 INFO [TezTR-518813_1_37_2_0_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:29:07,929 DEBUG [TezTR-518813_1_37_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,929 INFO [TezTaskEventRouter{attempt_1515180518813_0001_37_02_000000_0}] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 3: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:29:07,929 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=37&reduce=0&map=attempt_1515180518813_0001_37_01_000000_0_10267 mapId: [attempt_1515180518813_0001_37_01_000000_0_10267] reduceId: [0] jobId: [job_1515180518813_0001] dagId: [37] keepAlive: false 2018-01-05T11:29:07,929 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=37&reduce=0&map=attempt_1515180518813_0001_37_00_000000_0_10264 mapId: [attempt_1515180518813_0001_37_00_000000_0_10264] reduceId: [0] jobId: [job_1515180518813_0001] dagId: [37] keepAlive: false 2018-01-05T11:29:07,929 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=37&reduce=0&map=attempt_1515180518813_0001_37_00_000000_0_10264; hash=...k8hL8qQL2MSPg 2018-01-05T11:29:07,929 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=37&reduce=0&map=attempt_1515180518813_0001_37_01_000000_0_10267; hash=...NboMbCULLpFQg 2018-01-05T11:29:07,930 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=37&reduce=0&map=attempt_1515180518813_0001_37_00_000000_0_10264;reply=qRskJTc7OCFo0 2018-01-05T11:29:07,930 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=37&reduce=0&map=attempt_1515180518813_0001_37_01_000000_0_10267;reply=zX+muQnrYDEUk 2018-01-05T11:29:07,930 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=37, user='djaiswal', attemptId='attempt_1515180518813_0001_37_00_000000_0_10264'} via loader 2018-01-05T11:29:07,930 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=37, user='djaiswal', attemptId='attempt_1515180518813_0001_37_01_000000_0_10267'} via loader 2018-01-05T11:29:07,930 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=37, user='djaiswal', attemptId='attempt_1515180518813_0001_37_00_000000_0_10264'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:29:07,930 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_37_00_000000_0_10264 not found 2018-01-05T11:29:07,930 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=37, user='djaiswal', attemptId='attempt_1515180518813_0001_37_01_000000_0_10267'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:29:07,930 DEBUG [New I/O worker #3] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_37_01_000000_0_10267 not found 2018-01-05T11:29:07,930 DEBUG [TezTR-518813_1_37_2_0_0] log.PerfLogger: 2018-01-05T11:29:07,930 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_37_00_000000_0_10264,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/37/output/attempt_1515180518813_0001_37_00_000000_0_10264/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/37/output/attempt_1515180518813_0001_37_00_000000_0_10264/file.out.index 2018-01-05T11:29:07,930 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Content Length in shuffle : 6298 2018-01-05T11:29:07,930 INFO [TezTR-518813_1_37_2_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 2.00KB 2018-01-05T11:29:07,930 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_37_01_000000_0_10267,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/37/output/attempt_1515180518813_0001_37_01_000000_0_10267/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/37/output/attempt_1515180518813_0001_37_01_000000_0_10267/file.out.index 2018-01-05T11:29:07,930 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Content Length in shuffle : 12906 2018-01-05T11:29:07,930 INFO [Fetcher_O {Map_1} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=37&reduce=0&map=attempt_1515180518813_0001_37_00_000000_0_10264 sent hash and receievd reply 0 ms 2018-01-05T11:29:07,930 INFO [Fetcher_O {Map_3} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=37&reduce=0&map=attempt_1515180518813_0001_37_01_000000_0_10267 sent hash and receievd reply 0 ms 2018-01-05T11:29:07,930 DEBUG [TezTR-518813_1_37_2_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:07,931 DEBUG [TezTR-518813_1_37_2_0_0] log.PerfLogger: 2018-01-05T11:29:07,931 DEBUG [TezTR-518813_1_37_2_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:07,931 DEBUG [TezTR-518813_1_37_2_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:07,931 INFO [Fetcher_O {Map_1} #0] orderedgrouped.MergeManager: CloseInMemoryFile. Current state: inMemoryMapOutputs.size=1, commitMemory=0, usedMemory=6239. Since last log: count=1, min=6239, max=6239, total=6239, avg=6239.0 2018-01-05T11:29:07,931 INFO [Fetcher_O {Map_1} #0] ShuffleScheduler.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_37_00_000000_0_10264} to MEMORY, csize=6243, dsize=6239, EndTime=1515180547931, TimeTaken=1, Rate=5.95 MB/s 2018-01-05T11:29:07,931 INFO [Fetcher_O {Map_1} #0] orderedgrouped.ShuffleScheduler: All inputs fetched for input vertex : Map 1 2018-01-05T11:29:07,931 INFO [Fetcher_O {Map_1} #0] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.01 MB/s) 2018-01-05T11:29:07,931 INFO [Fetcher_O {Map_3} #0] orderedgrouped.MergeManager: CloseInMemoryFile. Current state: inMemoryMapOutputs.size=1, commitMemory=0, usedMemory=12847. Since last log: count=1, min=12847, max=12847, total=12847, avg=12847.0 2018-01-05T11:29:07,931 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down FetchScheduler for input: Map_1, wasInterrupted=false 2018-01-05T11:29:07,931 INFO [Fetcher_O {Map_3} #0] ShuffleScheduler.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_37_01_000000_0_10267} to MEMORY, csize=12851, dsize=12847, EndTime=1515180547931, TimeTaken=1, Rate=12.25 MB/s 2018-01-05T11:29:07,931 INFO [Fetcher_O {Map_3} #0] orderedgrouped.ShuffleScheduler: All inputs fetched for input vertex : Map 3 2018-01-05T11:29:07,932 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.01 MB/s) 2018-01-05T11:29:07,932 INFO [Fetcher_O {Map_3} #0] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.01 MB/s) 2018-01-05T11:29:07,932 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down fetchers for input: Map_1, shutdown timetaken: 0 ms, hasFetcherExecutorStopped: true 2018-01-05T11:29:07,932 INFO [ShuffleAndMergeRunner {Map_3}] orderedgrouped.ShuffleScheduler: Shutting down FetchScheduler for input: Map_3, wasInterrupted=false 2018-01-05T11:29:07,932 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: TotalInMemFetchStats: count=1, totalSize=6239, min=6239, max=6239, avg=1.0 2018-01-05T11:29:07,932 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: finalMerge with #inMemoryOutputs=1, size=6239 and #onDiskOutputs=0, size=0 2018-01-05T11:29:07,932 DEBUG [TezTR-518813_1_37_2_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:07,932 INFO [TezTR-518813_1_37_2_0_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:29:07,933 INFO [ShuffleAndMergeRunner {Map_3}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.01 MB/s) 2018-01-05T11:29:07,934 INFO [ShuffleAndMergeRunner {Map_3}] orderedgrouped.ShuffleScheduler: Shutting down fetchers for input: Map_3, shutdown timetaken: 0 ms, hasFetcherExecutorStopped: true 2018-01-05T11:29:07,934 INFO [ShuffleAndMergeRunner {Map_3}] orderedgrouped.MergeManager: TotalInMemFetchStats: count=1, totalSize=12847, min=12847, max=12847, avg=1.0 2018-01-05T11:29:07,934 INFO [ShuffleAndMergeRunner {Map_3}] orderedgrouped.MergeManager: finalMerge with #inMemoryOutputs=1, size=12847 and #onDiskOutputs=0, size=0 2018-01-05T11:29:07,946 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: merge complete for input vertex : Map_1 2018-01-05T11:29:07,946 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: Map_1: Shuffle Runner thread complete 2018-01-05T11:29:07,946 INFO [ShuffleAndMergeRunner {Map_3}] orderedgrouped.Shuffle: merge complete for input vertex : Map_3 2018-01-05T11:29:07,946 INFO [ShuffleAndMergeRunner {Map_3}] orderedgrouped.Shuffle: Map_3: Shuffle Runner thread complete 2018-01-05T11:29:07,946 INFO [TezTR-518813_1_37_2_0_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:29:07,946 INFO [TezTR-518813_1_37_2_0_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:07,946 DEBUG [TezTR-518813_1_37_2_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:07,946 INFO [TezTR-518813_1_37_2_0_0] input.OrderedGroupedKVInput: Map 1: creating ValuesIterator with comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator, keyClass=org.apache.hadoop.hive.ql.io.HiveKey, valClass=org.apache.hadoop.io.BytesWritable 2018-01-05T11:29:07,946 DEBUG [TezTR-518813_1_37_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,946 DEBUG [TezTR-518813_1_37_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,946 DEBUG [TezTR-518813_1_37_2_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[string, string] 2018-01-05T11:29:07,946 DEBUG [TezTR-518813_1_37_2_0_0] log.PerfLogger: 2018-01-05T11:29:07,946 INFO [TezTR-518813_1_37_2_0_0] input.OrderedGroupedKVInput: Map 3: creating ValuesIterator with comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator, keyClass=org.apache.hadoop.hive.ql.io.HiveKey, valClass=org.apache.hadoop.io.BytesWritable 2018-01-05T11:29:07,946 DEBUG [TezTR-518813_1_37_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,946 DEBUG [TezTR-518813_1_37_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,946 DEBUG [TezTR-518813_1_37_2_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[string, string] 2018-01-05T11:29:07,946 DEBUG [TezTR-518813_1_37_2_0_0] log.PerfLogger: 2018-01-05T11:29:07,946 INFO [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Initializing operator MERGEJOIN[25] 2018-01-05T11:29:07,946 DEBUG [TezTR-518813_1_37_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,947 DEBUG [TezTR-518813_1_37_2_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[0_VALUE_0, 0_VALUE_1, 0_VALUE_2] columnTypes=[int, string, string] 2018-01-05T11:29:07,947 DEBUG [TezTR-518813_1_37_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,947 DEBUG [TezTR-518813_1_37_2_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[0_VALUE_0, 0_VALUE_1, 0_VALUE_2] columnTypes=[int, string, string] 2018-01-05T11:29:07,947 DEBUG [TezTR-518813_1_37_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,947 DEBUG [TezTR-518813_1_37_2_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[1_VALUE_0, 1_VALUE_1, 1_VALUE_2] columnTypes=[int, string, string] 2018-01-05T11:29:07,947 DEBUG [TezTR-518813_1_37_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,947 DEBUG [TezTR-518813_1_37_2_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[1_VALUE_0, 1_VALUE_1, 1_VALUE_2] columnTypes=[int, string, string] 2018-01-05T11:29:07,947 INFO [TezTR-518813_1_37_2_0_0] exec.CommonJoinOperator: JOIN struct<_col0:int,_col1:string,_col2:string,_col3:int,_col4:string,_col5:string> totalsz = 6 2018-01-05T11:29:07,947 DEBUG [TezTR-518813_1_37_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,947 DEBUG [TezTR-518813_1_37_2_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[1_VALUE_0, 1_VALUE_1, 1_VALUE_2] columnTypes=[int, string, string] 2018-01-05T11:29:07,947 DEBUG [TezTR-518813_1_37_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,947 DEBUG [TezTR-518813_1_37_2_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[0_VALUE_0, 0_VALUE_1, 0_VALUE_2] columnTypes=[int, string, string] 2018-01-05T11:29:07,947 DEBUG [TezTR-518813_1_37_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,947 DEBUG [TezTR-518813_1_37_2_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[0_VALUE_0, 0_VALUE_1, 0_VALUE_2] columnTypes=[int, string, string] 2018-01-05T11:29:07,947 DEBUG [TezTR-518813_1_37_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,947 DEBUG [TezTR-518813_1_37_2_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[1_VALUE_0, 1_VALUE_1, 1_VALUE_2] columnTypes=[int, string, string] 2018-01-05T11:29:07,947 DEBUG [TezTR-518813_1_37_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,947 DEBUG [TezTR-518813_1_37_2_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[1_VALUE_0, 1_VALUE_1, 1_VALUE_2] columnTypes=[int, string, string] 2018-01-05T11:29:07,947 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Initialization Done 25 MERGEJOIN 2018-01-05T11:29:07,947 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Operator 25 MERGEJOIN initialized 2018-01-05T11:29:07,947 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Initializing children of 25 MERGEJOIN 2018-01-05T11:29:07,947 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Initializing child 10 FS 2018-01-05T11:29:07,947 INFO [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Initializing operator FS[10] 2018-01-05T11:29:07,947 DEBUG [TezTR-518813_1_37_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:07,947 INFO [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@69438995]:[_col0, _col1, _col2, _col3, _col4, _col5]:[int, string, string, int, string, string]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@dc1bdaf 2018-01-05T11:29:07,947 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:29:07,947 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Initialization Done 10 FS 2018-01-05T11:29:07,947 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Operator 10 FS initialized 2018-01-05T11:29:07,947 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Initialization Done 10 FS done is reset. 2018-01-05T11:29:07,947 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Initialization Done 25 MERGEJOIN done is reset. 2018-01-05T11:29:07,947 INFO [TezTR-518813_1_37_2_0_0] tez.ReduceRecordProcessor: Id =25 Id =10 <\Children> Id = 25 null<\Parent> <\FS> <\Children> <\Parent> <\MERGEJOIN> 2018-01-05T11:29:07,947 DEBUG [TezTR-518813_1_37_2_0_0] log.PerfLogger: 2018-01-05T11:29:07,947 INFO [TezTR-518813_1_37_2_0_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 2 2018-01-05T11:29:07,948 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,948 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,948 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,948 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,948 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,948 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,948 WARN [TezTR-518813_1_37_2_0_0] objectinspector.StandardStructObjectInspector: Trying to access 2 fields inside a list of 1 elements: [[0]] 2018-01-05T11:29:07,948 WARN [TezTR-518813_1_37_2_0_0] objectinspector.StandardStructObjectInspector: ignoring similar errors. 2018-01-05T11:29:07,949 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,949 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,949 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,949 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,949 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,949 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,949 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,949 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,949 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,949 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,949 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,949 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,949 INFO [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-07_490_5077619032005156997-1/_tmp.-ext-10002/000000_0 2018-01-05T11:29:07,949 INFO [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-07_490_5077619032005156997-1/_task_tmp.-ext-10002/_tmp.000000_0 2018-01-05T11:29:07,949 INFO [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-07_490_5077619032005156997-1/_tmp.-ext-10002/000000_0 2018-01-05T11:29:07,984 INFO [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: FS[10]: records written - 1 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,984 INFO [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: FS[10]: records written - 10 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,984 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,985 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,986 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,987 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,988 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,989 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,990 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,991 INFO [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: FS[10]: records written - 100 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,991 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,992 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,993 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,994 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,995 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,996 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,997 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,998 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:07,999 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,000 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,000 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,000 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,000 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,000 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,000 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,000 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,000 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,000 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,000 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,000 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,000 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,002 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,003 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,004 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,005 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,006 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,007 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,009 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,010 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,010 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: Status: Running (Executing on YARN cluster with App id application_1515180518813_0001) 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,010 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,011 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,011 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,011 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,011 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,011 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,011 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 Map 3: 1/1 Reducer 2: 0(+1)/1 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,011 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,012 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,013 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,014 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,015 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,016 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,017 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,018 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,019 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,020 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,020 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,020 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,020 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,020 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,020 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,020 DEBUG [TezTR-518813_1_37_2_0_0] log.PerfLogger: 2018-01-05T11:29:08,020 DEBUG [TezTR-518813_1_37_2_0_0] mr.ObjectCache: djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:29:08,020 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: close called for operator MERGEJOIN[25] 2018-01-05T11:29:08,020 INFO [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Closing operator MERGEJOIN[25] 2018-01-05T11:29:08,020 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Cleaning up the operator state 2018-01-05T11:29:08,020 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: Closing child = FS[10] 2018-01-05T11:29:08,020 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: close called for operator FS[10] 2018-01-05T11:29:08,020 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:08,020 INFO [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: Closing operator FS[10] 2018-01-05T11:29:08,020 INFO [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: FS[10]: records written - 480 2018-01-05T11:29:08,021 INFO [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: RECORDS_OUT_0:480, 2018-01-05T11:29:08,021 DEBUG [TezTR-518813_1_37_2_0_0] exec.FileSinkOperator: 10 Close done 2018-01-05T11:29:08,021 DEBUG [TezTR-518813_1_37_2_0_0] exec.CommonMergeJoinOperator: 25 Close done 2018-01-05T11:29:08,021 INFO [TezTR-518813_1_37_2_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_37_02_000000_0 2018-01-05T11:29:08,021 INFO [TezTR-518813_1_37_2_0_0] orderedgrouped.Shuffle: Shutting down Shuffle for source: Map_1 2018-01-05T11:29:08,021 INFO [TezTR-518813_1_37_2_0_0] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:29:08,021 INFO [TezTR-518813_1_37_2_0_0] orderedgrouped.Shuffle: Shutting down Shuffle for source: Map_3 2018-01-05T11:29:08,021 INFO [TezTR-518813_1_37_2_0_0] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 3: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:29:08,021 INFO [TezTR-518813_1_37_2_0_0] output.MROutput: out_Reducer 2 closed 2018-01-05T11:29:08,021 DEBUG [TezTR-518813_1_37_2_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:08,021 INFO [TezTR-518813_1_37_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:08,021 INFO [TezTR-518813_1_37_2_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_37_02_000000_0, askedToStop=false 2018-01-05T11:29:08,021 INFO [TezTR-518813_1_37_2_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_37_02_000000_0, stopRequested=false 2018-01-05T11:29:08,021 INFO [TezTR-518813_1_37_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_37_02_000000_0: Counters: 30 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=462, REDUCE_INPUT_RECORDS=742, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=742, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=19094, ADDITIONAL_SPILLS_BYTES_READ=19094, SHUFFLE_BYTES=19094, SHUFFLE_BYTES_DECOMPRESSED=19086, SHUFFLE_BYTES_TO_MEM=19094, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=14, MERGE_PHASE_TIME=40, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][HIVE CREATED_FILES=1, RECORDS_OUT_0=480][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0]] 2018-01-05T11:29:08,021 DEBUG [TezTR-518813_1_37_2_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_37_2_0_0 thread id: 2552 scheme: file bytesRead: 10902 bytesWritten: 27815 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:08,021 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_37_02_000000_0, isTaskDone=true 2018-01-05T11:29:08,021 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000089, requestId=2, startIndex=2, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_37_02_000000_0, eventCount=2 } 2018-01-05T11:29:08,022 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_37][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_37_02_000000_0, creationTime=1515180547919, allocationTime=1515180547920, startTime=1515180547920, finishTime=1515180548022, timeTaken=102, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:08,022 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_37_02_000000_0 2018-01-05T11:29:08,022 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_37_02_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:08,022 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_37_02_000000_0; false 2018-01-05T11:29:08,022 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=2, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:08,022 INFO [Task-Executor-1] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_37_02_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:08,022 INFO [Task-Executor-1] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_37_02_000000_0 2018-01-05T11:29:08,022 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_37_02_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:08,022 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_37][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_37_02_000000, startTime=1515180547920, finishTime=1515180548022, timeTaken=102, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_37_02_000000_0, diagnostics=, counters=Counters: 35, File System Counters, FILE_BYTES_READ=10902, FILE_BYTES_WRITTEN=27815, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=462, REDUCE_INPUT_RECORDS=742, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=742, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=19094, ADDITIONAL_SPILLS_BYTES_READ=19094, SHUFFLE_BYTES=19094, SHUFFLE_BYTES_DECOMPRESSED=19086, SHUFFLE_BYTES_TO_MEM=19094, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=14, MERGE_PHASE_TIME=40, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, HIVE, CREATED_FILES=1, RECORDS_OUT_0=480, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0 2018-01-05T11:29:08,023 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:08,023 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:08,023 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_37_02 [Reducer 2], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:08,023 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_37_02 [Reducer 2] 2018-01-05T11:29:08,023 INFO [Task-Executor-1] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000089=101 2018-01-05T11:29:08,023 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000088, QueryId=djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949-37, DagId=37, VertexName=Map 3, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-1, Succeeded=true, StartTime=1515180547857, EndTime=1515180548023 2018-01-05T11:29:08,023 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_37_02_000000_0: true 2018-01-05T11:29:08,023 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_37_02_000000_0 2018-01-05T11:29:08,023 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000089, QueryId=djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949-37, Vertex=Reducer 2, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:08,023 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_37_02_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:29:08,023 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_37_02_000000_0 2018-01-05T11:29:08,023 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_37][Event:VERTEX_FINISHED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_37_02, initRequestedTime=1515180547800, initedTime=1515180547806, startRequestedTime=1515180547855, startedTime=1515180547855, finishTime=1515180548023, timeTaken=168, status=SUCCEEDED, diagnostics=, counters=Counters: 35, File System Counters, FILE_BYTES_READ=10902, FILE_BYTES_WRITTEN=27815, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=462, REDUCE_INPUT_RECORDS=742, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=742, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=19094, ADDITIONAL_SPILLS_BYTES_READ=19094, SHUFFLE_BYTES=19094, SHUFFLE_BYTES_DECOMPRESSED=19086, SHUFFLE_BYTES_TO_MEM=19094, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=14, MERGE_PHASE_TIME=40, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, HIVE, CREATED_FILES=1, RECORDS_OUT_0=480, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, vertexStats=firstTaskStartTime=1515180547920, firstTasksToStart=[ task_1515180518813_0001_37_02_000000 ], lastTaskFinishTime=1515180548022, lastTasksToFinish=[ task_1515180518813_0001_37_02_000000 ], minTaskDuration=102, maxTaskDuration=102, avgTaskDuration=102.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_37_02_000000 ], longestDurationTasks=[ task_1515180518813_0001_37_02_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:08,023 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_37_02 [Reducer 2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:08,023 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_37][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000089, stoppedTime=1515180548023, exitStatus=0 2018-01-05T11:29:08,023 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000089 exited with diagnostics set to Container completed. 2018-01-05T11:29:08,023 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_37_02 [Reducer 2] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3 2018-01-05T11:29:08,023 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null 2018-01-05T11:29:08,023 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_37_01 [Map 3] 2018-01-05T11:29:08,023 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_37_00 [Map 1] 2018-01-05T11:29:08,023 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_37_02 [Reducer 2] 2018-01-05T11:29:08,024 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_37, queueSize=0 2018-01-05T11:29:08,025 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_37][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_37, startTime=1515180547784, finishTime=1515180548023, timeTaken=239, status=SUCCEEDED, diagnostics=, counters=Counters: 66, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=3, TOTAL_LAUNCHED_TASKS=3, OTHER_LOCAL_TASKS=2, AM_CPU_MILLISECONDS=495, AM_GC_TIME_MILLIS=0, File System Counters, FILE_BYTES_READ=27970, FILE_BYTES_WRITTEN=46973, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=462, REDUCE_INPUT_RECORDS=742, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=1484, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, INPUT_RECORDS_PROCESSED=6, INPUT_SPLIT_LENGTH_BYTES=3591, OUTPUT_RECORDS=742, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=19150, OUTPUT_BYTES_WITH_OVERHEAD=19086, OUTPUT_BYTES_PHYSICAL=19094, ADDITIONAL_SPILLS_BYTES_WRITTEN=19094, ADDITIONAL_SPILLS_BYTES_READ=19094, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=2, SHUFFLE_BYTES=19094, SHUFFLE_BYTES_DECOMPRESSED=19086, SHUFFLE_BYTES_TO_MEM=19094, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=14, MERGE_PHASE_TIME=40, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, HIVE, CREATED_FILES=1, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=242, RECORDS_IN_Map_3=500, RECORDS_OUT_0=480, RECORDS_OUT_INTERMEDIATE_Map_1=242, RECORDS_OUT_INTERMEDIATE_Map_3=500, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, org.apache.hadoop.hive.llap.counters.LlapIOCounters, ALLOCATED_BYTES=11010048, ALLOCATED_USED_BYTES=5474, CACHE_HIT_BYTES=0, CACHE_MISS_BYTES=3231, CONSUMER_TIME_NS=4251137, DECODE_TIME_NS=2185888, HDFS_TIME_NS=14330195, METADATA_CACHE_MISS=12, NUM_DECODED_BATCHES=6, NUM_VECTOR_BATCHES=6, ROWS_EMITTED=742, SELECTED_ROWGROUPS=6, TOTAL_IO_TIME_NS=18506847 2018-01-05T11:29:08,025 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_37 finished with state: SUCCEEDED 2018-01-05T11:29:08,025 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_37 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:29:08,025 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000089 2018-01-05T11:29:08,025 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_37, dagState=SUCCEEDED 2018-01-05T11:29:08,025 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_37_post 2018-01-05T11:29:08,025 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:29:08,025 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:29:08,025 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:29:08,025 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 37 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=3, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=3,NumRequestsWithoutLocation=3, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=3} 2018-01-05T11:29:08,025 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:29:08,025 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=select * from tab a join tab_part b ...b.key (Stage-1), with id=dag_1515180518813_0001_37 2018-01-05T11:29:08,025 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 37, to 10.22.8.182:49818 2018-01-05T11:29:08,025 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=3, Remaining=0 2018-01-05T11:29:08,025 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=select * from tab a join tab_part b ...b.key (Stage-1), with id=dag_1515180518813_0001_37 2018-01-05T11:29:08,026 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=37} 2018-01-05T11:29:08,026 INFO [IPC Server handler 4 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=37}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:29:08,026 INFO [IPC Server handler 4 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/37 after 300 seconds 2018-01-05T11:29:08,026 ERROR [IPC Server handler 4 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949, dag_1515180518813_0001_37 2018-01-05T11:29:08,026 INFO [IPC Server handler 4 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949 2018-01-05T11:29:08,026 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=37} = 0 2018-01-05T11:29:08,026 DEBUG [IPC Server handler 4 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=37} 2018-01-05T11:29:08,026 DEBUG [IPC Server handler 4 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:29:08,026 DEBUG [IPC Server handler 4 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=37} 2018-01-05T11:29:08,027 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,027 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 Map 3: 1/1 Reducer 2: 1/1 2018-01-05T11:29:08,027 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,029 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,029 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: TaskId for 000000_0 = 000000 2018-01-05T11:29:08,029 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,029 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,029 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,030 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,030 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,030 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: select * from tab a join tab_part b on a.key = b.key 2018-01-05T11:29:08,030 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:29:08,030 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@tab 2018-01-05T11:29:08,030 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@tab@ds=2008-04-08 2018-01-05T11:29:08,030 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@tab_part 2018-01-05T11:29:08,030 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@tab_part@ds=2008-04-08 2018-01-05T11:29:08,030 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/-mr-10001 2018-01-05T11:29:08,030 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,030 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,030 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:08,030 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {} 2018-01-05T11:29:08,030 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949); Time taken: 0.409 seconds 2018-01-05T11:29:08,030 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:08,030 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,030 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:08,030 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,030 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query select * from tab a join tab_part b on a.key = b.key 2018-01-05T11:29:08,045 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] mapred.FileInputFormat: Time taken to get FileStatuses: 14 2018-01-05T11:29:08,045 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:29:08,045 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 14 2018-01-05T11:29:08,045 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FetchOperator: Creating fetchTask with deserializer typeinfo: struct<_col0:int,_col1:string,_col2:string,_col3:int,_col4:string,_col5:string> 2018-01-05T11:29:08,045 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FetchOperator: deserializer properties: table properties: {columns=_col0,_col1,_col2,_col3,_col4,_col5, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=int:string:string:int:string:string, escape.delim=\} partition properties: {columns=_col0,_col1,_col2,_col3,_col4,_col5, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=int:string:string:int:string:string, escape.delim=\} 2018-01-05T11:29:08,051 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: close called for operator LIST_SINK[32] 2018-01-05T11:29:08,052 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: Closing operator LIST_SINK[32] 2018-01-05T11:29:08,052 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: 32 Close done 2018-01-05T11:29:08,052 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting result dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/-mr-10001 2018-01-05T11:29:08,053 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-07_490_5077619032005156997-1 2018-01-05T11:29:08,053 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-07_490_5077619032005156997-1 2018-01-05T11:29:08,056 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.54 seconds, Fetched: 480 row(s) 2018-01-05T11:29:08,056 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:08,056 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:08,056 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:08,056 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:08,056 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: 2 2018-01-05T11:29:08,057 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:08,057 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:08,057 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:08,057 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:08,057 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,057 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:08,057 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,057 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,057 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,057 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: select * from tab a join tab_part b on a.key = b.key 2018-01-05T11:29:08,057 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01): select * from tab a join tab_part b on a.key = b.key 2018-01-05T11:29:08,057 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,057 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: select * from tab a join tab_part b on a.key = b.key 2018-01-05T11:29:08,057 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:08,057 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,057 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,057 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,057 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,057 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:08,057 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:29:08,057 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed phase 1 of Semantic Analysis 2018-01-05T11:29:08,057 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:08,057 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,057 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab 2018-01-05T11:29:08,057 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab 2018-01-05T11:29:08,060 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,061 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,061 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,061 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab_part 2018-01-05T11:29:08,061 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab_part 2018-01-05T11:29:08,063 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,063 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,063 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:08,063 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:08,075 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed getting MetaData in Semantic Analysis 2018-01-05T11:29:08,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,076 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_not_null_constraints : db=default tbl=tab 2018-01-05T11:29:08,076 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_not_null_constraints : db=default tbl=tab 2018-01-05T11:29:08,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,076 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab 2018-01-05T11:29:08,076 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab 2018-01-05T11:29:08,077 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,077 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,077 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,077 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab 2018-01-05T11:29:08,077 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab 2018-01-05T11:29:08,077 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,077 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,077 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,077 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_unique_constraints : db=default tbl=tab 2018-01-05T11:29:08,077 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_unique_constraints : db=default tbl=tab 2018-01-05T11:29:08,077 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,077 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,077 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,077 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab 2018-01-05T11:29:08,077 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab 2018-01-05T11:29:08,077 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,078 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,078 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,078 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_not_null_constraints : db=default tbl=tab_part 2018-01-05T11:29:08,078 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_not_null_constraints : db=default tbl=tab_part 2018-01-05T11:29:08,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,079 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:08,079 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:08,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,079 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:08,079 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:08,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,080 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,080 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,080 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_unique_constraints : db=default tbl=tab_part 2018-01-05T11:29:08,080 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_unique_constraints : db=default tbl=tab_part 2018-01-05T11:29:08,080 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,080 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,081 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,081 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab_part 2018-01-05T11:29:08,081 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab_part 2018-01-05T11:29:08,081 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,081 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,082 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:29:08,082 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.sql2rel: Plan after trimming unused fields HiveJoin(condition=[=($0, $3)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1], ds=[$2]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0], value=[$1], ds=[$2]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:08,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan before removing subquery: HiveProject(key=[$0], value=[$1], ds=[$2], key1=[$6], value1=[$7], ds1=[$8]) HiveJoin(condition=[=($0, $6)], joinType=[inner], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:08,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan just after removing subquery: HiveProject(key=[$0], value=[$1], ds=[$2], key1=[$6], value1=[$7], ds1=[$8]) HiveJoin(condition=[=($0, $6)], joinType=[inner], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:08,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan after decorrelation: HiveProject(key=[$0], value=[$1], ds=[$2], key1=[$6], value1=[$7], ds1=[$8]) HiveJoin(condition=[=($0, $6)], joinType=[inner], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:08,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,095 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,095 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,095 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,095 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,095 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partitions : db=default tbl=tab 2018-01-05T11:29:08,095 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions : db=default tbl=tab 2018-01-05T11:29:08,098 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,099 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.172689ms + 0.004681ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? ] 2018-01-05T11:29:08,099 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.142909ms + 0.047543ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (15) order by "PART_NAME" asc] 2018-01-05T11:29:08,099 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.083198ms + 0.057226ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (15) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:29:08,099 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.069299ms + 0.028681ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (15) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:08,099 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.066172ms + 0.014835ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (43) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:29:08,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.070482ms + 0.021454ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (43) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:08,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.056996ms + 0.015729ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (43) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:08,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.051852ms + 0.005955ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (43) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:08,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.080301ms + 0.020468ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (28) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:08,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.062327ms + 0.019011ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (43) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:29:08,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,103 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,103 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,103 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,103 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,103 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,103 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partitions : db=default tbl=tab_part 2018-01-05T11:29:08,104 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions : db=default tbl=tab_part 2018-01-05T11:29:08,106 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,107 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.15928ms + 0.004208ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? ] 2018-01-05T11:29:08,107 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.119299ms + 0.040197ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (14) order by "PART_NAME" asc] 2018-01-05T11:29:08,107 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.095608ms + 0.070247ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (14) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:29:08,107 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.063252ms + 0.017859ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (14) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:08,107 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.055544ms + 0.006492ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (41) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:29:08,107 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.054944ms + 0.018922ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (41) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:08,107 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.053121ms + 0.015145ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (41) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:08,107 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.051233ms + 0.006083ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (41) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:08,107 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.074154ms + 0.018638ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (26) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:08,108 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.070268ms + 0.018482ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (41) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:29:08,108 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,110 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,110 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,110 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,110 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,110 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,123 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000089, QueryId=djaiswal_20180105112907_bb9d39c9-26d4-44f6-87bc-8fc5d93c3949-37, DagId=37, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-1, Succeeded=true, StartTime=1515180547921, EndTime=1515180548123 2018-01-05T11:29:08,125 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.sql2rel: Plan after trimming unused fields HiveJoin(condition=[=($0, $3)], joinType=[inner], algorithm=[none], cost=[not available]) HiveFilter(condition=[IS NOT NULL($0)]) HiveProject(key=[$0], value=[$1], ds=[$2]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveFilter(condition=[IS NOT NULL($0)]) HiveProject(key=[$0], value=[$1], ds=[$2]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:08,125 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,125 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,125 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,125 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,126 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,126 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,126 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,126 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,127 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,127 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,127 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,127 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,127 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,127 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,128 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,128 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Original plan for PlanModifier HiveJoin(condition=[=($0, $3)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1], ds=[$2]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0], value=[$1], ds=[$2]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:08,128 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after top-level introduceDerivedTable HiveProject(key=[$0], value=[$1], ds=[$2], key0=[$3], value0=[$4], ds0=[$5]) HiveJoin(condition=[=($0, $3)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1], ds=[$2]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0], value=[$1], ds=[$2]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:08,128 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after nested convertOpTree HiveProject(key=[$0], value=[$1], ds=[$2], key0=[$3], value0=[$4], ds0=[$5]) HiveJoin(condition=[=($0, $3)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1], ds=[$2]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0], value=[$1], ds=[$2]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:08,129 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after propagating order HiveProject(key=[$0], value=[$1], ds=[$2], key0=[$3], value0=[$4], ds0=[$5]) HiveJoin(condition=[=($0, $3)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1], ds=[$2]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0], value=[$1], ds=[$2]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:08,129 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after fixTopOBSchema HiveProject(key=[$0], value=[$1], ds=[$2], key0=[$3], value0=[$4], ds0=[$5]) HiveJoin(condition=[=($0, $3)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1], ds=[$2]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0], value=[$1], ds=[$2]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:08,129 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Final plan after modifier HiveProject(a.key=[$0], a.value=[$1], a.ds=[$2], b.key=[$3], b.value=[$4], b.ds=[$5]) HiveJoin(condition=[=($0, $3)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1], ds=[$2]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0], value=[$1], ds=[$2]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:08,129 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:08,129 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:08,129 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:08,129 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,129 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab 2018-01-05T11:29:08,129 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab 2018-01-05T11:29:08,132 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,132 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,132 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:08,132 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:08,132 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:08,132 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,132 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab_part 2018-01-05T11:29:08,132 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab_part 2018-01-05T11:29:08,135 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,136 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,136 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:08,136 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:08,136 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:08,136 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Table Plan for a TS[0] 2018-01-05T11:29:08,136 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Filter Plan for $hdt$_0 row schema: a{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:08,136 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB a{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB a{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:08,136 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (. (tok_table_or_col a) key) key) (tok_selexpr (. (tok_table_or_col a) value) value) (tok_selexpr (. (tok_table_or_col a) ds) ds)) 2018-01-05T11:29:08,136 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = a{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} starRr = null 2018-01-05T11:29:08,136 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(key,_col0: int)(value,_col1: string)(ds,_col2: string)} 2018-01-05T11:29:08,136 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:08,136 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block $hdt$_0 2018-01-05T11:29:08,136 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block $hdt$_0 2018-01-05T11:29:08,137 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Table Plan for b TS[3] 2018-01-05T11:29:08,137 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Filter Plan for $hdt$_1 row schema: b{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:08,137 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB b{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB b{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:08,137 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (. (tok_table_or_col b) key) key) (tok_selexpr (. (tok_table_or_col b) value) value) (tok_selexpr (. (tok_table_or_col b) ds) ds)) 2018-01-05T11:29:08,137 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = b{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} starRr = null 2018-01-05T11:29:08,137 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(key,_col0: int)(value,_col1: string)(ds,_col2: string)} 2018-01-05T11:29:08,137 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:08,137 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block $hdt$_1 2018-01-05T11:29:08,137 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block $hdt$_1 2018-01-05T11:29:08,138 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB $hdt$_0{(key,_col0: int)(value,_col1: string)(ds,_col2: string)} $hdt$_1{(key,_col3: int)(value,_col4: string)(ds,_col5: string)} after GB $hdt$_0{(key,_col0: int)(value,_col1: string)(ds,_col2: string)} $hdt$_1{(key,_col3: int)(value,_col4: string)(ds,_col5: string)} 2018-01-05T11:29:08,138 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (. (tok_table_or_col $hdt$_0) key) a.key) (tok_selexpr (. (tok_table_or_col $hdt$_0) value) a.value) (tok_selexpr (. (tok_table_or_col $hdt$_0) ds) a.ds) (tok_selexpr (. (tok_table_or_col $hdt$_1) key) b.key) (tok_selexpr (. (tok_table_or_col $hdt$_1) value) b.value) (tok_selexpr (. (tok_table_or_col $hdt$_1) ds) b.ds)) 2018-01-05T11:29:08,138 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = $hdt$_0{(key,_col0: int)(value,_col1: string)(ds,_col2: string)} $hdt$_1{(key,_col3: int)(value,_col4: string)(ds,_col5: string)} starRr = null 2018-01-05T11:29:08,138 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(a.key,_col0: int)(a.value,_col1: string)(a.ds,_col2: string)(b.key,_col3: int)(b.value,_col4: string)(b.ds,_col5: string)} 2018-01-05T11:29:08,138 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:08,138 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-08_057_3221244085031576356-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/-mr-10001 2018-01-05T11:29:08,138 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-08_057_3221244085031576356-1 2018-01-05T11:29:08,160 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/-mr-10003 2018-01-05T11:29:08,161 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/-mr-10001 row schema: null{(a.key,_col0: int)(a.value,_col1: string)(a.ds,_col2: string)(b.key,_col3: int)(b.value,_col4: string)(b.ds,_col5: string)} 2018-01-05T11:29:08,161 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block null 2018-01-05T11:29:08,161 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:29:08,161 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: CBO Succeeded; optimized logical plan. 2018-01-05T11:29:08,161 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Before logical optimization TS[0]-FIL[1]-SEL[2]-RS[6]-JOIN[8]-SEL[9]-FS[10] TS[3]-FIL[4]-SEL[5]-RS[7]-JOIN[8] 2018-01-05T11:29:08,161 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,161 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,161 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,162 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] lineage.Generator: Time taken for lineage transform=1 2018-01-05T11:29:08,162 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,162 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,162 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.PartitionColumnsSeparator: Partition columns not separated for null, is not IN operator : 2018-01-05T11:29:08,162 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.PartitionColumnsSeparator: Partition columns not separated for null, is not IN operator : 2018-01-05T11:29:08,162 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,162 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,162 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SyntheticJoinPredicate: Synthetic predicate: 0 --> 1 2018-01-05T11:29:08,162 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SyntheticJoinPredicate: Synthetic predicate: 1 --> 0 2018-01-05T11:29:08,162 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,162 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,162 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FS(10) 2018-01-05T11:29:08,162 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(9) 2018-01-05T11:29:08,162 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for JOIN(8) 2018-01-05T11:29:08,162 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(6) 2018-01-05T11:29:08,162 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(11) 2018-01-05T11:29:08,162 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias $hdt$_0: (_col0) IN (RS[7]) 2018-01-05T11:29:08,162 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(2) 2018-01-05T11:29:08,162 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of SEL for alias $hdt$_0: (key) IN (RS[7]) 2018-01-05T11:29:08,162 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(1) 2018-01-05T11:29:08,162 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias a: key is not null 2018-01-05T11:29:08,162 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias a: (key) IN (RS[7]) 2018-01-05T11:29:08,162 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:29:08,162 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of TS for alias a: key is not null; (key) IN (RS[7]) 2018-01-05T11:29:08,162 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(7) 2018-01-05T11:29:08,162 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(12) 2018-01-05T11:29:08,162 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias $hdt$_1: (_col0) IN (RS[6]) 2018-01-05T11:29:08,162 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(5) 2018-01-05T11:29:08,162 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of SEL for alias $hdt$_1: (key) IN (RS[6]) 2018-01-05T11:29:08,162 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(4) 2018-01-05T11:29:08,162 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias b: key is not null 2018-01-05T11:29:08,162 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias b: (key) IN (RS[6]) 2018-01-05T11:29:08,162 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for TS(3) 2018-01-05T11:29:08,163 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of TS for alias b: key is not null; (key) IN (RS[6]) 2018-01-05T11:29:08,163 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SimplePredicatePushDown: After PPD: TS[0]-FIL[13]-SEL[2]-RS[6]-JOIN[8]-SEL[9]-FS[10] TS[3]-FIL[14]-SEL[5]-RS[7]-JOIN[8] 2018-01-05T11:29:08,163 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,163 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,163 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,163 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,163 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,163 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,163 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,163 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,163 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pcr.PcrExprProcFactory: Processing key is not null org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNotNull outputs {UNKNOWN, Column[key]}, 2018-01-05T11:29:08,163 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pcr.PcrExprProcFactory: Processing (key) IN (RS[7]) org.apache.hadoop.hive.ql.udf.generic.GenericUDFIn outputs {UNKNOWN, Column[key]}, {UNKNOWN, RS[7]}, 2018-01-05T11:29:08,163 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pcr.PcrExprProcFactory: Processing (key is not null and (key) IN (RS[7])) org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPAnd outputs {UNKNOWN, GenericUDFOPNotNull(Column[key])}, {UNKNOWN, GenericUDFIn(Column[key], RS[7])}, 2018-01-05T11:29:08,163 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pcr.PcrExprProcFactory: Processing key is not null org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNotNull outputs {UNKNOWN, Column[key]}, 2018-01-05T11:29:08,163 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pcr.PcrExprProcFactory: Processing (key) IN (RS[6]) org.apache.hadoop.hive.ql.udf.generic.GenericUDFIn outputs {UNKNOWN, Column[key]}, {UNKNOWN, RS[6]}, 2018-01-05T11:29:08,163 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pcr.PcrExprProcFactory: Processing (key is not null and (key) IN (RS[6])) org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPAnd outputs {UNKNOWN, GenericUDFOPNotNull(Column[key])}, {UNKNOWN, GenericUDFIn(Column[key], RS[6])}, 2018-01-05T11:29:08,163 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,163 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,163 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,163 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,163 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: JOIN 8 oldExprs: {0=[Column[KEY.reducesinkkey0], Column[VALUE._col0], Column[VALUE._col1]], 1=[Column[KEY.reducesinkkey0], Column[VALUE._col0], Column[VALUE._col1]]} 2018-01-05T11:29:08,163 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: JOIN 8 newExprs: {0=[Column[KEY.reducesinkkey0], Column[VALUE._col0], Column[VALUE._col1]], 1=[Column[KEY.reducesinkkey0], Column[VALUE._col0], Column[VALUE._col1]]} 2018-01-05T11:29:08,163 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 6 key:[Column[_col0]] 2018-01-05T11:29:08,163 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 6 oldColExprMap: {VALUE._col0=Column[_col1], VALUE._col1=Column[_col2], KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:08,163 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 6 newColExprMap: {VALUE._col0=Column[_col1], VALUE._col1=Column[_col2], KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:08,163 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 7 key:[Column[_col0]] 2018-01-05T11:29:08,163 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 7 oldColExprMap: {VALUE._col0=Column[_col1], VALUE._col1=Column[_col2], KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:08,163 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 7 newColExprMap: {VALUE._col0=Column[_col1], VALUE._col1=Column[_col2], KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:08,163 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,163 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.IdentityProjectRemover: Identity project remover optimization removed : SEL[9] 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: After logical optimization TS[0]-FIL[13]-SEL[2]-RS[6]-JOIN[8]-FS[10] TS[3]-FIL[14]-SEL[5]-RS[7]-JOIN[8] 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Parent: TS[0] 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Filter: (key is not null and (key) IN (RS[7])) (type: boolean) 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: TableScan: TS[0] 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Column key is not a partition column 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Initiate semijoin reduction for key ((key is not null and (key) IN (RS[7])) 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: DynamicSemiJoinPushdown: Saving RS to TS mapping: RS[19]: TS[0] 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Parent: TS[3] 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Filter: (key is not null and (key) IN (RS[6])) (type: boolean) 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: TableScan: TS[3] 2018-01-05T11:29:08,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Column key is not a partition column 2018-01-05T11:29:08,165 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Initiate semijoin reduction for key ((key is not null and (key) IN (RS[6])) 2018-01-05T11:29:08,165 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:08,165 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:08,165 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:08,165 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:08,165 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:08,165 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:08,165 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: DynamicSemiJoinPushdown: Saving RS to TS mapping: RS[24]: TS[3] 2018-01-05T11:29:08,165 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,165 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,165 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Estimated average row size: 104 2018-01-05T11:29:08,165 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,165 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_aggr_stats_for: db=default table=tab 2018-01-05T11:29:08,165 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_aggr_stats_for: db=default table=tab 2018-01-05T11:29:08,168 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.AggregateStatsCache: Returning aggregate stats from the cache; total hits: 3, total misses: 0, hit ratio: 1.0 2018-01-05T11:29:08,168 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.AggregateStatsCache: Returning aggregate stats from the cache; total hits: 4, total misses: 0, hit ratio: 1.0 2018-01-05T11:29:08,168 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: useDensityFunctionForNDVEstimation = false partsFound = 1 ColumnStatisticsObj = [ColumnStatisticsObj(colName:key, colType:int, statsData:), ColumnStatisticsObj(colName:value, colType:string, statsData:)] 2018-01-05T11:29:08,169 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,169 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,169 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting stats (Num rows: 242 Data size: 67518 Basic stats: COMPLETE Column stats: COMPLETE) on TS[0] 2018-01-05T11:29:08,169 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (tab): numRows: 242 dataSize: 67518 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {value= colName: value colType: string countDistincts: 153 numNulls: 0 avgColLen: 6.801652892561983 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, key= colName: key colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false, ds= colName: ds colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,169 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Estimated average row size: 104 2018-01-05T11:29:08,169 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,169 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_aggr_stats_for: db=default table=tab_part 2018-01-05T11:29:08,169 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_aggr_stats_for: db=default table=tab_part 2018-01-05T11:29:08,171 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.AggregateStatsCache: Returning aggregate stats from the cache; total hits: 5, total misses: 0, hit ratio: 1.0 2018-01-05T11:29:08,171 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.AggregateStatsCache: Returning aggregate stats from the cache; total hits: 6, total misses: 0, hit ratio: 1.0 2018-01-05T11:29:08,171 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: useDensityFunctionForNDVEstimation = false partsFound = 1 ColumnStatisticsObj = [ColumnStatisticsObj(colName:key, colType:int, statsData:), ColumnStatisticsObj(colName:value, colType:string, statsData:)] 2018-01-05T11:29:08,172 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,172 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,172 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting stats (Num rows: 500 Data size: 139500 Basic stats: COMPLETE Column stats: COMPLETE) on TS[3] 2018-01-05T11:29:08,172 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-TS[3] (tab_part): numRows: 500 dataSize: 139500 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {value= colName: value colType: string countDistincts: 309 numNulls: 0 avgColLen: 6.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, key= colName: key colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false, ds= colName: ds colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,172 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_7_b_key_min), DynamicValue(RS_7_b_key_max)) Original num rows: 242 New num rows: 242 2018-01-05T11:29:08,172 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_7_b_key_min), DynamicValue(RS_7_b_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_7_b_key_bloom_filter))) Original num rows: 242 New num rows: 242 2018-01-05T11:29:08,172 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_7_b_key_min), DynamicValue(RS_7_b_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_7_b_key_bloom_filter)))) Original num rows: 242 New num rows: 242 2018-01-05T11:29:08,172 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-FIL[13]: numRows: 242 dataSize: 67518 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {value= colName: value colType: string countDistincts: 153 numNulls: 0 avgColLen: 6.801652892561983 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, key= colName: key colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false, ds= colName: ds colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,172 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting stats (Num rows: 242 Data size: 67518 Basic stats: COMPLETE Column stats: COMPLETE) on FIL[13] 2018-01-05T11:29:08,172 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_6_a_key_min), DynamicValue(RS_6_a_key_max)) Original num rows: 500 New num rows: 500 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_6_a_key_min), DynamicValue(RS_6_a_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_6_a_key_bloom_filter))) Original num rows: 500 New num rows: 500 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_6_a_key_min), DynamicValue(RS_6_a_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_6_a_key_bloom_filter)))) Original num rows: 500 New num rows: 500 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-FIL[14]: numRows: 500 dataSize: 139500 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {value= colName: value colType: string countDistincts: 309 numNulls: 0 avgColLen: 6.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, key= colName: key colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false, ds= colName: ds colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting stats (Num rows: 500 Data size: 139500 Basic stats: COMPLETE Column stats: COMPLETE) on FIL[14] 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 242 Data size: 67518 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[2] 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[2]: numRows: 242 dataSize: 67518 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: string countDistincts: 153 numNulls: 0 avgColLen: 6.801652892561983 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 500 Data size: 139500 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[5] 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[5]: numRows: 500 dataSize: 139500 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: string countDistincts: 309 numNulls: 0 avgColLen: 6.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 242 Data size: 67518 Basic stats: COMPLETE Column stats: COMPLETE) on RS[6] 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[6]: numRows: 242 dataSize: 67518 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {VALUE._col0= colName: VALUE._col0 colType: string countDistincts: 153 numNulls: 0 avgColLen: 6.801652892561983 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col1= colName: VALUE._col1 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, KEY.reducesinkkey0= colName: KEY.reducesinkkey0 colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 242 Data size: 968 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[20] 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[20]: numRows: 242 dataSize: 968 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 500 Data size: 139500 Basic stats: COMPLETE Column stats: COMPLETE) on RS[7] 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[7]: numRows: 500 dataSize: 139500 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {VALUE._col0= colName: VALUE._col0 colType: string countDistincts: 309 numNulls: 0 avgColLen: 6.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, VALUE._col1= colName: VALUE._col1 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, KEY.reducesinkkey0= colName: KEY.reducesinkkey0 colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[15] 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[15]: numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-JOIN[8]: detects none/multiple PK parents. 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.JoinOperator: Setting stats (Num rows: 392 Data size: 218736 Basic stats: COMPLETE Column stats: COMPLETE) on JOIN[8] 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-JOIN[8]: numRows: 392 dataSize: 218736 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: string countDistincts: 153 numNulls: 0 avgColLen: 6.801652892561983 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col4= colName: _col4 colType: string countDistincts: 243 numNulls: 0 avgColLen: 6.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: int countDistincts: 120 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false, _col5= colName: _col5 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[21]: inputSize: 67518 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[21] hashAgg: true 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 3] STATS-GBY[21]: cardinality: 1 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[21] 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[21]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[16]: inputSize: 139500 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[16] hashAgg: true 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 3] STATS-GBY[16]: cardinality: 1 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[16] 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[16]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-FS[10]: numRows: 392 dataSize: 218736 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: string countDistincts: 153 numNulls: 0 avgColLen: 6.801652892561983 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col4= colName: _col4 colType: string countDistincts: 243 numNulls: 0 avgColLen: 6.812 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col3= colName: _col3 colType: int countDistincts: 120 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false, _col5= colName: _col5 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on RS[22] 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[22]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on RS[17] 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[17]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[23]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[23] hashAgg: false 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 9] STATS-GBY[23]: cardinality: 1 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[23] 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[23]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[18]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[18] hashAgg: false 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 9] STATS-GBY[18]: cardinality: 1 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[18] 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[18]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on RS[24] 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[24]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on RS[19] 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[19]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[key]]; bucket count: 2 }) on TS[0] 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[key]]; bucket count: 4 }) on TS[3] 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[key]]; bucket count: 2 }) on FIL[13] 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[key]]; bucket count: 4 }) on FIL[14] 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: 2 }) on SEL[2] 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: 4 }) on SEL[5] 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[KEY.reducesinkkey0]]; sort column names: [[KEY.reducesinkkey0]]; bucket count: 2 }) on RS[6] 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: 2 }) on SEL[20] 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[KEY.reducesinkkey0]]; sort column names: [[KEY.reducesinkkey0]]; bucket count: 4 }) on RS[7] 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: 4 }) on SEL[15] 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.JoinOperator: Setting traits ({ bucket column names: [[_col0], [_col3]]; sort column names: [[_col0], [_col3]]; bucket count: -1 }) on JOIN[8] 2018-01-05T11:29:08,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[21] 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[16] 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FileSinkOperator: Setting traits ({ bucket column names: [[_col0], [_col3]]; sort column names: [[_col0], [_col3]]; bucket count: -1 }) on FS[10] 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[22] 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[17] 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[23] 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[18] 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[24] 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[19] 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,174 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Set parallelism for reduce sink RS[6] to: 1 2018-01-05T11:29:08,174 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Set parallelism for reduce sink RS[7] to: 1 2018-01-05T11:29:08,174 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConvertJoinMapJoin: Memory monitor info set to : { isLlap: true executorsPerNode: 4 maxExecutorsOverSubscribeMemory: 3 memoryOverSubscriptionFactor: 0.20000000298023224 memoryCheckInterval: 100000 noConditionalTaskSize: 10000000 adjustedNoConditionalTaskSize: 16000000 hashTableInflationFactor: 2.0 threshold: 32000000 } 2018-01-05T11:29:08,174 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConvertJoinMapJoin: Fallback to common merge join operator 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.CommonMergeJoinOperator: Setting traits ({ bucket column names: [[_col0], [_col3]]; sort column names: [[_col0], [_col3]]; bucket count: 0 }) on MERGEJOIN[25] 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.CommonMergeJoinOperator: Setting stats (Num rows: 392 Data size: 218736 Basic stats: COMPLETE Column stats: COMPLETE) on MERGEJOIN[25] 2018-01-05T11:29:08,174 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:08,174 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:08,174 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:08,174 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing BloomFilter cost/benefit for RS[24] - TS[3] (b) (Column[key]) 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing key domain cardinality, keyDomainCardinality=461, semiJoinKeyIsPK=false, selColStat= colName: _col0 colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false, selColSourceStat= colName: key colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false, tsColStat= colName: key colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit for Column[_col0] to Column[key], selKeyCardinality=153, tsKeyCardinality=308, tsRows=500, keyDomainCardinality=461 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: SemiJoin key selectivity=0.3318872017353579, benefit=334.056399132321 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit=334.056399132321, cost=242.0, tsDataSize=500, netBenefit=92.05639913232102 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: netBenefit=0.18411279826464202 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing BloomFilter cost/benefit for RS[19] - TS[0] (a) (Column[key]) 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing key domain cardinality, keyDomainCardinality=461, semiJoinKeyIsPK=false, selColStat= colName: _col0 colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false, selColSourceStat= colName: key colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false, tsColStat= colName: key colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit for Column[_col0] to Column[key], selKeyCardinality=308, tsKeyCardinality=153, tsRows=242, keyDomainCardinality=461 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: SemiJoin key selectivity=0.6681127982646421, benefit=80.31670281995662 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit=80.31670281995662, cost=500.0, tsDataSize=242, netBenefit=-419.6832971800434 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: netBenefit=-1.7342285007439808 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Reduction factor not satisfied for RS[24]-TS[3] (b). Removing semijoin optimization. 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Removing ReduceSink RS[24] and TableScan TS[3] 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Reduction factor not satisfied for RS[19]-TS[0] (a). Removing semijoin optimization. 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Removing ReduceSink RS[19] and TableScan TS[0] 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: FS, 10 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: MERGEJOIN, 25 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 6 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 2 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: FIL, 13 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 7 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 5 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: FIL, 14 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: TS, 3 2018-01-05T11:29:08,174 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Cycle free: true 2018-01-05T11:29:08,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: Before SharedWorkOptimizer: TS[0]-FIL[13]-SEL[2]-RS[6]-MERGEJOIN[25]-FS[10] TS[3]-FIL[14]-SEL[5]-RS[7]-MERGEJOIN[25] 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: DPP information stored in the cache: {} 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: Sorted tables by size: [default.tab_part=139500, default.tab=67518] 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: After SharedWorkOptimizer: TS[0]-FIL[13]-SEL[2]-RS[6]-MERGEJOIN[25]-FS[10] TS[3]-FIL[14]-SEL[5]-RS[7]-MERGEJOIN[25] 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: int|{a}key,value: string|{a}value,ds: string|{a}ds,BLOCK__OFFSET__INSIDE__FILE: bigint|{a}block__offset__inside__file,INPUT__FILE__NAME: string|{a}input__file__name,ROW__ID: struct|{a}row__id) 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(Const boolean true, Const boolean true) -> Const boolean true 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPAnd(Const boolean true, Const boolean true)) -> GenericUDFOPNotNull(Column[key]) 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[3] with rs:(key: int|{b}key,value: string|{b}value,ds: string|{b}ds,BLOCK__OFFSET__INSIDE__FILE: bigint|{b}block__offset__inside__file,INPUT__FILE__NAME: string|{b}input__file__name,ROW__ID: struct|{b}row__id) 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(Const boolean true, Const boolean true) -> Const boolean true 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPAnd(Const boolean true, Const boolean true)) -> GenericUDFOPNotNull(Column[key]) 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FIL[13] with rs:(key: int|{a}key,value: string|{a}value,ds: string|{a}ds) 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FIL[13] 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Old filter FIL[13] conditions:(key is not null and (true and true)) 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(Const boolean true, Const boolean true) -> Const boolean true 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPAnd(Const boolean true, Const boolean true)) -> GenericUDFOPNotNull(Column[key]) 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New filter FIL[13] conditions:key is not null 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FIL[14] with rs:(key: int|{b}key,value: string|{b}value,ds: string|{b}ds) 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_3 {} 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FIL[14] 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Old filter FIL[14] conditions:(key is not null and (true and true)) 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(Const boolean true, Const boolean true) -> Const boolean true 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPAnd(Const boolean true, Const boolean true)) -> GenericUDFOPNotNull(Column[key]) 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New filter FIL[14] conditions:key is not null 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[2] with rs:(_col0: int|{$hdt$_0}key,_col1: string|{$hdt$_0}value,_col2: string|{$hdt$_0}ds) 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op FIL_13 {} 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[2] 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value] Column[ds]) 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: int|{$hdt$_1}key,_col1: string|{$hdt$_1}value,_col2: string|{$hdt$_1}ds) 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op FIL_14 {} 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value] Column[ds]) 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[6] with rs:(KEY.reducesinkkey0: int|{$hdt$_0}key,VALUE._col0: string|{$hdt$_0}value,VALUE._col1: string|{$hdt$_0}ds) 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_2 {} 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[6] 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[7] with rs:(KEY.reducesinkkey0: int|{$hdt$_1}key,VALUE._col0: string|{$hdt$_1}value,VALUE._col1: string|{$hdt$_1}ds) 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[7] 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:MERGEJOIN[25] with rs:(_col0: int|{$hdt$_0}key,_col1: string|{$hdt$_0}value,_col2: string|{$hdt$_0}ds,_col3: int|{$hdt$_1}key,_col4: string|{$hdt$_1}value,_col5: string|{$hdt$_1}ds) 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_6 {} 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_7 {} 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator MERGEJOIN[25] 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[10] with rs:(_col0: int|{},_col1: string|{},_col2: string|{},_col3: int|{},_col4: string|{},_col5: string|{}) 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op MERGEJOIN_25 {} 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[10] 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: TS[0] 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[6] 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 of table a 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[6] 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: MERGEJOIN[25] 2018-01-05T11:29:08,175 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: FS[10] 2018-01-05T11:29:08,176 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding reduce work (Reducer 2) for MERGEJOIN[25] 2018-01-05T11:29:08,176 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Setting up reduce sink: RS[6] with following reduce work: Reducer 2 2018-01-05T11:29:08,176 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[6] as parent from MERGEJOIN[25] 2018-01-05T11:29:08,176 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[7] as parent from MERGEJOIN[25] 2018-01-05T11:29:08,176 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: FS[10] 2018-01-05T11:29:08,176 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: TS[3] 2018-01-05T11:29:08,176 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[7] 2018-01-05T11:29:08,176 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding map work (Map 3) for TS[3] 2018-01-05T11:29:08,176 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 of table b 2018-01-05T11:29:08,176 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:08,176 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Second pass. Leaf operator: RS[7] has common downstream work: org.apache.hadoop.hive.ql.plan.MergeJoinWork@4da443aa 2018-01-05T11:29:08,176 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:29:08,176 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,176 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,176 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:29:08,176 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:08,176 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:08,176 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 3 2018-01-05T11:29:08,176 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:08,176 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:08,176 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:29:08,176 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:08,176 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:08,176 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 3 2018-01-05T11:29:08,176 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:08,176 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:08,176 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:29:08,176 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Examining input format to see if vectorization is enabled. 2018-01-05T11:29:08,176 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorization is enabled for input format(s) [org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] 2018-01-05T11:29:08,176 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing MapWork... 2018-01-05T11:29:08,176 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFOPNotNull(Column[key]), Vectorized Expression = SelectColumnIsNotNull(col 0:int) 2018-01-05T11:29:08,176 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator 2018-01-05T11:29:08,176 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.FilterDesc 2018-01-05T11:29:08,176 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[key], Vectorized Expression = col 0:int 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[value], Vectorized Expression = col 1:string 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[ds], Vectorized Expression = col 2:string 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.SelectDesc 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:int 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col1], Vectorized Expression = col 1:string 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col2], Vectorized Expression = col 2:string 2018-01-05T11:29:08,177 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator groupby typeName int 2018-01-05T11:29:08,177 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkLongOperator 2018-01-05T11:29:08,177 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkLongOperator 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [key, value, ds, ROW__ID] 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [int, string, string, struct] 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE, NONE, NONE, NONE] 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 1 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 1 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:08,177 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Examining input format to see if vectorization is enabled. 2018-01-05T11:29:08,177 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorization is enabled for input format(s) [org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] 2018-01-05T11:29:08,177 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing MapWork... 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFOPNotNull(Column[key]), Vectorized Expression = SelectColumnIsNotNull(col 0:int) 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.FilterDesc 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[key], Vectorized Expression = col 0:int 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[value], Vectorized Expression = col 1:string 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[ds], Vectorized Expression = col 2:string 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.SelectDesc 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:int 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col1], Vectorized Expression = col 1:string 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col2], Vectorized Expression = col 2:string 2018-01-05T11:29:08,177 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator groupby typeName int 2018-01-05T11:29:08,177 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkLongOperator 2018-01-05T11:29:08,177 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkLongOperator 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [key, value, ds, ROW__ID] 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [int, string, string, struct] 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE, NONE, NONE, NONE] 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 1 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 1 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:29:08,177 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: llap mode: only 2018-01-05T11:29:08,177 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key' 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'value' 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'ds' 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key is not null' 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key is not null' 2018-01-05T11:29:08,177 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@27a65db7 can be done in LLAP 2018-01-05T11:29:08,177 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Map 3 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key' 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'value' 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'ds' 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key is not null' 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key is not null' 2018-01-05T11:29:08,177 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:08,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@1c1469d7 can be done in LLAP 2018-01-05T11:29:08,177 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Reducer 2 2018-01-05T11:29:08,177 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:08,178 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MergeJoinWork@4da443aa can be done in LLAP 2018-01-05T11:29:08,178 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.SerializeFilter: Serializing: key is not null 2018-01-05T11:29:08,178 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.SerializeFilter: Serializing: key is not null 2018-01-05T11:29:08,178 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,178 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed plan generation 2018-01-05T11:29:08,178 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:08,178 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: validation start 2018-01-05T11:29:08,178 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:29:08,178 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,178 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:a.key, type:int, comment:null), FieldSchema(name:a.value, type:string, comment:null), FieldSchema(name:a.ds, type:string, comment:null), FieldSchema(name:b.key, type:int, comment:null), FieldSchema(name:b.value, type:string, comment:null), FieldSchema(name:b.ds, type:string, comment:null)], properties:null) 2018-01-05T11:29:08,178 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: Initializing operator LIST_SINK[32] 2018-01-05T11:29:08,178 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: Initialization Done 32 LIST_SINK 2018-01-05T11:29:08,178 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: Operator 32 LIST_SINK initialized 2018-01-05T11:29:08,178 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: Initialization Done 32 LIST_SINK done is reset. 2018-01-05T11:29:08,178 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,178 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:08,178 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {getNotNullConstraints_(NotNullConstraintsRequest, )=1, isCompatibleWith_(Configuration, )=0, getAggrColStatsFor_(String, String, List, List, )=7, getTable_(String, String, )=13, flushCache_()=0, getUniqueConstraints_(UniqueConstraintsRequest, )=0, getPrimaryKeys_(PrimaryKeysRequest, )=2, listPartitions_(String, String, short, )=15, getForeignKeys_(ForeignKeysRequest, )=2} 2018-01-05T11:29:08,178 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01); Time taken: 0.121 seconds 2018-01-05T11:29:08,178 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:08,178 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,178 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01): select * from tab a join tab_part b on a.key = b.key 2018-01-05T11:29:08,179 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,179 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: select * from tab a join tab_part b on a.key = b.key 2018-01-05T11:29:08,179 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:29:08,179 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@tab 2018-01-05T11:29:08,179 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@tab@ds=2008-04-08 2018-01-05T11:29:08,179 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@tab_part 2018-01-05T11:29:08,179 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@tab_part@ds=2008-04-08 2018-01-05T11:29:08,179 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/-mr-10001 2018-01-05T11:29:08,179 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,179 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,179 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,179 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Query ID = djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01 2018-01-05T11:29:08,179 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Total jobs = 1 2018-01-05T11:29:08,180 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,180 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:29:08,180 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-1:MAPRED] in serial mode 2018-01-05T11:29:08,190 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:29:08,212 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:29:08,212 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:29:08,212 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:29:08,212 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01 2018-01-05T11:29:08,212 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Session is already open 2018-01-05T11:29:08,213 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:29:08,213 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,213 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Dag name: select * from tab a join tab_part b ...b.key (Stage-1) 2018-01-05T11:29:08,213 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: DagInfo: {"context":"Hive","description":"\n\nselect * from tab a join tab_part b on a.key = b.key"} 2018-01-05T11:29:08,213 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01 with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:29:08,213 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,225 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,225 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:29:08,226 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,227 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 2 size: 2.00KB 2018-01-05T11:29:08,230 DEBUG [AMReporterQueueDrainer] impl.AMReporter: Removing am 10.22.8.182:49824 with last associated dag QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=22} from heartbeat with taskCount=0, amFailed=false, isDone=true 2018-01-05T11:29:08,244 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,244 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,255 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:29:08,255 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,255 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:29:08,256 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,256 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: Serialized plan (via RPC) - name: Map 3 size: 3.97KB 2018-01-05T11:29:08,261 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:08,262 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,263 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,275 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:29:08,275 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,275 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:29:08,276 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,276 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 3.96KB 2018-01-05T11:29:08,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:08,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,284 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=select * from tab a join tab_part b ...b.key (Stage-1), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01 } 2018-01-05T11:29:08,284 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:08,284 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:08,284 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:08,286 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: select * from tab a join tab_part b ...b.key (Stage-1) 2018-01-05T11:29:08,286 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:29:08,286 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_38, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_38.dot 2018-01-05T11:29:08,287 INFO [IPC Server handler 0 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_38 2018-01-05T11:29:08,287 WARN [IPC Server handler 0 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:29:08,287 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Running DAG: select * from tab a join tab_part b ...b.key (Stage-1), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01 } 2018-01-05T11:29:08,310 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_38][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_38, submitTime=1515180548286, queueName=null 2018-01-05T11:29:08,310 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_38_00 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_38_00 [Map 1], Default Resources= 2018-01-05T11:29:08,310 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_38_00 [Map 1] 2018-01-05T11:29:08,311 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:29:08,311 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_38_00 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:08,311 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_38_01 [Map 3]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_38_01 [Map 3], Default Resources= 2018-01-05T11:29:08,311 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_38_01 [Map 3] 2018-01-05T11:29:08,311 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 3 2018-01-05T11:29:08,311 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_38_01 [Map 3] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:08,312 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_38_02 [Reducer 2]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_38_02 [Reducer 2], Default Resources= 2018-01-05T11:29:08,312 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_38_02 [Reducer 2] 2018-01-05T11:29:08,312 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 2 2018-01-05T11:29:08,312 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_38_02 [Reducer 2] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:08,312 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:29:08,312 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_38][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_38, initTime=1515180548312 2018-01-05T11:29:08,312 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: dag_1515180518813_0001_38 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:29:08,312 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:29:08,312 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_38][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_38, startTime=1515180548312 2018-01-05T11:29:08,312 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_38 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:29:08,312 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {a={InputName=a}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:29:08,312 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: a, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:29:08,312 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_38_00 [Map 1] 2018-01-05T11:29:08,312 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_38, dagName=select * from tab a join tab_part b ...b.key (Stage-1) 2018-01-05T11:29:08,312 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,312 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,312 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,318 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_38_00 [Map 1] 2018-01-05T11:29:08,318 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_38_00 [Map 1] 2018-01-05T11:29:08,318 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:08,318 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_38_00 [Map 1] 2018-01-05T11:29:08,320 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:29:08,320 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@1f796199 2018-01-05T11:29:08,320 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/3d30533a-11b6-4e2e-abb6-f1b1ae86ada4/map.xml 2018-01-05T11:29:08,320 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/3d30533a-11b6-4e2e-abb6-f1b1ae86ada4/map.xml 2018-01-05T11:29:08,320 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/3d30533a-11b6-4e2e-abb6-f1b1ae86ada4/map.xml 2018-01-05T11:29:08,320 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:08,320 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:08,320 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:08,321 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:08,321 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.96KB 2018-01-05T11:29:08,321 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_38_00 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:29:08,321 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 3 : {b={InputName=b}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:29:08,321 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: b, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:29:08,321 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_38_01 [Map 3] 2018-01-05T11:29:08,321 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: a on vertex vertex_1515180518813_0001_38_00 [Map 1] 2018-01-05T11:29:08,321 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:29:08,322 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:08,322 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:29:08,322 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:08,322 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/3d30533a-11b6-4e2e-abb6-f1b1ae86ada4/map.xml 2018-01-05T11:29:08,322 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:08,322 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias a 2018-01-05T11:29:08,322 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:08,322 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:08,322 DEBUG [InputInitializer {Map 1} #0] io.HiveInputFormat: Pushdown initiated with filterText = key is not null, filterExpr = GenericUDFOPNotNull(Column[key]), serializedFilterExpr = AQEAamF2YS51dGlsLkFycmF5TGlz9AEBAQFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVDb2x1bW5EZXPjAQFrZfkAAAGCYQECb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5zZXJkZTIudHlwZWluZm8uUHJpbWl0aXZlVHlwZUluZu8BAWlu9AEDb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGT1BOb3ROdWzsAQEEb3JnLmFwYWNoZS5oYWRvb3AuaW8uQm9vbGVhbldyaXRhYmzlAQAAAQIBAWJvb2xlYe4= (desc) 2018-01-05T11:29:08,322 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 0,1 2018-01-05T11:29:08,322 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = key,value 2018-01-05T11:29:08,322 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:08,323 DEBUG [InputInitializer {Map 1} #0] io.HiveInputFormat: Pushdown initiated with filterText = key is not null, filterExpr = GenericUDFOPNotNull(Column[key]), serializedFilterExpr = AQEAamF2YS51dGlsLkFycmF5TGlz9AEBAQFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVDb2x1bW5EZXPjAQFrZfkAAAGCYQECb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5zZXJkZTIudHlwZWluZm8uUHJpbWl0aXZlVHlwZUluZu8BAWlu9AEDb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGT1BOb3ROdWzsAQEEb3JnLmFwYWNoZS5oYWRvb3AuaW8uQm9vbGVhbldyaXRhYmzlAQAAAQIBAWJvb2xlYe4= (desc) 2018-01-05T11:29:08,323 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:08,323 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:08,323 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:08,323 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:08,323 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: getSplits started 2018-01-05T11:29:08,323 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: Number of buckets specified by conf file is 2 2018-01-05T11:29:08,323 INFO [InputInitializer {Map 1} #0] orc.OrcInputFormat: ORC pushdown predicate: leaf-0 = (IS_NULL key), expr = (not leaf-0) 2018-01-05T11:29:08,323 INFO [InputInitializer {Map 1} #0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:08,323 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: Generate splits schema evolution property true reader schema struct transactional scan property false 2018-01-05T11:29:08,323 ERROR [ORC_GET_SPLITS #7] io.AcidUtils: Failed to get files with ID; using regular API: Only supported for DFS; got class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:08,326 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_38_01 [Map 3] 2018-01-05T11:29:08,326 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_38_01 [Map 3] 2018-01-05T11:29:08,326 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:08,326 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_38_01 [Map 3] 2018-01-05T11:29:08,328 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:29:08,328 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@3f3dd36b 2018-01-05T11:29:08,328 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/7b385878-392d-4bf1-a4fe-47a3392c1f93/map.xml 2018-01-05T11:29:08,328 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/7b385878-392d-4bf1-a4fe-47a3392c1f93/map.xml 2018-01-05T11:29:08,328 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/7b385878-392d-4bf1-a4fe-47a3392c1f93/map.xml 2018-01-05T11:29:08,328 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:08,328 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:08,328 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:08,329 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:08,329 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 3 size: 3.97KB 2018-01-05T11:29:08,329 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_38_01 [Map 3] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:29:08,330 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to ShuffleVertexManager for vertex_1515180518813_0001_38_02 [Reducer 2] 2018-01-05T11:29:08,330 INFO [InputInitializer {Map 3} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: b on vertex vertex_1515180518813_0001_38_01 [Map 3] 2018-01-05T11:29:08,330 INFO [InputInitializer {Map 3} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:29:08,330 DEBUG [InputInitializer {Map 3} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:08,330 INFO [InputInitializer {Map 3} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:29:08,330 DEBUG [InputInitializer {Map 3} #0] log.PerfLogger: 2018-01-05T11:29:08,330 INFO [InputInitializer {Map 3} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/7b385878-392d-4bf1-a4fe-47a3392c1f93/map.xml 2018-01-05T11:29:08,330 DEBUG [InputInitializer {Map 3} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:08,330 INFO [InputInitializer {Map 3} #0] exec.Utilities: Processing alias b 2018-01-05T11:29:08,330 DEBUG [InputInitializer {Map 3} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:08,330 INFO [InputInitializer {Map 3} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:08,331 DEBUG [InputInitializer {Map 3} #0] io.HiveInputFormat: Pushdown initiated with filterText = key is not null, filterExpr = GenericUDFOPNotNull(Column[key]), serializedFilterExpr = AQEAamF2YS51dGlsLkFycmF5TGlz9AEBAQFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVDb2x1bW5EZXPjAQFrZfkAAAGCYgECb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5zZXJkZTIudHlwZWluZm8uUHJpbWl0aXZlVHlwZUluZu8BAWlu9AEDb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGT1BOb3ROdWzsAQEEb3JnLmFwYWNoZS5oYWRvb3AuaW8uQm9vbGVhbldyaXRhYmzlAQAAAQIBAWJvb2xlYe4= (desc) 2018-01-05T11:29:08,331 INFO [InputInitializer {Map 3} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 0,1 2018-01-05T11:29:08,331 INFO [InputInitializer {Map 3} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = key,value 2018-01-05T11:29:08,331 INFO [InputInitializer {Map 3} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:08,331 DEBUG [InputInitializer {Map 3} #0] io.HiveInputFormat: Pushdown initiated with filterText = key is not null, filterExpr = GenericUDFOPNotNull(Column[key]), serializedFilterExpr = AQEAamF2YS51dGlsLkFycmF5TGlz9AEBAQFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVDb2x1bW5EZXPjAQFrZfkAAAGCYgECb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5zZXJkZTIudHlwZWluZm8uUHJpbWl0aXZlVHlwZUluZu8BAWlu9AEDb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGT1BOb3ROdWzsAQEEb3JnLmFwYWNoZS5oYWRvb3AuaW8uQm9vbGVhbldyaXRhYmzlAQAAAQIBAWJvb2xlYe4= (desc) 2018-01-05T11:29:08,331 DEBUG [InputInitializer {Map 3} #0] orc.OrcInputFormat: getSplits started 2018-01-05T11:29:08,331 DEBUG [InputInitializer {Map 3} #0] orc.OrcInputFormat: Number of buckets specified by conf file is 4 2018-01-05T11:29:08,331 INFO [InputInitializer {Map 3} #0] orc.OrcInputFormat: ORC pushdown predicate: leaf-0 = (IS_NULL key), expr = (not leaf-0) 2018-01-05T11:29:08,331 INFO [InputInitializer {Map 3} #0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:08,331 DEBUG [InputInitializer {Map 3} #0] orc.OrcInputFormat: Generate splits schema evolution property true reader schema struct transactional scan property false 2018-01-05T11:29:08,331 ERROR [ORC_GET_SPLITS #0] io.AcidUtils: Failed to get files with ID; using regular API: Only supported for DFS; got class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:08,335 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.05 maxFrac: 0.75 auto: false desiredTaskIput: 104857600 2018-01-05T11:29:08,335 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:29:08,335 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 1 tasks for vertex: vertex_1515180518813_0001_38_02 [Reducer 2] 2018-01-05T11:29:08,335 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_38_02 [Reducer 2] 2018-01-05T11:29:08,335 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_38][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_38_02, reconfigureDoneTime=1515180548335, numTasks=1, vertexLocationHint=null, edgeManagersCount=2, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:29:08,335 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_38_02 [Reducer 2], numAdditionalOutputs=1 2018-01-05T11:29:08,335 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_38][Event:VERTEX_INITIALIZED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_38_02, initRequestedTime=1515180548329, initedTime=1515180548335, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:08,335 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_38_02 [Reducer 2] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:29:08,352 DEBUG [ORC_GET_SPLITS #7] io.AcidUtils: in directory file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 base = null deltas = 0 2018-01-05T11:29:08,352 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: Split strategy: ETLSplitStrategy strategy for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:08,352 DEBUG [ORC_GET_SPLITS #9] orc.LocalCache: Serialized tail cached for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0 2018-01-05T11:29:08,352 DEBUG [ORC_GET_SPLITS #9] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,352 DEBUG [ORC_GET_SPLITS #9] orc.LocalCache: Serialized tail cached for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0 2018-01-05T11:29:08,353 DEBUG [ORC_GET_SPLITS #9] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,353 DEBUG [ORC_GET_SPLITS #9] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,353 DEBUG [ORC_GET_SPLITS #8] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,353 DEBUG [ORC_GET_SPLITS #9] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:29:08,353 DEBUG [ORC_GET_SPLITS #8] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:29:08,353 INFO [InputInitializer {Map 1} #0] orc.OrcInputFormat: FooterCacheHitRatio: 2/2 2018-01-05T11:29:08,353 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0, start=3, length=566, isOriginal=true, fileLength=744, hasFooter=false, hasBase=true, deltas=0] projected_columns_uncompressed_size: 8724 2018-01-05T11:29:08,353 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0, start=3, length=583, isOriginal=true, fileLength=767, hasFooter=false, hasBase=true, deltas=0] projected_columns_uncompressed_size: 8573 2018-01-05T11:29:08,353 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: getSplits finished 2018-01-05T11:29:08,353 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 2 2018-01-05T11:29:08,353 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:08,353 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 2. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:29:08,353 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/3d30533a-11b6-4e2e-abb6-f1b1ae86ada4/map.xml 2018-01-05T11:29:08,353 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/3d30533a-11b6-4e2e-abb6-f1b1ae86ada4/map.xml 2018-01-05T11:29:08,353 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/3d30533a-11b6-4e2e-abb6-f1b1ae86ada4/map.xml 2018-01-05T11:29:08,353 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:08,353 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:08,353 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:08,354 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:08,355 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.96KB 2018-01-05T11:29:08,355 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0 to src new group? true 2018-01-05T11:29:08,355 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0 to src new group? false 2018-01-05T11:29:08,355 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:29:08,355 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:29:08,355 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:29:08,355 DEBUG [InputInitializer {Map 1} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 8573 2018-01-05T11:29:08,355 DEBUG [InputInitializer {Map 1} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 8724 2018-01-05T11:29:08,355 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 320 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 17297 Original splits: 2 2018-01-05T11:29:08,355 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired numSplits: 1 lengthPerGroup: 17297 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 17297 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false 2018-01-05T11:29:08,355 DEBUG [InputInitializer {Map 1} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 8573 2018-01-05T11:29:08,355 DEBUG [InputInitializer {Map 1} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 8724 2018-01-05T11:29:08,355 DEBUG [InputInitializer {Map 1} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 8724 2018-01-05T11:29:08,355 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Number of splits desired: 1 created: 1 splitsProcessed: 2 2018-01-05T11:29:08,355 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 2 grouped split count is 1, for bucket: 1 2018-01-05T11:29:08,355 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:29:08,355 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: a on vertex vertex_1515180518813_0001_38_00 [Map 1] 2018-01-05T11:29:08,355 INFO [App Shared Pool - #2] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_38_00 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:08,355 INFO [App Shared Pool - #2] impl.VertexImpl: Got updated RootInputsSpecs: {a=forAllWorkUnits=true, update=[1]} 2018-01-05T11:29:08,356 INFO [App Shared Pool - #2] impl.VertexImpl: Vertex vertex_1515180518813_0001_38_00 [Map 1] parallelism set to 1 2018-01-05T11:29:08,356 INFO [App Shared Pool - #2] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_38_00 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:08,356 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_38][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_38_00, initRequestedTime=1515180548312, initedTime=1515180548356, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:08,356 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_38_00 [Map 1] with distanceFromRoot: 0 2018-01-05T11:29:08,356 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_38_00 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:29:08,356 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_38][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_38_00, startRequestedTime=1515180548329, startedTime=1515180548356 2018-01-05T11:29:08,356 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_38][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_38_00, reconfigureDoneTime=1515180548356, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:29:08,356 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_38_00 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:08,356 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_38_00 for vertex: vertex_1515180518813_0001_38_02 [Reducer 2] numStartedSources: 1 numSources: 2 2018-01-05T11:29:08,356 INFO [Dispatcher thread {Central}] impl.VertexImpl: Cannot start vertex: vertex_1515180518813_0001_38_02 [Reducer 2] numStartedSources: 1 numSources: 2 2018-01-05T11:29:08,356 INFO [App Shared Pool - #2] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:29:08,356 INFO [App Shared Pool - #2] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:29:08,356 INFO [App Shared Pool - #2] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_38_00 [Map 1] 2018-01-05T11:29:08,357 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_38][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_38_00_000000, scheduledTime=1515180548356, launchTime=1515180548356 2018-01-05T11:29:08,357 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_38_00 [Map 1] 2018-01-05T11:29:08,357 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_38_00_000000_0, priority=8, capability=, hosts=null 2018-01-05T11:29:08,357 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=8,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:08,357 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=8,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:08,357 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:08,357 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_38_00_000000_0 on hosts=null 2018-01-05T11:29:08,357 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_38_00_000000_0 on hosts=null 2018-01-05T11:29:08,357 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_38_00_000000_0 2018-01-05T11:29:08,357 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:08,357 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #90, task=TaskInfo{task=attempt_1515180518813_0001_38_00_000000_0, priority=8, startTime=0, containerId=null, uniqueId=89, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000090 2018-01-05T11:29:08,357 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_38_00_000000_0; false 2018-01-05T11:29:08,357 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_38_00_000000_0, priority=8, startTime=311568866, containerId=container_111101111_0001_00_000090assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=89, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:08,357 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000090 for node: 10.22.8.182:49818 2018-01-05T11:29:08,360 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 38, name=select * from tab a join tab_part b ...b.key (Stage-1), queryId=djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01 2018-01-05T11:29:08,360 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000090, attempt_1515180518813_0001_38_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:08,360 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000090 for node: 10.22.8.182:49818 2018-01-05T11:29:08,361 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000090 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:08,361 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000090, launchTime=1515180548361 2018-01-05T11:29:08,362 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_38_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000090, dagName=select * from tab a join tab_part b ...b.key (Stage-1), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{a,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{Reducer 2,org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput,1}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180548312, firstAttemptStartTime=1515180548357, currentAttemptStartTime=1515180548360} 2018-01-05T11:29:08,362 INFO [IPC Server handler 3 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000090, QueryId=djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01-38, DagId=38, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180548362 2018-01-05T11:29:08,362 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_38_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000090] on NM: [10.22.8.182:49818] 2018-01-05T11:29:08,362 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:08,362 DEBUG [IPC Server handler 3 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=38} with the ShuffleHandler 2018-01-05T11:29:08,362 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:08,362 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_38][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_38_00_000000_0, startTime=1515180548361, containerId=container_111101111_0001_00_000090, nodeId=10.22.8.182:49818 2018-01-05T11:29:08,377 INFO [IPC Server handler 3 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:29:08,378 DEBUG [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/38] 2018-01-05T11:29:08,378 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:29:08,378 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_38_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:08,378 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_38_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180548357, dagStartTime=1515180548312, withinDagPriority=8, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:29:08,378 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_38_00_000000_0 : ACCEPTED 2018-01-05T11:29:08,378 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_38_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:29:08,378 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_38_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180548357, dagStartTime=1515180548312, withinDagPriority=8, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:29:08,378 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_38_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:08,378 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_38_00_000000_0 to pre-emption queue 2018-01-05T11:29:08,378 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_38_00_000000_0: true 2018-01-05T11:29:08,378 INFO [TaskCommunicator # 5] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_38_00_000000_0 2018-01-05T11:29:08,378 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: taskOwner hashCode:1463545238 2018-01-05T11:29:08,378 INFO [TaskCommunicator # 5] GuaranteedTasks: Received response for attempt_1515180518813_0001_38_00_000000_0, true 2018-01-05T11:29:08,379 INFO [Task-Executor-1] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : select * from tab a join tab_part b ...b.key (Stage-1), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_38_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=a, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=Reducer 2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ] 2018-01-05T11:29:08,379 INFO [Task-Executor-1] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1021837312, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:08,380 INFO [Task-Executor-1] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_38_00_000000_0 vertexName: Map 1 2018-01-05T11:29:08,380 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000090, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_38_00_000000_0, eventCount=1 } 2018-01-05T11:29:08,380 DEBUG [TezTR-518813_1_38_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_38 queryId: djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01 instanceType: TaskRunner2Callable 2018-01-05T11:29:08,380 INFO [TezTR-518813_1_38_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_38_00_000000_0 2018-01-05T11:29:08,380 DEBUG [TezTR-518813_1_38_0_0_0] log.PerfLogger: 2018-01-05T11:29:08,382 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:08,383 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:08,384 DEBUG [TezTR-518813_1_38_0_0_0] log.PerfLogger: 2018-01-05T11:29:08,384 INFO [TezTR-518813_1_38_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:08,384 INFO [TezTR-518813_1_38_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:08,384 INFO [TezTR-518813_1_38_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:08,384 INFO [I/O Setup 0 Initialize: {a}] input.MRInput: a using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:29:08,384 INFO [I/O Setup 0 Initialize: {a}] input.MRInputLegacy: a MRInputLegacy deferring initialization 2018-01-05T11:29:08,384 INFO [TezTR-518813_1_38_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:08,384 INFO [TezTR-518813_1_38_0_0_0] resources.MemoryDistributor: InitialRequests=[a:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [Reducer 2:OUTPUT:10485760:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput] 2018-01-05T11:29:08,384 INFO [TezTR-518813_1_38_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:08,384 INFO [TezTR-518813_1_38_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:08,384 INFO [IPC Server handler 12 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_38_00_000000_0 sent events: (0-1). 2018-01-05T11:29:08,384 INFO [TezTR-518813_1_38_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 10485760, TotalRequestedScaled: 9679163.076923078, TotalJVMHeap: 1021837312, TotalAvailable: 684630999, TotalRequested/TotalJVMHeap:0.01 2018-01-05T11:29:08,384 INFO [TezTR-518813_1_38_0_0_0] resources.MemoryDistributor: Allocations=[a:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:10485760:10485760] 2018-01-05T11:29:08,384 INFO [TezTR-518813_1_38_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:08,384 INFO [TezTR-518813_1_38_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: a being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:08,384 INFO [TezTR-518813_1_38_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:29:08,384 INFO [TezTR-518813_1_38_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:08,385 INFO [I/O Setup 1 Start: {a}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: a 2018-01-05T11:29:08,385 INFO [TezTR-518813_1_38_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:08,385 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:29:08,385 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_38_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:29:08,385 INFO [TezTR-518813_1_38_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_38_00_000000_0 2018-01-05T11:29:08,385 DEBUG [TezTR-518813_1_38_0_0_0] log.PerfLogger: 2018-01-05T11:29:08,385 DEBUG [TezTR-518813_1_38_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_38_00_000000_0_10272 2018-01-05T11:29:08,385 DEBUG [TezTR-518813_1_38_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:08,385 DEBUG [TezTR-518813_1_38_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_38_00_000000_0 2018-01-05T11:29:08,385 INFO [TezTR-518813_1_38_0_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01 2018-01-05T11:29:08,385 DEBUG [TezTR-518813_1_38_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:08,385 DEBUG [TezTR-518813_1_38_0_0_0] log.PerfLogger: 2018-01-05T11:29:08,385 DEBUG [TezTR-518813_1_38_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:08,385 DEBUG [TezTR-518813_1_38_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:08,385 DEBUG [TezTR-518813_1_38_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01_Map 1__MAP_PLAN__ 2018-01-05T11:29:08,385 INFO [TezTR-518813_1_38_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/3d30533a-11b6-4e2e-abb6-f1b1ae86ada4/map.xml 2018-01-05T11:29:08,385 DEBUG [TezTR-518813_1_38_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/3d30533a-11b6-4e2e-abb6-f1b1ae86ada4/map.xml 2018-01-05T11:29:08,385 DEBUG [TezTR-518813_1_38_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/3d30533a-11b6-4e2e-abb6-f1b1ae86ada4/map.xml 2018-01-05T11:29:08,386 DEBUG [TezTR-518813_1_38_0_0_0] log.PerfLogger: 2018-01-05T11:29:08,386 INFO [TezTR-518813_1_38_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:08,386 DEBUG [TezTR-518813_1_38_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:08,387 DEBUG [TezTR-518813_1_38_0_0_0] log.PerfLogger: 2018-01-05T11:29:08,387 INFO [TezTR-518813_1_38_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.96KB 2018-01-05T11:29:08,387 DEBUG [TezTR-518813_1_38_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:08,387 INFO [TezTR-518813_1_38_0_0_0] tez.MapRecordProcessor: The input names are: [a] 2018-01-05T11:29:08,387 INFO [TezTR-518813_1_38_0_0_0] input.MRInput: Initialized MRInput: a 2018-01-05T11:29:08,388 DEBUG [TezTR-518813_1_38_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 2 2018-01-05T11:29:08,388 INFO [TezTR-518813_1_38_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/3d30533a-11b6-4e2e-abb6-f1b1ae86ada4/map.xml 2018-01-05T11:29:08,388 DEBUG [TezTR-518813_1_38_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:08,388 DEBUG [TezTR-518813_1_38_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180546}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, numFiles=2, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, rawDataSize=22748, partition_columns.types=string, numRows=242, partition_columns=ds, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=1511, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08, transient_lastDdlTime=1515180547}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180546}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, numFiles=2, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, rawDataSize=22748, partition_columns.types=string, numRows=242, partition_columns=ds, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=1511, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08, transient_lastDdlTime=1515180547}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat]} 2018-01-05T11:29:08,388 DEBUG [TezTR-518813_1_38_0_0_0] io.HiveInputFormat: Pushdown initiated with filterText = key is not null, filterExpr = GenericUDFOPNotNull(Column[key]), serializedFilterExpr = AQEAamF2YS51dGlsLkFycmF5TGlz9AEBAQFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVDb2x1bW5EZXPjAQFrZfkAAAGCYQECb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5zZXJkZTIudHlwZWluZm8uUHJpbWl0aXZlVHlwZUluZu8BAWlu9AEDb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGT1BOb3ROdWzsAQEEb3JnLmFwYWNoZS5oYWRvb3AuaW8uQm9vbGVhbldyaXRhYmzlAQAAAQIBAWJvb2xlYe4= (desc) 2018-01-05T11:29:08,388 INFO [TezTR-518813_1_38_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/3d30533a-11b6-4e2e-abb6-f1b1ae86ada4/map.xml 2018-01-05T11:29:08,388 DEBUG [TezTR-518813_1_38_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:08,388 DEBUG [TezTR-518813_1_38_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.hive.ql.io.orc.OrcInputFormat 2018-01-05T11:29:08,390 DEBUG [TezTR-518813_1_38_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:29:08,390 INFO [TezTR-518813_1_38_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/3d30533a-11b6-4e2e-abb6-f1b1ae86ada4/map.xml 2018-01-05T11:29:08,390 DEBUG [TezTR-518813_1_38_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:08,390 INFO [TezTR-518813_1_38_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_38_00_000000_0 2018-01-05T11:29:08,390 INFO [TezTR-518813_1_38_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:08,390 DEBUG [TezTR-518813_1_38_0_0_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:08,390 DEBUG [TezTR-518813_1_38_0_0_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:08,390 DEBUG [TezTR-518813_1_38_0_0_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:08,390 DEBUG [TezTR-518813_1_38_0_0_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:08,390 DEBUG [TezTR-518813_1_38_0_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,390 DEBUG [TezTR-518813_1_38_0_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,390 INFO [TezTR-518813_1_38_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/3d30533a-11b6-4e2e-abb6-f1b1ae86ada4/map.xml 2018-01-05T11:29:08,390 DEBUG [TezTR-518813_1_38_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:08,391 DEBUG [TezTR-518813_1_38_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0 2018-01-05T11:29:08,391 DEBUG [IO-Elevator-Thread-6] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_38 queryId: djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01 instanceType: OrcEncodedDataReader 2018-01-05T11:29:08,391 INFO [TezTR-518813_1_38_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0, start=3, length=583, isOriginal=true, fileLength=767, hasFooter=false, hasBase=true, deltas=0], org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0, start=3, length=566, isOriginal=true, fileLength=744, hasFooter=false, hasBase=true, deltas=0]], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=1149} 2018-01-05T11:29:08,391 INFO [IO-Elevator-Thread-6] LlapIoImpl: Processing data for file [-5122461581150239451, 1515180546000, 767]: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0 2018-01-05T11:29:08,391 INFO [TezTR-518813_1_38_0_0_0] input.MRInput: a initialized RecordReader from event 2018-01-05T11:29:08,391 DEBUG [IO-Elevator-Thread-6] LlapIoOrc: FileSplit {3, 583}; stripes {3, 583}, 2018-01-05T11:29:08,391 DEBUG [TezTR-518813_1_38_0_0_0] tez.MapRecordProcessor: Starting Output: Reducer 2 2018-01-05T11:29:08,392 DEBUG [IO-Elevator-Thread-6] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,392 DEBUG [IO-Elevator-Thread-6] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,392 INFO [IO-Elevator-Thread-6] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [-5122461581150239451, 1515180546000, 767]): [{range start: 0 end: 76}] 2018-01-05T11:29:08,392 INFO [TezTR-518813_1_38_0_0_0] impl.ExternalSorter: Reducer 2 using: memoryMb=10, keySerializerClass=class org.apache.hadoop.hive.ql.io.HiveKey, valueSerializerClass=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization$TezBytesWritableSerializer@792f58, comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator@32ba9cdc, partitioner=org.apache.tez.mapreduce.partition.MRPartitioner, serialization=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.hadoop.io.serializer.WritableSerialization, org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization, org.apache.hadoop.io.serializer.avro.AvroReflectSerialization, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:29:08,392 INFO [IO-Elevator-Thread-6] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [-5122461581150239451, 1515180546000, 767], base offset 3): [{start: 0 end: 11 cache buffer: 0x317f0390(1)}, {start: 11 end: 38 cache buffer: 0x4252621d(1)}, {start: 38 end: 76 cache buffer: 0x479a13fd(1)}] 2018-01-05T11:29:08,392 INFO [TezTR-518813_1_38_0_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 2: , UsingHashComparator=true 2018-01-05T11:29:08,392 DEBUG [IO-Elevator-Thread-6] impl.RecordReaderImpl: Row group 0 to 9999 is included. 2018-01-05T11:29:08,392 DEBUG [IO-Elevator-Thread-6] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,392 INFO [IO-Elevator-Thread-6] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [-5122461581150239451, 1515180546000, 767]): [{range start: 76 end: 523}] 2018-01-05T11:29:08,393 INFO [IO-Elevator-Thread-6] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [-5122461581150239451, 1515180546000, 767], base offset 3): [{start: 76 end: 230 cache buffer: 0x112f2b3b(1)}, {start: 230 end: 330 cache buffer: 0x28b0a379(1)}, {start: 330 end: 361 cache buffer: 0xf056330(1)}, {start: 361 end: 523 cache buffer: 0x67fff5bf(1)}] 2018-01-05T11:29:08,393 DEBUG [IO-Elevator-Thread-6] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: int streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 164 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:29:08,393 DEBUG [IO-Elevator-Thread-6] encoded.EncodedTreeReaderFactory: columnIndex: 2 columnType: string streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DICTIONARY_V2 dictionarySize: 73 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 97 bytes, ], indexBaseOffset=0] dictionary: true lengths: true secondary: false tz: 2018-01-05T11:29:08,393 DEBUG [IO-Elevator-Thread-6] LlapIoOrc: Done with decode 2018-01-05T11:29:08,393 DEBUG [IO-Elevator-Thread-6] LlapIoImpl: setDone called; closed false, done false, err null, pending 1 2018-01-05T11:29:08,393 DEBUG [IO-Elevator-Thread-6] impl.StatsRecordingThreadPool: Updated stats: instance: OrcEncodedDataReader thread name: IO-Elevator-Thread-6 thread id: 2198 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:08,393 INFO [TezTR-518813_1_38_0_0_0] impl.PipelinedSorter: Newly allocated block size=10485760, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=10485760, total=10485760 2018-01-05T11:29:08,393 INFO [TezTR-518813_1_38_0_0_0] impl.PipelinedSorter: Pre allocating rest of memory buffers upfront 2018-01-05T11:29:08,393 INFO [TezTR-518813_1_38_0_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 2: , UsingHashComparator=true#blocks=1, maxMemUsage=10485760, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=10485760, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=10 2018-01-05T11:29:08,393 INFO [TezTR-518813_1_38_0_0_0] impl.PipelinedSorter: Reducer 2: reserved.remaining()=10485760, reserved.metasize=5242880 2018-01-05T11:29:08,394 INFO [TezTR-518813_1_38_0_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:29:08,394 INFO [TezTR-518813_1_38_0_0_0] vector.VectorMapOperator: Initializing operator MAP[0] 2018-01-05T11:29:08,394 DEBUG [TezTR-518813_1_38_0_0_0] vector.VectorMapOperator: Initialization Done 0 MAP 2018-01-05T11:29:08,394 DEBUG [TezTR-518813_1_38_0_0_0] vector.VectorMapOperator: Operator 0 MAP initialized 2018-01-05T11:29:08,394 DEBUG [TezTR-518813_1_38_0_0_0] vector.VectorMapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:29:08,394 INFO [TezTR-518813_1_38_0_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:29:08,394 INFO [TezTR-518813_1_38_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:08,394 DEBUG [TezTR-518813_1_38_0_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:29:08,394 INFO [TezTR-518813_1_38_0_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:29:08,394 DEBUG [TezTR-518813_1_38_0_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:29:08,394 DEBUG [TezTR-518813_1_38_0_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:29:08,394 DEBUG [TezTR-518813_1_38_0_0_0] exec.TableScanOperator: Initializing children of 0 TS 2018-01-05T11:29:08,394 DEBUG [TezTR-518813_1_38_0_0_0] vector.VectorFilterOperator: Initializing child 26 FIL 2018-01-05T11:29:08,394 INFO [TezTR-518813_1_38_0_0_0] vector.VectorFilterOperator: Initializing operator FIL[26] 2018-01-05T11:29:08,394 DEBUG [TezTR-518813_1_38_0_0_0] vector.VectorFilterOperator: Initialization Done 26 FIL 2018-01-05T11:29:08,394 DEBUG [TezTR-518813_1_38_0_0_0] vector.VectorFilterOperator: Operator 26 FIL initialized 2018-01-05T11:29:08,394 DEBUG [TezTR-518813_1_38_0_0_0] vector.VectorFilterOperator: Initializing children of 26 FIL 2018-01-05T11:29:08,394 DEBUG [TezTR-518813_1_38_0_0_0] vector.VectorSelectOperator: Initializing child 27 SEL 2018-01-05T11:29:08,394 INFO [TezTR-518813_1_38_0_0_0] vector.VectorSelectOperator: Initializing operator SEL[27] 2018-01-05T11:29:08,394 DEBUG [TezTR-518813_1_38_0_0_0] vector.VectorSelectOperator: Initialization Done 27 SEL 2018-01-05T11:29:08,394 DEBUG [TezTR-518813_1_38_0_0_0] vector.VectorSelectOperator: Operator 27 SEL initialized 2018-01-05T11:29:08,394 DEBUG [TezTR-518813_1_38_0_0_0] vector.VectorSelectOperator: Initializing children of 27 SEL 2018-01-05T11:29:08,394 DEBUG [TezTR-518813_1_38_0_0_0] reducesink.VectorReduceSinkLongOperator: Initializing child 28 RS 2018-01-05T11:29:08,394 INFO [TezTR-518813_1_38_0_0_0] reducesink.VectorReduceSinkLongOperator: Initializing operator RS[28] 2018-01-05T11:29:08,394 INFO [TezTR-518813_1_38_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/3d30533a-11b6-4e2e-abb6-f1b1ae86ada4/map.xml 2018-01-05T11:29:08,394 DEBUG [TezTR-518813_1_38_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:08,394 INFO [TezTR-518813_1_38_0_0_0] reducesink.VectorReduceSinkCommonOperator: Using tag = 0 2018-01-05T11:29:08,394 DEBUG [TezTR-518813_1_38_0_0_0] reducesink.VectorReduceSinkLongOperator: Initialization Done 28 RS 2018-01-05T11:29:08,394 DEBUG [TezTR-518813_1_38_0_0_0] reducesink.VectorReduceSinkLongOperator: Operator 28 RS initialized 2018-01-05T11:29:08,394 DEBUG [TezTR-518813_1_38_0_0_0] reducesink.VectorReduceSinkLongOperator: Initialization Done 28 RS done is reset. 2018-01-05T11:29:08,394 DEBUG [TezTR-518813_1_38_0_0_0] vector.VectorSelectOperator: Initialization Done 27 SEL done is reset. 2018-01-05T11:29:08,397 DEBUG [TezTR-518813_1_38_0_0_0] vector.VectorFilterOperator: Initialization Done 26 FIL done is reset. 2018-01-05T11:29:08,397 DEBUG [TezTR-518813_1_38_0_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:29:08,397 INFO [TezTR-518813_1_38_0_0_0] exec.OperatorUtils: Setting output collector: RS[28] --> Reducer 2 2018-01-05T11:29:08,397 DEBUG [TezTR-518813_1_38_0_0_0] log.PerfLogger: 2018-01-05T11:29:08,397 INFO [TezTR-518813_1_38_0_0_0] reducesink.VectorReduceSinkCommonOperator: RS[28]: records written - 1 2018-01-05T11:29:08,398 INFO [TezTR-518813_1_38_0_0_0] reducesink.VectorReduceSinkCommonOperator: RS[28]: records written - 10 2018-01-05T11:29:08,398 INFO [TezTR-518813_1_38_0_0_0] reducesink.VectorReduceSinkCommonOperator: RS[28]: records written - 100 2018-01-05T11:29:08,398 INFO [TezTR-518813_1_38_0_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:08,398 INFO [TezTR-518813_1_38_0_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, warehouse.tab, file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0 ([-5122461581150239451, 1515180546000, 767]), 0,1]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=1, NUM_ERRORS=0, ROWS_EMITTED=120, METADATA_CACHE_HIT=2, METADATA_CACHE_MISS=0, CACHE_HIT_BYTES=523, CACHE_MISS_BYTES=0, ALLOCATED_BYTES=0, ALLOCATED_USED_BYTES=0, TOTAL_IO_TIME_NS=2610123, DECODE_TIME_NS=496670, HDFS_TIME_NS=71822, CONSUMER_TIME_NS=534518 ] 2018-01-05T11:29:08,398 DEBUG [TezTR-518813_1_38_0_0_0] encoded.OrcEncodedDataReader: Encoded reader is being stopped 2018-01-05T11:29:08,398 DEBUG [TezTR-518813_1_38_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 1 of 2 2018-01-05T11:29:08,398 DEBUG [TezTR-518813_1_38_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180546}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, numFiles=2, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, rawDataSize=22748, partition_columns.types=string, numRows=242, partition_columns=ds, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=1511, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08, transient_lastDdlTime=1515180547}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180546}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, numFiles=2, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, rawDataSize=22748, partition_columns.types=string, numRows=242, partition_columns=ds, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=1511, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08, transient_lastDdlTime=1515180547}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat]} 2018-01-05T11:29:08,398 DEBUG [TezTR-518813_1_38_0_0_0] io.HiveInputFormat: Pushdown initiated with filterText = key is not null, filterExpr = GenericUDFOPNotNull(Column[key]), serializedFilterExpr = AQEAamF2YS51dGlsLkFycmF5TGlz9AEBAQFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVDb2x1bW5EZXPjAQFrZfkAAAGCYQECb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5zZXJkZTIudHlwZWluZm8uUHJpbWl0aXZlVHlwZUluZu8BAWlu9AEDb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGT1BOb3ROdWzsAQEEb3JnLmFwYWNoZS5oYWRvb3AuaW8uQm9vbGVhbldyaXRhYmzlAQAAAQIBAWJvb2xlYe4= (desc) 2018-01-05T11:29:08,398 INFO [TezTR-518813_1_38_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/3d30533a-11b6-4e2e-abb6-f1b1ae86ada4/map.xml 2018-01-05T11:29:08,398 DEBUG [TezTR-518813_1_38_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:08,398 DEBUG [TezTR-518813_1_38_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.hive.ql.io.orc.OrcInputFormat 2018-01-05T11:29:08,400 DEBUG [TezTR-518813_1_38_0_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:29:08,400 INFO [TezTR-518813_1_38_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/3d30533a-11b6-4e2e-abb6-f1b1ae86ada4/map.xml 2018-01-05T11:29:08,400 DEBUG [TezTR-518813_1_38_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:08,400 INFO [TezTR-518813_1_38_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_38_00_000000_0 2018-01-05T11:29:08,400 INFO [TezTR-518813_1_38_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:08,400 DEBUG [TezTR-518813_1_38_0_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,400 DEBUG [ORC_GET_SPLITS #0] io.AcidUtils: in directory file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 base = null deltas = 0 2018-01-05T11:29:08,400 DEBUG [TezTR-518813_1_38_0_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,400 DEBUG [InputInitializer {Map 3} #0] orc.OrcInputFormat: Split strategy: ETLSplitStrategy strategy for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:08,400 DEBUG [ORC_GET_SPLITS #5] orc.LocalCache: Serialized tail cached for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0 2018-01-05T11:29:08,400 DEBUG [ORC_GET_SPLITS #5] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,400 INFO [TezTR-518813_1_38_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/3d30533a-11b6-4e2e-abb6-f1b1ae86ada4/map.xml 2018-01-05T11:29:08,400 DEBUG [TezTR-518813_1_38_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:08,400 DEBUG [TezTR-518813_1_38_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0 2018-01-05T11:29:08,400 DEBUG [ORC_GET_SPLITS #5] orc.LocalCache: Serialized tail cached for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0 2018-01-05T11:29:08,400 DEBUG [ORC_GET_SPLITS #5] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,400 DEBUG [IO-Elevator-Thread-7] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_38 queryId: djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01 instanceType: OrcEncodedDataReader 2018-01-05T11:29:08,400 DEBUG [ORC_GET_SPLITS #5] orc.LocalCache: Serialized tail cached for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0 2018-01-05T11:29:08,401 INFO [IO-Elevator-Thread-7] LlapIoImpl: Processing data for file [-5122461581151735180, 1515180546000, 744]: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0 2018-01-05T11:29:08,401 DEBUG [IO-Elevator-Thread-7] LlapIoOrc: FileSplit {3, 566}; stripes {3, 566}, 2018-01-05T11:29:08,401 DEBUG [ORC_GET_SPLITS #5] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,401 DEBUG [ORC_GET_SPLITS #5] orc.LocalCache: Serialized tail cached for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0 2018-01-05T11:29:08,401 DEBUG [IO-Elevator-Thread-7] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,401 DEBUG [ORC_GET_SPLITS #5] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,401 DEBUG [IO-Elevator-Thread-7] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,401 INFO [IO-Elevator-Thread-7] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [-5122461581151735180, 1515180546000, 744]): [{range start: 0 end: 73}] 2018-01-05T11:29:08,401 INFO [IO-Elevator-Thread-7] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [-5122461581151735180, 1515180546000, 744], base offset 3): [{start: 0 end: 11 cache buffer: 0x4d1ed212(1)}, {start: 11 end: 38 cache buffer: 0x6497a38f(1)}, {start: 38 end: 73 cache buffer: 0x23ea0f51(1)}] 2018-01-05T11:29:08,401 DEBUG [ORC_GET_SPLITS #5] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,401 INFO [ORC_GET_SPLITS #1] impl.OrcCodecPool: Got brand-new codec ZLIB 2018-01-05T11:29:08,401 DEBUG [IO-Elevator-Thread-7] impl.RecordReaderImpl: Row group 0 to 9999 is included. 2018-01-05T11:29:08,401 DEBUG [IO-Elevator-Thread-7] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,401 DEBUG [ORC_GET_SPLITS #5] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:29:08,401 INFO [IO-Elevator-Thread-7] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [-5122461581151735180, 1515180546000, 744]): [{range start: 73 end: 506}] 2018-01-05T11:29:08,401 DEBUG [ORC_GET_SPLITS #1] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:29:08,401 INFO [IO-Elevator-Thread-7] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [-5122461581151735180, 1515180546000, 744], base offset 3): [{start: 73 end: 204 cache buffer: 0x3e73bafa(1)}, {start: 204 end: 296 cache buffer: 0x67a1fd60(1)}, {start: 296 end: 337 cache buffer: 0x14c2f642(1)}, {start: 337 end: 506 cache buffer: 0x2282f43e(1)}] 2018-01-05T11:29:08,401 DEBUG [ORC_GET_SPLITS #5] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,401 DEBUG [ORC_GET_SPLITS #2] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,401 DEBUG [IO-Elevator-Thread-7] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: int streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 136 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:29:08,401 DEBUG [ORC_GET_SPLITS #5] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:29:08,401 DEBUG [ORC_GET_SPLITS #2] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:29:08,401 DEBUG [IO-Elevator-Thread-7] encoded.EncodedTreeReaderFactory: columnIndex: 2 columnType: string streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DICTIONARY_V2 dictionarySize: 80 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 89 bytes, ], indexBaseOffset=0] dictionary: true lengths: true secondary: false tz: 2018-01-05T11:29:08,401 INFO [InputInitializer {Map 3} #0] orc.OrcInputFormat: FooterCacheHitRatio: 4/4 2018-01-05T11:29:08,402 DEBUG [InputInitializer {Map 3} #0] orc.OrcInputFormat: OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0, start=3, length=574, isOriginal=true, fileLength=752, hasFooter=false, hasBase=true, deltas=0] projected_columns_uncompressed_size: 8538 2018-01-05T11:29:08,402 DEBUG [InputInitializer {Map 3} #0] orc.OrcInputFormat: OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0, start=3, length=641, isOriginal=true, fileLength=827, hasFooter=false, hasBase=true, deltas=0] projected_columns_uncompressed_size: 10054 2018-01-05T11:29:08,402 DEBUG [InputInitializer {Map 3} #0] orc.OrcInputFormat: OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0, start=3, length=641, isOriginal=true, fileLength=823, hasFooter=false, hasBase=true, deltas=0] projected_columns_uncompressed_size: 9371 2018-01-05T11:29:08,402 DEBUG [InputInitializer {Map 3} #0] orc.OrcInputFormat: OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0, start=3, length=586, isOriginal=true, fileLength=770, hasFooter=false, hasBase=true, deltas=0] projected_columns_uncompressed_size: 8226 2018-01-05T11:29:08,402 DEBUG [InputInitializer {Map 3} #0] orc.OrcInputFormat: getSplits finished 2018-01-05T11:29:08,402 INFO [InputInitializer {Map 3} #0] io.HiveInputFormat: number of splits 4 2018-01-05T11:29:08,402 DEBUG [InputInitializer {Map 3} #0] log.PerfLogger: 2018-01-05T11:29:08,402 INFO [InputInitializer {Map 3} #0] tez.HiveSplitGenerator: Number of input splits: 4. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:29:08,402 DEBUG [IO-Elevator-Thread-7] LlapIoOrc: Done with decode 2018-01-05T11:29:08,402 INFO [InputInitializer {Map 3} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/7b385878-392d-4bf1-a4fe-47a3392c1f93/map.xml 2018-01-05T11:29:08,402 DEBUG [InputInitializer {Map 3} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/7b385878-392d-4bf1-a4fe-47a3392c1f93/map.xml 2018-01-05T11:29:08,402 DEBUG [InputInitializer {Map 3} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/7b385878-392d-4bf1-a4fe-47a3392c1f93/map.xml 2018-01-05T11:29:08,402 DEBUG [IO-Elevator-Thread-7] LlapIoImpl: setDone called; closed false, done false, err null, pending 0 2018-01-05T11:29:08,402 DEBUG [InputInitializer {Map 3} #0] log.PerfLogger: 2018-01-05T11:29:08,402 INFO [InputInitializer {Map 3} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:08,402 DEBUG [IO-Elevator-Thread-7] impl.StatsRecordingThreadPool: Updated stats: instance: OrcEncodedDataReader thread name: IO-Elevator-Thread-7 thread id: 2200 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:08,402 DEBUG [InputInitializer {Map 3} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:08,403 INFO [TezTR-518813_1_38_0_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:08,403 INFO [TezTR-518813_1_38_0_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, warehouse.tab, file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0 ([-5122461581151735180, 1515180546000, 744]), 0,1]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=1, NUM_ERRORS=0, ROWS_EMITTED=122, METADATA_CACHE_HIT=2, METADATA_CACHE_MISS=0, CACHE_HIT_BYTES=506, CACHE_MISS_BYTES=0, ALLOCATED_BYTES=0, ALLOCATED_USED_BYTES=0, TOTAL_IO_TIME_NS=1277253, DECODE_TIME_NS=461938, HDFS_TIME_NS=93053, CONSUMER_TIME_NS=881807 ] 2018-01-05T11:29:08,403 DEBUG [TezTR-518813_1_38_0_0_0] encoded.OrcEncodedDataReader: Encoded reader is being stopped 2018-01-05T11:29:08,403 DEBUG [TezTR-518813_1_38_0_0_0] log.PerfLogger: 2018-01-05T11:29:08,403 DEBUG [TezTR-518813_1_38_0_0_0] mr.ObjectCache: djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:29:08,403 DEBUG [TezTR-518813_1_38_0_0_0] vector.VectorMapOperator: close called for operator MAP[0] 2018-01-05T11:29:08,403 INFO [TezTR-518813_1_38_0_0_0] vector.VectorMapOperator: Closing operator MAP[0] 2018-01-05T11:29:08,403 INFO [TezTR-518813_1_38_0_0_0] vector.VectorMapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:242, 2018-01-05T11:29:08,403 DEBUG [TezTR-518813_1_38_0_0_0] vector.VectorMapOperator: Closing child = TS[0] 2018-01-05T11:29:08,403 DEBUG [TezTR-518813_1_38_0_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:29:08,403 INFO [TezTR-518813_1_38_0_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:29:08,403 DEBUG [TezTR-518813_1_38_0_0_0] exec.TableScanOperator: Closing child = FIL[26] 2018-01-05T11:29:08,403 DEBUG [TezTR-518813_1_38_0_0_0] vector.VectorFilterOperator: close called for operator FIL[26] 2018-01-05T11:29:08,403 DEBUG [TezTR-518813_1_38_0_0_0] vector.VectorFilterOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:08,403 INFO [TezTR-518813_1_38_0_0_0] vector.VectorFilterOperator: Closing operator FIL[26] 2018-01-05T11:29:08,403 DEBUG [TezTR-518813_1_38_0_0_0] vector.VectorFilterOperator: Closing child = SEL[27] 2018-01-05T11:29:08,403 DEBUG [TezTR-518813_1_38_0_0_0] vector.VectorSelectOperator: close called for operator SEL[27] 2018-01-05T11:29:08,403 DEBUG [TezTR-518813_1_38_0_0_0] vector.VectorSelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:08,403 INFO [TezTR-518813_1_38_0_0_0] vector.VectorSelectOperator: Closing operator SEL[27] 2018-01-05T11:29:08,403 DEBUG [TezTR-518813_1_38_0_0_0] vector.VectorSelectOperator: Closing child = RS[28] 2018-01-05T11:29:08,403 DEBUG [TezTR-518813_1_38_0_0_0] reducesink.VectorReduceSinkLongOperator: close called for operator RS[28] 2018-01-05T11:29:08,403 DEBUG [TezTR-518813_1_38_0_0_0] reducesink.VectorReduceSinkLongOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:08,403 INFO [TezTR-518813_1_38_0_0_0] reducesink.VectorReduceSinkLongOperator: Closing operator RS[28] 2018-01-05T11:29:08,403 INFO [TezTR-518813_1_38_0_0_0] reducesink.VectorReduceSinkCommonOperator: RS[28]: records written - 242 2018-01-05T11:29:08,403 INFO [TezTR-518813_1_38_0_0_0] reducesink.VectorReduceSinkLongOperator: RECORDS_OUT_INTERMEDIATE_Map_1:242, 2018-01-05T11:29:08,403 DEBUG [TezTR-518813_1_38_0_0_0] reducesink.VectorReduceSinkLongOperator: 28 Close done 2018-01-05T11:29:08,403 DEBUG [TezTR-518813_1_38_0_0_0] vector.VectorSelectOperator: 27 Close done 2018-01-05T11:29:08,403 DEBUG [TezTR-518813_1_38_0_0_0] vector.VectorFilterOperator: 26 Close done 2018-01-05T11:29:08,403 DEBUG [TezTR-518813_1_38_0_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:29:08,403 DEBUG [TezTR-518813_1_38_0_0_0] vector.VectorMapOperator: 0 Close done 2018-01-05T11:29:08,403 INFO [TezTR-518813_1_38_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_38_00_000000_0 2018-01-05T11:29:08,403 INFO [TezTR-518813_1_38_0_0_0] impl.PipelinedSorter: Reducer 2: Starting flush of map output 2018-01-05T11:29:08,403 INFO [TezTR-518813_1_38_0_0_0] impl.PipelinedSorter: Reducer 2: Span0.length = 242, perItem = 25 2018-01-05T11:29:08,403 INFO [TezTR-518813_1_38_0_0_0] impl.PipelinedSorter: Reducer 2: done sorting span=0, length=242, time=0 2018-01-05T11:29:08,403 INFO [TezTR-518813_1_38_0_0_0] impl.PipelinedSorter: Reducer 2: Heap = SpanIterator<0:241> (span=Span[5242880,6244]), 2018-01-05T11:29:08,404 DEBUG [InputInitializer {Map 3} #0] log.PerfLogger: 2018-01-05T11:29:08,404 INFO [InputInitializer {Map 3} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 3 size: 3.97KB 2018-01-05T11:29:08,404 DEBUG [InputInitializer {Map 3} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0 to src new group? true 2018-01-05T11:29:08,404 DEBUG [InputInitializer {Map 3} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0 to src new group? false 2018-01-05T11:29:08,404 DEBUG [InputInitializer {Map 3} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0 to src new group? false 2018-01-05T11:29:08,404 DEBUG [InputInitializer {Map 3} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0 to src new group? false 2018-01-05T11:29:08,404 INFO [InputInitializer {Map 3} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:29:08,404 INFO [InputInitializer {Map 3} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:29:08,404 INFO [InputInitializer {Map 3} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:29:08,404 DEBUG [InputInitializer {Map 3} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 10054 2018-01-05T11:29:08,404 DEBUG [InputInitializer {Map 3} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 9371 2018-01-05T11:29:08,404 DEBUG [InputInitializer {Map 3} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 8226 2018-01-05T11:29:08,404 DEBUG [InputInitializer {Map 3} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 8538 2018-01-05T11:29:08,404 INFO [InputInitializer {Map 3} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 670 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 36189 Original splits: 4 2018-01-05T11:29:08,404 INFO [InputInitializer {Map 3} #0] grouper.TezSplitGrouper: Desired numSplits: 1 lengthPerGroup: 36189 numLocations: 1 numSplitsPerLocation: 4 numSplitsInGroup: 4 totalLength: 36189 numOriginalSplits: 4 . Grouping by length: true count: false nodeLocalOnly: false 2018-01-05T11:29:08,404 DEBUG [InputInitializer {Map 3} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 10054 2018-01-05T11:29:08,404 DEBUG [InputInitializer {Map 3} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 9371 2018-01-05T11:29:08,404 DEBUG [InputInitializer {Map 3} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 9371 2018-01-05T11:29:08,404 DEBUG [InputInitializer {Map 3} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 8226 2018-01-05T11:29:08,404 DEBUG [InputInitializer {Map 3} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 8226 2018-01-05T11:29:08,404 DEBUG [InputInitializer {Map 3} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 8538 2018-01-05T11:29:08,404 DEBUG [InputInitializer {Map 3} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 8538 2018-01-05T11:29:08,404 INFO [InputInitializer {Map 3} #0] grouper.TezSplitGrouper: Number of splits desired: 1 created: 1 splitsProcessed: 4 2018-01-05T11:29:08,404 INFO [InputInitializer {Map 3} #0] tez.SplitGrouper: Original split count is 4 grouped split count is 1, for bucket: 1 2018-01-05T11:29:08,404 INFO [InputInitializer {Map 3} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:29:08,404 INFO [InputInitializer {Map 3} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: b on vertex vertex_1515180518813_0001_38_01 [Map 3] 2018-01-05T11:29:08,405 INFO [App Shared Pool - #2] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_38_01 [Map 3] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:08,405 INFO [App Shared Pool - #2] impl.VertexImpl: Got updated RootInputsSpecs: {b=forAllWorkUnits=true, update=[1]} 2018-01-05T11:29:08,405 INFO [App Shared Pool - #2] impl.VertexImpl: Vertex vertex_1515180518813_0001_38_01 [Map 3] parallelism set to 1 2018-01-05T11:29:08,405 INFO [App Shared Pool - #2] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_38_01 [Map 3] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:08,405 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_38][Event:VERTEX_INITIALIZED]: vertexName=Map 3, vertexId=vertex_1515180518813_0001_38_01, initRequestedTime=1515180548321, initedTime=1515180548405, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:08,405 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_38_01 [Map 3] with distanceFromRoot: 0 2018-01-05T11:29:08,405 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_38_01 [Map 3] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:29:08,405 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_38][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_38_01, startRequestedTime=1515180548329, startedTime=1515180548405 2018-01-05T11:29:08,405 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_38][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_38_01, reconfigureDoneTime=1515180548405, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:29:08,405 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_38_01 [Map 3] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:08,405 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_38_01 for vertex: vertex_1515180518813_0001_38_02 [Reducer 2] numStartedSources: 2 numSources: 2 2018-01-05T11:29:08,405 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_38_02 [Reducer 2] with distanceFromRoot: 1 2018-01-05T11:29:08,405 INFO [App Shared Pool - #2] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 3 2018-01-05T11:29:08,405 INFO [App Shared Pool - #2] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 3 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:29:08,405 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_38][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_38_02, startRequestedTime=1515180548405, startedTime=1515180548405 2018-01-05T11:29:08,405 INFO [App Shared Pool - #2] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_38_01 [Map 3] 2018-01-05T11:29:08,405 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_38_02 [Reducer 2] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:08,405 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_38][Event:TASK_STARTED]: vertexName=Map 3, taskId=task_1515180518813_0001_38_01_000000, scheduledTime=1515180548405, launchTime=1515180548405 2018-01-05T11:29:08,405 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: OnVertexStarted vertex: Reducer 2 with 0 source tasks and 1 pending tasks 2018-01-05T11:29:08,405 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_38_01 [Map 3] 2018-01-05T11:29:08,406 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Map 3 in vertex: Reducer 2 numBipartiteSourceTasks: 1 2018-01-05T11:29:08,406 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_38_01_000000_0, priority=11, capability=, hosts=null 2018-01-05T11:29:08,406 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Map 1 in vertex: Reducer 2 numBipartiteSourceTasks: 2 2018-01-05T11:29:08,406 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=11,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:08,406 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=11,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:08,406 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:08,406 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_38_01_000000_0 on hosts=null 2018-01-05T11:29:08,406 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_38_01_000000_0 on hosts=null 2018-01-05T11:29:08,406 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_38_01_000000_0 2018-01-05T11:29:08,406 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:08,406 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #91, task=TaskInfo{task=attempt_1515180518813_0001_38_01_000000_0, priority=11, startTime=0, containerId=null, uniqueId=90, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, to container=container_111101111_0001_00_000091 2018-01-05T11:29:08,406 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_38_01_000000_0; false 2018-01-05T11:29:08,406 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_38_01_000000_0, priority=11, startTime=311568915, containerId=container_111101111_0001_00_000091assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=2, ac=6, commF=false, disabled=false}, uniqueId=90, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:08,406 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000091 for node: 10.22.8.182:49818 2018-01-05T11:29:08,406 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000091, attempt_1515180518813_0001_38_01_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:08,406 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000091 for node: 10.22.8.182:49818 2018-01-05T11:29:08,407 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000091 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:08,407 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000091, launchTime=1515180548407 2018-01-05T11:29:08,407 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_38_01_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000091] on NM: [10.22.8.182:49818] 2018-01-05T11:29:08,407 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:08,407 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_38_01_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000091, dagName=select * from tab a join tab_part b ...b.key (Stage-1), vertexName=Map 3, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{b,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{Reducer 2,org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput,1}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180548312, firstAttemptStartTime=1515180548405, currentAttemptStartTime=1515180548406} 2018-01-05T11:29:08,407 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:08,407 INFO [IPC Server handler 1 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000091, QueryId=djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01-38, DagId=38, VertexName=Map 3, TaskId=0, TaskAttemptId=0, SubmitTime=1515180548407 2018-01-05T11:29:08,407 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_38][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 3, taskAttemptId=attempt_1515180518813_0001_38_01_000000_0, startTime=1515180548407, containerId=container_111101111_0001_00_000091, nodeId=10.22.8.182:49818 2018-01-05T11:29:08,408 DEBUG [IPC Server handler 1 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=38} with the ShuffleHandler 2018-01-05T11:29:08,408 DEBUG [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/38] 2018-01-05T11:29:08,408 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=3, runningFragmentCount=1 2018-01-05T11:29:08,408 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_38_01_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:08,408 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_38_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180548405, dagStartTime=1515180548312, withinDagPriority=11, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:29:08,408 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_38_01_000000_0 : ACCEPTED 2018-01-05T11:29:08,408 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_38_01_000000_0, canFinish=true. Current state: preemptionQueueSize=1, numSlotsAvailable=3, waitQueueSize=1 2018-01-05T11:29:08,408 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_38_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180548405, dagStartTime=1515180548312, withinDagPriority=11, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:29:08,408 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_38_01_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:08,408 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_38_01_000000_0 to pre-emption queue 2018-01-05T11:29:08,408 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_38_01_000000_0: true 2018-01-05T11:29:08,408 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: taskOwner hashCode:2029022705 2018-01-05T11:29:08,408 INFO [TaskCommunicator # 6] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_38_01_000000_0 2018-01-05T11:29:08,408 INFO [TaskCommunicator # 6] GuaranteedTasks: Received response for attempt_1515180518813_0001_38_01_000000_0, true 2018-01-05T11:29:08,409 INFO [Task-Executor-2] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : select * from tab a join tab_part b ...b.key (Stage-1), VertexName: Map 3, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_38_01_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=b, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=Reducer 2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ] 2018-01-05T11:29:08,409 INFO [Task-Executor-2] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1021837312, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:08,409 INFO [Task-Executor-2] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_38_01_000000_0 vertexName: Map 3 2018-01-05T11:29:08,409 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000091, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_38_01_000000_0, eventCount=1 } 2018-01-05T11:29:08,409 DEBUG [TezTR-518813_1_38_1_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_38 queryId: djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01 instanceType: TaskRunner2Callable 2018-01-05T11:29:08,409 INFO [TezTR-518813_1_38_1_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_38_01_000000_0 2018-01-05T11:29:08,410 DEBUG [TezTR-518813_1_38_1_0_0] log.PerfLogger: 2018-01-05T11:29:08,411 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:08,412 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:08,414 INFO [IPC Server handler 19 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_38_01_000000_0 sent events: (0-1). 2018-01-05T11:29:08,414 INFO [I/O Setup 0 Initialize: {b}] input.MRInput: b using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:29:08,414 INFO [I/O Setup 0 Initialize: {b}] input.MRInputLegacy: b MRInputLegacy deferring initialization 2018-01-05T11:29:08,414 DEBUG [TezTR-518813_1_38_1_0_0] log.PerfLogger: 2018-01-05T11:29:08,414 INFO [TezTR-518813_1_38_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:08,414 INFO [TezTR-518813_1_38_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:08,414 INFO [TezTR-518813_1_38_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:08,414 INFO [TezTR-518813_1_38_1_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:08,414 INFO [TezTR-518813_1_38_1_0_0] resources.MemoryDistributor: InitialRequests=[b:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [Reducer 2:OUTPUT:10485760:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput] 2018-01-05T11:29:08,414 INFO [TezTR-518813_1_38_1_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:08,414 INFO [TezTR-518813_1_38_1_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:08,414 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:29:08,414 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_38_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:29:08,414 INFO [TezTR-518813_1_38_1_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 10485760, TotalRequestedScaled: 9679163.076923078, TotalJVMHeap: 1021837312, TotalAvailable: 684630999, TotalRequested/TotalJVMHeap:0.01 2018-01-05T11:29:08,414 INFO [TezTR-518813_1_38_1_0_0] resources.MemoryDistributor: Allocations=[b:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:10485760:10485760] 2018-01-05T11:29:08,414 INFO [TezTR-518813_1_38_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:08,414 INFO [TezTR-518813_1_38_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: b being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:08,414 INFO [TezTR-518813_1_38_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:29:08,414 INFO [TezTR-518813_1_38_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:08,414 INFO [I/O Setup 1 Start: {b}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: b 2018-01-05T11:29:08,414 INFO [TezTR-518813_1_38_1_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:08,414 INFO [TezTR-518813_1_38_1_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_38_01_000000_0 2018-01-05T11:29:08,414 DEBUG [TezTR-518813_1_38_1_0_0] log.PerfLogger: 2018-01-05T11:29:08,414 DEBUG [TezTR-518813_1_38_1_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_38_01_000000_0_10275 2018-01-05T11:29:08,414 DEBUG [TezTR-518813_1_38_1_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:08,414 DEBUG [TezTR-518813_1_38_1_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_38_01_000000_0 2018-01-05T11:29:08,415 DEBUG [TezTR-518813_1_38_1_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:08,415 DEBUG [TezTR-518813_1_38_1_0_0] log.PerfLogger: 2018-01-05T11:29:08,415 DEBUG [TezTR-518813_1_38_1_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:08,415 DEBUG [TezTR-518813_1_38_1_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:08,415 DEBUG [TezTR-518813_1_38_1_0_0] mr.ObjectCache: Creating djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01_Map 3__MAP_PLAN__ 2018-01-05T11:29:08,415 INFO [TezTR-518813_1_38_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/7b385878-392d-4bf1-a4fe-47a3392c1f93/map.xml 2018-01-05T11:29:08,415 DEBUG [TezTR-518813_1_38_1_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/7b385878-392d-4bf1-a4fe-47a3392c1f93/map.xml 2018-01-05T11:29:08,415 DEBUG [TezTR-518813_1_38_1_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/7b385878-392d-4bf1-a4fe-47a3392c1f93/map.xml 2018-01-05T11:29:08,415 DEBUG [TezTR-518813_1_38_1_0_0] log.PerfLogger: 2018-01-05T11:29:08,415 INFO [TezTR-518813_1_38_1_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:08,415 DEBUG [TezTR-518813_1_38_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:08,416 DEBUG [TezTR-518813_1_38_1_0_0] log.PerfLogger: 2018-01-05T11:29:08,416 INFO [TezTR-518813_1_38_1_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 3 size: 3.97KB 2018-01-05T11:29:08,416 DEBUG [TezTR-518813_1_38_1_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:08,416 INFO [TezTR-518813_1_38_1_0_0] tez.MapRecordProcessor: The input names are: [b] 2018-01-05T11:29:08,417 INFO [TezTR-518813_1_38_1_0_0] input.MRInput: Initialized MRInput: b 2018-01-05T11:29:08,417 DEBUG [TezTR-518813_1_38_1_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 4 2018-01-05T11:29:08,417 INFO [TezTR-518813_1_38_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/7b385878-392d-4bf1-a4fe-47a3392c1f93/map.xml 2018-01-05T11:29:08,417 DEBUG [TezTR-518813_1_38_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:08,417 DEBUG [TezTR-518813_1_38_1_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat]} 2018-01-05T11:29:08,417 DEBUG [TezTR-518813_1_38_1_0_0] io.HiveInputFormat: Pushdown initiated with filterText = key is not null, filterExpr = GenericUDFOPNotNull(Column[key]), serializedFilterExpr = AQEAamF2YS51dGlsLkFycmF5TGlz9AEBAQFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVDb2x1bW5EZXPjAQFrZfkAAAGCYgECb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5zZXJkZTIudHlwZWluZm8uUHJpbWl0aXZlVHlwZUluZu8BAWlu9AEDb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGT1BOb3ROdWzsAQEEb3JnLmFwYWNoZS5oYWRvb3AuaW8uQm9vbGVhbldyaXRhYmzlAQAAAQIBAWJvb2xlYe4= (desc) 2018-01-05T11:29:08,417 INFO [TezTR-518813_1_38_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/7b385878-392d-4bf1-a4fe-47a3392c1f93/map.xml 2018-01-05T11:29:08,417 DEBUG [TezTR-518813_1_38_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:08,417 DEBUG [TezTR-518813_1_38_1_0_0] io.HiveInputFormat: Processing org.apache.hadoop.hive.ql.io.orc.OrcInputFormat 2018-01-05T11:29:08,418 DEBUG [TezTR-518813_1_38_1_0_0] impl.LlapRecordReader: Initializing for input Map 3 2018-01-05T11:29:08,418 INFO [TezTR-518813_1_38_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/7b385878-392d-4bf1-a4fe-47a3392c1f93/map.xml 2018-01-05T11:29:08,418 DEBUG [TezTR-518813_1_38_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:08,418 INFO [TezTR-518813_1_38_1_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_38_01_000000_0 2018-01-05T11:29:08,418 INFO [TezTR-518813_1_38_1_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:08,418 DEBUG [TezTR-518813_1_38_1_0_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:08,418 DEBUG [TezTR-518813_1_38_1_0_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:08,418 DEBUG [TezTR-518813_1_38_1_0_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:08,418 DEBUG [TezTR-518813_1_38_1_0_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:08,418 DEBUG [TezTR-518813_1_38_1_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,419 DEBUG [TezTR-518813_1_38_1_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,419 INFO [TezTR-518813_1_38_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/7b385878-392d-4bf1-a4fe-47a3392c1f93/map.xml 2018-01-05T11:29:08,419 DEBUG [TezTR-518813_1_38_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:08,419 DEBUG [TezTR-518813_1_38_1_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0 2018-01-05T11:29:08,419 DEBUG [IO-Elevator-Thread-8] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_38 queryId: djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01 instanceType: OrcEncodedDataReader 2018-01-05T11:29:08,419 INFO [IO-Elevator-Thread-8] LlapIoImpl: Processing data for file [6462268725460710401, 1515180546000, 827]: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0 2018-01-05T11:29:08,419 INFO [TezTR-518813_1_38_1_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0, start=3, length=641, isOriginal=true, fileLength=827, hasFooter=false, hasBase=true, deltas=0], org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0, start=3, length=641, isOriginal=true, fileLength=823, hasFooter=false, hasBase=true, deltas=0], org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0, start=3, length=586, isOriginal=true, fileLength=770, hasFooter=false, hasBase=true, deltas=0], org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0, start=3, length=574, isOriginal=true, fileLength=752, hasFooter=false, hasBase=true, deltas=0]], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=2442} 2018-01-05T11:29:08,419 DEBUG [IO-Elevator-Thread-8] LlapIoOrc: FileSplit {3, 641}; stripes {3, 641}, 2018-01-05T11:29:08,419 INFO [TezTR-518813_1_38_1_0_0] input.MRInput: b initialized RecordReader from event 2018-01-05T11:29:08,419 DEBUG [TezTR-518813_1_38_1_0_0] tez.MapRecordProcessor: Starting Output: Reducer 2 2018-01-05T11:29:08,420 DEBUG [IO-Elevator-Thread-8] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,420 DEBUG [IO-Elevator-Thread-8] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,420 INFO [IO-Elevator-Thread-8] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725460710401, 1515180546000, 827]): [{range start: 0 end: 79}] 2018-01-05T11:29:08,420 INFO [TezTR-518813_1_38_1_0_0] impl.ExternalSorter: Reducer 2 using: memoryMb=10, keySerializerClass=class org.apache.hadoop.hive.ql.io.HiveKey, valueSerializerClass=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization$TezBytesWritableSerializer@13335adb, comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator@5ab4e2b6, partitioner=org.apache.tez.mapreduce.partition.MRPartitioner, serialization=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.hadoop.io.serializer.WritableSerialization, org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization, org.apache.hadoop.io.serializer.avro.AvroReflectSerialization, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:29:08,420 INFO [IO-Elevator-Thread-8] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [6462268725460710401, 1515180546000, 827], base offset 3): [{start: 0 end: 12 cache buffer: 0x7302251(1)}, {start: 12 end: 40 cache buffer: 0x2cb64d2b(1)}, {start: 40 end: 79 cache buffer: 0x2488a149(1)}] 2018-01-05T11:29:08,420 INFO [TezTR-518813_1_38_1_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 2: , UsingHashComparator=true 2018-01-05T11:29:08,420 DEBUG [IO-Elevator-Thread-8] impl.RecordReaderImpl: Row group 0 to 9999 is included. 2018-01-05T11:29:08,420 DEBUG [IO-Elevator-Thread-8] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,420 INFO [IO-Elevator-Thread-8] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725460710401, 1515180546000, 827]): [{range start: 79 end: 581}] 2018-01-05T11:29:08,420 INFO [IO-Elevator-Thread-8] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [6462268725460710401, 1515180546000, 827], base offset 3): [{start: 79 end: 240 cache buffer: 0x24272ac5(1)}, {start: 240 end: 374 cache buffer: 0x3ae6f47f(1)}, {start: 374 end: 406 cache buffer: 0x5b058883(1)}, {start: 406 end: 581 cache buffer: 0x640a65db(1)}] 2018-01-05T11:29:08,420 DEBUG [IO-Elevator-Thread-8] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: int streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 200 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:29:08,420 DEBUG [IO-Elevator-Thread-8] encoded.EncodedTreeReaderFactory: columnIndex: 2 columnType: string streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DICTIONARY_V2 dictionarySize: 81 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 131 bytes, ], indexBaseOffset=0] dictionary: true lengths: true secondary: false tz: 2018-01-05T11:29:08,421 INFO [TezTR-518813_1_38_0_0_0] impl.PipelinedSorter: Reducer 2: Spilling to /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/38/output/attempt_1515180518813_0001_38_00_000000_0_10274_0/file.out 2018-01-05T11:29:08,421 DEBUG [IO-Elevator-Thread-8] LlapIoOrc: Done with decode 2018-01-05T11:29:08,421 DEBUG [IO-Elevator-Thread-8] LlapIoImpl: setDone called; closed false, done false, err null, pending 1 2018-01-05T11:29:08,421 DEBUG [IO-Elevator-Thread-8] impl.StatsRecordingThreadPool: Updated stats: instance: OrcEncodedDataReader thread name: IO-Elevator-Thread-8 thread id: 2387 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:08,421 INFO [TezTR-518813_1_38_1_0_0] impl.PipelinedSorter: Newly allocated block size=10485760, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=10485760, total=10485760 2018-01-05T11:29:08,421 INFO [TezTR-518813_1_38_1_0_0] impl.PipelinedSorter: Pre allocating rest of memory buffers upfront 2018-01-05T11:29:08,421 INFO [TezTR-518813_1_38_1_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 2: , UsingHashComparator=true#blocks=1, maxMemUsage=10485760, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=10485760, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=10 2018-01-05T11:29:08,421 INFO [TezTR-518813_1_38_1_0_0] impl.PipelinedSorter: Reducer 2: reserved.remaining()=10485760, reserved.metasize=5242880 2018-01-05T11:29:08,421 INFO [TezTR-518813_1_38_1_0_0] tez.MapRecordProcessor: Main input name is Map 3 2018-01-05T11:29:08,421 INFO [TezTR-518813_1_38_1_0_0] vector.VectorMapOperator: Initializing operator MAP[0] 2018-01-05T11:29:08,421 DEBUG [TezTR-518813_1_38_1_0_0] vector.VectorMapOperator: Initialization Done 0 MAP 2018-01-05T11:29:08,421 DEBUG [TezTR-518813_1_38_1_0_0] vector.VectorMapOperator: Operator 0 MAP initialized 2018-01-05T11:29:08,421 DEBUG [TezTR-518813_1_38_1_0_0] vector.VectorMapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:29:08,421 INFO [TezTR-518813_1_38_1_0_0] tez.MapRecordProcessor: Id =0 Id =3 Id =29 Id =30 Id =31 <\Children> Id = 30 null<\Parent> <\RS> <\Children> Id = 29 null<\Parent> <\SEL> <\Children> Id = 3 null<\Parent> <\FIL> <\Children> <\Parent> <\TS> <\Children> <\Parent> <\MAP> 2018-01-05T11:29:08,422 INFO [TezTR-518813_1_38_1_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:08,422 DEBUG [TezTR-518813_1_38_1_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:08,422 INFO [TezTR-518813_1_38_1_0_0] exec.TableScanOperator: Initializing operator TS[3] 2018-01-05T11:29:08,422 DEBUG [TezTR-518813_1_38_1_0_0] exec.TableScanOperator: Initialization Done 3 TS 2018-01-05T11:29:08,422 DEBUG [TezTR-518813_1_38_1_0_0] exec.TableScanOperator: Operator 3 TS initialized 2018-01-05T11:29:08,422 DEBUG [TezTR-518813_1_38_1_0_0] exec.TableScanOperator: Initializing children of 3 TS 2018-01-05T11:29:08,422 DEBUG [TezTR-518813_1_38_1_0_0] vector.VectorFilterOperator: Initializing child 29 FIL 2018-01-05T11:29:08,422 INFO [TezTR-518813_1_38_1_0_0] vector.VectorFilterOperator: Initializing operator FIL[29] 2018-01-05T11:29:08,422 DEBUG [TezTR-518813_1_38_1_0_0] vector.VectorFilterOperator: Initialization Done 29 FIL 2018-01-05T11:29:08,422 DEBUG [TezTR-518813_1_38_1_0_0] vector.VectorFilterOperator: Operator 29 FIL initialized 2018-01-05T11:29:08,422 DEBUG [TezTR-518813_1_38_1_0_0] vector.VectorFilterOperator: Initializing children of 29 FIL 2018-01-05T11:29:08,422 DEBUG [TezTR-518813_1_38_1_0_0] vector.VectorSelectOperator: Initializing child 30 SEL 2018-01-05T11:29:08,422 INFO [TezTR-518813_1_38_1_0_0] vector.VectorSelectOperator: Initializing operator SEL[30] 2018-01-05T11:29:08,422 DEBUG [TezTR-518813_1_38_1_0_0] vector.VectorSelectOperator: Initialization Done 30 SEL 2018-01-05T11:29:08,422 DEBUG [TezTR-518813_1_38_1_0_0] vector.VectorSelectOperator: Operator 30 SEL initialized 2018-01-05T11:29:08,422 DEBUG [TezTR-518813_1_38_1_0_0] vector.VectorSelectOperator: Initializing children of 30 SEL 2018-01-05T11:29:08,422 DEBUG [TezTR-518813_1_38_1_0_0] reducesink.VectorReduceSinkLongOperator: Initializing child 31 RS 2018-01-05T11:29:08,422 INFO [TezTR-518813_1_38_1_0_0] reducesink.VectorReduceSinkLongOperator: Initializing operator RS[31] 2018-01-05T11:29:08,422 INFO [TezTR-518813_1_38_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/7b385878-392d-4bf1-a4fe-47a3392c1f93/map.xml 2018-01-05T11:29:08,422 DEBUG [TezTR-518813_1_38_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:08,422 INFO [TezTR-518813_1_38_1_0_0] reducesink.VectorReduceSinkCommonOperator: Using tag = 1 2018-01-05T11:29:08,422 DEBUG [TezTR-518813_1_38_1_0_0] reducesink.VectorReduceSinkLongOperator: Initialization Done 31 RS 2018-01-05T11:29:08,422 DEBUG [TezTR-518813_1_38_1_0_0] reducesink.VectorReduceSinkLongOperator: Operator 31 RS initialized 2018-01-05T11:29:08,422 DEBUG [TezTR-518813_1_38_1_0_0] reducesink.VectorReduceSinkLongOperator: Initialization Done 31 RS done is reset. 2018-01-05T11:29:08,422 DEBUG [TezTR-518813_1_38_1_0_0] vector.VectorSelectOperator: Initialization Done 30 SEL done is reset. 2018-01-05T11:29:08,422 DEBUG [TezTR-518813_1_38_1_0_0] vector.VectorFilterOperator: Initialization Done 29 FIL done is reset. 2018-01-05T11:29:08,422 DEBUG [TezTR-518813_1_38_1_0_0] exec.TableScanOperator: Initialization Done 3 TS done is reset. 2018-01-05T11:29:08,422 INFO [TezTR-518813_1_38_1_0_0] exec.OperatorUtils: Setting output collector: RS[31] --> Reducer 2 2018-01-05T11:29:08,422 DEBUG [TezTR-518813_1_38_1_0_0] log.PerfLogger: 2018-01-05T11:29:08,422 INFO [TezTR-518813_1_38_1_0_0] reducesink.VectorReduceSinkCommonOperator: RS[31]: records written - 1 2018-01-05T11:29:08,422 INFO [TezTR-518813_1_38_1_0_0] reducesink.VectorReduceSinkCommonOperator: RS[31]: records written - 10 2018-01-05T11:29:08,422 INFO [TezTR-518813_1_38_1_0_0] reducesink.VectorReduceSinkCommonOperator: RS[31]: records written - 100 2018-01-05T11:29:08,422 INFO [TezTR-518813_1_38_1_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:08,422 INFO [TezTR-518813_1_38_1_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, warehouse.tab_part, file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0 ([6462268725460710401, 1515180546000, 827]), 0,1]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=1, NUM_ERRORS=0, ROWS_EMITTED=138, METADATA_CACHE_HIT=2, METADATA_CACHE_MISS=0, CACHE_HIT_BYTES=581, CACHE_MISS_BYTES=0, ALLOCATED_BYTES=0, ALLOCATED_USED_BYTES=0, TOTAL_IO_TIME_NS=1781836, DECODE_TIME_NS=324129, HDFS_TIME_NS=40606, CONSUMER_TIME_NS=427909 ] 2018-01-05T11:29:08,423 DEBUG [TezTR-518813_1_38_1_0_0] encoded.OrcEncodedDataReader: Encoded reader is being stopped 2018-01-05T11:29:08,423 DEBUG [TezTR-518813_1_38_1_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 1 of 4 2018-01-05T11:29:08,423 DEBUG [TezTR-518813_1_38_1_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat]} 2018-01-05T11:29:08,423 DEBUG [TezTR-518813_1_38_1_0_0] io.HiveInputFormat: Pushdown initiated with filterText = key is not null, filterExpr = GenericUDFOPNotNull(Column[key]), serializedFilterExpr = AQEAamF2YS51dGlsLkFycmF5TGlz9AEBAQFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVDb2x1bW5EZXPjAQFrZfkAAAGCYgECb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5zZXJkZTIudHlwZWluZm8uUHJpbWl0aXZlVHlwZUluZu8BAWlu9AEDb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGT1BOb3ROdWzsAQEEb3JnLmFwYWNoZS5oYWRvb3AuaW8uQm9vbGVhbldyaXRhYmzlAQAAAQIBAWJvb2xlYe4= (desc) 2018-01-05T11:29:08,423 INFO [TezTR-518813_1_38_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/7b385878-392d-4bf1-a4fe-47a3392c1f93/map.xml 2018-01-05T11:29:08,423 DEBUG [TezTR-518813_1_38_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:08,423 DEBUG [TezTR-518813_1_38_1_0_0] io.HiveInputFormat: Processing org.apache.hadoop.hive.ql.io.orc.OrcInputFormat 2018-01-05T11:29:08,424 DEBUG [TezTR-518813_1_38_1_0_0] impl.LlapRecordReader: Initializing for input Map 3 2018-01-05T11:29:08,424 INFO [TezTR-518813_1_38_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/7b385878-392d-4bf1-a4fe-47a3392c1f93/map.xml 2018-01-05T11:29:08,424 DEBUG [TezTR-518813_1_38_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:08,424 INFO [TezTR-518813_1_38_1_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_38_01_000000_0 2018-01-05T11:29:08,424 INFO [TezTR-518813_1_38_1_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:08,424 DEBUG [TezTR-518813_1_38_1_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,424 DEBUG [TezTR-518813_1_38_1_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,424 INFO [TezTR-518813_1_38_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/7b385878-392d-4bf1-a4fe-47a3392c1f93/map.xml 2018-01-05T11:29:08,424 DEBUG [TezTR-518813_1_38_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:08,424 DEBUG [TezTR-518813_1_38_1_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0 2018-01-05T11:29:08,424 DEBUG [IO-Elevator-Thread-9] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_38 queryId: djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01 instanceType: OrcEncodedDataReader 2018-01-05T11:29:08,425 INFO [IO-Elevator-Thread-9] LlapIoImpl: Processing data for file [6462268725462206130, 1515180546000, 823]: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0 2018-01-05T11:29:08,425 DEBUG [IO-Elevator-Thread-9] LlapIoOrc: FileSplit {3, 641}; stripes {3, 641}, 2018-01-05T11:29:08,425 DEBUG [IO-Elevator-Thread-9] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,425 DEBUG [IO-Elevator-Thread-9] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,425 INFO [IO-Elevator-Thread-9] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725462206130, 1515180546000, 823]): [{range start: 0 end: 76}] 2018-01-05T11:29:08,425 INFO [IO-Elevator-Thread-9] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [6462268725462206130, 1515180546000, 823], base offset 3): [{start: 0 end: 12 cache buffer: 0x469c2dcf(1)}, {start: 12 end: 40 cache buffer: 0x2f556695(1)}, {start: 40 end: 76 cache buffer: 0x51d31fa3(1)}] 2018-01-05T11:29:08,425 DEBUG [IO-Elevator-Thread-9] impl.RecordReaderImpl: Row group 0 to 9999 is included. 2018-01-05T11:29:08,425 DEBUG [IO-Elevator-Thread-9] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,425 INFO [IO-Elevator-Thread-9] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725462206130, 1515180546000, 823]): [{range start: 76 end: 581}] 2018-01-05T11:29:08,425 INFO [IO-Elevator-Thread-9] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [6462268725462206130, 1515180546000, 823], base offset 3): [{start: 76 end: 242 cache buffer: 0x6e3a2cb3(1)}, {start: 242 end: 373 cache buffer: 0x6a5f9e2f(1)}, {start: 373 end: 410 cache buffer: 0x23b8aac0(1)}, {start: 410 end: 581 cache buffer: 0x16f27311(1)}] 2018-01-05T11:29:08,425 DEBUG [IO-Elevator-Thread-9] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: int streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 200 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:29:08,425 DEBUG [IO-Elevator-Thread-9] encoded.EncodedTreeReaderFactory: columnIndex: 2 columnType: string streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DICTIONARY_V2 dictionarySize: 80 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 129 bytes, ], indexBaseOffset=0] dictionary: true lengths: true secondary: false tz: 2018-01-05T11:29:08,425 DEBUG [IO-Elevator-Thread-9] LlapIoOrc: Done with decode 2018-01-05T11:29:08,426 DEBUG [IO-Elevator-Thread-9] LlapIoImpl: setDone called; closed false, done false, err null, pending 0 2018-01-05T11:29:08,426 DEBUG [IO-Elevator-Thread-9] impl.StatsRecordingThreadPool: Updated stats: instance: OrcEncodedDataReader thread name: IO-Elevator-Thread-9 thread id: 2389 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:08,426 INFO [TezTR-518813_1_38_1_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:08,426 INFO [TezTR-518813_1_38_1_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, warehouse.tab_part, file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0 ([6462268725462206130, 1515180546000, 823]), 0,1]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=1, NUM_ERRORS=0, ROWS_EMITTED=128, METADATA_CACHE_HIT=2, METADATA_CACHE_MISS=0, CACHE_HIT_BYTES=581, CACHE_MISS_BYTES=0, ALLOCATED_BYTES=0, ALLOCATED_USED_BYTES=0, TOTAL_IO_TIME_NS=1032886, DECODE_TIME_NS=311754, HDFS_TIME_NS=48067, CONSUMER_TIME_NS=667960 ] 2018-01-05T11:29:08,426 DEBUG [TezTR-518813_1_38_1_0_0] encoded.OrcEncodedDataReader: Encoded reader is being stopped 2018-01-05T11:29:08,426 DEBUG [TezTR-518813_1_38_1_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 2 of 4 2018-01-05T11:29:08,426 DEBUG [TezTR-518813_1_38_1_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat]} 2018-01-05T11:29:08,426 DEBUG [TezTR-518813_1_38_1_0_0] io.HiveInputFormat: Pushdown initiated with filterText = key is not null, filterExpr = GenericUDFOPNotNull(Column[key]), serializedFilterExpr = AQEAamF2YS51dGlsLkFycmF5TGlz9AEBAQFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVDb2x1bW5EZXPjAQFrZfkAAAGCYgECb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5zZXJkZTIudHlwZWluZm8uUHJpbWl0aXZlVHlwZUluZu8BAWlu9AEDb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGT1BOb3ROdWzsAQEEb3JnLmFwYWNoZS5oYWRvb3AuaW8uQm9vbGVhbldyaXRhYmzlAQAAAQIBAWJvb2xlYe4= (desc) 2018-01-05T11:29:08,426 INFO [TezTR-518813_1_38_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/7b385878-392d-4bf1-a4fe-47a3392c1f93/map.xml 2018-01-05T11:29:08,426 DEBUG [TezTR-518813_1_38_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:08,426 DEBUG [TezTR-518813_1_38_1_0_0] io.HiveInputFormat: Processing org.apache.hadoop.hive.ql.io.orc.OrcInputFormat 2018-01-05T11:29:08,427 DEBUG [TezTR-518813_1_38_1_0_0] impl.LlapRecordReader: Initializing for input Map 3 2018-01-05T11:29:08,427 INFO [TezTR-518813_1_38_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/7b385878-392d-4bf1-a4fe-47a3392c1f93/map.xml 2018-01-05T11:29:08,427 DEBUG [TezTR-518813_1_38_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:08,427 INFO [TezTR-518813_1_38_1_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_38_01_000000_0 2018-01-05T11:29:08,428 INFO [TezTR-518813_1_38_1_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:08,428 DEBUG [TezTR-518813_1_38_1_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,428 DEBUG [TezTR-518813_1_38_1_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,428 INFO [TezTR-518813_1_38_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/7b385878-392d-4bf1-a4fe-47a3392c1f93/map.xml 2018-01-05T11:29:08,428 DEBUG [TezTR-518813_1_38_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:08,428 DEBUG [TezTR-518813_1_38_1_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0 2018-01-05T11:29:08,428 DEBUG [IO-Elevator-Thread-0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_38 queryId: djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01 instanceType: OrcEncodedDataReader 2018-01-05T11:29:08,428 INFO [IO-Elevator-Thread-0] LlapIoImpl: Processing data for file [6462268725463701859, 1515180546000, 770]: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0 2018-01-05T11:29:08,428 DEBUG [IO-Elevator-Thread-0] LlapIoOrc: FileSplit {3, 586}; stripes {3, 586}, 2018-01-05T11:29:08,428 DEBUG [IO-Elevator-Thread-0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,428 DEBUG [IO-Elevator-Thread-0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,428 INFO [IO-Elevator-Thread-0] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725463701859, 1515180546000, 770]): [{range start: 0 end: 76}] 2018-01-05T11:29:08,428 INFO [IO-Elevator-Thread-0] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [6462268725463701859, 1515180546000, 770], base offset 3): [{start: 0 end: 11 cache buffer: 0x5a722c41(1)}, {start: 11 end: 38 cache buffer: 0x7b40d4ac(1)}, {start: 38 end: 76 cache buffer: 0xbfc9cb2(1)}] 2018-01-05T11:29:08,429 DEBUG [IO-Elevator-Thread-0] impl.RecordReaderImpl: Row group 0 to 9999 is included. 2018-01-05T11:29:08,429 INFO [IO-Elevator-Thread-0] impl.OrcCodecPool: Got brand-new codec ZLIB 2018-01-05T11:29:08,429 INFO [IO-Elevator-Thread-0] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725463701859, 1515180546000, 770]): [{range start: 76 end: 526}] 2018-01-05T11:29:08,429 INFO [IO-Elevator-Thread-0] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [6462268725463701859, 1515180546000, 770], base offset 3): [{start: 76 end: 223 cache buffer: 0x1ddca8ea(1)}, {start: 223 end: 336 cache buffer: 0x6044fa49(1)}, {start: 336 end: 367 cache buffer: 0xfb2e3ff(1)}, {start: 367 end: 526 cache buffer: 0x656d39b8(1)}] 2018-01-05T11:29:08,429 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: int streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 171 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:29:08,429 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 2 columnType: string streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DICTIONARY_V2 dictionarySize: 73 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 110 bytes, ], indexBaseOffset=0] dictionary: true lengths: true secondary: false tz: 2018-01-05T11:29:08,429 DEBUG [IO-Elevator-Thread-0] LlapIoOrc: Done with decode 2018-01-05T11:29:08,429 DEBUG [IO-Elevator-Thread-0] LlapIoImpl: setDone called; closed false, done false, err null, pending 0 2018-01-05T11:29:08,429 DEBUG [IO-Elevator-Thread-0] impl.StatsRecordingThreadPool: Updated stats: instance: OrcEncodedDataReader thread name: IO-Elevator-Thread-0 thread id: 1252 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:08,429 INFO [TezTR-518813_1_38_1_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:08,429 INFO [TezTR-518813_1_38_1_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, warehouse.tab_part, file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0 ([6462268725463701859, 1515180546000, 770]), 0,1]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=1, NUM_ERRORS=0, ROWS_EMITTED=115, METADATA_CACHE_HIT=2, METADATA_CACHE_MISS=0, CACHE_HIT_BYTES=526, CACHE_MISS_BYTES=0, ALLOCATED_BYTES=0, ALLOCATED_USED_BYTES=0, TOTAL_IO_TIME_NS=1032738, DECODE_TIME_NS=296802, HDFS_TIME_NS=38110, CONSUMER_TIME_NS=288144 ] 2018-01-05T11:29:08,429 DEBUG [TezTR-518813_1_38_1_0_0] encoded.OrcEncodedDataReader: Encoded reader is being stopped 2018-01-05T11:29:08,429 DEBUG [TezTR-518813_1_38_1_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 3 of 4 2018-01-05T11:29:08,430 DEBUG [TezTR-518813_1_38_1_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat]} 2018-01-05T11:29:08,430 DEBUG [TezTR-518813_1_38_1_0_0] io.HiveInputFormat: Pushdown initiated with filterText = key is not null, filterExpr = GenericUDFOPNotNull(Column[key]), serializedFilterExpr = AQEAamF2YS51dGlsLkFycmF5TGlz9AEBAQFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVDb2x1bW5EZXPjAQFrZfkAAAGCYgECb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5zZXJkZTIudHlwZWluZm8uUHJpbWl0aXZlVHlwZUluZu8BAWlu9AEDb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGT1BOb3ROdWzsAQEEb3JnLmFwYWNoZS5oYWRvb3AuaW8uQm9vbGVhbldyaXRhYmzlAQAAAQIBAWJvb2xlYe4= (desc) 2018-01-05T11:29:08,430 INFO [TezTR-518813_1_38_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/7b385878-392d-4bf1-a4fe-47a3392c1f93/map.xml 2018-01-05T11:29:08,430 DEBUG [TezTR-518813_1_38_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:08,430 DEBUG [TezTR-518813_1_38_1_0_0] io.HiveInputFormat: Processing org.apache.hadoop.hive.ql.io.orc.OrcInputFormat 2018-01-05T11:29:08,431 DEBUG [TezTR-518813_1_38_1_0_0] impl.LlapRecordReader: Initializing for input Map 3 2018-01-05T11:29:08,431 INFO [TezTR-518813_1_38_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/7b385878-392d-4bf1-a4fe-47a3392c1f93/map.xml 2018-01-05T11:29:08,431 DEBUG [TezTR-518813_1_38_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:08,431 INFO [TezTR-518813_1_38_1_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_38_01_000000_0 2018-01-05T11:29:08,431 INFO [TezTR-518813_1_38_1_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:08,431 DEBUG [TezTR-518813_1_38_1_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,431 DEBUG [TezTR-518813_1_38_1_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,431 INFO [TezTR-518813_1_38_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/7b385878-392d-4bf1-a4fe-47a3392c1f93/map.xml 2018-01-05T11:29:08,431 DEBUG [TezTR-518813_1_38_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:08,431 DEBUG [TezTR-518813_1_38_1_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0 2018-01-05T11:29:08,431 DEBUG [IO-Elevator-Thread-1] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_38 queryId: djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01 instanceType: OrcEncodedDataReader 2018-01-05T11:29:08,432 INFO [IO-Elevator-Thread-1] LlapIoImpl: Processing data for file [6462268725459214672, 1515180546000, 752]: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0 2018-01-05T11:29:08,432 DEBUG [IO-Elevator-Thread-1] LlapIoOrc: FileSplit {3, 574}; stripes {3, 574}, 2018-01-05T11:29:08,432 DEBUG [IO-Elevator-Thread-1] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,432 DEBUG [IO-Elevator-Thread-1] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:08,432 INFO [IO-Elevator-Thread-1] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725459214672, 1515180546000, 752]): [{range start: 0 end: 73}] 2018-01-05T11:29:08,432 INFO [IO-Elevator-Thread-1] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [6462268725459214672, 1515180546000, 752], base offset 3): [{start: 0 end: 11 cache buffer: 0x1c0198da(1)}, {start: 11 end: 38 cache buffer: 0x55974329(1)}, {start: 38 end: 73 cache buffer: 0x4d494b43(1)}] 2018-01-05T11:29:08,432 DEBUG [IO-Elevator-Thread-1] impl.RecordReaderImpl: Row group 0 to 9999 is included. 2018-01-05T11:29:08,432 INFO [IO-Elevator-Thread-1] impl.OrcCodecPool: Got brand-new codec ZLIB 2018-01-05T11:29:08,432 INFO [IO-Elevator-Thread-1] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725459214672, 1515180546000, 752]): [{range start: 73 end: 514}] 2018-01-05T11:29:08,432 INFO [IO-Elevator-Thread-1] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [6462268725459214672, 1515180546000, 752], base offset 3): [{start: 73 end: 208 cache buffer: 0x21723a65(1)}, {start: 208 end: 315 cache buffer: 0x4e93ab69(1)}, {start: 315 end: 353 cache buffer: 0x4907f2ab(1)}, {start: 353 end: 514 cache buffer: 0x55a06b73(1)}] 2018-01-05T11:29:08,432 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: int streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 150 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:29:08,432 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 2 columnType: string streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DICTIONARY_V2 dictionarySize: 75 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 104 bytes, ], indexBaseOffset=0] dictionary: true lengths: true secondary: false tz: 2018-01-05T11:29:08,433 DEBUG [IO-Elevator-Thread-1] LlapIoOrc: Done with decode 2018-01-05T11:29:08,433 DEBUG [IO-Elevator-Thread-1] LlapIoImpl: setDone called; closed false, done false, err null, pending 0 2018-01-05T11:29:08,433 DEBUG [IO-Elevator-Thread-1] impl.StatsRecordingThreadPool: Updated stats: instance: OrcEncodedDataReader thread name: IO-Elevator-Thread-1 thread id: 1309 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:08,433 INFO [TezTR-518813_1_38_1_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:08,433 INFO [TezTR-518813_1_38_1_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, warehouse.tab_part, file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0 ([6462268725459214672, 1515180546000, 752]), 0,1]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=1, NUM_ERRORS=0, ROWS_EMITTED=119, METADATA_CACHE_HIT=2, METADATA_CACHE_MISS=0, CACHE_HIT_BYTES=514, CACHE_MISS_BYTES=0, ALLOCATED_BYTES=0, ALLOCATED_USED_BYTES=0, TOTAL_IO_TIME_NS=1104072, DECODE_TIME_NS=349303, HDFS_TIME_NS=43442, CONSUMER_TIME_NS=303140 ] 2018-01-05T11:29:08,433 DEBUG [TezTR-518813_1_38_1_0_0] encoded.OrcEncodedDataReader: Encoded reader is being stopped 2018-01-05T11:29:08,433 DEBUG [TezTR-518813_1_38_1_0_0] log.PerfLogger: 2018-01-05T11:29:08,433 DEBUG [TezTR-518813_1_38_1_0_0] mr.ObjectCache: djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01_Map 3__MAP_PLAN__ no longer needed 2018-01-05T11:29:08,433 DEBUG [TezTR-518813_1_38_1_0_0] vector.VectorMapOperator: close called for operator MAP[0] 2018-01-05T11:29:08,433 INFO [TezTR-518813_1_38_1_0_0] vector.VectorMapOperator: Closing operator MAP[0] 2018-01-05T11:29:08,433 INFO [TezTR-518813_1_38_1_0_0] vector.VectorMapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_3:500, 2018-01-05T11:29:08,433 DEBUG [TezTR-518813_1_38_1_0_0] vector.VectorMapOperator: Closing child = TS[3] 2018-01-05T11:29:08,433 DEBUG [TezTR-518813_1_38_1_0_0] exec.TableScanOperator: close called for operator TS[3] 2018-01-05T11:29:08,433 INFO [TezTR-518813_1_38_1_0_0] exec.TableScanOperator: Closing operator TS[3] 2018-01-05T11:29:08,433 DEBUG [TezTR-518813_1_38_1_0_0] exec.TableScanOperator: Closing child = FIL[29] 2018-01-05T11:29:08,433 DEBUG [TezTR-518813_1_38_1_0_0] vector.VectorFilterOperator: close called for operator FIL[29] 2018-01-05T11:29:08,433 DEBUG [TezTR-518813_1_38_1_0_0] vector.VectorFilterOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:08,433 INFO [TezTR-518813_1_38_1_0_0] vector.VectorFilterOperator: Closing operator FIL[29] 2018-01-05T11:29:08,433 DEBUG [TezTR-518813_1_38_1_0_0] vector.VectorFilterOperator: Closing child = SEL[30] 2018-01-05T11:29:08,433 DEBUG [TezTR-518813_1_38_1_0_0] vector.VectorSelectOperator: close called for operator SEL[30] 2018-01-05T11:29:08,433 DEBUG [TezTR-518813_1_38_1_0_0] vector.VectorSelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:08,433 INFO [TezTR-518813_1_38_1_0_0] vector.VectorSelectOperator: Closing operator SEL[30] 2018-01-05T11:29:08,433 DEBUG [TezTR-518813_1_38_1_0_0] vector.VectorSelectOperator: Closing child = RS[31] 2018-01-05T11:29:08,433 DEBUG [TezTR-518813_1_38_1_0_0] reducesink.VectorReduceSinkLongOperator: close called for operator RS[31] 2018-01-05T11:29:08,433 DEBUG [TezTR-518813_1_38_1_0_0] reducesink.VectorReduceSinkLongOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:08,433 INFO [TezTR-518813_1_38_1_0_0] reducesink.VectorReduceSinkLongOperator: Closing operator RS[31] 2018-01-05T11:29:08,433 INFO [TezTR-518813_1_38_1_0_0] reducesink.VectorReduceSinkCommonOperator: RS[31]: records written - 500 2018-01-05T11:29:08,433 INFO [TezTR-518813_1_38_1_0_0] reducesink.VectorReduceSinkLongOperator: RECORDS_OUT_INTERMEDIATE_Map_3:500, 2018-01-05T11:29:08,433 DEBUG [TezTR-518813_1_38_1_0_0] reducesink.VectorReduceSinkLongOperator: 31 Close done 2018-01-05T11:29:08,433 DEBUG [TezTR-518813_1_38_1_0_0] vector.VectorSelectOperator: 30 Close done 2018-01-05T11:29:08,433 DEBUG [TezTR-518813_1_38_1_0_0] vector.VectorFilterOperator: 29 Close done 2018-01-05T11:29:08,433 DEBUG [TezTR-518813_1_38_1_0_0] exec.TableScanOperator: 3 Close done 2018-01-05T11:29:08,433 DEBUG [TezTR-518813_1_38_1_0_0] vector.VectorMapOperator: 0 Close done 2018-01-05T11:29:08,433 INFO [TezTR-518813_1_38_1_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_38_01_000000_0 2018-01-05T11:29:08,433 INFO [TezTR-518813_1_38_1_0_0] impl.PipelinedSorter: Reducer 2: Starting flush of map output 2018-01-05T11:29:08,433 INFO [TezTR-518813_1_38_1_0_0] impl.PipelinedSorter: Reducer 2: Span0.length = 500, perItem = 25 2018-01-05T11:29:08,434 INFO [TezTR-518813_1_38_1_0_0] impl.PipelinedSorter: Reducer 2: done sorting span=0, length=500, time=1 2018-01-05T11:29:08,434 INFO [TezTR-518813_1_38_1_0_0] impl.PipelinedSorter: Reducer 2: Heap = SpanIterator<0:499> (span=Span[5242880,12906]), 2018-01-05T11:29:08,437 DEBUG [TezTR-518813_1_38_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:08,437 INFO [TezTR-518813_1_38_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:08,437 INFO [TezTR-518813_1_38_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_38_00_000000_0, askedToStop=false 2018-01-05T11:29:08,437 INFO [TezTR-518813_1_38_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_38_00_000000_0, stopRequested=false 2018-01-05T11:29:08,437 INFO [TezTR-518813_1_38_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_38_00_000000_0: Counters: 31 [[File System Counters FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=0, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=242, INPUT_RECORDS_PROCESSED=2, INPUT_SPLIT_LENGTH_BYTES=1149, OUTPUT_RECORDS=242, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=6244, OUTPUT_BYTES_WITH_OVERHEAD=6239, OUTPUT_BYTES_PHYSICAL=6243, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=242, RECORDS_OUT_INTERMEDIATE_Map_1=242][org.apache.hadoop.hive.llap.counters.LlapIOCounters CACHE_HIT_BYTES=1029, CACHE_MISS_BYTES=0, CONSUMER_TIME_NS=1416325, DECODE_TIME_NS=958608, HDFS_TIME_NS=164875, METADATA_CACHE_HIT=4, NUM_DECODED_BATCHES=2, NUM_VECTOR_BATCHES=2, ROWS_EMITTED=242, SELECTED_ROWGROUPS=2, TOTAL_IO_TIME_NS=3887376]] 2018-01-05T11:29:08,437 DEBUG [TezTR-518813_1_38_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_38_0_0_0 thread id: 2600 scheme: file bytesRead: 64 bytesWritten: 6275 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:08,437 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_38_00_000000_0, isTaskDone=true 2018-01-05T11:29:08,437 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000090, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_38_00_000000_0, eventCount=4 } 2018-01-05T11:29:08,438 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_38][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_38_00_000000_0, creationTime=1515180548357, allocationTime=1515180548357, startTime=1515180548361, finishTime=1515180548438, timeTaken=77, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:08,438 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_38_00_000000_0 2018-01-05T11:29:08,438 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_38_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:08,438 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_38_00_000000_0; false 2018-01-05T11:29:08,438 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:29:08,438 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_38][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_38_00_000000, startTime=1515180548361, finishTime=1515180548438, timeTaken=77, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_38_00_000000_0, diagnostics=, counters=Counters: 32, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=64, FILE_BYTES_WRITTEN=6275, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=242, INPUT_RECORDS_PROCESSED=2, INPUT_SPLIT_LENGTH_BYTES=1149, OUTPUT_RECORDS=242, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=6244, OUTPUT_BYTES_WITH_OVERHEAD=6239, OUTPUT_BYTES_PHYSICAL=6243, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=242, RECORDS_OUT_INTERMEDIATE_Map_1=242, org.apache.hadoop.hive.llap.counters.LlapIOCounters, CACHE_HIT_BYTES=1029, CACHE_MISS_BYTES=0, CONSUMER_TIME_NS=1416325, DECODE_TIME_NS=958608, HDFS_TIME_NS=164875, METADATA_CACHE_HIT=4, NUM_DECODED_BATCHES=2, NUM_VECTOR_BATCHES=2, ROWS_EMITTED=242, SELECTED_ROWGROUPS=2, TOTAL_IO_TIME_NS=3887376 2018-01-05T11:29:08,438 INFO [Task-Executor-1] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_38_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:08,438 INFO [Task-Executor-1] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_38_00_000000_0 2018-01-05T11:29:08,438 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_38_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:08,438 INFO [Task-Executor-1] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000090=60 2018-01-05T11:29:08,438 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_38_00_000000_0: true 2018-01-05T11:29:08,438 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_38_00 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:08,438 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_38_00 [Map 1] 2018-01-05T11:29:08,438 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:08,438 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:08,439 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_38_00_000000_0 2018-01-05T11:29:08,439 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000090, QueryId=djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01-38, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:08,439 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_38_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=3, preemptionQueueSize=1 2018-01-05T11:29:08,439 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_38_00_000000_0 2018-01-05T11:29:08,439 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_38][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_38_00, initRequestedTime=1515180548312, initedTime=1515180548356, startRequestedTime=1515180548329, startedTime=1515180548356, finishTime=1515180548438, timeTaken=82, status=SUCCEEDED, diagnostics=, counters=Counters: 32, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=64, FILE_BYTES_WRITTEN=6275, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=242, INPUT_RECORDS_PROCESSED=2, INPUT_SPLIT_LENGTH_BYTES=1149, OUTPUT_RECORDS=242, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=6244, OUTPUT_BYTES_WITH_OVERHEAD=6239, OUTPUT_BYTES_PHYSICAL=6243, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=242, RECORDS_OUT_INTERMEDIATE_Map_1=242, org.apache.hadoop.hive.llap.counters.LlapIOCounters, CACHE_HIT_BYTES=1029, CACHE_MISS_BYTES=0, CONSUMER_TIME_NS=1416325, DECODE_TIME_NS=958608, HDFS_TIME_NS=164875, METADATA_CACHE_HIT=4, NUM_DECODED_BATCHES=2, NUM_VECTOR_BATCHES=2, ROWS_EMITTED=242, SELECTED_ROWGROUPS=2, TOTAL_IO_TIME_NS=3887376, vertexStats=firstTaskStartTime=1515180548361, firstTasksToStart=[ task_1515180518813_0001_38_00_000000 ], lastTaskFinishTime=1515180548438, lastTasksToFinish=[ task_1515180518813_0001_38_00_000000 ], minTaskDuration=77, maxTaskDuration=77, avgTaskDuration=77.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_38_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_38_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:08,439 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_38_00 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:08,439 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_38_02 [Reducer 2] attempt: attempt_1515180518813_0001_38_00_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:29:08,439 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000090, QueryId=djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01-38, DagId=38, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-1, Succeeded=true, StartTime=1515180548378, EndTime=1515180548439 2018-01-05T11:29:08,439 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_38][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000090, stoppedTime=1515180548439, exitStatus=0 2018-01-05T11:29:08,439 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000090 exited with diagnostics set to Container completed. 2018-01-05T11:29:08,439 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_38_00 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3 2018-01-05T11:29:08,439 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null 2018-01-05T11:29:08,439 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000090 2018-01-05T11:29:08,447 INFO [TezTR-518813_1_38_1_0_0] impl.PipelinedSorter: Reducer 2: Spilling to /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/38/output/attempt_1515180518813_0001_38_01_000000_0_10277_0/file.out 2018-01-05T11:29:08,462 DEBUG [TezTR-518813_1_38_1_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:08,462 INFO [TezTR-518813_1_38_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:08,462 INFO [TezTR-518813_1_38_1_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_38_01_000000_0, askedToStop=false 2018-01-05T11:29:08,462 INFO [TezTR-518813_1_38_1_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_38_01_000000_0, stopRequested=false 2018-01-05T11:29:08,462 INFO [TezTR-518813_1_38_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_38_01_000000_0: Counters: 31 [[File System Counters FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=0, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=500, INPUT_RECORDS_PROCESSED=4, INPUT_SPLIT_LENGTH_BYTES=2442, OUTPUT_RECORDS=500, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=12906, OUTPUT_BYTES_WITH_OVERHEAD=12847, OUTPUT_BYTES_PHYSICAL=12851, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_3=500, RECORDS_OUT_INTERMEDIATE_Map_3=500][org.apache.hadoop.hive.llap.counters.LlapIOCounters CACHE_HIT_BYTES=2202, CACHE_MISS_BYTES=0, CONSUMER_TIME_NS=1687153, DECODE_TIME_NS=1281988, HDFS_TIME_NS=170225, METADATA_CACHE_HIT=8, NUM_DECODED_BATCHES=4, NUM_VECTOR_BATCHES=4, ROWS_EMITTED=500, SELECTED_ROWGROUPS=4, TOTAL_IO_TIME_NS=4951532]] 2018-01-05T11:29:08,462 DEBUG [TezTR-518813_1_38_1_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_38_1_0_0 thread id: 2609 scheme: file bytesRead: 64 bytesWritten: 12883 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:08,462 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_38_01_000000_0, isTaskDone=true 2018-01-05T11:29:08,462 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000091, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_38_01_000000_0, eventCount=4 } 2018-01-05T11:29:08,463 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_38][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 3, taskAttemptId=attempt_1515180518813_0001_38_01_000000_0, creationTime=1515180548405, allocationTime=1515180548406, startTime=1515180548407, finishTime=1515180548463, timeTaken=56, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:08,463 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_38_01_000000_0 2018-01-05T11:29:08,463 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_38_01_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:08,463 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_38_01_000000_0; false 2018-01-05T11:29:08,463 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_38_01_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:08,463 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:29:08,463 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:08,463 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_38][Event:TASK_FINISHED]: vertexName=Map 3, taskId=task_1515180518813_0001_38_01_000000, startTime=1515180548407, finishTime=1515180548463, timeTaken=56, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_38_01_000000_0, diagnostics=, counters=Counters: 32, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=64, FILE_BYTES_WRITTEN=12883, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=500, INPUT_RECORDS_PROCESSED=4, INPUT_SPLIT_LENGTH_BYTES=2442, OUTPUT_RECORDS=500, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=12906, OUTPUT_BYTES_WITH_OVERHEAD=12847, OUTPUT_BYTES_PHYSICAL=12851, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_3=500, RECORDS_OUT_INTERMEDIATE_Map_3=500, org.apache.hadoop.hive.llap.counters.LlapIOCounters, CACHE_HIT_BYTES=2202, CACHE_MISS_BYTES=0, CONSUMER_TIME_NS=1687153, DECODE_TIME_NS=1281988, HDFS_TIME_NS=170225, METADATA_CACHE_HIT=8, NUM_DECODED_BATCHES=4, NUM_VECTOR_BATCHES=4, ROWS_EMITTED=500, SELECTED_ROWGROUPS=4, TOTAL_IO_TIME_NS=4951532 2018-01-05T11:29:08,463 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:08,463 INFO [Task-Executor-2] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_38_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:08,464 INFO [Task-Executor-2] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_38_01_000000_0 2018-01-05T11:29:08,464 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_38_01 [Map 3], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:08,464 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_38_01 [Map 3] 2018-01-05T11:29:08,464 INFO [Task-Executor-2] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000091=55 2018-01-05T11:29:08,464 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_38_01_000000_0: true 2018-01-05T11:29:08,464 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_38_01_000000_0 2018-01-05T11:29:08,464 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000091, QueryId=djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01-38, Vertex=Map 3, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:08,464 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_38_01_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:29:08,464 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_38_01_000000_0 2018-01-05T11:29:08,464 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_38][Event:VERTEX_FINISHED]: vertexName=Map 3, vertexId=vertex_1515180518813_0001_38_01, initRequestedTime=1515180548321, initedTime=1515180548405, startRequestedTime=1515180548329, startedTime=1515180548405, finishTime=1515180548464, timeTaken=59, status=SUCCEEDED, diagnostics=, counters=Counters: 32, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=64, FILE_BYTES_WRITTEN=12883, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=500, INPUT_RECORDS_PROCESSED=4, INPUT_SPLIT_LENGTH_BYTES=2442, OUTPUT_RECORDS=500, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=12906, OUTPUT_BYTES_WITH_OVERHEAD=12847, OUTPUT_BYTES_PHYSICAL=12851, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_3=500, RECORDS_OUT_INTERMEDIATE_Map_3=500, org.apache.hadoop.hive.llap.counters.LlapIOCounters, CACHE_HIT_BYTES=2202, CACHE_MISS_BYTES=0, CONSUMER_TIME_NS=1687153, DECODE_TIME_NS=1281988, HDFS_TIME_NS=170225, METADATA_CACHE_HIT=8, NUM_DECODED_BATCHES=4, NUM_VECTOR_BATCHES=4, ROWS_EMITTED=500, SELECTED_ROWGROUPS=4, TOTAL_IO_TIME_NS=4951532, vertexStats=firstTaskStartTime=1515180548407, firstTasksToStart=[ task_1515180518813_0001_38_01_000000 ], lastTaskFinishTime=1515180548463, lastTasksToFinish=[ task_1515180518813_0001_38_01_000000 ], minTaskDuration=56, maxTaskDuration=56, avgTaskDuration=56.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_38_01_000000 ], longestDurationTasks=[ task_1515180518813_0001_38_01_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:08,464 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_38_01 [Map 3] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:08,464 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_38][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000091, stoppedTime=1515180548464, exitStatus=0 2018-01-05T11:29:08,464 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000091 exited with diagnostics set to Container completed. 2018-01-05T11:29:08,464 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_38_02 [Reducer 2] attempt: attempt_1515180518813_0001_38_01_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:29:08,464 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_38_01 [Map 3] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3 2018-01-05T11:29:08,464 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null 2018-01-05T11:29:08,464 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000091 2018-01-05T11:29:08,464 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: All source tasks completed. Ramping up 1 remaining tasks for vertex: Reducer 2 2018-01-05T11:29:08,464 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Scheduling 1 tasks for vertex: Reducer 2 with totalTasks: 1. 2 source tasks completed out of 2. MinSourceTaskCompletedFraction: 1.0 min: 0.05 max: 0.75 2018-01-05T11:29:08,464 INFO [App Shared Pool - #0] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Map 3, destinationVertexName=Reducer 2. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:29:08,464 INFO [App Shared Pool - #0] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Map 1, destinationVertexName=Reducer 2. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:29:08,464 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_38_02 [Reducer 2] 2018-01-05T11:29:08,464 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_38][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_38_02_000000, scheduledTime=1515180548464, launchTime=1515180548464 2018-01-05T11:29:08,464 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_38_02 [Reducer 2] 2018-01-05T11:29:08,465 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_38_02_000000_0, priority=23, capability=, hosts=null 2018-01-05T11:29:08,465 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=23,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:08,465 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=23,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:08,465 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:08,465 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_38_02_000000_0 on hosts=null 2018-01-05T11:29:08,465 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_38_02_000000_0 on hosts=null 2018-01-05T11:29:08,465 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_38_02_000000_0 2018-01-05T11:29:08,465 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:08,465 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #92, task=TaskInfo{task=attempt_1515180518813_0001_38_02_000000_0, priority=23, startTime=0, containerId=null, uniqueId=91, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000092 2018-01-05T11:29:08,465 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_38_02_000000_0; false 2018-01-05T11:29:08,465 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_38_02_000000_0, priority=23, startTime=311568974, containerId=container_111101111_0001_00_000092assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=91, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:08,465 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000092 for node: 10.22.8.182:49818 2018-01-05T11:29:08,465 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000092, attempt_1515180518813_0001_38_02_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:08,465 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000092 for node: 10.22.8.182:49818 2018-01-05T11:29:08,465 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000092 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:08,465 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000092, launchTime=1515180548465 2018-01-05T11:29:08,466 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=38}, sourceName=Map 3, state=S_SUCCEEDED 2018-01-05T11:29:08,466 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=38}, sourceName=Map 1, state=S_SUCCEEDED 2018-01-05T11:29:08,466 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_38_02_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000092] on NM: [10.22.8.182:49818] 2018-01-05T11:29:08,466 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:08,466 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:08,466 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_38][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_38_02_000000_0, startTime=1515180548465, containerId=container_111101111_0001_00_000092, nodeId=10.22.8.182:49818 2018-01-05T11:29:08,467 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_38_02_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000092, dagName=select * from tab a join tab_part b ...b.key (Stage-1), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=2, numOutputs=1, numGroupedInputs=0, Inputs={{Map 3,org.apache.tez.runtime.library.input.OrderedGroupedKVInput,1}{Map 1,org.apache.tez.runtime.library.input.OrderedGroupedKVInput,1}}, Outputs={{out_Reducer 2,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=3, completedTaskCount=2, dagStartTime=1515180548312, firstAttemptStartTime=1515180548465, currentAttemptStartTime=1515180548465} 2018-01-05T11:29:08,467 INFO [IPC Server handler 4 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000092, QueryId=djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01-38, DagId=38, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, SubmitTime=1515180548467 2018-01-05T11:29:08,467 DEBUG [IPC Server handler 4 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=38} with the ShuffleHandler 2018-01-05T11:29:08,467 DEBUG [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/38] 2018-01-05T11:29:08,467 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:29:08,467 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_38_02_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:08,467 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_38_02_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180548465, dagStartTime=1515180548312, withinDagPriority=23, vertexParallelism= 1, selfAndUpstreamParallelism= 3, selfAndUpstreamComplete= 2}] 2018-01-05T11:29:08,467 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_38_02_000000_0 : ACCEPTED 2018-01-05T11:29:08,467 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_38_02_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:29:08,467 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_38_02_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180548465, dagStartTime=1515180548312, withinDagPriority=23, vertexParallelism= 1, selfAndUpstreamParallelism= 3, selfAndUpstreamComplete= 2} 2018-01-05T11:29:08,467 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_38_02_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:08,467 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_38_02_000000_0 to pre-emption queue 2018-01-05T11:29:08,467 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_38_02_000000_0: true 2018-01-05T11:29:08,468 INFO [TaskCommunicator # 9] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_38_02_000000_0 2018-01-05T11:29:08,468 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: taskOwner hashCode:1463545238 2018-01-05T11:29:08,468 INFO [TaskCommunicator # 9] GuaranteedTasks: Received response for attempt_1515180518813_0001_38_02_000000_0, true 2018-01-05T11:29:08,468 INFO [Task-Executor-2] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : select * from tab a join tab_part b ...b.key (Stage-1), VertexName: Reducer 2, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_38_02_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 3, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:29:08,468 INFO [Task-Executor-2] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=2, numOutputs=1, JVM.maxFree=1021837312, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:08,468 INFO [Task-Executor-2] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_38_02_000000_0 vertexName: Reducer 2 2018-01-05T11:29:08,468 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000092, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_38_02_000000_0, eventCount=1 } 2018-01-05T11:29:08,468 DEBUG [TezTR-518813_1_38_2_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_38 queryId: djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01 instanceType: TaskRunner2Callable 2018-01-05T11:29:08,468 INFO [TezTR-518813_1_38_2_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_38_02_000000_0 2018-01-05T11:29:08,469 DEBUG [TezTR-518813_1_38_2_0_0] log.PerfLogger: 2018-01-05T11:29:08,469 INFO [I/O Setup 0 Initialize: {Map 3}] orderedgrouped.MergeManager: Initial Memory required for SHUFFLE_BUFFER=131404920 based on INPUT_BUFFER_FACTOR=0.7, for final merged output=0, using factor: 0.0 2018-01-05T11:29:08,469 INFO [I/O Setup 1 Initialize: {Map 1}] orderedgrouped.MergeManager: Initial Memory required for SHUFFLE_BUFFER=131404920 based on INPUT_BUFFER_FACTOR=0.7, for final merged output=0, using factor: 0.0 2018-01-05T11:29:08,469 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:08,470 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:08,471 INFO [IPC Server handler 22 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_38_02_000000_0 2 events [0,2) total 2 vertex_1515180518813_0001_38_02 [Reducer 2] 2018-01-05T11:29:08,471 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=2, nextPreRoutedEventId=0, eventCount=2 } 2018-01-05T11:29:08,471 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_38_02_000000_0, eventCount=2 fromEventId=0 nextFromEventId=2 2018-01-05T11:29:08,472 DEBUG [TezTR-518813_1_38_2_0_0] log.PerfLogger: 2018-01-05T11:29:08,472 INFO [TezTR-518813_1_38_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:08,472 INFO [TezTR-518813_1_38_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 3 initializers to finish 2018-01-05T11:29:08,472 INFO [TezTR-518813_1_38_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:08,472 INFO [TezTR-518813_1_38_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:08,472 INFO [I/O Setup 2 Initialize: {out_Reducer 2}] output.MROutput: out_Reducer 2: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:29:08,472 DEBUG [I/O Setup 2 Initialize: {out_Reducer 2}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:08,472 DEBUG [I/O Setup 2 Initialize: {out_Reducer 2}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:08,472 DEBUG [I/O Setup 2 Initialize: {out_Reducer 2}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:08,472 DEBUG [I/O Setup 2 Initialize: {out_Reducer 2}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:08,472 INFO [TezTR-518813_1_38_2_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:08,472 INFO [TezTR-518813_1_38_2_0_0] resources.MemoryDistributor: InitialRequests=[out_Reducer 2:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [Map 3:INPUT:131404920:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.OrderedGroupedKVInput] 2018-01-05T11:29:08,472 INFO [TezTR-518813_1_38_2_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:08,472 INFO [TezTR-518813_1_38_2_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.045, finalReserveFractionUsed=0.345 2018-01-05T11:29:08,472 INFO [TezTR-518813_1_38_2_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 3, numScaledRequests: 25, TotalRequested: 262809840, TotalRequestedScaled: 1.2614872319999999E8, TotalJVMHeap: 1021837312, TotalAvailable: 669303439, TotalRequested/TotalJVMHeap:0.26 2018-01-05T11:29:08,472 INFO [TezTR-518813_1_38_2_0_0] resources.MemoryDistributor: Allocations=[out_Reducer 2:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [Map 3:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:131404920:131404920], [Map 1:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:131404920:131404920] 2018-01-05T11:29:08,472 INFO [TezTR-518813_1_38_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:08,472 INFO [TezTR-518813_1_38_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: Map 3 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:08,473 INFO [TezTR-518813_1_38_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: Map 1 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:08,473 INFO [TezTR-518813_1_38_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 2 2018-01-05T11:29:08,473 INFO [TezTR-518813_1_38_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 IOs to start 2018-01-05T11:29:08,473 INFO [I/O Setup 0 Start: {Map 3}] orderedgrouped.Shuffle: Map_3: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true 2018-01-05T11:29:08,473 INFO [I/O Setup 1 Start: {Map 1}] orderedgrouped.Shuffle: Map_1: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true 2018-01-05T11:29:08,473 INFO [I/O Setup 0 Start: {Map 3}] orderedgrouped.MergeManager: Map 3: MergerManager: memoryLimit=131404920, maxSingleShuffleLimit=32851230, mergeThreshold=86727248, ioSortFactor=10, postMergeMem=0, memToMemMergeOutputsThreshold=10 2018-01-05T11:29:08,473 INFO [I/O Setup 1 Start: {Map 1}] orderedgrouped.MergeManager: Map 1: MergerManager: memoryLimit=131404920, maxSingleShuffleLimit=32851230, mergeThreshold=86727248, ioSortFactor=10, postMergeMem=0, memToMemMergeOutputsThreshold=10 2018-01-05T11:29:08,474 INFO [I/O Setup 0 Start: {Map 3}] orderedgrouped.ShuffleScheduler: ShuffleScheduler running for sourceVertex: Map 3 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true 2018-01-05T11:29:08,474 INFO [I/O Setup 1 Start: {Map 1}] orderedgrouped.ShuffleScheduler: ShuffleScheduler running for sourceVertex: Map 1 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true 2018-01-05T11:29:08,475 INFO [I/O Setup 0 Start: {Map 3}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: Map 3 2018-01-05T11:29:08,475 INFO [TezTR-518813_1_38_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:08,475 INFO [I/O Setup 1 Start: {Map 1}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: Map 1 2018-01-05T11:29:08,475 INFO [TezTR-518813_1_38_2_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:08,475 INFO [TezTR-518813_1_38_2_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_38_02_000000_0 2018-01-05T11:29:08,475 DEBUG [TezTR-518813_1_38_2_0_0] log.PerfLogger: 2018-01-05T11:29:08,475 DEBUG [TezTR-518813_1_38_2_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_38_02_000000_0_10278 2018-01-05T11:29:08,475 DEBUG [TezTR-518813_1_38_2_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:08,475 DEBUG [TezTR-518813_1_38_2_0_0] mr.ObjectCache: Creating djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:29:08,475 INFO [TezTR-518813_1_38_2_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/c7e13d34-9ad4-48e5-95c1-a4e020bd1432/reduce.xml 2018-01-05T11:29:08,475 INFO [TezTaskEventRouter{attempt_1515180518813_0001_38_02_000000_0}] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:29:08,476 DEBUG [TezTR-518813_1_38_2_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/c7e13d34-9ad4-48e5-95c1-a4e020bd1432/reduce.xml 2018-01-05T11:29:08,476 DEBUG [TezTR-518813_1_38_2_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/djaiswal/_tez_scratch_dir/c7e13d34-9ad4-48e5-95c1-a4e020bd1432/reduce.xml 2018-01-05T11:29:08,476 DEBUG [TezTR-518813_1_38_2_0_0] log.PerfLogger: 2018-01-05T11:29:08,476 INFO [TezTR-518813_1_38_2_0_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:29:08,476 INFO [TezTaskEventRouter{attempt_1515180518813_0001_38_02_000000_0}] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 3: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:29:08,476 DEBUG [TezTR-518813_1_38_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:08,477 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=38&reduce=0&map=attempt_1515180518813_0001_38_01_000000_0_10277 mapId: [attempt_1515180518813_0001_38_01_000000_0_10277] reduceId: [0] jobId: [job_1515180518813_0001] dagId: [38] keepAlive: false 2018-01-05T11:29:08,477 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=38&reduce=0&map=attempt_1515180518813_0001_38_00_000000_0_10274 mapId: [attempt_1515180518813_0001_38_00_000000_0_10274] reduceId: [0] jobId: [job_1515180518813_0001] dagId: [38] keepAlive: false 2018-01-05T11:29:08,477 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=38&reduce=0&map=attempt_1515180518813_0001_38_01_000000_0_10277; hash=...ewFS4dmwje4dQ 2018-01-05T11:29:08,477 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=38&reduce=0&map=attempt_1515180518813_0001_38_00_000000_0_10274; hash=...TMZ6jPE2WPSnQ 2018-01-05T11:29:08,477 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=38&reduce=0&map=attempt_1515180518813_0001_38_01_000000_0_10277;reply=sS/Ehb1IR1PR8 2018-01-05T11:29:08,477 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=38&reduce=0&map=attempt_1515180518813_0001_38_00_000000_0_10274;reply=WpRHfu/LMNCuQ 2018-01-05T11:29:08,477 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=38, user='djaiswal', attemptId='attempt_1515180518813_0001_38_00_000000_0_10274'} via loader 2018-01-05T11:29:08,477 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=38, user='djaiswal', attemptId='attempt_1515180518813_0001_38_01_000000_0_10277'} via loader 2018-01-05T11:29:08,477 DEBUG [TezTR-518813_1_38_2_0_0] log.PerfLogger: 2018-01-05T11:29:08,477 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=38, user='djaiswal', attemptId='attempt_1515180518813_0001_38_00_000000_0_10274'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:29:08,477 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=38, user='djaiswal', attemptId='attempt_1515180518813_0001_38_01_000000_0_10277'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:29:08,477 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_38_01_000000_0_10277 not found 2018-01-05T11:29:08,477 DEBUG [New I/O worker #3] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_38_00_000000_0_10274 not found 2018-01-05T11:29:08,477 INFO [TezTR-518813_1_38_2_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 2.00KB 2018-01-05T11:29:08,477 DEBUG [TezTR-518813_1_38_2_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:08,477 DEBUG [TezTR-518813_1_38_2_0_0] log.PerfLogger: 2018-01-05T11:29:08,477 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_38_01_000000_0_10277,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/38/output/attempt_1515180518813_0001_38_01_000000_0_10277/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/38/output/attempt_1515180518813_0001_38_01_000000_0_10277/file.out.index 2018-01-05T11:29:08,477 DEBUG [TezTR-518813_1_38_2_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:08,479 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Content Length in shuffle : 12906 2018-01-05T11:29:08,479 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_38_00_000000_0_10274,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/38/output/attempt_1515180518813_0001_38_00_000000_0_10274/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/38/output/attempt_1515180518813_0001_38_00_000000_0_10274/file.out.index 2018-01-05T11:29:08,479 DEBUG [TezTR-518813_1_38_2_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:08,479 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Content Length in shuffle : 6298 2018-01-05T11:29:08,479 INFO [Fetcher_O {Map_3} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=38&reduce=0&map=attempt_1515180518813_0001_38_01_000000_0_10277 sent hash and receievd reply 0 ms 2018-01-05T11:29:08,479 INFO [Fetcher_O {Map_1} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=38&reduce=0&map=attempt_1515180518813_0001_38_00_000000_0_10274 sent hash and receievd reply 0 ms 2018-01-05T11:29:08,479 DEBUG [TezTR-518813_1_38_2_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:08,479 INFO [Fetcher_O {Map_3} #0] orderedgrouped.MergeManager: CloseInMemoryFile. Current state: inMemoryMapOutputs.size=1, commitMemory=0, usedMemory=12847. Since last log: count=1, min=12847, max=12847, total=12847, avg=12847.0 2018-01-05T11:29:08,479 INFO [TezTR-518813_1_38_2_0_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:29:08,480 INFO [Fetcher_O {Map_3} #0] ShuffleScheduler.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_38_01_000000_0_10277} to MEMORY, csize=12851, dsize=12847, EndTime=1515180548480, TimeTaken=0, Rate=0.00 MB/s 2018-01-05T11:29:08,480 INFO [Fetcher_O {Map_3} #0] orderedgrouped.ShuffleScheduler: All inputs fetched for input vertex : Map 3 2018-01-05T11:29:08,480 INFO [Fetcher_O {Map_1} #0] orderedgrouped.MergeManager: CloseInMemoryFile. Current state: inMemoryMapOutputs.size=1, commitMemory=0, usedMemory=6239. Since last log: count=1, min=6239, max=6239, total=6239, avg=6239.0 2018-01-05T11:29:08,480 INFO [Fetcher_O {Map_3} #0] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.01 MB/s) 2018-01-05T11:29:08,480 INFO [Fetcher_O {Map_1} #0] ShuffleScheduler.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_38_00_000000_0_10274} to MEMORY, csize=6243, dsize=6239, EndTime=1515180548480, TimeTaken=1, Rate=5.95 MB/s 2018-01-05T11:29:08,480 INFO [Fetcher_O {Map_1} #0] orderedgrouped.ShuffleScheduler: All inputs fetched for input vertex : Map 1 2018-01-05T11:29:08,480 INFO [ShuffleAndMergeRunner {Map_3}] orderedgrouped.ShuffleScheduler: Shutting down FetchScheduler for input: Map_3, wasInterrupted=false 2018-01-05T11:29:08,480 INFO [Fetcher_O {Map_1} #0] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.01 MB/s) 2018-01-05T11:29:08,480 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down FetchScheduler for input: Map_1, wasInterrupted=false 2018-01-05T11:29:08,480 INFO [ShuffleAndMergeRunner {Map_3}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.01 MB/s) 2018-01-05T11:29:08,481 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.01 MB/s) 2018-01-05T11:29:08,481 INFO [ShuffleAndMergeRunner {Map_3}] orderedgrouped.ShuffleScheduler: Shutting down fetchers for input: Map_3, shutdown timetaken: 0 ms, hasFetcherExecutorStopped: true 2018-01-05T11:29:08,481 INFO [ShuffleAndMergeRunner {Map_3}] orderedgrouped.MergeManager: TotalInMemFetchStats: count=1, totalSize=12847, min=12847, max=12847, avg=1.0 2018-01-05T11:29:08,481 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down fetchers for input: Map_1, shutdown timetaken: 0 ms, hasFetcherExecutorStopped: true 2018-01-05T11:29:08,481 INFO [ShuffleAndMergeRunner {Map_3}] orderedgrouped.MergeManager: finalMerge with #inMemoryOutputs=1, size=12847 and #onDiskOutputs=0, size=0 2018-01-05T11:29:08,481 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: TotalInMemFetchStats: count=1, totalSize=6239, min=6239, max=6239, avg=1.0 2018-01-05T11:29:08,481 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: finalMerge with #inMemoryOutputs=1, size=6239 and #onDiskOutputs=0, size=0 2018-01-05T11:29:08,502 INFO [ShuffleAndMergeRunner {Map_3}] orderedgrouped.Shuffle: merge complete for input vertex : Map_3 2018-01-05T11:29:08,502 INFO [ShuffleAndMergeRunner {Map_3}] orderedgrouped.Shuffle: Map_3: Shuffle Runner thread complete 2018-01-05T11:29:08,502 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: merge complete for input vertex : Map_1 2018-01-05T11:29:08,502 INFO [TezTR-518813_1_38_2_0_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:29:08,502 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: Map_1: Shuffle Runner thread complete 2018-01-05T11:29:08,502 INFO [TezTR-518813_1_38_2_0_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:08,503 DEBUG [TezTR-518813_1_38_2_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:08,503 INFO [TezTR-518813_1_38_2_0_0] input.OrderedGroupedKVInput: Map 1: creating ValuesIterator with comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator, keyClass=org.apache.hadoop.hive.ql.io.HiveKey, valClass=org.apache.hadoop.io.BytesWritable 2018-01-05T11:29:08,503 DEBUG [TezTR-518813_1_38_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:08,503 DEBUG [TezTR-518813_1_38_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:08,503 DEBUG [TezTR-518813_1_38_2_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[string, string] 2018-01-05T11:29:08,503 DEBUG [TezTR-518813_1_38_2_0_0] log.PerfLogger: 2018-01-05T11:29:08,503 INFO [TezTR-518813_1_38_2_0_0] input.OrderedGroupedKVInput: Map 3: creating ValuesIterator with comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator, keyClass=org.apache.hadoop.hive.ql.io.HiveKey, valClass=org.apache.hadoop.io.BytesWritable 2018-01-05T11:29:08,503 DEBUG [TezTR-518813_1_38_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:08,503 DEBUG [TezTR-518813_1_38_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:08,503 DEBUG [TezTR-518813_1_38_2_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1] columnTypes=[string, string] 2018-01-05T11:29:08,503 DEBUG [TezTR-518813_1_38_2_0_0] log.PerfLogger: 2018-01-05T11:29:08,503 INFO [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Initializing operator MERGEJOIN[25] 2018-01-05T11:29:08,503 DEBUG [TezTR-518813_1_38_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:08,503 DEBUG [TezTR-518813_1_38_2_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[0_VALUE_0, 0_VALUE_1, 0_VALUE_2] columnTypes=[int, string, string] 2018-01-05T11:29:08,503 DEBUG [TezTR-518813_1_38_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:08,503 DEBUG [TezTR-518813_1_38_2_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[0_VALUE_0, 0_VALUE_1, 0_VALUE_2] columnTypes=[int, string, string] 2018-01-05T11:29:08,503 DEBUG [TezTR-518813_1_38_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:08,503 DEBUG [TezTR-518813_1_38_2_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[1_VALUE_0, 1_VALUE_1, 1_VALUE_2] columnTypes=[int, string, string] 2018-01-05T11:29:08,503 DEBUG [TezTR-518813_1_38_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:08,503 DEBUG [TezTR-518813_1_38_2_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[1_VALUE_0, 1_VALUE_1, 1_VALUE_2] columnTypes=[int, string, string] 2018-01-05T11:29:08,503 INFO [TezTR-518813_1_38_2_0_0] exec.CommonJoinOperator: JOIN struct<_col0:int,_col1:string,_col2:string,_col3:int,_col4:string,_col5:string> totalsz = 6 2018-01-05T11:29:08,503 DEBUG [TezTR-518813_1_38_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:08,504 DEBUG [TezTR-518813_1_38_2_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[1_VALUE_0, 1_VALUE_1, 1_VALUE_2] columnTypes=[int, string, string] 2018-01-05T11:29:08,504 DEBUG [TezTR-518813_1_38_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:08,504 DEBUG [TezTR-518813_1_38_2_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[0_VALUE_0, 0_VALUE_1, 0_VALUE_2] columnTypes=[int, string, string] 2018-01-05T11:29:08,504 DEBUG [TezTR-518813_1_38_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:08,504 DEBUG [TezTR-518813_1_38_2_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[0_VALUE_0, 0_VALUE_1, 0_VALUE_2] columnTypes=[int, string, string] 2018-01-05T11:29:08,504 DEBUG [TezTR-518813_1_38_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:08,504 DEBUG [TezTR-518813_1_38_2_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[1_VALUE_0, 1_VALUE_1, 1_VALUE_2] columnTypes=[int, string, string] 2018-01-05T11:29:08,504 DEBUG [TezTR-518813_1_38_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:08,504 DEBUG [TezTR-518813_1_38_2_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[1_VALUE_0, 1_VALUE_1, 1_VALUE_2] columnTypes=[int, string, string] 2018-01-05T11:29:08,504 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Initialization Done 25 MERGEJOIN 2018-01-05T11:29:08,504 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Operator 25 MERGEJOIN initialized 2018-01-05T11:29:08,504 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Initializing children of 25 MERGEJOIN 2018-01-05T11:29:08,504 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Initializing child 10 FS 2018-01-05T11:29:08,504 INFO [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Initializing operator FS[10] 2018-01-05T11:29:08,504 DEBUG [TezTR-518813_1_38_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:08,504 INFO [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@7e7cfe50]:[_col0, _col1, _col2, _col3, _col4, _col5]:[int, string, string, int, string, string]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@79cac049 2018-01-05T11:29:08,504 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:29:08,504 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Initialization Done 10 FS 2018-01-05T11:29:08,504 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Operator 10 FS initialized 2018-01-05T11:29:08,504 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Initialization Done 10 FS done is reset. 2018-01-05T11:29:08,504 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Initialization Done 25 MERGEJOIN done is reset. 2018-01-05T11:29:08,504 INFO [TezTR-518813_1_38_2_0_0] tez.ReduceRecordProcessor: Id =25 Id =10 <\Children> Id = 25 null<\Parent> <\FS> <\Children> <\Parent> <\MERGEJOIN> 2018-01-05T11:29:08,504 DEBUG [TezTR-518813_1_38_2_0_0] log.PerfLogger: 2018-01-05T11:29:08,504 INFO [TezTR-518813_1_38_2_0_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 2 2018-01-05T11:29:08,506 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,506 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,506 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,506 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,506 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,506 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,507 INFO [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:08,507 INFO [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-08_057_3221244085031576356-1/_tmp.-ext-10002/000000_0 2018-01-05T11:29:08,507 INFO [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-08_057_3221244085031576356-1/_task_tmp.-ext-10002/_tmp.000000_0 2018-01-05T11:29:08,507 INFO [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-08_057_3221244085031576356-1/_tmp.-ext-10002/000000_0 2018-01-05T11:29:08,540 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,540 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: Status: Running (Executing on YARN cluster with App id application_1515180518813_0001) 2018-01-05T11:29:08,542 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,542 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,542 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,542 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,542 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,542 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 Map 3: 1/1 Reducer 2: 0(+1)/1 2018-01-05T11:29:08,553 INFO [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: FS[10]: records written - 1 2018-01-05T11:29:08,553 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,553 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,553 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,553 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,553 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,553 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,553 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,553 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,553 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,553 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,553 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,553 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,554 INFO [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: FS[10]: records written - 10 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,554 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,555 INFO [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,555 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,556 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,557 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,558 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,559 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,560 INFO [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:08,560 INFO [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: FS[10]: records written - 100 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,560 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,561 INFO [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,561 INFO [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,561 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,562 INFO [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,562 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,563 INFO [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,563 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,564 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,565 INFO [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,565 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,566 INFO [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,566 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,567 INFO [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,567 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,568 INFO [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,568 INFO [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,568 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000091, QueryId=djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01-38, DagId=38, VertexName=Map 3, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-2, Succeeded=true, StartTime=1515180548408, EndTime=1515180548568 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,568 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,569 DEBUG [AMReporterQueueDrainer] impl.AMReporter: Removing am 10.22.8.182:49824 with last associated dag QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=23} from heartbeat with taskCount=0, amFailed=false, isDone=true 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,569 INFO [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,569 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,570 INFO [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,570 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,571 INFO [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,571 INFO [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,571 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,572 INFO [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,572 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,573 INFO [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,573 INFO [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,573 INFO [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:08,573 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,574 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000092, requestId=2, startIndex=2, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_38_02_000000_0, eventCount=1 } 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,574 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,575 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=2, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Ending group for children: 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: End group Done 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Starting group for children: 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Start group Done 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] log.PerfLogger: 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] mr.ObjectCache: djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: close called for operator MERGEJOIN[25] 2018-01-05T11:29:08,575 INFO [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Closing operator MERGEJOIN[25] 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Cleaning up the operator state 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: Closing child = FS[10] 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: close called for operator FS[10] 2018-01-05T11:29:08,575 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:08,575 INFO [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: Closing operator FS[10] 2018-01-05T11:29:08,575 INFO [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: FS[10]: records written - 480 2018-01-05T11:29:08,576 INFO [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: RECORDS_OUT_0:480, 2018-01-05T11:29:08,576 DEBUG [TezTR-518813_1_38_2_0_0] exec.FileSinkOperator: 10 Close done 2018-01-05T11:29:08,576 DEBUG [TezTR-518813_1_38_2_0_0] exec.CommonMergeJoinOperator: 25 Close done 2018-01-05T11:29:08,576 INFO [TezTR-518813_1_38_2_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_38_02_000000_0 2018-01-05T11:29:08,576 INFO [TezTR-518813_1_38_2_0_0] orderedgrouped.Shuffle: Shutting down Shuffle for source: Map_3 2018-01-05T11:29:08,576 INFO [TezTR-518813_1_38_2_0_0] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 3: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:29:08,576 INFO [TezTR-518813_1_38_2_0_0] orderedgrouped.Shuffle: Shutting down Shuffle for source: Map_1 2018-01-05T11:29:08,576 INFO [TezTR-518813_1_38_2_0_0] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:29:08,576 INFO [TezTR-518813_1_38_2_0_0] output.MROutput: out_Reducer 2 closed 2018-01-05T11:29:08,576 DEBUG [TezTR-518813_1_38_2_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:08,576 INFO [TezTR-518813_1_38_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:08,576 INFO [TezTR-518813_1_38_2_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_38_02_000000_0, askedToStop=false 2018-01-05T11:29:08,576 INFO [TezTR-518813_1_38_2_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_38_02_000000_0, stopRequested=false 2018-01-05T11:29:08,576 INFO [TezTR-518813_1_38_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_38_02_000000_0: Counters: 30 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=462, REDUCE_INPUT_RECORDS=742, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=742, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=19094, ADDITIONAL_SPILLS_BYTES_READ=19094, SHUFFLE_BYTES=19094, SHUFFLE_BYTES_DECOMPRESSED=19086, SHUFFLE_BYTES_TO_MEM=19094, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=16, MERGE_PHASE_TIME=58, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][HIVE CREATED_FILES=1, RECORDS_OUT_0=480][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0]] 2018-01-05T11:29:08,576 DEBUG [TezTR-518813_1_38_2_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_38_2_0_0 thread id: 2621 scheme: file bytesRead: 10902 bytesWritten: 27815 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:08,576 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_38_02_000000_0, isTaskDone=true 2018-01-05T11:29:08,577 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000092, requestId=3, startIndex=2, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_38_02_000000_0, eventCount=2 } 2018-01-05T11:29:08,577 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_38][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_38_02_000000_0, creationTime=1515180548464, allocationTime=1515180548465, startTime=1515180548465, finishTime=1515180548577, timeTaken=112, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:08,577 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=3, shouldDie=false, nextFromEventId=2, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:08,577 INFO [Task-Executor-2] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_38_02_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:08,577 INFO [Task-Executor-2] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_38_02_000000_0 2018-01-05T11:29:08,577 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_38_02_000000_0 2018-01-05T11:29:08,577 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_38_02_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:08,578 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_38_02_000000_0; false 2018-01-05T11:29:08,578 INFO [Task-Executor-2] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000092=110 2018-01-05T11:29:08,578 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_38_02_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:08,578 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_38_02_000000_0: true 2018-01-05T11:29:08,578 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:08,578 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_38_02_000000_0 2018-01-05T11:29:08,578 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_38][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_38_02_000000, startTime=1515180548465, finishTime=1515180548577, timeTaken=112, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_38_02_000000_0, diagnostics=, counters=Counters: 35, File System Counters, FILE_BYTES_READ=10902, FILE_BYTES_WRITTEN=27815, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=462, REDUCE_INPUT_RECORDS=742, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=742, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=19094, ADDITIONAL_SPILLS_BYTES_READ=19094, SHUFFLE_BYTES=19094, SHUFFLE_BYTES_DECOMPRESSED=19086, SHUFFLE_BYTES_TO_MEM=19094, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=16, MERGE_PHASE_TIME=58, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, HIVE, CREATED_FILES=1, RECORDS_OUT_0=480, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0 2018-01-05T11:29:08,578 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000092, QueryId=djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01-38, Vertex=Reducer 2, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:08,578 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_38_02_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:29:08,578 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_38_02_000000_0 2018-01-05T11:29:08,578 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:08,578 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_38_02 [Reducer 2], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:08,578 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_38_02 [Reducer 2] 2018-01-05T11:29:08,578 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_38][Event:VERTEX_FINISHED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_38_02, initRequestedTime=1515180548329, initedTime=1515180548335, startRequestedTime=1515180548405, startedTime=1515180548405, finishTime=1515180548578, timeTaken=173, status=SUCCEEDED, diagnostics=, counters=Counters: 35, File System Counters, FILE_BYTES_READ=10902, FILE_BYTES_WRITTEN=27815, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=462, REDUCE_INPUT_RECORDS=742, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=742, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=19094, ADDITIONAL_SPILLS_BYTES_READ=19094, SHUFFLE_BYTES=19094, SHUFFLE_BYTES_DECOMPRESSED=19086, SHUFFLE_BYTES_TO_MEM=19094, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=16, MERGE_PHASE_TIME=58, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, HIVE, CREATED_FILES=1, RECORDS_OUT_0=480, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, vertexStats=firstTaskStartTime=1515180548465, firstTasksToStart=[ task_1515180518813_0001_38_02_000000 ], lastTaskFinishTime=1515180548577, lastTasksToFinish=[ task_1515180518813_0001_38_02_000000 ], minTaskDuration=112, maxTaskDuration=112, avgTaskDuration=112.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_38_02_000000 ], longestDurationTasks=[ task_1515180518813_0001_38_02_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:08,578 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_38_02 [Reducer 2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:08,579 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_38][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000092, stoppedTime=1515180548579, exitStatus=0 2018-01-05T11:29:08,579 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000092 exited with diagnostics set to Container completed. 2018-01-05T11:29:08,579 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_38_02 [Reducer 2] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3 2018-01-05T11:29:08,579 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3, commitInProgress=0, terminationCause=null 2018-01-05T11:29:08,579 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_38_00 [Map 1] 2018-01-05T11:29:08,579 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_38_02 [Reducer 2] 2018-01-05T11:29:08,579 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_38_01 [Map 3] 2018-01-05T11:29:08,580 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_38, queueSize=0 2018-01-05T11:29:08,580 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_38][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_38, startTime=1515180548312, finishTime=1515180548579, timeTaken=267, status=SUCCEEDED, diagnostics=, counters=Counters: 64, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=3, TOTAL_LAUNCHED_TASKS=3, OTHER_LOCAL_TASKS=2, AM_CPU_MILLISECONDS=979, AM_GC_TIME_MILLIS=0, File System Counters, FILE_BYTES_READ=11030, FILE_BYTES_WRITTEN=46973, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=462, REDUCE_INPUT_RECORDS=742, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=1484, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, INPUT_RECORDS_PROCESSED=6, INPUT_SPLIT_LENGTH_BYTES=3591, OUTPUT_RECORDS=742, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=19150, OUTPUT_BYTES_WITH_OVERHEAD=19086, OUTPUT_BYTES_PHYSICAL=19094, ADDITIONAL_SPILLS_BYTES_WRITTEN=19094, ADDITIONAL_SPILLS_BYTES_READ=19094, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=2, SHUFFLE_BYTES=19094, SHUFFLE_BYTES_DECOMPRESSED=19086, SHUFFLE_BYTES_TO_MEM=19094, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=16, MERGE_PHASE_TIME=58, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, HIVE, CREATED_FILES=1, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=242, RECORDS_IN_Map_3=500, RECORDS_OUT_0=480, RECORDS_OUT_INTERMEDIATE_Map_1=242, RECORDS_OUT_INTERMEDIATE_Map_3=500, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, org.apache.hadoop.hive.llap.counters.LlapIOCounters, CACHE_HIT_BYTES=3231, CACHE_MISS_BYTES=0, CONSUMER_TIME_NS=3103478, DECODE_TIME_NS=2240596, HDFS_TIME_NS=335100, METADATA_CACHE_HIT=12, NUM_DECODED_BATCHES=6, NUM_VECTOR_BATCHES=6, ROWS_EMITTED=742, SELECTED_ROWGROUPS=6, TOTAL_IO_TIME_NS=8838908 2018-01-05T11:29:08,580 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_38 finished with state: SUCCEEDED 2018-01-05T11:29:08,580 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_38 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:29:08,580 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000092 2018-01-05T11:29:08,580 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_38, dagState=SUCCEEDED 2018-01-05T11:29:08,580 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_38_post 2018-01-05T11:29:08,580 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:29:08,580 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:29:08,580 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:29:08,580 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 38 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=3, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=3,NumRequestsWithoutLocation=3, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=3} 2018-01-05T11:29:08,580 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:29:08,580 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=select * from tab a join tab_part b ...b.key (Stage-1), with id=dag_1515180518813_0001_38 2018-01-05T11:29:08,581 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 38, to 10.22.8.182:49818 2018-01-05T11:29:08,581 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=3, Remaining=0 2018-01-05T11:29:08,581 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=select * from tab a join tab_part b ...b.key (Stage-1), with id=dag_1515180518813_0001_38 2018-01-05T11:29:08,581 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=38} 2018-01-05T11:29:08,581 INFO [IPC Server handler 3 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=38}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:29:08,581 INFO [IPC Server handler 3 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/38 after 300 seconds 2018-01-05T11:29:08,581 ERROR [IPC Server handler 3 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01, dag_1515180518813_0001_38 2018-01-05T11:29:08,581 INFO [IPC Server handler 3 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01 2018-01-05T11:29:08,581 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=38} = 0 2018-01-05T11:29:08,581 DEBUG [IPC Server handler 3 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=38} 2018-01-05T11:29:08,581 DEBUG [IPC Server handler 3 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:29:08,581 DEBUG [IPC Server handler 3 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=38} 2018-01-05T11:29:08,582 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,582 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 Map 3: 1/1 Reducer 2: 1/1 2018-01-05T11:29:08,582 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,584 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,584 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: TaskId for 000000_0 = 000000 2018-01-05T11:29:08,584 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,584 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,584 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,584 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,584 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,584 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: select * from tab a join tab_part b on a.key = b.key 2018-01-05T11:29:08,584 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:29:08,584 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@tab 2018-01-05T11:29:08,584 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@tab@ds=2008-04-08 2018-01-05T11:29:08,584 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@tab_part 2018-01-05T11:29:08,584 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@tab_part@ds=2008-04-08 2018-01-05T11:29:08,584 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/-mr-10001 2018-01-05T11:29:08,584 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,584 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,584 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:08,584 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {} 2018-01-05T11:29:08,584 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01); Time taken: 0.406 seconds 2018-01-05T11:29:08,584 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:08,584 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,584 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:08,584 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,584 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query select * from tab a join tab_part b on a.key = b.key 2018-01-05T11:29:08,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] mapred.FileInputFormat: Time taken to get FileStatuses: 13 2018-01-05T11:29:08,598 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:29:08,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 13 2018-01-05T11:29:08,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FetchOperator: Creating fetchTask with deserializer typeinfo: struct<_col0:int,_col1:string,_col2:string,_col3:int,_col4:string,_col5:string> 2018-01-05T11:29:08,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FetchOperator: deserializer properties: table properties: {columns=_col0,_col1,_col2,_col3,_col4,_col5, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=int:string:string:int:string:string, escape.delim=\} partition properties: {columns=_col0,_col1,_col2,_col3,_col4,_col5, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=int:string:string:int:string:string, escape.delim=\} 2018-01-05T11:29:08,603 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: close called for operator LIST_SINK[32] 2018-01-05T11:29:08,603 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: Closing operator LIST_SINK[32] 2018-01-05T11:29:08,603 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: 32 Close done 2018-01-05T11:29:08,603 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting result dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/-mr-10001 2018-01-05T11:29:08,604 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-08_057_3221244085031576356-1 2018-01-05T11:29:08,604 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_057_3221244085031576356-1 2018-01-05T11:29:08,605 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.527 seconds, Fetched: 480 row(s) 2018-01-05T11:29:08,605 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:08,605 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:08,606 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:08,606 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:08,606 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,606 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:08,606 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,606 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,606 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,606 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: explain vectorization detail select count(*) from tab a left outer join tab_part b on a.key = b.key 2018-01-05T11:29:08,606 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112908_18aa9945-d13c-4022-863e-c6e8efbf6261): explain vectorization detail select count(*) from tab a left outer join tab_part b on a.key = b.key 2018-01-05T11:29:08,606 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,606 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: explain vectorization detail select count(*) from tab a left outer join tab_part b on a.key = b.key 2018-01-05T11:29:08,606 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:08,606 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,606 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,607 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,607 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,607 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:08,607 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:29:08,607 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:08,607 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed phase 1 of Semantic Analysis 2018-01-05T11:29:08,607 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:08,607 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,607 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab 2018-01-05T11:29:08,607 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab 2018-01-05T11:29:08,610 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,610 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,610 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,610 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab_part 2018-01-05T11:29:08,610 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab_part 2018-01-05T11:29:08,613 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,613 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,613 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:08,613 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:08,613 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed getting MetaData in Semantic Analysis 2018-01-05T11:29:08,614 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,615 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_not_null_constraints : db=default tbl=tab 2018-01-05T11:29:08,615 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_not_null_constraints : db=default tbl=tab 2018-01-05T11:29:08,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,615 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab 2018-01-05T11:29:08,615 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab 2018-01-05T11:29:08,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,616 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab 2018-01-05T11:29:08,616 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab 2018-01-05T11:29:08,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,616 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_unique_constraints : db=default tbl=tab 2018-01-05T11:29:08,616 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_unique_constraints : db=default tbl=tab 2018-01-05T11:29:08,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,616 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,617 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab 2018-01-05T11:29:08,617 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab 2018-01-05T11:29:08,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,618 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_not_null_constraints : db=default tbl=tab_part 2018-01-05T11:29:08,618 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_not_null_constraints : db=default tbl=tab_part 2018-01-05T11:29:08,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,618 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,618 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:08,618 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:08,619 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,619 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,619 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,619 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:08,619 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:08,619 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,619 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,619 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,619 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_unique_constraints : db=default tbl=tab_part 2018-01-05T11:29:08,619 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_unique_constraints : db=default tbl=tab_part 2018-01-05T11:29:08,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,620 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab_part 2018-01-05T11:29:08,620 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab_part 2018-01-05T11:29:08,621 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,621 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,625 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:08,625 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:08,641 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:08,641 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:29:08,641 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,645 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.sql2rel: Plan after trimming unused fields HiveAggregate(group=[{}], agg#0=[count()]) HiveProject(DUMMY=[0]) HiveJoin(condition=[=($0, $1)], joinType=[left], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:08,645 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan before removing subquery: HiveProject(_o__c0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveProject($f0=[$0]) HiveJoin(condition=[=($0, $6)], joinType=[left], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:08,646 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan just after removing subquery: HiveProject(_o__c0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveProject($f0=[$0]) HiveJoin(condition=[=($0, $6)], joinType=[left], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:08,646 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan after decorrelation: HiveProject(_o__c0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveProject($f0=[$0]) HiveJoin(condition=[=($0, $6)], joinType=[left], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:08,646 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,647 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,647 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,648 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,648 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,649 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,649 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,649 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,649 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,650 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,650 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,652 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,652 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,654 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,654 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,655 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.sql2rel: Plan after trimming unused fields HiveAggregate(group=[{}], agg#0=[count()]) HiveProject(DUMMY=[0]) HiveJoin(condition=[=($0, $1)], joinType=[left], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:08,655 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,655 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,656 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,656 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,656 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,656 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,657 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,657 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,657 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,657 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,658 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,658 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,658 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,658 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,659 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,668 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Original plan for PlanModifier HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[left], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:08,669 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after top-level introduceDerivedTable HiveProject($f0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[left], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:08,669 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after nested convertOpTree HiveProject($f0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[left], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:08,669 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after propagating order HiveProject($f0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[left], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:08,669 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after fixTopOBSchema HiveProject($f0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[left], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:08,669 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Final plan after modifier HiveProject(_c0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[left], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:08,670 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:08,670 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:08,670 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:08,670 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:08,670 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,670 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab 2018-01-05T11:29:08,670 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab 2018-01-05T11:29:08,674 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,674 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,674 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:08,674 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:08,675 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:08,675 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,675 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab_part 2018-01-05T11:29:08,675 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab_part 2018-01-05T11:29:08,678 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,678 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,679 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:08,679 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:08,679 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:08,679 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Table Plan for a TS[0] 2018-01-05T11:29:08,679 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB a{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB a{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:08,679 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (. (tok_table_or_col a) key) key)) 2018-01-05T11:29:08,679 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = a{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} starRr = null 2018-01-05T11:29:08,679 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(key,_col0: int)} 2018-01-05T11:29:08,679 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:08,679 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block $hdt$_0 2018-01-05T11:29:08,679 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block $hdt$_0 2018-01-05T11:29:08,679 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Table Plan for b TS[2] 2018-01-05T11:29:08,679 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB b{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB b{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:08,679 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (. (tok_table_or_col b) key) key)) 2018-01-05T11:29:08,679 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = b{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} starRr = null 2018-01-05T11:29:08,679 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(key,_col0: int)} 2018-01-05T11:29:08,679 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:08,679 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block $hdt$_1 2018-01-05T11:29:08,679 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block $hdt$_1 2018-01-05T11:29:08,680 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:08,680 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB $hdt$_0{(key,_col0: int)} $hdt$_1{(key,_col1: int)} after GB {((tok_functionstar count),_col0: bigint)} 2018-01-05T11:29:08,680 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (tok_functionstar count) _c0)) 2018-01-05T11:29:08,680 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = {((tok_functionstar count),_col0: bigint)} starRr = $hdt$_0{(key,_col0: int)} $hdt$_1{(key,_col1: int)} 2018-01-05T11:29:08,680 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(_c0,_col0: bigint)} 2018-01-05T11:29:08,680 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:08,680 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_606_6145014980571214667-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-08_606_6145014980571214667-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_606_6145014980571214667-1/-mr-10001 2018-01-05T11:29:08,680 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_606_6145014980571214667-1/-mr-10003 2018-01-05T11:29:08,680 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_606_6145014980571214667-1/-mr-10001 row schema: null{(_c0,_col0: bigint)} 2018-01-05T11:29:08,680 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block null 2018-01-05T11:29:08,680 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:29:08,680 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: CBO Succeeded; optimized logical plan. 2018-01-05T11:29:08,680 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Before logical optimization TS[0]-SEL[1]-RS[4]-JOIN[6]-SEL[7]-GBY[8]-RS[9]-GBY[10]-SEL[11]-FS[12] TS[2]-SEL[3]-RS[5]-JOIN[6] 2018-01-05T11:29:08,681 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,681 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,681 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,681 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] lineage.Generator: Time taken for lineage transform=0 2018-01-05T11:29:08,681 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,681 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,682 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,682 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,682 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SyntheticJoinPredicate: Synthetic predicate: 1 --> 0 2018-01-05T11:29:08,682 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,682 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,682 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000092, QueryId=djaiswal_20180105112908_bd7a1804-7bd9-4502-9b01-377f9950fb01-38, DagId=38, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-2, Succeeded=true, StartTime=1515180548467, EndTime=1515180548682 2018-01-05T11:29:08,682 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FS(12) 2018-01-05T11:29:08,682 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(11) 2018-01-05T11:29:08,682 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for GBY(10) 2018-01-05T11:29:08,682 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(9) 2018-01-05T11:29:08,682 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for GBY(8) 2018-01-05T11:29:08,682 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(7) 2018-01-05T11:29:08,682 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for JOIN(6) 2018-01-05T11:29:08,682 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(4) 2018-01-05T11:29:08,682 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(1) 2018-01-05T11:29:08,682 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:29:08,682 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(5) 2018-01-05T11:29:08,682 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(13) 2018-01-05T11:29:08,682 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias $hdt$_1: (_col0) IN (RS[4]) 2018-01-05T11:29:08,682 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(3) 2018-01-05T11:29:08,682 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of SEL for alias $hdt$_1: (key) IN (RS[4]) 2018-01-05T11:29:08,682 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for TS(2) 2018-01-05T11:29:08,683 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of TS for alias b: (key) IN (RS[4]) 2018-01-05T11:29:08,683 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SimplePredicatePushDown: After PPD: TS[0]-SEL[1]-RS[4]-JOIN[6]-SEL[7]-GBY[8]-RS[9]-GBY[10]-SEL[11]-FS[12] TS[2]-FIL[14]-SEL[3]-RS[5]-JOIN[6] 2018-01-05T11:29:08,683 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,683 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,683 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,683 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,683 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,683 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,683 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,683 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,683 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,683 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,683 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partitions : db=default tbl=tab_part 2018-01-05T11:29:08,683 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions : db=default tbl=tab_part 2018-01-05T11:29:08,686 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,686 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.178225ms + 0.005013ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? ] 2018-01-05T11:29:08,687 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.143772ms + 0.052916ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (14) order by "PART_NAME" asc] 2018-01-05T11:29:08,687 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.087369ms + 0.063088ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (14) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:29:08,687 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.073668ms + 0.020717ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (14) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:08,687 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.065829ms + 0.007ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (41) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:29:08,687 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.063159ms + 0.021875ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (41) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:08,687 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.061573ms + 0.030909ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (41) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:08,687 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.0625ms + 0.007078ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (41) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:08,687 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.089397ms + 0.022869ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (26) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:08,688 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.069333ms + 0.022033ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (41) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:29:08,688 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,690 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,691 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,691 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,691 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pcr.PcrExprProcFactory: Processing (key) IN (RS[4]) org.apache.hadoop.hive.ql.udf.generic.GenericUDFIn outputs {UNKNOWN, Column[key]}, {UNKNOWN, RS[4]}, 2018-01-05T11:29:08,691 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,691 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,691 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,691 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,691 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 9 key:[] 2018-01-05T11:29:08,691 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 9 oldColExprMap: {VALUE._col0=Column[_col0]} 2018-01-05T11:29:08,691 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 9 newColExprMap: {VALUE._col0=Column[_col0]} 2018-01-05T11:29:08,691 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: JOIN 6 oldExprs: {0=[Column[KEY.reducesinkkey0]], 1=[Column[KEY.reducesinkkey0]]} 2018-01-05T11:29:08,691 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: JOIN 6 newExprs: {0=[], 1=[]} 2018-01-05T11:29:08,691 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 4 key:[Column[_col0]] 2018-01-05T11:29:08,691 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 4 oldColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:08,691 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 4 newColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:08,691 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 5 key:[Column[_col0]] 2018-01-05T11:29:08,691 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 5 oldColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:08,691 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 5 newColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:08,691 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,691 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,691 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,691 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,691 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,691 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,691 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,691 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,691 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,691 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,691 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,691 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,691 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,692 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,692 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,692 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,692 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.IdentityProjectRemover: Identity project remover optimization removed : SEL[11] 2018-01-05T11:29:08,692 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.IdentityProjectRemover: Identity project remover optimization removed : SEL[7] 2018-01-05T11:29:08,692 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,692 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,692 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,692 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,692 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,692 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,692 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,692 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: After logical optimization TS[0]-SEL[1]-RS[4]-JOIN[6]-GBY[8]-RS[9]-GBY[10]-FS[12] TS[2]-FIL[14]-SEL[3]-RS[5]-JOIN[6] 2018-01-05T11:29:08,692 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,692 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Parent: TS[2] 2018-01-05T11:29:08,692 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Filter: (key) IN (RS[4]) (type: boolean) 2018-01-05T11:29:08,692 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: TableScan: TS[2] 2018-01-05T11:29:08,692 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Column key is not a partition column 2018-01-05T11:29:08,692 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Initiate semijoin reduction for key ((key) IN (RS[4]) 2018-01-05T11:29:08,692 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:08,692 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:08,692 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:08,692 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:08,692 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:08,692 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:08,692 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: DynamicSemiJoinPushdown: Saving RS to TS mapping: RS[19]: TS[2] 2018-01-05T11:29:08,692 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,692 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,692 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,693 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,693 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partitions : db=default tbl=tab 2018-01-05T11:29:08,693 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions : db=default tbl=tab 2018-01-05T11:29:08,696 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,699 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.202632ms + 0.03328ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? ] 2018-01-05T11:29:08,699 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.13436ms + 0.06121ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (15) order by "PART_NAME" asc] 2018-01-05T11:29:08,699 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.093545ms + 0.056385ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (15) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:29:08,699 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.069926ms + 0.01782ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (15) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:08,699 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.061857ms + 0.005995ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (43) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:29:08,699 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.062291ms + 0.02027ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (43) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:08,699 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.069338ms + 0.015873ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (43) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:08,700 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.055486ms + 0.005798ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (43) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:08,700 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.081946ms + 0.020701ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (28) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:08,700 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.0669ms + 0.042457ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (43) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:29:08,700 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,702 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,703 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,703 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,703 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Estimated average row size: 4 2018-01-05T11:29:08,703 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,703 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_aggr_stats_for: db=default table=tab 2018-01-05T11:29:08,703 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_aggr_stats_for: db=default table=tab 2018-01-05T11:29:08,704 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.AggregateStatsCache: Returning aggregate stats from the cache; total hits: 7, total misses: 0, hit ratio: 1.0 2018-01-05T11:29:08,704 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: useDensityFunctionForNDVEstimation = false partsFound = 1 ColumnStatisticsObj = [ColumnStatisticsObj(colName:key, colType:int, statsData:)] 2018-01-05T11:29:08,705 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,705 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,706 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting stats (Num rows: 242 Data size: 968 Basic stats: COMPLETE Column stats: COMPLETE) on TS[0] 2018-01-05T11:29:08,706 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (tab): numRows: 242 dataSize: 968 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {key= colName: key colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,706 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Estimated average row size: 4 2018-01-05T11:29:08,706 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,706 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_aggr_stats_for: db=default table=tab_part 2018-01-05T11:29:08,706 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_aggr_stats_for: db=default table=tab_part 2018-01-05T11:29:08,707 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.AggregateStatsCache: Returning aggregate stats from the cache; total hits: 8, total misses: 0, hit ratio: 1.0 2018-01-05T11:29:08,707 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: useDensityFunctionForNDVEstimation = false partsFound = 1 ColumnStatisticsObj = [ColumnStatisticsObj(colName:key, colType:int, statsData:)] 2018-01-05T11:29:08,708 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,708 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on TS[2] 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-TS[2] (tab_part): numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {key= colName: key colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 242 Data size: 968 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[1] 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[1]: numRows: 242 dataSize: 968 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_4_a_key_min), DynamicValue(RS_4_a_key_max)) Original num rows: 500 New num rows: 500 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_4_a_key_min), DynamicValue(RS_4_a_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_4_a_key_bloom_filter))) Original num rows: 500 New num rows: 500 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-FIL[14]: numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {key= colName: key colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on FIL[14] 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 242 Data size: 968 Basic stats: COMPLETE Column stats: COMPLETE) on RS[4] 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[4]: numRows: 242 dataSize: 968 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {KEY.reducesinkkey0= colName: KEY.reducesinkkey0 colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 242 Data size: 968 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[15] 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[15]: numRows: 242 dataSize: 968 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[3] 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[3]: numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[16]: inputSize: 968 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[16] hashAgg: true 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 3] STATS-GBY[16]: cardinality: 1 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[16] 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[16]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on RS[5] 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[5]: numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {KEY.reducesinkkey0= colName: KEY.reducesinkkey0 colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on RS[17] 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[17]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-JOIN[6]: detects none/multiple PK parents. 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.JoinOperator: Setting stats (Num rows: 392 Data size: 3136 Basic stats: COMPLETE Column stats: COMPLETE) on JOIN[6] 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-JOIN[6]: numRows: 392 dataSize: 3136 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[18]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[18] hashAgg: false 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 9] STATS-GBY[18]: cardinality: 1 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[18] 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[18]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[8]: inputSize: 3136 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[8] hashAgg: true 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 3] STATS-GBY[8]: cardinality: 1 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[8] 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[8]: numRows: 1 dataSize: 8 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on RS[19] 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[19]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE) on RS[9] 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[9]: numRows: 1 dataSize: 8 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {VALUE._col0= colName: VALUE._col0 colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[10]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[10] hashAgg: false 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 9] STATS-GBY[10]: cardinality: 1 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[10] 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[10]: numRows: 1 dataSize: 8 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-FS[12]: numRows: 1 dataSize: 8 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[key]]; bucket count: 2 }) on TS[0] 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[key]]; bucket count: 4 }) on TS[2] 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: 2 }) on SEL[1] 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[key]]; bucket count: 4 }) on FIL[14] 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[KEY.reducesinkkey0]]; sort column names: [[KEY.reducesinkkey0]]; bucket count: 2 }) on RS[4] 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: 2 }) on SEL[15] 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: 4 }) on SEL[3] 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[16] 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[KEY.reducesinkkey0]]; sort column names: [[KEY.reducesinkkey0]]; bucket count: 4 }) on RS[5] 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[17] 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.JoinOperator: Setting traits ({ bucket column names: [[], []]; sort column names: [[], []]; bucket count: -1 }) on JOIN[6] 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[18] 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[8] 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[19] 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[9] 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[10] 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FileSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on FS[12] 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,709 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,710 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Set parallelism for reduce sink RS[4] to: 1 2018-01-05T11:29:08,710 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Set parallelism for reduce sink RS[5] to: 1 2018-01-05T11:29:08,710 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConvertJoinMapJoin: Memory monitor info set to : { isLlap: true executorsPerNode: 4 maxExecutorsOverSubscribeMemory: 3 memoryOverSubscriptionFactor: 0.20000000298023224 memoryCheckInterval: 100000 noConditionalTaskSize: 10000000 adjustedNoConditionalTaskSize: 16000000 hashTableInflationFactor: 2.0 threshold: 32000000 } 2018-01-05T11:29:08,710 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConvertJoinMapJoin: Fallback to common merge join operator 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.CommonMergeJoinOperator: Setting traits ({ bucket column names: [[], []]; sort column names: [[], []]; bucket count: 0 }) on MERGEJOIN[20] 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.CommonMergeJoinOperator: Setting stats (Num rows: 392 Data size: 3136 Basic stats: COMPLETE Column stats: COMPLETE) on MERGEJOIN[20] 2018-01-05T11:29:08,710 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:08,710 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:08,710 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing BloomFilter cost/benefit for RS[19] - TS[2] (b) (Column[key]) 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing key domain cardinality, keyDomainCardinality=461, semiJoinKeyIsPK=false, selColStat= colName: _col0 colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false, selColSourceStat= colName: key colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false, tsColStat= colName: key colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit for Column[_col0] to Column[key], selKeyCardinality=153, tsKeyCardinality=308, tsRows=500, keyDomainCardinality=461 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: SemiJoin key selectivity=0.3318872017353579, benefit=334.056399132321 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit=334.056399132321, cost=242.0, tsDataSize=500, netBenefit=92.05639913232102 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: netBenefit=0.18411279826464202 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Reduction factor not satisfied for RS[19]-TS[2] (b). Removing semijoin optimization. 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Removing ReduceSink RS[19] and TableScan TS[2] 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: FS, 12 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: GBY, 10 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 9 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: GBY, 8 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: MERGEJOIN, 20 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 4 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 1 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 5 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 3 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: FIL, 14 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: TS, 2 2018-01-05T11:29:08,710 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Cycle free: true 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: Before SharedWorkOptimizer: TS[0]-SEL[1]-RS[4]-MERGEJOIN[20]-GBY[8]-RS[9]-GBY[10]-FS[12] TS[2]-FIL[14]-SEL[3]-RS[5]-MERGEJOIN[20] 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: DPP information stored in the cache: {} 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: Sorted tables by size: [default.tab_part=2000, default.tab=968] 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: After SharedWorkOptimizer: TS[0]-SEL[1]-RS[4]-MERGEJOIN[20]-GBY[8]-RS[9]-GBY[10]-FS[12] TS[2]-FIL[14]-SEL[3]-RS[5]-MERGEJOIN[20] 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: int|{a}key,value: string|{a}value,ds: string|{a}ds,BLOCK__OFFSET__INSIDE__FILE: bigint|{a}block__offset__inside__file,INPUT__FILE__NAME: string|{a}input__file__name,ROW__ID: struct|{a}row__id) 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[2] with rs:(key: int|{b}key,value: string|{b}value,ds: string|{b}ds,BLOCK__OFFSET__INSIDE__FILE: bigint|{b}block__offset__inside__file,INPUT__FILE__NAME: string|{b}input__file__name,ROW__ID: struct|{b}row__id) 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(_col0: int|{$hdt$_0}key) 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key]) 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FIL[14] with rs:(key: int|{b}key) 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_2 {} 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FIL[14] 2018-01-05T11:29:08,710 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Old filter FIL[14] conditions:(true and true) 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(Const boolean true, Const boolean true) -> Const boolean true 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Filter expression GenericUDFOPAnd(Const boolean true, Const boolean true) holds true. Will delete it. 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New filter FIL[14] conditions:true 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[4] with rs:(KEY.reducesinkkey0: int|{$hdt$_0}key) 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[4] 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[3] with rs:(_col0: int|{$hdt$_1}key) 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op FIL_14 {} 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[3] 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key]) 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[4] with rs:(KEY.reducesinkkey0: int|{$hdt$_0}key) 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[4] 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[5] with rs:(KEY.reducesinkkey0: int|{$hdt$_1}key) 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_3 {} 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[5] 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:MERGEJOIN[20] with rs:() 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_4 {} 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_5 {} 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator MERGEJOIN[20] 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[8] with rs:(_col0: bigint|{}(tok_functionstar count)) 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op MERGEJOIN_20 {} 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[8] 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[9] with rs:(VALUE._col0: bigint|{}(tok_functionstar count)) 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_8 {} 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[9] 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[10] with rs:(_col0: bigint|{}(tok_functionstar count)) 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_9 {} 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[10] 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[12] with rs:(_col0: bigint|{}) 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_10 {} 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[12] 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: TS[0] 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[4] 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 of table a 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[4] 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: MERGEJOIN[20] 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[9] 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding reduce work (Reducer 2) for MERGEJOIN[20] 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Setting up reduce sink: RS[4] with following reduce work: Reducer 2 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[4] as parent from MERGEJOIN[20] 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[5] as parent from MERGEJOIN[20] 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[9] 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: GBY[10] 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: FS[12] 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding reduce work (Reducer 3) for GBY[10] 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Setting up reduce sink: RS[9] with following reduce work: Reducer 3 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[9] as parent from GBY[10] 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: FS[12] 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: TS[2] 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[5] 2018-01-05T11:29:08,711 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding map work (Map 4) for TS[2] 2018-01-05T11:29:08,712 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 of table b 2018-01-05T11:29:08,712 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:08,712 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Second pass. Leaf operator: RS[5] has common downstream work: org.apache.hadoop.hive.ql.plan.MergeJoinWork@4926f297 2018-01-05T11:29:08,712 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:29:08,712 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,712 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,712 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:29:08,712 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:08,712 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:08,712 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 4 2018-01-05T11:29:08,712 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:08,712 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:08,712 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:29:08,712 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:08,712 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:08,712 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 4 2018-01-05T11:29:08,712 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:08,712 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:08,712 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:29:08,712 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Examining input format to see if vectorization is enabled. 2018-01-05T11:29:08,712 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorization is enabled for input format(s) [org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] 2018-01-05T11:29:08,712 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing MapWork... 2018-01-05T11:29:08,712 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[key], Vectorized Expression = col 0:int 2018-01-05T11:29:08,712 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator 2018-01-05T11:29:08,712 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.SelectDesc 2018-01-05T11:29:08,712 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:int 2018-01-05T11:29:08,712 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator groupby typeName int 2018-01-05T11:29:08,712 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkLongOperator 2018-01-05T11:29:08,712 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:08,712 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkLongOperator 2018-01-05T11:29:08,712 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [key, value, ds, ROW__ID] 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [int, string, string, struct] 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE, NONE, NONE, NONE] 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 1 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 1 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:08,713 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Examining input format to see if vectorization is enabled. 2018-01-05T11:29:08,713 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorization is enabled for input format(s) [org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] 2018-01-05T11:29:08,713 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing MapWork... 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[key], Vectorized Expression = col 0:int 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.SelectDesc 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:int 2018-01-05T11:29:08,713 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator groupby typeName int 2018-01-05T11:29:08,713 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkLongOperator 2018-01-05T11:29:08,713 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkLongOperator 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [key, value, ds, ROW__ID] 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [int, string, string, struct] 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE, NONE, NONE, NONE] 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 1 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 1 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Using reduce tag 0 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0] columnTypes=[bigint] 2018-01-05T11:29:08,713 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing ReduceWork... 2018-01-05T11:29:08,713 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vector GROUP BY operator will use processing mode GLOBAL 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[VALUE._col0], Vectorized Expression = col 0:bigint 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.GroupByDesc 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorFileSinkOperator 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.FileSinkDesc 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [VALUE._col0] 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [bigint] 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE] 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 0 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 0 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:29:08,713 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: llap mode: only 2018-01-05T11:29:08,713 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key' 2018-01-05T11:29:08,713 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@6ec69d02 can be done in LLAP 2018-01-05T11:29:08,713 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Map 4 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key' 2018-01-05T11:29:08,713 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@39138cce can be done in LLAP 2018-01-05T11:29:08,713 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Reducer 2 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'count()' 2018-01-05T11:29:08,713 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MergeJoinWork@4926f297 can be done in LLAP 2018-01-05T11:29:08,713 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Reducer 3 2018-01-05T11:29:08,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'count(VALUE._col0)' 2018-01-05T11:29:08,714 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:29:08,714 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:08,714 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@63e7333d can be done in LLAP 2018-01-05T11:29:08,714 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,714 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed plan generation 2018-01-05T11:29:08,714 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: validation start 2018-01-05T11:29:08,714 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:29:08,714 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:08,714 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,714 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:Explain, type:string, comment:null)], properties:null) 2018-01-05T11:29:08,714 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,714 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:08,714 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {getNotNullConstraints_(NotNullConstraintsRequest, )=0, isCompatibleWith_(Configuration, )=0, getAggrColStatsFor_(String, String, List, List, )=4, getTable_(String, String, )=13, flushCache_()=0, getUniqueConstraints_(UniqueConstraintsRequest, )=1, getPrimaryKeys_(PrimaryKeysRequest, )=2, listPartitions_(String, String, short, )=19, getForeignKeys_(ForeignKeysRequest, )=2} 2018-01-05T11:29:08,714 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112908_18aa9945-d13c-4022-863e-c6e8efbf6261); Time taken: 0.108 seconds 2018-01-05T11:29:08,714 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:08,714 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,714 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112908_18aa9945-d13c-4022-863e-c6e8efbf6261): explain vectorization detail select count(*) from tab a left outer join tab_part b on a.key = b.key 2018-01-05T11:29:08,715 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,715 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: explain vectorization detail select count(*) from tab a left outer join tab_part b on a.key = b.key 2018-01-05T11:29:08,715 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:29:08,715 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,715 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,715 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,715 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,715 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-3:EXPLAIN] in serial mode 2018-01-05T11:29:08,752 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,752 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,752 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: explain vectorization detail select count(*) from tab a left outer join tab_part b on a.key = b.key 2018-01-05T11:29:08,752 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:29:08,752 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,752 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,752 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:08,752 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {} 2018-01-05T11:29:08,752 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112908_18aa9945-d13c-4022-863e-c6e8efbf6261); Time taken: 0.038 seconds 2018-01-05T11:29:08,752 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:08,752 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,752 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:08,752 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,752 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query explain vectorization detail select count(*) from tab a left outer join tab_part b on a.key = b.key 2018-01-05T11:29:08,753 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting result file: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_606_6145014980571214667-1/-local-10004 2018-01-05T11:29:08,753 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_606_6145014980571214667-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-08_606_6145014980571214667-1 2018-01-05T11:29:08,754 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_606_6145014980571214667-1 2018-01-05T11:29:08,754 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.146 seconds, Fetched: 171 row(s) 2018-01-05T11:29:08,754 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:08,754 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:08,754 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:08,754 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:08,754 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,754 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:08,754 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,754 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,754 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,754 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: select count(*) from tab a left outer join tab_part b on a.key = b.key 2018-01-05T11:29:08,754 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827): select count(*) from tab a left outer join tab_part b on a.key = b.key 2018-01-05T11:29:08,754 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,754 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: select count(*) from tab a left outer join tab_part b on a.key = b.key 2018-01-05T11:29:08,755 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:08,755 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,755 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,755 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,755 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,755 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:08,755 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:29:08,755 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:08,755 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed phase 1 of Semantic Analysis 2018-01-05T11:29:08,755 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:08,755 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,755 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab 2018-01-05T11:29:08,755 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab 2018-01-05T11:29:08,758 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,758 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,758 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,758 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab_part 2018-01-05T11:29:08,758 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab_part 2018-01-05T11:29:08,761 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,761 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,761 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:08,761 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:08,772 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed getting MetaData in Semantic Analysis 2018-01-05T11:29:08,773 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,774 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,774 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_not_null_constraints : db=default tbl=tab 2018-01-05T11:29:08,774 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_not_null_constraints : db=default tbl=tab 2018-01-05T11:29:08,774 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,774 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,774 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,774 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab 2018-01-05T11:29:08,774 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab 2018-01-05T11:29:08,774 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,774 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,774 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,774 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab 2018-01-05T11:29:08,774 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab 2018-01-05T11:29:08,774 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,774 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,775 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,775 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_unique_constraints : db=default tbl=tab 2018-01-05T11:29:08,775 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_unique_constraints : db=default tbl=tab 2018-01-05T11:29:08,775 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,775 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,775 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,775 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab 2018-01-05T11:29:08,775 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab 2018-01-05T11:29:08,775 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,775 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,775 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,775 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_not_null_constraints : db=default tbl=tab_part 2018-01-05T11:29:08,775 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_not_null_constraints : db=default tbl=tab_part 2018-01-05T11:29:08,776 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,776 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,776 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,776 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:08,776 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:08,776 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,776 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,776 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,776 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:08,776 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:08,777 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,777 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,777 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,777 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_unique_constraints : db=default tbl=tab_part 2018-01-05T11:29:08,777 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_unique_constraints : db=default tbl=tab_part 2018-01-05T11:29:08,777 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,777 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,777 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,777 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab_part 2018-01-05T11:29:08,777 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab_part 2018-01-05T11:29:08,778 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,778 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,778 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:08,778 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:08,779 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:08,779 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:29:08,779 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,789 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.sql2rel: Plan after trimming unused fields HiveAggregate(group=[{}], agg#0=[count()]) HiveProject(DUMMY=[0]) HiveJoin(condition=[=($0, $1)], joinType=[left], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:08,789 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan before removing subquery: HiveProject(_o__c0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveProject($f0=[$0]) HiveJoin(condition=[=($0, $6)], joinType=[left], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:08,789 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan just after removing subquery: HiveProject(_o__c0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveProject($f0=[$0]) HiveJoin(condition=[=($0, $6)], joinType=[left], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:08,789 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan after decorrelation: HiveProject(_o__c0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveProject($f0=[$0]) HiveJoin(condition=[=($0, $6)], joinType=[left], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:08,790 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,790 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,790 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,790 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,790 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,791 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,791 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,791 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,791 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,791 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,791 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,801 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,801 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,801 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,801 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,802 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.sql2rel: Plan after trimming unused fields HiveAggregate(group=[{}], agg#0=[count()]) HiveProject(DUMMY=[0]) HiveJoin(condition=[=($0, $1)], joinType=[left], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:08,802 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,802 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,802 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,802 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,803 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,803 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,803 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,803 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,803 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,803 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,804 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,804 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,804 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,804 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,804 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,810 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Original plan for PlanModifier HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[left], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:08,811 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after top-level introduceDerivedTable HiveProject($f0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[left], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:08,811 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after nested convertOpTree HiveProject($f0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[left], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:08,811 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after propagating order HiveProject($f0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[left], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:08,811 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after fixTopOBSchema HiveProject($f0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[left], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:08,811 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Final plan after modifier HiveProject(_c0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[left], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:08,811 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:08,811 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:08,811 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:08,811 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:08,811 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,811 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab 2018-01-05T11:29:08,812 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab 2018-01-05T11:29:08,815 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,815 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,815 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:08,815 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:08,815 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:08,815 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,815 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab_part 2018-01-05T11:29:08,815 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab_part 2018-01-05T11:29:08,817 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,818 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,818 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:08,818 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:08,818 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:08,818 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Table Plan for a TS[0] 2018-01-05T11:29:08,818 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB a{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB a{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:08,818 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (. (tok_table_or_col a) key) key)) 2018-01-05T11:29:08,818 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = a{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} starRr = null 2018-01-05T11:29:08,818 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(key,_col0: int)} 2018-01-05T11:29:08,818 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:08,818 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block $hdt$_0 2018-01-05T11:29:08,818 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block $hdt$_0 2018-01-05T11:29:08,818 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Table Plan for b TS[2] 2018-01-05T11:29:08,818 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB b{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB b{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:08,818 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (. (tok_table_or_col b) key) key)) 2018-01-05T11:29:08,818 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = b{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} starRr = null 2018-01-05T11:29:08,818 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(key,_col0: int)} 2018-01-05T11:29:08,818 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:08,818 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block $hdt$_1 2018-01-05T11:29:08,818 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block $hdt$_1 2018-01-05T11:29:08,819 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:08,819 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB $hdt$_0{(key,_col0: int)} $hdt$_1{(key,_col1: int)} after GB {((tok_functionstar count),_col0: bigint)} 2018-01-05T11:29:08,819 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (tok_functionstar count) _c0)) 2018-01-05T11:29:08,819 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = {((tok_functionstar count),_col0: bigint)} starRr = $hdt$_0{(key,_col0: int)} $hdt$_1{(key,_col1: int)} 2018-01-05T11:29:08,819 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(_c0,_col0: bigint)} 2018-01-05T11:29:08,819 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:08,819 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-08_754_5378951951843078555-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/-mr-10001 2018-01-05T11:29:08,819 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-08_754_5378951951843078555-1 2018-01-05T11:29:08,841 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/-mr-10003 2018-01-05T11:29:08,841 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/-mr-10001 row schema: null{(_c0,_col0: bigint)} 2018-01-05T11:29:08,842 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block null 2018-01-05T11:29:08,842 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:29:08,842 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: CBO Succeeded; optimized logical plan. 2018-01-05T11:29:08,842 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Before logical optimization TS[0]-SEL[1]-RS[4]-JOIN[6]-SEL[7]-GBY[8]-RS[9]-GBY[10]-SEL[11]-FS[12] TS[2]-SEL[3]-RS[5]-JOIN[6] 2018-01-05T11:29:08,842 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,842 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,842 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,842 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] lineage.Generator: Time taken for lineage transform=0 2018-01-05T11:29:08,842 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,842 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,843 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,843 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,843 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SyntheticJoinPredicate: Synthetic predicate: 1 --> 0 2018-01-05T11:29:08,843 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,843 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,843 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FS(12) 2018-01-05T11:29:08,843 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(11) 2018-01-05T11:29:08,843 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for GBY(10) 2018-01-05T11:29:08,843 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(9) 2018-01-05T11:29:08,843 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for GBY(8) 2018-01-05T11:29:08,843 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(7) 2018-01-05T11:29:08,843 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for JOIN(6) 2018-01-05T11:29:08,843 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(4) 2018-01-05T11:29:08,843 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(1) 2018-01-05T11:29:08,843 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:29:08,843 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(5) 2018-01-05T11:29:08,843 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(13) 2018-01-05T11:29:08,843 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias $hdt$_1: (_col0) IN (RS[4]) 2018-01-05T11:29:08,843 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(3) 2018-01-05T11:29:08,843 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of SEL for alias $hdt$_1: (key) IN (RS[4]) 2018-01-05T11:29:08,843 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for TS(2) 2018-01-05T11:29:08,843 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of TS for alias b: (key) IN (RS[4]) 2018-01-05T11:29:08,843 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SimplePredicatePushDown: After PPD: TS[0]-SEL[1]-RS[4]-JOIN[6]-SEL[7]-GBY[8]-RS[9]-GBY[10]-SEL[11]-FS[12] TS[2]-FIL[14]-SEL[3]-RS[5]-JOIN[6] 2018-01-05T11:29:08,843 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,843 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,843 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,843 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,843 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,843 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,843 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,843 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,843 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,844 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,844 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partitions : db=default tbl=tab_part 2018-01-05T11:29:08,844 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions : db=default tbl=tab_part 2018-01-05T11:29:08,846 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,847 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.163568ms + 0.003709ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? ] 2018-01-05T11:29:08,847 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.164843ms + 0.06386ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (14) order by "PART_NAME" asc] 2018-01-05T11:29:08,847 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.113813ms + 0.076867ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (14) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:29:08,847 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.094455ms + 0.022641ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (14) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:08,848 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.085597ms + 0.007853ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (41) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:29:08,848 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.086052ms + 0.026257ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (41) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:08,848 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.085893ms + 0.02196ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (41) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:08,848 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.079944ms + 0.007934ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (41) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:08,848 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.120619ms + 0.026738ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (26) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:08,848 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.092068ms + 0.025902ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (41) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:29:08,848 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,851 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,851 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,851 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,851 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pcr.PcrExprProcFactory: Processing (key) IN (RS[4]) org.apache.hadoop.hive.ql.udf.generic.GenericUDFIn outputs {UNKNOWN, Column[key]}, {UNKNOWN, RS[4]}, 2018-01-05T11:29:08,851 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,851 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,851 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,851 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 9 key:[] 2018-01-05T11:29:08,852 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 9 oldColExprMap: {VALUE._col0=Column[_col0]} 2018-01-05T11:29:08,852 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 9 newColExprMap: {VALUE._col0=Column[_col0]} 2018-01-05T11:29:08,852 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: JOIN 6 oldExprs: {0=[Column[KEY.reducesinkkey0]], 1=[Column[KEY.reducesinkkey0]]} 2018-01-05T11:29:08,852 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: JOIN 6 newExprs: {0=[], 1=[]} 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 4 key:[Column[_col0]] 2018-01-05T11:29:08,852 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 4 oldColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:08,852 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 4 newColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 5 key:[Column[_col0]] 2018-01-05T11:29:08,852 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 5 oldColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:08,852 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 5 newColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.IdentityProjectRemover: Identity project remover optimization removed : SEL[11] 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.IdentityProjectRemover: Identity project remover optimization removed : SEL[7] 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: After logical optimization TS[0]-SEL[1]-RS[4]-JOIN[6]-GBY[8]-RS[9]-GBY[10]-FS[12] TS[2]-FIL[14]-SEL[3]-RS[5]-JOIN[6] 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Parent: TS[2] 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Filter: (key) IN (RS[4]) (type: boolean) 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: TableScan: TS[2] 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Column key is not a partition column 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Initiate semijoin reduction for key ((key) IN (RS[4]) 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:08,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:08,853 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:08,853 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:08,853 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: DynamicSemiJoinPushdown: Saving RS to TS mapping: RS[19]: TS[2] 2018-01-05T11:29:08,853 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,853 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,853 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,853 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,853 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partitions : db=default tbl=tab 2018-01-05T11:29:08,853 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions : db=default tbl=tab 2018-01-05T11:29:08,855 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,856 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.148347ms + 0.002555ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? ] 2018-01-05T11:29:08,856 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.122234ms + 0.038217ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (15) order by "PART_NAME" asc] 2018-01-05T11:29:08,856 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.072998ms + 0.047152ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (15) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:29:08,856 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.065196ms + 0.015325ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (15) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:08,856 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.059711ms + 0.005572ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (43) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:29:08,856 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.060157ms + 0.017739ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (43) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:08,856 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.059ms + 0.014761ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (43) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:08,856 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.054059ms + 0.004969ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (43) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:08,857 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.079254ms + 0.018571ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (28) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:08,857 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.063167ms + 0.017203ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (43) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:29:08,857 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,859 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,860 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,860 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,860 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Estimated average row size: 4 2018-01-05T11:29:08,860 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,860 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_aggr_stats_for: db=default table=tab 2018-01-05T11:29:08,860 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_aggr_stats_for: db=default table=tab 2018-01-05T11:29:08,861 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.AggregateStatsCache: Returning aggregate stats from the cache; total hits: 9, total misses: 0, hit ratio: 1.0 2018-01-05T11:29:08,861 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: useDensityFunctionForNDVEstimation = false partsFound = 1 ColumnStatisticsObj = [ColumnStatisticsObj(colName:key, colType:int, statsData:)] 2018-01-05T11:29:08,862 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,863 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,863 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting stats (Num rows: 242 Data size: 968 Basic stats: COMPLETE Column stats: COMPLETE) on TS[0] 2018-01-05T11:29:08,863 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (tab): numRows: 242 dataSize: 968 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {key= colName: key colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,863 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Estimated average row size: 4 2018-01-05T11:29:08,863 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,863 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_aggr_stats_for: db=default table=tab_part 2018-01-05T11:29:08,863 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_aggr_stats_for: db=default table=tab_part 2018-01-05T11:29:08,865 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.AggregateStatsCache: Returning aggregate stats from the cache; total hits: 10, total misses: 0, hit ratio: 1.0 2018-01-05T11:29:08,865 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: useDensityFunctionForNDVEstimation = false partsFound = 1 ColumnStatisticsObj = [ColumnStatisticsObj(colName:key, colType:int, statsData:)] 2018-01-05T11:29:08,866 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:08,866 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:08,866 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on TS[2] 2018-01-05T11:29:08,866 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-TS[2] (tab_part): numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {key= colName: key colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,866 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 242 Data size: 968 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[1] 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[1]: numRows: 242 dataSize: 968 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_4_a_key_min), DynamicValue(RS_4_a_key_max)) Original num rows: 500 New num rows: 500 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_4_a_key_min), DynamicValue(RS_4_a_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_4_a_key_bloom_filter))) Original num rows: 500 New num rows: 500 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-FIL[14]: numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {key= colName: key colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on FIL[14] 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 242 Data size: 968 Basic stats: COMPLETE Column stats: COMPLETE) on RS[4] 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[4]: numRows: 242 dataSize: 968 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {KEY.reducesinkkey0= colName: KEY.reducesinkkey0 colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 242 Data size: 968 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[15] 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[15]: numRows: 242 dataSize: 968 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[3] 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[3]: numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[16]: inputSize: 968 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[16] hashAgg: true 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 3] STATS-GBY[16]: cardinality: 1 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[16] 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[16]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on RS[5] 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[5]: numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {KEY.reducesinkkey0= colName: KEY.reducesinkkey0 colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on RS[17] 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[17]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-JOIN[6]: detects none/multiple PK parents. 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.JoinOperator: Setting stats (Num rows: 392 Data size: 3136 Basic stats: COMPLETE Column stats: COMPLETE) on JOIN[6] 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-JOIN[6]: numRows: 392 dataSize: 3136 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[18]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[18] hashAgg: false 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 9] STATS-GBY[18]: cardinality: 1 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[18] 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[18]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[8]: inputSize: 3136 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[8] hashAgg: true 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 3] STATS-GBY[8]: cardinality: 1 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[8] 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[8]: numRows: 1 dataSize: 8 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on RS[19] 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[19]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE) on RS[9] 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[9]: numRows: 1 dataSize: 8 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {VALUE._col0= colName: VALUE._col0 colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[10]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[10] hashAgg: false 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 9] STATS-GBY[10]: cardinality: 1 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[10] 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[10]: numRows: 1 dataSize: 8 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-FS[12]: numRows: 1 dataSize: 8 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[key]]; bucket count: 2 }) on TS[0] 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[key]]; bucket count: 4 }) on TS[2] 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: 2 }) on SEL[1] 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[key]]; bucket count: 4 }) on FIL[14] 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[KEY.reducesinkkey0]]; sort column names: [[KEY.reducesinkkey0]]; bucket count: 2 }) on RS[4] 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: 2 }) on SEL[15] 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: 4 }) on SEL[3] 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[16] 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[KEY.reducesinkkey0]]; sort column names: [[KEY.reducesinkkey0]]; bucket count: 4 }) on RS[5] 2018-01-05T11:29:08,867 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[17] 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.JoinOperator: Setting traits ({ bucket column names: [[], []]; sort column names: [[], []]; bucket count: -1 }) on JOIN[6] 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[18] 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[8] 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[19] 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[9] 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[10] 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FileSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on FS[12] 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,868 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Set parallelism for reduce sink RS[4] to: 1 2018-01-05T11:29:08,868 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Set parallelism for reduce sink RS[5] to: 1 2018-01-05T11:29:08,868 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConvertJoinMapJoin: Memory monitor info set to : { isLlap: true executorsPerNode: 4 maxExecutorsOverSubscribeMemory: 3 memoryOverSubscriptionFactor: 0.20000000298023224 memoryCheckInterval: 100000 noConditionalTaskSize: 10000000 adjustedNoConditionalTaskSize: 16000000 hashTableInflationFactor: 2.0 threshold: 32000000 } 2018-01-05T11:29:08,868 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConvertJoinMapJoin: Fallback to common merge join operator 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.CommonMergeJoinOperator: Setting traits ({ bucket column names: [[], []]; sort column names: [[], []]; bucket count: 0 }) on MERGEJOIN[20] 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.CommonMergeJoinOperator: Setting stats (Num rows: 392 Data size: 3136 Basic stats: COMPLETE Column stats: COMPLETE) on MERGEJOIN[20] 2018-01-05T11:29:08,868 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:08,868 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:08,868 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing BloomFilter cost/benefit for RS[19] - TS[2] (b) (Column[key]) 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing key domain cardinality, keyDomainCardinality=461, semiJoinKeyIsPK=false, selColStat= colName: _col0 colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false, selColSourceStat= colName: key colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false, tsColStat= colName: key colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit for Column[_col0] to Column[key], selKeyCardinality=153, tsKeyCardinality=308, tsRows=500, keyDomainCardinality=461 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: SemiJoin key selectivity=0.3318872017353579, benefit=334.056399132321 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit=334.056399132321, cost=242.0, tsDataSize=500, netBenefit=92.05639913232102 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: netBenefit=0.18411279826464202 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Reduction factor not satisfied for RS[19]-TS[2] (b). Removing semijoin optimization. 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Removing ReduceSink RS[19] and TableScan TS[2] 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: FS, 12 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: GBY, 10 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 9 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: GBY, 8 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: MERGEJOIN, 20 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 4 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 1 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 5 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 3 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: FIL, 14 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: TS, 2 2018-01-05T11:29:08,868 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Cycle free: true 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: Before SharedWorkOptimizer: TS[0]-SEL[1]-RS[4]-MERGEJOIN[20]-GBY[8]-RS[9]-GBY[10]-FS[12] TS[2]-FIL[14]-SEL[3]-RS[5]-MERGEJOIN[20] 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: DPP information stored in the cache: {} 2018-01-05T11:29:08,868 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: Sorted tables by size: [default.tab_part=2000, default.tab=968] 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: After SharedWorkOptimizer: TS[0]-SEL[1]-RS[4]-MERGEJOIN[20]-GBY[8]-RS[9]-GBY[10]-FS[12] TS[2]-FIL[14]-SEL[3]-RS[5]-MERGEJOIN[20] 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: int|{a}key,value: string|{a}value,ds: string|{a}ds,BLOCK__OFFSET__INSIDE__FILE: bigint|{a}block__offset__inside__file,INPUT__FILE__NAME: string|{a}input__file__name,ROW__ID: struct|{a}row__id) 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[2] with rs:(key: int|{b}key,value: string|{b}value,ds: string|{b}ds,BLOCK__OFFSET__INSIDE__FILE: bigint|{b}block__offset__inside__file,INPUT__FILE__NAME: string|{b}input__file__name,ROW__ID: struct|{b}row__id) 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(_col0: int|{$hdt$_0}key) 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key]) 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FIL[14] with rs:(key: int|{b}key) 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_2 {} 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FIL[14] 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Old filter FIL[14] conditions:(true and true) 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(Const boolean true, Const boolean true) -> Const boolean true 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Filter expression GenericUDFOPAnd(Const boolean true, Const boolean true) holds true. Will delete it. 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New filter FIL[14] conditions:true 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[4] with rs:(KEY.reducesinkkey0: int|{$hdt$_0}key) 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[4] 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[3] with rs:(_col0: int|{$hdt$_1}key) 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op FIL_14 {} 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[3] 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key]) 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[4] with rs:(KEY.reducesinkkey0: int|{$hdt$_0}key) 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[4] 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[5] with rs:(KEY.reducesinkkey0: int|{$hdt$_1}key) 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_3 {} 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[5] 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:MERGEJOIN[20] with rs:() 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_4 {} 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_5 {} 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator MERGEJOIN[20] 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[8] with rs:(_col0: bigint|{}(tok_functionstar count)) 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op MERGEJOIN_20 {} 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[8] 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[9] with rs:(VALUE._col0: bigint|{}(tok_functionstar count)) 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_8 {} 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[9] 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[10] with rs:(_col0: bigint|{}(tok_functionstar count)) 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_9 {} 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[10] 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[12] with rs:(_col0: bigint|{}) 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_10 {} 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[12] 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: TS[0] 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[4] 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 of table a 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[4] 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: MERGEJOIN[20] 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[9] 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding reduce work (Reducer 2) for MERGEJOIN[20] 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Setting up reduce sink: RS[4] with following reduce work: Reducer 2 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[4] as parent from MERGEJOIN[20] 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[5] as parent from MERGEJOIN[20] 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[9] 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: GBY[10] 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: FS[12] 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding reduce work (Reducer 3) for GBY[10] 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Setting up reduce sink: RS[9] with following reduce work: Reducer 3 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[9] as parent from GBY[10] 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: FS[12] 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: TS[2] 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[5] 2018-01-05T11:29:08,869 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding map work (Map 4) for TS[2] 2018-01-05T11:29:08,870 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 of table b 2018-01-05T11:29:08,870 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:08,870 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Second pass. Leaf operator: RS[5] has common downstream work: org.apache.hadoop.hive.ql.plan.MergeJoinWork@5236b18c 2018-01-05T11:29:08,870 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:29:08,870 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,870 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,870 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:29:08,870 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:08,870 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:08,870 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 4 2018-01-05T11:29:08,870 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:08,870 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:08,870 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:29:08,870 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:08,870 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:08,870 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 4 2018-01-05T11:29:08,870 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:08,870 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:08,870 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:29:08,870 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Examining input format to see if vectorization is enabled. 2018-01-05T11:29:08,870 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorization is enabled for input format(s) [org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] 2018-01-05T11:29:08,870 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing MapWork... 2018-01-05T11:29:08,870 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[key], Vectorized Expression = col 0:int 2018-01-05T11:29:08,870 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator 2018-01-05T11:29:08,870 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.SelectDesc 2018-01-05T11:29:08,870 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:int 2018-01-05T11:29:08,871 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator groupby typeName int 2018-01-05T11:29:08,871 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkLongOperator 2018-01-05T11:29:08,871 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkLongOperator 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [key, value, ds, ROW__ID] 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [int, string, string, struct] 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE, NONE, NONE, NONE] 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 1 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 1 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:08,871 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Examining input format to see if vectorization is enabled. 2018-01-05T11:29:08,871 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorization is enabled for input format(s) [org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] 2018-01-05T11:29:08,871 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing MapWork... 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[key], Vectorized Expression = col 0:int 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.SelectDesc 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:int 2018-01-05T11:29:08,871 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator groupby typeName int 2018-01-05T11:29:08,871 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkLongOperator 2018-01-05T11:29:08,871 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkLongOperator 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [key, value, ds, ROW__ID] 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [int, string, string, struct] 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE, NONE, NONE, NONE] 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 1 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 1 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Using reduce tag 0 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0] columnTypes=[bigint] 2018-01-05T11:29:08,871 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing ReduceWork... 2018-01-05T11:29:08,871 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vector GROUP BY operator will use processing mode GLOBAL 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[VALUE._col0], Vectorized Expression = col 0:bigint 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.GroupByDesc 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorFileSinkOperator 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.FileSinkDesc 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [VALUE._col0] 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [bigint] 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE] 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 0 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 0 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:29:08,871 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: llap mode: only 2018-01-05T11:29:08,871 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key' 2018-01-05T11:29:08,871 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@7f82e3d8 can be done in LLAP 2018-01-05T11:29:08,871 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Map 4 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key' 2018-01-05T11:29:08,871 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@41c853ce can be done in LLAP 2018-01-05T11:29:08,871 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Reducer 2 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'count()' 2018-01-05T11:29:08,871 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:08,871 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MergeJoinWork@5236b18c can be done in LLAP 2018-01-05T11:29:08,871 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Reducer 3 2018-01-05T11:29:08,872 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'count(VALUE._col0)' 2018-01-05T11:29:08,872 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:29:08,872 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:08,872 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@29bcf902 can be done in LLAP 2018-01-05T11:29:08,872 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,872 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed plan generation 2018-01-05T11:29:08,872 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:08,872 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: validation start 2018-01-05T11:29:08,872 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:29:08,872 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,872 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0, type:bigint, comment:null)], properties:null) 2018-01-05T11:29:08,872 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: Initializing operator LIST_SINK[27] 2018-01-05T11:29:08,872 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: Initialization Done 27 LIST_SINK 2018-01-05T11:29:08,872 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: Operator 27 LIST_SINK initialized 2018-01-05T11:29:08,872 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: Initialization Done 27 LIST_SINK done is reset. 2018-01-05T11:29:08,872 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,872 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:08,872 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {getNotNullConstraints_(NotNullConstraintsRequest, )=1, isCompatibleWith_(Configuration, )=0, getAggrColStatsFor_(String, String, List, List, )=6, getTable_(String, String, )=13, flushCache_()=0, getUniqueConstraints_(UniqueConstraintsRequest, )=0, getPrimaryKeys_(PrimaryKeysRequest, )=2, listPartitions_(String, String, short, )=14, getForeignKeys_(ForeignKeysRequest, )=1} 2018-01-05T11:29:08,872 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827); Time taken: 0.118 seconds 2018-01-05T11:29:08,872 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:08,872 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,872 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827): select count(*) from tab a left outer join tab_part b on a.key = b.key 2018-01-05T11:29:08,873 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,873 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: select count(*) from tab a left outer join tab_part b on a.key = b.key 2018-01-05T11:29:08,873 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:29:08,873 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@tab 2018-01-05T11:29:08,873 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@tab@ds=2008-04-08 2018-01-05T11:29:08,874 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@tab_part 2018-01-05T11:29:08,874 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@tab_part@ds=2008-04-08 2018-01-05T11:29:08,874 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/-mr-10001 2018-01-05T11:29:08,874 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,874 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,874 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,874 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Query ID = djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827 2018-01-05T11:29:08,874 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Total jobs = 1 2018-01-05T11:29:08,874 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,874 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:29:08,874 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-1:MAPRED] in serial mode 2018-01-05T11:29:08,884 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:29:08,906 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:29:08,906 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:29:08,906 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:29:08,906 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827 2018-01-05T11:29:08,906 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Session is already open 2018-01-05T11:29:08,906 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:29:08,906 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,906 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Dag name: select count(*) from tab a left oute...b.key (Stage-1) 2018-01-05T11:29:08,906 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: DagInfo: {"context":"Hive","description":"\n\nselect count(*)\nfrom tab a left outer join tab_part b on a.key = b.key"} 2018-01-05T11:29:08,906 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827 with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:29:08,906 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,918 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,918 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:29:08,919 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,919 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 3 size: 1.91KB 2018-01-05T11:29:08,937 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,937 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,948 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,948 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:29:08,949 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,949 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 2 size: 1.49KB 2018-01-05T11:29:08,952 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,954 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,967 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:29:08,967 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,967 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:29:08,968 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,968 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: Serialized plan (via RPC) - name: Map 4 size: 3.07KB 2018-01-05T11:29:08,973 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:08,973 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,975 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,987 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:29:08,987 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,987 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:29:08,987 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,987 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 3.05KB 2018-01-05T11:29:08,992 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:08,992 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,994 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,994 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:08,994 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=select count(*) from tab a left oute...b.key (Stage-1), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827 } 2018-01-05T11:29:08,994 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:08,994 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:08,994 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:08,994 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:08,996 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: select count(*) from tab a left oute...b.key (Stage-1) 2018-01-05T11:29:08,996 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:29:08,997 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_39, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_39.dot 2018-01-05T11:29:08,997 INFO [IPC Server handler 1 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_39 2018-01-05T11:29:08,997 WARN [IPC Server handler 1 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:29:08,997 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Running DAG: select count(*) from tab a left oute...b.key (Stage-1), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827 } 2018-01-05T11:29:09,020 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_39, submitTime=1515180548996, queueName=null 2018-01-05T11:29:09,020 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_39_00 [Map 4]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_39_00 [Map 4], Default Resources= 2018-01-05T11:29:09,020 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_39_00 [Map 4] 2018-01-05T11:29:09,020 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 4 2018-01-05T11:29:09,020 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_39_00 [Map 4] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:09,021 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_39_01 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_39_01 [Map 1], Default Resources= 2018-01-05T11:29:09,021 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_39_01 [Map 1] 2018-01-05T11:29:09,021 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:29:09,021 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_39_01 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:09,021 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_39_02 [Reducer 2]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_39_02 [Reducer 2], Default Resources= 2018-01-05T11:29:09,021 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 2 2018-01-05T11:29:09,021 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_39_02 [Reducer 2] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:09,022 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_39_03 [Reducer 3]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_39_03 [Reducer 3], Default Resources= 2018-01-05T11:29:09,022 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_39_03 [Reducer 3] 2018-01-05T11:29:09,022 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 3 2018-01-05T11:29:09,022 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_39_03 [Reducer 3] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:09,022 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:29:09,022 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_39, initTime=1515180549022 2018-01-05T11:29:09,022 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: dag_1515180518813_0001_39 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:29:09,022 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:29:09,022 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_39, startTime=1515180549022 2018-01-05T11:29:09,022 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_39 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:29:09,022 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {a={InputName=a}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:29:09,022 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: a, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:29:09,022 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_39_01 [Map 1] 2018-01-05T11:29:09,022 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_39, dagName=select count(*) from tab a left oute...b.key (Stage-1) 2018-01-05T11:29:09,022 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,022 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,022 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,026 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_39_01 [Map 1] 2018-01-05T11:29:09,026 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_39_01 [Map 1] 2018-01-05T11:29:09,026 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:09,027 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_39_01 [Map 1] 2018-01-05T11:29:09,028 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:29:09,028 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@3e7f1d25 2018-01-05T11:29:09,029 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/b1618906-d66b-4401-932e-90b4d1f79b04/map.xml 2018-01-05T11:29:09,029 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/b1618906-d66b-4401-932e-90b4d1f79b04/map.xml 2018-01-05T11:29:09,029 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/b1618906-d66b-4401-932e-90b4d1f79b04/map.xml 2018-01-05T11:29:09,029 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:09,029 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:09,029 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:09,029 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:09,029 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.05KB 2018-01-05T11:29:09,030 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_39_01 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:29:09,030 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 4 : {b={InputName=b}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:29:09,030 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: b, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:29:09,030 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_39_00 [Map 4] 2018-01-05T11:29:09,044 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: a on vertex vertex_1515180518813_0001_39_01 [Map 1] 2018-01-05T11:29:09,044 DEBUG [AMReporterQueueDrainer] impl.AMReporter: Removing am 10.22.8.182:49824 with last associated dag QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=24} from heartbeat with taskCount=0, amFailed=false, isDone=true 2018-01-05T11:29:09,044 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:29:09,044 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:09,044 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:29:09,044 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:09,044 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/b1618906-d66b-4401-932e-90b4d1f79b04/map.xml 2018-01-05T11:29:09,044 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,044 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias a 2018-01-05T11:29:09,044 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:09,044 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:09,045 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 0 2018-01-05T11:29:09,045 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = key 2018-01-05T11:29:09,045 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:09,045 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:09,045 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:09,045 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:09,045 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:09,045 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: getSplits started 2018-01-05T11:29:09,045 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: Number of buckets specified by conf file is 2 2018-01-05T11:29:09,045 INFO [InputInitializer {Map 1} #0] orc.OrcInputFormat: ORC pushdown predicate: null 2018-01-05T11:29:09,045 INFO [InputInitializer {Map 1} #0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:09,045 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: Generate splits schema evolution property true reader schema struct transactional scan property false 2018-01-05T11:29:09,045 ERROR [ORC_GET_SPLITS #7] io.AcidUtils: Failed to get files with ID; using regular API: Only supported for DFS; got class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:09,048 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_39_00 [Map 4] 2018-01-05T11:29:09,048 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_39_00 [Map 4] 2018-01-05T11:29:09,049 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:09,049 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_39_00 [Map 4] 2018-01-05T11:29:09,051 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:29:09,051 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@2628a4a5 2018-01-05T11:29:09,051 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/dcdeb337-7358-435a-9630-6d83a88e69f1/map.xml 2018-01-05T11:29:09,051 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/dcdeb337-7358-435a-9630-6d83a88e69f1/map.xml 2018-01-05T11:29:09,051 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/dcdeb337-7358-435a-9630-6d83a88e69f1/map.xml 2018-01-05T11:29:09,051 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:09,051 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:09,051 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:09,052 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:09,052 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 4 size: 3.07KB 2018-01-05T11:29:09,052 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_39_00 [Map 4] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:29:09,052 INFO [InputInitializer {Map 4} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: b on vertex vertex_1515180518813_0001_39_00 [Map 4] 2018-01-05T11:29:09,052 INFO [InputInitializer {Map 4} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:29:09,053 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to ShuffleVertexManager for vertex_1515180518813_0001_39_02 [Reducer 2] 2018-01-05T11:29:09,053 DEBUG [InputInitializer {Map 4} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:09,053 INFO [InputInitializer {Map 4} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:29:09,053 DEBUG [InputInitializer {Map 4} #0] log.PerfLogger: 2018-01-05T11:29:09,053 INFO [InputInitializer {Map 4} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/dcdeb337-7358-435a-9630-6d83a88e69f1/map.xml 2018-01-05T11:29:09,053 DEBUG [InputInitializer {Map 4} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,053 INFO [InputInitializer {Map 4} #0] exec.Utilities: Processing alias b 2018-01-05T11:29:09,053 DEBUG [InputInitializer {Map 4} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:09,053 INFO [InputInitializer {Map 4} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:09,054 INFO [InputInitializer {Map 4} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 0 2018-01-05T11:29:09,054 INFO [InputInitializer {Map 4} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = key 2018-01-05T11:29:09,054 INFO [InputInitializer {Map 4} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:09,054 DEBUG [InputInitializer {Map 4} #0] orc.OrcInputFormat: getSplits started 2018-01-05T11:29:09,054 DEBUG [InputInitializer {Map 4} #0] orc.OrcInputFormat: Number of buckets specified by conf file is 4 2018-01-05T11:29:09,054 INFO [InputInitializer {Map 4} #0] orc.OrcInputFormat: ORC pushdown predicate: null 2018-01-05T11:29:09,054 INFO [InputInitializer {Map 4} #0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:09,054 DEBUG [InputInitializer {Map 4} #0] orc.OrcInputFormat: Generate splits schema evolution property true reader schema struct transactional scan property false 2018-01-05T11:29:09,054 ERROR [ORC_GET_SPLITS #6] io.AcidUtils: Failed to get files with ID; using regular API: Only supported for DFS; got class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:09,057 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.05 maxFrac: 0.75 auto: false desiredTaskIput: 104857600 2018-01-05T11:29:09,057 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:29:09,057 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 1 tasks for vertex: vertex_1515180518813_0001_39_02 [Reducer 2] 2018-01-05T11:29:09,057 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_39_02 [Reducer 2] 2018-01-05T11:29:09,058 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_39_02, reconfigureDoneTime=1515180549058, numTasks=1, vertexLocationHint=null, edgeManagersCount=2, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:29:09,058 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:VERTEX_INITIALIZED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_39_02, initRequestedTime=1515180549052, initedTime=1515180549058, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:09,058 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_39_02 [Reducer 2] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:29:09,058 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to ShuffleVertexManager for vertex_1515180518813_0001_39_03 [Reducer 3] 2018-01-05T11:29:09,062 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.05 maxFrac: 0.75 auto: false desiredTaskIput: 104857600 2018-01-05T11:29:09,062 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:29:09,062 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 1 tasks for vertex: vertex_1515180518813_0001_39_03 [Reducer 3] 2018-01-05T11:29:09,062 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_39_03 [Reducer 3] 2018-01-05T11:29:09,063 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_39_03, reconfigureDoneTime=1515180549062, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:29:09,063 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_39_03 [Reducer 3], numAdditionalOutputs=1 2018-01-05T11:29:09,063 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:VERTEX_INITIALIZED]: vertexName=Reducer 3, vertexId=vertex_1515180518813_0001_39_03, initRequestedTime=1515180549058, initedTime=1515180549063, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:09,063 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_39_03 [Reducer 3] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:29:09,073 DEBUG [ORC_GET_SPLITS #7] io.AcidUtils: in directory file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 base = null deltas = 0 2018-01-05T11:29:09,073 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: Split strategy: ETLSplitStrategy strategy for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:09,073 DEBUG [ORC_GET_SPLITS #9] orc.LocalCache: Serialized tail cached for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0 2018-01-05T11:29:09,073 DEBUG [ORC_GET_SPLITS #9] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,073 DEBUG [ORC_GET_SPLITS #9] orc.LocalCache: Serialized tail cached for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0 2018-01-05T11:29:09,073 DEBUG [ORC_GET_SPLITS #9] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,073 DEBUG [ORC_GET_SPLITS #8] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,073 DEBUG [ORC_GET_SPLITS #9] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,074 DEBUG [ORC_GET_SPLITS #8] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:29:09,074 DEBUG [ORC_GET_SPLITS #9] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:29:09,074 INFO [InputInitializer {Map 1} #0] orc.OrcInputFormat: FooterCacheHitRatio: 2/2 2018-01-05T11:29:09,074 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0, start=3, length=566, isOriginal=true, fileLength=744, hasFooter=false, hasBase=true, deltas=0] projected_columns_uncompressed_size: 371 2018-01-05T11:29:09,074 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0, start=3, length=583, isOriginal=true, fileLength=767, hasFooter=false, hasBase=true, deltas=0] projected_columns_uncompressed_size: 364 2018-01-05T11:29:09,074 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: getSplits finished 2018-01-05T11:29:09,074 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 2 2018-01-05T11:29:09,074 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:09,074 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 2. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:29:09,074 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/b1618906-d66b-4401-932e-90b4d1f79b04/map.xml 2018-01-05T11:29:09,074 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/b1618906-d66b-4401-932e-90b4d1f79b04/map.xml 2018-01-05T11:29:09,074 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/b1618906-d66b-4401-932e-90b4d1f79b04/map.xml 2018-01-05T11:29:09,074 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:09,074 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:09,074 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:09,074 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:09,075 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.05KB 2018-01-05T11:29:09,075 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0 to src new group? true 2018-01-05T11:29:09,075 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0 to src new group? false 2018-01-05T11:29:09,075 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:29:09,075 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:29:09,075 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:29:09,075 DEBUG [InputInitializer {Map 1} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 364 2018-01-05T11:29:09,075 DEBUG [InputInitializer {Map 1} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 371 2018-01-05T11:29:09,075 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 13 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 735 Original splits: 2 2018-01-05T11:29:09,075 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired numSplits: 1 lengthPerGroup: 735 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 735 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false 2018-01-05T11:29:09,075 DEBUG [InputInitializer {Map 1} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 364 2018-01-05T11:29:09,075 DEBUG [InputInitializer {Map 1} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 371 2018-01-05T11:29:09,075 DEBUG [InputInitializer {Map 1} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 371 2018-01-05T11:29:09,075 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Number of splits desired: 1 created: 1 splitsProcessed: 2 2018-01-05T11:29:09,075 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 2 grouped split count is 1, for bucket: 1 2018-01-05T11:29:09,075 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:29:09,075 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: a on vertex vertex_1515180518813_0001_39_01 [Map 1] 2018-01-05T11:29:09,075 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_39_01 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:09,075 INFO [App Shared Pool - #0] impl.VertexImpl: Got updated RootInputsSpecs: {a=forAllWorkUnits=true, update=[1]} 2018-01-05T11:29:09,076 INFO [App Shared Pool - #0] impl.VertexImpl: Vertex vertex_1515180518813_0001_39_01 [Map 1] parallelism set to 1 2018-01-05T11:29:09,076 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_39_01 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:09,076 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_39_01, initRequestedTime=1515180549022, initedTime=1515180549076, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:09,076 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_39_01 [Map 1] with distanceFromRoot: 0 2018-01-05T11:29:09,076 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_39_01 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:29:09,077 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_39_01, startRequestedTime=1515180549052, startedTime=1515180549076 2018-01-05T11:29:09,077 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_39_01, reconfigureDoneTime=1515180549077, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:29:09,077 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_39_01 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:09,077 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_39_01 for vertex: vertex_1515180518813_0001_39_02 [Reducer 2] numStartedSources: 1 numSources: 2 2018-01-05T11:29:09,077 INFO [Dispatcher thread {Central}] impl.VertexImpl: Cannot start vertex: vertex_1515180518813_0001_39_02 [Reducer 2] numStartedSources: 1 numSources: 2 2018-01-05T11:29:09,077 INFO [App Shared Pool - #0] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:29:09,077 INFO [App Shared Pool - #0] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:29:09,077 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_39_01 [Map 1] 2018-01-05T11:29:09,077 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_39_01_000000, scheduledTime=1515180549077, launchTime=1515180549077 2018-01-05T11:29:09,077 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_39_01 [Map 1] 2018-01-05T11:29:09,077 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_39_01_000000_0, priority=14, capability=, hosts=null 2018-01-05T11:29:09,077 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:09,077 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:09,077 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:09,077 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_39_01_000000_0 on hosts=null 2018-01-05T11:29:09,077 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_39_01_000000_0 on hosts=null 2018-01-05T11:29:09,077 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_39_01_000000_0 2018-01-05T11:29:09,077 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:09,077 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #93, task=TaskInfo{task=attempt_1515180518813_0001_39_01_000000_0, priority=14, startTime=0, containerId=null, uniqueId=92, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000093 2018-01-05T11:29:09,077 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_39_01_000000_0; false 2018-01-05T11:29:09,077 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_39_01_000000_0, priority=14, startTime=311569587, containerId=container_111101111_0001_00_000093assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=92, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:09,078 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000093 for node: 10.22.8.182:49818 2018-01-05T11:29:09,080 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 39, name=select count(*) from tab a left oute...b.key (Stage-1), queryId=djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827 2018-01-05T11:29:09,080 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000093, attempt_1515180518813_0001_39_01_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:09,080 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000093 for node: 10.22.8.182:49818 2018-01-05T11:29:09,080 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000093 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:09,080 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000093, launchTime=1515180549080 2018-01-05T11:29:09,080 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_39_01_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000093, dagName=select count(*) from tab a left oute...b.key (Stage-1), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{a,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{Reducer 2,org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput,1}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180549022, firstAttemptStartTime=1515180549077, currentAttemptStartTime=1515180549080} 2018-01-05T11:29:09,080 INFO [IPC Server handler 1 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000093, QueryId=djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827-39, DagId=39, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180549080 2018-01-05T11:29:09,081 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_39_01_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000093] on NM: [10.22.8.182:49818] 2018-01-05T11:29:09,081 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:09,081 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:09,081 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_39_01_000000_0, startTime=1515180549080, containerId=container_111101111_0001_00_000093, nodeId=10.22.8.182:49818 2018-01-05T11:29:09,081 DEBUG [IPC Server handler 1 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=39} with the ShuffleHandler 2018-01-05T11:29:09,092 INFO [IPC Server handler 1 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:29:09,092 DEBUG [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/39] 2018-01-05T11:29:09,092 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:29:09,092 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_39_01_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:09,092 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_39_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180549077, dagStartTime=1515180549022, withinDagPriority=14, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:29:09,092 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_39_01_000000_0 : ACCEPTED 2018-01-05T11:29:09,092 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_39_01_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:29:09,092 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_39_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180549077, dagStartTime=1515180549022, withinDagPriority=14, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:29:09,092 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_39_01_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:09,092 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_39_01_000000_0 to pre-emption queue 2018-01-05T11:29:09,092 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_39_01_000000_0: true 2018-01-05T11:29:09,093 INFO [TaskCommunicator # 1] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_39_01_000000_0 2018-01-05T11:29:09,093 INFO [TaskCommunicator # 1] GuaranteedTasks: Received response for attempt_1515180518813_0001_39_01_000000_0, true 2018-01-05T11:29:09,093 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: taskOwner hashCode:1230345669 2018-01-05T11:29:09,094 INFO [Task-Executor-2] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : select count(*) from tab a left oute...b.key (Stage-1), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_39_01_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=a, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=Reducer 2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ] 2018-01-05T11:29:09,094 INFO [Task-Executor-2] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1024458752, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:09,094 INFO [Task-Executor-2] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_39_01_000000_0 vertexName: Map 1 2018-01-05T11:29:09,094 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000093, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_39_01_000000_0, eventCount=1 } 2018-01-05T11:29:09,094 DEBUG [TezTR-518813_1_39_1_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_39 queryId: djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827 instanceType: TaskRunner2Callable 2018-01-05T11:29:09,094 INFO [TezTR-518813_1_39_1_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_39_01_000000_0 2018-01-05T11:29:09,095 DEBUG [TezTR-518813_1_39_1_0_0] log.PerfLogger: 2018-01-05T11:29:09,095 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:09,096 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:09,097 INFO [IPC Server handler 6 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_39_01_000000_0 sent events: (0-1). 2018-01-05T11:29:09,097 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:29:09,097 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_39_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:29:09,100 INFO [I/O Setup 0 Initialize: {a}] input.MRInput: a using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:29:09,100 INFO [I/O Setup 0 Initialize: {a}] input.MRInputLegacy: a MRInputLegacy deferring initialization 2018-01-05T11:29:09,100 DEBUG [TezTR-518813_1_39_1_0_0] log.PerfLogger: 2018-01-05T11:29:09,100 INFO [TezTR-518813_1_39_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:09,100 INFO [TezTR-518813_1_39_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:09,100 INFO [TezTR-518813_1_39_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:09,100 INFO [TezTR-518813_1_39_1_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:09,100 INFO [TezTR-518813_1_39_1_0_0] resources.MemoryDistributor: InitialRequests=[a:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [Reducer 2:OUTPUT:10485760:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput] 2018-01-05T11:29:09,100 INFO [TezTR-518813_1_39_1_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:09,100 INFO [TezTR-518813_1_39_1_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:09,100 INFO [TezTR-518813_1_39_1_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 10485760, TotalRequestedScaled: 9679163.076923078, TotalJVMHeap: 1024458752, TotalAvailable: 686387363, TotalRequested/TotalJVMHeap:0.01 2018-01-05T11:29:09,100 INFO [TezTR-518813_1_39_1_0_0] resources.MemoryDistributor: Allocations=[a:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:10485760:10485760] 2018-01-05T11:29:09,100 INFO [TezTR-518813_1_39_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:09,100 INFO [TezTR-518813_1_39_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: a being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:09,100 INFO [TezTR-518813_1_39_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:29:09,100 INFO [TezTR-518813_1_39_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:09,100 INFO [I/O Setup 1 Start: {a}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: a 2018-01-05T11:29:09,100 INFO [TezTR-518813_1_39_1_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:09,100 INFO [TezTR-518813_1_39_1_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_39_01_000000_0 2018-01-05T11:29:09,100 DEBUG [TezTR-518813_1_39_1_0_0] log.PerfLogger: 2018-01-05T11:29:09,100 DEBUG [TezTR-518813_1_39_1_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_39_01_000000_0_10282 2018-01-05T11:29:09,100 DEBUG [TezTR-518813_1_39_1_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:09,100 DEBUG [TezTR-518813_1_39_1_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_39_01_000000_0 2018-01-05T11:29:09,100 INFO [TezTR-518813_1_39_1_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827 2018-01-05T11:29:09,101 DEBUG [TezTR-518813_1_39_1_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:09,101 DEBUG [TezTR-518813_1_39_1_0_0] log.PerfLogger: 2018-01-05T11:29:09,101 DEBUG [TezTR-518813_1_39_1_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:09,101 DEBUG [TezTR-518813_1_39_1_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:09,101 DEBUG [TezTR-518813_1_39_1_0_0] mr.ObjectCache: Creating djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827_Map 1__MAP_PLAN__ 2018-01-05T11:29:09,101 INFO [TezTR-518813_1_39_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/b1618906-d66b-4401-932e-90b4d1f79b04/map.xml 2018-01-05T11:29:09,101 DEBUG [TezTR-518813_1_39_1_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/b1618906-d66b-4401-932e-90b4d1f79b04/map.xml 2018-01-05T11:29:09,101 DEBUG [TezTR-518813_1_39_1_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/b1618906-d66b-4401-932e-90b4d1f79b04/map.xml 2018-01-05T11:29:09,101 DEBUG [TezTR-518813_1_39_1_0_0] log.PerfLogger: 2018-01-05T11:29:09,101 INFO [TezTR-518813_1_39_1_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:09,101 DEBUG [TezTR-518813_1_39_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:09,101 DEBUG [TezTR-518813_1_39_1_0_0] log.PerfLogger: 2018-01-05T11:29:09,101 INFO [TezTR-518813_1_39_1_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.05KB 2018-01-05T11:29:09,102 DEBUG [TezTR-518813_1_39_1_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:09,102 INFO [TezTR-518813_1_39_1_0_0] tez.MapRecordProcessor: The input names are: [a] 2018-01-05T11:29:09,102 INFO [TezTR-518813_1_39_1_0_0] input.MRInput: Initialized MRInput: a 2018-01-05T11:29:09,102 DEBUG [TezTR-518813_1_39_1_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 2 2018-01-05T11:29:09,102 INFO [TezTR-518813_1_39_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/b1618906-d66b-4401-932e-90b4d1f79b04/map.xml 2018-01-05T11:29:09,102 DEBUG [TezTR-518813_1_39_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,102 DEBUG [TezTR-518813_1_39_1_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180546}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, numFiles=2, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, rawDataSize=22748, partition_columns.types=string, numRows=242, partition_columns=ds, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=1511, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08, transient_lastDdlTime=1515180547}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180546}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, numFiles=2, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, rawDataSize=22748, partition_columns.types=string, numRows=242, partition_columns=ds, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=1511, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08, transient_lastDdlTime=1515180547}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat]} 2018-01-05T11:29:09,103 INFO [TezTR-518813_1_39_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/b1618906-d66b-4401-932e-90b4d1f79b04/map.xml 2018-01-05T11:29:09,103 DEBUG [TezTR-518813_1_39_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,103 DEBUG [TezTR-518813_1_39_1_0_0] io.HiveInputFormat: Processing org.apache.hadoop.hive.ql.io.orc.OrcInputFormat 2018-01-05T11:29:09,103 DEBUG [TezTR-518813_1_39_1_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:29:09,103 INFO [TezTR-518813_1_39_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/b1618906-d66b-4401-932e-90b4d1f79b04/map.xml 2018-01-05T11:29:09,103 DEBUG [TezTR-518813_1_39_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,103 INFO [TezTR-518813_1_39_1_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_39_01_000000_0 2018-01-05T11:29:09,104 INFO [TezTR-518813_1_39_1_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:09,104 DEBUG [TezTR-518813_1_39_1_0_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:09,104 DEBUG [TezTR-518813_1_39_1_0_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:09,104 DEBUG [TezTR-518813_1_39_1_0_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:09,104 DEBUG [TezTR-518813_1_39_1_0_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:09,104 DEBUG [TezTR-518813_1_39_1_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,104 DEBUG [TezTR-518813_1_39_1_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,104 INFO [TezTR-518813_1_39_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/b1618906-d66b-4401-932e-90b4d1f79b04/map.xml 2018-01-05T11:29:09,104 DEBUG [TezTR-518813_1_39_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,104 DEBUG [TezTR-518813_1_39_1_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0 2018-01-05T11:29:09,104 DEBUG [IO-Elevator-Thread-2] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_39 queryId: djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827 instanceType: OrcEncodedDataReader 2018-01-05T11:29:09,104 INFO [TezTR-518813_1_39_1_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0, start=3, length=583, isOriginal=true, fileLength=767, hasFooter=false, hasBase=true, deltas=0], org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0, start=3, length=566, isOriginal=true, fileLength=744, hasFooter=false, hasBase=true, deltas=0]], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=1149} 2018-01-05T11:29:09,104 INFO [TezTR-518813_1_39_1_0_0] input.MRInput: a initialized RecordReader from event 2018-01-05T11:29:09,104 DEBUG [TezTR-518813_1_39_1_0_0] tez.MapRecordProcessor: Starting Output: Reducer 2 2018-01-05T11:29:09,104 INFO [IO-Elevator-Thread-2] LlapIoImpl: Processing data for file [-5122461581150239451, 1515180546000, 767]: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0 2018-01-05T11:29:09,105 DEBUG [IO-Elevator-Thread-2] LlapIoOrc: FileSplit {3, 583}; stripes {3, 583}, 2018-01-05T11:29:09,105 DEBUG [IO-Elevator-Thread-2] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,105 DEBUG [IO-Elevator-Thread-2] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,105 INFO [TezTR-518813_1_39_1_0_0] impl.ExternalSorter: Reducer 2 using: memoryMb=10, keySerializerClass=class org.apache.hadoop.hive.ql.io.HiveKey, valueSerializerClass=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization$TezBytesWritableSerializer@217275ac, comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator@55282ebd, partitioner=org.apache.tez.mapreduce.partition.MRPartitioner, serialization=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.hadoop.io.serializer.WritableSerialization, org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization, org.apache.hadoop.io.serializer.avro.AvroReflectSerialization, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:29:09,105 INFO [IO-Elevator-Thread-2] impl.OrcCodecPool: Got brand-new codec ZLIB 2018-01-05T11:29:09,105 INFO [TezTR-518813_1_39_1_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 2: , UsingHashComparator=true 2018-01-05T11:29:09,105 INFO [IO-Elevator-Thread-2] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [-5122461581150239451, 1515180546000, 767]): [{range start: 0 end: 38}] 2018-01-05T11:29:09,105 INFO [IO-Elevator-Thread-2] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [-5122461581150239451, 1515180546000, 767], base offset 3): [{start: 0 end: 11 cache buffer: 0x317f0390(1)}, {start: 11 end: 38 cache buffer: 0x4252621d(1)}] 2018-01-05T11:29:09,106 INFO [IO-Elevator-Thread-2] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [-5122461581150239451, 1515180546000, 767]): [{range start: 76 end: 230}] 2018-01-05T11:29:09,106 INFO [IO-Elevator-Thread-2] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [-5122461581150239451, 1515180546000, 767], base offset 3): [{start: 76 end: 230 cache buffer: 0x112f2b3b(1)}] 2018-01-05T11:29:09,106 DEBUG [IO-Elevator-Thread-2] encoded.EncodedTreeReaderFactory: Column at 1 2:string has no data 2018-01-05T11:29:09,106 DEBUG [IO-Elevator-Thread-2] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: int streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 164 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:29:09,106 DEBUG [IO-Elevator-Thread-2] LlapIoOrc: Done with decode 2018-01-05T11:29:09,106 DEBUG [IO-Elevator-Thread-2] LlapIoImpl: setDone called; closed false, done false, err null, pending 1 2018-01-05T11:29:09,106 DEBUG [IO-Elevator-Thread-2] impl.StatsRecordingThreadPool: Updated stats: instance: OrcEncodedDataReader thread name: IO-Elevator-Thread-2 thread id: 2076 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:09,107 INFO [TezTR-518813_1_39_1_0_0] impl.PipelinedSorter: Newly allocated block size=10485760, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=10485760, total=10485760 2018-01-05T11:29:09,107 INFO [TezTR-518813_1_39_1_0_0] impl.PipelinedSorter: Pre allocating rest of memory buffers upfront 2018-01-05T11:29:09,107 INFO [TezTR-518813_1_39_1_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 2: , UsingHashComparator=true#blocks=1, maxMemUsage=10485760, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=10485760, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=10 2018-01-05T11:29:09,107 INFO [TezTR-518813_1_39_1_0_0] impl.PipelinedSorter: Reducer 2: reserved.remaining()=10485760, reserved.metasize=5242880 2018-01-05T11:29:09,107 INFO [TezTR-518813_1_39_1_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:29:09,107 INFO [TezTR-518813_1_39_1_0_0] vector.VectorMapOperator: Initializing operator MAP[0] 2018-01-05T11:29:09,107 DEBUG [TezTR-518813_1_39_1_0_0] vector.VectorMapOperator: Initialization Done 0 MAP 2018-01-05T11:29:09,107 DEBUG [TezTR-518813_1_39_1_0_0] vector.VectorMapOperator: Operator 0 MAP initialized 2018-01-05T11:29:09,107 DEBUG [TezTR-518813_1_39_1_0_0] vector.VectorMapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:29:09,107 INFO [TezTR-518813_1_39_1_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:29:09,107 INFO [TezTR-518813_1_39_1_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:09,107 DEBUG [TezTR-518813_1_39_1_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:29:09,107 INFO [TezTR-518813_1_39_1_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:29:09,107 DEBUG [TezTR-518813_1_39_1_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:29:09,107 DEBUG [TezTR-518813_1_39_1_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:29:09,107 DEBUG [TezTR-518813_1_39_1_0_0] exec.TableScanOperator: Initializing children of 0 TS 2018-01-05T11:29:09,107 DEBUG [TezTR-518813_1_39_1_0_0] vector.VectorSelectOperator: Initializing child 21 SEL 2018-01-05T11:29:09,107 INFO [TezTR-518813_1_39_1_0_0] vector.VectorSelectOperator: Initializing operator SEL[21] 2018-01-05T11:29:09,107 DEBUG [TezTR-518813_1_39_1_0_0] vector.VectorSelectOperator: Initialization Done 21 SEL 2018-01-05T11:29:09,107 DEBUG [TezTR-518813_1_39_1_0_0] vector.VectorSelectOperator: Operator 21 SEL initialized 2018-01-05T11:29:09,107 DEBUG [TezTR-518813_1_39_1_0_0] vector.VectorSelectOperator: Initializing children of 21 SEL 2018-01-05T11:29:09,107 DEBUG [TezTR-518813_1_39_1_0_0] reducesink.VectorReduceSinkLongOperator: Initializing child 22 RS 2018-01-05T11:29:09,107 INFO [TezTR-518813_1_39_1_0_0] reducesink.VectorReduceSinkLongOperator: Initializing operator RS[22] 2018-01-05T11:29:09,107 INFO [TezTR-518813_1_39_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/b1618906-d66b-4401-932e-90b4d1f79b04/map.xml 2018-01-05T11:29:09,107 DEBUG [TezTR-518813_1_39_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,107 INFO [TezTR-518813_1_39_1_0_0] reducesink.VectorReduceSinkCommonOperator: Using tag = 0 2018-01-05T11:29:09,107 DEBUG [TezTR-518813_1_39_1_0_0] reducesink.VectorReduceSinkLongOperator: Initialization Done 22 RS 2018-01-05T11:29:09,107 DEBUG [TezTR-518813_1_39_1_0_0] reducesink.VectorReduceSinkLongOperator: Operator 22 RS initialized 2018-01-05T11:29:09,107 DEBUG [TezTR-518813_1_39_1_0_0] reducesink.VectorReduceSinkLongOperator: Initialization Done 22 RS done is reset. 2018-01-05T11:29:09,107 DEBUG [TezTR-518813_1_39_1_0_0] vector.VectorSelectOperator: Initialization Done 21 SEL done is reset. 2018-01-05T11:29:09,107 DEBUG [TezTR-518813_1_39_1_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:29:09,107 INFO [TezTR-518813_1_39_1_0_0] exec.OperatorUtils: Setting output collector: RS[22] --> Reducer 2 2018-01-05T11:29:09,107 DEBUG [TezTR-518813_1_39_1_0_0] log.PerfLogger: 2018-01-05T11:29:09,107 INFO [TezTR-518813_1_39_1_0_0] reducesink.VectorReduceSinkCommonOperator: RS[22]: records written - 1 2018-01-05T11:29:09,107 INFO [TezTR-518813_1_39_1_0_0] reducesink.VectorReduceSinkCommonOperator: RS[22]: records written - 10 2018-01-05T11:29:09,107 INFO [TezTR-518813_1_39_1_0_0] reducesink.VectorReduceSinkCommonOperator: RS[22]: records written - 100 2018-01-05T11:29:09,107 INFO [TezTR-518813_1_39_1_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:09,107 INFO [TezTR-518813_1_39_1_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, warehouse.tab, file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0 ([-5122461581150239451, 1515180546000, 767]), 0,1]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=1, NUM_ERRORS=0, ROWS_EMITTED=120, METADATA_CACHE_HIT=2, METADATA_CACHE_MISS=0, CACHE_HIT_BYTES=192, CACHE_MISS_BYTES=0, ALLOCATED_BYTES=0, ALLOCATED_USED_BYTES=0, TOTAL_IO_TIME_NS=1819740, DECODE_TIME_NS=250179, HDFS_TIME_NS=45453, CONSUMER_TIME_NS=166223 ] 2018-01-05T11:29:09,107 DEBUG [TezTR-518813_1_39_1_0_0] encoded.OrcEncodedDataReader: Encoded reader is being stopped 2018-01-05T11:29:09,107 DEBUG [TezTR-518813_1_39_1_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 1 of 2 2018-01-05T11:29:09,107 DEBUG [TezTR-518813_1_39_1_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180546}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, numFiles=2, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, rawDataSize=22748, partition_columns.types=string, numRows=242, partition_columns=ds, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=1511, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08, transient_lastDdlTime=1515180547}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180546}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, numFiles=2, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, rawDataSize=22748, partition_columns.types=string, numRows=242, partition_columns=ds, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=1511, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08, transient_lastDdlTime=1515180547}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat]} 2018-01-05T11:29:09,108 INFO [TezTR-518813_1_39_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/b1618906-d66b-4401-932e-90b4d1f79b04/map.xml 2018-01-05T11:29:09,108 DEBUG [TezTR-518813_1_39_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,108 DEBUG [TezTR-518813_1_39_1_0_0] io.HiveInputFormat: Processing org.apache.hadoop.hive.ql.io.orc.OrcInputFormat 2018-01-05T11:29:09,109 DEBUG [TezTR-518813_1_39_1_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:29:09,109 INFO [TezTR-518813_1_39_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/b1618906-d66b-4401-932e-90b4d1f79b04/map.xml 2018-01-05T11:29:09,109 DEBUG [TezTR-518813_1_39_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,109 INFO [TezTR-518813_1_39_1_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_39_01_000000_0 2018-01-05T11:29:09,109 INFO [TezTR-518813_1_39_1_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:09,109 DEBUG [TezTR-518813_1_39_1_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,109 DEBUG [TezTR-518813_1_39_1_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,109 INFO [TezTR-518813_1_39_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/b1618906-d66b-4401-932e-90b4d1f79b04/map.xml 2018-01-05T11:29:09,109 DEBUG [TezTR-518813_1_39_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,109 DEBUG [TezTR-518813_1_39_1_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0 2018-01-05T11:29:09,109 DEBUG [IO-Elevator-Thread-3] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_39 queryId: djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827 instanceType: OrcEncodedDataReader 2018-01-05T11:29:09,109 INFO [IO-Elevator-Thread-3] LlapIoImpl: Processing data for file [-5122461581151735180, 1515180546000, 744]: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0 2018-01-05T11:29:09,109 DEBUG [IO-Elevator-Thread-3] LlapIoOrc: FileSplit {3, 566}; stripes {3, 566}, 2018-01-05T11:29:09,109 DEBUG [IO-Elevator-Thread-3] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,109 DEBUG [IO-Elevator-Thread-3] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,109 INFO [IO-Elevator-Thread-3] impl.OrcCodecPool: Got brand-new codec ZLIB 2018-01-05T11:29:09,109 INFO [IO-Elevator-Thread-3] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [-5122461581151735180, 1515180546000, 744]): [{range start: 0 end: 38}] 2018-01-05T11:29:09,109 INFO [IO-Elevator-Thread-3] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [-5122461581151735180, 1515180546000, 744], base offset 3): [{start: 0 end: 11 cache buffer: 0x4d1ed212(1)}, {start: 11 end: 38 cache buffer: 0x6497a38f(1)}] 2018-01-05T11:29:09,109 INFO [IO-Elevator-Thread-3] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [-5122461581151735180, 1515180546000, 744]): [{range start: 73 end: 204}] 2018-01-05T11:29:09,110 INFO [IO-Elevator-Thread-3] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [-5122461581151735180, 1515180546000, 744], base offset 3): [{start: 73 end: 204 cache buffer: 0x3e73bafa(1)}] 2018-01-05T11:29:09,110 DEBUG [IO-Elevator-Thread-3] encoded.EncodedTreeReaderFactory: Column at 1 2:string has no data 2018-01-05T11:29:09,110 DEBUG [IO-Elevator-Thread-3] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: int streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 136 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:29:09,110 DEBUG [IO-Elevator-Thread-3] LlapIoOrc: Done with decode 2018-01-05T11:29:09,110 DEBUG [IO-Elevator-Thread-3] LlapIoImpl: setDone called; closed false, done false, err null, pending 0 2018-01-05T11:29:09,110 DEBUG [IO-Elevator-Thread-3] impl.StatsRecordingThreadPool: Updated stats: instance: OrcEncodedDataReader thread name: IO-Elevator-Thread-3 thread id: 2077 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:09,110 INFO [TezTR-518813_1_39_1_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:09,110 INFO [TezTR-518813_1_39_1_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, warehouse.tab, file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0 ([-5122461581151735180, 1515180546000, 744]), 0,1]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=1, NUM_ERRORS=0, ROWS_EMITTED=122, METADATA_CACHE_HIT=2, METADATA_CACHE_MISS=0, CACHE_HIT_BYTES=169, CACHE_MISS_BYTES=0, ALLOCATED_BYTES=0, ALLOCATED_USED_BYTES=0, TOTAL_IO_TIME_NS=659370, DECODE_TIME_NS=139636, HDFS_TIME_NS=27703, CONSUMER_TIME_NS=568042 ] 2018-01-05T11:29:09,110 DEBUG [TezTR-518813_1_39_1_0_0] encoded.OrcEncodedDataReader: Encoded reader is being stopped 2018-01-05T11:29:09,110 DEBUG [TezTR-518813_1_39_1_0_0] log.PerfLogger: 2018-01-05T11:29:09,110 DEBUG [TezTR-518813_1_39_1_0_0] mr.ObjectCache: djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:29:09,110 DEBUG [TezTR-518813_1_39_1_0_0] vector.VectorMapOperator: close called for operator MAP[0] 2018-01-05T11:29:09,110 INFO [TezTR-518813_1_39_1_0_0] vector.VectorMapOperator: Closing operator MAP[0] 2018-01-05T11:29:09,110 INFO [TezTR-518813_1_39_1_0_0] vector.VectorMapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:242, 2018-01-05T11:29:09,110 DEBUG [TezTR-518813_1_39_1_0_0] vector.VectorMapOperator: Closing child = TS[0] 2018-01-05T11:29:09,110 DEBUG [TezTR-518813_1_39_1_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:29:09,110 INFO [TezTR-518813_1_39_1_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:29:09,110 DEBUG [TezTR-518813_1_39_1_0_0] exec.TableScanOperator: Closing child = SEL[21] 2018-01-05T11:29:09,110 DEBUG [TezTR-518813_1_39_1_0_0] vector.VectorSelectOperator: close called for operator SEL[21] 2018-01-05T11:29:09,110 DEBUG [TezTR-518813_1_39_1_0_0] vector.VectorSelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:09,110 INFO [TezTR-518813_1_39_1_0_0] vector.VectorSelectOperator: Closing operator SEL[21] 2018-01-05T11:29:09,110 DEBUG [TezTR-518813_1_39_1_0_0] vector.VectorSelectOperator: Closing child = RS[22] 2018-01-05T11:29:09,110 DEBUG [TezTR-518813_1_39_1_0_0] reducesink.VectorReduceSinkLongOperator: close called for operator RS[22] 2018-01-05T11:29:09,110 DEBUG [TezTR-518813_1_39_1_0_0] reducesink.VectorReduceSinkLongOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:09,110 INFO [TezTR-518813_1_39_1_0_0] reducesink.VectorReduceSinkLongOperator: Closing operator RS[22] 2018-01-05T11:29:09,110 INFO [TezTR-518813_1_39_1_0_0] reducesink.VectorReduceSinkCommonOperator: RS[22]: records written - 242 2018-01-05T11:29:09,110 INFO [TezTR-518813_1_39_1_0_0] reducesink.VectorReduceSinkLongOperator: RECORDS_OUT_INTERMEDIATE_Map_1:242, 2018-01-05T11:29:09,110 DEBUG [TezTR-518813_1_39_1_0_0] reducesink.VectorReduceSinkLongOperator: 22 Close done 2018-01-05T11:29:09,110 DEBUG [TezTR-518813_1_39_1_0_0] vector.VectorSelectOperator: 21 Close done 2018-01-05T11:29:09,110 DEBUG [TezTR-518813_1_39_1_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:29:09,110 DEBUG [TezTR-518813_1_39_1_0_0] vector.VectorMapOperator: 0 Close done 2018-01-05T11:29:09,111 INFO [TezTR-518813_1_39_1_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_39_01_000000_0 2018-01-05T11:29:09,111 INFO [TezTR-518813_1_39_1_0_0] impl.PipelinedSorter: Reducer 2: Starting flush of map output 2018-01-05T11:29:09,111 INFO [TezTR-518813_1_39_1_0_0] impl.PipelinedSorter: Reducer 2: Span0.length = 242, perItem = 6 2018-01-05T11:29:09,111 INFO [TezTR-518813_1_39_1_0_0] impl.PipelinedSorter: Reducer 2: done sorting span=0, length=242, time=0 2018-01-05T11:29:09,111 INFO [TezTR-518813_1_39_1_0_0] impl.PipelinedSorter: Reducer 2: Heap = SpanIterator<0:241> (span=Span[5242880,1452]), 2018-01-05T11:29:09,112 DEBUG [ORC_GET_SPLITS #6] io.AcidUtils: in directory file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 base = null deltas = 0 2018-01-05T11:29:09,112 DEBUG [InputInitializer {Map 4} #0] orc.OrcInputFormat: Split strategy: ETLSplitStrategy strategy for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:09,112 DEBUG [ORC_GET_SPLITS #1] orc.LocalCache: Serialized tail cached for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0 2018-01-05T11:29:09,112 DEBUG [ORC_GET_SPLITS #1] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,112 DEBUG [ORC_GET_SPLITS #1] orc.LocalCache: Serialized tail cached for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0 2018-01-05T11:29:09,112 DEBUG [ORC_GET_SPLITS #1] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,113 DEBUG [ORC_GET_SPLITS #1] orc.LocalCache: Serialized tail cached for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0 2018-01-05T11:29:09,113 DEBUG [ORC_GET_SPLITS #1] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,113 DEBUG [ORC_GET_SPLITS #1] orc.LocalCache: Serialized tail cached for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0 2018-01-05T11:29:09,113 DEBUG [ORC_GET_SPLITS #1] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,113 DEBUG [ORC_GET_SPLITS #1] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,113 DEBUG [ORC_GET_SPLITS #3] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,113 INFO [ORC_GET_SPLITS #5] impl.OrcCodecPool: Got brand-new codec ZLIB 2018-01-05T11:29:09,113 INFO [ORC_GET_SPLITS #4] impl.OrcCodecPool: Got brand-new codec ZLIB 2018-01-05T11:29:09,113 DEBUG [ORC_GET_SPLITS #3] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:29:09,113 DEBUG [ORC_GET_SPLITS #5] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:29:09,113 DEBUG [ORC_GET_SPLITS #4] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:29:09,113 DEBUG [ORC_GET_SPLITS #1] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:29:09,113 INFO [InputInitializer {Map 4} #0] orc.OrcInputFormat: FooterCacheHitRatio: 4/4 2018-01-05T11:29:09,113 DEBUG [InputInitializer {Map 4} #0] orc.OrcInputFormat: OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0, start=3, length=574, isOriginal=true, fileLength=752, hasFooter=false, hasBase=true, deltas=0] projected_columns_uncompressed_size: 363 2018-01-05T11:29:09,113 DEBUG [InputInitializer {Map 4} #0] orc.OrcInputFormat: OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0, start=3, length=641, isOriginal=true, fileLength=827, hasFooter=false, hasBase=true, deltas=0] projected_columns_uncompressed_size: 427 2018-01-05T11:29:09,113 DEBUG [InputInitializer {Map 4} #0] orc.OrcInputFormat: OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0, start=3, length=641, isOriginal=true, fileLength=823, hasFooter=false, hasBase=true, deltas=0] projected_columns_uncompressed_size: 398 2018-01-05T11:29:09,113 DEBUG [InputInitializer {Map 4} #0] orc.OrcInputFormat: OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0, start=3, length=586, isOriginal=true, fileLength=770, hasFooter=false, hasBase=true, deltas=0] projected_columns_uncompressed_size: 350 2018-01-05T11:29:09,113 DEBUG [InputInitializer {Map 4} #0] orc.OrcInputFormat: getSplits finished 2018-01-05T11:29:09,113 INFO [InputInitializer {Map 4} #0] io.HiveInputFormat: number of splits 4 2018-01-05T11:29:09,113 DEBUG [InputInitializer {Map 4} #0] log.PerfLogger: 2018-01-05T11:29:09,113 INFO [InputInitializer {Map 4} #0] tez.HiveSplitGenerator: Number of input splits: 4. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:29:09,113 INFO [InputInitializer {Map 4} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/dcdeb337-7358-435a-9630-6d83a88e69f1/map.xml 2018-01-05T11:29:09,113 DEBUG [InputInitializer {Map 4} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/dcdeb337-7358-435a-9630-6d83a88e69f1/map.xml 2018-01-05T11:29:09,113 DEBUG [InputInitializer {Map 4} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/dcdeb337-7358-435a-9630-6d83a88e69f1/map.xml 2018-01-05T11:29:09,113 DEBUG [InputInitializer {Map 4} #0] log.PerfLogger: 2018-01-05T11:29:09,113 INFO [InputInitializer {Map 4} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:09,113 DEBUG [InputInitializer {Map 4} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:09,114 DEBUG [InputInitializer {Map 4} #0] log.PerfLogger: 2018-01-05T11:29:09,114 INFO [InputInitializer {Map 4} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 4 size: 3.07KB 2018-01-05T11:29:09,114 DEBUG [InputInitializer {Map 4} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0 to src new group? true 2018-01-05T11:29:09,114 DEBUG [InputInitializer {Map 4} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0 to src new group? false 2018-01-05T11:29:09,114 DEBUG [InputInitializer {Map 4} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0 to src new group? false 2018-01-05T11:29:09,114 DEBUG [InputInitializer {Map 4} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0 to src new group? false 2018-01-05T11:29:09,114 INFO [InputInitializer {Map 4} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:29:09,114 INFO [InputInitializer {Map 4} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:29:09,114 INFO [InputInitializer {Map 4} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:29:09,114 DEBUG [InputInitializer {Map 4} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 427 2018-01-05T11:29:09,114 DEBUG [InputInitializer {Map 4} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 398 2018-01-05T11:29:09,114 DEBUG [InputInitializer {Map 4} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 350 2018-01-05T11:29:09,114 DEBUG [InputInitializer {Map 4} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 363 2018-01-05T11:29:09,114 INFO [InputInitializer {Map 4} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 28 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 1538 Original splits: 4 2018-01-05T11:29:09,114 INFO [InputInitializer {Map 4} #0] grouper.TezSplitGrouper: Desired numSplits: 1 lengthPerGroup: 1538 numLocations: 1 numSplitsPerLocation: 4 numSplitsInGroup: 4 totalLength: 1538 numOriginalSplits: 4 . Grouping by length: true count: false nodeLocalOnly: false 2018-01-05T11:29:09,114 DEBUG [InputInitializer {Map 4} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 427 2018-01-05T11:29:09,114 DEBUG [InputInitializer {Map 4} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 398 2018-01-05T11:29:09,114 DEBUG [InputInitializer {Map 4} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 398 2018-01-05T11:29:09,114 DEBUG [InputInitializer {Map 4} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 350 2018-01-05T11:29:09,114 DEBUG [InputInitializer {Map 4} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 350 2018-01-05T11:29:09,114 DEBUG [InputInitializer {Map 4} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 363 2018-01-05T11:29:09,114 DEBUG [InputInitializer {Map 4} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 363 2018-01-05T11:29:09,114 INFO [InputInitializer {Map 4} #0] grouper.TezSplitGrouper: Number of splits desired: 1 created: 1 splitsProcessed: 4 2018-01-05T11:29:09,114 INFO [InputInitializer {Map 4} #0] tez.SplitGrouper: Original split count is 4 grouped split count is 1, for bucket: 1 2018-01-05T11:29:09,114 INFO [InputInitializer {Map 4} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:29:09,115 INFO [InputInitializer {Map 4} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: b on vertex vertex_1515180518813_0001_39_00 [Map 4] 2018-01-05T11:29:09,115 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_39_00 [Map 4] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:09,115 INFO [App Shared Pool - #0] impl.VertexImpl: Got updated RootInputsSpecs: {b=forAllWorkUnits=true, update=[1]} 2018-01-05T11:29:09,115 INFO [App Shared Pool - #0] impl.VertexImpl: Vertex vertex_1515180518813_0001_39_00 [Map 4] parallelism set to 1 2018-01-05T11:29:09,115 INFO [App Shared Pool - #0] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_39_00 [Map 4] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:09,115 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:VERTEX_INITIALIZED]: vertexName=Map 4, vertexId=vertex_1515180518813_0001_39_00, initRequestedTime=1515180549030, initedTime=1515180549115, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:09,115 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_39_00 [Map 4] with distanceFromRoot: 0 2018-01-05T11:29:09,115 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_39_00 [Map 4] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:29:09,115 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_39_00, startRequestedTime=1515180549052, startedTime=1515180549115 2018-01-05T11:29:09,115 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_39_00, reconfigureDoneTime=1515180549115, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:29:09,115 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_39_00 [Map 4] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:09,115 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_39_00 for vertex: vertex_1515180518813_0001_39_02 [Reducer 2] numStartedSources: 2 numSources: 2 2018-01-05T11:29:09,115 INFO [App Shared Pool - #0] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 4 2018-01-05T11:29:09,115 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_39_02 [Reducer 2] with distanceFromRoot: 1 2018-01-05T11:29:09,115 INFO [App Shared Pool - #0] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 4 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:29:09,115 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_39_02, startRequestedTime=1515180549115, startedTime=1515180549115 2018-01-05T11:29:09,115 INFO [App Shared Pool - #0] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_39_00 [Map 4] 2018-01-05T11:29:09,115 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_39_02 [Reducer 2] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:09,115 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_39_02 for vertex: vertex_1515180518813_0001_39_03 [Reducer 3] numStartedSources: 1 numSources: 1 2018-01-05T11:29:09,115 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_39_03 [Reducer 3] with distanceFromRoot: 2 2018-01-05T11:29:09,115 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_39_03, startRequestedTime=1515180549115, startedTime=1515180549115 2018-01-05T11:29:09,115 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: OnVertexStarted vertex: Reducer 2 with 0 source tasks and 1 pending tasks 2018-01-05T11:29:09,115 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_39_03 [Reducer 3] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:09,115 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:TASK_STARTED]: vertexName=Map 4, taskId=task_1515180518813_0001_39_00_000000, scheduledTime=1515180549115, launchTime=1515180549115 2018-01-05T11:29:09,116 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Map 4 in vertex: Reducer 2 numBipartiteSourceTasks: 1 2018-01-05T11:29:09,116 INFO [App Shared Pool - #1] vertexmanager.ShuffleVertexManagerBase: OnVertexStarted vertex: Reducer 3 with 0 source tasks and 1 pending tasks 2018-01-05T11:29:09,116 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_39_00 [Map 4] 2018-01-05T11:29:09,116 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Reducer 2 in vertex: Reducer 3 numBipartiteSourceTasks: 1 2018-01-05T11:29:09,116 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Map 1 in vertex: Reducer 2 numBipartiteSourceTasks: 2 2018-01-05T11:29:09,116 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_39_00_000000_0, priority=11, capability=, hosts=null 2018-01-05T11:29:09,116 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=11,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:09,116 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=11,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:09,116 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:09,116 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_39_00_000000_0 on hosts=null 2018-01-05T11:29:09,116 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_39_00_000000_0 on hosts=null 2018-01-05T11:29:09,116 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_39_00_000000_0 2018-01-05T11:29:09,116 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:09,117 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #94, task=TaskInfo{task=attempt_1515180518813_0001_39_00_000000_0, priority=11, startTime=0, containerId=null, uniqueId=93, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, to container=container_111101111_0001_00_000094 2018-01-05T11:29:09,117 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_39_00_000000_0; false 2018-01-05T11:29:09,117 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_39_00_000000_0, priority=11, startTime=311569626, containerId=container_111101111_0001_00_000094assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=2, ac=6, commF=false, disabled=false}, uniqueId=93, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:09,117 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000094 for node: 10.22.8.182:49818 2018-01-05T11:29:09,117 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000094, attempt_1515180518813_0001_39_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:09,117 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000094 for node: 10.22.8.182:49818 2018-01-05T11:29:09,117 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000094 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:09,117 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000094, launchTime=1515180549117 2018-01-05T11:29:09,118 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_39_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000094, dagName=select count(*) from tab a left oute...b.key (Stage-1), vertexName=Map 4, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{b,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{Reducer 2,org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput,1}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180549022, firstAttemptStartTime=1515180549116, currentAttemptStartTime=1515180549117} 2018-01-05T11:29:09,118 INFO [IPC Server handler 0 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000094, QueryId=djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827-39, DagId=39, VertexName=Map 4, TaskId=0, TaskAttemptId=0, SubmitTime=1515180549118 2018-01-05T11:29:09,118 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_39_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000094] on NM: [10.22.8.182:49818] 2018-01-05T11:29:09,118 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:09,118 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:09,118 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 4, taskAttemptId=attempt_1515180518813_0001_39_00_000000_0, startTime=1515180549117, containerId=container_111101111_0001_00_000094, nodeId=10.22.8.182:49818 2018-01-05T11:29:09,118 DEBUG [IPC Server handler 0 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=39} with the ShuffleHandler 2018-01-05T11:29:09,118 DEBUG [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/39] 2018-01-05T11:29:09,118 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=3, runningFragmentCount=1 2018-01-05T11:29:09,118 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_39_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:09,118 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_39_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180549116, dagStartTime=1515180549022, withinDagPriority=11, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:29:09,118 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_39_00_000000_0 : ACCEPTED 2018-01-05T11:29:09,118 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_39_00_000000_0, canFinish=true. Current state: preemptionQueueSize=1, numSlotsAvailable=3, waitQueueSize=1 2018-01-05T11:29:09,118 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_39_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180549116, dagStartTime=1515180549022, withinDagPriority=11, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:29:09,118 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_39_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:09,118 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_39_00_000000_0 to pre-emption queue 2018-01-05T11:29:09,118 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_39_00_000000_0: true 2018-01-05T11:29:09,118 INFO [TaskCommunicator # 3] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_39_00_000000_0 2018-01-05T11:29:09,118 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: taskOwner hashCode:1400700185 2018-01-05T11:29:09,118 INFO [TaskCommunicator # 3] GuaranteedTasks: Received response for attempt_1515180518813_0001_39_00_000000_0, true 2018-01-05T11:29:09,119 INFO [Task-Executor-1] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : select count(*) from tab a left oute...b.key (Stage-1), VertexName: Map 4, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_39_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=b, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=Reducer 2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ] 2018-01-05T11:29:09,119 INFO [Task-Executor-1] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1024458752, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:09,119 INFO [Task-Executor-1] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_39_00_000000_0 vertexName: Map 4 2018-01-05T11:29:09,119 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000094, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_39_00_000000_0, eventCount=1 } 2018-01-05T11:29:09,119 DEBUG [TezTR-518813_1_39_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_39 queryId: djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827 instanceType: TaskRunner2Callable 2018-01-05T11:29:09,119 INFO [TezTR-518813_1_39_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_39_00_000000_0 2018-01-05T11:29:09,120 DEBUG [TezTR-518813_1_39_0_0_0] log.PerfLogger: 2018-01-05T11:29:09,120 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:09,121 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:09,121 INFO [IPC Server handler 27 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_39_00_000000_0 sent events: (0-1). 2018-01-05T11:29:09,122 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:29:09,122 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_39_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:29:09,122 DEBUG [TezTR-518813_1_39_0_0_0] log.PerfLogger: 2018-01-05T11:29:09,122 INFO [TezTR-518813_1_39_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:09,122 INFO [TezTR-518813_1_39_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:09,122 INFO [TezTR-518813_1_39_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:09,123 INFO [I/O Setup 0 Initialize: {b}] input.MRInput: b using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:29:09,123 INFO [I/O Setup 0 Initialize: {b}] input.MRInputLegacy: b MRInputLegacy deferring initialization 2018-01-05T11:29:09,123 INFO [TezTR-518813_1_39_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:09,123 INFO [TezTR-518813_1_39_0_0_0] resources.MemoryDistributor: InitialRequests=[b:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [Reducer 2:OUTPUT:10485760:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput] 2018-01-05T11:29:09,123 INFO [TezTR-518813_1_39_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:09,123 INFO [TezTR-518813_1_39_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:09,123 INFO [TezTR-518813_1_39_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 10485760, TotalRequestedScaled: 9679163.076923078, TotalJVMHeap: 1024458752, TotalAvailable: 686387363, TotalRequested/TotalJVMHeap:0.01 2018-01-05T11:29:09,123 INFO [TezTR-518813_1_39_0_0_0] resources.MemoryDistributor: Allocations=[b:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:10485760:10485760] 2018-01-05T11:29:09,123 INFO [TezTR-518813_1_39_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:09,123 INFO [TezTR-518813_1_39_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: b being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:09,123 INFO [TezTR-518813_1_39_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:29:09,123 INFO [TezTR-518813_1_39_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:09,123 INFO [I/O Setup 1 Start: {b}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: b 2018-01-05T11:29:09,123 INFO [TezTR-518813_1_39_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:09,123 INFO [TezTR-518813_1_39_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_39_00_000000_0 2018-01-05T11:29:09,123 DEBUG [TezTR-518813_1_39_0_0_0] log.PerfLogger: 2018-01-05T11:29:09,123 DEBUG [TezTR-518813_1_39_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_39_00_000000_0_10285 2018-01-05T11:29:09,123 DEBUG [TezTR-518813_1_39_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:09,123 DEBUG [TezTR-518813_1_39_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_39_00_000000_0 2018-01-05T11:29:09,123 DEBUG [TezTR-518813_1_39_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:09,123 DEBUG [TezTR-518813_1_39_0_0_0] log.PerfLogger: 2018-01-05T11:29:09,123 DEBUG [TezTR-518813_1_39_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:09,123 DEBUG [TezTR-518813_1_39_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:09,123 DEBUG [TezTR-518813_1_39_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827_Map 4__MAP_PLAN__ 2018-01-05T11:29:09,123 INFO [TezTR-518813_1_39_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/dcdeb337-7358-435a-9630-6d83a88e69f1/map.xml 2018-01-05T11:29:09,123 DEBUG [TezTR-518813_1_39_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/dcdeb337-7358-435a-9630-6d83a88e69f1/map.xml 2018-01-05T11:29:09,123 DEBUG [TezTR-518813_1_39_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/dcdeb337-7358-435a-9630-6d83a88e69f1/map.xml 2018-01-05T11:29:09,123 DEBUG [TezTR-518813_1_39_0_0_0] log.PerfLogger: 2018-01-05T11:29:09,123 INFO [TezTR-518813_1_39_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:09,123 DEBUG [TezTR-518813_1_39_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:09,124 DEBUG [TezTR-518813_1_39_0_0_0] log.PerfLogger: 2018-01-05T11:29:09,124 INFO [TezTR-518813_1_39_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 4 size: 3.07KB 2018-01-05T11:29:09,125 INFO [TezTR-518813_1_39_1_0_0] impl.PipelinedSorter: Reducer 2: Spilling to /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/39/output/attempt_1515180518813_0001_39_01_000000_0_10284_0/file.out 2018-01-05T11:29:09,125 DEBUG [TezTR-518813_1_39_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:09,125 INFO [TezTR-518813_1_39_0_0_0] tez.MapRecordProcessor: The input names are: [b] 2018-01-05T11:29:09,125 INFO [TezTR-518813_1_39_0_0_0] input.MRInput: Initialized MRInput: b 2018-01-05T11:29:09,125 DEBUG [TezTR-518813_1_39_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 4 2018-01-05T11:29:09,125 INFO [TezTR-518813_1_39_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/dcdeb337-7358-435a-9630-6d83a88e69f1/map.xml 2018-01-05T11:29:09,125 DEBUG [TezTR-518813_1_39_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,125 DEBUG [TezTR-518813_1_39_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat]} 2018-01-05T11:29:09,125 INFO [TezTR-518813_1_39_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/dcdeb337-7358-435a-9630-6d83a88e69f1/map.xml 2018-01-05T11:29:09,125 DEBUG [TezTR-518813_1_39_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,125 DEBUG [TezTR-518813_1_39_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.hive.ql.io.orc.OrcInputFormat 2018-01-05T11:29:09,126 DEBUG [TezTR-518813_1_39_0_0_0] impl.LlapRecordReader: Initializing for input Map 4 2018-01-05T11:29:09,126 INFO [TezTR-518813_1_39_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/dcdeb337-7358-435a-9630-6d83a88e69f1/map.xml 2018-01-05T11:29:09,126 DEBUG [TezTR-518813_1_39_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,126 INFO [TezTR-518813_1_39_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_39_00_000000_0 2018-01-05T11:29:09,126 INFO [TezTR-518813_1_39_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:09,126 DEBUG [TezTR-518813_1_39_0_0_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:09,126 DEBUG [TezTR-518813_1_39_0_0_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:09,126 DEBUG [TezTR-518813_1_39_0_0_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:09,126 DEBUG [TezTR-518813_1_39_0_0_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:09,126 DEBUG [TezTR-518813_1_39_0_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,127 DEBUG [TezTR-518813_1_39_0_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,127 INFO [TezTR-518813_1_39_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/dcdeb337-7358-435a-9630-6d83a88e69f1/map.xml 2018-01-05T11:29:09,127 DEBUG [TezTR-518813_1_39_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,127 DEBUG [IO-Elevator-Thread-4] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_39 queryId: djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827 instanceType: OrcEncodedDataReader 2018-01-05T11:29:09,127 DEBUG [TezTR-518813_1_39_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0 2018-01-05T11:29:09,127 INFO [IO-Elevator-Thread-4] LlapIoImpl: Processing data for file [6462268725460710401, 1515180546000, 827]: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0 2018-01-05T11:29:09,127 INFO [TezTR-518813_1_39_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0, start=3, length=641, isOriginal=true, fileLength=827, hasFooter=false, hasBase=true, deltas=0], org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0, start=3, length=641, isOriginal=true, fileLength=823, hasFooter=false, hasBase=true, deltas=0], org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0, start=3, length=586, isOriginal=true, fileLength=770, hasFooter=false, hasBase=true, deltas=0], org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0, start=3, length=574, isOriginal=true, fileLength=752, hasFooter=false, hasBase=true, deltas=0]], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=2442} 2018-01-05T11:29:09,127 DEBUG [IO-Elevator-Thread-4] LlapIoOrc: FileSplit {3, 641}; stripes {3, 641}, 2018-01-05T11:29:09,127 INFO [TezTR-518813_1_39_0_0_0] input.MRInput: b initialized RecordReader from event 2018-01-05T11:29:09,127 DEBUG [TezTR-518813_1_39_0_0_0] tez.MapRecordProcessor: Starting Output: Reducer 2 2018-01-05T11:29:09,128 DEBUG [IO-Elevator-Thread-4] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,128 DEBUG [IO-Elevator-Thread-4] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,128 INFO [TezTR-518813_1_39_0_0_0] impl.ExternalSorter: Reducer 2 using: memoryMb=10, keySerializerClass=class org.apache.hadoop.hive.ql.io.HiveKey, valueSerializerClass=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization$TezBytesWritableSerializer@e98a788, comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator@16c25bdf, partitioner=org.apache.tez.mapreduce.partition.MRPartitioner, serialization=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.hadoop.io.serializer.WritableSerialization, org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization, org.apache.hadoop.io.serializer.avro.AvroReflectSerialization, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:29:09,128 DEBUG [IO-Elevator-Thread-4] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,128 INFO [TezTR-518813_1_39_0_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 2: , UsingHashComparator=true 2018-01-05T11:29:09,128 INFO [IO-Elevator-Thread-4] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725460710401, 1515180546000, 827]): [{range start: 0 end: 40}] 2018-01-05T11:29:09,128 INFO [IO-Elevator-Thread-4] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [6462268725460710401, 1515180546000, 827], base offset 3): [{start: 0 end: 12 cache buffer: 0x7302251(1)}, {start: 12 end: 40 cache buffer: 0x2cb64d2b(1)}] 2018-01-05T11:29:09,128 INFO [IO-Elevator-Thread-4] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725460710401, 1515180546000, 827]): [{range start: 79 end: 240}] 2018-01-05T11:29:09,129 INFO [IO-Elevator-Thread-4] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [6462268725460710401, 1515180546000, 827], base offset 3): [{start: 79 end: 240 cache buffer: 0x24272ac5(1)}] 2018-01-05T11:29:09,129 DEBUG [IO-Elevator-Thread-4] encoded.EncodedTreeReaderFactory: Column at 1 2:string has no data 2018-01-05T11:29:09,129 DEBUG [IO-Elevator-Thread-4] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: int streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 200 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:29:09,129 DEBUG [IO-Elevator-Thread-4] LlapIoOrc: Done with decode 2018-01-05T11:29:09,129 DEBUG [IO-Elevator-Thread-4] LlapIoImpl: setDone called; closed false, done false, err null, pending 1 2018-01-05T11:29:09,129 DEBUG [IO-Elevator-Thread-4] impl.StatsRecordingThreadPool: Updated stats: instance: OrcEncodedDataReader thread name: IO-Elevator-Thread-4 thread id: 2194 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:09,130 INFO [TezTR-518813_1_39_0_0_0] impl.PipelinedSorter: Newly allocated block size=10485760, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=10485760, total=10485760 2018-01-05T11:29:09,130 INFO [TezTR-518813_1_39_0_0_0] impl.PipelinedSorter: Pre allocating rest of memory buffers upfront 2018-01-05T11:29:09,130 INFO [TezTR-518813_1_39_0_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 2: , UsingHashComparator=true#blocks=1, maxMemUsage=10485760, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=10485760, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=10 2018-01-05T11:29:09,130 INFO [TezTR-518813_1_39_0_0_0] impl.PipelinedSorter: Reducer 2: reserved.remaining()=10485760, reserved.metasize=5242880 2018-01-05T11:29:09,130 INFO [TezTR-518813_1_39_0_0_0] tez.MapRecordProcessor: Main input name is Map 4 2018-01-05T11:29:09,130 INFO [TezTR-518813_1_39_0_0_0] vector.VectorMapOperator: Initializing operator MAP[0] 2018-01-05T11:29:09,130 DEBUG [TezTR-518813_1_39_0_0_0] vector.VectorMapOperator: Initialization Done 0 MAP 2018-01-05T11:29:09,130 DEBUG [TezTR-518813_1_39_0_0_0] vector.VectorMapOperator: Operator 0 MAP initialized 2018-01-05T11:29:09,130 DEBUG [TezTR-518813_1_39_0_0_0] vector.VectorMapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:29:09,130 INFO [TezTR-518813_1_39_0_0_0] tez.MapRecordProcessor: Id =0 Id =2 Id =23 Id =24 <\Children> Id = 23 null<\Parent> <\RS> <\Children> Id = 2 null<\Parent> <\SEL> <\Children> <\Parent> <\TS> <\Children> <\Parent> <\MAP> 2018-01-05T11:29:09,130 INFO [TezTR-518813_1_39_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:09,130 DEBUG [TezTR-518813_1_39_0_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:09,130 INFO [TezTR-518813_1_39_0_0_0] exec.TableScanOperator: Initializing operator TS[2] 2018-01-05T11:29:09,130 DEBUG [TezTR-518813_1_39_0_0_0] exec.TableScanOperator: Initialization Done 2 TS 2018-01-05T11:29:09,130 DEBUG [TezTR-518813_1_39_0_0_0] exec.TableScanOperator: Operator 2 TS initialized 2018-01-05T11:29:09,130 DEBUG [TezTR-518813_1_39_0_0_0] exec.TableScanOperator: Initializing children of 2 TS 2018-01-05T11:29:09,130 DEBUG [TezTR-518813_1_39_0_0_0] vector.VectorSelectOperator: Initializing child 23 SEL 2018-01-05T11:29:09,130 INFO [TezTR-518813_1_39_0_0_0] vector.VectorSelectOperator: Initializing operator SEL[23] 2018-01-05T11:29:09,130 DEBUG [TezTR-518813_1_39_0_0_0] vector.VectorSelectOperator: Initialization Done 23 SEL 2018-01-05T11:29:09,130 DEBUG [TezTR-518813_1_39_0_0_0] vector.VectorSelectOperator: Operator 23 SEL initialized 2018-01-05T11:29:09,130 DEBUG [TezTR-518813_1_39_0_0_0] vector.VectorSelectOperator: Initializing children of 23 SEL 2018-01-05T11:29:09,130 DEBUG [TezTR-518813_1_39_0_0_0] reducesink.VectorReduceSinkLongOperator: Initializing child 24 RS 2018-01-05T11:29:09,130 INFO [TezTR-518813_1_39_0_0_0] reducesink.VectorReduceSinkLongOperator: Initializing operator RS[24] 2018-01-05T11:29:09,130 INFO [TezTR-518813_1_39_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/dcdeb337-7358-435a-9630-6d83a88e69f1/map.xml 2018-01-05T11:29:09,130 DEBUG [TezTR-518813_1_39_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,130 INFO [TezTR-518813_1_39_0_0_0] reducesink.VectorReduceSinkCommonOperator: Using tag = 1 2018-01-05T11:29:09,130 DEBUG [TezTR-518813_1_39_0_0_0] reducesink.VectorReduceSinkLongOperator: Initialization Done 24 RS 2018-01-05T11:29:09,130 DEBUG [TezTR-518813_1_39_0_0_0] reducesink.VectorReduceSinkLongOperator: Operator 24 RS initialized 2018-01-05T11:29:09,130 DEBUG [TezTR-518813_1_39_0_0_0] reducesink.VectorReduceSinkLongOperator: Initialization Done 24 RS done is reset. 2018-01-05T11:29:09,130 DEBUG [TezTR-518813_1_39_0_0_0] vector.VectorSelectOperator: Initialization Done 23 SEL done is reset. 2018-01-05T11:29:09,130 DEBUG [TezTR-518813_1_39_0_0_0] exec.TableScanOperator: Initialization Done 2 TS done is reset. 2018-01-05T11:29:09,130 INFO [TezTR-518813_1_39_0_0_0] exec.OperatorUtils: Setting output collector: RS[24] --> Reducer 2 2018-01-05T11:29:09,130 DEBUG [TezTR-518813_1_39_0_0_0] log.PerfLogger: 2018-01-05T11:29:09,130 INFO [TezTR-518813_1_39_0_0_0] reducesink.VectorReduceSinkCommonOperator: RS[24]: records written - 1 2018-01-05T11:29:09,130 INFO [TezTR-518813_1_39_0_0_0] reducesink.VectorReduceSinkCommonOperator: RS[24]: records written - 10 2018-01-05T11:29:09,130 INFO [TezTR-518813_1_39_0_0_0] reducesink.VectorReduceSinkCommonOperator: RS[24]: records written - 100 2018-01-05T11:29:09,130 INFO [TezTR-518813_1_39_0_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:09,131 INFO [TezTR-518813_1_39_0_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, warehouse.tab_part, file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0 ([6462268725460710401, 1515180546000, 827]), 0,1]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=1, NUM_ERRORS=0, ROWS_EMITTED=138, METADATA_CACHE_HIT=2, METADATA_CACHE_MISS=0, CACHE_HIT_BYTES=201, CACHE_MISS_BYTES=0, ALLOCATED_BYTES=0, ALLOCATED_USED_BYTES=0, TOTAL_IO_TIME_NS=2100491, DECODE_TIME_NS=320850, HDFS_TIME_NS=464618, CONSUMER_TIME_NS=210158 ] 2018-01-05T11:29:09,131 DEBUG [TezTR-518813_1_39_0_0_0] encoded.OrcEncodedDataReader: Encoded reader is being stopped 2018-01-05T11:29:09,131 DEBUG [TezTR-518813_1_39_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 1 of 4 2018-01-05T11:29:09,131 DEBUG [TezTR-518813_1_39_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat]} 2018-01-05T11:29:09,131 INFO [TezTR-518813_1_39_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/dcdeb337-7358-435a-9630-6d83a88e69f1/map.xml 2018-01-05T11:29:09,131 DEBUG [TezTR-518813_1_39_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,131 DEBUG [TezTR-518813_1_39_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.hive.ql.io.orc.OrcInputFormat 2018-01-05T11:29:09,132 DEBUG [TezTR-518813_1_39_0_0_0] impl.LlapRecordReader: Initializing for input Map 4 2018-01-05T11:29:09,132 INFO [TezTR-518813_1_39_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/dcdeb337-7358-435a-9630-6d83a88e69f1/map.xml 2018-01-05T11:29:09,132 DEBUG [TezTR-518813_1_39_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,132 INFO [TezTR-518813_1_39_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_39_00_000000_0 2018-01-05T11:29:09,132 INFO [TezTR-518813_1_39_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:09,132 DEBUG [TezTR-518813_1_39_0_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,132 DEBUG [TezTR-518813_1_39_0_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,132 INFO [TezTR-518813_1_39_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/dcdeb337-7358-435a-9630-6d83a88e69f1/map.xml 2018-01-05T11:29:09,132 DEBUG [TezTR-518813_1_39_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,133 DEBUG [TezTR-518813_1_39_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0 2018-01-05T11:29:09,133 DEBUG [IO-Elevator-Thread-5] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_39 queryId: djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827 instanceType: OrcEncodedDataReader 2018-01-05T11:29:09,133 INFO [IO-Elevator-Thread-5] LlapIoImpl: Processing data for file [6462268725462206130, 1515180546000, 823]: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0 2018-01-05T11:29:09,133 DEBUG [IO-Elevator-Thread-5] LlapIoOrc: FileSplit {3, 641}; stripes {3, 641}, 2018-01-05T11:29:09,133 DEBUG [IO-Elevator-Thread-5] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,133 DEBUG [IO-Elevator-Thread-5] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,133 DEBUG [IO-Elevator-Thread-5] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,133 INFO [IO-Elevator-Thread-5] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725462206130, 1515180546000, 823]): [{range start: 0 end: 40}] 2018-01-05T11:29:09,133 INFO [IO-Elevator-Thread-5] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [6462268725462206130, 1515180546000, 823], base offset 3): [{start: 0 end: 12 cache buffer: 0x469c2dcf(1)}, {start: 12 end: 40 cache buffer: 0x2f556695(1)}] 2018-01-05T11:29:09,133 INFO [IO-Elevator-Thread-5] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725462206130, 1515180546000, 823]): [{range start: 76 end: 242}] 2018-01-05T11:29:09,133 INFO [IO-Elevator-Thread-5] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [6462268725462206130, 1515180546000, 823], base offset 3): [{start: 76 end: 242 cache buffer: 0x6e3a2cb3(1)}] 2018-01-05T11:29:09,133 DEBUG [IO-Elevator-Thread-5] encoded.EncodedTreeReaderFactory: Column at 1 2:string has no data 2018-01-05T11:29:09,133 DEBUG [IO-Elevator-Thread-5] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: int streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 200 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:29:09,133 DEBUG [IO-Elevator-Thread-5] LlapIoOrc: Done with decode 2018-01-05T11:29:09,134 DEBUG [IO-Elevator-Thread-5] LlapIoImpl: setDone called; closed false, done false, err null, pending 0 2018-01-05T11:29:09,134 DEBUG [IO-Elevator-Thread-5] impl.StatsRecordingThreadPool: Updated stats: instance: OrcEncodedDataReader thread name: IO-Elevator-Thread-5 thread id: 2196 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:09,134 INFO [TezTR-518813_1_39_0_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:09,134 INFO [TezTR-518813_1_39_0_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, warehouse.tab_part, file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0 ([6462268725462206130, 1515180546000, 823]), 0,1]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=1, NUM_ERRORS=0, ROWS_EMITTED=128, METADATA_CACHE_HIT=2, METADATA_CACHE_MISS=0, CACHE_HIT_BYTES=206, CACHE_MISS_BYTES=0, ALLOCATED_BYTES=0, ALLOCATED_USED_BYTES=0, TOTAL_IO_TIME_NS=887447, DECODE_TIME_NS=191079, HDFS_TIME_NS=59765, CONSUMER_TIME_NS=695221 ] 2018-01-05T11:29:09,134 DEBUG [TezTR-518813_1_39_0_0_0] encoded.OrcEncodedDataReader: Encoded reader is being stopped 2018-01-05T11:29:09,134 DEBUG [TezTR-518813_1_39_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 2 of 4 2018-01-05T11:29:09,134 DEBUG [TezTR-518813_1_39_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat]} 2018-01-05T11:29:09,134 INFO [TezTR-518813_1_39_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/dcdeb337-7358-435a-9630-6d83a88e69f1/map.xml 2018-01-05T11:29:09,134 DEBUG [TezTR-518813_1_39_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,135 DEBUG [TezTR-518813_1_39_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.hive.ql.io.orc.OrcInputFormat 2018-01-05T11:29:09,136 DEBUG [TezTR-518813_1_39_0_0_0] impl.LlapRecordReader: Initializing for input Map 4 2018-01-05T11:29:09,136 INFO [TezTR-518813_1_39_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/dcdeb337-7358-435a-9630-6d83a88e69f1/map.xml 2018-01-05T11:29:09,136 DEBUG [TezTR-518813_1_39_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,136 INFO [TezTR-518813_1_39_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_39_00_000000_0 2018-01-05T11:29:09,136 INFO [TezTR-518813_1_39_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:09,136 DEBUG [TezTR-518813_1_39_0_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,136 DEBUG [TezTR-518813_1_39_0_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,136 INFO [TezTR-518813_1_39_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/dcdeb337-7358-435a-9630-6d83a88e69f1/map.xml 2018-01-05T11:29:09,136 DEBUG [TezTR-518813_1_39_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,136 DEBUG [TezTR-518813_1_39_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0 2018-01-05T11:29:09,136 DEBUG [IO-Elevator-Thread-6] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_39 queryId: djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827 instanceType: OrcEncodedDataReader 2018-01-05T11:29:09,136 INFO [IO-Elevator-Thread-6] LlapIoImpl: Processing data for file [6462268725463701859, 1515180546000, 770]: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0 2018-01-05T11:29:09,136 DEBUG [IO-Elevator-Thread-6] LlapIoOrc: FileSplit {3, 586}; stripes {3, 586}, 2018-01-05T11:29:09,136 DEBUG [IO-Elevator-Thread-6] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,136 DEBUG [IO-Elevator-Thread-6] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,136 INFO [IO-Elevator-Thread-6] impl.OrcCodecPool: Got brand-new codec ZLIB 2018-01-05T11:29:09,136 INFO [IO-Elevator-Thread-6] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725463701859, 1515180546000, 770]): [{range start: 0 end: 38}] 2018-01-05T11:29:09,136 INFO [IO-Elevator-Thread-6] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [6462268725463701859, 1515180546000, 770], base offset 3): [{start: 0 end: 11 cache buffer: 0x5a722c41(1)}, {start: 11 end: 38 cache buffer: 0x7b40d4ac(1)}] 2018-01-05T11:29:09,137 INFO [IO-Elevator-Thread-6] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725463701859, 1515180546000, 770]): [{range start: 76 end: 223}] 2018-01-05T11:29:09,137 INFO [IO-Elevator-Thread-6] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [6462268725463701859, 1515180546000, 770], base offset 3): [{start: 76 end: 223 cache buffer: 0x1ddca8ea(1)}] 2018-01-05T11:29:09,137 DEBUG [IO-Elevator-Thread-6] encoded.EncodedTreeReaderFactory: Column at 1 2:string has no data 2018-01-05T11:29:09,137 DEBUG [IO-Elevator-Thread-6] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: int streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 171 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:29:09,137 DEBUG [IO-Elevator-Thread-6] LlapIoOrc: Done with decode 2018-01-05T11:29:09,137 DEBUG [IO-Elevator-Thread-6] LlapIoImpl: setDone called; closed false, done false, err null, pending 0 2018-01-05T11:29:09,137 DEBUG [IO-Elevator-Thread-6] impl.StatsRecordingThreadPool: Updated stats: instance: OrcEncodedDataReader thread name: IO-Elevator-Thread-6 thread id: 2198 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:09,137 INFO [TezTR-518813_1_39_0_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:09,137 INFO [TezTR-518813_1_39_0_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, warehouse.tab_part, file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0 ([6462268725463701859, 1515180546000, 770]), 0,1]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=1, NUM_ERRORS=0, ROWS_EMITTED=115, METADATA_CACHE_HIT=2, METADATA_CACHE_MISS=0, CACHE_HIT_BYTES=185, CACHE_MISS_BYTES=0, ALLOCATED_BYTES=0, ALLOCATED_USED_BYTES=0, TOTAL_IO_TIME_NS=533023, DECODE_TIME_NS=119646, HDFS_TIME_NS=23234, CONSUMER_TIME_NS=222681 ] 2018-01-05T11:29:09,137 DEBUG [TezTR-518813_1_39_0_0_0] encoded.OrcEncodedDataReader: Encoded reader is being stopped 2018-01-05T11:29:09,137 DEBUG [TezTR-518813_1_39_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 3 of 4 2018-01-05T11:29:09,137 DEBUG [TezTR-518813_1_39_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat]} 2018-01-05T11:29:09,137 INFO [TezTR-518813_1_39_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/dcdeb337-7358-435a-9630-6d83a88e69f1/map.xml 2018-01-05T11:29:09,137 DEBUG [TezTR-518813_1_39_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,137 DEBUG [TezTR-518813_1_39_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.hive.ql.io.orc.OrcInputFormat 2018-01-05T11:29:09,138 DEBUG [TezTR-518813_1_39_0_0_0] impl.LlapRecordReader: Initializing for input Map 4 2018-01-05T11:29:09,138 INFO [TezTR-518813_1_39_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/dcdeb337-7358-435a-9630-6d83a88e69f1/map.xml 2018-01-05T11:29:09,138 DEBUG [TezTR-518813_1_39_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,138 INFO [TezTR-518813_1_39_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_39_00_000000_0 2018-01-05T11:29:09,138 INFO [TezTR-518813_1_39_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:09,138 DEBUG [TezTR-518813_1_39_0_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,138 DEBUG [TezTR-518813_1_39_0_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,138 INFO [TezTR-518813_1_39_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/dcdeb337-7358-435a-9630-6d83a88e69f1/map.xml 2018-01-05T11:29:09,138 DEBUG [TezTR-518813_1_39_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,139 DEBUG [IO-Elevator-Thread-7] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_39 queryId: djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827 instanceType: OrcEncodedDataReader 2018-01-05T11:29:09,139 DEBUG [TezTR-518813_1_39_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0 2018-01-05T11:29:09,139 INFO [IO-Elevator-Thread-7] LlapIoImpl: Processing data for file [6462268725459214672, 1515180546000, 752]: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0 2018-01-05T11:29:09,139 DEBUG [IO-Elevator-Thread-7] LlapIoOrc: FileSplit {3, 574}; stripes {3, 574}, 2018-01-05T11:29:09,139 DEBUG [IO-Elevator-Thread-7] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,139 DEBUG [IO-Elevator-Thread-7] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,139 INFO [IO-Elevator-Thread-7] impl.OrcCodecPool: Got brand-new codec ZLIB 2018-01-05T11:29:09,139 INFO [IO-Elevator-Thread-7] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725459214672, 1515180546000, 752]): [{range start: 0 end: 38}] 2018-01-05T11:29:09,139 INFO [IO-Elevator-Thread-7] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [6462268725459214672, 1515180546000, 752], base offset 3): [{start: 0 end: 11 cache buffer: 0x1c0198da(1)}, {start: 11 end: 38 cache buffer: 0x55974329(1)}] 2018-01-05T11:29:09,139 INFO [IO-Elevator-Thread-7] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725459214672, 1515180546000, 752]): [{range start: 73 end: 208}] 2018-01-05T11:29:09,139 INFO [IO-Elevator-Thread-7] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [6462268725459214672, 1515180546000, 752], base offset 3): [{start: 73 end: 208 cache buffer: 0x21723a65(1)}] 2018-01-05T11:29:09,139 DEBUG [IO-Elevator-Thread-7] encoded.EncodedTreeReaderFactory: Column at 1 2:string has no data 2018-01-05T11:29:09,139 DEBUG [IO-Elevator-Thread-7] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: int streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 150 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:29:09,139 DEBUG [IO-Elevator-Thread-7] LlapIoOrc: Done with decode 2018-01-05T11:29:09,139 DEBUG [IO-Elevator-Thread-7] LlapIoImpl: setDone called; closed false, done false, err null, pending 0 2018-01-05T11:29:09,139 DEBUG [IO-Elevator-Thread-7] impl.StatsRecordingThreadPool: Updated stats: instance: OrcEncodedDataReader thread name: IO-Elevator-Thread-7 thread id: 2200 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:09,139 INFO [TezTR-518813_1_39_0_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:09,139 INFO [TezTR-518813_1_39_0_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, warehouse.tab_part, file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0 ([6462268725459214672, 1515180546000, 752]), 0,1]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=1, NUM_ERRORS=0, ROWS_EMITTED=119, METADATA_CACHE_HIT=2, METADATA_CACHE_MISS=0, CACHE_HIT_BYTES=173, CACHE_MISS_BYTES=0, ALLOCATED_BYTES=0, ALLOCATED_USED_BYTES=0, TOTAL_IO_TIME_NS=509642, DECODE_TIME_NS=127138, HDFS_TIME_NS=25624, CONSUMER_TIME_NS=143843 ] 2018-01-05T11:29:09,139 DEBUG [TezTR-518813_1_39_0_0_0] encoded.OrcEncodedDataReader: Encoded reader is being stopped 2018-01-05T11:29:09,139 DEBUG [TezTR-518813_1_39_1_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:09,139 DEBUG [TezTR-518813_1_39_0_0_0] log.PerfLogger: 2018-01-05T11:29:09,139 INFO [TezTR-518813_1_39_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:09,139 DEBUG [TezTR-518813_1_39_0_0_0] mr.ObjectCache: djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827_Map 4__MAP_PLAN__ no longer needed 2018-01-05T11:29:09,139 DEBUG [TezTR-518813_1_39_0_0_0] vector.VectorMapOperator: close called for operator MAP[0] 2018-01-05T11:29:09,139 INFO [TezTR-518813_1_39_0_0_0] vector.VectorMapOperator: Closing operator MAP[0] 2018-01-05T11:29:09,139 INFO [TezTR-518813_1_39_0_0_0] vector.VectorMapOperator: RECORDS_IN_Map_4:500, DESERIALIZE_ERRORS:0, 2018-01-05T11:29:09,139 DEBUG [TezTR-518813_1_39_0_0_0] vector.VectorMapOperator: Closing child = TS[2] 2018-01-05T11:29:09,139 DEBUG [TezTR-518813_1_39_0_0_0] exec.TableScanOperator: close called for operator TS[2] 2018-01-05T11:29:09,139 INFO [TezTR-518813_1_39_0_0_0] exec.TableScanOperator: Closing operator TS[2] 2018-01-05T11:29:09,139 DEBUG [TezTR-518813_1_39_0_0_0] exec.TableScanOperator: Closing child = SEL[23] 2018-01-05T11:29:09,139 DEBUG [TezTR-518813_1_39_0_0_0] vector.VectorSelectOperator: close called for operator SEL[23] 2018-01-05T11:29:09,139 DEBUG [TezTR-518813_1_39_0_0_0] vector.VectorSelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:09,139 INFO [TezTR-518813_1_39_0_0_0] vector.VectorSelectOperator: Closing operator SEL[23] 2018-01-05T11:29:09,139 DEBUG [TezTR-518813_1_39_0_0_0] vector.VectorSelectOperator: Closing child = RS[24] 2018-01-05T11:29:09,139 DEBUG [TezTR-518813_1_39_0_0_0] reducesink.VectorReduceSinkLongOperator: close called for operator RS[24] 2018-01-05T11:29:09,139 DEBUG [TezTR-518813_1_39_0_0_0] reducesink.VectorReduceSinkLongOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:09,139 INFO [TezTR-518813_1_39_0_0_0] reducesink.VectorReduceSinkLongOperator: Closing operator RS[24] 2018-01-05T11:29:09,139 INFO [TezTR-518813_1_39_0_0_0] reducesink.VectorReduceSinkCommonOperator: RS[24]: records written - 500 2018-01-05T11:29:09,139 INFO [TezTR-518813_1_39_1_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_39_01_000000_0, askedToStop=false 2018-01-05T11:29:09,139 INFO [TezTR-518813_1_39_0_0_0] reducesink.VectorReduceSinkLongOperator: RECORDS_OUT_INTERMEDIATE_Map_4:500, 2018-01-05T11:29:09,139 INFO [TezTR-518813_1_39_1_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_39_01_000000_0, stopRequested=false 2018-01-05T11:29:09,139 DEBUG [TezTR-518813_1_39_0_0_0] reducesink.VectorReduceSinkLongOperator: 24 Close done 2018-01-05T11:29:09,139 DEBUG [TezTR-518813_1_39_0_0_0] vector.VectorSelectOperator: 23 Close done 2018-01-05T11:29:09,139 DEBUG [TezTR-518813_1_39_0_0_0] exec.TableScanOperator: 2 Close done 2018-01-05T11:29:09,139 DEBUG [TezTR-518813_1_39_0_0_0] vector.VectorMapOperator: 0 Close done 2018-01-05T11:29:09,139 INFO [TezTR-518813_1_39_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_39_00_000000_0 2018-01-05T11:29:09,140 INFO [TezTR-518813_1_39_0_0_0] impl.PipelinedSorter: Reducer 2: Starting flush of map output 2018-01-05T11:29:09,140 INFO [TezTR-518813_1_39_0_0_0] impl.PipelinedSorter: Reducer 2: Span0.length = 500, perItem = 6 2018-01-05T11:29:09,140 INFO [TezTR-518813_1_39_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_39_01_000000_0: Counters: 31 [[File System Counters FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=0, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=242, INPUT_RECORDS_PROCESSED=2, INPUT_SPLIT_LENGTH_BYTES=1149, OUTPUT_RECORDS=242, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=1452, OUTPUT_BYTES_WITH_OVERHEAD=1447, OUTPUT_BYTES_PHYSICAL=1451, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=242, RECORDS_OUT_INTERMEDIATE_Map_1=242][org.apache.hadoop.hive.llap.counters.LlapIOCounters CACHE_HIT_BYTES=361, CACHE_MISS_BYTES=0, CONSUMER_TIME_NS=734265, DECODE_TIME_NS=389815, HDFS_TIME_NS=73156, METADATA_CACHE_HIT=4, NUM_DECODED_BATCHES=2, NUM_VECTOR_BATCHES=2, ROWS_EMITTED=242, SELECTED_ROWGROUPS=2, TOTAL_IO_TIME_NS=2479110]] 2018-01-05T11:29:09,140 DEBUG [TezTR-518813_1_39_1_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_39_1_0_0 thread id: 2674 scheme: file bytesRead: 64 bytesWritten: 1483 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:09,140 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_39_01_000000_0, isTaskDone=true 2018-01-05T11:29:09,140 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000093, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_39_01_000000_0, eventCount=4 } 2018-01-05T11:29:09,140 INFO [TezTR-518813_1_39_0_0_0] impl.PipelinedSorter: Reducer 2: done sorting span=0, length=500, time=0 2018-01-05T11:29:09,140 INFO [TezTR-518813_1_39_0_0_0] impl.PipelinedSorter: Reducer 2: Heap = SpanIterator<0:499> (span=Span[5242880,3000]), 2018-01-05T11:29:09,141 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_39_01_000000_0, creationTime=1515180549077, allocationTime=1515180549078, startTime=1515180549080, finishTime=1515180549140, timeTaken=60, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:09,141 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:29:09,141 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_39_01_000000_0 2018-01-05T11:29:09,141 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_39_01_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:09,141 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_39_01_000000_0; false 2018-01-05T11:29:09,141 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_39_01_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:09,141 INFO [Task-Executor-2] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_39_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:09,141 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_39_01_000000, startTime=1515180549080, finishTime=1515180549141, timeTaken=61, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_39_01_000000_0, diagnostics=, counters=Counters: 32, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=64, FILE_BYTES_WRITTEN=1483, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=242, INPUT_RECORDS_PROCESSED=2, INPUT_SPLIT_LENGTH_BYTES=1149, OUTPUT_RECORDS=242, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=1452, OUTPUT_BYTES_WITH_OVERHEAD=1447, OUTPUT_BYTES_PHYSICAL=1451, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=242, RECORDS_OUT_INTERMEDIATE_Map_1=242, org.apache.hadoop.hive.llap.counters.LlapIOCounters, CACHE_HIT_BYTES=361, CACHE_MISS_BYTES=0, CONSUMER_TIME_NS=734265, DECODE_TIME_NS=389815, HDFS_TIME_NS=73156, METADATA_CACHE_HIT=4, NUM_DECODED_BATCHES=2, NUM_VECTOR_BATCHES=2, ROWS_EMITTED=242, SELECTED_ROWGROUPS=2, TOTAL_IO_TIME_NS=2479110 2018-01-05T11:29:09,141 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:09,141 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:09,141 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_39_01 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:09,141 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_39_01 [Map 1] 2018-01-05T11:29:09,141 INFO [Task-Executor-2] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_39_01_000000_0 2018-01-05T11:29:09,141 INFO [Task-Executor-2] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000093=48 2018-01-05T11:29:09,141 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_39_01_000000_0: true 2018-01-05T11:29:09,141 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_39_01_000000_0 2018-01-05T11:29:09,141 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_39_01, initRequestedTime=1515180549022, initedTime=1515180549076, startRequestedTime=1515180549052, startedTime=1515180549076, finishTime=1515180549141, timeTaken=65, status=SUCCEEDED, diagnostics=, counters=Counters: 32, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=64, FILE_BYTES_WRITTEN=1483, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=242, INPUT_RECORDS_PROCESSED=2, INPUT_SPLIT_LENGTH_BYTES=1149, OUTPUT_RECORDS=242, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=1452, OUTPUT_BYTES_WITH_OVERHEAD=1447, OUTPUT_BYTES_PHYSICAL=1451, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=242, RECORDS_OUT_INTERMEDIATE_Map_1=242, org.apache.hadoop.hive.llap.counters.LlapIOCounters, CACHE_HIT_BYTES=361, CACHE_MISS_BYTES=0, CONSUMER_TIME_NS=734265, DECODE_TIME_NS=389815, HDFS_TIME_NS=73156, METADATA_CACHE_HIT=4, NUM_DECODED_BATCHES=2, NUM_VECTOR_BATCHES=2, ROWS_EMITTED=242, SELECTED_ROWGROUPS=2, TOTAL_IO_TIME_NS=2479110, vertexStats=firstTaskStartTime=1515180549080, firstTasksToStart=[ task_1515180518813_0001_39_01_000000 ], lastTaskFinishTime=1515180549140, lastTasksToFinish=[ task_1515180518813_0001_39_01_000000 ], minTaskDuration=60, maxTaskDuration=60, avgTaskDuration=60.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_39_01_000000 ], longestDurationTasks=[ task_1515180518813_0001_39_01_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:09,141 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000093, QueryId=djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827-39, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:09,141 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_39_01 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:09,141 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_39_01_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=3, preemptionQueueSize=1 2018-01-05T11:29:09,141 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_39_01_000000_0 2018-01-05T11:29:09,141 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_39_02 [Reducer 2] attempt: attempt_1515180518813_0001_39_01_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:29:09,142 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000093, QueryId=djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827-39, DagId=39, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-2, Succeeded=true, StartTime=1515180549092, EndTime=1515180549142 2018-01-05T11:29:09,142 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000093, stoppedTime=1515180549142, exitStatus=0 2018-01-05T11:29:09,142 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000093 exited with diagnostics set to Container completed. 2018-01-05T11:29:09,142 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_39_01 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=4 2018-01-05T11:29:09,142 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=4, commitInProgress=0, terminationCause=null 2018-01-05T11:29:09,142 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000093 2018-01-05T11:29:09,152 INFO [TezTR-518813_1_39_0_0_0] impl.PipelinedSorter: Reducer 2: Spilling to /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/39/output/attempt_1515180518813_0001_39_00_000000_0_10287_0/file.out 2018-01-05T11:29:09,164 DEBUG [TezTR-518813_1_39_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:09,164 INFO [TezTR-518813_1_39_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:09,164 INFO [TezTR-518813_1_39_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_39_00_000000_0, askedToStop=false 2018-01-05T11:29:09,164 INFO [TezTR-518813_1_39_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_39_00_000000_0, stopRequested=false 2018-01-05T11:29:09,165 INFO [TezTR-518813_1_39_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_39_00_000000_0: Counters: 31 [[File System Counters FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=0, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=500, INPUT_RECORDS_PROCESSED=4, INPUT_SPLIT_LENGTH_BYTES=2442, OUTPUT_RECORDS=500, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=3000, OUTPUT_BYTES_WITH_OVERHEAD=2941, OUTPUT_BYTES_PHYSICAL=2945, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_4=500, RECORDS_OUT_INTERMEDIATE_Map_4=500][org.apache.hadoop.hive.llap.counters.LlapIOCounters CACHE_HIT_BYTES=765, CACHE_MISS_BYTES=0, CONSUMER_TIME_NS=1271903, DECODE_TIME_NS=758713, HDFS_TIME_NS=573241, METADATA_CACHE_HIT=8, NUM_DECODED_BATCHES=4, NUM_VECTOR_BATCHES=4, ROWS_EMITTED=500, SELECTED_ROWGROUPS=4, TOTAL_IO_TIME_NS=4030603]] 2018-01-05T11:29:09,165 DEBUG [TezTR-518813_1_39_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_39_0_0_0 thread id: 2683 scheme: file bytesRead: 64 bytesWritten: 2977 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:09,165 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_39_00_000000_0, isTaskDone=true 2018-01-05T11:29:09,165 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000094, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_39_00_000000_0, eventCount=4 } 2018-01-05T11:29:09,165 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 4, taskAttemptId=attempt_1515180518813_0001_39_00_000000_0, creationTime=1515180549116, allocationTime=1515180549117, startTime=1515180549117, finishTime=1515180549165, timeTaken=48, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:09,165 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:29:09,165 INFO [Task-Executor-1] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_39_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:09,165 INFO [Task-Executor-1] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_39_00_000000_0 2018-01-05T11:29:09,165 INFO [Task-Executor-1] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000094=46 2018-01-05T11:29:09,165 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_39_00_000000_0 2018-01-05T11:29:09,165 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_39_00_000000_0: true 2018-01-05T11:29:09,165 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_39_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:09,165 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_39_00_000000_0 2018-01-05T11:29:09,165 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_39_00_000000_0; false 2018-01-05T11:29:09,166 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000094, QueryId=djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827-39, Vertex=Map 4, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:09,166 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_39_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:09,166 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_39_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:29:09,166 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:09,166 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:09,166 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_39_00_000000_0 2018-01-05T11:29:09,166 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:TASK_FINISHED]: vertexName=Map 4, taskId=task_1515180518813_0001_39_00_000000, startTime=1515180549117, finishTime=1515180549166, timeTaken=49, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_39_00_000000_0, diagnostics=, counters=Counters: 32, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=64, FILE_BYTES_WRITTEN=2977, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=500, INPUT_RECORDS_PROCESSED=4, INPUT_SPLIT_LENGTH_BYTES=2442, OUTPUT_RECORDS=500, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=3000, OUTPUT_BYTES_WITH_OVERHEAD=2941, OUTPUT_BYTES_PHYSICAL=2945, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_4=500, RECORDS_OUT_INTERMEDIATE_Map_4=500, org.apache.hadoop.hive.llap.counters.LlapIOCounters, CACHE_HIT_BYTES=765, CACHE_MISS_BYTES=0, CONSUMER_TIME_NS=1271903, DECODE_TIME_NS=758713, HDFS_TIME_NS=573241, METADATA_CACHE_HIT=8, NUM_DECODED_BATCHES=4, NUM_VECTOR_BATCHES=4, ROWS_EMITTED=500, SELECTED_ROWGROUPS=4, TOTAL_IO_TIME_NS=4030603 2018-01-05T11:29:09,166 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_39_00 [Map 4], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:09,166 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_39_00 [Map 4] 2018-01-05T11:29:09,166 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:VERTEX_FINISHED]: vertexName=Map 4, vertexId=vertex_1515180518813_0001_39_00, initRequestedTime=1515180549030, initedTime=1515180549115, startRequestedTime=1515180549052, startedTime=1515180549115, finishTime=1515180549166, timeTaken=51, status=SUCCEEDED, diagnostics=, counters=Counters: 32, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=64, FILE_BYTES_WRITTEN=2977, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=500, INPUT_RECORDS_PROCESSED=4, INPUT_SPLIT_LENGTH_BYTES=2442, OUTPUT_RECORDS=500, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=3000, OUTPUT_BYTES_WITH_OVERHEAD=2941, OUTPUT_BYTES_PHYSICAL=2945, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_4=500, RECORDS_OUT_INTERMEDIATE_Map_4=500, org.apache.hadoop.hive.llap.counters.LlapIOCounters, CACHE_HIT_BYTES=765, CACHE_MISS_BYTES=0, CONSUMER_TIME_NS=1271903, DECODE_TIME_NS=758713, HDFS_TIME_NS=573241, METADATA_CACHE_HIT=8, NUM_DECODED_BATCHES=4, NUM_VECTOR_BATCHES=4, ROWS_EMITTED=500, SELECTED_ROWGROUPS=4, TOTAL_IO_TIME_NS=4030603, vertexStats=firstTaskStartTime=1515180549117, firstTasksToStart=[ task_1515180518813_0001_39_00_000000 ], lastTaskFinishTime=1515180549165, lastTasksToFinish=[ task_1515180518813_0001_39_00_000000 ], minTaskDuration=48, maxTaskDuration=48, avgTaskDuration=48.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_39_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_39_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:09,166 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_39_00 [Map 4] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:09,166 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000094, stoppedTime=1515180549166, exitStatus=0 2018-01-05T11:29:09,166 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000094 exited with diagnostics set to Container completed. 2018-01-05T11:29:09,166 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_39_02 [Reducer 2] attempt: attempt_1515180518813_0001_39_00_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:29:09,166 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_39_00 [Map 4] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=4 2018-01-05T11:29:09,166 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=4, commitInProgress=0, terminationCause=null 2018-01-05T11:29:09,166 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: All source tasks completed. Ramping up 1 remaining tasks for vertex: Reducer 2 2018-01-05T11:29:09,166 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: Scheduling 1 tasks for vertex: Reducer 2 with totalTasks: 1. 2 source tasks completed out of 2. MinSourceTaskCompletedFraction: 1.0 min: 0.05 max: 0.75 2018-01-05T11:29:09,166 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000094 2018-01-05T11:29:09,166 INFO [App Shared Pool - #2] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Map 1, destinationVertexName=Reducer 2. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:29:09,166 INFO [App Shared Pool - #2] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Map 4, destinationVertexName=Reducer 2. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:29:09,166 INFO [App Shared Pool - #2] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_39_02 [Reducer 2] 2018-01-05T11:29:09,166 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_39_02_000000, scheduledTime=1515180549166, launchTime=1515180549166 2018-01-05T11:29:09,167 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_39_02 [Reducer 2] 2018-01-05T11:29:09,167 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_39_02_000000_0, priority=29, capability=, hosts=null 2018-01-05T11:29:09,167 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=29,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:09,167 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=29,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:09,167 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:09,167 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_39_02_000000_0 on hosts=null 2018-01-05T11:29:09,167 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_39_02_000000_0 on hosts=null 2018-01-05T11:29:09,167 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_39_02_000000_0 2018-01-05T11:29:09,167 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:09,167 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #95, task=TaskInfo{task=attempt_1515180518813_0001_39_02_000000_0, priority=29, startTime=0, containerId=null, uniqueId=94, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000095 2018-01-05T11:29:09,167 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_39_02_000000_0; false 2018-01-05T11:29:09,167 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_39_02_000000_0, priority=29, startTime=311569676, containerId=container_111101111_0001_00_000095assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=94, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:09,167 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000095 for node: 10.22.8.182:49818 2018-01-05T11:29:09,167 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000095, attempt_1515180518813_0001_39_02_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:09,167 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000095 for node: 10.22.8.182:49818 2018-01-05T11:29:09,167 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000095 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:09,167 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000095, launchTime=1515180549167 2018-01-05T11:29:09,168 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=39}, sourceName=Map 1, state=S_SUCCEEDED 2018-01-05T11:29:09,168 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=39}, sourceName=Map 4, state=S_SUCCEEDED 2018-01-05T11:29:09,168 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_39_02_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000095] on NM: [10.22.8.182:49818] 2018-01-05T11:29:09,168 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:09,168 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:09,168 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_39_02_000000_0, startTime=1515180549167, containerId=container_111101111_0001_00_000095, nodeId=10.22.8.182:49818 2018-01-05T11:29:09,168 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_39_02_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000095, dagName=select count(*) from tab a left oute...b.key (Stage-1), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=2, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.OrderedGroupedKVInput,1}{Map 4,org.apache.tez.runtime.library.input.OrderedGroupedKVInput,1}}, Outputs={{Reducer 3,org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput,1}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=3, completedTaskCount=2, dagStartTime=1515180549022, firstAttemptStartTime=1515180549167, currentAttemptStartTime=1515180549167} 2018-01-05T11:29:09,169 INFO [IPC Server handler 3 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000095, QueryId=djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827-39, DagId=39, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, SubmitTime=1515180549168 2018-01-05T11:29:09,169 DEBUG [IPC Server handler 3 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=39} with the ShuffleHandler 2018-01-05T11:29:09,169 DEBUG [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/39] 2018-01-05T11:29:09,169 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:29:09,169 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_39_02_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:09,169 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_39_02_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180549167, dagStartTime=1515180549022, withinDagPriority=29, vertexParallelism= 1, selfAndUpstreamParallelism= 3, selfAndUpstreamComplete= 2}] 2018-01-05T11:29:09,169 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_39_02_000000_0 : ACCEPTED 2018-01-05T11:29:09,169 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_39_02_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:29:09,169 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_39_02_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180549167, dagStartTime=1515180549022, withinDagPriority=29, vertexParallelism= 1, selfAndUpstreamParallelism= 3, selfAndUpstreamComplete= 2} 2018-01-05T11:29:09,169 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_39_02_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:09,169 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_39_02_000000_0 to pre-emption queue 2018-01-05T11:29:09,169 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_39_02_000000_0: true 2018-01-05T11:29:09,169 INFO [TaskCommunicator # 5] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_39_02_000000_0 2018-01-05T11:29:09,169 INFO [TaskCommunicator # 5] GuaranteedTasks: Received response for attempt_1515180518813_0001_39_02_000000_0, true 2018-01-05T11:29:09,169 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: taskOwner hashCode:1230345669 2018-01-05T11:29:09,170 INFO [Task-Executor-1] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : select count(*) from tab a left oute...b.key (Stage-1), VertexName: Reducer 2, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_39_02_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=Map 4, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=Reducer 3, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput }}, ] 2018-01-05T11:29:09,170 INFO [Task-Executor-1] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=2, numOutputs=1, JVM.maxFree=1024458752, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:09,170 INFO [Task-Executor-1] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_39_02_000000_0 vertexName: Reducer 2 2018-01-05T11:29:09,170 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000095, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_39_02_000000_0, eventCount=1 } 2018-01-05T11:29:09,170 DEBUG [TezTR-518813_1_39_2_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_39 queryId: djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827 instanceType: TaskRunner2Callable 2018-01-05T11:29:09,170 INFO [TezTR-518813_1_39_2_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_39_02_000000_0 2018-01-05T11:29:09,170 DEBUG [TezTR-518813_1_39_2_0_0] log.PerfLogger: 2018-01-05T11:29:09,171 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:09,171 INFO [I/O Setup 0 Initialize: {Map 1}] orderedgrouped.MergeManager: Initial Memory required for SHUFFLE_BUFFER=131404920 based on INPUT_BUFFER_FACTOR=0.7, for final merged output=0, using factor: 0.0 2018-01-05T11:29:09,171 INFO [I/O Setup 1 Initialize: {Map 4}] orderedgrouped.MergeManager: Initial Memory required for SHUFFLE_BUFFER=131404920 based on INPUT_BUFFER_FACTOR=0.7, for final merged output=0, using factor: 0.0 2018-01-05T11:29:09,171 INFO [I/O Setup 2 Initialize: {Reducer 3}] writers.UnorderedPartitionedKVWriter: Requested BufferSize (tez.runtime.unordered.output.buffer.size-mb) : 100 2018-01-05T11:29:09,171 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:09,172 INFO [IPC Server handler 11 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_39_02_000000_0 2 events [0,2) total 2 vertex_1515180518813_0001_39_02 [Reducer 2] 2018-01-05T11:29:09,172 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=2, nextPreRoutedEventId=0, eventCount=2 } 2018-01-05T11:29:09,172 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_39_02_000000_0, eventCount=2 fromEventId=0 nextFromEventId=2 2018-01-05T11:29:09,172 DEBUG [TezTR-518813_1_39_2_0_0] log.PerfLogger: 2018-01-05T11:29:09,172 INFO [TezTR-518813_1_39_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:09,172 INFO [TezTR-518813_1_39_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 3 initializers to finish 2018-01-05T11:29:09,172 INFO [TezTR-518813_1_39_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:09,172 INFO [TezTR-518813_1_39_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:09,173 INFO [TezTR-518813_1_39_2_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:09,173 INFO [TezTR-518813_1_39_2_0_0] resources.MemoryDistributor: InitialRequests=[Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [Map 4:INPUT:131404920:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [Reducer 3:OUTPUT:104857600:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput] 2018-01-05T11:29:09,173 INFO [TezTR-518813_1_39_2_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:09,173 INFO [TezTR-518813_1_39_2_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.045, finalReserveFractionUsed=0.345 2018-01-05T11:29:09,173 INFO [TezTR-518813_1_39_2_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 3, numScaledRequests: 25, TotalRequested: 367667440, TotalRequestedScaled: 1.3034302719999999E8, TotalJVMHeap: 1024458752, TotalAvailable: 671020482, TotalRequested/TotalJVMHeap:0.36 2018-01-05T11:29:09,173 INFO [TezTR-518813_1_39_2_0_0] resources.MemoryDistributor: Allocations=[Map 1:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:131404920:131404920], [Map 4:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:131404920:131404920], [Reducer 3:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput:OUTPUT:104857600:21592746] 2018-01-05T11:29:09,173 INFO [TezTR-518813_1_39_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:09,173 INFO [TezTR-518813_1_39_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: Map 1 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:09,173 INFO [TezTR-518813_1_39_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: Map 4 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:09,173 INFO [TezTR-518813_1_39_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 2 2018-01-05T11:29:09,173 INFO [TezTR-518813_1_39_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 IOs to start 2018-01-05T11:29:09,173 DEBUG [I/O Setup 1 Start: {Map 4}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:09,173 DEBUG [I/O Setup 1 Start: {Map 4}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:09,173 DEBUG [I/O Setup 0 Start: {Map 1}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:09,173 DEBUG [I/O Setup 0 Start: {Map 1}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:09,173 DEBUG [I/O Setup 0 Start: {Map 1}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:09,173 DEBUG [I/O Setup 1 Start: {Map 4}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:09,173 DEBUG [I/O Setup 0 Start: {Map 1}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:09,173 DEBUG [I/O Setup 1 Start: {Map 4}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:09,173 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.Shuffle: Map_1: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true 2018-01-05T11:29:09,173 INFO [I/O Setup 1 Start: {Map 4}] orderedgrouped.Shuffle: Map_4: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true 2018-01-05T11:29:09,173 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.MergeManager: Map 1: MergerManager: memoryLimit=131404920, maxSingleShuffleLimit=32851230, mergeThreshold=86727248, ioSortFactor=10, postMergeMem=0, memToMemMergeOutputsThreshold=10 2018-01-05T11:29:09,173 INFO [I/O Setup 1 Start: {Map 4}] orderedgrouped.MergeManager: Map 4: MergerManager: memoryLimit=131404920, maxSingleShuffleLimit=32851230, mergeThreshold=86727248, ioSortFactor=10, postMergeMem=0, memToMemMergeOutputsThreshold=10 2018-01-05T11:29:09,174 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.ShuffleScheduler: ShuffleScheduler running for sourceVertex: Map 1 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true 2018-01-05T11:29:09,174 INFO [I/O Setup 1 Start: {Map 4}] orderedgrouped.ShuffleScheduler: ShuffleScheduler running for sourceVertex: Map 4 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true 2018-01-05T11:29:09,174 INFO [I/O Setup 0 Start: {Map 1}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: Map 1 2018-01-05T11:29:09,174 INFO [TezTR-518813_1_39_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:09,175 INFO [I/O Setup 1 Start: {Map 4}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: Map 4 2018-01-05T11:29:09,175 INFO [TezTR-518813_1_39_2_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:09,175 INFO [TezTR-518813_1_39_2_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_39_02_000000_0 2018-01-05T11:29:09,175 DEBUG [TezTR-518813_1_39_2_0_0] log.PerfLogger: 2018-01-05T11:29:09,175 DEBUG [TezTR-518813_1_39_2_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_39_02_000000_0_10288 2018-01-05T11:29:09,175 DEBUG [TezTR-518813_1_39_2_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:09,175 DEBUG [TezTR-518813_1_39_2_0_0] mr.ObjectCache: Creating djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:29:09,175 INFO [TezTR-518813_1_39_2_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/9f8acef3-141c-4020-8a99-6646f59e492c/reduce.xml 2018-01-05T11:29:09,175 DEBUG [TezTR-518813_1_39_2_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/9f8acef3-141c-4020-8a99-6646f59e492c/reduce.xml 2018-01-05T11:29:09,175 DEBUG [TezTR-518813_1_39_2_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/9f8acef3-141c-4020-8a99-6646f59e492c/reduce.xml 2018-01-05T11:29:09,175 DEBUG [TezTR-518813_1_39_2_0_0] log.PerfLogger: 2018-01-05T11:29:09,175 INFO [TezTaskEventRouter{attempt_1515180518813_0001_39_02_000000_0}] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:29:09,175 INFO [TezTR-518813_1_39_2_0_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:29:09,176 DEBUG [TezTR-518813_1_39_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:09,177 INFO [TezTaskEventRouter{attempt_1515180518813_0001_39_02_000000_0}] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 4: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:29:09,177 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=39&reduce=0&map=attempt_1515180518813_0001_39_00_000000_0_10287 mapId: [attempt_1515180518813_0001_39_00_000000_0_10287] reduceId: [0] jobId: [job_1515180518813_0001] dagId: [39] keepAlive: false 2018-01-05T11:29:09,177 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=39&reduce=0&map=attempt_1515180518813_0001_39_01_000000_0_10284 mapId: [attempt_1515180518813_0001_39_01_000000_0_10284] reduceId: [0] jobId: [job_1515180518813_0001] dagId: [39] keepAlive: false 2018-01-05T11:29:09,177 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=39&reduce=0&map=attempt_1515180518813_0001_39_00_000000_0_10287; hash=...9u58FeQAI1YJc 2018-01-05T11:29:09,177 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=39&reduce=0&map=attempt_1515180518813_0001_39_01_000000_0_10284; hash=...ErWBU/NBwwmQ4 2018-01-05T11:29:09,177 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=39&reduce=0&map=attempt_1515180518813_0001_39_00_000000_0_10287;reply=cWHy11AbU5RsE 2018-01-05T11:29:09,177 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=39&reduce=0&map=attempt_1515180518813_0001_39_01_000000_0_10284;reply=OPpELyR9Wc7mI 2018-01-05T11:29:09,178 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=39, user='djaiswal', attemptId='attempt_1515180518813_0001_39_00_000000_0_10287'} via loader 2018-01-05T11:29:09,178 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=39, user='djaiswal', attemptId='attempt_1515180518813_0001_39_00_000000_0_10287'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:29:09,178 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_39_00_000000_0_10287 not found 2018-01-05T11:29:09,178 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=39, user='djaiswal', attemptId='attempt_1515180518813_0001_39_01_000000_0_10284'} via loader 2018-01-05T11:29:09,178 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=39, user='djaiswal', attemptId='attempt_1515180518813_0001_39_01_000000_0_10284'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:29:09,178 DEBUG [New I/O worker #3] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_39_01_000000_0_10284 not found 2018-01-05T11:29:09,178 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_39_00_000000_0_10287,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/39/output/attempt_1515180518813_0001_39_00_000000_0_10287/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/39/output/attempt_1515180518813_0001_39_00_000000_0_10287/file.out.index 2018-01-05T11:29:09,178 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Content Length in shuffle : 3000 2018-01-05T11:29:09,178 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_39_01_000000_0_10284,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/39/output/attempt_1515180518813_0001_39_01_000000_0_10284/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/39/output/attempt_1515180518813_0001_39_01_000000_0_10284/file.out.index 2018-01-05T11:29:09,178 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Content Length in shuffle : 1506 2018-01-05T11:29:09,178 INFO [Fetcher_O {Map_4} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=39&reduce=0&map=attempt_1515180518813_0001_39_00_000000_0_10287 sent hash and receievd reply 0 ms 2018-01-05T11:29:09,178 DEBUG [TezTR-518813_1_39_2_0_0] log.PerfLogger: 2018-01-05T11:29:09,178 INFO [Fetcher_O {Map_1} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=39&reduce=0&map=attempt_1515180518813_0001_39_01_000000_0_10284 sent hash and receievd reply 0 ms 2018-01-05T11:29:09,178 INFO [TezTR-518813_1_39_2_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 1.49KB 2018-01-05T11:29:09,180 INFO [Fetcher_O {Map_4} #0] orderedgrouped.MergeManager: CloseInMemoryFile. Current state: inMemoryMapOutputs.size=1, commitMemory=0, usedMemory=2941. Since last log: count=1, min=2941, max=2941, total=2941, avg=2941.0 2018-01-05T11:29:09,180 INFO [Fetcher_O {Map_4} #0] ShuffleScheduler.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_39_00_000000_0_10287} to MEMORY, csize=2945, dsize=2941, EndTime=1515180549180, TimeTaken=1, Rate=2.80 MB/s 2018-01-05T11:29:09,180 INFO [Fetcher_O {Map_1} #0] orderedgrouped.MergeManager: CloseInMemoryFile. Current state: inMemoryMapOutputs.size=1, commitMemory=0, usedMemory=1447. Since last log: count=1, min=1447, max=1447, total=1447, avg=1447.0 2018-01-05T11:29:09,180 INFO [Fetcher_O {Map_1} #0] ShuffleScheduler.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_39_01_000000_0_10284} to MEMORY, csize=1451, dsize=1447, EndTime=1515180549180, TimeTaken=2, Rate=0.69 MB/s 2018-01-05T11:29:09,180 INFO [Fetcher_O {Map_4} #0] orderedgrouped.ShuffleScheduler: All inputs fetched for input vertex : Map 4 2018-01-05T11:29:09,180 INFO [Fetcher_O {Map_1} #0] orderedgrouped.ShuffleScheduler: All inputs fetched for input vertex : Map 1 2018-01-05T11:29:09,180 INFO [Fetcher_O {Map_4} #0] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:09,180 DEBUG [TezTR-518813_1_39_2_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:09,180 DEBUG [TezTR-518813_1_39_2_0_0] log.PerfLogger: 2018-01-05T11:29:09,180 INFO [ShuffleAndMergeRunner {Map_4}] orderedgrouped.ShuffleScheduler: Shutting down FetchScheduler for input: Map_4, wasInterrupted=false 2018-01-05T11:29:09,180 DEBUG [TezTR-518813_1_39_2_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:09,180 DEBUG [TezTR-518813_1_39_2_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:09,180 INFO [ShuffleAndMergeRunner {Map_4}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:09,180 INFO [Fetcher_O {Map_1} #0] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:09,181 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down FetchScheduler for input: Map_1, wasInterrupted=false 2018-01-05T11:29:09,181 INFO [ShuffleAndMergeRunner {Map_4}] orderedgrouped.ShuffleScheduler: Shutting down fetchers for input: Map_4, shutdown timetaken: 0 ms, hasFetcherExecutorStopped: true 2018-01-05T11:29:09,181 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:09,181 INFO [ShuffleAndMergeRunner {Map_4}] orderedgrouped.MergeManager: TotalInMemFetchStats: count=1, totalSize=2941, min=2941, max=2941, avg=1.0 2018-01-05T11:29:09,181 INFO [ShuffleAndMergeRunner {Map_4}] orderedgrouped.MergeManager: finalMerge with #inMemoryOutputs=1, size=2941 and #onDiskOutputs=0, size=0 2018-01-05T11:29:09,181 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down fetchers for input: Map_1, shutdown timetaken: 0 ms, hasFetcherExecutorStopped: true 2018-01-05T11:29:09,182 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: TotalInMemFetchStats: count=1, totalSize=1447, min=1447, max=1447, avg=1.0 2018-01-05T11:29:09,182 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: finalMerge with #inMemoryOutputs=1, size=1447 and #onDiskOutputs=0, size=0 2018-01-05T11:29:09,182 DEBUG [TezTR-518813_1_39_2_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:09,182 INFO [TezTR-518813_1_39_2_0_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:29:09,194 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: merge complete for input vertex : Map_1 2018-01-05T11:29:09,194 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: Map_1: Shuffle Runner thread complete 2018-01-05T11:29:09,194 INFO [ShuffleAndMergeRunner {Map_4}] orderedgrouped.Shuffle: merge complete for input vertex : Map_4 2018-01-05T11:29:09,194 INFO [ShuffleAndMergeRunner {Map_4}] orderedgrouped.Shuffle: Map_4: Shuffle Runner thread complete 2018-01-05T11:29:09,194 INFO [TezTR-518813_1_39_2_0_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:29:09,194 INFO [TezTR-518813_1_39_2_0_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:09,194 DEBUG [TezTR-518813_1_39_2_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:09,194 INFO [TezTR-518813_1_39_2_0_0] input.OrderedGroupedKVInput: Map 1: creating ValuesIterator with comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator, keyClass=org.apache.hadoop.hive.ql.io.HiveKey, valClass=org.apache.hadoop.io.BytesWritable 2018-01-05T11:29:09,194 DEBUG [TezTR-518813_1_39_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:09,194 DEBUG [TezTR-518813_1_39_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:09,194 DEBUG [TezTR-518813_1_39_2_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[] columnTypes=[] 2018-01-05T11:29:09,194 DEBUG [TezTR-518813_1_39_2_0_0] log.PerfLogger: 2018-01-05T11:29:09,194 INFO [TezTR-518813_1_39_2_0_0] input.OrderedGroupedKVInput: Map 4: creating ValuesIterator with comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator, keyClass=org.apache.hadoop.hive.ql.io.HiveKey, valClass=org.apache.hadoop.io.BytesWritable 2018-01-05T11:29:09,194 DEBUG [TezTR-518813_1_39_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:09,194 DEBUG [TezTR-518813_1_39_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:09,194 DEBUG [TezTR-518813_1_39_2_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[] columnTypes=[] 2018-01-05T11:29:09,194 DEBUG [TezTR-518813_1_39_2_0_0] log.PerfLogger: 2018-01-05T11:29:09,194 INFO [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Initializing operator MERGEJOIN[20] 2018-01-05T11:29:09,195 INFO [TezTR-518813_1_39_2_0_0] exec.CommonJoinOperator: JOIN struct<> totalsz = 0 2018-01-05T11:29:09,195 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Initialization Done 20 MERGEJOIN 2018-01-05T11:29:09,195 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Operator 20 MERGEJOIN initialized 2018-01-05T11:29:09,195 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Initializing children of 20 MERGEJOIN 2018-01-05T11:29:09,195 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Initializing child 8 GBY 2018-01-05T11:29:09,195 INFO [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Initializing operator GBY[8] 2018-01-05T11:29:09,195 DEBUG [TezTR-518813_1_39_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:09,195 INFO [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Max hash table memory: 93860656 bytes 2018-01-05T11:29:09,195 INFO [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:29:09,195 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Initialization Done 8 GBY 2018-01-05T11:29:09,195 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Operator 8 GBY initialized 2018-01-05T11:29:09,195 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Initializing children of 8 GBY 2018-01-05T11:29:09,195 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Initializing child 9 RS 2018-01-05T11:29:09,195 INFO [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Initializing operator RS[9] 2018-01-05T11:29:09,195 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: keys size is 0 2018-01-05T11:29:09,195 INFO [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Using tag = -1 2018-01-05T11:29:09,195 DEBUG [TezTR-518813_1_39_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:09,195 DEBUG [TezTR-518813_1_39_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:09,195 DEBUG [TezTR-518813_1_39_2_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0] columnTypes=[bigint] 2018-01-05T11:29:09,195 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Initialization Done 9 RS 2018-01-05T11:29:09,195 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Operator 9 RS initialized 2018-01-05T11:29:09,195 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Initialization Done 9 RS done is reset. 2018-01-05T11:29:09,195 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Initialization Done 8 GBY done is reset. 2018-01-05T11:29:09,195 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Initialization Done 20 MERGEJOIN done is reset. 2018-01-05T11:29:09,195 INFO [TezTR-518813_1_39_2_0_0] tez.ReduceRecordProcessor: Id =20 Id =8 Id =9 <\Children> Id = 8 null<\Parent> <\RS> <\Children> Id = 20 null<\Parent> <\GBY> <\Children> <\Parent> <\MERGEJOIN> 2018-01-05T11:29:09,195 INFO [TezTR-518813_1_39_2_0_0] exec.OperatorUtils: Setting output collector: RS[9] --> Reducer 3 2018-01-05T11:29:09,195 DEBUG [TezTR-518813_1_39_2_0_0] log.PerfLogger: 2018-01-05T11:29:09,195 INFO [TezTR-518813_1_39_2_0_0] tez.ReduceRecordProcessor: Starting Output: Reducer 3 2018-01-05T11:29:09,195 INFO [TezTR-518813_1_39_2_0_0] writers.BaseUnorderedPartitionedKVWriter: Instantiating Partitioner: [org.apache.tez.mapreduce.partition.MRPartitioner] 2018-01-05T11:29:09,208 INFO [TezTR-518813_1_39_2_0_0] writers.UnorderedPartitionedKVWriter: Reducer_3: numBuffers=2, sizePerBuffer=10796372, skipBuffers=true, numPartitions=1, availableMemory=21592746, maxSingleBufferSizeBytes=2147483647, pipelinedShuffle=false, isFinalMergeEnabled=true, numPartitions=1, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:29:09,208 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,208 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,208 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,208 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,208 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,208 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,208 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,208 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,208 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,209 WARN [TezTR-518813_1_39_2_0_0] objectinspector.StandardStructObjectInspector: Trying to access 2 fields inside a list of 1 elements: [[0]] 2018-01-05T11:29:09,209 WARN [TezTR-518813_1_39_2_0_0] objectinspector.StandardStructObjectInspector: ignoring similar errors. 2018-01-05T11:29:09,210 INFO [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,211 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,212 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,213 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,214 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,215 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,216 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,217 INFO [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,217 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,218 INFO [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,218 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,219 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,220 INFO [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,220 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,221 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,222 INFO [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] log.PerfLogger: 2018-01-05T11:29:09,222 DEBUG [TezTR-518813_1_39_2_0_0] mr.ObjectCache: djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:29:09,223 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: close called for operator MERGEJOIN[20] 2018-01-05T11:29:09,223 INFO [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Closing operator MERGEJOIN[20] 2018-01-05T11:29:09,223 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Cleaning up the operator state 2018-01-05T11:29:09,223 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: Closing child = GBY[8] 2018-01-05T11:29:09,223 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: close called for operator GBY[8] 2018-01-05T11:29:09,223 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:09,223 INFO [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Closing operator GBY[8] 2018-01-05T11:29:09,223 INFO [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Begin Hash Table flush: size = 1 2018-01-05T11:29:09,223 INFO [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: keys are [] num distributions: 0 2018-01-05T11:29:09,223 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: Closing child = RS[9] 2018-01-05T11:29:09,223 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: close called for operator RS[9] 2018-01-05T11:29:09,223 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:09,223 INFO [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: Closing operator RS[9] 2018-01-05T11:29:09,223 INFO [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: RECORDS_OUT_INTERMEDIATE_Reducer_2:1, 2018-01-05T11:29:09,223 DEBUG [TezTR-518813_1_39_2_0_0] exec.ReduceSinkOperator: 9 Close done 2018-01-05T11:29:09,223 DEBUG [TezTR-518813_1_39_2_0_0] exec.GroupByOperator: 8 Close done 2018-01-05T11:29:09,223 DEBUG [TezTR-518813_1_39_2_0_0] exec.CommonMergeJoinOperator: 20 Close done 2018-01-05T11:29:09,223 INFO [TezTR-518813_1_39_2_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_39_02_000000_0 2018-01-05T11:29:09,223 INFO [TezTR-518813_1_39_2_0_0] orderedgrouped.Shuffle: Shutting down Shuffle for source: Map_1 2018-01-05T11:29:09,223 INFO [TezTR-518813_1_39_2_0_0] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:29:09,223 INFO [TezTR-518813_1_39_2_0_0] orderedgrouped.Shuffle: Shutting down Shuffle for source: Map_4 2018-01-05T11:29:09,223 INFO [TezTR-518813_1_39_2_0_0] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 4: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:29:09,223 INFO [TezTR-518813_1_39_2_0_0] writers.UnorderedPartitionedKVWriter: Reducer_3: Waiting for all spills to complete : Pending : 0 2018-01-05T11:29:09,223 INFO [TezTR-518813_1_39_2_0_0] writers.UnorderedPartitionedKVWriter: Reducer_3: All spills complete 2018-01-05T11:29:09,234 DEBUG [TezTR-518813_1_39_2_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:09,235 INFO [TezTR-518813_1_39_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:09,235 INFO [TezTR-518813_1_39_2_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_39_02_000000_0, askedToStop=false 2018-01-05T11:29:09,235 INFO [TezTR-518813_1_39_2_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_39_02_000000_0, stopRequested=false 2018-01-05T11:29:09,235 INFO [TezTR-518813_1_39_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_39_02_000000_0: Counters: 34 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=462, REDUCE_INPUT_RECORDS=742, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=742, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=5, OUTPUT_BYTES_WITH_OVERHEAD=13, OUTPUT_BYTES_PHYSICAL=41, ADDITIONAL_SPILLS_BYTES_WRITTEN=4396, ADDITIONAL_SPILLS_BYTES_READ=4396, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=4396, SHUFFLE_BYTES_DECOMPRESSED=4388, SHUFFLE_BYTES_TO_MEM=4396, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=16, MERGE_PHASE_TIME=42, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][HIVE RECORDS_OUT_INTERMEDIATE_Reducer_2=1][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0]] 2018-01-05T11:29:09,235 DEBUG [TezTR-518813_1_39_2_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_39_2_0_0 thread id: 2695 scheme: file bytesRead: 0 bytesWritten: 49 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:09,235 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_39_02_000000_0, isTaskDone=true 2018-01-05T11:29:09,235 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000095, requestId=2, startIndex=2, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_39_02_000000_0, eventCount=4 } 2018-01-05T11:29:09,236 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_39_02_000000_0, creationTime=1515180549166, allocationTime=1515180549167, startTime=1515180549167, finishTime=1515180549236, timeTaken=69, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:09,236 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=2, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:09,236 INFO [Task-Executor-1] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_39_02_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:09,236 INFO [Task-Executor-1] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_39_02_000000_0 2018-01-05T11:29:09,236 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_39_02_000000_0 2018-01-05T11:29:09,236 INFO [Task-Executor-1] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000095=66 2018-01-05T11:29:09,236 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_39_02_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:09,236 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_39_02_000000_0; false 2018-01-05T11:29:09,236 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_39_02_000000_0: true 2018-01-05T11:29:09,236 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_39_02_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:09,236 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:09,236 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:09,236 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_39_02_000000, startTime=1515180549167, finishTime=1515180549236, timeTaken=69, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_39_02_000000_0, diagnostics=, counters=Counters: 39, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=49, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=462, REDUCE_INPUT_RECORDS=742, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=742, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=5, OUTPUT_BYTES_WITH_OVERHEAD=13, OUTPUT_BYTES_PHYSICAL=41, ADDITIONAL_SPILLS_BYTES_WRITTEN=4396, ADDITIONAL_SPILLS_BYTES_READ=4396, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=4396, SHUFFLE_BYTES_DECOMPRESSED=4388, SHUFFLE_BYTES_TO_MEM=4396, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=16, MERGE_PHASE_TIME=42, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, HIVE, RECORDS_OUT_INTERMEDIATE_Reducer_2=1, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0 2018-01-05T11:29:09,236 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_39_02 [Reducer 2], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:09,236 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_39_02 [Reducer 2] 2018-01-05T11:29:09,236 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:VERTEX_FINISHED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_39_02, initRequestedTime=1515180549052, initedTime=1515180549058, startRequestedTime=1515180549115, startedTime=1515180549115, finishTime=1515180549236, timeTaken=121, status=SUCCEEDED, diagnostics=, counters=Counters: 39, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=49, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=462, REDUCE_INPUT_RECORDS=742, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=742, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=5, OUTPUT_BYTES_WITH_OVERHEAD=13, OUTPUT_BYTES_PHYSICAL=41, ADDITIONAL_SPILLS_BYTES_WRITTEN=4396, ADDITIONAL_SPILLS_BYTES_READ=4396, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=4396, SHUFFLE_BYTES_DECOMPRESSED=4388, SHUFFLE_BYTES_TO_MEM=4396, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=16, MERGE_PHASE_TIME=42, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, HIVE, RECORDS_OUT_INTERMEDIATE_Reducer_2=1, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, vertexStats=firstTaskStartTime=1515180549167, firstTasksToStart=[ task_1515180518813_0001_39_02_000000 ], lastTaskFinishTime=1515180549236, lastTasksToFinish=[ task_1515180518813_0001_39_02_000000 ], minTaskDuration=69, maxTaskDuration=69, avgTaskDuration=69.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_39_02_000000 ], longestDurationTasks=[ task_1515180518813_0001_39_02_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:09,236 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_39_02 [Reducer 2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:09,237 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000095, stoppedTime=1515180549237, exitStatus=0 2018-01-05T11:29:09,237 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000095 exited with diagnostics set to Container completed. 2018-01-05T11:29:09,237 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_39_03 [Reducer 3] attempt: attempt_1515180518813_0001_39_02_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:29:09,237 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_39_02 [Reducer 2] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=4 2018-01-05T11:29:09,237 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=4, commitInProgress=0, terminationCause=null 2018-01-05T11:29:09,237 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000095 2018-01-05T11:29:09,237 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: All source tasks completed. Ramping up 1 remaining tasks for vertex: Reducer 3 2018-01-05T11:29:09,237 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: Scheduling 1 tasks for vertex: Reducer 3 with totalTasks: 1. 1 source tasks completed out of 1. MinSourceTaskCompletedFraction: 1.0 min: 0.05 max: 0.75 2018-01-05T11:29:09,237 INFO [App Shared Pool - #2] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Reducer 2, destinationVertexName=Reducer 3. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.UnorderedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:29:09,237 INFO [App Shared Pool - #2] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_39_03 [Reducer 3] 2018-01-05T11:29:09,237 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:TASK_STARTED]: vertexName=Reducer 3, taskId=task_1515180518813_0001_39_03_000000, scheduledTime=1515180549237, launchTime=1515180549237 2018-01-05T11:29:09,237 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_39_03 [Reducer 3] 2018-01-05T11:29:09,237 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_39_03_000000_0, priority=44, capability=, hosts=null 2018-01-05T11:29:09,237 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=44,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:09,237 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=44,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:09,237 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:09,237 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_39_03_000000_0 on hosts=null 2018-01-05T11:29:09,237 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_39_03_000000_0 on hosts=null 2018-01-05T11:29:09,237 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_39_03_000000_0 2018-01-05T11:29:09,237 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:09,237 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #96, task=TaskInfo{task=attempt_1515180518813_0001_39_03_000000_0, priority=44, startTime=0, containerId=null, uniqueId=95, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000096 2018-01-05T11:29:09,237 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_39_03_000000_0; false 2018-01-05T11:29:09,237 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_39_03_000000_0, priority=44, startTime=311569747, containerId=container_111101111_0001_00_000096assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=95, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:09,237 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000096 for node: 10.22.8.182:49818 2018-01-05T11:29:09,237 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000096, attempt_1515180518813_0001_39_03_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:09,237 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000096 for node: 10.22.8.182:49818 2018-01-05T11:29:09,238 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000096 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:09,238 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000096, launchTime=1515180549238 2018-01-05T11:29:09,238 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=39}, sourceName=Reducer 2, state=S_SUCCEEDED 2018-01-05T11:29:09,238 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_39_03_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000096] on NM: [10.22.8.182:49818] 2018-01-05T11:29:09,238 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_39_03_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000096, dagName=select count(*) from tab a left oute...b.key (Stage-1), vertexName=Reducer 3, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Reducer 2,org.apache.tez.runtime.library.input.UnorderedKVInput,1}}, Outputs={{out_Reducer 3,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=4, completedTaskCount=3, dagStartTime=1515180549022, firstAttemptStartTime=1515180549237, currentAttemptStartTime=1515180549238} 2018-01-05T11:29:09,238 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:09,238 INFO [IPC Server handler 0 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000096, QueryId=djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827-39, DagId=39, VertexName=Reducer 3, TaskId=0, TaskAttemptId=0, SubmitTime=1515180549238 2018-01-05T11:29:09,238 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:09,239 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 3, taskAttemptId=attempt_1515180518813_0001_39_03_000000_0, startTime=1515180549238, containerId=container_111101111_0001_00_000096, nodeId=10.22.8.182:49818 2018-01-05T11:29:09,239 DEBUG [IPC Server handler 0 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=39} with the ShuffleHandler 2018-01-05T11:29:09,239 DEBUG [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/39] 2018-01-05T11:29:09,239 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=3, runningFragmentCount=0 2018-01-05T11:29:09,239 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_39_03_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:09,239 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_39_03_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180549237, dagStartTime=1515180549022, withinDagPriority=44, vertexParallelism= 1, selfAndUpstreamParallelism= 4, selfAndUpstreamComplete= 3}] 2018-01-05T11:29:09,239 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_39_03_000000_0 : ACCEPTED 2018-01-05T11:29:09,239 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_39_03_000000_0, canFinish=true. Current state: preemptionQueueSize=1, numSlotsAvailable=3, waitQueueSize=1 2018-01-05T11:29:09,239 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_39_03_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180549237, dagStartTime=1515180549022, withinDagPriority=44, vertexParallelism= 1, selfAndUpstreamParallelism= 4, selfAndUpstreamComplete= 3} 2018-01-05T11:29:09,239 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_39_03_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:09,239 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_39_03_000000_0 to pre-emption queue 2018-01-05T11:29:09,239 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_39_03_000000_0: true 2018-01-05T11:29:09,239 INFO [TaskCommunicator # 7] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_39_03_000000_0 2018-01-05T11:29:09,239 INFO [TaskCommunicator # 7] GuaranteedTasks: Received response for attempt_1515180518813_0001_39_03_000000_0, true 2018-01-05T11:29:09,239 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: taskOwner hashCode:1400700185 2018-01-05T11:29:09,240 INFO [Task-Executor-1] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : select count(*) from tab a left oute...b.key (Stage-1), VertexName: Reducer 3, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_39_03_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Reducer 2, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:29:09,240 INFO [Task-Executor-1] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1024458752, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:09,240 INFO [Task-Executor-1] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_39_03_000000_0 vertexName: Reducer 3 2018-01-05T11:29:09,240 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000096, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_39_03_000000_0, eventCount=1 } 2018-01-05T11:29:09,240 DEBUG [TezTR-518813_1_39_3_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_39 queryId: djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827 instanceType: TaskRunner2Callable 2018-01-05T11:29:09,240 INFO [TezTR-518813_1_39_3_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_39_03_000000_0 2018-01-05T11:29:09,240 DEBUG [TezTR-518813_1_39_3_0_0] log.PerfLogger: 2018-01-05T11:29:09,241 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:09,241 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:09,242 INFO [IPC Server handler 19 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_39_03_000000_0 1 events [0,1) total 1 vertex_1515180518813_0001_39_03 [Reducer 3] 2018-01-05T11:29:09,242 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:29:09,242 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_39_03_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:29:09,243 INFO [I/O Setup 1 Initialize: {out_Reducer 3}] output.MROutput: out_Reducer 3: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:29:09,243 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:09,243 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:09,243 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:09,243 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:09,243 DEBUG [TezTR-518813_1_39_3_0_0] log.PerfLogger: 2018-01-05T11:29:09,243 INFO [TezTR-518813_1_39_3_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:09,243 INFO [TezTR-518813_1_39_3_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:09,243 INFO [TezTR-518813_1_39_3_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:09,243 INFO [TezTR-518813_1_39_3_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:09,243 INFO [TezTR-518813_1_39_3_0_0] resources.MemoryDistributor: InitialRequests=[out_Reducer 3:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [Reducer 2:INPUT:131404920:org.apache.tez.runtime.library.input.UnorderedKVInput] 2018-01-05T11:29:09,243 INFO [TezTR-518813_1_39_3_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:09,243 INFO [TezTR-518813_1_39_3_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:09,243 INFO [TezTR-518813_1_39_3_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 131404920, TotalRequestedScaled: 6.570246E7, TotalJVMHeap: 1024458752, TotalAvailable: 686387363, TotalRequested/TotalJVMHeap:0.13 2018-01-05T11:29:09,243 INFO [TezTR-518813_1_39_3_0_0] resources.MemoryDistributor: Allocations=[out_Reducer 3:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.input.UnorderedKVInput:INPUT:131404920:131404920] 2018-01-05T11:29:09,243 INFO [TezTR-518813_1_39_3_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:09,243 INFO [TezTR-518813_1_39_3_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 0 2018-01-05T11:29:09,243 INFO [TezTR-518813_1_39_3_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:09,243 INFO [TezTR-518813_1_39_3_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_39_03_000000_0 2018-01-05T11:29:09,243 DEBUG [TezTR-518813_1_39_3_0_0] log.PerfLogger: 2018-01-05T11:29:09,243 DEBUG [TezTR-518813_1_39_3_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_39_03_000000_0_10292 2018-01-05T11:29:09,244 DEBUG [TezTR-518813_1_39_3_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:09,244 DEBUG [TezTR-518813_1_39_3_0_0] mr.ObjectCache: Creating djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827_Reducer 3__REDUCE_PLAN__ 2018-01-05T11:29:09,244 INFO [TezTR-518813_1_39_3_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/70931752-5b34-4578-915c-3958ba0e2e53/reduce.xml 2018-01-05T11:29:09,244 DEBUG [TezTR-518813_1_39_3_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/70931752-5b34-4578-915c-3958ba0e2e53/reduce.xml 2018-01-05T11:29:09,244 DEBUG [TezTR-518813_1_39_3_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/djaiswal/_tez_scratch_dir/70931752-5b34-4578-915c-3958ba0e2e53/reduce.xml 2018-01-05T11:29:09,244 DEBUG [TezTR-518813_1_39_3_0_0] log.PerfLogger: 2018-01-05T11:29:09,244 INFO [TezTR-518813_1_39_3_0_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:29:09,244 DEBUG [TezTR-518813_1_39_3_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:09,248 DEBUG [TezTR-518813_1_39_3_0_0] log.PerfLogger: 2018-01-05T11:29:09,248 INFO [TezTR-518813_1_39_3_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 3 size: 1.91KB 2018-01-05T11:29:09,248 DEBUG [TezTR-518813_1_39_3_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:09,248 DEBUG [TezTR-518813_1_39_3_0_0] log.PerfLogger: 2018-01-05T11:29:09,248 DEBUG [TezTR-518813_1_39_3_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:09,248 DEBUG [TezTR-518813_1_39_3_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:09,248 DEBUG [TezTR-518813_1_39_3_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:09,248 INFO [TezTR-518813_1_39_3_0_0] impl.SimpleFetchedInputAllocator: Reducer_2: RequestedMemory=131404920, AssignedMemory=131404920, maxSingleShuffleLimit=32851230 2018-01-05T11:29:09,249 INFO [TezTR-518813_1_39_3_0_0] impl.ShuffleManager: Reducer_2: numInputs=1, compressionCodec=NoCompressionCodec, numFetchers=1, ifileBufferSize=4096, ifileReadAheadEnabled=true, ifileReadAheadLength=4194304, localDiskFetchEnabled=false, sharedFetchEnabled=false, keepAlive=false, keepAliveMaxConnections=20, connectionTimeout=180000, readTimeout=180000, bufferSize=8192, bufferSize=8192, maxTaskOutputAtOnce=20 2018-01-05T11:29:09,249 INFO [TezTR-518813_1_39_3_0_0] impl.ShuffleInputEventHandlerImpl: Reducer 2: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:29:09,249 INFO [TezTR-518813_1_39_3_0_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:29:09,250 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=39&reduce=0&map=attempt_1515180518813_0001_39_02_000000_0_10291 mapId: [attempt_1515180518813_0001_39_02_000000_0_10291] reduceId: [0] jobId: [job_1515180518813_0001] dagId: [39] keepAlive: false 2018-01-05T11:29:09,250 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=39&reduce=0&map=attempt_1515180518813_0001_39_02_000000_0_10291; hash=...lOnX8MAEn+eEQ 2018-01-05T11:29:09,250 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=39&reduce=0&map=attempt_1515180518813_0001_39_02_000000_0_10291;reply=9jHKlM5Vt+Hiw 2018-01-05T11:29:09,250 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=39, user='djaiswal', attemptId='attempt_1515180518813_0001_39_02_000000_0_10291'} via loader 2018-01-05T11:29:09,250 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=39, user='djaiswal', attemptId='attempt_1515180518813_0001_39_02_000000_0_10291'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:29:09,250 DEBUG [New I/O worker #3] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_39_02_000000_0_10291 not found 2018-01-05T11:29:09,250 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_39_02_000000_0_10291,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/39/output/attempt_1515180518813_0001_39_02_000000_0_10291/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/39/output/attempt_1515180518813_0001_39_02_000000_0_10291/file.out.index 2018-01-05T11:29:09,250 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Content Length in shuffle : 68 2018-01-05T11:29:09,250 INFO [Fetcher_B {Reducer_2} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=39&reduce=0&map=attempt_1515180518813_0001_39_02_000000_0_10291 sent hash and receievd reply 0 ms 2018-01-05T11:29:09,251 INFO [Fetcher_B {Reducer_2} #0] ShuffleManager.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_39_02_000000_0_10291} to MEMORY, csize=17, dsize=13, EndTime=1515180549251, TimeTaken=0, Rate=0.00 MB/s 2018-01-05T11:29:09,251 INFO [Fetcher_B {Reducer_2} #0] impl.ShuffleManager: All inputs fetched for input vertex : Reducer 2 2018-01-05T11:29:09,251 INFO [TezTR-518813_1_39_3_0_0] tez.ReduceRecordProcessor: Main work is Reducer 3 2018-01-05T11:29:09,251 INFO [Fetcher_B {Reducer_2} #0] impl.ShuffleManager: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:09,251 INFO [TezTR-518813_1_39_3_0_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:09,251 INFO [ShuffleRunner {Reducer_2}] impl.ShuffleManager: Reducer_2: Shutting down FetchScheduler, Was Interrupted: false 2018-01-05T11:29:09,251 DEBUG [TezTR-518813_1_39_3_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:09,251 INFO [ShuffleRunner {Reducer_2}] impl.ShuffleManager: Reducer_2: Scheduler thread completed 2018-01-05T11:29:09,251 DEBUG [TezTR-518813_1_39_3_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:09,251 DEBUG [TezTR-518813_1_39_3_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:09,251 DEBUG [TezTR-518813_1_39_3_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0] columnTypes=[bigint] 2018-01-05T11:29:09,251 DEBUG [TezTR-518813_1_39_3_0_0] log.PerfLogger: 2018-01-05T11:29:09,251 INFO [TezTR-518813_1_39_3_0_0] vector.VectorGroupByOperator: Initializing operator GBY[25] 2018-01-05T11:29:09,252 INFO [TezTR-518813_1_39_3_0_0] vector.VectorGroupByOperator: using global aggregation processing mode 2018-01-05T11:29:09,252 DEBUG [TezTR-518813_1_39_3_0_0] vector.VectorGroupByOperator: Initialization Done 25 GBY 2018-01-05T11:29:09,252 DEBUG [TezTR-518813_1_39_3_0_0] vector.VectorGroupByOperator: Operator 25 GBY initialized 2018-01-05T11:29:09,252 DEBUG [TezTR-518813_1_39_3_0_0] vector.VectorGroupByOperator: Initializing children of 25 GBY 2018-01-05T11:29:09,252 DEBUG [TezTR-518813_1_39_3_0_0] vector.VectorFileSinkOperator: Initializing child 26 FS 2018-01-05T11:29:09,252 INFO [TezTR-518813_1_39_3_0_0] vector.VectorFileSinkOperator: Initializing operator FS[26] 2018-01-05T11:29:09,252 DEBUG [TezTR-518813_1_39_3_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:09,252 INFO [TezTR-518813_1_39_3_0_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@63e0dd45]:[_col0]:[bigint]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@6ecbd468 2018-01-05T11:29:09,252 DEBUG [TezTR-518813_1_39_3_0_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:29:09,252 DEBUG [TezTR-518813_1_39_3_0_0] vector.VectorFileSinkOperator: Initialization Done 26 FS 2018-01-05T11:29:09,252 DEBUG [TezTR-518813_1_39_3_0_0] vector.VectorFileSinkOperator: Operator 26 FS initialized 2018-01-05T11:29:09,252 DEBUG [TezTR-518813_1_39_3_0_0] vector.VectorFileSinkOperator: Initialization Done 26 FS done is reset. 2018-01-05T11:29:09,252 DEBUG [TezTR-518813_1_39_3_0_0] vector.VectorGroupByOperator: Initialization Done 25 GBY done is reset. 2018-01-05T11:29:09,252 INFO [TezTR-518813_1_39_3_0_0] tez.ReduceRecordProcessor: Id =25 Id =26 <\Children> Id = 25 null<\Parent> <\FS> <\Children> <\Parent> <\GBY> 2018-01-05T11:29:09,252 DEBUG [TezTR-518813_1_39_3_0_0] log.PerfLogger: 2018-01-05T11:29:09,252 INFO [TezTR-518813_1_39_3_0_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 3 2018-01-05T11:29:09,253 INFO [TezTR-518813_1_39_3_0_0] readers.UnorderedKVReader: Num Records read: 1 2018-01-05T11:29:09,253 DEBUG [TezTR-518813_1_39_3_0_0] log.PerfLogger: 2018-01-05T11:29:09,253 DEBUG [TezTR-518813_1_39_3_0_0] mr.ObjectCache: djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827_Reducer 3__REDUCE_PLAN__ no longer needed 2018-01-05T11:29:09,253 DEBUG [TezTR-518813_1_39_3_0_0] vector.VectorGroupByOperator: close called for operator GBY[25] 2018-01-05T11:29:09,253 INFO [TezTR-518813_1_39_3_0_0] vector.VectorGroupByOperator: Closing operator GBY[25] 2018-01-05T11:29:09,253 INFO [TezTR-518813_1_39_3_0_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-08_754_5378951951843078555-1/_tmp.-ext-10002/000000_0 2018-01-05T11:29:09,253 INFO [TezTR-518813_1_39_3_0_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-08_754_5378951951843078555-1/_task_tmp.-ext-10002/_tmp.000000_0 2018-01-05T11:29:09,253 INFO [TezTR-518813_1_39_3_0_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-08_754_5378951951843078555-1/_tmp.-ext-10002/000000_0 2018-01-05T11:29:09,268 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000094, QueryId=djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827-39, DagId=39, VertexName=Map 4, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-1, Succeeded=true, StartTime=1515180549118, EndTime=1515180549268 2018-01-05T11:29:09,268 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_39_02_000000_0 2018-01-05T11:29:09,268 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000095, QueryId=djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827-39, Vertex=Reducer 2, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:09,268 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_39_02_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=3, preemptionQueueSize=1 2018-01-05T11:29:09,268 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_39_02_000000_0 2018-01-05T11:29:09,268 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000095, QueryId=djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827-39, DagId=39, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-1, Succeeded=true, StartTime=1515180549169, EndTime=1515180549268 2018-01-05T11:29:09,269 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,269 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: Status: Running (Executing on YARN cluster with App id application_1515180518813_0001) 2018-01-05T11:29:09,270 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,270 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,270 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,270 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,270 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,270 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,270 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,270 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 Map 4: 1/1 Reducer 2: 1/1 Reducer 3: 0(+1)/1 2018-01-05T11:29:09,284 INFO [TezTR-518813_1_39_3_0_0] exec.FileSinkOperator: FS[26]: records written - 1 2018-01-05T11:29:09,285 DEBUG [TezTR-518813_1_39_3_0_0] vector.VectorGroupByOperator: Closing child = FS[26] 2018-01-05T11:29:09,285 DEBUG [TezTR-518813_1_39_3_0_0] vector.VectorFileSinkOperator: close called for operator FS[26] 2018-01-05T11:29:09,285 DEBUG [TezTR-518813_1_39_3_0_0] vector.VectorFileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:09,285 INFO [TezTR-518813_1_39_3_0_0] vector.VectorFileSinkOperator: Closing operator FS[26] 2018-01-05T11:29:09,285 INFO [TezTR-518813_1_39_3_0_0] exec.FileSinkOperator: FS[26]: records written - 1 2018-01-05T11:29:09,285 INFO [TezTR-518813_1_39_3_0_0] vector.VectorFileSinkOperator: RECORDS_OUT_0:1, 2018-01-05T11:29:09,285 DEBUG [TezTR-518813_1_39_3_0_0] vector.VectorFileSinkOperator: 26 Close done 2018-01-05T11:29:09,285 DEBUG [TezTR-518813_1_39_3_0_0] vector.VectorGroupByOperator: 25 Close done 2018-01-05T11:29:09,285 INFO [TezTR-518813_1_39_3_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_39_03_000000_0 2018-01-05T11:29:09,285 INFO [TezTR-518813_1_39_3_0_0] impl.ShuffleInputEventHandlerImpl: Reducer 2: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:29:09,285 INFO [TezTR-518813_1_39_3_0_0] impl.ShuffleManager: Shutting down pending fetchers on sourceReducer_2: 0 2018-01-05T11:29:09,285 INFO [TezTR-518813_1_39_3_0_0] output.MROutput: out_Reducer 3 closed 2018-01-05T11:29:09,285 DEBUG [TezTR-518813_1_39_3_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:09,285 INFO [TezTR-518813_1_39_3_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:09,285 INFO [TezTR-518813_1_39_3_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_39_03_000000_0, askedToStop=false 2018-01-05T11:29:09,285 INFO [TezTR-518813_1_39_3_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_39_03_000000_0, stopRequested=false 2018-01-05T11:29:09,285 INFO [TezTR-518813_1_39_3_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_39_03_000000_0: Counters: 14 [[org.apache.tez.common.counters.TaskCounter NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=17, SHUFFLE_BYTES_DECOMPRESSED=13, SHUFFLE_BYTES_TO_MEM=17, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=3, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1][HIVE CREATED_FILES=1, RECORDS_OUT_0=1]] 2018-01-05T11:29:09,285 DEBUG [TezTR-518813_1_39_3_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_39_3_0_0 thread id: 2718 scheme: file bytesRead: 0 bytesWritten: 115 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:09,285 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_39_03_000000_0, isTaskDone=true 2018-01-05T11:29:09,285 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000096, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_39_03_000000_0, eventCount=2 } 2018-01-05T11:29:09,286 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 3, taskAttemptId=attempt_1515180518813_0001_39_03_000000_0, creationTime=1515180549237, allocationTime=1515180549237, startTime=1515180549238, finishTime=1515180549286, timeTaken=48, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:09,286 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:09,286 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_39_03_000000_0 2018-01-05T11:29:09,286 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_39_03_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:09,286 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_39_03_000000_0; false 2018-01-05T11:29:09,286 INFO [Task-Executor-1] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_39_03_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:09,286 INFO [Task-Executor-1] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_39_03_000000_0 2018-01-05T11:29:09,286 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_39_03_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:09,286 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:09,286 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:09,286 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:TASK_FINISHED]: vertexName=Reducer 3, taskId=task_1515180518813_0001_39_03_000000, startTime=1515180549238, finishTime=1515180549286, timeTaken=48, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_39_03_000000_0, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=115, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=17, SHUFFLE_BYTES_DECOMPRESSED=13, SHUFFLE_BYTES_TO_MEM=17, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=3, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1 2018-01-05T11:29:09,287 INFO [Task-Executor-1] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000096=47 2018-01-05T11:29:09,287 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_39_03_000000_0: true 2018-01-05T11:29:09,287 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_39_03 [Reducer 3], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:09,287 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_39_03 [Reducer 3] 2018-01-05T11:29:09,287 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_39_03_000000_0 2018-01-05T11:29:09,287 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000096, QueryId=djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827-39, Vertex=Reducer 3, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:09,287 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_39_03_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:29:09,287 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_39_03_000000_0 2018-01-05T11:29:09,287 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:VERTEX_FINISHED]: vertexName=Reducer 3, vertexId=vertex_1515180518813_0001_39_03, initRequestedTime=1515180549058, initedTime=1515180549063, startRequestedTime=1515180549115, startedTime=1515180549115, finishTime=1515180549287, timeTaken=172, status=SUCCEEDED, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=115, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=17, SHUFFLE_BYTES_DECOMPRESSED=13, SHUFFLE_BYTES_TO_MEM=17, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=3, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1, vertexStats=firstTaskStartTime=1515180549238, firstTasksToStart=[ task_1515180518813_0001_39_03_000000 ], lastTaskFinishTime=1515180549286, lastTasksToFinish=[ task_1515180518813_0001_39_03_000000 ], minTaskDuration=48, maxTaskDuration=48, avgTaskDuration=48.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_39_03_000000 ], longestDurationTasks=[ task_1515180518813_0001_39_03_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:09,287 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_39_03 [Reducer 3] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:09,287 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000096, stoppedTime=1515180549287, exitStatus=0 2018-01-05T11:29:09,287 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000096 exited with diagnostics set to Container completed. 2018-01-05T11:29:09,287 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_39_03 [Reducer 3] completed., numCompletedVertices=4, numSuccessfulVertices=4, numFailedVertices=0, numKilledVertices=0, numVertices=4 2018-01-05T11:29:09,287 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=4, numSuccessfulVertices=4, numFailedVertices=0, numKilledVertices=0, numVertices=4, commitInProgress=0, terminationCause=null 2018-01-05T11:29:09,287 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_39_03 [Reducer 3] 2018-01-05T11:29:09,287 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_39_02 [Reducer 2] 2018-01-05T11:29:09,287 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_39_01 [Map 1] 2018-01-05T11:29:09,287 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_39_00 [Map 4] 2018-01-05T11:29:09,288 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_39, queueSize=0 2018-01-05T11:29:09,288 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_39][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_39, startTime=1515180549022, finishTime=1515180549287, timeTaken=265, status=SUCCEEDED, diagnostics=, counters=Counters: 65, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=4, TOTAL_LAUNCHED_TASKS=4, OTHER_LOCAL_TASKS=2, AM_CPU_MILLISECONDS=604, AM_GC_TIME_MILLIS=14, File System Counters, FILE_BYTES_READ=128, FILE_BYTES_WRITTEN=4624, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=462, REDUCE_INPUT_RECORDS=742, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=1484, NUM_SHUFFLED_INPUTS=3, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, INPUT_RECORDS_PROCESSED=7, INPUT_SPLIT_LENGTH_BYTES=3591, OUTPUT_RECORDS=743, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=4457, OUTPUT_BYTES_WITH_OVERHEAD=4401, OUTPUT_BYTES_PHYSICAL=4437, ADDITIONAL_SPILLS_BYTES_WRITTEN=4396, ADDITIONAL_SPILLS_BYTES_READ=4396, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=2, SHUFFLE_BYTES=4413, SHUFFLE_BYTES_DECOMPRESSED=4401, SHUFFLE_BYTES_TO_MEM=4413, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=19, MERGE_PHASE_TIME=42, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, HIVE, CREATED_FILES=1, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=242, RECORDS_IN_Map_4=500, RECORDS_OUT_0=1, RECORDS_OUT_INTERMEDIATE_Map_1=242, RECORDS_OUT_INTERMEDIATE_Map_4=500, RECORDS_OUT_INTERMEDIATE_Reducer_2=1, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, org.apache.hadoop.hive.llap.counters.LlapIOCounters, CACHE_HIT_BYTES=1126, CACHE_MISS_BYTES=0, CONSUMER_TIME_NS=2006168, DECODE_TIME_NS=1148528, HDFS_TIME_NS=646397, METADATA_CACHE_HIT=12, NUM_DECODED_BATCHES=6, NUM_VECTOR_BATCHES=6, ROWS_EMITTED=742, SELECTED_ROWGROUPS=6, TOTAL_IO_TIME_NS=6509713 2018-01-05T11:29:09,288 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_39 finished with state: SUCCEEDED 2018-01-05T11:29:09,288 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_39 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:29:09,288 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000096 2018-01-05T11:29:09,288 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_39, dagState=SUCCEEDED 2018-01-05T11:29:09,288 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_39_post 2018-01-05T11:29:09,288 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:29:09,288 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:29:09,288 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:29:09,288 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 39 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=4, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=4,NumRequestsWithoutLocation=4, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=4} 2018-01-05T11:29:09,288 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:29:09,288 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=select count(*) from tab a left oute...b.key (Stage-1), with id=dag_1515180518813_0001_39 2018-01-05T11:29:09,289 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 39, to 10.22.8.182:49818 2018-01-05T11:29:09,289 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=4, Remaining=0 2018-01-05T11:29:09,289 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=select count(*) from tab a left oute...b.key (Stage-1), with id=dag_1515180518813_0001_39 2018-01-05T11:29:09,289 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=39} 2018-01-05T11:29:09,289 INFO [IPC Server handler 2 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=39}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:29:09,289 INFO [IPC Server handler 2 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/39 after 300 seconds 2018-01-05T11:29:09,289 ERROR [IPC Server handler 2 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827, dag_1515180518813_0001_39 2018-01-05T11:29:09,289 INFO [IPC Server handler 2 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827 2018-01-05T11:29:09,289 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=39} = 0 2018-01-05T11:29:09,289 DEBUG [IPC Server handler 2 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=39} 2018-01-05T11:29:09,289 DEBUG [IPC Server handler 2 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:29:09,289 DEBUG [IPC Server handler 2 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=39} 2018-01-05T11:29:09,290 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,290 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 Map 4: 1/1 Reducer 2: 1/1 Reducer 3: 1/1 2018-01-05T11:29:09,290 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,292 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,292 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: TaskId for 000000_0 = 000000 2018-01-05T11:29:09,292 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,292 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,292 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,292 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,292 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,292 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: select count(*) from tab a left outer join tab_part b on a.key = b.key 2018-01-05T11:29:09,292 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:29:09,292 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@tab 2018-01-05T11:29:09,292 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@tab@ds=2008-04-08 2018-01-05T11:29:09,292 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@tab_part 2018-01-05T11:29:09,292 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@tab_part@ds=2008-04-08 2018-01-05T11:29:09,292 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/-mr-10001 2018-01-05T11:29:09,292 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,292 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,292 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:09,292 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {} 2018-01-05T11:29:09,292 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827); Time taken: 0.42 seconds 2018-01-05T11:29:09,292 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:09,292 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,292 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:09,292 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,292 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query select count(*) from tab a left outer join tab_part b on a.key = b.key 2018-01-05T11:29:09,306 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] mapred.FileInputFormat: Time taken to get FileStatuses: 13 2018-01-05T11:29:09,306 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:29:09,306 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 13 2018-01-05T11:29:09,306 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FetchOperator: Creating fetchTask with deserializer typeinfo: struct<_col0:bigint> 2018-01-05T11:29:09,306 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FetchOperator: deserializer properties: table properties: {columns=_col0, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=bigint, escape.delim=\} partition properties: {columns=_col0, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=bigint, escape.delim=\} 2018-01-05T11:29:09,306 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: close called for operator LIST_SINK[27] 2018-01-05T11:29:09,306 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: Closing operator LIST_SINK[27] 2018-01-05T11:29:09,306 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: 27 Close done 2018-01-05T11:29:09,306 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting result dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/-mr-10001 2018-01-05T11:29:09,307 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-08_754_5378951951843078555-1 2018-01-05T11:29:09,307 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-08_754_5378951951843078555-1 2018-01-05T11:29:09,308 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.538 seconds, Fetched: 1 row(s) 2018-01-05T11:29:09,308 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:09,308 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:09,308 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:09,308 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:09,308 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,308 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:09,308 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,308 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,308 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,308 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: explain vectorization detail select count (*) from tab a right outer join tab_part b on a.key = b.key 2018-01-05T11:29:09,308 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112909_558f83a9-3340-4175-ba52-6cd82364c2ac): explain vectorization detail select count (*) from tab a right outer join tab_part b on a.key = b.key 2018-01-05T11:29:09,308 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,308 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: explain vectorization detail select count (*) from tab a right outer join tab_part b on a.key = b.key 2018-01-05T11:29:09,309 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:09,309 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,309 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,309 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,309 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,309 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:09,309 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:29:09,309 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:09,309 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed phase 1 of Semantic Analysis 2018-01-05T11:29:09,309 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:09,309 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,309 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab 2018-01-05T11:29:09,309 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab 2018-01-05T11:29:09,312 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,313 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,313 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,313 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab_part 2018-01-05T11:29:09,313 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab_part 2018-01-05T11:29:09,315 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,316 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,316 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:09,316 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:09,316 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed getting MetaData in Semantic Analysis 2018-01-05T11:29:09,317 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,317 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,317 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_not_null_constraints : db=default tbl=tab 2018-01-05T11:29:09,318 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_not_null_constraints : db=default tbl=tab 2018-01-05T11:29:09,318 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,318 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,318 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,318 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab 2018-01-05T11:29:09,318 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab 2018-01-05T11:29:09,318 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,318 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,318 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,318 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab 2018-01-05T11:29:09,318 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab 2018-01-05T11:29:09,319 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,319 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,319 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,319 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_unique_constraints : db=default tbl=tab 2018-01-05T11:29:09,319 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_unique_constraints : db=default tbl=tab 2018-01-05T11:29:09,319 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,319 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,319 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,319 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab 2018-01-05T11:29:09,319 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab 2018-01-05T11:29:09,319 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,319 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,320 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,320 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_not_null_constraints : db=default tbl=tab_part 2018-01-05T11:29:09,320 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_not_null_constraints : db=default tbl=tab_part 2018-01-05T11:29:09,320 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,320 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,320 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,320 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:09,320 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:09,320 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,320 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,320 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,320 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:09,321 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:09,321 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,321 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,321 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,321 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_unique_constraints : db=default tbl=tab_part 2018-01-05T11:29:09,321 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_unique_constraints : db=default tbl=tab_part 2018-01-05T11:29:09,321 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,321 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,321 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,321 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab_part 2018-01-05T11:29:09,321 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab_part 2018-01-05T11:29:09,321 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,321 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,322 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:09,322 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:09,322 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:09,322 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:29:09,322 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,333 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.sql2rel: Plan after trimming unused fields HiveAggregate(group=[{}], agg#0=[count()]) HiveProject(DUMMY=[0]) HiveJoin(condition=[=($0, $1)], joinType=[right], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:09,333 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan before removing subquery: HiveProject(_o__c0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveProject($f0=[$0]) HiveJoin(condition=[=($0, $6)], joinType=[right], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:09,334 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan just after removing subquery: HiveProject(_o__c0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveProject($f0=[$0]) HiveJoin(condition=[=($0, $6)], joinType=[right], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:09,334 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan after decorrelation: HiveProject(_o__c0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveProject($f0=[$0]) HiveJoin(condition=[=($0, $6)], joinType=[right], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:09,334 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,335 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,335 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,335 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,335 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,336 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,336 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,336 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,336 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,336 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,337 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,348 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,348 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,348 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,348 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,349 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.sql2rel: Plan after trimming unused fields HiveAggregate(group=[{}], agg#0=[count()]) HiveProject(DUMMY=[0]) HiveJoin(condition=[=($0, $1)], joinType=[right], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:09,349 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,349 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,349 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,349 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,350 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,350 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,350 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,350 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,350 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,350 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,351 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,351 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,351 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,351 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,351 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,358 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Original plan for PlanModifier HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[right], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:09,358 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after top-level introduceDerivedTable HiveProject($f0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[right], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:09,358 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after nested convertOpTree HiveProject($f0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[right], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:09,359 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after propagating order HiveProject($f0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[right], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:09,359 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after fixTopOBSchema HiveProject($f0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[right], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:09,359 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Final plan after modifier HiveProject(_c0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[right], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:09,359 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:09,359 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:09,359 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:09,359 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:09,359 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,359 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab 2018-01-05T11:29:09,359 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab 2018-01-05T11:29:09,361 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,362 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,362 DEBUG [AMReporterQueueDrainer] impl.AMReporter: Removing am 10.22.8.182:49824 with last associated dag QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=25} from heartbeat with taskCount=0, amFailed=false, isDone=true 2018-01-05T11:29:09,362 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:09,362 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:09,362 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:09,362 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,362 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab_part 2018-01-05T11:29:09,362 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab_part 2018-01-05T11:29:09,365 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,365 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,365 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:09,365 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:09,365 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:09,366 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Table Plan for a TS[0] 2018-01-05T11:29:09,366 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB a{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB a{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:09,366 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (. (tok_table_or_col a) key) key)) 2018-01-05T11:29:09,366 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = a{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} starRr = null 2018-01-05T11:29:09,366 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(key,_col0: int)} 2018-01-05T11:29:09,366 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:09,366 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block $hdt$_0 2018-01-05T11:29:09,366 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block $hdt$_0 2018-01-05T11:29:09,366 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Table Plan for b TS[2] 2018-01-05T11:29:09,366 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB b{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB b{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:09,366 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (. (tok_table_or_col b) key) key)) 2018-01-05T11:29:09,366 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = b{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} starRr = null 2018-01-05T11:29:09,366 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(key,_col0: int)} 2018-01-05T11:29:09,366 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:09,366 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block $hdt$_1 2018-01-05T11:29:09,366 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block $hdt$_1 2018-01-05T11:29:09,367 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:09,367 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB $hdt$_0{(key,_col0: int)} $hdt$_1{(key,_col1: int)} after GB {((tok_functionstar count),_col0: bigint)} 2018-01-05T11:29:09,367 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (tok_functionstar count) _c0)) 2018-01-05T11:29:09,367 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = {((tok_functionstar count),_col0: bigint)} starRr = $hdt$_0{(key,_col0: int)} $hdt$_1{(key,_col1: int)} 2018-01-05T11:29:09,367 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(_c0,_col0: bigint)} 2018-01-05T11:29:09,367 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:09,367 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_308_474189882829300338-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-09_308_474189882829300338-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_308_474189882829300338-1/-mr-10001 2018-01-05T11:29:09,367 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_308_474189882829300338-1/-mr-10003 2018-01-05T11:29:09,367 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_308_474189882829300338-1/-mr-10001 row schema: null{(_c0,_col0: bigint)} 2018-01-05T11:29:09,367 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block null 2018-01-05T11:29:09,367 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:29:09,367 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: CBO Succeeded; optimized logical plan. 2018-01-05T11:29:09,367 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Before logical optimization TS[0]-SEL[1]-RS[4]-JOIN[6]-SEL[7]-GBY[8]-RS[9]-GBY[10]-SEL[11]-FS[12] TS[2]-SEL[3]-RS[5]-JOIN[6] 2018-01-05T11:29:09,367 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,367 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,367 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,368 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] lineage.Generator: Time taken for lineage transform=1 2018-01-05T11:29:09,368 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,368 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,368 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,368 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,368 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SyntheticJoinPredicate: Synthetic predicate: 0 --> 1 2018-01-05T11:29:09,368 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,368 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,368 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FS(12) 2018-01-05T11:29:09,368 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(11) 2018-01-05T11:29:09,368 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for GBY(10) 2018-01-05T11:29:09,368 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(9) 2018-01-05T11:29:09,368 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for GBY(8) 2018-01-05T11:29:09,368 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(7) 2018-01-05T11:29:09,368 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for JOIN(6) 2018-01-05T11:29:09,368 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(4) 2018-01-05T11:29:09,368 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(13) 2018-01-05T11:29:09,368 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias $hdt$_0: (_col0) IN (RS[5]) 2018-01-05T11:29:09,368 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(1) 2018-01-05T11:29:09,368 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of SEL for alias $hdt$_0: (key) IN (RS[5]) 2018-01-05T11:29:09,368 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:29:09,369 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of TS for alias a: (key) IN (RS[5]) 2018-01-05T11:29:09,369 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(5) 2018-01-05T11:29:09,369 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(3) 2018-01-05T11:29:09,369 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for TS(2) 2018-01-05T11:29:09,369 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SimplePredicatePushDown: After PPD: TS[0]-FIL[14]-SEL[1]-RS[4]-JOIN[6]-SEL[7]-GBY[8]-RS[9]-GBY[10]-SEL[11]-FS[12] TS[2]-SEL[3]-RS[5]-JOIN[6] 2018-01-05T11:29:09,369 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,369 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,369 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,369 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,369 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,369 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,369 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,369 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,369 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,369 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,369 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partitions : db=default tbl=tab 2018-01-05T11:29:09,369 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions : db=default tbl=tab 2018-01-05T11:29:09,371 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.252797ms + 0.003689ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? ] 2018-01-05T11:29:09,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.167293ms + 0.045208ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (15) order by "PART_NAME" asc] 2018-01-05T11:29:09,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.072279ms + 0.051368ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (15) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:29:09,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.062409ms + 0.016395ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (15) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:09,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.054644ms + 0.00553ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (43) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:29:09,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.05461ms + 0.018353ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (43) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:09,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.067908ms + 0.014573ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (43) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:09,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.048769ms + 0.005083ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (43) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:09,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.068456ms + 0.018174ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (28) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:09,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.086599ms + 0.027484ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (43) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:29:09,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pcr.PcrExprProcFactory: Processing (key) IN (RS[5]) org.apache.hadoop.hive.ql.udf.generic.GenericUDFIn outputs {UNKNOWN, Column[key]}, {UNKNOWN, RS[5]}, 2018-01-05T11:29:09,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 9 key:[] 2018-01-05T11:29:09,376 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 9 oldColExprMap: {VALUE._col0=Column[_col0]} 2018-01-05T11:29:09,376 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 9 newColExprMap: {VALUE._col0=Column[_col0]} 2018-01-05T11:29:09,376 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: JOIN 6 oldExprs: {0=[Column[KEY.reducesinkkey0]], 1=[Column[KEY.reducesinkkey0]]} 2018-01-05T11:29:09,376 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: JOIN 6 newExprs: {0=[], 1=[]} 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 4 key:[Column[_col0]] 2018-01-05T11:29:09,376 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 4 oldColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:09,376 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 4 newColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 5 key:[Column[_col0]] 2018-01-05T11:29:09,376 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 5 oldColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:09,376 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 5 newColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.IdentityProjectRemover: Identity project remover optimization removed : SEL[11] 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.IdentityProjectRemover: Identity project remover optimization removed : SEL[7] 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: After logical optimization TS[0]-FIL[14]-SEL[1]-RS[4]-JOIN[6]-GBY[8]-RS[9]-GBY[10]-FS[12] TS[2]-SEL[3]-RS[5]-JOIN[6] 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Parent: TS[0] 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Filter: (key) IN (RS[5]) (type: boolean) 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: TableScan: TS[0] 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Column key is not a partition column 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Initiate semijoin reduction for key ((key) IN (RS[5]) 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:09,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:09,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: DynamicSemiJoinPushdown: Saving RS to TS mapping: RS[19]: TS[0] 2018-01-05T11:29:09,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Estimated average row size: 4 2018-01-05T11:29:09,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,377 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_aggr_stats_for: db=default table=tab 2018-01-05T11:29:09,377 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_aggr_stats_for: db=default table=tab 2018-01-05T11:29:09,379 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.AggregateStatsCache: Returning aggregate stats from the cache; total hits: 11, total misses: 0, hit ratio: 1.0 2018-01-05T11:29:09,379 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: useDensityFunctionForNDVEstimation = false partsFound = 1 ColumnStatisticsObj = [ColumnStatisticsObj(colName:key, colType:int, statsData:)] 2018-01-05T11:29:09,380 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,380 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,380 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting stats (Num rows: 242 Data size: 968 Basic stats: COMPLETE Column stats: COMPLETE) on TS[0] 2018-01-05T11:29:09,380 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (tab): numRows: 242 dataSize: 968 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {key= colName: key colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:09,380 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,380 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,380 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partitions : db=default tbl=tab_part 2018-01-05T11:29:09,380 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions : db=default tbl=tab_part 2018-01-05T11:29:09,382 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.165407ms + 0.002694ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? ] 2018-01-05T11:29:09,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.127988ms + 0.042348ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (14) order by "PART_NAME" asc] 2018-01-05T11:29:09,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.075814ms + 0.052278ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (14) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:29:09,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.063378ms + 0.017192ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (14) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:09,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.057747ms + 0.005764ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (41) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:29:09,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.054601ms + 0.019505ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (41) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:09,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.055004ms + 0.015136ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (41) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:09,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.050417ms + 0.005453ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (41) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:09,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.077842ms + 0.019393ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (26) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:09,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.068947ms + 0.022066ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (41) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:29:09,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,386 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,387 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,387 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,387 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Estimated average row size: 4 2018-01-05T11:29:09,387 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,387 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_aggr_stats_for: db=default table=tab_part 2018-01-05T11:29:09,387 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_aggr_stats_for: db=default table=tab_part 2018-01-05T11:29:09,387 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000096, QueryId=djaiswal_20180105112908_5436fea8-dbcd-433f-8d85-23aef4a81827-39, DagId=39, VertexName=Reducer 3, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-1, Succeeded=true, StartTime=1515180549239, EndTime=1515180549387 2018-01-05T11:29:09,388 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.AggregateStatsCache: Returning aggregate stats from the cache; total hits: 12, total misses: 0, hit ratio: 1.0 2018-01-05T11:29:09,389 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: useDensityFunctionForNDVEstimation = false partsFound = 1 ColumnStatisticsObj = [ColumnStatisticsObj(colName:key, colType:int, statsData:)] 2018-01-05T11:29:09,389 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on TS[2] 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-TS[2] (tab_part): numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {key= colName: key colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_5_b_key_min), DynamicValue(RS_5_b_key_max)) Original num rows: 242 New num rows: 242 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_5_b_key_min), DynamicValue(RS_5_b_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_5_b_key_bloom_filter))) Original num rows: 242 New num rows: 242 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-FIL[14]: numRows: 242 dataSize: 968 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {key= colName: key colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting stats (Num rows: 242 Data size: 968 Basic stats: COMPLETE Column stats: COMPLETE) on FIL[14] 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[3] 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[3]: numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 242 Data size: 968 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[1] 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[1]: numRows: 242 dataSize: 968 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on RS[5] 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[5]: numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {KEY.reducesinkkey0= colName: KEY.reducesinkkey0 colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[15] 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[15]: numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 242 Data size: 968 Basic stats: COMPLETE Column stats: COMPLETE) on RS[4] 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[4]: numRows: 242 dataSize: 968 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {KEY.reducesinkkey0= colName: KEY.reducesinkkey0 colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-JOIN[6]: detects none/multiple PK parents. 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.JoinOperator: Setting stats (Num rows: 500 Data size: 4000 Basic stats: COMPLETE Column stats: COMPLETE) on JOIN[6] 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-JOIN[6]: numRows: 500 dataSize: 4000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[16]: inputSize: 2000 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[16] hashAgg: true 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 3] STATS-GBY[16]: cardinality: 1 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[16] 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[16]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[8]: inputSize: 4000 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[8] hashAgg: true 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 3] STATS-GBY[8]: cardinality: 1 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[8] 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[8]: numRows: 1 dataSize: 8 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on RS[17] 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[17]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE) on RS[9] 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[9]: numRows: 1 dataSize: 8 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {VALUE._col0= colName: VALUE._col0 colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[18]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[18] hashAgg: false 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 9] STATS-GBY[18]: cardinality: 1 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[18] 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[18]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[10]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[10] hashAgg: false 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 9] STATS-GBY[10]: cardinality: 1 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[10] 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[10]: numRows: 1 dataSize: 8 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on RS[19] 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[19]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:09,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-FS[12]: numRows: 1 dataSize: 8 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[key]]; bucket count: 2 }) on TS[0] 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[key]]; bucket count: 4 }) on TS[2] 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[key]]; bucket count: 2 }) on FIL[14] 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: 4 }) on SEL[3] 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: 2 }) on SEL[1] 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[KEY.reducesinkkey0]]; sort column names: [[KEY.reducesinkkey0]]; bucket count: 4 }) on RS[5] 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: 4 }) on SEL[15] 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[KEY.reducesinkkey0]]; sort column names: [[KEY.reducesinkkey0]]; bucket count: 2 }) on RS[4] 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.JoinOperator: Setting traits ({ bucket column names: [[], []]; sort column names: [[], []]; bucket count: -1 }) on JOIN[6] 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[16] 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[8] 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[17] 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[9] 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[18] 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[10] 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[19] 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FileSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on FS[12] 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,391 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Set parallelism for reduce sink RS[5] to: 1 2018-01-05T11:29:09,391 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Set parallelism for reduce sink RS[4] to: 1 2018-01-05T11:29:09,391 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConvertJoinMapJoin: Memory monitor info set to : { isLlap: true executorsPerNode: 4 maxExecutorsOverSubscribeMemory: 3 memoryOverSubscriptionFactor: 0.20000000298023224 memoryCheckInterval: 100000 noConditionalTaskSize: 10000000 adjustedNoConditionalTaskSize: 16000000 hashTableInflationFactor: 2.0 threshold: 32000000 } 2018-01-05T11:29:09,391 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConvertJoinMapJoin: Fallback to common merge join operator 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.CommonMergeJoinOperator: Setting traits ({ bucket column names: [[], []]; sort column names: [[], []]; bucket count: 0 }) on MERGEJOIN[20] 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.CommonMergeJoinOperator: Setting stats (Num rows: 500 Data size: 4000 Basic stats: COMPLETE Column stats: COMPLETE) on MERGEJOIN[20] 2018-01-05T11:29:09,391 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:09,391 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:09,391 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing BloomFilter cost/benefit for RS[19] - TS[0] (a) (Column[key]) 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing key domain cardinality, keyDomainCardinality=461, semiJoinKeyIsPK=false, selColStat= colName: _col0 colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false, selColSourceStat= colName: key colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false, tsColStat= colName: key colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit for Column[_col0] to Column[key], selKeyCardinality=308, tsKeyCardinality=153, tsRows=242, keyDomainCardinality=461 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: SemiJoin key selectivity=0.6681127982646421, benefit=80.31670281995662 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit=80.31670281995662, cost=500.0, tsDataSize=242, netBenefit=-419.6832971800434 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: netBenefit=-1.7342285007439808 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Reduction factor not satisfied for RS[19]-TS[0] (a). Removing semijoin optimization. 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Removing ReduceSink RS[19] and TableScan TS[0] 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: FS, 12 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: GBY, 10 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 9 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: GBY, 8 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: MERGEJOIN, 20 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 4 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 1 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: FIL, 14 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:29:09,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 5 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 3 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: TS, 2 2018-01-05T11:29:09,392 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Cycle free: true 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: Before SharedWorkOptimizer: TS[0]-FIL[14]-SEL[1]-RS[4]-MERGEJOIN[20]-GBY[8]-RS[9]-GBY[10]-FS[12] TS[2]-SEL[3]-RS[5]-MERGEJOIN[20] 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: DPP information stored in the cache: {} 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: Sorted tables by size: [default.tab_part=2000, default.tab=968] 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: After SharedWorkOptimizer: TS[0]-FIL[14]-SEL[1]-RS[4]-MERGEJOIN[20]-GBY[8]-RS[9]-GBY[10]-FS[12] TS[2]-SEL[3]-RS[5]-MERGEJOIN[20] 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: int|{a}key,value: string|{a}value,ds: string|{a}ds,BLOCK__OFFSET__INSIDE__FILE: bigint|{a}block__offset__inside__file,INPUT__FILE__NAME: string|{a}input__file__name,ROW__ID: struct|{a}row__id) 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[2] with rs:(key: int|{b}key,value: string|{b}value,ds: string|{b}ds,BLOCK__OFFSET__INSIDE__FILE: bigint|{b}block__offset__inside__file,INPUT__FILE__NAME: string|{b}input__file__name,ROW__ID: struct|{b}row__id) 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FIL[14] with rs:(key: int|{a}key) 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FIL[14] 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Old filter FIL[14] conditions:(true and true) 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(Const boolean true, Const boolean true) -> Const boolean true 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Filter expression GenericUDFOPAnd(Const boolean true, Const boolean true) holds true. Will delete it. 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New filter FIL[14] conditions:true 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[3] with rs:(_col0: int|{$hdt$_1}key) 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_2 {} 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[3] 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key]) 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(_col0: int|{$hdt$_0}key) 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op FIL_14 {} 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key]) 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[5] with rs:(KEY.reducesinkkey0: int|{$hdt$_1}key) 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_3 {} 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[5] 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[4] with rs:(KEY.reducesinkkey0: int|{$hdt$_0}key) 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[4] 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:MERGEJOIN[20] with rs:() 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_4 {} 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_5 {} 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator MERGEJOIN[20] 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[8] with rs:(_col0: bigint|{}(tok_functionstar count)) 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op MERGEJOIN_20 {} 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[8] 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[9] with rs:(VALUE._col0: bigint|{}(tok_functionstar count)) 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_8 {} 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[9] 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[10] with rs:(_col0: bigint|{}(tok_functionstar count)) 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_9 {} 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[10] 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[12] with rs:(_col0: bigint|{}) 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_10 {} 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[12] 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: TS[0] 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[4] 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 of table a 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[4] 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: MERGEJOIN[20] 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[9] 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding reduce work (Reducer 2) for MERGEJOIN[20] 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Setting up reduce sink: RS[4] with following reduce work: Reducer 2 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[4] as parent from MERGEJOIN[20] 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[5] as parent from MERGEJOIN[20] 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[9] 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: GBY[10] 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: FS[12] 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding reduce work (Reducer 3) for GBY[10] 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Setting up reduce sink: RS[9] with following reduce work: Reducer 3 2018-01-05T11:29:09,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[9] as parent from GBY[10] 2018-01-05T11:29:09,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: FS[12] 2018-01-05T11:29:09,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: TS[2] 2018-01-05T11:29:09,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[5] 2018-01-05T11:29:09,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding map work (Map 4) for TS[2] 2018-01-05T11:29:09,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 of table b 2018-01-05T11:29:09,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:09,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Second pass. Leaf operator: RS[5] has common downstream work: org.apache.hadoop.hive.ql.plan.MergeJoinWork@6907aa14 2018-01-05T11:29:09,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:29:09,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:29:09,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:09,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:09,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 4 2018-01-05T11:29:09,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:09,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:09,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:29:09,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:09,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:09,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 4 2018-01-05T11:29:09,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:09,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:09,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:29:09,393 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Examining input format to see if vectorization is enabled. 2018-01-05T11:29:09,393 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorization is enabled for input format(s) [org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] 2018-01-05T11:29:09,393 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing MapWork... 2018-01-05T11:29:09,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[key], Vectorized Expression = col 0:int 2018-01-05T11:29:09,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator 2018-01-05T11:29:09,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.SelectDesc 2018-01-05T11:29:09,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:int 2018-01-05T11:29:09,393 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator groupby typeName int 2018-01-05T11:29:09,393 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkLongOperator 2018-01-05T11:29:09,393 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:09,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkLongOperator 2018-01-05T11:29:09,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:09,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [key, value, ds, ROW__ID] 2018-01-05T11:29:09,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [int, string, string, struct] 2018-01-05T11:29:09,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE, NONE, NONE, NONE] 2018-01-05T11:29:09,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 1 2018-01-05T11:29:09,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 1 2018-01-05T11:29:09,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:09,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:09,393 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Examining input format to see if vectorization is enabled. 2018-01-05T11:29:09,394 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorization is enabled for input format(s) [org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] 2018-01-05T11:29:09,394 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing MapWork... 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[key], Vectorized Expression = col 0:int 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.SelectDesc 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:int 2018-01-05T11:29:09,394 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator groupby typeName int 2018-01-05T11:29:09,394 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkLongOperator 2018-01-05T11:29:09,394 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkLongOperator 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [key, value, ds, ROW__ID] 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [int, string, string, struct] 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE, NONE, NONE, NONE] 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 1 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 1 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Using reduce tag 0 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0] columnTypes=[bigint] 2018-01-05T11:29:09,394 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing ReduceWork... 2018-01-05T11:29:09,394 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vector GROUP BY operator will use processing mode GLOBAL 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[VALUE._col0], Vectorized Expression = col 0:bigint 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.GroupByDesc 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorFileSinkOperator 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.FileSinkDesc 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [VALUE._col0] 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [bigint] 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE] 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 0 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 0 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:29:09,394 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: llap mode: only 2018-01-05T11:29:09,394 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key' 2018-01-05T11:29:09,394 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@7fe5d800 can be done in LLAP 2018-01-05T11:29:09,394 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Map 4 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key' 2018-01-05T11:29:09,394 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@8e35473 can be done in LLAP 2018-01-05T11:29:09,394 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Reducer 2 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'count()' 2018-01-05T11:29:09,394 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MergeJoinWork@6907aa14 can be done in LLAP 2018-01-05T11:29:09,394 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Reducer 3 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'count(VALUE._col0)' 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:29:09,394 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@5997a299 can be done in LLAP 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,394 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed plan generation 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: validation start 2018-01-05T11:29:09,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:29:09,395 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:09,395 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,395 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:Explain, type:string, comment:null)], properties:null) 2018-01-05T11:29:09,395 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,395 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:09,395 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {getNotNullConstraints_(NotNullConstraintsRequest, )=1, isCompatibleWith_(Configuration, )=0, getAggrColStatsFor_(String, String, List, List, )=6, getTable_(String, String, )=13, flushCache_()=0, getUniqueConstraints_(UniqueConstraintsRequest, )=0, getPrimaryKeys_(PrimaryKeysRequest, )=2, listPartitions_(String, String, short, )=13, getForeignKeys_(ForeignKeysRequest, )=0} 2018-01-05T11:29:09,395 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112909_558f83a9-3340-4175-ba52-6cd82364c2ac); Time taken: 0.087 seconds 2018-01-05T11:29:09,395 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:09,395 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,395 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112909_558f83a9-3340-4175-ba52-6cd82364c2ac): explain vectorization detail select count (*) from tab a right outer join tab_part b on a.key = b.key 2018-01-05T11:29:09,396 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,396 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: explain vectorization detail select count (*) from tab a right outer join tab_part b on a.key = b.key 2018-01-05T11:29:09,396 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:29:09,396 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,396 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,396 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,396 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,396 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-3:EXPLAIN] in serial mode 2018-01-05T11:29:09,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,432 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: explain vectorization detail select count (*) from tab a right outer join tab_part b on a.key = b.key 2018-01-05T11:29:09,432 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:29:09,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,432 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:09,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {} 2018-01-05T11:29:09,432 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112909_558f83a9-3340-4175-ba52-6cd82364c2ac); Time taken: 0.037 seconds 2018-01-05T11:29:09,432 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:09,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,432 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:09,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query explain vectorization detail select count (*) from tab a right outer join tab_part b on a.key = b.key 2018-01-05T11:29:09,434 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting result file: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_308_474189882829300338-1/-local-10004 2018-01-05T11:29:09,434 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_308_474189882829300338-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-09_308_474189882829300338-1 2018-01-05T11:29:09,434 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_308_474189882829300338-1 2018-01-05T11:29:09,435 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.124 seconds, Fetched: 171 row(s) 2018-01-05T11:29:09,435 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:09,435 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:09,435 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:09,435 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:09,435 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,435 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:09,435 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,435 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,435 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,435 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: select count (*) from tab a right outer join tab_part b on a.key = b.key 2018-01-05T11:29:09,435 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd): select count (*) from tab a right outer join tab_part b on a.key = b.key 2018-01-05T11:29:09,435 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,435 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: select count (*) from tab a right outer join tab_part b on a.key = b.key 2018-01-05T11:29:09,435 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:09,435 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,435 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,436 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,436 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,436 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:09,436 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:29:09,436 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:09,436 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed phase 1 of Semantic Analysis 2018-01-05T11:29:09,436 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:09,436 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,436 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab 2018-01-05T11:29:09,436 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab 2018-01-05T11:29:09,438 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,439 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,439 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,439 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab_part 2018-01-05T11:29:09,439 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab_part 2018-01-05T11:29:09,441 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,441 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,441 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:09,442 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:09,453 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed getting MetaData in Semantic Analysis 2018-01-05T11:29:09,454 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,455 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,455 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_not_null_constraints : db=default tbl=tab 2018-01-05T11:29:09,455 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_not_null_constraints : db=default tbl=tab 2018-01-05T11:29:09,455 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,455 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,455 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,455 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab 2018-01-05T11:29:09,455 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab 2018-01-05T11:29:09,455 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,456 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,456 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,456 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab 2018-01-05T11:29:09,456 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab 2018-01-05T11:29:09,456 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,456 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,456 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,456 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_unique_constraints : db=default tbl=tab 2018-01-05T11:29:09,456 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_unique_constraints : db=default tbl=tab 2018-01-05T11:29:09,456 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,456 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,456 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,456 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab 2018-01-05T11:29:09,456 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab 2018-01-05T11:29:09,456 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,456 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,457 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,457 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_not_null_constraints : db=default tbl=tab_part 2018-01-05T11:29:09,457 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_not_null_constraints : db=default tbl=tab_part 2018-01-05T11:29:09,457 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,457 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,457 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,457 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:09,457 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:09,457 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,457 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,457 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,457 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:09,457 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:09,457 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,457 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,457 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,457 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_unique_constraints : db=default tbl=tab_part 2018-01-05T11:29:09,457 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_unique_constraints : db=default tbl=tab_part 2018-01-05T11:29:09,458 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,458 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,458 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,458 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab_part 2018-01-05T11:29:09,458 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab_part 2018-01-05T11:29:09,458 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,458 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,458 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:09,458 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:09,459 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:09,459 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:29:09,459 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,468 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.sql2rel: Plan after trimming unused fields HiveAggregate(group=[{}], agg#0=[count()]) HiveProject(DUMMY=[0]) HiveJoin(condition=[=($0, $1)], joinType=[right], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:09,468 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan before removing subquery: HiveProject(_o__c0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveProject($f0=[$0]) HiveJoin(condition=[=($0, $6)], joinType=[right], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:09,468 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan just after removing subquery: HiveProject(_o__c0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveProject($f0=[$0]) HiveJoin(condition=[=($0, $6)], joinType=[right], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:09,468 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan after decorrelation: HiveProject(_o__c0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveProject($f0=[$0]) HiveJoin(condition=[=($0, $6)], joinType=[right], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:09,468 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,469 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,469 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,469 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,469 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,469 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,469 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,469 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,469 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,470 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,470 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,479 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,479 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,479 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,479 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,480 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.sql2rel: Plan after trimming unused fields HiveAggregate(group=[{}], agg#0=[count()]) HiveProject(DUMMY=[0]) HiveJoin(condition=[=($0, $1)], joinType=[right], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:09,480 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,480 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,480 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,480 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,481 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,481 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,483 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,483 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,484 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,484 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,484 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,484 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,484 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,484 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,484 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,491 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Original plan for PlanModifier HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[right], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:09,491 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after top-level introduceDerivedTable HiveProject($f0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[right], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:09,491 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after nested convertOpTree HiveProject($f0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[right], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:09,491 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after propagating order HiveProject($f0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[right], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:09,491 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after fixTopOBSchema HiveProject($f0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[right], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:09,491 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Final plan after modifier HiveProject(_c0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[right], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:09,492 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:09,492 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:09,492 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:09,492 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:09,492 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,492 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab 2018-01-05T11:29:09,492 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab 2018-01-05T11:29:09,495 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,495 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,495 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:09,495 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:09,495 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:09,495 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,495 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab_part 2018-01-05T11:29:09,495 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab_part 2018-01-05T11:29:09,497 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,498 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,498 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:09,498 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:09,498 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:09,498 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Table Plan for a TS[0] 2018-01-05T11:29:09,498 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB a{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB a{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:09,498 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (. (tok_table_or_col a) key) key)) 2018-01-05T11:29:09,498 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = a{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} starRr = null 2018-01-05T11:29:09,498 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(key,_col0: int)} 2018-01-05T11:29:09,498 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:09,498 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block $hdt$_0 2018-01-05T11:29:09,498 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block $hdt$_0 2018-01-05T11:29:09,498 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Table Plan for b TS[2] 2018-01-05T11:29:09,498 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB b{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB b{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:09,498 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (. (tok_table_or_col b) key) key)) 2018-01-05T11:29:09,498 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = b{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} starRr = null 2018-01-05T11:29:09,498 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(key,_col0: int)} 2018-01-05T11:29:09,498 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:09,498 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block $hdt$_1 2018-01-05T11:29:09,498 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block $hdt$_1 2018-01-05T11:29:09,499 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:09,499 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB $hdt$_0{(key,_col0: int)} $hdt$_1{(key,_col1: int)} after GB {((tok_functionstar count),_col0: bigint)} 2018-01-05T11:29:09,499 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (tok_functionstar count) _c0)) 2018-01-05T11:29:09,499 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = {((tok_functionstar count),_col0: bigint)} starRr = $hdt$_0{(key,_col0: int)} $hdt$_1{(key,_col1: int)} 2018-01-05T11:29:09,499 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(_c0,_col0: bigint)} 2018-01-05T11:29:09,499 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:09,499 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-09_435_1124461022760547009-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/-mr-10001 2018-01-05T11:29:09,499 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-09_435_1124461022760547009-1 2018-01-05T11:29:09,522 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/-mr-10003 2018-01-05T11:29:09,522 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/-mr-10001 row schema: null{(_c0,_col0: bigint)} 2018-01-05T11:29:09,522 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block null 2018-01-05T11:29:09,522 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:29:09,522 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: CBO Succeeded; optimized logical plan. 2018-01-05T11:29:09,522 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Before logical optimization TS[0]-SEL[1]-RS[4]-JOIN[6]-SEL[7]-GBY[8]-RS[9]-GBY[10]-SEL[11]-FS[12] TS[2]-SEL[3]-RS[5]-JOIN[6] 2018-01-05T11:29:09,522 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,522 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,522 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,523 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] lineage.Generator: Time taken for lineage transform=1 2018-01-05T11:29:09,523 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,523 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,523 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,523 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,523 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SyntheticJoinPredicate: Synthetic predicate: 0 --> 1 2018-01-05T11:29:09,523 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,523 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,523 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FS(12) 2018-01-05T11:29:09,523 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(11) 2018-01-05T11:29:09,523 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for GBY(10) 2018-01-05T11:29:09,523 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(9) 2018-01-05T11:29:09,523 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for GBY(8) 2018-01-05T11:29:09,523 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(7) 2018-01-05T11:29:09,523 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for JOIN(6) 2018-01-05T11:29:09,523 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(4) 2018-01-05T11:29:09,523 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(13) 2018-01-05T11:29:09,523 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias $hdt$_0: (_col0) IN (RS[5]) 2018-01-05T11:29:09,523 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(1) 2018-01-05T11:29:09,523 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of SEL for alias $hdt$_0: (key) IN (RS[5]) 2018-01-05T11:29:09,523 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:29:09,523 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of TS for alias a: (key) IN (RS[5]) 2018-01-05T11:29:09,523 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(5) 2018-01-05T11:29:09,523 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(3) 2018-01-05T11:29:09,523 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for TS(2) 2018-01-05T11:29:09,523 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SimplePredicatePushDown: After PPD: TS[0]-FIL[14]-SEL[1]-RS[4]-JOIN[6]-SEL[7]-GBY[8]-RS[9]-GBY[10]-SEL[11]-FS[12] TS[2]-SEL[3]-RS[5]-JOIN[6] 2018-01-05T11:29:09,523 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,523 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,523 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,523 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,524 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,524 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,524 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,524 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,524 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,524 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,524 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partitions : db=default tbl=tab 2018-01-05T11:29:09,524 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions : db=default tbl=tab 2018-01-05T11:29:09,526 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,527 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.159258ms + 0.003478ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? ] 2018-01-05T11:29:09,527 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.120706ms + 0.050552ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (15) order by "PART_NAME" asc] 2018-01-05T11:29:09,527 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.072025ms + 0.054223ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (15) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:29:09,527 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.058949ms + 0.015912ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (15) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:09,527 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.053227ms + 0.005492ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (43) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:29:09,527 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.051412ms + 0.01788ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (43) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:09,528 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.053454ms + 0.014551ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (43) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:09,528 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.047364ms + 0.005142ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (43) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:09,528 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.072131ms + 0.018317ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (28) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:09,528 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.057162ms + 0.017128ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (43) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:29:09,528 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,530 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,531 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,531 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,531 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pcr.PcrExprProcFactory: Processing (key) IN (RS[5]) org.apache.hadoop.hive.ql.udf.generic.GenericUDFIn outputs {UNKNOWN, Column[key]}, {UNKNOWN, RS[5]}, 2018-01-05T11:29:09,531 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,531 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,531 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,531 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,531 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 9 key:[] 2018-01-05T11:29:09,531 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 9 oldColExprMap: {VALUE._col0=Column[_col0]} 2018-01-05T11:29:09,531 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 9 newColExprMap: {VALUE._col0=Column[_col0]} 2018-01-05T11:29:09,531 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: JOIN 6 oldExprs: {0=[Column[KEY.reducesinkkey0]], 1=[Column[KEY.reducesinkkey0]]} 2018-01-05T11:29:09,531 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: JOIN 6 newExprs: {0=[], 1=[]} 2018-01-05T11:29:09,531 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 4 key:[Column[_col0]] 2018-01-05T11:29:09,531 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 4 oldColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:09,531 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 4 newColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:09,531 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 5 key:[Column[_col0]] 2018-01-05T11:29:09,531 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 5 oldColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:09,531 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 5 newColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:09,531 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,531 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,531 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,531 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,531 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,531 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,531 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,531 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,531 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,531 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,531 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,531 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,531 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,531 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,531 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,531 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,531 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.IdentityProjectRemover: Identity project remover optimization removed : SEL[11] 2018-01-05T11:29:09,531 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.IdentityProjectRemover: Identity project remover optimization removed : SEL[7] 2018-01-05T11:29:09,531 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,531 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,532 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,532 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,532 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,532 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,532 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,532 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: After logical optimization TS[0]-FIL[14]-SEL[1]-RS[4]-JOIN[6]-GBY[8]-RS[9]-GBY[10]-FS[12] TS[2]-SEL[3]-RS[5]-JOIN[6] 2018-01-05T11:29:09,532 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,532 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Parent: TS[0] 2018-01-05T11:29:09,532 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Filter: (key) IN (RS[5]) (type: boolean) 2018-01-05T11:29:09,532 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: TableScan: TS[0] 2018-01-05T11:29:09,532 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Column key is not a partition column 2018-01-05T11:29:09,532 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Initiate semijoin reduction for key ((key) IN (RS[5]) 2018-01-05T11:29:09,532 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:09,532 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:09,532 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:09,532 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:09,532 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:09,532 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:09,532 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: DynamicSemiJoinPushdown: Saving RS to TS mapping: RS[19]: TS[0] 2018-01-05T11:29:09,532 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,532 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,533 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Estimated average row size: 4 2018-01-05T11:29:09,533 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,533 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_aggr_stats_for: db=default table=tab 2018-01-05T11:29:09,533 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_aggr_stats_for: db=default table=tab 2018-01-05T11:29:09,534 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.AggregateStatsCache: Returning aggregate stats from the cache; total hits: 13, total misses: 0, hit ratio: 1.0 2018-01-05T11:29:09,534 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: useDensityFunctionForNDVEstimation = false partsFound = 1 ColumnStatisticsObj = [ColumnStatisticsObj(colName:key, colType:int, statsData:)] 2018-01-05T11:29:09,535 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,535 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,535 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting stats (Num rows: 242 Data size: 968 Basic stats: COMPLETE Column stats: COMPLETE) on TS[0] 2018-01-05T11:29:09,535 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (tab): numRows: 242 dataSize: 968 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {key= colName: key colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:09,535 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,535 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,535 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partitions : db=default tbl=tab_part 2018-01-05T11:29:09,535 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions : db=default tbl=tab_part 2018-01-05T11:29:09,537 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,538 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.331341ms + 0.002707ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? ] 2018-01-05T11:29:09,538 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.130965ms + 0.040593ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (14) order by "PART_NAME" asc] 2018-01-05T11:29:09,538 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.066512ms + 0.04713ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (14) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:29:09,539 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.065669ms + 0.01566ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (14) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:09,539 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.052321ms + 0.005667ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (41) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:29:09,539 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.050792ms + 0.016878ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (41) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:09,539 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.050717ms + 0.013841ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (41) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:09,539 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.047021ms + 0.00507ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (41) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:09,539 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.0715ms + 0.017305ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (26) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:09,539 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.057786ms + 0.016756ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (41) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:29:09,539 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,541 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,542 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,542 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,542 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Estimated average row size: 4 2018-01-05T11:29:09,542 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,542 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_aggr_stats_for: db=default table=tab_part 2018-01-05T11:29:09,542 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_aggr_stats_for: db=default table=tab_part 2018-01-05T11:29:09,545 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.AggregateStatsCache: Returning aggregate stats from the cache; total hits: 14, total misses: 0, hit ratio: 1.0 2018-01-05T11:29:09,545 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: useDensityFunctionForNDVEstimation = false partsFound = 1 ColumnStatisticsObj = [ColumnStatisticsObj(colName:key, colType:int, statsData:)] 2018-01-05T11:29:09,545 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:09,546 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:09,546 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on TS[2] 2018-01-05T11:29:09,546 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-TS[2] (tab_part): numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {key= colName: key colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:09,546 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_5_b_key_min), DynamicValue(RS_5_b_key_max)) Original num rows: 242 New num rows: 242 2018-01-05T11:29:09,546 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_5_b_key_min), DynamicValue(RS_5_b_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_5_b_key_bloom_filter))) Original num rows: 242 New num rows: 242 2018-01-05T11:29:09,546 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-FIL[14]: numRows: 242 dataSize: 968 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {key= colName: key colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:09,546 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting stats (Num rows: 242 Data size: 968 Basic stats: COMPLETE Column stats: COMPLETE) on FIL[14] 2018-01-05T11:29:09,546 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[3] 2018-01-05T11:29:09,546 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[3]: numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:09,546 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 242 Data size: 968 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[1] 2018-01-05T11:29:09,546 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[1]: numRows: 242 dataSize: 968 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:09,546 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on RS[5] 2018-01-05T11:29:09,546 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[5]: numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {KEY.reducesinkkey0= colName: KEY.reducesinkkey0 colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:09,546 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[15] 2018-01-05T11:29:09,546 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[15]: numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:09,546 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 242 Data size: 968 Basic stats: COMPLETE Column stats: COMPLETE) on RS[4] 2018-01-05T11:29:09,546 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[4]: numRows: 242 dataSize: 968 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {KEY.reducesinkkey0= colName: KEY.reducesinkkey0 colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:09,546 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-JOIN[6]: detects none/multiple PK parents. 2018-01-05T11:29:09,546 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.JoinOperator: Setting stats (Num rows: 500 Data size: 4000 Basic stats: COMPLETE Column stats: COMPLETE) on JOIN[6] 2018-01-05T11:29:09,546 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-JOIN[6]: numRows: 500 dataSize: 4000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:09,546 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[16]: inputSize: 2000 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:09,546 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[16] hashAgg: true 2018-01-05T11:29:09,546 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 3] STATS-GBY[16]: cardinality: 1 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[16] 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[16]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[8]: inputSize: 4000 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[8] hashAgg: true 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 3] STATS-GBY[8]: cardinality: 1 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[8] 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[8]: numRows: 1 dataSize: 8 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on RS[17] 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[17]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE) on RS[9] 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[9]: numRows: 1 dataSize: 8 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {VALUE._col0= colName: VALUE._col0 colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[18]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[18] hashAgg: false 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 9] STATS-GBY[18]: cardinality: 1 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[18] 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[18]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[10]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[10] hashAgg: false 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 9] STATS-GBY[10]: cardinality: 1 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[10] 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[10]: numRows: 1 dataSize: 8 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on RS[19] 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[19]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-FS[12]: numRows: 1 dataSize: 8 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[key]]; bucket count: 2 }) on TS[0] 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[key]]; bucket count: 4 }) on TS[2] 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[key]]; bucket count: 2 }) on FIL[14] 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: 4 }) on SEL[3] 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: 2 }) on SEL[1] 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[KEY.reducesinkkey0]]; sort column names: [[KEY.reducesinkkey0]]; bucket count: 4 }) on RS[5] 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: 4 }) on SEL[15] 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[KEY.reducesinkkey0]]; sort column names: [[KEY.reducesinkkey0]]; bucket count: 2 }) on RS[4] 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.JoinOperator: Setting traits ({ bucket column names: [[], []]; sort column names: [[], []]; bucket count: -1 }) on JOIN[6] 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[16] 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[8] 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[17] 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[9] 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[18] 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[10] 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[19] 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FileSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on FS[12] 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,547 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Set parallelism for reduce sink RS[5] to: 1 2018-01-05T11:29:09,547 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Set parallelism for reduce sink RS[4] to: 1 2018-01-05T11:29:09,547 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConvertJoinMapJoin: Memory monitor info set to : { isLlap: true executorsPerNode: 4 maxExecutorsOverSubscribeMemory: 3 memoryOverSubscriptionFactor: 0.20000000298023224 memoryCheckInterval: 100000 noConditionalTaskSize: 10000000 adjustedNoConditionalTaskSize: 16000000 hashTableInflationFactor: 2.0 threshold: 32000000 } 2018-01-05T11:29:09,547 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConvertJoinMapJoin: Fallback to common merge join operator 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.CommonMergeJoinOperator: Setting traits ({ bucket column names: [[], []]; sort column names: [[], []]; bucket count: 0 }) on MERGEJOIN[20] 2018-01-05T11:29:09,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.CommonMergeJoinOperator: Setting stats (Num rows: 500 Data size: 4000 Basic stats: COMPLETE Column stats: COMPLETE) on MERGEJOIN[20] 2018-01-05T11:29:09,547 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:09,548 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:09,548 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing BloomFilter cost/benefit for RS[19] - TS[0] (a) (Column[key]) 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing key domain cardinality, keyDomainCardinality=461, semiJoinKeyIsPK=false, selColStat= colName: _col0 colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false, selColSourceStat= colName: key colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false, tsColStat= colName: key colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit for Column[_col0] to Column[key], selKeyCardinality=308, tsKeyCardinality=153, tsRows=242, keyDomainCardinality=461 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: SemiJoin key selectivity=0.6681127982646421, benefit=80.31670281995662 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit=80.31670281995662, cost=500.0, tsDataSize=242, netBenefit=-419.6832971800434 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: netBenefit=-1.7342285007439808 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Reduction factor not satisfied for RS[19]-TS[0] (a). Removing semijoin optimization. 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Removing ReduceSink RS[19] and TableScan TS[0] 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: FS, 12 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: GBY, 10 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 9 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: GBY, 8 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: MERGEJOIN, 20 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 4 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 1 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: FIL, 14 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 5 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 3 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: TS, 2 2018-01-05T11:29:09,548 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Cycle free: true 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: Before SharedWorkOptimizer: TS[0]-FIL[14]-SEL[1]-RS[4]-MERGEJOIN[20]-GBY[8]-RS[9]-GBY[10]-FS[12] TS[2]-SEL[3]-RS[5]-MERGEJOIN[20] 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: DPP information stored in the cache: {} 2018-01-05T11:29:09,548 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: Sorted tables by size: [default.tab_part=2000, default.tab=968] 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: After SharedWorkOptimizer: TS[0]-FIL[14]-SEL[1]-RS[4]-MERGEJOIN[20]-GBY[8]-RS[9]-GBY[10]-FS[12] TS[2]-SEL[3]-RS[5]-MERGEJOIN[20] 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: int|{a}key,value: string|{a}value,ds: string|{a}ds,BLOCK__OFFSET__INSIDE__FILE: bigint|{a}block__offset__inside__file,INPUT__FILE__NAME: string|{a}input__file__name,ROW__ID: struct|{a}row__id) 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[2] with rs:(key: int|{b}key,value: string|{b}value,ds: string|{b}ds,BLOCK__OFFSET__INSIDE__FILE: bigint|{b}block__offset__inside__file,INPUT__FILE__NAME: string|{b}input__file__name,ROW__ID: struct|{b}row__id) 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FIL[14] with rs:(key: int|{a}key) 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FIL[14] 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Old filter FIL[14] conditions:(true and true) 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(Const boolean true, Const boolean true) -> Const boolean true 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Filter expression GenericUDFOPAnd(Const boolean true, Const boolean true) holds true. Will delete it. 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New filter FIL[14] conditions:true 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[3] with rs:(_col0: int|{$hdt$_1}key) 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_2 {} 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[3] 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key]) 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(_col0: int|{$hdt$_0}key) 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op FIL_14 {} 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key]) 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[5] with rs:(KEY.reducesinkkey0: int|{$hdt$_1}key) 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_3 {} 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[5] 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[4] with rs:(KEY.reducesinkkey0: int|{$hdt$_0}key) 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[4] 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:MERGEJOIN[20] with rs:() 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_4 {} 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_5 {} 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator MERGEJOIN[20] 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[8] with rs:(_col0: bigint|{}(tok_functionstar count)) 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op MERGEJOIN_20 {} 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[8] 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[9] with rs:(VALUE._col0: bigint|{}(tok_functionstar count)) 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_8 {} 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[9] 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[10] with rs:(_col0: bigint|{}(tok_functionstar count)) 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_9 {} 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[10] 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[12] with rs:(_col0: bigint|{}) 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_10 {} 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[12] 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: TS[0] 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[4] 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:29:09,549 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 of table a 2018-01-05T11:29:09,550 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:09,550 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[4] 2018-01-05T11:29:09,550 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: MERGEJOIN[20] 2018-01-05T11:29:09,550 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[9] 2018-01-05T11:29:09,550 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding reduce work (Reducer 2) for MERGEJOIN[20] 2018-01-05T11:29:09,550 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Setting up reduce sink: RS[4] with following reduce work: Reducer 2 2018-01-05T11:29:09,550 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[4] as parent from MERGEJOIN[20] 2018-01-05T11:29:09,550 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[5] as parent from MERGEJOIN[20] 2018-01-05T11:29:09,550 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[9] 2018-01-05T11:29:09,550 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: GBY[10] 2018-01-05T11:29:09,550 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: FS[12] 2018-01-05T11:29:09,550 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding reduce work (Reducer 3) for GBY[10] 2018-01-05T11:29:09,550 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Setting up reduce sink: RS[9] with following reduce work: Reducer 3 2018-01-05T11:29:09,550 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[9] as parent from GBY[10] 2018-01-05T11:29:09,550 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: FS[12] 2018-01-05T11:29:09,550 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: TS[2] 2018-01-05T11:29:09,550 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[5] 2018-01-05T11:29:09,550 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding map work (Map 4) for TS[2] 2018-01-05T11:29:09,550 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 of table b 2018-01-05T11:29:09,550 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:09,550 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Second pass. Leaf operator: RS[5] has common downstream work: org.apache.hadoop.hive.ql.plan.MergeJoinWork@1ff98d32 2018-01-05T11:29:09,550 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:29:09,550 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,550 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,550 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:29:09,550 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:09,550 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:09,550 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 4 2018-01-05T11:29:09,550 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:09,550 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:09,550 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:29:09,550 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:09,550 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:09,550 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 4 2018-01-05T11:29:09,550 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:09,550 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:09,550 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:29:09,550 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Examining input format to see if vectorization is enabled. 2018-01-05T11:29:09,551 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorization is enabled for input format(s) [org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] 2018-01-05T11:29:09,551 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing MapWork... 2018-01-05T11:29:09,551 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[key], Vectorized Expression = col 0:int 2018-01-05T11:29:09,551 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator 2018-01-05T11:29:09,551 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.SelectDesc 2018-01-05T11:29:09,551 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:int 2018-01-05T11:29:09,551 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator groupby typeName int 2018-01-05T11:29:09,551 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkLongOperator 2018-01-05T11:29:09,551 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:09,551 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkLongOperator 2018-01-05T11:29:09,551 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:09,551 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [key, value, ds, ROW__ID] 2018-01-05T11:29:09,551 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [int, string, string, struct] 2018-01-05T11:29:09,551 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE, NONE, NONE, NONE] 2018-01-05T11:29:09,551 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 1 2018-01-05T11:29:09,551 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 1 2018-01-05T11:29:09,551 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:09,551 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:09,551 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Examining input format to see if vectorization is enabled. 2018-01-05T11:29:09,551 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorization is enabled for input format(s) [org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] 2018-01-05T11:29:09,551 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing MapWork... 2018-01-05T11:29:09,551 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[key], Vectorized Expression = col 0:int 2018-01-05T11:29:09,551 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator 2018-01-05T11:29:09,551 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.SelectDesc 2018-01-05T11:29:09,551 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:int 2018-01-05T11:29:09,551 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator groupby typeName int 2018-01-05T11:29:09,551 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkLongOperator 2018-01-05T11:29:09,551 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:09,551 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkLongOperator 2018-01-05T11:29:09,551 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:09,551 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [key, value, ds, ROW__ID] 2018-01-05T11:29:09,551 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [int, string, string, struct] 2018-01-05T11:29:09,551 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE, NONE, NONE, NONE] 2018-01-05T11:29:09,551 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 1 2018-01-05T11:29:09,551 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 1 2018-01-05T11:29:09,551 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:09,551 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:09,551 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Using reduce tag 0 2018-01-05T11:29:09,551 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0] columnTypes=[bigint] 2018-01-05T11:29:09,551 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing ReduceWork... 2018-01-05T11:29:09,551 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vector GROUP BY operator will use processing mode GLOBAL 2018-01-05T11:29:09,551 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[VALUE._col0], Vectorized Expression = col 0:bigint 2018-01-05T11:29:09,551 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator 2018-01-05T11:29:09,551 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.GroupByDesc 2018-01-05T11:29:09,551 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorFileSinkOperator 2018-01-05T11:29:09,551 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.FileSinkDesc 2018-01-05T11:29:09,552 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [VALUE._col0] 2018-01-05T11:29:09,552 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [bigint] 2018-01-05T11:29:09,552 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE] 2018-01-05T11:29:09,552 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 0 2018-01-05T11:29:09,552 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 0 2018-01-05T11:29:09,552 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:09,552 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:09,552 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:29:09,552 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: llap mode: only 2018-01-05T11:29:09,552 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:29:09,552 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key' 2018-01-05T11:29:09,552 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:09,552 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@43a0015b can be done in LLAP 2018-01-05T11:29:09,552 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Map 4 2018-01-05T11:29:09,552 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key' 2018-01-05T11:29:09,552 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:09,552 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@3df8a7e6 can be done in LLAP 2018-01-05T11:29:09,552 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Reducer 2 2018-01-05T11:29:09,552 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'count()' 2018-01-05T11:29:09,552 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:09,552 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MergeJoinWork@1ff98d32 can be done in LLAP 2018-01-05T11:29:09,552 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Reducer 3 2018-01-05T11:29:09,552 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'count(VALUE._col0)' 2018-01-05T11:29:09,552 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:29:09,552 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:09,552 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@7e11cba6 can be done in LLAP 2018-01-05T11:29:09,552 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,552 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed plan generation 2018-01-05T11:29:09,552 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:09,552 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: validation start 2018-01-05T11:29:09,552 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:29:09,552 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,552 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0, type:bigint, comment:null)], properties:null) 2018-01-05T11:29:09,553 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: Initializing operator LIST_SINK[27] 2018-01-05T11:29:09,553 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: Initialization Done 27 LIST_SINK 2018-01-05T11:29:09,553 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: Operator 27 LIST_SINK initialized 2018-01-05T11:29:09,553 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: Initialization Done 27 LIST_SINK done is reset. 2018-01-05T11:29:09,553 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,553 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:09,553 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {getNotNullConstraints_(NotNullConstraintsRequest, )=0, isCompatibleWith_(Configuration, )=1, getAggrColStatsFor_(String, String, List, List, )=6, getTable_(String, String, )=11, flushCache_()=0, getUniqueConstraints_(UniqueConstraintsRequest, )=1, getPrimaryKeys_(PrimaryKeysRequest, )=1, listPartitions_(String, String, short, )=14, getForeignKeys_(ForeignKeysRequest, )=0} 2018-01-05T11:29:09,553 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd); Time taken: 0.118 seconds 2018-01-05T11:29:09,553 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:09,553 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,553 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd): select count (*) from tab a right outer join tab_part b on a.key = b.key 2018-01-05T11:29:09,554 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,554 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: select count (*) from tab a right outer join tab_part b on a.key = b.key 2018-01-05T11:29:09,554 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:29:09,554 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@tab 2018-01-05T11:29:09,554 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@tab@ds=2008-04-08 2018-01-05T11:29:09,554 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@tab_part 2018-01-05T11:29:09,554 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@tab_part@ds=2008-04-08 2018-01-05T11:29:09,554 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/-mr-10001 2018-01-05T11:29:09,554 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,554 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,554 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,554 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Query ID = djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd 2018-01-05T11:29:09,554 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Total jobs = 1 2018-01-05T11:29:09,554 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,554 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:29:09,554 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-1:MAPRED] in serial mode 2018-01-05T11:29:09,563 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:29:09,585 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:29:09,585 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:29:09,585 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:29:09,585 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd 2018-01-05T11:29:09,585 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Session is already open 2018-01-05T11:29:09,585 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:29:09,585 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,586 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Dag name: select count (*) from tab a right ou...b.key (Stage-1) 2018-01-05T11:29:09,586 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: DagInfo: {"context":"Hive","description":"\n\nselect count (*)\nfrom tab a right outer join tab_part b on a.key = b.key"} 2018-01-05T11:29:09,586 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:29:09,586 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,598 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:29:09,598 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,599 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 3 size: 1.91KB 2018-01-05T11:29:09,614 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,614 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,628 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,628 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:29:09,629 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,629 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 2 size: 1.50KB 2018-01-05T11:29:09,631 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,633 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,653 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:29:09,654 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,654 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:29:09,654 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,654 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: Serialized plan (via RPC) - name: Map 4 size: 3.07KB 2018-01-05T11:29:09,661 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:09,661 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,662 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,675 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:29:09,676 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,676 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:29:09,676 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,677 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 3.05KB 2018-01-05T11:29:09,683 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:09,683 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,685 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,685 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,685 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=select count (*) from tab a right ou...b.key (Stage-1), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd } 2018-01-05T11:29:09,686 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:09,686 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:09,686 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:09,686 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:09,688 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: select count (*) from tab a right ou...b.key (Stage-1) 2018-01-05T11:29:09,688 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:29:09,689 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_40, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_40.dot 2018-01-05T11:29:09,689 INFO [IPC Server handler 0 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_40 2018-01-05T11:29:09,689 WARN [IPC Server handler 0 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:29:09,689 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Running DAG: select count (*) from tab a right ou...b.key (Stage-1), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd } 2018-01-05T11:29:09,714 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_40, submitTime=1515180549688, queueName=null 2018-01-05T11:29:09,715 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_40_00 [Map 4]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_40_00 [Map 4], Default Resources= 2018-01-05T11:29:09,715 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_40_00 [Map 4] 2018-01-05T11:29:09,715 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 4 2018-01-05T11:29:09,715 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_40_00 [Map 4] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:09,715 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_40_01 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_40_01 [Map 1], Default Resources= 2018-01-05T11:29:09,715 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_40_01 [Map 1] 2018-01-05T11:29:09,715 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:29:09,716 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_40_01 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:09,716 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_40_02 [Reducer 2]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_40_02 [Reducer 2], Default Resources= 2018-01-05T11:29:09,716 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 2 2018-01-05T11:29:09,716 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_40_02 [Reducer 2] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:09,716 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_40_03 [Reducer 3]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_40_03 [Reducer 3], Default Resources= 2018-01-05T11:29:09,716 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_40_03 [Reducer 3] 2018-01-05T11:29:09,716 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 3 2018-01-05T11:29:09,716 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_40_03 [Reducer 3] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:09,716 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:29:09,717 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_40, initTime=1515180549717 2018-01-05T11:29:09,717 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: dag_1515180518813_0001_40 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:29:09,717 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:29:09,717 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_40, startTime=1515180549717 2018-01-05T11:29:09,717 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_40 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:29:09,717 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {a={InputName=a}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:29:09,717 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: a, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:29:09,717 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_40_01 [Map 1] 2018-01-05T11:29:09,717 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_40, dagName=select count (*) from tab a right ou...b.key (Stage-1) 2018-01-05T11:29:09,717 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,717 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,717 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,721 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_40_01 [Map 1] 2018-01-05T11:29:09,721 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_40_01 [Map 1] 2018-01-05T11:29:09,721 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:09,721 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_40_01 [Map 1] 2018-01-05T11:29:09,724 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:29:09,724 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@1820b155 2018-01-05T11:29:09,724 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/20827036-a3cc-49f0-ad6a-722d5650e02b/map.xml 2018-01-05T11:29:09,724 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/20827036-a3cc-49f0-ad6a-722d5650e02b/map.xml 2018-01-05T11:29:09,724 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/20827036-a3cc-49f0-ad6a-722d5650e02b/map.xml 2018-01-05T11:29:09,725 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:09,725 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:09,725 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:09,725 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:09,725 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.05KB 2018-01-05T11:29:09,726 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_40_01 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:29:09,726 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 4 : {b={InputName=b}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:29:09,726 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: b, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:29:09,726 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_40_00 [Map 4] 2018-01-05T11:29:09,726 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: a on vertex vertex_1515180518813_0001_40_01 [Map 1] 2018-01-05T11:29:09,726 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:29:09,726 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:09,726 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:29:09,726 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:09,726 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/20827036-a3cc-49f0-ad6a-722d5650e02b/map.xml 2018-01-05T11:29:09,726 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,726 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias a 2018-01-05T11:29:09,726 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:09,726 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:09,727 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 0 2018-01-05T11:29:09,727 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = key 2018-01-05T11:29:09,727 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:09,727 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:09,727 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:09,727 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:09,727 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:09,727 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: getSplits started 2018-01-05T11:29:09,727 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: Number of buckets specified by conf file is 2 2018-01-05T11:29:09,727 INFO [InputInitializer {Map 1} #0] orc.OrcInputFormat: ORC pushdown predicate: null 2018-01-05T11:29:09,727 INFO [InputInitializer {Map 1} #0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:09,727 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: Generate splits schema evolution property true reader schema struct transactional scan property false 2018-01-05T11:29:09,727 ERROR [ORC_GET_SPLITS #7] io.AcidUtils: Failed to get files with ID; using regular API: Only supported for DFS; got class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:09,731 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_40_00 [Map 4] 2018-01-05T11:29:09,731 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_40_00 [Map 4] 2018-01-05T11:29:09,731 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:09,731 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_40_00 [Map 4] 2018-01-05T11:29:09,733 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:29:09,733 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@2a709810 2018-01-05T11:29:09,733 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/713c5849-a0d5-47ce-8236-f3bb3bf77c58/map.xml 2018-01-05T11:29:09,733 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/713c5849-a0d5-47ce-8236-f3bb3bf77c58/map.xml 2018-01-05T11:29:09,733 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/713c5849-a0d5-47ce-8236-f3bb3bf77c58/map.xml 2018-01-05T11:29:09,733 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:09,733 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:09,733 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:09,734 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:09,734 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 4 size: 3.07KB 2018-01-05T11:29:09,734 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_40_00 [Map 4] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:29:09,735 INFO [InputInitializer {Map 4} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: b on vertex vertex_1515180518813_0001_40_00 [Map 4] 2018-01-05T11:29:09,735 INFO [InputInitializer {Map 4} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:29:09,735 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to ShuffleVertexManager for vertex_1515180518813_0001_40_02 [Reducer 2] 2018-01-05T11:29:09,735 DEBUG [InputInitializer {Map 4} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:09,736 INFO [InputInitializer {Map 4} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:29:09,736 DEBUG [InputInitializer {Map 4} #0] log.PerfLogger: 2018-01-05T11:29:09,736 INFO [InputInitializer {Map 4} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/713c5849-a0d5-47ce-8236-f3bb3bf77c58/map.xml 2018-01-05T11:29:09,736 DEBUG [InputInitializer {Map 4} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,736 INFO [InputInitializer {Map 4} #0] exec.Utilities: Processing alias b 2018-01-05T11:29:09,736 DEBUG [InputInitializer {Map 4} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:09,736 INFO [InputInitializer {Map 4} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:09,736 INFO [InputInitializer {Map 4} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 0 2018-01-05T11:29:09,736 INFO [InputInitializer {Map 4} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = key 2018-01-05T11:29:09,736 INFO [InputInitializer {Map 4} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:09,736 DEBUG [InputInitializer {Map 4} #0] orc.OrcInputFormat: getSplits started 2018-01-05T11:29:09,736 DEBUG [InputInitializer {Map 4} #0] orc.OrcInputFormat: Number of buckets specified by conf file is 4 2018-01-05T11:29:09,736 INFO [InputInitializer {Map 4} #0] orc.OrcInputFormat: ORC pushdown predicate: null 2018-01-05T11:29:09,737 INFO [InputInitializer {Map 4} #0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:09,737 ERROR [ORC_GET_SPLITS #0] io.AcidUtils: Failed to get files with ID; using regular API: Only supported for DFS; got class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:09,737 DEBUG [InputInitializer {Map 4} #0] orc.OrcInputFormat: Generate splits schema evolution property true reader schema struct transactional scan property false 2018-01-05T11:29:09,741 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.05 maxFrac: 0.75 auto: false desiredTaskIput: 104857600 2018-01-05T11:29:09,741 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:29:09,741 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 1 tasks for vertex: vertex_1515180518813_0001_40_02 [Reducer 2] 2018-01-05T11:29:09,741 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_40_02 [Reducer 2] 2018-01-05T11:29:09,741 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_40_02, reconfigureDoneTime=1515180549741, numTasks=1, vertexLocationHint=null, edgeManagersCount=2, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:29:09,741 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:VERTEX_INITIALIZED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_40_02, initRequestedTime=1515180549734, initedTime=1515180549741, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:09,741 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_40_02 [Reducer 2] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:29:09,741 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to ShuffleVertexManager for vertex_1515180518813_0001_40_03 [Reducer 3] 2018-01-05T11:29:09,741 DEBUG [AMReporterQueueDrainer] impl.AMReporter: Removing am 10.22.8.182:49824 with last associated dag QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=26} from heartbeat with taskCount=0, amFailed=false, isDone=true 2018-01-05T11:29:09,747 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.05 maxFrac: 0.75 auto: false desiredTaskIput: 104857600 2018-01-05T11:29:09,747 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:29:09,747 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 1 tasks for vertex: vertex_1515180518813_0001_40_03 [Reducer 3] 2018-01-05T11:29:09,747 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_40_03 [Reducer 3] 2018-01-05T11:29:09,747 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_40_03, reconfigureDoneTime=1515180549747, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:29:09,747 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_40_03 [Reducer 3], numAdditionalOutputs=1 2018-01-05T11:29:09,747 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:VERTEX_INITIALIZED]: vertexName=Reducer 3, vertexId=vertex_1515180518813_0001_40_03, initRequestedTime=1515180549741, initedTime=1515180549747, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:09,747 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_40_03 [Reducer 3] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:29:09,757 DEBUG [ORC_GET_SPLITS #7] io.AcidUtils: in directory file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 base = null deltas = 0 2018-01-05T11:29:09,758 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: Split strategy: ETLSplitStrategy strategy for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:09,758 DEBUG [ORC_GET_SPLITS #8] orc.LocalCache: Serialized tail cached for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0 2018-01-05T11:29:09,758 DEBUG [ORC_GET_SPLITS #8] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,758 DEBUG [ORC_GET_SPLITS #8] orc.LocalCache: Serialized tail cached for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0 2018-01-05T11:29:09,758 DEBUG [ORC_GET_SPLITS #8] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,758 DEBUG [ORC_GET_SPLITS #8] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,758 DEBUG [ORC_GET_SPLITS #9] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,758 DEBUG [ORC_GET_SPLITS #8] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:29:09,758 DEBUG [ORC_GET_SPLITS #9] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:29:09,758 INFO [InputInitializer {Map 1} #0] orc.OrcInputFormat: FooterCacheHitRatio: 2/2 2018-01-05T11:29:09,758 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0, start=3, length=566, isOriginal=true, fileLength=744, hasFooter=false, hasBase=true, deltas=0] projected_columns_uncompressed_size: 371 2018-01-05T11:29:09,758 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0, start=3, length=583, isOriginal=true, fileLength=767, hasFooter=false, hasBase=true, deltas=0] projected_columns_uncompressed_size: 364 2018-01-05T11:29:09,758 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: getSplits finished 2018-01-05T11:29:09,758 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 2 2018-01-05T11:29:09,758 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:09,758 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 2. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:29:09,758 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/20827036-a3cc-49f0-ad6a-722d5650e02b/map.xml 2018-01-05T11:29:09,758 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/20827036-a3cc-49f0-ad6a-722d5650e02b/map.xml 2018-01-05T11:29:09,758 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/20827036-a3cc-49f0-ad6a-722d5650e02b/map.xml 2018-01-05T11:29:09,758 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:09,758 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:09,758 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:09,759 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:09,759 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.05KB 2018-01-05T11:29:09,759 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0 to src new group? true 2018-01-05T11:29:09,759 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0 to src new group? false 2018-01-05T11:29:09,759 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:29:09,759 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:29:09,759 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:29:09,759 DEBUG [InputInitializer {Map 1} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 364 2018-01-05T11:29:09,759 DEBUG [InputInitializer {Map 1} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 371 2018-01-05T11:29:09,759 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 13 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 735 Original splits: 2 2018-01-05T11:29:09,759 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired numSplits: 1 lengthPerGroup: 735 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 735 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false 2018-01-05T11:29:09,759 DEBUG [InputInitializer {Map 1} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 364 2018-01-05T11:29:09,759 DEBUG [InputInitializer {Map 1} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 371 2018-01-05T11:29:09,759 DEBUG [InputInitializer {Map 1} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 371 2018-01-05T11:29:09,759 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Number of splits desired: 1 created: 1 splitsProcessed: 2 2018-01-05T11:29:09,759 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 2 grouped split count is 1, for bucket: 1 2018-01-05T11:29:09,759 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:29:09,759 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: a on vertex vertex_1515180518813_0001_40_01 [Map 1] 2018-01-05T11:29:09,760 INFO [App Shared Pool - #2] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_40_01 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:09,760 INFO [App Shared Pool - #2] impl.VertexImpl: Got updated RootInputsSpecs: {a=forAllWorkUnits=true, update=[1]} 2018-01-05T11:29:09,760 INFO [App Shared Pool - #2] impl.VertexImpl: Vertex vertex_1515180518813_0001_40_01 [Map 1] parallelism set to 1 2018-01-05T11:29:09,760 INFO [App Shared Pool - #2] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_40_01 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:09,760 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_40_01, initRequestedTime=1515180549717, initedTime=1515180549760, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:09,760 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_40_01 [Map 1] with distanceFromRoot: 0 2018-01-05T11:29:09,760 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_40_01 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:29:09,760 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_40_01, startRequestedTime=1515180549734, startedTime=1515180549760 2018-01-05T11:29:09,760 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_40_01, reconfigureDoneTime=1515180549760, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:29:09,760 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_40_01 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:09,760 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_40_01 for vertex: vertex_1515180518813_0001_40_02 [Reducer 2] numStartedSources: 1 numSources: 2 2018-01-05T11:29:09,760 INFO [Dispatcher thread {Central}] impl.VertexImpl: Cannot start vertex: vertex_1515180518813_0001_40_02 [Reducer 2] numStartedSources: 1 numSources: 2 2018-01-05T11:29:09,760 INFO [App Shared Pool - #2] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:29:09,760 INFO [App Shared Pool - #2] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:29:09,760 INFO [App Shared Pool - #2] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_40_01 [Map 1] 2018-01-05T11:29:09,760 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_40_01_000000, scheduledTime=1515180549760, launchTime=1515180549760 2018-01-05T11:29:09,760 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_40_01 [Map 1] 2018-01-05T11:29:09,760 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_40_01_000000_0, priority=14, capability=, hosts=null 2018-01-05T11:29:09,760 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:09,760 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:09,760 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:09,760 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_40_01_000000_0 on hosts=null 2018-01-05T11:29:09,760 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_40_01_000000_0 on hosts=null 2018-01-05T11:29:09,760 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_40_01_000000_0 2018-01-05T11:29:09,760 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:09,760 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #97, task=TaskInfo{task=attempt_1515180518813_0001_40_01_000000_0, priority=14, startTime=0, containerId=null, uniqueId=96, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000097 2018-01-05T11:29:09,760 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_40_01_000000_0; false 2018-01-05T11:29:09,761 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_40_01_000000_0, priority=14, startTime=311570270, containerId=container_111101111_0001_00_000097assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=96, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:09,761 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000097 for node: 10.22.8.182:49818 2018-01-05T11:29:09,763 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 40, name=select count (*) from tab a right ou...b.key (Stage-1), queryId=djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd 2018-01-05T11:29:09,763 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000097, attempt_1515180518813_0001_40_01_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:09,763 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000097 for node: 10.22.8.182:49818 2018-01-05T11:29:09,765 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000097 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:09,765 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000097, launchTime=1515180549765 2018-01-05T11:29:09,766 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_40_01_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000097, dagName=select count (*) from tab a right ou...b.key (Stage-1), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{a,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{Reducer 2,org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput,1}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180549717, firstAttemptStartTime=1515180549760, currentAttemptStartTime=1515180549763} 2018-01-05T11:29:09,766 INFO [IPC Server handler 4 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000097, QueryId=djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd-40, DagId=40, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180549766 2018-01-05T11:29:09,766 DEBUG [IPC Server handler 4 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=40} with the ShuffleHandler 2018-01-05T11:29:09,766 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_40_01_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000097] on NM: [10.22.8.182:49818] 2018-01-05T11:29:09,766 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:09,766 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:09,766 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_40_01_000000_0, startTime=1515180549765, containerId=container_111101111_0001_00_000097, nodeId=10.22.8.182:49818 2018-01-05T11:29:09,777 INFO [IPC Server handler 4 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:29:09,777 DEBUG [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/40] 2018-01-05T11:29:09,777 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:29:09,777 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_40_01_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:09,777 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_40_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180549760, dagStartTime=1515180549717, withinDagPriority=14, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:29:09,777 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_40_01_000000_0 : ACCEPTED 2018-01-05T11:29:09,777 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_40_01_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:29:09,777 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_40_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180549760, dagStartTime=1515180549717, withinDagPriority=14, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:29:09,777 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_40_01_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:09,777 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_40_01_000000_0 to pre-emption queue 2018-01-05T11:29:09,777 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_40_01_000000_0: true 2018-01-05T11:29:09,777 INFO [TaskCommunicator # 9] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_40_01_000000_0 2018-01-05T11:29:09,778 INFO [TaskCommunicator # 9] GuaranteedTasks: Received response for attempt_1515180518813_0001_40_01_000000_0, true 2018-01-05T11:29:09,778 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: taskOwner hashCode:64272534 2018-01-05T11:29:09,778 INFO [Task-Executor-1] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : select count (*) from tab a right ou...b.key (Stage-1), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_40_01_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=a, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=Reducer 2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ] 2018-01-05T11:29:09,778 INFO [Task-Executor-1] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1024458752, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:09,778 INFO [Task-Executor-1] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_40_01_000000_0 vertexName: Map 1 2018-01-05T11:29:09,778 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000097, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_40_01_000000_0, eventCount=1 } 2018-01-05T11:29:09,778 DEBUG [TezTR-518813_1_40_1_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_40 queryId: djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd instanceType: TaskRunner2Callable 2018-01-05T11:29:09,778 INFO [TezTR-518813_1_40_1_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_40_01_000000_0 2018-01-05T11:29:09,778 DEBUG [TezTR-518813_1_40_1_0_0] log.PerfLogger: 2018-01-05T11:29:09,779 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:09,779 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:09,780 INFO [IPC Server handler 8 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_40_01_000000_0 sent events: (0-1). 2018-01-05T11:29:09,780 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:29:09,780 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_40_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:29:09,781 DEBUG [TezTR-518813_1_40_1_0_0] log.PerfLogger: 2018-01-05T11:29:09,781 INFO [TezTR-518813_1_40_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:09,781 INFO [TezTR-518813_1_40_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:09,781 INFO [TezTR-518813_1_40_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:09,781 INFO [I/O Setup 0 Initialize: {a}] input.MRInput: a using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:29:09,781 INFO [I/O Setup 0 Initialize: {a}] input.MRInputLegacy: a MRInputLegacy deferring initialization 2018-01-05T11:29:09,781 INFO [TezTR-518813_1_40_1_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:09,781 INFO [TezTR-518813_1_40_1_0_0] resources.MemoryDistributor: InitialRequests=[a:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [Reducer 2:OUTPUT:10485760:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput] 2018-01-05T11:29:09,781 INFO [TezTR-518813_1_40_1_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:09,781 INFO [TezTR-518813_1_40_1_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:09,781 INFO [TezTR-518813_1_40_1_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 10485760, TotalRequestedScaled: 9679163.076923078, TotalJVMHeap: 1024458752, TotalAvailable: 686387363, TotalRequested/TotalJVMHeap:0.01 2018-01-05T11:29:09,781 INFO [TezTR-518813_1_40_1_0_0] resources.MemoryDistributor: Allocations=[a:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:10485760:10485760] 2018-01-05T11:29:09,781 INFO [TezTR-518813_1_40_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:09,781 INFO [TezTR-518813_1_40_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: a being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:09,781 INFO [TezTR-518813_1_40_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:29:09,781 INFO [TezTR-518813_1_40_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:09,781 INFO [I/O Setup 1 Start: {a}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: a 2018-01-05T11:29:09,781 INFO [TezTR-518813_1_40_1_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:09,782 INFO [TezTR-518813_1_40_1_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_40_01_000000_0 2018-01-05T11:29:09,782 DEBUG [TezTR-518813_1_40_1_0_0] log.PerfLogger: 2018-01-05T11:29:09,782 DEBUG [TezTR-518813_1_40_1_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_40_01_000000_0_10295 2018-01-05T11:29:09,782 DEBUG [TezTR-518813_1_40_1_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:09,782 DEBUG [TezTR-518813_1_40_1_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_40_01_000000_0 2018-01-05T11:29:09,782 INFO [TezTR-518813_1_40_1_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd 2018-01-05T11:29:09,782 DEBUG [TezTR-518813_1_40_1_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:09,782 DEBUG [TezTR-518813_1_40_1_0_0] log.PerfLogger: 2018-01-05T11:29:09,782 DEBUG [TezTR-518813_1_40_1_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:09,782 DEBUG [TezTR-518813_1_40_1_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:09,782 DEBUG [TezTR-518813_1_40_1_0_0] mr.ObjectCache: Creating djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd_Map 1__MAP_PLAN__ 2018-01-05T11:29:09,782 INFO [TezTR-518813_1_40_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/20827036-a3cc-49f0-ad6a-722d5650e02b/map.xml 2018-01-05T11:29:09,782 DEBUG [TezTR-518813_1_40_1_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/20827036-a3cc-49f0-ad6a-722d5650e02b/map.xml 2018-01-05T11:29:09,782 DEBUG [TezTR-518813_1_40_1_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/20827036-a3cc-49f0-ad6a-722d5650e02b/map.xml 2018-01-05T11:29:09,782 DEBUG [TezTR-518813_1_40_1_0_0] log.PerfLogger: 2018-01-05T11:29:09,782 INFO [TezTR-518813_1_40_1_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:09,782 DEBUG [TezTR-518813_1_40_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:09,783 DEBUG [TezTR-518813_1_40_1_0_0] log.PerfLogger: 2018-01-05T11:29:09,783 INFO [TezTR-518813_1_40_1_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.05KB 2018-01-05T11:29:09,783 DEBUG [TezTR-518813_1_40_1_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:09,783 INFO [TezTR-518813_1_40_1_0_0] tez.MapRecordProcessor: The input names are: [a] 2018-01-05T11:29:09,783 INFO [TezTR-518813_1_40_1_0_0] input.MRInput: Initialized MRInput: a 2018-01-05T11:29:09,783 DEBUG [TezTR-518813_1_40_1_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 2 2018-01-05T11:29:09,783 INFO [TezTR-518813_1_40_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/20827036-a3cc-49f0-ad6a-722d5650e02b/map.xml 2018-01-05T11:29:09,783 DEBUG [TezTR-518813_1_40_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,783 DEBUG [TezTR-518813_1_40_1_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180546}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, numFiles=2, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, rawDataSize=22748, partition_columns.types=string, numRows=242, partition_columns=ds, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=1511, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08, transient_lastDdlTime=1515180547}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180546}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, numFiles=2, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, rawDataSize=22748, partition_columns.types=string, numRows=242, partition_columns=ds, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=1511, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08, transient_lastDdlTime=1515180547}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat]} 2018-01-05T11:29:09,784 INFO [TezTR-518813_1_40_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/20827036-a3cc-49f0-ad6a-722d5650e02b/map.xml 2018-01-05T11:29:09,784 DEBUG [TezTR-518813_1_40_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,784 DEBUG [TezTR-518813_1_40_1_0_0] io.HiveInputFormat: Processing org.apache.hadoop.hive.ql.io.orc.OrcInputFormat 2018-01-05T11:29:09,784 DEBUG [TezTR-518813_1_40_1_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:29:09,784 INFO [TezTR-518813_1_40_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/20827036-a3cc-49f0-ad6a-722d5650e02b/map.xml 2018-01-05T11:29:09,784 DEBUG [TezTR-518813_1_40_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,784 INFO [TezTR-518813_1_40_1_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_40_01_000000_0 2018-01-05T11:29:09,785 INFO [TezTR-518813_1_40_1_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:09,785 DEBUG [TezTR-518813_1_40_1_0_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:09,785 DEBUG [TezTR-518813_1_40_1_0_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:09,785 DEBUG [TezTR-518813_1_40_1_0_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:09,785 DEBUG [TezTR-518813_1_40_1_0_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:09,785 DEBUG [TezTR-518813_1_40_1_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,785 DEBUG [TezTR-518813_1_40_1_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,785 INFO [TezTR-518813_1_40_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/20827036-a3cc-49f0-ad6a-722d5650e02b/map.xml 2018-01-05T11:29:09,785 DEBUG [TezTR-518813_1_40_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,785 DEBUG [TezTR-518813_1_40_1_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0 2018-01-05T11:29:09,785 DEBUG [IO-Elevator-Thread-8] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_40 queryId: djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd instanceType: OrcEncodedDataReader 2018-01-05T11:29:09,785 INFO [TezTR-518813_1_40_1_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0, start=3, length=583, isOriginal=true, fileLength=767, hasFooter=false, hasBase=true, deltas=0], org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0, start=3, length=566, isOriginal=true, fileLength=744, hasFooter=false, hasBase=true, deltas=0]], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=1149} 2018-01-05T11:29:09,785 INFO [IO-Elevator-Thread-8] LlapIoImpl: Processing data for file [-5122461581150239451, 1515180546000, 767]: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0 2018-01-05T11:29:09,785 INFO [TezTR-518813_1_40_1_0_0] input.MRInput: a initialized RecordReader from event 2018-01-05T11:29:09,785 DEBUG [TezTR-518813_1_40_1_0_0] tez.MapRecordProcessor: Starting Output: Reducer 2 2018-01-05T11:29:09,785 DEBUG [IO-Elevator-Thread-8] LlapIoOrc: FileSplit {3, 583}; stripes {3, 583}, 2018-01-05T11:29:09,787 DEBUG [IO-Elevator-Thread-8] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,787 DEBUG [IO-Elevator-Thread-8] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,787 INFO [TezTR-518813_1_40_1_0_0] impl.ExternalSorter: Reducer 2 using: memoryMb=10, keySerializerClass=class org.apache.hadoop.hive.ql.io.HiveKey, valueSerializerClass=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization$TezBytesWritableSerializer@1df29ec2, comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator@7447e816, partitioner=org.apache.tez.mapreduce.partition.MRPartitioner, serialization=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.hadoop.io.serializer.WritableSerialization, org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization, org.apache.hadoop.io.serializer.avro.AvroReflectSerialization, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:29:09,787 INFO [TezTR-518813_1_40_1_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 2: , UsingHashComparator=true 2018-01-05T11:29:09,787 INFO [IO-Elevator-Thread-8] impl.OrcCodecPool: Got brand-new codec ZLIB 2018-01-05T11:29:09,787 INFO [IO-Elevator-Thread-8] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [-5122461581150239451, 1515180546000, 767]): [{range start: 0 end: 38}] 2018-01-05T11:29:09,787 INFO [IO-Elevator-Thread-8] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [-5122461581150239451, 1515180546000, 767], base offset 3): [{start: 0 end: 11 cache buffer: 0x317f0390(1)}, {start: 11 end: 38 cache buffer: 0x4252621d(1)}] 2018-01-05T11:29:09,787 INFO [IO-Elevator-Thread-8] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [-5122461581150239451, 1515180546000, 767]): [{range start: 76 end: 230}] 2018-01-05T11:29:09,787 INFO [IO-Elevator-Thread-8] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [-5122461581150239451, 1515180546000, 767], base offset 3): [{start: 76 end: 230 cache buffer: 0x112f2b3b(1)}] 2018-01-05T11:29:09,787 DEBUG [IO-Elevator-Thread-8] encoded.EncodedTreeReaderFactory: Column at 1 2:string has no data 2018-01-05T11:29:09,787 DEBUG [IO-Elevator-Thread-8] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: int streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 164 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:29:09,787 DEBUG [IO-Elevator-Thread-8] LlapIoOrc: Done with decode 2018-01-05T11:29:09,787 DEBUG [IO-Elevator-Thread-8] LlapIoImpl: setDone called; closed false, done false, err null, pending 1 2018-01-05T11:29:09,787 DEBUG [IO-Elevator-Thread-8] impl.StatsRecordingThreadPool: Updated stats: instance: OrcEncodedDataReader thread name: IO-Elevator-Thread-8 thread id: 2387 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:09,788 INFO [TezTR-518813_1_40_1_0_0] impl.PipelinedSorter: Newly allocated block size=10485760, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=10485760, total=10485760 2018-01-05T11:29:09,788 INFO [TezTR-518813_1_40_1_0_0] impl.PipelinedSorter: Pre allocating rest of memory buffers upfront 2018-01-05T11:29:09,788 INFO [TezTR-518813_1_40_1_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 2: , UsingHashComparator=true#blocks=1, maxMemUsage=10485760, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=10485760, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=10 2018-01-05T11:29:09,788 INFO [TezTR-518813_1_40_1_0_0] impl.PipelinedSorter: Reducer 2: reserved.remaining()=10485760, reserved.metasize=5242880 2018-01-05T11:29:09,788 INFO [TezTR-518813_1_40_1_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:29:09,788 INFO [TezTR-518813_1_40_1_0_0] vector.VectorMapOperator: Initializing operator MAP[0] 2018-01-05T11:29:09,788 DEBUG [TezTR-518813_1_40_1_0_0] vector.VectorMapOperator: Initialization Done 0 MAP 2018-01-05T11:29:09,788 DEBUG [TezTR-518813_1_40_1_0_0] vector.VectorMapOperator: Operator 0 MAP initialized 2018-01-05T11:29:09,788 DEBUG [TezTR-518813_1_40_1_0_0] vector.VectorMapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:29:09,788 INFO [TezTR-518813_1_40_1_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:29:09,788 INFO [TezTR-518813_1_40_1_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:09,788 DEBUG [TezTR-518813_1_40_1_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:29:09,788 INFO [TezTR-518813_1_40_1_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:29:09,788 DEBUG [TezTR-518813_1_40_1_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:29:09,788 DEBUG [TezTR-518813_1_40_1_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:29:09,788 DEBUG [TezTR-518813_1_40_1_0_0] exec.TableScanOperator: Initializing children of 0 TS 2018-01-05T11:29:09,788 DEBUG [TezTR-518813_1_40_1_0_0] vector.VectorSelectOperator: Initializing child 21 SEL 2018-01-05T11:29:09,788 INFO [TezTR-518813_1_40_1_0_0] vector.VectorSelectOperator: Initializing operator SEL[21] 2018-01-05T11:29:09,788 DEBUG [TezTR-518813_1_40_1_0_0] vector.VectorSelectOperator: Initialization Done 21 SEL 2018-01-05T11:29:09,788 DEBUG [TezTR-518813_1_40_1_0_0] vector.VectorSelectOperator: Operator 21 SEL initialized 2018-01-05T11:29:09,788 DEBUG [TezTR-518813_1_40_1_0_0] vector.VectorSelectOperator: Initializing children of 21 SEL 2018-01-05T11:29:09,788 DEBUG [TezTR-518813_1_40_1_0_0] reducesink.VectorReduceSinkLongOperator: Initializing child 22 RS 2018-01-05T11:29:09,788 INFO [TezTR-518813_1_40_1_0_0] reducesink.VectorReduceSinkLongOperator: Initializing operator RS[22] 2018-01-05T11:29:09,788 INFO [TezTR-518813_1_40_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/20827036-a3cc-49f0-ad6a-722d5650e02b/map.xml 2018-01-05T11:29:09,788 DEBUG [TezTR-518813_1_40_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,788 INFO [TezTR-518813_1_40_1_0_0] reducesink.VectorReduceSinkCommonOperator: Using tag = 0 2018-01-05T11:29:09,788 DEBUG [TezTR-518813_1_40_1_0_0] reducesink.VectorReduceSinkLongOperator: Initialization Done 22 RS 2018-01-05T11:29:09,788 DEBUG [TezTR-518813_1_40_1_0_0] reducesink.VectorReduceSinkLongOperator: Operator 22 RS initialized 2018-01-05T11:29:09,788 DEBUG [TezTR-518813_1_40_1_0_0] reducesink.VectorReduceSinkLongOperator: Initialization Done 22 RS done is reset. 2018-01-05T11:29:09,788 DEBUG [TezTR-518813_1_40_1_0_0] vector.VectorSelectOperator: Initialization Done 21 SEL done is reset. 2018-01-05T11:29:09,788 DEBUG [TezTR-518813_1_40_1_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:29:09,788 INFO [TezTR-518813_1_40_1_0_0] exec.OperatorUtils: Setting output collector: RS[22] --> Reducer 2 2018-01-05T11:29:09,788 DEBUG [TezTR-518813_1_40_1_0_0] log.PerfLogger: 2018-01-05T11:29:09,789 INFO [TezTR-518813_1_40_1_0_0] reducesink.VectorReduceSinkCommonOperator: RS[22]: records written - 1 2018-01-05T11:29:09,789 INFO [TezTR-518813_1_40_1_0_0] reducesink.VectorReduceSinkCommonOperator: RS[22]: records written - 10 2018-01-05T11:29:09,789 INFO [TezTR-518813_1_40_1_0_0] reducesink.VectorReduceSinkCommonOperator: RS[22]: records written - 100 2018-01-05T11:29:09,789 INFO [TezTR-518813_1_40_1_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:09,789 INFO [TezTR-518813_1_40_1_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, warehouse.tab, file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0 ([-5122461581150239451, 1515180546000, 767]), 0,1]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=1, NUM_ERRORS=0, ROWS_EMITTED=120, METADATA_CACHE_HIT=2, METADATA_CACHE_MISS=0, CACHE_HIT_BYTES=192, CACHE_MISS_BYTES=0, ALLOCATED_BYTES=0, ALLOCATED_USED_BYTES=0, TOTAL_IO_TIME_NS=2201696, DECODE_TIME_NS=157098, HDFS_TIME_NS=36842, CONSUMER_TIME_NS=179808 ] 2018-01-05T11:29:09,789 DEBUG [TezTR-518813_1_40_1_0_0] encoded.OrcEncodedDataReader: Encoded reader is being stopped 2018-01-05T11:29:09,789 DEBUG [TezTR-518813_1_40_1_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 1 of 2 2018-01-05T11:29:09,789 DEBUG [TezTR-518813_1_40_1_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180546}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, numFiles=2, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, rawDataSize=22748, partition_columns.types=string, numRows=242, partition_columns=ds, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=1511, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08, transient_lastDdlTime=1515180547}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180546}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, numFiles=2, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, rawDataSize=22748, partition_columns.types=string, numRows=242, partition_columns=ds, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=1511, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08, transient_lastDdlTime=1515180547}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat]} 2018-01-05T11:29:09,789 INFO [TezTR-518813_1_40_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/20827036-a3cc-49f0-ad6a-722d5650e02b/map.xml 2018-01-05T11:29:09,789 DEBUG [TezTR-518813_1_40_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,789 DEBUG [TezTR-518813_1_40_1_0_0] io.HiveInputFormat: Processing org.apache.hadoop.hive.ql.io.orc.OrcInputFormat 2018-01-05T11:29:09,790 DEBUG [TezTR-518813_1_40_1_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:29:09,790 INFO [TezTR-518813_1_40_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/20827036-a3cc-49f0-ad6a-722d5650e02b/map.xml 2018-01-05T11:29:09,790 DEBUG [TezTR-518813_1_40_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,790 INFO [TezTR-518813_1_40_1_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_40_01_000000_0 2018-01-05T11:29:09,790 INFO [TezTR-518813_1_40_1_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:09,790 DEBUG [TezTR-518813_1_40_1_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,790 DEBUG [TezTR-518813_1_40_1_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,790 INFO [TezTR-518813_1_40_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/20827036-a3cc-49f0-ad6a-722d5650e02b/map.xml 2018-01-05T11:29:09,790 DEBUG [TezTR-518813_1_40_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,790 DEBUG [TezTR-518813_1_40_1_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0 2018-01-05T11:29:09,790 DEBUG [IO-Elevator-Thread-9] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_40 queryId: djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd instanceType: OrcEncodedDataReader 2018-01-05T11:29:09,790 INFO [IO-Elevator-Thread-9] LlapIoImpl: Processing data for file [-5122461581151735180, 1515180546000, 744]: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0 2018-01-05T11:29:09,790 DEBUG [IO-Elevator-Thread-9] LlapIoOrc: FileSplit {3, 566}; stripes {3, 566}, 2018-01-05T11:29:09,790 DEBUG [IO-Elevator-Thread-9] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,790 DEBUG [IO-Elevator-Thread-9] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,790 INFO [IO-Elevator-Thread-9] impl.OrcCodecPool: Got brand-new codec ZLIB 2018-01-05T11:29:09,790 INFO [IO-Elevator-Thread-9] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [-5122461581151735180, 1515180546000, 744]): [{range start: 0 end: 38}] 2018-01-05T11:29:09,790 INFO [IO-Elevator-Thread-9] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [-5122461581151735180, 1515180546000, 744], base offset 3): [{start: 0 end: 11 cache buffer: 0x4d1ed212(1)}, {start: 11 end: 38 cache buffer: 0x6497a38f(1)}] 2018-01-05T11:29:09,790 INFO [IO-Elevator-Thread-9] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [-5122461581151735180, 1515180546000, 744]): [{range start: 73 end: 204}] 2018-01-05T11:29:09,791 INFO [IO-Elevator-Thread-9] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [-5122461581151735180, 1515180546000, 744], base offset 3): [{start: 73 end: 204 cache buffer: 0x3e73bafa(1)}] 2018-01-05T11:29:09,791 DEBUG [IO-Elevator-Thread-9] encoded.EncodedTreeReaderFactory: Column at 1 2:string has no data 2018-01-05T11:29:09,791 DEBUG [IO-Elevator-Thread-9] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: int streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 136 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:29:09,791 DEBUG [IO-Elevator-Thread-9] LlapIoOrc: Done with decode 2018-01-05T11:29:09,791 DEBUG [IO-Elevator-Thread-9] LlapIoImpl: setDone called; closed false, done false, err null, pending 0 2018-01-05T11:29:09,791 DEBUG [IO-Elevator-Thread-9] impl.StatsRecordingThreadPool: Updated stats: instance: OrcEncodedDataReader thread name: IO-Elevator-Thread-9 thread id: 2389 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:09,791 INFO [TezTR-518813_1_40_1_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:09,791 INFO [TezTR-518813_1_40_1_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, warehouse.tab, file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0 ([-5122461581151735180, 1515180546000, 744]), 0,1]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=1, NUM_ERRORS=0, ROWS_EMITTED=122, METADATA_CACHE_HIT=2, METADATA_CACHE_MISS=0, CACHE_HIT_BYTES=169, CACHE_MISS_BYTES=0, ALLOCATED_BYTES=0, ALLOCATED_USED_BYTES=0, TOTAL_IO_TIME_NS=549515, DECODE_TIME_NS=120997, HDFS_TIME_NS=26700, CONSUMER_TIME_NS=499756 ] 2018-01-05T11:29:09,791 DEBUG [TezTR-518813_1_40_1_0_0] encoded.OrcEncodedDataReader: Encoded reader is being stopped 2018-01-05T11:29:09,791 DEBUG [TezTR-518813_1_40_1_0_0] log.PerfLogger: 2018-01-05T11:29:09,791 DEBUG [TezTR-518813_1_40_1_0_0] mr.ObjectCache: djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:29:09,791 DEBUG [TezTR-518813_1_40_1_0_0] vector.VectorMapOperator: close called for operator MAP[0] 2018-01-05T11:29:09,791 INFO [TezTR-518813_1_40_1_0_0] vector.VectorMapOperator: Closing operator MAP[0] 2018-01-05T11:29:09,791 INFO [TezTR-518813_1_40_1_0_0] vector.VectorMapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:242, 2018-01-05T11:29:09,791 DEBUG [TezTR-518813_1_40_1_0_0] vector.VectorMapOperator: Closing child = TS[0] 2018-01-05T11:29:09,791 DEBUG [TezTR-518813_1_40_1_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:29:09,791 INFO [TezTR-518813_1_40_1_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:29:09,791 DEBUG [TezTR-518813_1_40_1_0_0] exec.TableScanOperator: Closing child = SEL[21] 2018-01-05T11:29:09,791 DEBUG [TezTR-518813_1_40_1_0_0] vector.VectorSelectOperator: close called for operator SEL[21] 2018-01-05T11:29:09,791 DEBUG [TezTR-518813_1_40_1_0_0] vector.VectorSelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:09,791 INFO [TezTR-518813_1_40_1_0_0] vector.VectorSelectOperator: Closing operator SEL[21] 2018-01-05T11:29:09,791 DEBUG [TezTR-518813_1_40_1_0_0] vector.VectorSelectOperator: Closing child = RS[22] 2018-01-05T11:29:09,791 DEBUG [TezTR-518813_1_40_1_0_0] reducesink.VectorReduceSinkLongOperator: close called for operator RS[22] 2018-01-05T11:29:09,791 DEBUG [TezTR-518813_1_40_1_0_0] reducesink.VectorReduceSinkLongOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:09,791 INFO [TezTR-518813_1_40_1_0_0] reducesink.VectorReduceSinkLongOperator: Closing operator RS[22] 2018-01-05T11:29:09,791 INFO [TezTR-518813_1_40_1_0_0] reducesink.VectorReduceSinkCommonOperator: RS[22]: records written - 242 2018-01-05T11:29:09,791 INFO [TezTR-518813_1_40_1_0_0] reducesink.VectorReduceSinkLongOperator: RECORDS_OUT_INTERMEDIATE_Map_1:242, 2018-01-05T11:29:09,791 DEBUG [TezTR-518813_1_40_1_0_0] reducesink.VectorReduceSinkLongOperator: 22 Close done 2018-01-05T11:29:09,791 DEBUG [TezTR-518813_1_40_1_0_0] vector.VectorSelectOperator: 21 Close done 2018-01-05T11:29:09,791 DEBUG [TezTR-518813_1_40_1_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:29:09,791 DEBUG [TezTR-518813_1_40_1_0_0] vector.VectorMapOperator: 0 Close done 2018-01-05T11:29:09,791 INFO [TezTR-518813_1_40_1_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_40_01_000000_0 2018-01-05T11:29:09,791 INFO [TezTR-518813_1_40_1_0_0] impl.PipelinedSorter: Reducer 2: Starting flush of map output 2018-01-05T11:29:09,791 INFO [TezTR-518813_1_40_1_0_0] impl.PipelinedSorter: Reducer 2: Span0.length = 242, perItem = 6 2018-01-05T11:29:09,791 INFO [TezTR-518813_1_40_1_0_0] impl.PipelinedSorter: Reducer 2: done sorting span=0, length=242, time=0 2018-01-05T11:29:09,792 INFO [TezTR-518813_1_40_1_0_0] impl.PipelinedSorter: Reducer 2: Heap = SpanIterator<0:241> (span=Span[5242880,1452]), 2018-01-05T11:29:09,795 DEBUG [ORC_GET_SPLITS #0] io.AcidUtils: in directory file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 base = null deltas = 0 2018-01-05T11:29:09,795 DEBUG [InputInitializer {Map 4} #0] orc.OrcInputFormat: Split strategy: ETLSplitStrategy strategy for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:09,795 DEBUG [ORC_GET_SPLITS #2] orc.LocalCache: Serialized tail cached for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0 2018-01-05T11:29:09,795 DEBUG [ORC_GET_SPLITS #2] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,795 DEBUG [ORC_GET_SPLITS #2] orc.LocalCache: Serialized tail cached for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0 2018-01-05T11:29:09,795 DEBUG [ORC_GET_SPLITS #2] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,796 DEBUG [ORC_GET_SPLITS #2] orc.LocalCache: Serialized tail cached for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0 2018-01-05T11:29:09,796 DEBUG [ORC_GET_SPLITS #2] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,796 DEBUG [ORC_GET_SPLITS #2] orc.LocalCache: Serialized tail cached for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0 2018-01-05T11:29:09,796 DEBUG [ORC_GET_SPLITS #2] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,796 DEBUG [ORC_GET_SPLITS #2] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,796 DEBUG [ORC_GET_SPLITS #5] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,796 INFO [ORC_GET_SPLITS #3] impl.OrcCodecPool: Got brand-new codec ZLIB 2018-01-05T11:29:09,796 INFO [ORC_GET_SPLITS #4] impl.OrcCodecPool: Got brand-new codec ZLIB 2018-01-05T11:29:09,796 DEBUG [ORC_GET_SPLITS #2] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:29:09,796 DEBUG [ORC_GET_SPLITS #5] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:29:09,796 DEBUG [ORC_GET_SPLITS #3] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:29:09,796 DEBUG [ORC_GET_SPLITS #4] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:29:09,796 INFO [InputInitializer {Map 4} #0] orc.OrcInputFormat: FooterCacheHitRatio: 4/4 2018-01-05T11:29:09,796 DEBUG [InputInitializer {Map 4} #0] orc.OrcInputFormat: OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0, start=3, length=574, isOriginal=true, fileLength=752, hasFooter=false, hasBase=true, deltas=0] projected_columns_uncompressed_size: 363 2018-01-05T11:29:09,796 DEBUG [InputInitializer {Map 4} #0] orc.OrcInputFormat: OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0, start=3, length=641, isOriginal=true, fileLength=827, hasFooter=false, hasBase=true, deltas=0] projected_columns_uncompressed_size: 427 2018-01-05T11:29:09,796 DEBUG [InputInitializer {Map 4} #0] orc.OrcInputFormat: OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0, start=3, length=641, isOriginal=true, fileLength=823, hasFooter=false, hasBase=true, deltas=0] projected_columns_uncompressed_size: 398 2018-01-05T11:29:09,796 DEBUG [InputInitializer {Map 4} #0] orc.OrcInputFormat: OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0, start=3, length=586, isOriginal=true, fileLength=770, hasFooter=false, hasBase=true, deltas=0] projected_columns_uncompressed_size: 350 2018-01-05T11:29:09,796 DEBUG [InputInitializer {Map 4} #0] orc.OrcInputFormat: getSplits finished 2018-01-05T11:29:09,796 INFO [InputInitializer {Map 4} #0] io.HiveInputFormat: number of splits 4 2018-01-05T11:29:09,796 DEBUG [InputInitializer {Map 4} #0] log.PerfLogger: 2018-01-05T11:29:09,796 INFO [InputInitializer {Map 4} #0] tez.HiveSplitGenerator: Number of input splits: 4. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:29:09,796 INFO [InputInitializer {Map 4} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/713c5849-a0d5-47ce-8236-f3bb3bf77c58/map.xml 2018-01-05T11:29:09,796 DEBUG [InputInitializer {Map 4} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/713c5849-a0d5-47ce-8236-f3bb3bf77c58/map.xml 2018-01-05T11:29:09,796 DEBUG [InputInitializer {Map 4} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/713c5849-a0d5-47ce-8236-f3bb3bf77c58/map.xml 2018-01-05T11:29:09,796 DEBUG [InputInitializer {Map 4} #0] log.PerfLogger: 2018-01-05T11:29:09,796 INFO [InputInitializer {Map 4} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:09,796 DEBUG [InputInitializer {Map 4} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:09,797 DEBUG [InputInitializer {Map 4} #0] log.PerfLogger: 2018-01-05T11:29:09,797 INFO [InputInitializer {Map 4} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 4 size: 3.07KB 2018-01-05T11:29:09,797 DEBUG [InputInitializer {Map 4} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0 to src new group? true 2018-01-05T11:29:09,797 DEBUG [InputInitializer {Map 4} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0 to src new group? false 2018-01-05T11:29:09,797 DEBUG [InputInitializer {Map 4} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0 to src new group? false 2018-01-05T11:29:09,797 DEBUG [InputInitializer {Map 4} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0 to src new group? false 2018-01-05T11:29:09,797 INFO [InputInitializer {Map 4} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:29:09,797 INFO [InputInitializer {Map 4} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:29:09,797 INFO [InputInitializer {Map 4} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:29:09,797 DEBUG [InputInitializer {Map 4} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 427 2018-01-05T11:29:09,797 DEBUG [InputInitializer {Map 4} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 398 2018-01-05T11:29:09,797 DEBUG [InputInitializer {Map 4} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 350 2018-01-05T11:29:09,797 DEBUG [InputInitializer {Map 4} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 363 2018-01-05T11:29:09,797 INFO [InputInitializer {Map 4} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 28 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 1538 Original splits: 4 2018-01-05T11:29:09,797 INFO [InputInitializer {Map 4} #0] grouper.TezSplitGrouper: Desired numSplits: 1 lengthPerGroup: 1538 numLocations: 1 numSplitsPerLocation: 4 numSplitsInGroup: 4 totalLength: 1538 numOriginalSplits: 4 . Grouping by length: true count: false nodeLocalOnly: false 2018-01-05T11:29:09,797 DEBUG [InputInitializer {Map 4} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 427 2018-01-05T11:29:09,797 DEBUG [InputInitializer {Map 4} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 398 2018-01-05T11:29:09,797 DEBUG [InputInitializer {Map 4} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 398 2018-01-05T11:29:09,797 DEBUG [InputInitializer {Map 4} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 350 2018-01-05T11:29:09,797 DEBUG [InputInitializer {Map 4} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 350 2018-01-05T11:29:09,797 DEBUG [InputInitializer {Map 4} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 363 2018-01-05T11:29:09,797 DEBUG [InputInitializer {Map 4} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 363 2018-01-05T11:29:09,797 INFO [InputInitializer {Map 4} #0] grouper.TezSplitGrouper: Number of splits desired: 1 created: 1 splitsProcessed: 4 2018-01-05T11:29:09,797 INFO [InputInitializer {Map 4} #0] tez.SplitGrouper: Original split count is 4 grouped split count is 1, for bucket: 1 2018-01-05T11:29:09,797 INFO [InputInitializer {Map 4} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:29:09,797 INFO [InputInitializer {Map 4} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: b on vertex vertex_1515180518813_0001_40_00 [Map 4] 2018-01-05T11:29:09,798 INFO [App Shared Pool - #2] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_40_00 [Map 4] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:09,798 INFO [App Shared Pool - #2] impl.VertexImpl: Got updated RootInputsSpecs: {b=forAllWorkUnits=true, update=[1]} 2018-01-05T11:29:09,798 INFO [App Shared Pool - #2] impl.VertexImpl: Vertex vertex_1515180518813_0001_40_00 [Map 4] parallelism set to 1 2018-01-05T11:29:09,798 INFO [App Shared Pool - #2] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_40_00 [Map 4] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:09,798 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:VERTEX_INITIALIZED]: vertexName=Map 4, vertexId=vertex_1515180518813_0001_40_00, initRequestedTime=1515180549726, initedTime=1515180549798, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:09,798 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_40_00 [Map 4] with distanceFromRoot: 0 2018-01-05T11:29:09,798 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_40_00 [Map 4] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:29:09,798 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_40_00, startRequestedTime=1515180549734, startedTime=1515180549798 2018-01-05T11:29:09,798 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_40_00, reconfigureDoneTime=1515180549798, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:29:09,798 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_40_00 [Map 4] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:09,798 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_40_00 for vertex: vertex_1515180518813_0001_40_02 [Reducer 2] numStartedSources: 2 numSources: 2 2018-01-05T11:29:09,798 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_40_02 [Reducer 2] with distanceFromRoot: 1 2018-01-05T11:29:09,798 INFO [App Shared Pool - #2] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 4 2018-01-05T11:29:09,798 INFO [App Shared Pool - #2] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 4 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:29:09,798 INFO [App Shared Pool - #2] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_40_00 [Map 4] 2018-01-05T11:29:09,798 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_40_02, startRequestedTime=1515180549798, startedTime=1515180549798 2018-01-05T11:29:09,798 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_40_02 [Reducer 2] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:09,798 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_40_02 for vertex: vertex_1515180518813_0001_40_03 [Reducer 3] numStartedSources: 1 numSources: 1 2018-01-05T11:29:09,798 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_40_03 [Reducer 3] with distanceFromRoot: 2 2018-01-05T11:29:09,798 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_40_03, startRequestedTime=1515180549798, startedTime=1515180549798 2018-01-05T11:29:09,798 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: OnVertexStarted vertex: Reducer 2 with 0 source tasks and 1 pending tasks 2018-01-05T11:29:09,798 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_40_03 [Reducer 3] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:09,799 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Map 4 in vertex: Reducer 2 numBipartiteSourceTasks: 1 2018-01-05T11:29:09,799 INFO [App Shared Pool - #1] vertexmanager.ShuffleVertexManagerBase: OnVertexStarted vertex: Reducer 3 with 0 source tasks and 1 pending tasks 2018-01-05T11:29:09,799 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:TASK_STARTED]: vertexName=Map 4, taskId=task_1515180518813_0001_40_00_000000, scheduledTime=1515180549798, launchTime=1515180549798 2018-01-05T11:29:09,799 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Map 1 in vertex: Reducer 2 numBipartiteSourceTasks: 2 2018-01-05T11:29:09,799 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Reducer 2 in vertex: Reducer 3 numBipartiteSourceTasks: 1 2018-01-05T11:29:09,799 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_40_00 [Map 4] 2018-01-05T11:29:09,799 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_40_00_000000_0, priority=11, capability=, hosts=null 2018-01-05T11:29:09,799 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=11,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:09,799 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=11,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:09,799 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:09,799 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_40_00_000000_0 on hosts=null 2018-01-05T11:29:09,799 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_40_00_000000_0 on hosts=null 2018-01-05T11:29:09,799 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_40_00_000000_0 2018-01-05T11:29:09,799 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:09,799 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #98, task=TaskInfo{task=attempt_1515180518813_0001_40_00_000000_0, priority=11, startTime=0, containerId=null, uniqueId=97, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, to container=container_111101111_0001_00_000098 2018-01-05T11:29:09,799 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_40_00_000000_0; false 2018-01-05T11:29:09,799 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_40_00_000000_0, priority=11, startTime=311570309, containerId=container_111101111_0001_00_000098assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=2, ac=6, commF=false, disabled=false}, uniqueId=97, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:09,800 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000098 for node: 10.22.8.182:49818 2018-01-05T11:29:09,800 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000098, attempt_1515180518813_0001_40_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:09,800 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000098 for node: 10.22.8.182:49818 2018-01-05T11:29:09,800 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000098 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:09,800 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000098, launchTime=1515180549800 2018-01-05T11:29:09,800 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_40_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000098, dagName=select count (*) from tab a right ou...b.key (Stage-1), vertexName=Map 4, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{b,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{Reducer 2,org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput,1}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180549717, firstAttemptStartTime=1515180549799, currentAttemptStartTime=1515180549800} 2018-01-05T11:29:09,800 INFO [IPC Server handler 3 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000098, QueryId=djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd-40, DagId=40, VertexName=Map 4, TaskId=0, TaskAttemptId=0, SubmitTime=1515180549800 2018-01-05T11:29:09,801 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_40_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000098] on NM: [10.22.8.182:49818] 2018-01-05T11:29:09,801 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:09,801 DEBUG [IPC Server handler 3 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=40} with the ShuffleHandler 2018-01-05T11:29:09,801 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:09,801 DEBUG [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/40] 2018-01-05T11:29:09,801 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 4, taskAttemptId=attempt_1515180518813_0001_40_00_000000_0, startTime=1515180549800, containerId=container_111101111_0001_00_000098, nodeId=10.22.8.182:49818 2018-01-05T11:29:09,801 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=3, runningFragmentCount=1 2018-01-05T11:29:09,801 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_40_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:09,801 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_40_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180549799, dagStartTime=1515180549717, withinDagPriority=11, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:29:09,801 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_40_00_000000_0 : ACCEPTED 2018-01-05T11:29:09,801 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_40_00_000000_0, canFinish=true. Current state: preemptionQueueSize=1, numSlotsAvailable=3, waitQueueSize=1 2018-01-05T11:29:09,801 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_40_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180549799, dagStartTime=1515180549717, withinDagPriority=11, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:29:09,801 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_40_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:09,801 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_40_00_000000_0 to pre-emption queue 2018-01-05T11:29:09,801 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_40_00_000000_0: true 2018-01-05T11:29:09,801 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: taskOwner hashCode:1365640661 2018-01-05T11:29:09,801 INFO [TaskCommunicator # 0] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_40_00_000000_0 2018-01-05T11:29:09,801 INFO [TaskCommunicator # 0] GuaranteedTasks: Received response for attempt_1515180518813_0001_40_00_000000_0, true 2018-01-05T11:29:09,802 INFO [Task-Executor-2] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : select count (*) from tab a right ou...b.key (Stage-1), VertexName: Map 4, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_40_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=b, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=Reducer 2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ] 2018-01-05T11:29:09,802 INFO [Task-Executor-2] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1024458752, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:09,802 INFO [Task-Executor-2] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_40_00_000000_0 vertexName: Map 4 2018-01-05T11:29:09,802 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000098, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_40_00_000000_0, eventCount=1 } 2018-01-05T11:29:09,802 DEBUG [TezTR-518813_1_40_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_40 queryId: djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd instanceType: TaskRunner2Callable 2018-01-05T11:29:09,802 INFO [TezTR-518813_1_40_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_40_00_000000_0 2018-01-05T11:29:09,802 DEBUG [TezTR-518813_1_40_0_0_0] log.PerfLogger: 2018-01-05T11:29:09,803 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:09,804 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:09,805 INFO [IPC Server handler 16 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_40_00_000000_0 sent events: (0-1). 2018-01-05T11:29:09,805 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:29:09,805 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_40_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:29:09,806 DEBUG [TezTR-518813_1_40_0_0_0] log.PerfLogger: 2018-01-05T11:29:09,806 INFO [TezTR-518813_1_40_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:09,806 INFO [TezTR-518813_1_40_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:09,806 INFO [TezTR-518813_1_40_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:09,806 INFO [I/O Setup 0 Initialize: {b}] input.MRInput: b using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:29:09,806 INFO [I/O Setup 0 Initialize: {b}] input.MRInputLegacy: b MRInputLegacy deferring initialization 2018-01-05T11:29:09,806 INFO [TezTR-518813_1_40_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:09,806 INFO [TezTR-518813_1_40_0_0_0] resources.MemoryDistributor: InitialRequests=[b:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [Reducer 2:OUTPUT:10485760:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput] 2018-01-05T11:29:09,806 INFO [TezTR-518813_1_40_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:09,806 INFO [TezTR-518813_1_40_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:09,806 INFO [TezTR-518813_1_40_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 10485760, TotalRequestedScaled: 9679163.076923078, TotalJVMHeap: 1024458752, TotalAvailable: 686387363, TotalRequested/TotalJVMHeap:0.01 2018-01-05T11:29:09,806 INFO [TezTR-518813_1_40_0_0_0] resources.MemoryDistributor: Allocations=[b:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:10485760:10485760] 2018-01-05T11:29:09,806 INFO [TezTR-518813_1_40_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:09,806 INFO [TezTR-518813_1_40_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: b being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:09,806 INFO [TezTR-518813_1_40_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:29:09,806 INFO [TezTR-518813_1_40_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:09,806 INFO [I/O Setup 1 Start: {b}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: b 2018-01-05T11:29:09,806 INFO [TezTR-518813_1_40_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:09,807 INFO [TezTR-518813_1_40_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_40_00_000000_0 2018-01-05T11:29:09,807 DEBUG [TezTR-518813_1_40_0_0_0] log.PerfLogger: 2018-01-05T11:29:09,807 DEBUG [TezTR-518813_1_40_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_40_00_000000_0_10298 2018-01-05T11:29:09,807 DEBUG [TezTR-518813_1_40_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:09,807 DEBUG [TezTR-518813_1_40_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_40_00_000000_0 2018-01-05T11:29:09,807 DEBUG [TezTR-518813_1_40_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:09,807 DEBUG [TezTR-518813_1_40_0_0_0] log.PerfLogger: 2018-01-05T11:29:09,807 DEBUG [TezTR-518813_1_40_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:09,807 DEBUG [TezTR-518813_1_40_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:09,807 DEBUG [TezTR-518813_1_40_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd_Map 4__MAP_PLAN__ 2018-01-05T11:29:09,807 INFO [TezTR-518813_1_40_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/713c5849-a0d5-47ce-8236-f3bb3bf77c58/map.xml 2018-01-05T11:29:09,807 DEBUG [TezTR-518813_1_40_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/713c5849-a0d5-47ce-8236-f3bb3bf77c58/map.xml 2018-01-05T11:29:09,807 DEBUG [TezTR-518813_1_40_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/713c5849-a0d5-47ce-8236-f3bb3bf77c58/map.xml 2018-01-05T11:29:09,807 DEBUG [TezTR-518813_1_40_0_0_0] log.PerfLogger: 2018-01-05T11:29:09,807 INFO [TezTR-518813_1_40_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:09,807 DEBUG [TezTR-518813_1_40_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:09,807 DEBUG [TezTR-518813_1_40_0_0_0] log.PerfLogger: 2018-01-05T11:29:09,807 INFO [TezTR-518813_1_40_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 4 size: 3.07KB 2018-01-05T11:29:09,808 INFO [TezTR-518813_1_40_1_0_0] impl.PipelinedSorter: Reducer 2: Spilling to /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/40/output/attempt_1515180518813_0001_40_01_000000_0_10297_0/file.out 2018-01-05T11:29:09,808 DEBUG [TezTR-518813_1_40_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:09,808 INFO [TezTR-518813_1_40_0_0_0] tez.MapRecordProcessor: The input names are: [b] 2018-01-05T11:29:09,808 INFO [TezTR-518813_1_40_0_0_0] input.MRInput: Initialized MRInput: b 2018-01-05T11:29:09,808 DEBUG [TezTR-518813_1_40_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 4 2018-01-05T11:29:09,808 INFO [TezTR-518813_1_40_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/713c5849-a0d5-47ce-8236-f3bb3bf77c58/map.xml 2018-01-05T11:29:09,808 DEBUG [TezTR-518813_1_40_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,808 DEBUG [TezTR-518813_1_40_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat]} 2018-01-05T11:29:09,808 INFO [TezTR-518813_1_40_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/713c5849-a0d5-47ce-8236-f3bb3bf77c58/map.xml 2018-01-05T11:29:09,808 DEBUG [TezTR-518813_1_40_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,808 DEBUG [TezTR-518813_1_40_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.hive.ql.io.orc.OrcInputFormat 2018-01-05T11:29:09,809 DEBUG [TezTR-518813_1_40_0_0_0] impl.LlapRecordReader: Initializing for input Map 4 2018-01-05T11:29:09,809 INFO [TezTR-518813_1_40_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/713c5849-a0d5-47ce-8236-f3bb3bf77c58/map.xml 2018-01-05T11:29:09,809 DEBUG [TezTR-518813_1_40_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,809 INFO [TezTR-518813_1_40_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_40_00_000000_0 2018-01-05T11:29:09,809 INFO [TezTR-518813_1_40_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:09,809 DEBUG [TezTR-518813_1_40_0_0_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:09,809 DEBUG [TezTR-518813_1_40_0_0_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:09,809 DEBUG [TezTR-518813_1_40_0_0_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:09,809 DEBUG [TezTR-518813_1_40_0_0_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:09,809 DEBUG [TezTR-518813_1_40_0_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,809 DEBUG [TezTR-518813_1_40_0_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,810 INFO [TezTR-518813_1_40_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/713c5849-a0d5-47ce-8236-f3bb3bf77c58/map.xml 2018-01-05T11:29:09,810 DEBUG [TezTR-518813_1_40_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,810 DEBUG [TezTR-518813_1_40_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0 2018-01-05T11:29:09,810 INFO [TezTR-518813_1_40_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0, start=3, length=641, isOriginal=true, fileLength=827, hasFooter=false, hasBase=true, deltas=0], org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0, start=3, length=641, isOriginal=true, fileLength=823, hasFooter=false, hasBase=true, deltas=0], org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0, start=3, length=586, isOriginal=true, fileLength=770, hasFooter=false, hasBase=true, deltas=0], org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0, start=3, length=574, isOriginal=true, fileLength=752, hasFooter=false, hasBase=true, deltas=0]], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=2442} 2018-01-05T11:29:09,810 DEBUG [IO-Elevator-Thread-0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_40 queryId: djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd instanceType: OrcEncodedDataReader 2018-01-05T11:29:09,810 INFO [TezTR-518813_1_40_0_0_0] input.MRInput: b initialized RecordReader from event 2018-01-05T11:29:09,810 DEBUG [TezTR-518813_1_40_0_0_0] tez.MapRecordProcessor: Starting Output: Reducer 2 2018-01-05T11:29:09,810 INFO [IO-Elevator-Thread-0] LlapIoImpl: Processing data for file [6462268725460710401, 1515180546000, 827]: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0 2018-01-05T11:29:09,810 DEBUG [IO-Elevator-Thread-0] LlapIoOrc: FileSplit {3, 641}; stripes {3, 641}, 2018-01-05T11:29:09,811 DEBUG [IO-Elevator-Thread-0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,811 INFO [TezTR-518813_1_40_0_0_0] impl.ExternalSorter: Reducer 2 using: memoryMb=10, keySerializerClass=class org.apache.hadoop.hive.ql.io.HiveKey, valueSerializerClass=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization$TezBytesWritableSerializer@33add007, comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator@4f134e1e, partitioner=org.apache.tez.mapreduce.partition.MRPartitioner, serialization=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.hadoop.io.serializer.WritableSerialization, org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization, org.apache.hadoop.io.serializer.avro.AvroReflectSerialization, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:29:09,811 DEBUG [IO-Elevator-Thread-0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,811 INFO [TezTR-518813_1_40_0_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 2: , UsingHashComparator=true 2018-01-05T11:29:09,811 DEBUG [IO-Elevator-Thread-0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,811 INFO [IO-Elevator-Thread-0] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725460710401, 1515180546000, 827]): [{range start: 0 end: 40}] 2018-01-05T11:29:09,811 INFO [IO-Elevator-Thread-0] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [6462268725460710401, 1515180546000, 827], base offset 3): [{start: 0 end: 12 cache buffer: 0x7302251(1)}, {start: 12 end: 40 cache buffer: 0x2cb64d2b(1)}] 2018-01-05T11:29:09,811 INFO [IO-Elevator-Thread-0] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725460710401, 1515180546000, 827]): [{range start: 79 end: 240}] 2018-01-05T11:29:09,811 INFO [IO-Elevator-Thread-0] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [6462268725460710401, 1515180546000, 827], base offset 3): [{start: 79 end: 240 cache buffer: 0x24272ac5(1)}] 2018-01-05T11:29:09,811 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: Column at 1 2:string has no data 2018-01-05T11:29:09,811 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: int streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 200 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:29:09,811 DEBUG [IO-Elevator-Thread-0] LlapIoOrc: Done with decode 2018-01-05T11:29:09,811 DEBUG [IO-Elevator-Thread-0] LlapIoImpl: setDone called; closed false, done false, err null, pending 1 2018-01-05T11:29:09,811 DEBUG [IO-Elevator-Thread-0] impl.StatsRecordingThreadPool: Updated stats: instance: OrcEncodedDataReader thread name: IO-Elevator-Thread-0 thread id: 1252 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:09,812 INFO [TezTR-518813_1_40_0_0_0] impl.PipelinedSorter: Newly allocated block size=10485760, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=10485760, total=10485760 2018-01-05T11:29:09,812 INFO [TezTR-518813_1_40_0_0_0] impl.PipelinedSorter: Pre allocating rest of memory buffers upfront 2018-01-05T11:29:09,812 INFO [TezTR-518813_1_40_0_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 2: , UsingHashComparator=true#blocks=1, maxMemUsage=10485760, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=10485760, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=10 2018-01-05T11:29:09,812 INFO [TezTR-518813_1_40_0_0_0] impl.PipelinedSorter: Reducer 2: reserved.remaining()=10485760, reserved.metasize=5242880 2018-01-05T11:29:09,812 INFO [TezTR-518813_1_40_0_0_0] tez.MapRecordProcessor: Main input name is Map 4 2018-01-05T11:29:09,812 INFO [TezTR-518813_1_40_0_0_0] vector.VectorMapOperator: Initializing operator MAP[0] 2018-01-05T11:29:09,812 DEBUG [TezTR-518813_1_40_0_0_0] vector.VectorMapOperator: Initialization Done 0 MAP 2018-01-05T11:29:09,812 DEBUG [TezTR-518813_1_40_0_0_0] vector.VectorMapOperator: Operator 0 MAP initialized 2018-01-05T11:29:09,812 DEBUG [TezTR-518813_1_40_0_0_0] vector.VectorMapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:29:09,812 INFO [TezTR-518813_1_40_0_0_0] tez.MapRecordProcessor: Id =0 Id =2 Id =23 Id =24 <\Children> Id = 23 null<\Parent> <\RS> <\Children> Id = 2 null<\Parent> <\SEL> <\Children> <\Parent> <\TS> <\Children> <\Parent> <\MAP> 2018-01-05T11:29:09,812 INFO [TezTR-518813_1_40_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:09,812 DEBUG [TezTR-518813_1_40_0_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:09,812 INFO [TezTR-518813_1_40_0_0_0] exec.TableScanOperator: Initializing operator TS[2] 2018-01-05T11:29:09,812 DEBUG [TezTR-518813_1_40_0_0_0] exec.TableScanOperator: Initialization Done 2 TS 2018-01-05T11:29:09,812 DEBUG [TezTR-518813_1_40_0_0_0] exec.TableScanOperator: Operator 2 TS initialized 2018-01-05T11:29:09,812 DEBUG [TezTR-518813_1_40_0_0_0] exec.TableScanOperator: Initializing children of 2 TS 2018-01-05T11:29:09,812 DEBUG [TezTR-518813_1_40_0_0_0] vector.VectorSelectOperator: Initializing child 23 SEL 2018-01-05T11:29:09,812 INFO [TezTR-518813_1_40_0_0_0] vector.VectorSelectOperator: Initializing operator SEL[23] 2018-01-05T11:29:09,812 DEBUG [TezTR-518813_1_40_0_0_0] vector.VectorSelectOperator: Initialization Done 23 SEL 2018-01-05T11:29:09,812 DEBUG [TezTR-518813_1_40_0_0_0] vector.VectorSelectOperator: Operator 23 SEL initialized 2018-01-05T11:29:09,812 DEBUG [TezTR-518813_1_40_0_0_0] vector.VectorSelectOperator: Initializing children of 23 SEL 2018-01-05T11:29:09,812 DEBUG [TezTR-518813_1_40_0_0_0] reducesink.VectorReduceSinkLongOperator: Initializing child 24 RS 2018-01-05T11:29:09,812 INFO [TezTR-518813_1_40_0_0_0] reducesink.VectorReduceSinkLongOperator: Initializing operator RS[24] 2018-01-05T11:29:09,812 INFO [TezTR-518813_1_40_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/713c5849-a0d5-47ce-8236-f3bb3bf77c58/map.xml 2018-01-05T11:29:09,812 DEBUG [TezTR-518813_1_40_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,812 INFO [TezTR-518813_1_40_0_0_0] reducesink.VectorReduceSinkCommonOperator: Using tag = 1 2018-01-05T11:29:09,812 DEBUG [TezTR-518813_1_40_0_0_0] reducesink.VectorReduceSinkLongOperator: Initialization Done 24 RS 2018-01-05T11:29:09,812 DEBUG [TezTR-518813_1_40_0_0_0] reducesink.VectorReduceSinkLongOperator: Operator 24 RS initialized 2018-01-05T11:29:09,812 DEBUG [TezTR-518813_1_40_0_0_0] reducesink.VectorReduceSinkLongOperator: Initialization Done 24 RS done is reset. 2018-01-05T11:29:09,812 DEBUG [TezTR-518813_1_40_0_0_0] vector.VectorSelectOperator: Initialization Done 23 SEL done is reset. 2018-01-05T11:29:09,812 DEBUG [TezTR-518813_1_40_0_0_0] exec.TableScanOperator: Initialization Done 2 TS done is reset. 2018-01-05T11:29:09,812 INFO [TezTR-518813_1_40_0_0_0] exec.OperatorUtils: Setting output collector: RS[24] --> Reducer 2 2018-01-05T11:29:09,812 DEBUG [TezTR-518813_1_40_0_0_0] log.PerfLogger: 2018-01-05T11:29:09,812 INFO [TezTR-518813_1_40_0_0_0] reducesink.VectorReduceSinkCommonOperator: RS[24]: records written - 1 2018-01-05T11:29:09,812 INFO [TezTR-518813_1_40_0_0_0] reducesink.VectorReduceSinkCommonOperator: RS[24]: records written - 10 2018-01-05T11:29:09,812 INFO [TezTR-518813_1_40_0_0_0] reducesink.VectorReduceSinkCommonOperator: RS[24]: records written - 100 2018-01-05T11:29:09,812 INFO [TezTR-518813_1_40_0_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:09,812 INFO [TezTR-518813_1_40_0_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, warehouse.tab_part, file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0 ([6462268725460710401, 1515180546000, 827]), 0,1]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=1, NUM_ERRORS=0, ROWS_EMITTED=138, METADATA_CACHE_HIT=2, METADATA_CACHE_MISS=0, CACHE_HIT_BYTES=201, CACHE_MISS_BYTES=0, ALLOCATED_BYTES=0, ALLOCATED_USED_BYTES=0, TOTAL_IO_TIME_NS=1345788, DECODE_TIME_NS=172037, HDFS_TIME_NS=37281, CONSUMER_TIME_NS=145793 ] 2018-01-05T11:29:09,812 DEBUG [TezTR-518813_1_40_0_0_0] encoded.OrcEncodedDataReader: Encoded reader is being stopped 2018-01-05T11:29:09,812 DEBUG [TezTR-518813_1_40_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 1 of 4 2018-01-05T11:29:09,812 DEBUG [TezTR-518813_1_40_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat]} 2018-01-05T11:29:09,813 INFO [TezTR-518813_1_40_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/713c5849-a0d5-47ce-8236-f3bb3bf77c58/map.xml 2018-01-05T11:29:09,813 DEBUG [TezTR-518813_1_40_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,813 DEBUG [TezTR-518813_1_40_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.hive.ql.io.orc.OrcInputFormat 2018-01-05T11:29:09,814 DEBUG [TezTR-518813_1_40_0_0_0] impl.LlapRecordReader: Initializing for input Map 4 2018-01-05T11:29:09,814 INFO [TezTR-518813_1_40_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/713c5849-a0d5-47ce-8236-f3bb3bf77c58/map.xml 2018-01-05T11:29:09,814 DEBUG [TezTR-518813_1_40_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,814 INFO [TezTR-518813_1_40_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_40_00_000000_0 2018-01-05T11:29:09,814 INFO [TezTR-518813_1_40_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:09,814 DEBUG [TezTR-518813_1_40_0_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,814 DEBUG [TezTR-518813_1_40_0_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,814 INFO [TezTR-518813_1_40_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/713c5849-a0d5-47ce-8236-f3bb3bf77c58/map.xml 2018-01-05T11:29:09,814 DEBUG [TezTR-518813_1_40_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,814 DEBUG [TezTR-518813_1_40_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0 2018-01-05T11:29:09,814 DEBUG [IO-Elevator-Thread-1] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_40 queryId: djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd instanceType: OrcEncodedDataReader 2018-01-05T11:29:09,814 INFO [IO-Elevator-Thread-1] LlapIoImpl: Processing data for file [6462268725462206130, 1515180546000, 823]: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0 2018-01-05T11:29:09,814 DEBUG [IO-Elevator-Thread-1] LlapIoOrc: FileSplit {3, 641}; stripes {3, 641}, 2018-01-05T11:29:09,814 DEBUG [IO-Elevator-Thread-1] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,814 DEBUG [IO-Elevator-Thread-1] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,814 DEBUG [IO-Elevator-Thread-1] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,814 INFO [IO-Elevator-Thread-1] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725462206130, 1515180546000, 823]): [{range start: 0 end: 40}] 2018-01-05T11:29:09,814 INFO [IO-Elevator-Thread-1] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [6462268725462206130, 1515180546000, 823], base offset 3): [{start: 0 end: 12 cache buffer: 0x469c2dcf(1)}, {start: 12 end: 40 cache buffer: 0x2f556695(1)}] 2018-01-05T11:29:09,814 INFO [IO-Elevator-Thread-1] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725462206130, 1515180546000, 823]): [{range start: 76 end: 242}] 2018-01-05T11:29:09,814 INFO [IO-Elevator-Thread-1] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [6462268725462206130, 1515180546000, 823], base offset 3): [{start: 76 end: 242 cache buffer: 0x6e3a2cb3(1)}] 2018-01-05T11:29:09,814 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: Column at 1 2:string has no data 2018-01-05T11:29:09,814 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: int streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 200 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:29:09,815 DEBUG [IO-Elevator-Thread-1] LlapIoOrc: Done with decode 2018-01-05T11:29:09,815 DEBUG [IO-Elevator-Thread-1] LlapIoImpl: setDone called; closed false, done false, err null, pending 0 2018-01-05T11:29:09,815 DEBUG [IO-Elevator-Thread-1] impl.StatsRecordingThreadPool: Updated stats: instance: OrcEncodedDataReader thread name: IO-Elevator-Thread-1 thread id: 1309 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:09,815 INFO [TezTR-518813_1_40_0_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:09,815 INFO [TezTR-518813_1_40_0_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, warehouse.tab_part, file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0 ([6462268725462206130, 1515180546000, 823]), 0,1]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=1, NUM_ERRORS=0, ROWS_EMITTED=128, METADATA_CACHE_HIT=2, METADATA_CACHE_MISS=0, CACHE_HIT_BYTES=206, CACHE_MISS_BYTES=0, ALLOCATED_BYTES=0, ALLOCATED_USED_BYTES=0, TOTAL_IO_TIME_NS=509656, DECODE_TIME_NS=110653, HDFS_TIME_NS=26710, CONSUMER_TIME_NS=583719 ] 2018-01-05T11:29:09,815 DEBUG [TezTR-518813_1_40_0_0_0] encoded.OrcEncodedDataReader: Encoded reader is being stopped 2018-01-05T11:29:09,815 DEBUG [TezTR-518813_1_40_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 2 of 4 2018-01-05T11:29:09,815 DEBUG [TezTR-518813_1_40_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat]} 2018-01-05T11:29:09,815 INFO [TezTR-518813_1_40_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/713c5849-a0d5-47ce-8236-f3bb3bf77c58/map.xml 2018-01-05T11:29:09,815 DEBUG [TezTR-518813_1_40_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,815 DEBUG [TezTR-518813_1_40_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.hive.ql.io.orc.OrcInputFormat 2018-01-05T11:29:09,816 DEBUG [TezTR-518813_1_40_0_0_0] impl.LlapRecordReader: Initializing for input Map 4 2018-01-05T11:29:09,816 INFO [TezTR-518813_1_40_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/713c5849-a0d5-47ce-8236-f3bb3bf77c58/map.xml 2018-01-05T11:29:09,816 DEBUG [TezTR-518813_1_40_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,816 INFO [TezTR-518813_1_40_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_40_00_000000_0 2018-01-05T11:29:09,816 INFO [TezTR-518813_1_40_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:09,816 DEBUG [TezTR-518813_1_40_0_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,817 DEBUG [TezTR-518813_1_40_0_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,817 INFO [TezTR-518813_1_40_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/713c5849-a0d5-47ce-8236-f3bb3bf77c58/map.xml 2018-01-05T11:29:09,817 DEBUG [TezTR-518813_1_40_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,817 DEBUG [TezTR-518813_1_40_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0 2018-01-05T11:29:09,817 DEBUG [IO-Elevator-Thread-2] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_40 queryId: djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd instanceType: OrcEncodedDataReader 2018-01-05T11:29:09,817 INFO [IO-Elevator-Thread-2] LlapIoImpl: Processing data for file [6462268725463701859, 1515180546000, 770]: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0 2018-01-05T11:29:09,817 DEBUG [IO-Elevator-Thread-2] LlapIoOrc: FileSplit {3, 586}; stripes {3, 586}, 2018-01-05T11:29:09,817 DEBUG [IO-Elevator-Thread-2] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,817 DEBUG [IO-Elevator-Thread-2] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,817 INFO [IO-Elevator-Thread-2] impl.OrcCodecPool: Got brand-new codec ZLIB 2018-01-05T11:29:09,817 INFO [IO-Elevator-Thread-2] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725463701859, 1515180546000, 770]): [{range start: 0 end: 38}] 2018-01-05T11:29:09,817 INFO [IO-Elevator-Thread-2] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [6462268725463701859, 1515180546000, 770], base offset 3): [{start: 0 end: 11 cache buffer: 0x5a722c41(1)}, {start: 11 end: 38 cache buffer: 0x7b40d4ac(1)}] 2018-01-05T11:29:09,817 INFO [IO-Elevator-Thread-2] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725463701859, 1515180546000, 770]): [{range start: 76 end: 223}] 2018-01-05T11:29:09,817 INFO [IO-Elevator-Thread-2] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [6462268725463701859, 1515180546000, 770], base offset 3): [{start: 76 end: 223 cache buffer: 0x1ddca8ea(1)}] 2018-01-05T11:29:09,817 DEBUG [IO-Elevator-Thread-2] encoded.EncodedTreeReaderFactory: Column at 1 2:string has no data 2018-01-05T11:29:09,817 DEBUG [IO-Elevator-Thread-2] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: int streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 171 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:29:09,817 DEBUG [IO-Elevator-Thread-2] LlapIoOrc: Done with decode 2018-01-05T11:29:09,817 DEBUG [IO-Elevator-Thread-2] LlapIoImpl: setDone called; closed false, done false, err null, pending 0 2018-01-05T11:29:09,817 DEBUG [IO-Elevator-Thread-2] impl.StatsRecordingThreadPool: Updated stats: instance: OrcEncodedDataReader thread name: IO-Elevator-Thread-2 thread id: 2076 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:09,817 INFO [TezTR-518813_1_40_0_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:09,817 INFO [TezTR-518813_1_40_0_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, warehouse.tab_part, file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0 ([6462268725463701859, 1515180546000, 770]), 0,1]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=1, NUM_ERRORS=0, ROWS_EMITTED=115, METADATA_CACHE_HIT=2, METADATA_CACHE_MISS=0, CACHE_HIT_BYTES=185, CACHE_MISS_BYTES=0, ALLOCATED_BYTES=0, ALLOCATED_USED_BYTES=0, TOTAL_IO_TIME_NS=482161, DECODE_TIME_NS=101674, HDFS_TIME_NS=27118, CONSUMER_TIME_NS=127049 ] 2018-01-05T11:29:09,817 DEBUG [TezTR-518813_1_40_0_0_0] encoded.OrcEncodedDataReader: Encoded reader is being stopped 2018-01-05T11:29:09,817 DEBUG [TezTR-518813_1_40_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 3 of 4 2018-01-05T11:29:09,817 DEBUG [TezTR-518813_1_40_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat]} 2018-01-05T11:29:09,818 INFO [TezTR-518813_1_40_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/713c5849-a0d5-47ce-8236-f3bb3bf77c58/map.xml 2018-01-05T11:29:09,818 DEBUG [TezTR-518813_1_40_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,818 DEBUG [TezTR-518813_1_40_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.hive.ql.io.orc.OrcInputFormat 2018-01-05T11:29:09,818 DEBUG [TezTR-518813_1_40_0_0_0] impl.LlapRecordReader: Initializing for input Map 4 2018-01-05T11:29:09,818 INFO [TezTR-518813_1_40_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/713c5849-a0d5-47ce-8236-f3bb3bf77c58/map.xml 2018-01-05T11:29:09,818 DEBUG [TezTR-518813_1_40_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,818 INFO [TezTR-518813_1_40_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_40_00_000000_0 2018-01-05T11:29:09,818 INFO [TezTR-518813_1_40_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:09,819 DEBUG [TezTR-518813_1_40_0_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,819 DEBUG [TezTR-518813_1_40_0_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,819 INFO [TezTR-518813_1_40_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/713c5849-a0d5-47ce-8236-f3bb3bf77c58/map.xml 2018-01-05T11:29:09,819 DEBUG [TezTR-518813_1_40_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:09,819 DEBUG [TezTR-518813_1_40_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0 2018-01-05T11:29:09,819 DEBUG [IO-Elevator-Thread-3] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_40 queryId: djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd instanceType: OrcEncodedDataReader 2018-01-05T11:29:09,819 INFO [IO-Elevator-Thread-3] LlapIoImpl: Processing data for file [6462268725459214672, 1515180546000, 752]: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0 2018-01-05T11:29:09,819 DEBUG [IO-Elevator-Thread-3] LlapIoOrc: FileSplit {3, 574}; stripes {3, 574}, 2018-01-05T11:29:09,819 DEBUG [IO-Elevator-Thread-3] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,819 DEBUG [IO-Elevator-Thread-3] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:09,819 INFO [IO-Elevator-Thread-3] impl.OrcCodecPool: Got brand-new codec ZLIB 2018-01-05T11:29:09,819 INFO [IO-Elevator-Thread-3] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725459214672, 1515180546000, 752]): [{range start: 0 end: 38}] 2018-01-05T11:29:09,819 INFO [IO-Elevator-Thread-3] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [6462268725459214672, 1515180546000, 752], base offset 3): [{start: 0 end: 11 cache buffer: 0x1c0198da(1)}, {start: 11 end: 38 cache buffer: 0x55974329(1)}] 2018-01-05T11:29:09,819 INFO [IO-Elevator-Thread-3] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725459214672, 1515180546000, 752]): [{range start: 73 end: 208}] 2018-01-05T11:29:09,819 INFO [IO-Elevator-Thread-3] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [6462268725459214672, 1515180546000, 752], base offset 3): [{start: 73 end: 208 cache buffer: 0x21723a65(1)}] 2018-01-05T11:29:09,819 DEBUG [IO-Elevator-Thread-3] encoded.EncodedTreeReaderFactory: Column at 1 2:string has no data 2018-01-05T11:29:09,819 DEBUG [IO-Elevator-Thread-3] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: int streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 150 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:29:09,819 DEBUG [IO-Elevator-Thread-3] LlapIoOrc: Done with decode 2018-01-05T11:29:09,819 DEBUG [IO-Elevator-Thread-3] LlapIoImpl: setDone called; closed false, done false, err null, pending 0 2018-01-05T11:29:09,819 DEBUG [IO-Elevator-Thread-3] impl.StatsRecordingThreadPool: Updated stats: instance: OrcEncodedDataReader thread name: IO-Elevator-Thread-3 thread id: 2077 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:09,820 INFO [TezTR-518813_1_40_0_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:09,820 INFO [TezTR-518813_1_40_0_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, warehouse.tab_part, file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0 ([6462268725459214672, 1515180546000, 752]), 0,1]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=1, NUM_ERRORS=0, ROWS_EMITTED=119, METADATA_CACHE_HIT=2, METADATA_CACHE_MISS=0, CACHE_HIT_BYTES=173, CACHE_MISS_BYTES=0, ALLOCATED_BYTES=0, ALLOCATED_USED_BYTES=0, TOTAL_IO_TIME_NS=576684, DECODE_TIME_NS=139471, HDFS_TIME_NS=27059, CONSUMER_TIME_NS=125710 ] 2018-01-05T11:29:09,820 DEBUG [TezTR-518813_1_40_0_0_0] encoded.OrcEncodedDataReader: Encoded reader is being stopped 2018-01-05T11:29:09,820 DEBUG [TezTR-518813_1_40_0_0_0] log.PerfLogger: 2018-01-05T11:29:09,820 DEBUG [TezTR-518813_1_40_0_0_0] mr.ObjectCache: djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd_Map 4__MAP_PLAN__ no longer needed 2018-01-05T11:29:09,820 DEBUG [TezTR-518813_1_40_0_0_0] vector.VectorMapOperator: close called for operator MAP[0] 2018-01-05T11:29:09,820 INFO [TezTR-518813_1_40_0_0_0] vector.VectorMapOperator: Closing operator MAP[0] 2018-01-05T11:29:09,820 INFO [TezTR-518813_1_40_0_0_0] vector.VectorMapOperator: RECORDS_IN_Map_4:500, DESERIALIZE_ERRORS:0, 2018-01-05T11:29:09,820 DEBUG [TezTR-518813_1_40_0_0_0] vector.VectorMapOperator: Closing child = TS[2] 2018-01-05T11:29:09,820 DEBUG [TezTR-518813_1_40_0_0_0] exec.TableScanOperator: close called for operator TS[2] 2018-01-05T11:29:09,820 INFO [TezTR-518813_1_40_0_0_0] exec.TableScanOperator: Closing operator TS[2] 2018-01-05T11:29:09,820 DEBUG [TezTR-518813_1_40_0_0_0] exec.TableScanOperator: Closing child = SEL[23] 2018-01-05T11:29:09,820 DEBUG [TezTR-518813_1_40_0_0_0] vector.VectorSelectOperator: close called for operator SEL[23] 2018-01-05T11:29:09,820 DEBUG [TezTR-518813_1_40_0_0_0] vector.VectorSelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:09,820 INFO [TezTR-518813_1_40_0_0_0] vector.VectorSelectOperator: Closing operator SEL[23] 2018-01-05T11:29:09,820 DEBUG [TezTR-518813_1_40_0_0_0] vector.VectorSelectOperator: Closing child = RS[24] 2018-01-05T11:29:09,820 DEBUG [TezTR-518813_1_40_0_0_0] reducesink.VectorReduceSinkLongOperator: close called for operator RS[24] 2018-01-05T11:29:09,820 DEBUG [TezTR-518813_1_40_0_0_0] reducesink.VectorReduceSinkLongOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:09,820 INFO [TezTR-518813_1_40_0_0_0] reducesink.VectorReduceSinkLongOperator: Closing operator RS[24] 2018-01-05T11:29:09,820 INFO [TezTR-518813_1_40_0_0_0] reducesink.VectorReduceSinkCommonOperator: RS[24]: records written - 500 2018-01-05T11:29:09,820 INFO [TezTR-518813_1_40_0_0_0] reducesink.VectorReduceSinkLongOperator: RECORDS_OUT_INTERMEDIATE_Map_4:500, 2018-01-05T11:29:09,820 DEBUG [TezTR-518813_1_40_0_0_0] reducesink.VectorReduceSinkLongOperator: 24 Close done 2018-01-05T11:29:09,820 DEBUG [TezTR-518813_1_40_0_0_0] vector.VectorSelectOperator: 23 Close done 2018-01-05T11:29:09,820 DEBUG [TezTR-518813_1_40_0_0_0] exec.TableScanOperator: 2 Close done 2018-01-05T11:29:09,820 DEBUG [TezTR-518813_1_40_0_0_0] vector.VectorMapOperator: 0 Close done 2018-01-05T11:29:09,820 INFO [TezTR-518813_1_40_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_40_00_000000_0 2018-01-05T11:29:09,820 INFO [TezTR-518813_1_40_0_0_0] impl.PipelinedSorter: Reducer 2: Starting flush of map output 2018-01-05T11:29:09,820 INFO [TezTR-518813_1_40_0_0_0] impl.PipelinedSorter: Reducer 2: Span0.length = 500, perItem = 6 2018-01-05T11:29:09,820 INFO [TezTR-518813_1_40_0_0_0] impl.PipelinedSorter: Reducer 2: done sorting span=0, length=500, time=0 2018-01-05T11:29:09,820 INFO [TezTR-518813_1_40_0_0_0] impl.PipelinedSorter: Reducer 2: Heap = SpanIterator<0:499> (span=Span[5242880,3000]), 2018-01-05T11:29:09,820 DEBUG [TezTR-518813_1_40_1_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:09,820 INFO [TezTR-518813_1_40_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:09,820 INFO [TezTR-518813_1_40_1_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_40_01_000000_0, askedToStop=false 2018-01-05T11:29:09,820 INFO [TezTR-518813_1_40_1_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_40_01_000000_0, stopRequested=false 2018-01-05T11:29:09,820 INFO [TezTR-518813_1_40_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_40_01_000000_0: Counters: 31 [[File System Counters FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=0, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=242, INPUT_RECORDS_PROCESSED=2, INPUT_SPLIT_LENGTH_BYTES=1149, OUTPUT_RECORDS=242, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=1452, OUTPUT_BYTES_WITH_OVERHEAD=1447, OUTPUT_BYTES_PHYSICAL=1451, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=242, RECORDS_OUT_INTERMEDIATE_Map_1=242][org.apache.hadoop.hive.llap.counters.LlapIOCounters CACHE_HIT_BYTES=361, CACHE_MISS_BYTES=0, CONSUMER_TIME_NS=679564, DECODE_TIME_NS=278095, HDFS_TIME_NS=63542, METADATA_CACHE_HIT=4, NUM_DECODED_BATCHES=2, NUM_VECTOR_BATCHES=2, ROWS_EMITTED=242, SELECTED_ROWGROUPS=2, TOTAL_IO_TIME_NS=2751211]] 2018-01-05T11:29:09,820 DEBUG [TezTR-518813_1_40_1_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_40_1_0_0 thread id: 2760 scheme: file bytesRead: 64 bytesWritten: 1483 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:09,820 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_40_01_000000_0, isTaskDone=true 2018-01-05T11:29:09,820 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000097, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_40_01_000000_0, eventCount=4 } 2018-01-05T11:29:09,821 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_40_01_000000_0, creationTime=1515180549760, allocationTime=1515180549761, startTime=1515180549765, finishTime=1515180549821, timeTaken=56, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:09,821 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_40_01_000000_0 2018-01-05T11:29:09,821 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:29:09,821 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_40_01_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:09,821 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_40_01_000000_0; false 2018-01-05T11:29:09,821 INFO [Task-Executor-1] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_40_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:09,821 INFO [Task-Executor-1] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_40_01_000000_0 2018-01-05T11:29:09,821 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_40_01_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:09,821 INFO [Task-Executor-1] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000097=43 2018-01-05T11:29:09,821 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:09,821 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:09,821 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_40_01_000000_0: true 2018-01-05T11:29:09,821 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_40_01_000000_0 2018-01-05T11:29:09,821 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_40_01_000000, startTime=1515180549765, finishTime=1515180549821, timeTaken=56, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_40_01_000000_0, diagnostics=, counters=Counters: 32, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=64, FILE_BYTES_WRITTEN=1483, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=242, INPUT_RECORDS_PROCESSED=2, INPUT_SPLIT_LENGTH_BYTES=1149, OUTPUT_RECORDS=242, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=1452, OUTPUT_BYTES_WITH_OVERHEAD=1447, OUTPUT_BYTES_PHYSICAL=1451, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=242, RECORDS_OUT_INTERMEDIATE_Map_1=242, org.apache.hadoop.hive.llap.counters.LlapIOCounters, CACHE_HIT_BYTES=361, CACHE_MISS_BYTES=0, CONSUMER_TIME_NS=679564, DECODE_TIME_NS=278095, HDFS_TIME_NS=63542, METADATA_CACHE_HIT=4, NUM_DECODED_BATCHES=2, NUM_VECTOR_BATCHES=2, ROWS_EMITTED=242, SELECTED_ROWGROUPS=2, TOTAL_IO_TIME_NS=2751211 2018-01-05T11:29:09,821 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000097, QueryId=djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd-40, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:09,821 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_40_01_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=3, preemptionQueueSize=1 2018-01-05T11:29:09,821 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_40_01_000000_0 2018-01-05T11:29:09,822 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000097, QueryId=djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd-40, DagId=40, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-1, Succeeded=true, StartTime=1515180549777, EndTime=1515180549822 2018-01-05T11:29:09,822 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_40_01 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:09,822 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_40_01 [Map 1] 2018-01-05T11:29:09,822 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_40_01, initRequestedTime=1515180549717, initedTime=1515180549760, startRequestedTime=1515180549734, startedTime=1515180549760, finishTime=1515180549822, timeTaken=62, status=SUCCEEDED, diagnostics=, counters=Counters: 32, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=64, FILE_BYTES_WRITTEN=1483, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=242, INPUT_RECORDS_PROCESSED=2, INPUT_SPLIT_LENGTH_BYTES=1149, OUTPUT_RECORDS=242, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=1452, OUTPUT_BYTES_WITH_OVERHEAD=1447, OUTPUT_BYTES_PHYSICAL=1451, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=242, RECORDS_OUT_INTERMEDIATE_Map_1=242, org.apache.hadoop.hive.llap.counters.LlapIOCounters, CACHE_HIT_BYTES=361, CACHE_MISS_BYTES=0, CONSUMER_TIME_NS=679564, DECODE_TIME_NS=278095, HDFS_TIME_NS=63542, METADATA_CACHE_HIT=4, NUM_DECODED_BATCHES=2, NUM_VECTOR_BATCHES=2, ROWS_EMITTED=242, SELECTED_ROWGROUPS=2, TOTAL_IO_TIME_NS=2751211, vertexStats=firstTaskStartTime=1515180549765, firstTasksToStart=[ task_1515180518813_0001_40_01_000000 ], lastTaskFinishTime=1515180549821, lastTasksToFinish=[ task_1515180518813_0001_40_01_000000 ], minTaskDuration=56, maxTaskDuration=56, avgTaskDuration=56.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_40_01_000000 ], longestDurationTasks=[ task_1515180518813_0001_40_01_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:09,822 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_40_01 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:09,822 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000097, stoppedTime=1515180549822, exitStatus=0 2018-01-05T11:29:09,822 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000097 exited with diagnostics set to Container completed. 2018-01-05T11:29:09,822 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_40_02 [Reducer 2] attempt: attempt_1515180518813_0001_40_01_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:29:09,822 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_40_01 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=4 2018-01-05T11:29:09,822 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=4, commitInProgress=0, terminationCause=null 2018-01-05T11:29:09,822 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000097 2018-01-05T11:29:09,831 INFO [TezTR-518813_1_40_0_0_0] impl.PipelinedSorter: Reducer 2: Spilling to /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/40/output/attempt_1515180518813_0001_40_00_000000_0_10300_0/file.out 2018-01-05T11:29:09,844 DEBUG [TezTR-518813_1_40_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:09,844 INFO [TezTR-518813_1_40_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:09,844 INFO [TezTR-518813_1_40_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_40_00_000000_0, askedToStop=false 2018-01-05T11:29:09,844 INFO [TezTR-518813_1_40_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_40_00_000000_0, stopRequested=false 2018-01-05T11:29:09,844 INFO [TezTR-518813_1_40_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_40_00_000000_0: Counters: 31 [[File System Counters FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=0, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=500, INPUT_RECORDS_PROCESSED=4, INPUT_SPLIT_LENGTH_BYTES=2442, OUTPUT_RECORDS=500, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=3000, OUTPUT_BYTES_WITH_OVERHEAD=2941, OUTPUT_BYTES_PHYSICAL=2945, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_4=500, RECORDS_OUT_INTERMEDIATE_Map_4=500][org.apache.hadoop.hive.llap.counters.LlapIOCounters CACHE_HIT_BYTES=765, CACHE_MISS_BYTES=0, CONSUMER_TIME_NS=982271, DECODE_TIME_NS=523835, HDFS_TIME_NS=118168, METADATA_CACHE_HIT=8, NUM_DECODED_BATCHES=4, NUM_VECTOR_BATCHES=4, ROWS_EMITTED=500, SELECTED_ROWGROUPS=4, TOTAL_IO_TIME_NS=2914289]] 2018-01-05T11:29:09,844 DEBUG [TezTR-518813_1_40_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_40_0_0_0 thread id: 2770 scheme: file bytesRead: 64 bytesWritten: 2977 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:09,844 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_40_00_000000_0, isTaskDone=true 2018-01-05T11:29:09,844 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000098, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_40_00_000000_0, eventCount=4 } 2018-01-05T11:29:09,844 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 4, taskAttemptId=attempt_1515180518813_0001_40_00_000000_0, creationTime=1515180549799, allocationTime=1515180549800, startTime=1515180549800, finishTime=1515180549844, timeTaken=44, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:09,845 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:29:09,845 INFO [Task-Executor-2] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_40_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:09,845 INFO [Task-Executor-2] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_40_00_000000_0 2018-01-05T11:29:09,845 INFO [Task-Executor-2] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000098=43 2018-01-05T11:29:09,845 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_40_00_000000_0 2018-01-05T11:29:09,845 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_40_00_000000_0: true 2018-01-05T11:29:09,845 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_40_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:09,845 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_40_00_000000_0 2018-01-05T11:29:09,845 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_40_00_000000_0; false 2018-01-05T11:29:09,845 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000098, QueryId=djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd-40, Vertex=Map 4, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:09,845 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_40_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:29:09,845 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:TASK_FINISHED]: vertexName=Map 4, taskId=task_1515180518813_0001_40_00_000000, startTime=1515180549800, finishTime=1515180549845, timeTaken=45, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_40_00_000000_0, diagnostics=, counters=Counters: 32, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=64, FILE_BYTES_WRITTEN=2977, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=500, INPUT_RECORDS_PROCESSED=4, INPUT_SPLIT_LENGTH_BYTES=2442, OUTPUT_RECORDS=500, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=3000, OUTPUT_BYTES_WITH_OVERHEAD=2941, OUTPUT_BYTES_PHYSICAL=2945, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_4=500, RECORDS_OUT_INTERMEDIATE_Map_4=500, org.apache.hadoop.hive.llap.counters.LlapIOCounters, CACHE_HIT_BYTES=765, CACHE_MISS_BYTES=0, CONSUMER_TIME_NS=982271, DECODE_TIME_NS=523835, HDFS_TIME_NS=118168, METADATA_CACHE_HIT=8, NUM_DECODED_BATCHES=4, NUM_VECTOR_BATCHES=4, ROWS_EMITTED=500, SELECTED_ROWGROUPS=4, TOTAL_IO_TIME_NS=2914289 2018-01-05T11:29:09,845 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_40_00_000000_0 2018-01-05T11:29:09,845 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_40_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:09,845 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:09,845 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:09,845 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_40_00 [Map 4], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:09,845 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_40_00 [Map 4] 2018-01-05T11:29:09,845 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:VERTEX_FINISHED]: vertexName=Map 4, vertexId=vertex_1515180518813_0001_40_00, initRequestedTime=1515180549726, initedTime=1515180549798, startRequestedTime=1515180549734, startedTime=1515180549798, finishTime=1515180549845, timeTaken=47, status=SUCCEEDED, diagnostics=, counters=Counters: 32, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=64, FILE_BYTES_WRITTEN=2977, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=500, INPUT_RECORDS_PROCESSED=4, INPUT_SPLIT_LENGTH_BYTES=2442, OUTPUT_RECORDS=500, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=3000, OUTPUT_BYTES_WITH_OVERHEAD=2941, OUTPUT_BYTES_PHYSICAL=2945, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_4=500, RECORDS_OUT_INTERMEDIATE_Map_4=500, org.apache.hadoop.hive.llap.counters.LlapIOCounters, CACHE_HIT_BYTES=765, CACHE_MISS_BYTES=0, CONSUMER_TIME_NS=982271, DECODE_TIME_NS=523835, HDFS_TIME_NS=118168, METADATA_CACHE_HIT=8, NUM_DECODED_BATCHES=4, NUM_VECTOR_BATCHES=4, ROWS_EMITTED=500, SELECTED_ROWGROUPS=4, TOTAL_IO_TIME_NS=2914289, vertexStats=firstTaskStartTime=1515180549800, firstTasksToStart=[ task_1515180518813_0001_40_00_000000 ], lastTaskFinishTime=1515180549844, lastTasksToFinish=[ task_1515180518813_0001_40_00_000000 ], minTaskDuration=44, maxTaskDuration=44, avgTaskDuration=44.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_40_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_40_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:09,845 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_40_00 [Map 4] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:09,845 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_40_02 [Reducer 2] attempt: attempt_1515180518813_0001_40_00_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:29:09,845 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000098, stoppedTime=1515180549845, exitStatus=0 2018-01-05T11:29:09,845 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000098 exited with diagnostics set to Container completed. 2018-01-05T11:29:09,845 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: All source tasks completed. Ramping up 1 remaining tasks for vertex: Reducer 2 2018-01-05T11:29:09,845 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_40_00 [Map 4] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=4 2018-01-05T11:29:09,845 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=4, commitInProgress=0, terminationCause=null 2018-01-05T11:29:09,845 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: Scheduling 1 tasks for vertex: Reducer 2 with totalTasks: 1. 2 source tasks completed out of 2. MinSourceTaskCompletedFraction: 1.0 min: 0.05 max: 0.75 2018-01-05T11:29:09,845 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000098 2018-01-05T11:29:09,845 INFO [App Shared Pool - #2] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Map 4, destinationVertexName=Reducer 2. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:29:09,845 INFO [App Shared Pool - #2] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Map 1, destinationVertexName=Reducer 2. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:29:09,845 INFO [App Shared Pool - #2] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_40_02 [Reducer 2] 2018-01-05T11:29:09,845 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_40_02_000000, scheduledTime=1515180549845, launchTime=1515180549845 2018-01-05T11:29:09,845 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_40_02 [Reducer 2] 2018-01-05T11:29:09,845 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_40_02_000000_0, priority=29, capability=, hosts=null 2018-01-05T11:29:09,845 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=29,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:09,845 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=29,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:09,846 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:09,846 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_40_02_000000_0 on hosts=null 2018-01-05T11:29:09,846 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_40_02_000000_0 on hosts=null 2018-01-05T11:29:09,846 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_40_02_000000_0 2018-01-05T11:29:09,846 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:09,846 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #99, task=TaskInfo{task=attempt_1515180518813_0001_40_02_000000_0, priority=29, startTime=0, containerId=null, uniqueId=98, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000099 2018-01-05T11:29:09,846 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_40_02_000000_0; false 2018-01-05T11:29:09,846 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_40_02_000000_0, priority=29, startTime=311570355, containerId=container_111101111_0001_00_000099assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=98, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:09,846 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000099 for node: 10.22.8.182:49818 2018-01-05T11:29:09,846 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000099, attempt_1515180518813_0001_40_02_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:09,846 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000099 for node: 10.22.8.182:49818 2018-01-05T11:29:09,846 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000099 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:09,846 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000099, launchTime=1515180549846 2018-01-05T11:29:09,846 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=40}, sourceName=Map 4, state=S_SUCCEEDED 2018-01-05T11:29:09,847 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=40}, sourceName=Map 1, state=S_SUCCEEDED 2018-01-05T11:29:09,847 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_40_02_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000099] on NM: [10.22.8.182:49818] 2018-01-05T11:29:09,847 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:09,847 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:09,847 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_40_02_000000_0, startTime=1515180549846, containerId=container_111101111_0001_00_000099, nodeId=10.22.8.182:49818 2018-01-05T11:29:09,847 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_40_02_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000099, dagName=select count (*) from tab a right ou...b.key (Stage-1), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=2, numOutputs=1, numGroupedInputs=0, Inputs={{Map 4,org.apache.tez.runtime.library.input.OrderedGroupedKVInput,1}{Map 1,org.apache.tez.runtime.library.input.OrderedGroupedKVInput,1}}, Outputs={{Reducer 3,org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput,1}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=3, completedTaskCount=2, dagStartTime=1515180549717, firstAttemptStartTime=1515180549845, currentAttemptStartTime=1515180549846} 2018-01-05T11:29:09,847 INFO [IPC Server handler 2 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000099, QueryId=djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd-40, DagId=40, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, SubmitTime=1515180549847 2018-01-05T11:29:09,847 DEBUG [IPC Server handler 2 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=40} with the ShuffleHandler 2018-01-05T11:29:09,847 DEBUG [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/40] 2018-01-05T11:29:09,847 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:29:09,847 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_40_02_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:09,847 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_40_02_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180549845, dagStartTime=1515180549717, withinDagPriority=29, vertexParallelism= 1, selfAndUpstreamParallelism= 3, selfAndUpstreamComplete= 2}] 2018-01-05T11:29:09,847 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_40_02_000000_0 : ACCEPTED 2018-01-05T11:29:09,848 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_40_02_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:29:09,848 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_40_02_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180549845, dagStartTime=1515180549717, withinDagPriority=29, vertexParallelism= 1, selfAndUpstreamParallelism= 3, selfAndUpstreamComplete= 2} 2018-01-05T11:29:09,848 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_40_02_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:09,848 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_40_02_000000_0 to pre-emption queue 2018-01-05T11:29:09,848 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_40_02_000000_0: true 2018-01-05T11:29:09,848 INFO [TaskCommunicator # 2] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_40_02_000000_0 2018-01-05T11:29:09,848 INFO [TaskCommunicator # 2] GuaranteedTasks: Received response for attempt_1515180518813_0001_40_02_000000_0, true 2018-01-05T11:29:09,848 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: taskOwner hashCode:64272534 2018-01-05T11:29:09,848 INFO [Task-Executor-2] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : select count (*) from tab a right ou...b.key (Stage-1), VertexName: Reducer 2, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_40_02_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 4, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=Reducer 3, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput }}, ] 2018-01-05T11:29:09,848 INFO [Task-Executor-2] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=2, numOutputs=1, JVM.maxFree=1024458752, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:09,848 INFO [Task-Executor-2] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_40_02_000000_0 vertexName: Reducer 2 2018-01-05T11:29:09,848 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000099, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_40_02_000000_0, eventCount=1 } 2018-01-05T11:29:09,848 DEBUG [TezTR-518813_1_40_2_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_40 queryId: djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd instanceType: TaskRunner2Callable 2018-01-05T11:29:09,848 INFO [TezTR-518813_1_40_2_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_40_02_000000_0 2018-01-05T11:29:09,849 DEBUG [TezTR-518813_1_40_2_0_0] log.PerfLogger: 2018-01-05T11:29:09,849 INFO [I/O Setup 0 Initialize: {Map 4}] orderedgrouped.MergeManager: Initial Memory required for SHUFFLE_BUFFER=131404920 based on INPUT_BUFFER_FACTOR=0.7, for final merged output=0, using factor: 0.0 2018-01-05T11:29:09,849 INFO [I/O Setup 1 Initialize: {Map 1}] orderedgrouped.MergeManager: Initial Memory required for SHUFFLE_BUFFER=131404920 based on INPUT_BUFFER_FACTOR=0.7, for final merged output=0, using factor: 0.0 2018-01-05T11:29:09,849 INFO [I/O Setup 2 Initialize: {Reducer 3}] writers.UnorderedPartitionedKVWriter: Requested BufferSize (tez.runtime.unordered.output.buffer.size-mb) : 100 2018-01-05T11:29:09,849 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:09,849 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:09,850 INFO [IPC Server handler 15 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_40_02_000000_0 2 events [0,2) total 2 vertex_1515180518813_0001_40_02 [Reducer 2] 2018-01-05T11:29:09,850 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=2, nextPreRoutedEventId=0, eventCount=2 } 2018-01-05T11:29:09,850 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_40_02_000000_0, eventCount=2 fromEventId=0 nextFromEventId=2 2018-01-05T11:29:09,851 DEBUG [TezTR-518813_1_40_2_0_0] log.PerfLogger: 2018-01-05T11:29:09,851 INFO [TezTR-518813_1_40_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:09,851 INFO [TezTR-518813_1_40_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 3 initializers to finish 2018-01-05T11:29:09,851 INFO [TezTR-518813_1_40_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:09,851 INFO [TezTR-518813_1_40_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:09,851 INFO [TezTR-518813_1_40_2_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:09,851 INFO [TezTR-518813_1_40_2_0_0] resources.MemoryDistributor: InitialRequests=[Map 4:INPUT:131404920:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [Reducer 3:OUTPUT:104857600:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput] 2018-01-05T11:29:09,851 INFO [TezTR-518813_1_40_2_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:09,851 INFO [TezTR-518813_1_40_2_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.045, finalReserveFractionUsed=0.345 2018-01-05T11:29:09,851 INFO [TezTR-518813_1_40_2_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 3, numScaledRequests: 25, TotalRequested: 367667440, TotalRequestedScaled: 1.3034302719999999E8, TotalJVMHeap: 1024458752, TotalAvailable: 671020482, TotalRequested/TotalJVMHeap:0.36 2018-01-05T11:29:09,851 INFO [TezTR-518813_1_40_2_0_0] resources.MemoryDistributor: Allocations=[Map 4:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:131404920:131404920], [Map 1:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:131404920:131404920], [Reducer 3:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput:OUTPUT:104857600:21592746] 2018-01-05T11:29:09,851 INFO [TezTR-518813_1_40_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:09,851 INFO [TezTR-518813_1_40_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: Map 4 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:09,851 INFO [TezTR-518813_1_40_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: Map 1 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:09,851 INFO [TezTR-518813_1_40_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 2 2018-01-05T11:29:09,851 INFO [TezTR-518813_1_40_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 IOs to start 2018-01-05T11:29:09,852 DEBUG [I/O Setup 1 Start: {Map 1}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:09,852 DEBUG [I/O Setup 1 Start: {Map 1}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:09,852 DEBUG [I/O Setup 0 Start: {Map 4}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:09,852 DEBUG [I/O Setup 1 Start: {Map 1}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:09,852 DEBUG [I/O Setup 1 Start: {Map 1}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:09,852 DEBUG [I/O Setup 0 Start: {Map 4}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:09,852 DEBUG [I/O Setup 0 Start: {Map 4}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:09,852 DEBUG [I/O Setup 0 Start: {Map 4}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:09,852 INFO [I/O Setup 1 Start: {Map 1}] orderedgrouped.Shuffle: Map_1: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true 2018-01-05T11:29:09,852 INFO [I/O Setup 0 Start: {Map 4}] orderedgrouped.Shuffle: Map_4: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true 2018-01-05T11:29:09,852 INFO [I/O Setup 1 Start: {Map 1}] orderedgrouped.MergeManager: Map 1: MergerManager: memoryLimit=131404920, maxSingleShuffleLimit=32851230, mergeThreshold=86727248, ioSortFactor=10, postMergeMem=0, memToMemMergeOutputsThreshold=10 2018-01-05T11:29:09,852 INFO [I/O Setup 0 Start: {Map 4}] orderedgrouped.MergeManager: Map 4: MergerManager: memoryLimit=131404920, maxSingleShuffleLimit=32851230, mergeThreshold=86727248, ioSortFactor=10, postMergeMem=0, memToMemMergeOutputsThreshold=10 2018-01-05T11:29:09,852 INFO [I/O Setup 1 Start: {Map 1}] orderedgrouped.ShuffleScheduler: ShuffleScheduler running for sourceVertex: Map 1 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true 2018-01-05T11:29:09,852 INFO [I/O Setup 0 Start: {Map 4}] orderedgrouped.ShuffleScheduler: ShuffleScheduler running for sourceVertex: Map 4 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true 2018-01-05T11:29:09,853 INFO [I/O Setup 1 Start: {Map 1}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: Map 1 2018-01-05T11:29:09,853 INFO [I/O Setup 0 Start: {Map 4}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: Map 4 2018-01-05T11:29:09,853 INFO [TezTR-518813_1_40_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:09,853 INFO [TezTR-518813_1_40_2_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:09,853 INFO [TezTR-518813_1_40_2_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_40_02_000000_0 2018-01-05T11:29:09,853 DEBUG [TezTR-518813_1_40_2_0_0] log.PerfLogger: 2018-01-05T11:29:09,853 DEBUG [TezTR-518813_1_40_2_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_40_02_000000_0_10301 2018-01-05T11:29:09,854 DEBUG [TezTR-518813_1_40_2_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:09,854 DEBUG [TezTR-518813_1_40_2_0_0] mr.ObjectCache: Creating djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:29:09,854 INFO [TezTR-518813_1_40_2_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/cb0fa143-30e3-42b3-89f2-f3d99a479985/reduce.xml 2018-01-05T11:29:09,854 DEBUG [TezTR-518813_1_40_2_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/cb0fa143-30e3-42b3-89f2-f3d99a479985/reduce.xml 2018-01-05T11:29:09,854 DEBUG [TezTR-518813_1_40_2_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/cb0fa143-30e3-42b3-89f2-f3d99a479985/reduce.xml 2018-01-05T11:29:09,854 INFO [TezTaskEventRouter{attempt_1515180518813_0001_40_02_000000_0}] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:29:09,854 DEBUG [TezTR-518813_1_40_2_0_0] log.PerfLogger: 2018-01-05T11:29:09,854 INFO [TezTR-518813_1_40_2_0_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:29:09,854 DEBUG [TezTR-518813_1_40_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:09,854 INFO [TezTaskEventRouter{attempt_1515180518813_0001_40_02_000000_0}] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 4: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:29:09,855 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=40&reduce=0&map=attempt_1515180518813_0001_40_00_000000_0_10300 mapId: [attempt_1515180518813_0001_40_00_000000_0_10300] reduceId: [0] jobId: [job_1515180518813_0001] dagId: [40] keepAlive: false 2018-01-05T11:29:09,855 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=40&reduce=0&map=attempt_1515180518813_0001_40_01_000000_0_10297 mapId: [attempt_1515180518813_0001_40_01_000000_0_10297] reduceId: [0] jobId: [job_1515180518813_0001] dagId: [40] keepAlive: false 2018-01-05T11:29:09,855 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=40&reduce=0&map=attempt_1515180518813_0001_40_00_000000_0_10300; hash=...bI1d712hcl+KY 2018-01-05T11:29:09,855 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=40&reduce=0&map=attempt_1515180518813_0001_40_01_000000_0_10297; hash=...svAxdR/ydFdkc 2018-01-05T11:29:09,855 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=40&reduce=0&map=attempt_1515180518813_0001_40_01_000000_0_10297;reply=6/G1LUdyTdDEQ 2018-01-05T11:29:09,855 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=40&reduce=0&map=attempt_1515180518813_0001_40_00_000000_0_10300;reply=FOogPh542pPAs 2018-01-05T11:29:09,855 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=40, user='djaiswal', attemptId='attempt_1515180518813_0001_40_01_000000_0_10297'} via loader 2018-01-05T11:29:09,855 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=40, user='djaiswal', attemptId='attempt_1515180518813_0001_40_01_000000_0_10297'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:29:09,855 DEBUG [New I/O worker #3] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_40_01_000000_0_10297 not found 2018-01-05T11:29:09,855 DEBUG [TezTR-518813_1_40_2_0_0] log.PerfLogger: 2018-01-05T11:29:09,855 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=40, user='djaiswal', attemptId='attempt_1515180518813_0001_40_00_000000_0_10300'} via loader 2018-01-05T11:29:09,855 INFO [TezTR-518813_1_40_2_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 1.50KB 2018-01-05T11:29:09,855 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=40, user='djaiswal', attemptId='attempt_1515180518813_0001_40_00_000000_0_10300'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:29:09,855 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_40_00_000000_0_10300 not found 2018-01-05T11:29:09,855 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_40_01_000000_0_10297,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/40/output/attempt_1515180518813_0001_40_01_000000_0_10297/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/40/output/attempt_1515180518813_0001_40_01_000000_0_10297/file.out.index 2018-01-05T11:29:09,855 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Content Length in shuffle : 1506 2018-01-05T11:29:09,855 DEBUG [TezTR-518813_1_40_2_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:09,855 DEBUG [TezTR-518813_1_40_2_0_0] log.PerfLogger: 2018-01-05T11:29:09,855 DEBUG [TezTR-518813_1_40_2_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:09,855 DEBUG [TezTR-518813_1_40_2_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:09,855 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_40_00_000000_0_10300,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/40/output/attempt_1515180518813_0001_40_00_000000_0_10300/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/40/output/attempt_1515180518813_0001_40_00_000000_0_10300/file.out.index 2018-01-05T11:29:09,855 INFO [Fetcher_O {Map_1} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=40&reduce=0&map=attempt_1515180518813_0001_40_01_000000_0_10297 sent hash and receievd reply 0 ms 2018-01-05T11:29:09,856 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Content Length in shuffle : 3000 2018-01-05T11:29:09,856 INFO [Fetcher_O {Map_1} #0] orderedgrouped.MergeManager: CloseInMemoryFile. Current state: inMemoryMapOutputs.size=1, commitMemory=0, usedMemory=1447. Since last log: count=1, min=1447, max=1447, total=1447, avg=1447.0 2018-01-05T11:29:09,856 INFO [Fetcher_O {Map_1} #0] ShuffleScheduler.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_40_01_000000_0_10297} to MEMORY, csize=1451, dsize=1447, EndTime=1515180549856, TimeTaken=0, Rate=0.00 MB/s 2018-01-05T11:29:09,856 INFO [Fetcher_O {Map_1} #0] orderedgrouped.ShuffleScheduler: All inputs fetched for input vertex : Map 1 2018-01-05T11:29:09,856 INFO [Fetcher_O {Map_4} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=40&reduce=0&map=attempt_1515180518813_0001_40_00_000000_0_10300 sent hash and receievd reply 0 ms 2018-01-05T11:29:09,856 INFO [Fetcher_O {Map_1} #0] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:09,857 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down FetchScheduler for input: Map_1, wasInterrupted=false 2018-01-05T11:29:09,857 INFO [Fetcher_O {Map_4} #0] orderedgrouped.MergeManager: CloseInMemoryFile. Current state: inMemoryMapOutputs.size=1, commitMemory=0, usedMemory=2941. Since last log: count=1, min=2941, max=2941, total=2941, avg=2941.0 2018-01-05T11:29:09,857 INFO [Fetcher_O {Map_4} #0] ShuffleScheduler.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_40_00_000000_0_10300} to MEMORY, csize=2945, dsize=2941, EndTime=1515180549857, TimeTaken=1, Rate=2.80 MB/s 2018-01-05T11:29:09,857 INFO [Fetcher_O {Map_4} #0] orderedgrouped.ShuffleScheduler: All inputs fetched for input vertex : Map 4 2018-01-05T11:29:09,857 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:09,857 INFO [Fetcher_O {Map_4} #0] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:09,857 DEBUG [TezTR-518813_1_40_2_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:09,857 INFO [ShuffleAndMergeRunner {Map_4}] orderedgrouped.ShuffleScheduler: Shutting down FetchScheduler for input: Map_4, wasInterrupted=false 2018-01-05T11:29:09,857 INFO [TezTR-518813_1_40_2_0_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:29:09,857 INFO [ShuffleAndMergeRunner {Map_4}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:09,857 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down fetchers for input: Map_1, shutdown timetaken: 0 ms, hasFetcherExecutorStopped: true 2018-01-05T11:29:09,857 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: TotalInMemFetchStats: count=1, totalSize=1447, min=1447, max=1447, avg=1.0 2018-01-05T11:29:09,857 INFO [ShuffleAndMergeRunner {Map_4}] orderedgrouped.ShuffleScheduler: Shutting down fetchers for input: Map_4, shutdown timetaken: 0 ms, hasFetcherExecutorStopped: true 2018-01-05T11:29:09,857 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: finalMerge with #inMemoryOutputs=1, size=1447 and #onDiskOutputs=0, size=0 2018-01-05T11:29:09,857 INFO [ShuffleAndMergeRunner {Map_4}] orderedgrouped.MergeManager: TotalInMemFetchStats: count=1, totalSize=2941, min=2941, max=2941, avg=1.0 2018-01-05T11:29:09,857 INFO [ShuffleAndMergeRunner {Map_4}] orderedgrouped.MergeManager: finalMerge with #inMemoryOutputs=1, size=2941 and #onDiskOutputs=0, size=0 2018-01-05T11:29:09,870 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: merge complete for input vertex : Map_1 2018-01-05T11:29:09,870 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: Map_1: Shuffle Runner thread complete 2018-01-05T11:29:09,870 INFO [ShuffleAndMergeRunner {Map_4}] orderedgrouped.Shuffle: merge complete for input vertex : Map_4 2018-01-05T11:29:09,870 INFO [TezTR-518813_1_40_2_0_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:29:09,870 INFO [ShuffleAndMergeRunner {Map_4}] orderedgrouped.Shuffle: Map_4: Shuffle Runner thread complete 2018-01-05T11:29:09,870 INFO [TezTR-518813_1_40_2_0_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:09,870 DEBUG [TezTR-518813_1_40_2_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:09,870 INFO [TezTR-518813_1_40_2_0_0] input.OrderedGroupedKVInput: Map 1: creating ValuesIterator with comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator, keyClass=org.apache.hadoop.hive.ql.io.HiveKey, valClass=org.apache.hadoop.io.BytesWritable 2018-01-05T11:29:09,871 DEBUG [TezTR-518813_1_40_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:09,871 DEBUG [TezTR-518813_1_40_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:09,871 DEBUG [TezTR-518813_1_40_2_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[] columnTypes=[] 2018-01-05T11:29:09,871 DEBUG [TezTR-518813_1_40_2_0_0] log.PerfLogger: 2018-01-05T11:29:09,871 INFO [TezTR-518813_1_40_2_0_0] input.OrderedGroupedKVInput: Map 4: creating ValuesIterator with comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator, keyClass=org.apache.hadoop.hive.ql.io.HiveKey, valClass=org.apache.hadoop.io.BytesWritable 2018-01-05T11:29:09,871 DEBUG [TezTR-518813_1_40_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:09,871 DEBUG [TezTR-518813_1_40_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:09,871 DEBUG [TezTR-518813_1_40_2_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[] columnTypes=[] 2018-01-05T11:29:09,871 DEBUG [TezTR-518813_1_40_2_0_0] log.PerfLogger: 2018-01-05T11:29:09,871 INFO [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Initializing operator MERGEJOIN[20] 2018-01-05T11:29:09,871 INFO [TezTR-518813_1_40_2_0_0] exec.CommonJoinOperator: JOIN struct<> totalsz = 0 2018-01-05T11:29:09,871 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Initialization Done 20 MERGEJOIN 2018-01-05T11:29:09,871 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Operator 20 MERGEJOIN initialized 2018-01-05T11:29:09,871 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Initializing children of 20 MERGEJOIN 2018-01-05T11:29:09,871 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Initializing child 8 GBY 2018-01-05T11:29:09,871 INFO [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Initializing operator GBY[8] 2018-01-05T11:29:09,871 DEBUG [TezTR-518813_1_40_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:09,871 INFO [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Max hash table memory: 93860656 bytes 2018-01-05T11:29:09,871 INFO [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:29:09,871 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Initialization Done 8 GBY 2018-01-05T11:29:09,871 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Operator 8 GBY initialized 2018-01-05T11:29:09,871 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Initializing children of 8 GBY 2018-01-05T11:29:09,871 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Initializing child 9 RS 2018-01-05T11:29:09,871 INFO [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Initializing operator RS[9] 2018-01-05T11:29:09,871 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: keys size is 0 2018-01-05T11:29:09,871 INFO [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Using tag = -1 2018-01-05T11:29:09,871 DEBUG [TezTR-518813_1_40_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:09,871 DEBUG [TezTR-518813_1_40_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:09,871 DEBUG [TezTR-518813_1_40_2_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0] columnTypes=[bigint] 2018-01-05T11:29:09,871 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Initialization Done 9 RS 2018-01-05T11:29:09,871 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Operator 9 RS initialized 2018-01-05T11:29:09,871 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Initialization Done 9 RS done is reset. 2018-01-05T11:29:09,871 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Initialization Done 8 GBY done is reset. 2018-01-05T11:29:09,871 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Initialization Done 20 MERGEJOIN done is reset. 2018-01-05T11:29:09,871 INFO [TezTR-518813_1_40_2_0_0] tez.ReduceRecordProcessor: Id =20 Id =8 Id =9 <\Children> Id = 8 null<\Parent> <\RS> <\Children> Id = 20 null<\Parent> <\GBY> <\Children> <\Parent> <\MERGEJOIN> 2018-01-05T11:29:09,871 INFO [TezTR-518813_1_40_2_0_0] exec.OperatorUtils: Setting output collector: RS[9] --> Reducer 3 2018-01-05T11:29:09,871 DEBUG [TezTR-518813_1_40_2_0_0] log.PerfLogger: 2018-01-05T11:29:09,871 INFO [TezTR-518813_1_40_2_0_0] tez.ReduceRecordProcessor: Starting Output: Reducer 3 2018-01-05T11:29:09,872 INFO [TezTR-518813_1_40_2_0_0] writers.BaseUnorderedPartitionedKVWriter: Instantiating Partitioner: [org.apache.tez.mapreduce.partition.MRPartitioner] 2018-01-05T11:29:09,889 INFO [TezTR-518813_1_40_2_0_0] writers.UnorderedPartitionedKVWriter: Reducer_3: numBuffers=2, sizePerBuffer=10796372, skipBuffers=true, numPartitions=1, availableMemory=21592746, maxSingleBufferSizeBytes=2147483647, pipelinedShuffle=false, isFinalMergeEnabled=true, numPartitions=1, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:29:09,889 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,889 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,889 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,889 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,889 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,889 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,889 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,889 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,889 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,891 INFO [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,891 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,892 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,893 INFO [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,893 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,894 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,895 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,896 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,897 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,897 INFO [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,898 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,899 INFO [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,899 INFO [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,899 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,900 INFO [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,900 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,901 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,902 INFO [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,902 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,903 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,904 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,905 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,906 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,908 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,909 INFO [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,909 INFO [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,909 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,910 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,911 INFO [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,911 INFO [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,911 INFO [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,911 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,912 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,913 INFO [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,913 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,914 INFO [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,914 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,915 INFO [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,915 INFO [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,915 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,916 INFO [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,916 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,917 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,918 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,919 INFO [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,919 INFO [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,919 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,920 INFO [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,920 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] log.PerfLogger: 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] mr.ObjectCache: djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: close called for operator MERGEJOIN[20] 2018-01-05T11:29:09,921 INFO [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Closing operator MERGEJOIN[20] 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Cleaning up the operator state 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: Closing child = GBY[8] 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: close called for operator GBY[8] 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:09,921 INFO [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Closing operator GBY[8] 2018-01-05T11:29:09,921 INFO [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Begin Hash Table flush: size = 1 2018-01-05T11:29:09,921 INFO [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: keys are [] num distributions: 0 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: Closing child = RS[9] 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: close called for operator RS[9] 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:09,921 INFO [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: Closing operator RS[9] 2018-01-05T11:29:09,921 INFO [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: RECORDS_OUT_INTERMEDIATE_Reducer_2:1, 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.ReduceSinkOperator: 9 Close done 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.GroupByOperator: 8 Close done 2018-01-05T11:29:09,921 DEBUG [TezTR-518813_1_40_2_0_0] exec.CommonMergeJoinOperator: 20 Close done 2018-01-05T11:29:09,921 INFO [TezTR-518813_1_40_2_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_40_02_000000_0 2018-01-05T11:29:09,921 INFO [TezTR-518813_1_40_2_0_0] orderedgrouped.Shuffle: Shutting down Shuffle for source: Map_4 2018-01-05T11:29:09,921 INFO [TezTR-518813_1_40_2_0_0] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 4: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:29:09,921 INFO [TezTR-518813_1_40_2_0_0] orderedgrouped.Shuffle: Shutting down Shuffle for source: Map_1 2018-01-05T11:29:09,921 INFO [TezTR-518813_1_40_2_0_0] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:29:09,921 INFO [TezTR-518813_1_40_2_0_0] writers.UnorderedPartitionedKVWriter: Reducer_3: Waiting for all spills to complete : Pending : 0 2018-01-05T11:29:09,921 INFO [TezTR-518813_1_40_2_0_0] writers.UnorderedPartitionedKVWriter: Reducer_3: All spills complete 2018-01-05T11:29:09,935 DEBUG [TezTR-518813_1_40_2_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:09,935 INFO [TezTR-518813_1_40_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:09,935 INFO [TezTR-518813_1_40_2_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_40_02_000000_0, askedToStop=false 2018-01-05T11:29:09,935 INFO [TezTR-518813_1_40_2_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_40_02_000000_0, stopRequested=false 2018-01-05T11:29:09,935 INFO [TezTR-518813_1_40_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_40_02_000000_0: Counters: 34 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=462, REDUCE_INPUT_RECORDS=742, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=742, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=5, OUTPUT_BYTES_WITH_OVERHEAD=13, OUTPUT_BYTES_PHYSICAL=41, ADDITIONAL_SPILLS_BYTES_WRITTEN=4396, ADDITIONAL_SPILLS_BYTES_READ=4396, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=4396, SHUFFLE_BYTES_DECOMPRESSED=4388, SHUFFLE_BYTES_TO_MEM=4396, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=36, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][HIVE RECORDS_OUT_INTERMEDIATE_Reducer_2=1][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0]] 2018-01-05T11:29:09,936 DEBUG [TezTR-518813_1_40_2_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_40_2_0_0 thread id: 2781 scheme: file bytesRead: 0 bytesWritten: 49 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:09,936 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_40_02_000000_0, isTaskDone=true 2018-01-05T11:29:09,936 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000099, requestId=2, startIndex=2, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_40_02_000000_0, eventCount=4 } 2018-01-05T11:29:09,936 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_40_02_000000_0, creationTime=1515180549845, allocationTime=1515180549846, startTime=1515180549846, finishTime=1515180549936, timeTaken=90, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:09,936 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=2, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:09,936 INFO [Task-Executor-2] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_40_02_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:09,936 INFO [Task-Executor-2] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_40_02_000000_0 2018-01-05T11:29:09,936 INFO [Task-Executor-2] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000099=88 2018-01-05T11:29:09,936 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_40_02_000000_0: true 2018-01-05T11:29:09,936 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_40_02_000000_0 2018-01-05T11:29:09,937 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_40_02_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:09,937 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_40_02_000000_0; false 2018-01-05T11:29:09,937 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_40_02_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:09,937 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:09,937 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:09,937 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_40_02_000000, startTime=1515180549846, finishTime=1515180549937, timeTaken=91, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_40_02_000000_0, diagnostics=, counters=Counters: 39, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=49, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=462, REDUCE_INPUT_RECORDS=742, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=742, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=5, OUTPUT_BYTES_WITH_OVERHEAD=13, OUTPUT_BYTES_PHYSICAL=41, ADDITIONAL_SPILLS_BYTES_WRITTEN=4396, ADDITIONAL_SPILLS_BYTES_READ=4396, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=4396, SHUFFLE_BYTES_DECOMPRESSED=4388, SHUFFLE_BYTES_TO_MEM=4396, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=36, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, HIVE, RECORDS_OUT_INTERMEDIATE_Reducer_2=1, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0 2018-01-05T11:29:09,937 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_40_02 [Reducer 2], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:09,937 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_40_02 [Reducer 2] 2018-01-05T11:29:09,937 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:VERTEX_FINISHED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_40_02, initRequestedTime=1515180549734, initedTime=1515180549741, startRequestedTime=1515180549798, startedTime=1515180549798, finishTime=1515180549937, timeTaken=139, status=SUCCEEDED, diagnostics=, counters=Counters: 39, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=49, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=462, REDUCE_INPUT_RECORDS=742, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=742, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=5, OUTPUT_BYTES_WITH_OVERHEAD=13, OUTPUT_BYTES_PHYSICAL=41, ADDITIONAL_SPILLS_BYTES_WRITTEN=4396, ADDITIONAL_SPILLS_BYTES_READ=4396, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=4396, SHUFFLE_BYTES_DECOMPRESSED=4388, SHUFFLE_BYTES_TO_MEM=4396, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=36, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, HIVE, RECORDS_OUT_INTERMEDIATE_Reducer_2=1, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, vertexStats=firstTaskStartTime=1515180549846, firstTasksToStart=[ task_1515180518813_0001_40_02_000000 ], lastTaskFinishTime=1515180549936, lastTasksToFinish=[ task_1515180518813_0001_40_02_000000 ], minTaskDuration=90, maxTaskDuration=90, avgTaskDuration=90.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_40_02_000000 ], longestDurationTasks=[ task_1515180518813_0001_40_02_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:09,937 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_40_02 [Reducer 2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:09,937 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000099, stoppedTime=1515180549937, exitStatus=0 2018-01-05T11:29:09,937 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000099 exited with diagnostics set to Container completed. 2018-01-05T11:29:09,937 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_40_03 [Reducer 3] attempt: attempt_1515180518813_0001_40_02_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:29:09,937 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_40_02 [Reducer 2] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=4 2018-01-05T11:29:09,937 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=4, commitInProgress=0, terminationCause=null 2018-01-05T11:29:09,937 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000099 2018-01-05T11:29:09,937 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: All source tasks completed. Ramping up 1 remaining tasks for vertex: Reducer 3 2018-01-05T11:29:09,937 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: Scheduling 1 tasks for vertex: Reducer 3 with totalTasks: 1. 1 source tasks completed out of 1. MinSourceTaskCompletedFraction: 1.0 min: 0.05 max: 0.75 2018-01-05T11:29:09,937 INFO [App Shared Pool - #2] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Reducer 2, destinationVertexName=Reducer 3. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.UnorderedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:29:09,937 INFO [App Shared Pool - #2] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_40_03 [Reducer 3] 2018-01-05T11:29:09,937 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:TASK_STARTED]: vertexName=Reducer 3, taskId=task_1515180518813_0001_40_03_000000, scheduledTime=1515180549937, launchTime=1515180549937 2018-01-05T11:29:09,938 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_40_03 [Reducer 3] 2018-01-05T11:29:09,938 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_40_03_000000_0, priority=44, capability=, hosts=null 2018-01-05T11:29:09,938 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=44,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:09,938 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=44,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:09,938 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:09,938 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_40_03_000000_0 on hosts=null 2018-01-05T11:29:09,938 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_40_03_000000_0 on hosts=null 2018-01-05T11:29:09,938 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_40_03_000000_0 2018-01-05T11:29:09,938 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:09,938 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #100, task=TaskInfo{task=attempt_1515180518813_0001_40_03_000000_0, priority=44, startTime=0, containerId=null, uniqueId=99, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000100 2018-01-05T11:29:09,938 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_40_03_000000_0; false 2018-01-05T11:29:09,938 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_40_03_000000_0, priority=44, startTime=311570447, containerId=container_111101111_0001_00_000100assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=99, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:09,938 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000100 for node: 10.22.8.182:49818 2018-01-05T11:29:09,938 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000100, attempt_1515180518813_0001_40_03_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:09,938 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000100 for node: 10.22.8.182:49818 2018-01-05T11:29:09,938 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000100 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:09,938 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000100, launchTime=1515180549938 2018-01-05T11:29:09,938 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=40}, sourceName=Reducer 2, state=S_SUCCEEDED 2018-01-05T11:29:09,939 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_40_03_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000100] on NM: [10.22.8.182:49818] 2018-01-05T11:29:09,939 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:09,939 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:09,939 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 3, taskAttemptId=attempt_1515180518813_0001_40_03_000000_0, startTime=1515180549938, containerId=container_111101111_0001_00_000100, nodeId=10.22.8.182:49818 2018-01-05T11:29:09,939 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_40_03_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000100, dagName=select count (*) from tab a right ou...b.key (Stage-1), vertexName=Reducer 3, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Reducer 2,org.apache.tez.runtime.library.input.UnorderedKVInput,1}}, Outputs={{out_Reducer 3,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=4, completedTaskCount=3, dagStartTime=1515180549717, firstAttemptStartTime=1515180549938, currentAttemptStartTime=1515180549938} 2018-01-05T11:29:09,939 INFO [IPC Server handler 3 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000100, QueryId=djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd-40, DagId=40, VertexName=Reducer 3, TaskId=0, TaskAttemptId=0, SubmitTime=1515180549939 2018-01-05T11:29:09,939 DEBUG [IPC Server handler 3 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=40} with the ShuffleHandler 2018-01-05T11:29:09,939 DEBUG [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/40] 2018-01-05T11:29:09,939 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=3, runningFragmentCount=0 2018-01-05T11:29:09,939 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_40_03_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:09,939 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_40_03_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180549938, dagStartTime=1515180549717, withinDagPriority=44, vertexParallelism= 1, selfAndUpstreamParallelism= 4, selfAndUpstreamComplete= 3}] 2018-01-05T11:29:09,939 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_40_03_000000_0 : ACCEPTED 2018-01-05T11:29:09,939 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_40_03_000000_0, canFinish=true. Current state: preemptionQueueSize=1, numSlotsAvailable=3, waitQueueSize=1 2018-01-05T11:29:09,939 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_40_03_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180549938, dagStartTime=1515180549717, withinDagPriority=44, vertexParallelism= 1, selfAndUpstreamParallelism= 4, selfAndUpstreamComplete= 3} 2018-01-05T11:29:09,939 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_40_03_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:09,940 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_40_03_000000_0 to pre-emption queue 2018-01-05T11:29:09,940 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_40_03_000000_0: true 2018-01-05T11:29:09,940 INFO [TaskCommunicator # 5] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_40_03_000000_0 2018-01-05T11:29:09,940 INFO [TaskCommunicator # 5] GuaranteedTasks: Received response for attempt_1515180518813_0001_40_03_000000_0, true 2018-01-05T11:29:09,940 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: taskOwner hashCode:1365640661 2018-01-05T11:29:09,940 INFO [Task-Executor-2] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : select count (*) from tab a right ou...b.key (Stage-1), VertexName: Reducer 3, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_40_03_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Reducer 2, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:29:09,940 INFO [Task-Executor-2] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1024458752, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:09,940 INFO [Task-Executor-2] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_40_03_000000_0 vertexName: Reducer 3 2018-01-05T11:29:09,940 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000100, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_40_03_000000_0, eventCount=1 } 2018-01-05T11:29:09,940 DEBUG [TezTR-518813_1_40_3_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_40 queryId: djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd instanceType: TaskRunner2Callable 2018-01-05T11:29:09,940 INFO [TezTR-518813_1_40_3_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_40_03_000000_0 2018-01-05T11:29:09,940 DEBUG [TezTR-518813_1_40_3_0_0] log.PerfLogger: 2018-01-05T11:29:09,941 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:09,941 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:09,942 INFO [IPC Server handler 5 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_40_03_000000_0 1 events [0,1) total 1 vertex_1515180518813_0001_40_03 [Reducer 3] 2018-01-05T11:29:09,942 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:29:09,942 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_40_03_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:29:09,943 DEBUG [TezTR-518813_1_40_3_0_0] log.PerfLogger: 2018-01-05T11:29:09,943 INFO [TezTR-518813_1_40_3_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:09,943 INFO [TezTR-518813_1_40_3_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:09,943 INFO [TezTR-518813_1_40_3_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:09,943 INFO [I/O Setup 1 Initialize: {out_Reducer 3}] output.MROutput: out_Reducer 3: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:29:09,943 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:09,943 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:09,943 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:09,943 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:09,943 INFO [TezTR-518813_1_40_3_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:09,943 INFO [TezTR-518813_1_40_3_0_0] resources.MemoryDistributor: InitialRequests=[out_Reducer 3:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [Reducer 2:INPUT:131404920:org.apache.tez.runtime.library.input.UnorderedKVInput] 2018-01-05T11:29:09,943 INFO [TezTR-518813_1_40_3_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:09,943 INFO [TezTR-518813_1_40_3_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:09,943 INFO [TezTR-518813_1_40_3_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 131404920, TotalRequestedScaled: 6.570246E7, TotalJVMHeap: 1024458752, TotalAvailable: 686387363, TotalRequested/TotalJVMHeap:0.13 2018-01-05T11:29:09,943 INFO [TezTR-518813_1_40_3_0_0] resources.MemoryDistributor: Allocations=[out_Reducer 3:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.input.UnorderedKVInput:INPUT:131404920:131404920] 2018-01-05T11:29:09,943 INFO [TezTR-518813_1_40_3_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:09,943 INFO [TezTR-518813_1_40_3_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 0 2018-01-05T11:29:09,943 INFO [TezTR-518813_1_40_3_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:09,943 INFO [TezTR-518813_1_40_3_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_40_03_000000_0 2018-01-05T11:29:09,943 DEBUG [TezTR-518813_1_40_3_0_0] log.PerfLogger: 2018-01-05T11:29:09,943 DEBUG [TezTR-518813_1_40_3_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_40_03_000000_0_10305 2018-01-05T11:29:09,943 DEBUG [TezTR-518813_1_40_3_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:09,943 DEBUG [TezTR-518813_1_40_3_0_0] mr.ObjectCache: Creating djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd_Reducer 3__REDUCE_PLAN__ 2018-01-05T11:29:09,944 INFO [TezTR-518813_1_40_3_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/a50fbfb6-1cc0-4010-ae1f-62dd758e99d5/reduce.xml 2018-01-05T11:29:09,944 DEBUG [TezTR-518813_1_40_3_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/a50fbfb6-1cc0-4010-ae1f-62dd758e99d5/reduce.xml 2018-01-05T11:29:09,944 DEBUG [TezTR-518813_1_40_3_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/djaiswal/_tez_scratch_dir/a50fbfb6-1cc0-4010-ae1f-62dd758e99d5/reduce.xml 2018-01-05T11:29:09,944 DEBUG [TezTR-518813_1_40_3_0_0] log.PerfLogger: 2018-01-05T11:29:09,944 INFO [TezTR-518813_1_40_3_0_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:29:09,944 DEBUG [TezTR-518813_1_40_3_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:09,945 DEBUG [TezTR-518813_1_40_3_0_0] log.PerfLogger: 2018-01-05T11:29:09,945 INFO [TezTR-518813_1_40_3_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 3 size: 1.91KB 2018-01-05T11:29:09,945 DEBUG [TezTR-518813_1_40_3_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:09,945 DEBUG [TezTR-518813_1_40_3_0_0] log.PerfLogger: 2018-01-05T11:29:09,945 DEBUG [TezTR-518813_1_40_3_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:09,945 DEBUG [TezTR-518813_1_40_3_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:09,945 DEBUG [TezTR-518813_1_40_3_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:09,945 INFO [TezTR-518813_1_40_3_0_0] impl.SimpleFetchedInputAllocator: Reducer_2: RequestedMemory=131404920, AssignedMemory=131404920, maxSingleShuffleLimit=32851230 2018-01-05T11:29:09,945 INFO [TezTR-518813_1_40_3_0_0] impl.ShuffleManager: Reducer_2: numInputs=1, compressionCodec=NoCompressionCodec, numFetchers=1, ifileBufferSize=4096, ifileReadAheadEnabled=true, ifileReadAheadLength=4194304, localDiskFetchEnabled=false, sharedFetchEnabled=false, keepAlive=false, keepAliveMaxConnections=20, connectionTimeout=180000, readTimeout=180000, bufferSize=8192, bufferSize=8192, maxTaskOutputAtOnce=20 2018-01-05T11:29:09,946 INFO [TezTR-518813_1_40_3_0_0] impl.ShuffleInputEventHandlerImpl: Reducer 2: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:29:09,946 INFO [TezTR-518813_1_40_3_0_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:29:09,947 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=40&reduce=0&map=attempt_1515180518813_0001_40_02_000000_0_10304 mapId: [attempt_1515180518813_0001_40_02_000000_0_10304] reduceId: [0] jobId: [job_1515180518813_0001] dagId: [40] keepAlive: false 2018-01-05T11:29:09,947 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=40&reduce=0&map=attempt_1515180518813_0001_40_02_000000_0_10304; hash=...PxgDYoa6svbz0 2018-01-05T11:29:09,947 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=40&reduce=0&map=attempt_1515180518813_0001_40_02_000000_0_10304;reply=zXIHoRPMiuBw8 2018-01-05T11:29:09,947 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=40, user='djaiswal', attemptId='attempt_1515180518813_0001_40_02_000000_0_10304'} via loader 2018-01-05T11:29:09,947 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=40, user='djaiswal', attemptId='attempt_1515180518813_0001_40_02_000000_0_10304'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:29:09,947 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_40_02_000000_0_10304 not found 2018-01-05T11:29:09,947 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_40_02_000000_0_10304,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/40/output/attempt_1515180518813_0001_40_02_000000_0_10304/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/40/output/attempt_1515180518813_0001_40_02_000000_0_10304/file.out.index 2018-01-05T11:29:09,947 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Content Length in shuffle : 68 2018-01-05T11:29:09,947 INFO [Fetcher_B {Reducer_2} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=40&reduce=0&map=attempt_1515180518813_0001_40_02_000000_0_10304 sent hash and receievd reply 0 ms 2018-01-05T11:29:09,947 INFO [Fetcher_B {Reducer_2} #0] ShuffleManager.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_40_02_000000_0_10304} to MEMORY, csize=17, dsize=13, EndTime=1515180549947, TimeTaken=0, Rate=0.00 MB/s 2018-01-05T11:29:09,947 INFO [Fetcher_B {Reducer_2} #0] impl.ShuffleManager: All inputs fetched for input vertex : Reducer 2 2018-01-05T11:29:09,947 INFO [Fetcher_B {Reducer_2} #0] impl.ShuffleManager: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:09,947 INFO [TezTR-518813_1_40_3_0_0] tez.ReduceRecordProcessor: Main work is Reducer 3 2018-01-05T11:29:09,947 INFO [ShuffleRunner {Reducer_2}] impl.ShuffleManager: Reducer_2: Shutting down FetchScheduler, Was Interrupted: false 2018-01-05T11:29:09,947 INFO [TezTR-518813_1_40_3_0_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:09,947 INFO [ShuffleRunner {Reducer_2}] impl.ShuffleManager: Reducer_2: Scheduler thread completed 2018-01-05T11:29:09,947 DEBUG [TezTR-518813_1_40_3_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:09,948 DEBUG [TezTR-518813_1_40_3_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:09,948 DEBUG [TezTR-518813_1_40_3_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:09,948 DEBUG [TezTR-518813_1_40_3_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0] columnTypes=[bigint] 2018-01-05T11:29:09,948 DEBUG [TezTR-518813_1_40_3_0_0] log.PerfLogger: 2018-01-05T11:29:09,948 INFO [TezTR-518813_1_40_3_0_0] vector.VectorGroupByOperator: Initializing operator GBY[25] 2018-01-05T11:29:09,948 INFO [TezTR-518813_1_40_3_0_0] vector.VectorGroupByOperator: using global aggregation processing mode 2018-01-05T11:29:09,948 DEBUG [TezTR-518813_1_40_3_0_0] vector.VectorGroupByOperator: Initialization Done 25 GBY 2018-01-05T11:29:09,948 DEBUG [TezTR-518813_1_40_3_0_0] vector.VectorGroupByOperator: Operator 25 GBY initialized 2018-01-05T11:29:09,948 DEBUG [TezTR-518813_1_40_3_0_0] vector.VectorGroupByOperator: Initializing children of 25 GBY 2018-01-05T11:29:09,948 DEBUG [TezTR-518813_1_40_3_0_0] vector.VectorFileSinkOperator: Initializing child 26 FS 2018-01-05T11:29:09,948 INFO [TezTR-518813_1_40_3_0_0] vector.VectorFileSinkOperator: Initializing operator FS[26] 2018-01-05T11:29:09,948 DEBUG [TezTR-518813_1_40_3_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:09,948 INFO [TezTR-518813_1_40_3_0_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@42d2dda1]:[_col0]:[bigint]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@97f3845 2018-01-05T11:29:09,948 DEBUG [TezTR-518813_1_40_3_0_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:29:09,948 DEBUG [TezTR-518813_1_40_3_0_0] vector.VectorFileSinkOperator: Initialization Done 26 FS 2018-01-05T11:29:09,948 DEBUG [TezTR-518813_1_40_3_0_0] vector.VectorFileSinkOperator: Operator 26 FS initialized 2018-01-05T11:29:09,948 DEBUG [TezTR-518813_1_40_3_0_0] vector.VectorFileSinkOperator: Initialization Done 26 FS done is reset. 2018-01-05T11:29:09,948 DEBUG [TezTR-518813_1_40_3_0_0] vector.VectorGroupByOperator: Initialization Done 25 GBY done is reset. 2018-01-05T11:29:09,948 INFO [TezTR-518813_1_40_3_0_0] tez.ReduceRecordProcessor: Id =25 Id =26 <\Children> Id = 25 null<\Parent> <\FS> <\Children> <\Parent> <\GBY> 2018-01-05T11:29:09,948 DEBUG [TezTR-518813_1_40_3_0_0] log.PerfLogger: 2018-01-05T11:29:09,948 INFO [TezTR-518813_1_40_3_0_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 3 2018-01-05T11:29:09,949 INFO [TezTR-518813_1_40_3_0_0] readers.UnorderedKVReader: Num Records read: 1 2018-01-05T11:29:09,949 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000098, QueryId=djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd-40, DagId=40, VertexName=Map 4, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-2, Succeeded=true, StartTime=1515180549801, EndTime=1515180549949 2018-01-05T11:29:09,949 DEBUG [TezTR-518813_1_40_3_0_0] log.PerfLogger: 2018-01-05T11:29:09,949 DEBUG [TezTR-518813_1_40_3_0_0] mr.ObjectCache: djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd_Reducer 3__REDUCE_PLAN__ no longer needed 2018-01-05T11:29:09,949 DEBUG [TezTR-518813_1_40_3_0_0] vector.VectorGroupByOperator: close called for operator GBY[25] 2018-01-05T11:29:09,949 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_40_02_000000_0 2018-01-05T11:29:09,949 INFO [TezTR-518813_1_40_3_0_0] vector.VectorGroupByOperator: Closing operator GBY[25] 2018-01-05T11:29:09,949 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000099, QueryId=djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd-40, Vertex=Reducer 2, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:09,949 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_40_02_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=3, preemptionQueueSize=1 2018-01-05T11:29:09,949 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_40_02_000000_0 2018-01-05T11:29:09,949 INFO [TezTR-518813_1_40_3_0_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-09_435_1124461022760547009-1/_tmp.-ext-10002/000000_0 2018-01-05T11:29:09,949 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000099, QueryId=djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd-40, DagId=40, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-2, Succeeded=true, StartTime=1515180549848, EndTime=1515180549949 2018-01-05T11:29:09,949 INFO [TezTR-518813_1_40_3_0_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-09_435_1124461022760547009-1/_task_tmp.-ext-10002/_tmp.000000_0 2018-01-05T11:29:09,949 INFO [TezTR-518813_1_40_3_0_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-09_435_1124461022760547009-1/_tmp.-ext-10002/000000_0 2018-01-05T11:29:09,953 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,953 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: Status: Running (Executing on YARN cluster with App id application_1515180518813_0001) 2018-01-05T11:29:09,954 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,954 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,954 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,954 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,954 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,954 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,954 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,954 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 Map 4: 1/1 Reducer 2: 1/1 Reducer 3: 0(+1)/1 2018-01-05T11:29:09,988 INFO [TezTR-518813_1_40_3_0_0] exec.FileSinkOperator: FS[26]: records written - 1 2018-01-05T11:29:09,988 DEBUG [TezTR-518813_1_40_3_0_0] vector.VectorGroupByOperator: Closing child = FS[26] 2018-01-05T11:29:09,988 DEBUG [TezTR-518813_1_40_3_0_0] vector.VectorFileSinkOperator: close called for operator FS[26] 2018-01-05T11:29:09,988 DEBUG [TezTR-518813_1_40_3_0_0] vector.VectorFileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:09,988 INFO [TezTR-518813_1_40_3_0_0] vector.VectorFileSinkOperator: Closing operator FS[26] 2018-01-05T11:29:09,988 INFO [TezTR-518813_1_40_3_0_0] exec.FileSinkOperator: FS[26]: records written - 1 2018-01-05T11:29:09,988 INFO [TezTR-518813_1_40_3_0_0] vector.VectorFileSinkOperator: RECORDS_OUT_0:1, 2018-01-05T11:29:09,988 DEBUG [TezTR-518813_1_40_3_0_0] vector.VectorFileSinkOperator: 26 Close done 2018-01-05T11:29:09,988 DEBUG [TezTR-518813_1_40_3_0_0] vector.VectorGroupByOperator: 25 Close done 2018-01-05T11:29:09,988 INFO [TezTR-518813_1_40_3_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_40_03_000000_0 2018-01-05T11:29:09,988 INFO [TezTR-518813_1_40_3_0_0] impl.ShuffleInputEventHandlerImpl: Reducer 2: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:29:09,988 INFO [TezTR-518813_1_40_3_0_0] impl.ShuffleManager: Shutting down pending fetchers on sourceReducer_2: 0 2018-01-05T11:29:09,988 INFO [TezTR-518813_1_40_3_0_0] output.MROutput: out_Reducer 3 closed 2018-01-05T11:29:09,988 DEBUG [TezTR-518813_1_40_3_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:09,988 INFO [TezTR-518813_1_40_3_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:09,989 INFO [TezTR-518813_1_40_3_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_40_03_000000_0, askedToStop=false 2018-01-05T11:29:09,989 INFO [TezTR-518813_1_40_3_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_40_03_000000_0, stopRequested=false 2018-01-05T11:29:09,989 INFO [TezTR-518813_1_40_3_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_40_03_000000_0: Counters: 14 [[org.apache.tez.common.counters.TaskCounter NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=17, SHUFFLE_BYTES_DECOMPRESSED=13, SHUFFLE_BYTES_TO_MEM=17, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=2, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1][HIVE CREATED_FILES=1, RECORDS_OUT_0=1]] 2018-01-05T11:29:09,989 DEBUG [TezTR-518813_1_40_3_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_40_3_0_0 thread id: 2804 scheme: file bytesRead: 0 bytesWritten: 115 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:09,989 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_40_03_000000_0, isTaskDone=true 2018-01-05T11:29:09,989 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000100, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_40_03_000000_0, eventCount=2 } 2018-01-05T11:29:09,989 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 3, taskAttemptId=attempt_1515180518813_0001_40_03_000000_0, creationTime=1515180549937, allocationTime=1515180549938, startTime=1515180549938, finishTime=1515180549989, timeTaken=51, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:09,990 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:09,990 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_40_03_000000_0 2018-01-05T11:29:09,990 INFO [Task-Executor-2] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_40_03_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:09,990 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_40_03_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:09,990 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_40_03_000000_0; false 2018-01-05T11:29:09,990 INFO [Task-Executor-2] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_40_03_000000_0 2018-01-05T11:29:09,990 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_40_03_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:09,990 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:09,990 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:TASK_FINISHED]: vertexName=Reducer 3, taskId=task_1515180518813_0001_40_03_000000, startTime=1515180549938, finishTime=1515180549990, timeTaken=52, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_40_03_000000_0, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=115, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=17, SHUFFLE_BYTES_DECOMPRESSED=13, SHUFFLE_BYTES_TO_MEM=17, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=2, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1 2018-01-05T11:29:09,990 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:09,990 INFO [Task-Executor-2] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000100=50 2018-01-05T11:29:09,990 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_40_03_000000_0: true 2018-01-05T11:29:09,990 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_40_03 [Reducer 3], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:09,990 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_40_03 [Reducer 3] 2018-01-05T11:29:09,990 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_40_03_000000_0 2018-01-05T11:29:09,990 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000100, QueryId=djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd-40, Vertex=Reducer 3, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:09,990 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_40_03_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:29:09,990 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_40_03_000000_0 2018-01-05T11:29:09,991 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:VERTEX_FINISHED]: vertexName=Reducer 3, vertexId=vertex_1515180518813_0001_40_03, initRequestedTime=1515180549741, initedTime=1515180549747, startRequestedTime=1515180549798, startedTime=1515180549798, finishTime=1515180549990, timeTaken=192, status=SUCCEEDED, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=115, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=17, SHUFFLE_BYTES_DECOMPRESSED=13, SHUFFLE_BYTES_TO_MEM=17, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=2, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1, vertexStats=firstTaskStartTime=1515180549938, firstTasksToStart=[ task_1515180518813_0001_40_03_000000 ], lastTaskFinishTime=1515180549989, lastTasksToFinish=[ task_1515180518813_0001_40_03_000000 ], minTaskDuration=51, maxTaskDuration=51, avgTaskDuration=51.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_40_03_000000 ], longestDurationTasks=[ task_1515180518813_0001_40_03_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:09,991 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_40_03 [Reducer 3] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:09,991 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000100, stoppedTime=1515180549991, exitStatus=0 2018-01-05T11:29:09,991 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000100 exited with diagnostics set to Container completed. 2018-01-05T11:29:09,991 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_40_03 [Reducer 3] completed., numCompletedVertices=4, numSuccessfulVertices=4, numFailedVertices=0, numKilledVertices=0, numVertices=4 2018-01-05T11:29:09,991 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=4, numSuccessfulVertices=4, numFailedVertices=0, numKilledVertices=0, numVertices=4, commitInProgress=0, terminationCause=null 2018-01-05T11:29:09,991 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_40_03 [Reducer 3] 2018-01-05T11:29:09,991 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_40_02 [Reducer 2] 2018-01-05T11:29:09,991 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_40_01 [Map 1] 2018-01-05T11:29:09,991 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_40_00 [Map 4] 2018-01-05T11:29:09,992 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_40, queueSize=0 2018-01-05T11:29:09,993 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_40][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_40, startTime=1515180549717, finishTime=1515180549991, timeTaken=274, status=SUCCEEDED, diagnostics=, counters=Counters: 65, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=4, TOTAL_LAUNCHED_TASKS=4, OTHER_LOCAL_TASKS=2, AM_CPU_MILLISECONDS=537, AM_GC_TIME_MILLIS=0, File System Counters, FILE_BYTES_READ=128, FILE_BYTES_WRITTEN=4624, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=462, REDUCE_INPUT_RECORDS=742, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=1484, NUM_SHUFFLED_INPUTS=3, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, INPUT_RECORDS_PROCESSED=7, INPUT_SPLIT_LENGTH_BYTES=3591, OUTPUT_RECORDS=743, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=4457, OUTPUT_BYTES_WITH_OVERHEAD=4401, OUTPUT_BYTES_PHYSICAL=4437, ADDITIONAL_SPILLS_BYTES_WRITTEN=4396, ADDITIONAL_SPILLS_BYTES_READ=4396, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=2, SHUFFLE_BYTES=4413, SHUFFLE_BYTES_DECOMPRESSED=4401, SHUFFLE_BYTES_TO_MEM=4413, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=36, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, HIVE, CREATED_FILES=1, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=242, RECORDS_IN_Map_4=500, RECORDS_OUT_0=1, RECORDS_OUT_INTERMEDIATE_Map_1=242, RECORDS_OUT_INTERMEDIATE_Map_4=500, RECORDS_OUT_INTERMEDIATE_Reducer_2=1, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, org.apache.hadoop.hive.llap.counters.LlapIOCounters, CACHE_HIT_BYTES=1126, CACHE_MISS_BYTES=0, CONSUMER_TIME_NS=1661835, DECODE_TIME_NS=801930, HDFS_TIME_NS=181710, METADATA_CACHE_HIT=12, NUM_DECODED_BATCHES=6, NUM_VECTOR_BATCHES=6, ROWS_EMITTED=742, SELECTED_ROWGROUPS=6, TOTAL_IO_TIME_NS=5665500 2018-01-05T11:29:09,993 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_40 finished with state: SUCCEEDED 2018-01-05T11:29:09,993 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_40 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:29:09,993 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000100 2018-01-05T11:29:09,993 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_40, dagState=SUCCEEDED 2018-01-05T11:29:09,993 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_40_post 2018-01-05T11:29:09,993 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:29:09,993 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:29:09,993 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:29:09,993 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 40 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=4, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=4,NumRequestsWithoutLocation=4, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=4} 2018-01-05T11:29:09,993 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:29:09,993 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=select count (*) from tab a right ou...b.key (Stage-1), with id=dag_1515180518813_0001_40 2018-01-05T11:29:09,993 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 40, to 10.22.8.182:49818 2018-01-05T11:29:09,993 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=4, Remaining=0 2018-01-05T11:29:09,993 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=select count (*) from tab a right ou...b.key (Stage-1), with id=dag_1515180518813_0001_40 2018-01-05T11:29:09,993 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=40} 2018-01-05T11:29:09,993 INFO [IPC Server handler 1 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=40}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:29:09,993 INFO [IPC Server handler 1 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/40 after 300 seconds 2018-01-05T11:29:09,993 ERROR [IPC Server handler 1 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd, dag_1515180518813_0001_40 2018-01-05T11:29:09,993 INFO [IPC Server handler 1 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd 2018-01-05T11:29:09,993 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=40} = 0 2018-01-05T11:29:09,993 DEBUG [IPC Server handler 1 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=40} 2018-01-05T11:29:09,993 DEBUG [IPC Server handler 1 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:29:09,993 DEBUG [IPC Server handler 1 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=40} 2018-01-05T11:29:09,995 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,995 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 Map 4: 1/1 Reducer 2: 1/1 Reducer 3: 1/1 2018-01-05T11:29:09,995 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,996 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,996 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: TaskId for 000000_0 = 000000 2018-01-05T11:29:09,997 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,997 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,997 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,997 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,997 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,997 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: select count (*) from tab a right outer join tab_part b on a.key = b.key 2018-01-05T11:29:09,997 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:29:09,997 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@tab 2018-01-05T11:29:09,997 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@tab@ds=2008-04-08 2018-01-05T11:29:09,997 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@tab_part 2018-01-05T11:29:09,997 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@tab_part@ds=2008-04-08 2018-01-05T11:29:09,997 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/-mr-10001 2018-01-05T11:29:09,997 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,997 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,997 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:09,997 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {} 2018-01-05T11:29:09,997 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd); Time taken: 0.444 seconds 2018-01-05T11:29:09,997 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:09,997 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,997 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:09,997 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:09,997 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query select count (*) from tab a right outer join tab_part b on a.key = b.key 2018-01-05T11:29:10,015 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] mapred.FileInputFormat: Time taken to get FileStatuses: 17 2018-01-05T11:29:10,015 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:29:10,015 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 17 2018-01-05T11:29:10,015 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FetchOperator: Creating fetchTask with deserializer typeinfo: struct<_col0:bigint> 2018-01-05T11:29:10,015 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FetchOperator: deserializer properties: table properties: {columns=_col0, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=bigint, escape.delim=\} partition properties: {columns=_col0, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=bigint, escape.delim=\} 2018-01-05T11:29:10,016 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: close called for operator LIST_SINK[27] 2018-01-05T11:29:10,016 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: Closing operator LIST_SINK[27] 2018-01-05T11:29:10,016 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: 27 Close done 2018-01-05T11:29:10,016 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting result dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/-mr-10001 2018-01-05T11:29:10,017 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-09_435_1124461022760547009-1 2018-01-05T11:29:10,017 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-09_435_1124461022760547009-1 2018-01-05T11:29:10,018 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.562 seconds, Fetched: 1 row(s) 2018-01-05T11:29:10,018 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:10,018 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:10,018 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:10,018 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:10,018 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,018 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:10,018 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,018 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,018 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,018 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: explain vectorization detail select count(*) from tab a full outer join tab_part b on a.key = b.key 2018-01-05T11:29:10,018 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112910_a92bdc40-6dd0-43ae-bb85-4eb3b06c895e): explain vectorization detail select count(*) from tab a full outer join tab_part b on a.key = b.key 2018-01-05T11:29:10,018 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,018 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: explain vectorization detail select count(*) from tab a full outer join tab_part b on a.key = b.key 2018-01-05T11:29:10,019 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:10,019 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,019 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,019 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,019 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,019 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:10,019 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:29:10,019 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:10,019 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed phase 1 of Semantic Analysis 2018-01-05T11:29:10,019 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:10,019 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,019 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab 2018-01-05T11:29:10,019 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab 2018-01-05T11:29:10,022 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,023 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,023 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,023 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab_part 2018-01-05T11:29:10,023 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab_part 2018-01-05T11:29:10,026 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,026 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,026 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:10,026 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:10,026 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed getting MetaData in Semantic Analysis 2018-01-05T11:29:10,028 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,028 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,028 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_not_null_constraints : db=default tbl=tab 2018-01-05T11:29:10,028 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_not_null_constraints : db=default tbl=tab 2018-01-05T11:29:10,028 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,028 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,028 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,028 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab 2018-01-05T11:29:10,028 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab 2018-01-05T11:29:10,029 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,029 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,029 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,029 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab 2018-01-05T11:29:10,029 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab 2018-01-05T11:29:10,029 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,029 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,029 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,029 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_unique_constraints : db=default tbl=tab 2018-01-05T11:29:10,029 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_unique_constraints : db=default tbl=tab 2018-01-05T11:29:10,029 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,029 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,029 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,029 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab 2018-01-05T11:29:10,029 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab 2018-01-05T11:29:10,029 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,030 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,030 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,030 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_not_null_constraints : db=default tbl=tab_part 2018-01-05T11:29:10,030 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_not_null_constraints : db=default tbl=tab_part 2018-01-05T11:29:10,030 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,030 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,030 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,030 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:10,030 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:10,030 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,030 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,030 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,030 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:10,030 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:10,030 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,031 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,031 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,031 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_unique_constraints : db=default tbl=tab_part 2018-01-05T11:29:10,031 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_unique_constraints : db=default tbl=tab_part 2018-01-05T11:29:10,031 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,031 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,031 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,031 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab_part 2018-01-05T11:29:10,031 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab_part 2018-01-05T11:29:10,031 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,031 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,032 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:10,032 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:10,032 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:10,032 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:29:10,032 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,037 DEBUG [AMReporterQueueDrainer] impl.AMReporter: Removing am 10.22.8.182:49824 with last associated dag QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=27} from heartbeat with taskCount=0, amFailed=false, isDone=true 2018-01-05T11:29:10,041 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.sql2rel: Plan after trimming unused fields HiveAggregate(group=[{}], agg#0=[count()]) HiveProject(DUMMY=[0]) HiveJoin(condition=[=($0, $1)], joinType=[full], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:10,041 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan before removing subquery: HiveProject(_o__c0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveProject($f0=[$0]) HiveJoin(condition=[=($0, $6)], joinType=[full], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:10,042 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan just after removing subquery: HiveProject(_o__c0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveProject($f0=[$0]) HiveJoin(condition=[=($0, $6)], joinType=[full], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:10,042 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan after decorrelation: HiveProject(_o__c0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveProject($f0=[$0]) HiveJoin(condition=[=($0, $6)], joinType=[full], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:10,042 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,042 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,042 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,043 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,043 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,043 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,043 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,043 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,043 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,044 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,044 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,057 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,057 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,057 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,057 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,058 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.sql2rel: Plan after trimming unused fields HiveAggregate(group=[{}], agg#0=[count()]) HiveProject(DUMMY=[0]) HiveJoin(condition=[=($0, $1)], joinType=[full], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:10,058 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,058 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,059 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,059 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,059 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,059 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,059 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,059 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,060 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,060 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,060 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,060 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,060 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,060 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,060 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,068 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Original plan for PlanModifier HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[full], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:10,068 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after top-level introduceDerivedTable HiveProject($f0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[full], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:10,068 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after nested convertOpTree HiveProject($f0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[full], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:10,068 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after propagating order HiveProject($f0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[full], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:10,068 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after fixTopOBSchema HiveProject($f0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[full], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:10,068 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Final plan after modifier HiveProject(_c0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[full], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:10,069 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:10,069 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:10,069 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:10,069 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:10,069 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,069 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab 2018-01-05T11:29:10,069 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab 2018-01-05T11:29:10,072 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,072 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,072 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:10,072 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:10,072 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:10,072 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,072 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab_part 2018-01-05T11:29:10,072 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab_part 2018-01-05T11:29:10,075 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,075 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,076 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:10,076 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:10,076 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:10,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Table Plan for a TS[0] 2018-01-05T11:29:10,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB a{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB a{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:10,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (. (tok_table_or_col a) key) key)) 2018-01-05T11:29:10,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = a{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} starRr = null 2018-01-05T11:29:10,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(key,_col0: int)} 2018-01-05T11:29:10,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:10,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block $hdt$_0 2018-01-05T11:29:10,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block $hdt$_0 2018-01-05T11:29:10,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Table Plan for b TS[2] 2018-01-05T11:29:10,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB b{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB b{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:10,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (. (tok_table_or_col b) key) key)) 2018-01-05T11:29:10,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = b{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} starRr = null 2018-01-05T11:29:10,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(key,_col0: int)} 2018-01-05T11:29:10,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:10,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block $hdt$_1 2018-01-05T11:29:10,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block $hdt$_1 2018-01-05T11:29:10,077 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:10,077 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB $hdt$_0{(key,_col0: int)} $hdt$_1{(key,_col1: int)} after GB {((tok_functionstar count),_col0: bigint)} 2018-01-05T11:29:10,077 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (tok_functionstar count) _c0)) 2018-01-05T11:29:10,077 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = {((tok_functionstar count),_col0: bigint)} starRr = $hdt$_0{(key,_col0: int)} $hdt$_1{(key,_col1: int)} 2018-01-05T11:29:10,077 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(_c0,_col0: bigint)} 2018-01-05T11:29:10,077 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:10,077 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_018_7309300305025151710-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-10_018_7309300305025151710-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_018_7309300305025151710-1/-mr-10001 2018-01-05T11:29:10,077 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_018_7309300305025151710-1/-mr-10003 2018-01-05T11:29:10,077 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_018_7309300305025151710-1/-mr-10001 row schema: null{(_c0,_col0: bigint)} 2018-01-05T11:29:10,077 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block null 2018-01-05T11:29:10,077 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:29:10,077 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: CBO Succeeded; optimized logical plan. 2018-01-05T11:29:10,077 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Before logical optimization TS[0]-SEL[1]-RS[4]-JOIN[6]-SEL[7]-GBY[8]-RS[9]-GBY[10]-SEL[11]-FS[12] TS[2]-SEL[3]-RS[5]-JOIN[6] 2018-01-05T11:29:10,077 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,077 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,077 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,078 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] lineage.Generator: Time taken for lineage transform=1 2018-01-05T11:29:10,078 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,078 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,078 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,078 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,078 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,078 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,078 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FS(12) 2018-01-05T11:29:10,078 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(11) 2018-01-05T11:29:10,078 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for GBY(10) 2018-01-05T11:29:10,078 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(9) 2018-01-05T11:29:10,078 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for GBY(8) 2018-01-05T11:29:10,078 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(7) 2018-01-05T11:29:10,078 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for JOIN(6) 2018-01-05T11:29:10,078 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(4) 2018-01-05T11:29:10,078 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(1) 2018-01-05T11:29:10,078 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:29:10,078 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(5) 2018-01-05T11:29:10,078 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(3) 2018-01-05T11:29:10,078 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for TS(2) 2018-01-05T11:29:10,078 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SimplePredicatePushDown: After PPD: TS[0]-SEL[1]-RS[4]-JOIN[6]-SEL[7]-GBY[8]-RS[9]-GBY[10]-SEL[11]-FS[12] TS[2]-SEL[3]-RS[5]-JOIN[6] 2018-01-05T11:29:10,078 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,078 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,078 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,078 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,078 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,078 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,078 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,078 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,078 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,078 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,078 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,078 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,078 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 9 key:[] 2018-01-05T11:29:10,078 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 9 oldColExprMap: {VALUE._col0=Column[_col0]} 2018-01-05T11:29:10,078 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 9 newColExprMap: {VALUE._col0=Column[_col0]} 2018-01-05T11:29:10,079 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: JOIN 6 oldExprs: {0=[Column[KEY.reducesinkkey0]], 1=[Column[KEY.reducesinkkey0]]} 2018-01-05T11:29:10,079 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: JOIN 6 newExprs: {0=[], 1=[]} 2018-01-05T11:29:10,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 4 key:[Column[_col0]] 2018-01-05T11:29:10,079 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 4 oldColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:10,079 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 4 newColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:10,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 5 key:[Column[_col0]] 2018-01-05T11:29:10,079 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 5 oldColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:10,079 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 5 newColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:10,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.IdentityProjectRemover: Identity project remover optimization removed : SEL[11] 2018-01-05T11:29:10,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.IdentityProjectRemover: Identity project remover optimization removed : SEL[7] 2018-01-05T11:29:10,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: After logical optimization TS[0]-SEL[1]-RS[4]-JOIN[6]-GBY[8]-RS[9]-GBY[10]-FS[12] TS[2]-SEL[3]-RS[5]-JOIN[6] 2018-01-05T11:29:10,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,079 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,079 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partitions : db=default tbl=tab 2018-01-05T11:29:10,079 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions : db=default tbl=tab 2018-01-05T11:29:10,082 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.224126ms + 0.00394ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? ] 2018-01-05T11:29:10,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.164245ms + 0.052013ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (15) order by "PART_NAME" asc] 2018-01-05T11:29:10,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.103616ms + 0.119087ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (15) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:29:10,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.093423ms + 0.031298ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (15) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:10,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.082775ms + 0.007352ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (43) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:29:10,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.063778ms + 0.02114ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (43) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:10,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.120994ms + 0.023781ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (43) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:10,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.068449ms + 0.006495ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (43) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:10,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.121163ms + 0.023031ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (28) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:10,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.074138ms + 0.02223ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (43) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:29:10,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Estimated average row size: 4 2018-01-05T11:29:10,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,088 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_aggr_stats_for: db=default table=tab 2018-01-05T11:29:10,088 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_aggr_stats_for: db=default table=tab 2018-01-05T11:29:10,090 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.AggregateStatsCache: Returning aggregate stats from the cache; total hits: 15, total misses: 0, hit ratio: 1.0 2018-01-05T11:29:10,090 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: useDensityFunctionForNDVEstimation = false partsFound = 1 ColumnStatisticsObj = [ColumnStatisticsObj(colName:key, colType:int, statsData:)] 2018-01-05T11:29:10,090 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting stats (Num rows: 242 Data size: 968 Basic stats: COMPLETE Column stats: COMPLETE) on TS[0] 2018-01-05T11:29:10,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (tab): numRows: 242 dataSize: 968 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {key= colName: key colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:10,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,091 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partitions : db=default tbl=tab_part 2018-01-05T11:29:10,091 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions : db=default tbl=tab_part 2018-01-05T11:29:10,093 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,094 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.142721ms + 0.002522ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? ] 2018-01-05T11:29:10,094 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.133266ms + 0.041625ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (14) order by "PART_NAME" asc] 2018-01-05T11:29:10,094 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.072009ms + 0.052711ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (14) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:29:10,094 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000100, QueryId=djaiswal_20180105112909_b28593fd-a85e-41c8-b2d8-693518da2fcd-40, DagId=40, VertexName=Reducer 3, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-2, Succeeded=true, StartTime=1515180549940, EndTime=1515180550094 2018-01-05T11:29:10,094 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.132504ms + 0.039548ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (14) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:10,094 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.06339ms + 0.006368ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (41) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:29:10,094 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.056413ms + 0.019722ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (41) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:10,094 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.05321ms + 0.016352ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (41) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:10,094 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.050738ms + 0.005131ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (41) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:10,095 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.075495ms + 0.018405ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (26) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:10,095 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.057852ms + 0.01788ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (41) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:29:10,095 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Estimated average row size: 4 2018-01-05T11:29:10,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,097 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_aggr_stats_for: db=default table=tab_part 2018-01-05T11:29:10,097 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_aggr_stats_for: db=default table=tab_part 2018-01-05T11:29:10,099 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.AggregateStatsCache: Returning aggregate stats from the cache; total hits: 16, total misses: 0, hit ratio: 1.0 2018-01-05T11:29:10,099 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: useDensityFunctionForNDVEstimation = false partsFound = 1 ColumnStatisticsObj = [ColumnStatisticsObj(colName:key, colType:int, statsData:)] 2018-01-05T11:29:10,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on TS[2] 2018-01-05T11:29:10,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-TS[2] (tab_part): numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {key= colName: key colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:10,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 242 Data size: 968 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[1] 2018-01-05T11:29:10,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[1]: numRows: 242 dataSize: 968 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:10,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[3] 2018-01-05T11:29:10,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[3]: numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:10,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 242 Data size: 968 Basic stats: COMPLETE Column stats: COMPLETE) on RS[4] 2018-01-05T11:29:10,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[4]: numRows: 242 dataSize: 968 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {KEY.reducesinkkey0= colName: KEY.reducesinkkey0 colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:10,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on RS[5] 2018-01-05T11:29:10,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[5]: numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {KEY.reducesinkkey0= colName: KEY.reducesinkkey0 colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:10,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-JOIN[6]: detects none/multiple PK parents. 2018-01-05T11:29:10,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.JoinOperator: Setting stats (Num rows: 742 Data size: 5936 Basic stats: COMPLETE Column stats: COMPLETE) on JOIN[6] 2018-01-05T11:29:10,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-JOIN[6]: numRows: 742 dataSize: 5936 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:10,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[8]: inputSize: 5936 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:10,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[8] hashAgg: true 2018-01-05T11:29:10,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 3] STATS-GBY[8]: cardinality: 1 2018-01-05T11:29:10,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[8] 2018-01-05T11:29:10,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[8]: numRows: 1 dataSize: 8 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:10,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE) on RS[9] 2018-01-05T11:29:10,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[9]: numRows: 1 dataSize: 8 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {VALUE._col0= colName: VALUE._col0 colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:10,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[10]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:10,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[10] hashAgg: false 2018-01-05T11:29:10,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 9] STATS-GBY[10]: cardinality: 1 2018-01-05T11:29:10,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[10] 2018-01-05T11:29:10,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[10]: numRows: 1 dataSize: 8 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:10,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-FS[12]: numRows: 1 dataSize: 8 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[key]]; bucket count: 2 }) on TS[0] 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[key]]; bucket count: 4 }) on TS[2] 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: 2 }) on SEL[1] 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: 4 }) on SEL[3] 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[KEY.reducesinkkey0]]; sort column names: [[KEY.reducesinkkey0]]; bucket count: 2 }) on RS[4] 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[KEY.reducesinkkey0]]; sort column names: [[KEY.reducesinkkey0]]; bucket count: 4 }) on RS[5] 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.JoinOperator: Setting traits ({ bucket column names: [[], []]; sort column names: [[], []]; bucket count: -1 }) on JOIN[6] 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[8] 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[9] 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[10] 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FileSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on FS[12] 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,101 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Set parallelism for reduce sink RS[4] to: 1 2018-01-05T11:29:10,101 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Set parallelism for reduce sink RS[5] to: 1 2018-01-05T11:29:10,101 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConvertJoinMapJoin: Memory monitor info set to : { isLlap: true executorsPerNode: 4 maxExecutorsOverSubscribeMemory: 3 memoryOverSubscriptionFactor: 0.20000000298023224 memoryCheckInterval: 100000 noConditionalTaskSize: 10000000 adjustedNoConditionalTaskSize: 16000000 hashTableInflationFactor: 2.0 threshold: 32000000 } 2018-01-05T11:29:10,101 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConvertJoinMapJoin: Could not get a valid join position. Defaulting to position 0 2018-01-05T11:29:10,101 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConvertJoinMapJoin: Fallback to common merge join operator 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.CommonMergeJoinOperator: Setting traits ({ bucket column names: [[], []]; sort column names: [[], []]; bucket count: 0 }) on MERGEJOIN[13] 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.CommonMergeJoinOperator: Setting stats (Num rows: 742 Data size: 5936 Basic stats: COMPLETE Column stats: COMPLETE) on MERGEJOIN[13] 2018-01-05T11:29:10,101 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: FS, 12 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: GBY, 10 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 9 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: GBY, 8 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: MERGEJOIN, 13 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 4 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 1 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 5 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 3 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: TS, 2 2018-01-05T11:29:10,101 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Cycle free: true 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: Before SharedWorkOptimizer: TS[0]-SEL[1]-RS[4]-MERGEJOIN[13]-GBY[8]-RS[9]-GBY[10]-FS[12] TS[2]-SEL[3]-RS[5]-MERGEJOIN[13] 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: DPP information stored in the cache: {} 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: Sorted tables by size: [default.tab_part=2000, default.tab=968] 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: After SharedWorkOptimizer: TS[0]-SEL[1]-RS[4]-MERGEJOIN[13]-GBY[8]-RS[9]-GBY[10]-FS[12] TS[2]-SEL[3]-RS[5]-MERGEJOIN[13] 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: int|{a}key,value: string|{a}value,ds: string|{a}ds,BLOCK__OFFSET__INSIDE__FILE: bigint|{a}block__offset__inside__file,INPUT__FILE__NAME: string|{a}input__file__name,ROW__ID: struct|{a}row__id) 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[2] with rs:(key: int|{b}key,value: string|{b}value,ds: string|{b}ds,BLOCK__OFFSET__INSIDE__FILE: bigint|{b}block__offset__inside__file,INPUT__FILE__NAME: string|{b}input__file__name,ROW__ID: struct|{b}row__id) 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(_col0: int|{$hdt$_0}key) 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key]) 2018-01-05T11:29:10,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[3] with rs:(_col0: int|{$hdt$_1}key) 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_2 {} 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[3] 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key]) 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[4] with rs:(KEY.reducesinkkey0: int|{$hdt$_0}key) 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[4] 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[5] with rs:(KEY.reducesinkkey0: int|{$hdt$_1}key) 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_3 {} 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[5] 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:MERGEJOIN[13] with rs:() 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_4 {} 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_5 {} 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator MERGEJOIN[13] 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[8] with rs:(_col0: bigint|{}(tok_functionstar count)) 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op MERGEJOIN_13 {} 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[8] 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[9] with rs:(VALUE._col0: bigint|{}(tok_functionstar count)) 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_8 {} 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[9] 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[10] with rs:(_col0: bigint|{}(tok_functionstar count)) 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_9 {} 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[10] 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[12] with rs:(_col0: bigint|{}) 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_10 {} 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[12] 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: TS[0] 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[4] 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 of table a 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[4] 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: MERGEJOIN[13] 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[9] 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding reduce work (Reducer 2) for MERGEJOIN[13] 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Setting up reduce sink: RS[4] with following reduce work: Reducer 2 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[4] as parent from MERGEJOIN[13] 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[5] as parent from MERGEJOIN[13] 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[9] 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: GBY[10] 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: FS[12] 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding reduce work (Reducer 3) for GBY[10] 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Setting up reduce sink: RS[9] with following reduce work: Reducer 3 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[9] as parent from GBY[10] 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: FS[12] 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: TS[2] 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[5] 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding map work (Map 4) for TS[2] 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 of table b 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Second pass. Leaf operator: RS[5] has common downstream work: org.apache.hadoop.hive.ql.plan.MergeJoinWork@18849058 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 4 2018-01-05T11:29:10,102 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:10,103 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:10,103 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:29:10,103 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:10,103 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:10,103 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 4 2018-01-05T11:29:10,103 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:10,103 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:10,103 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:29:10,103 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Examining input format to see if vectorization is enabled. 2018-01-05T11:29:10,103 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorization is enabled for input format(s) [org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] 2018-01-05T11:29:10,103 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing MapWork... 2018-01-05T11:29:10,103 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[key], Vectorized Expression = col 0:int 2018-01-05T11:29:10,103 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator 2018-01-05T11:29:10,103 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.SelectDesc 2018-01-05T11:29:10,103 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:int 2018-01-05T11:29:10,103 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator groupby typeName int 2018-01-05T11:29:10,103 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkLongOperator 2018-01-05T11:29:10,103 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:10,103 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkLongOperator 2018-01-05T11:29:10,103 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:10,103 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [key, value, ds, ROW__ID] 2018-01-05T11:29:10,103 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [int, string, string, struct] 2018-01-05T11:29:10,103 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE, NONE, NONE, NONE] 2018-01-05T11:29:10,103 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 1 2018-01-05T11:29:10,103 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 1 2018-01-05T11:29:10,103 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:10,103 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:10,103 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Examining input format to see if vectorization is enabled. 2018-01-05T11:29:10,103 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorization is enabled for input format(s) [org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] 2018-01-05T11:29:10,103 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing MapWork... 2018-01-05T11:29:10,103 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[key], Vectorized Expression = col 0:int 2018-01-05T11:29:10,103 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator 2018-01-05T11:29:10,103 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.SelectDesc 2018-01-05T11:29:10,103 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:int 2018-01-05T11:29:10,103 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator groupby typeName int 2018-01-05T11:29:10,103 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkLongOperator 2018-01-05T11:29:10,104 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:10,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkLongOperator 2018-01-05T11:29:10,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:10,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [key, value, ds, ROW__ID] 2018-01-05T11:29:10,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [int, string, string, struct] 2018-01-05T11:29:10,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE, NONE, NONE, NONE] 2018-01-05T11:29:10,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 1 2018-01-05T11:29:10,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 1 2018-01-05T11:29:10,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:10,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:10,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Using reduce tag 0 2018-01-05T11:29:10,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0] columnTypes=[bigint] 2018-01-05T11:29:10,104 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing ReduceWork... 2018-01-05T11:29:10,104 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vector GROUP BY operator will use processing mode GLOBAL 2018-01-05T11:29:10,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[VALUE._col0], Vectorized Expression = col 0:bigint 2018-01-05T11:29:10,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator 2018-01-05T11:29:10,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.GroupByDesc 2018-01-05T11:29:10,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorFileSinkOperator 2018-01-05T11:29:10,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.FileSinkDesc 2018-01-05T11:29:10,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [VALUE._col0] 2018-01-05T11:29:10,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [bigint] 2018-01-05T11:29:10,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE] 2018-01-05T11:29:10,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 0 2018-01-05T11:29:10,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 0 2018-01-05T11:29:10,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:10,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:10,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:29:10,104 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: llap mode: only 2018-01-05T11:29:10,104 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:29:10,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key' 2018-01-05T11:29:10,104 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:10,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@7a7a07c2 can be done in LLAP 2018-01-05T11:29:10,104 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Map 4 2018-01-05T11:29:10,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key' 2018-01-05T11:29:10,104 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:10,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@5e644480 can be done in LLAP 2018-01-05T11:29:10,104 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Reducer 2 2018-01-05T11:29:10,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'count()' 2018-01-05T11:29:10,104 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:10,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MergeJoinWork@18849058 can be done in LLAP 2018-01-05T11:29:10,104 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Reducer 3 2018-01-05T11:29:10,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'count(VALUE._col0)' 2018-01-05T11:29:10,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:29:10,104 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:10,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@748a19c4 can be done in LLAP 2018-01-05T11:29:10,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,104 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed plan generation 2018-01-05T11:29:10,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: validation start 2018-01-05T11:29:10,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:29:10,105 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:10,105 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,105 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:Explain, type:string, comment:null)], properties:null) 2018-01-05T11:29:10,105 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,105 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:10,105 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {getNotNullConstraints_(NotNullConstraintsRequest, )=0, isCompatibleWith_(Configuration, )=0, getAggrColStatsFor_(String, String, List, List, )=6, getTable_(String, String, )=14, flushCache_()=0, getUniqueConstraints_(UniqueConstraintsRequest, )=0, getPrimaryKeys_(PrimaryKeysRequest, )=2, listPartitions_(String, String, short, )=15, getForeignKeys_(ForeignKeysRequest, )=1} 2018-01-05T11:29:10,105 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112910_a92bdc40-6dd0-43ae-bb85-4eb3b06c895e); Time taken: 0.087 seconds 2018-01-05T11:29:10,105 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:10,105 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,105 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112910_a92bdc40-6dd0-43ae-bb85-4eb3b06c895e): explain vectorization detail select count(*) from tab a full outer join tab_part b on a.key = b.key 2018-01-05T11:29:10,106 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,106 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: explain vectorization detail select count(*) from tab a full outer join tab_part b on a.key = b.key 2018-01-05T11:29:10,106 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:29:10,106 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,106 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,106 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,106 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,106 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-3:EXPLAIN] in serial mode 2018-01-05T11:29:10,154 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,154 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,154 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: explain vectorization detail select count(*) from tab a full outer join tab_part b on a.key = b.key 2018-01-05T11:29:10,154 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:29:10,154 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,154 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,154 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:10,154 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {} 2018-01-05T11:29:10,154 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112910_a92bdc40-6dd0-43ae-bb85-4eb3b06c895e); Time taken: 0.049 seconds 2018-01-05T11:29:10,154 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:10,154 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,154 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:10,154 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,154 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query explain vectorization detail select count(*) from tab a full outer join tab_part b on a.key = b.key 2018-01-05T11:29:10,155 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting result file: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_018_7309300305025151710-1/-local-10004 2018-01-05T11:29:10,155 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_018_7309300305025151710-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-10_018_7309300305025151710-1 2018-01-05T11:29:10,155 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_018_7309300305025151710-1 2018-01-05T11:29:10,156 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.136 seconds, Fetched: 171 row(s) 2018-01-05T11:29:10,156 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:10,156 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:10,156 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:10,156 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:10,156 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,156 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:10,156 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,156 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,156 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,156 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: select count(*) from tab a full outer join tab_part b on a.key = b.key 2018-01-05T11:29:10,156 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca): select count(*) from tab a full outer join tab_part b on a.key = b.key 2018-01-05T11:29:10,156 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,156 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: select count(*) from tab a full outer join tab_part b on a.key = b.key 2018-01-05T11:29:10,156 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:10,156 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,156 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,156 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,156 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,156 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:10,156 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:29:10,157 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:10,157 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed phase 1 of Semantic Analysis 2018-01-05T11:29:10,157 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:10,157 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,157 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab 2018-01-05T11:29:10,157 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab 2018-01-05T11:29:10,160 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,160 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,160 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,160 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab_part 2018-01-05T11:29:10,160 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab_part 2018-01-05T11:29:10,162 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,163 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,163 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:10,163 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:10,175 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed getting MetaData in Semantic Analysis 2018-01-05T11:29:10,176 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,176 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,176 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_not_null_constraints : db=default tbl=tab 2018-01-05T11:29:10,176 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_not_null_constraints : db=default tbl=tab 2018-01-05T11:29:10,176 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,176 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,176 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,176 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab 2018-01-05T11:29:10,176 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab 2018-01-05T11:29:10,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,177 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab 2018-01-05T11:29:10,177 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab 2018-01-05T11:29:10,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,177 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_unique_constraints : db=default tbl=tab 2018-01-05T11:29:10,177 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_unique_constraints : db=default tbl=tab 2018-01-05T11:29:10,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,177 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab 2018-01-05T11:29:10,177 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab 2018-01-05T11:29:10,178 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,178 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,178 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,178 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_not_null_constraints : db=default tbl=tab_part 2018-01-05T11:29:10,178 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_not_null_constraints : db=default tbl=tab_part 2018-01-05T11:29:10,178 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,178 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,178 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,178 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:10,178 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:10,178 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,178 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,178 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,178 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:10,178 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:10,179 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,179 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,179 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,179 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_unique_constraints : db=default tbl=tab_part 2018-01-05T11:29:10,179 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_unique_constraints : db=default tbl=tab_part 2018-01-05T11:29:10,179 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,179 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,179 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,179 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab_part 2018-01-05T11:29:10,179 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab_part 2018-01-05T11:29:10,179 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,179 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,180 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:10,180 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:10,180 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:10,180 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:29:10,180 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,191 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.sql2rel: Plan after trimming unused fields HiveAggregate(group=[{}], agg#0=[count()]) HiveProject(DUMMY=[0]) HiveJoin(condition=[=($0, $1)], joinType=[full], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:10,192 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan before removing subquery: HiveProject(_o__c0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveProject($f0=[$0]) HiveJoin(condition=[=($0, $6)], joinType=[full], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:10,192 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan just after removing subquery: HiveProject(_o__c0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveProject($f0=[$0]) HiveJoin(condition=[=($0, $6)], joinType=[full], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:10,192 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan after decorrelation: HiveProject(_o__c0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveProject($f0=[$0]) HiveJoin(condition=[=($0, $6)], joinType=[full], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:10,192 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,193 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,193 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,193 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,193 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,193 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,193 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,193 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,193 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,194 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,194 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,203 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,203 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,203 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,203 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,204 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.sql2rel: Plan after trimming unused fields HiveAggregate(group=[{}], agg#0=[count()]) HiveProject(DUMMY=[0]) HiveJoin(condition=[=($0, $1)], joinType=[full], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:10,204 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,204 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,204 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,204 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,204 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,204 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,205 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,205 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,205 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,205 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,205 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,205 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,205 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,205 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,206 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,213 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Original plan for PlanModifier HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[full], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:10,214 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after top-level introduceDerivedTable HiveProject($f0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[full], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:10,214 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after nested convertOpTree HiveProject($f0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[full], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:10,214 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after propagating order HiveProject($f0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[full], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:10,214 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after fixTopOBSchema HiveProject($f0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[full], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:10,214 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Final plan after modifier HiveProject(_c0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($0, $1)], joinType=[full], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) 2018-01-05T11:29:10,214 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:10,215 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:10,215 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:10,215 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:10,215 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,215 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab 2018-01-05T11:29:10,215 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab 2018-01-05T11:29:10,218 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,219 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,219 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:10,219 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:10,219 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:10,219 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,219 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab_part 2018-01-05T11:29:10,219 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab_part 2018-01-05T11:29:10,222 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,222 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,222 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:10,222 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:10,222 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:10,222 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Table Plan for a TS[0] 2018-01-05T11:29:10,222 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB a{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB a{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:10,222 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (. (tok_table_or_col a) key) key)) 2018-01-05T11:29:10,222 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = a{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} starRr = null 2018-01-05T11:29:10,222 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(key,_col0: int)} 2018-01-05T11:29:10,222 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:10,222 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block $hdt$_0 2018-01-05T11:29:10,222 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block $hdt$_0 2018-01-05T11:29:10,223 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Table Plan for b TS[2] 2018-01-05T11:29:10,223 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB b{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB b{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:10,223 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (. (tok_table_or_col b) key) key)) 2018-01-05T11:29:10,223 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = b{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} starRr = null 2018-01-05T11:29:10,223 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(key,_col0: int)} 2018-01-05T11:29:10,223 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:10,223 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block $hdt$_1 2018-01-05T11:29:10,223 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block $hdt$_1 2018-01-05T11:29:10,223 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:10,223 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB $hdt$_0{(key,_col0: int)} $hdt$_1{(key,_col1: int)} after GB {((tok_functionstar count),_col0: bigint)} 2018-01-05T11:29:10,223 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (tok_functionstar count) _c0)) 2018-01-05T11:29:10,223 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = {((tok_functionstar count),_col0: bigint)} starRr = $hdt$_0{(key,_col0: int)} $hdt$_1{(key,_col1: int)} 2018-01-05T11:29:10,223 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(_c0,_col0: bigint)} 2018-01-05T11:29:10,223 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:10,223 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-10_156_6921175463455711345-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/-mr-10001 2018-01-05T11:29:10,223 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-10_156_6921175463455711345-1 2018-01-05T11:29:10,254 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/-mr-10003 2018-01-05T11:29:10,254 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/-mr-10001 row schema: null{(_c0,_col0: bigint)} 2018-01-05T11:29:10,254 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block null 2018-01-05T11:29:10,254 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:29:10,254 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: CBO Succeeded; optimized logical plan. 2018-01-05T11:29:10,254 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Before logical optimization TS[0]-SEL[1]-RS[4]-JOIN[6]-SEL[7]-GBY[8]-RS[9]-GBY[10]-SEL[11]-FS[12] TS[2]-SEL[3]-RS[5]-JOIN[6] 2018-01-05T11:29:10,254 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,254 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,254 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,255 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] lineage.Generator: Time taken for lineage transform=1 2018-01-05T11:29:10,255 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,255 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,255 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,255 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,255 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,255 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,255 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FS(12) 2018-01-05T11:29:10,255 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(11) 2018-01-05T11:29:10,255 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for GBY(10) 2018-01-05T11:29:10,255 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(9) 2018-01-05T11:29:10,255 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for GBY(8) 2018-01-05T11:29:10,255 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(7) 2018-01-05T11:29:10,255 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for JOIN(6) 2018-01-05T11:29:10,255 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(4) 2018-01-05T11:29:10,255 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(1) 2018-01-05T11:29:10,255 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:29:10,255 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(5) 2018-01-05T11:29:10,255 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(3) 2018-01-05T11:29:10,255 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for TS(2) 2018-01-05T11:29:10,255 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SimplePredicatePushDown: After PPD: TS[0]-SEL[1]-RS[4]-JOIN[6]-SEL[7]-GBY[8]-RS[9]-GBY[10]-SEL[11]-FS[12] TS[2]-SEL[3]-RS[5]-JOIN[6] 2018-01-05T11:29:10,255 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,255 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,255 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,255 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,255 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,255 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,255 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,255 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,256 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,256 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,256 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,256 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,256 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 9 key:[] 2018-01-05T11:29:10,256 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 9 oldColExprMap: {VALUE._col0=Column[_col0]} 2018-01-05T11:29:10,256 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 9 newColExprMap: {VALUE._col0=Column[_col0]} 2018-01-05T11:29:10,256 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: JOIN 6 oldExprs: {0=[Column[KEY.reducesinkkey0]], 1=[Column[KEY.reducesinkkey0]]} 2018-01-05T11:29:10,256 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: JOIN 6 newExprs: {0=[], 1=[]} 2018-01-05T11:29:10,256 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 4 key:[Column[_col0]] 2018-01-05T11:29:10,256 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 4 oldColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:10,256 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 4 newColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:10,256 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 5 key:[Column[_col0]] 2018-01-05T11:29:10,256 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 5 oldColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:10,256 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 5 newColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:10,256 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,256 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,256 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,256 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,256 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,256 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,256 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,256 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,256 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,256 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,256 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,256 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,257 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,257 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,257 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,257 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,257 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.IdentityProjectRemover: Identity project remover optimization removed : SEL[11] 2018-01-05T11:29:10,257 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.IdentityProjectRemover: Identity project remover optimization removed : SEL[7] 2018-01-05T11:29:10,257 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,257 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,257 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,257 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,257 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,257 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,257 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,257 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: After logical optimization TS[0]-SEL[1]-RS[4]-JOIN[6]-GBY[8]-RS[9]-GBY[10]-FS[12] TS[2]-SEL[3]-RS[5]-JOIN[6] 2018-01-05T11:29:10,257 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,257 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,257 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,257 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,257 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,257 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partitions : db=default tbl=tab 2018-01-05T11:29:10,257 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions : db=default tbl=tab 2018-01-05T11:29:10,261 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,262 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.194555ms + 0.005511ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? ] 2018-01-05T11:29:10,262 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.149386ms + 0.050649ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (15) order by "PART_NAME" asc] 2018-01-05T11:29:10,262 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.086667ms + 0.059429ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (15) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:29:10,263 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.08397ms + 0.02166ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (15) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:10,263 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.063422ms + 0.007015ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (43) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:29:10,263 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.060198ms + 0.020894ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (43) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:10,263 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.056465ms + 0.015879ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (43) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:10,263 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.051249ms + 0.005777ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (43) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:10,263 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.083881ms + 0.021678ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (28) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:10,263 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.065601ms + 0.018961ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (43) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:29:10,263 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,266 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,267 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,267 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,267 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Estimated average row size: 4 2018-01-05T11:29:10,267 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,267 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_aggr_stats_for: db=default table=tab 2018-01-05T11:29:10,267 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_aggr_stats_for: db=default table=tab 2018-01-05T11:29:10,269 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.AggregateStatsCache: Returning aggregate stats from the cache; total hits: 17, total misses: 0, hit ratio: 1.0 2018-01-05T11:29:10,269 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: useDensityFunctionForNDVEstimation = false partsFound = 1 ColumnStatisticsObj = [ColumnStatisticsObj(colName:key, colType:int, statsData:)] 2018-01-05T11:29:10,270 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,270 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,270 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting stats (Num rows: 242 Data size: 968 Basic stats: COMPLETE Column stats: COMPLETE) on TS[0] 2018-01-05T11:29:10,270 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (tab): numRows: 242 dataSize: 968 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {key= colName: key colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:10,270 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,270 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,270 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partitions : db=default tbl=tab_part 2018-01-05T11:29:10,271 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions : db=default tbl=tab_part 2018-01-05T11:29:10,273 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,274 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.266471ms + 0.002991ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? ] 2018-01-05T11:29:10,274 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.187596ms + 0.060923ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (14) order by "PART_NAME" asc] 2018-01-05T11:29:10,274 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.084821ms + 0.057789ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (14) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:29:10,274 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.067758ms + 0.018548ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (14) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:10,274 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.059531ms + 0.00605ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (41) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:29:10,275 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.057642ms + 0.019581ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (41) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:10,275 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.056166ms + 0.015693ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (41) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:10,275 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.054971ms + 0.005601ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (41) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:10,275 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.087841ms + 0.019866ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (26) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:10,275 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.092726ms + 0.023044ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (41) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:29:10,275 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,278 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,278 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,279 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,279 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Estimated average row size: 4 2018-01-05T11:29:10,279 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,279 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_aggr_stats_for: db=default table=tab_part 2018-01-05T11:29:10,279 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_aggr_stats_for: db=default table=tab_part 2018-01-05T11:29:10,280 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.AggregateStatsCache: Returning aggregate stats from the cache; total hits: 18, total misses: 0, hit ratio: 1.0 2018-01-05T11:29:10,280 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: useDensityFunctionForNDVEstimation = false partsFound = 1 ColumnStatisticsObj = [ColumnStatisticsObj(colName:key, colType:int, statsData:)] 2018-01-05T11:29:10,281 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on TS[2] 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-TS[2] (tab_part): numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {key= colName: key colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 242 Data size: 968 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[1] 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[1]: numRows: 242 dataSize: 968 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[3] 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[3]: numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 242 Data size: 968 Basic stats: COMPLETE Column stats: COMPLETE) on RS[4] 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[4]: numRows: 242 dataSize: 968 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {KEY.reducesinkkey0= colName: KEY.reducesinkkey0 colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on RS[5] 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[5]: numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {KEY.reducesinkkey0= colName: KEY.reducesinkkey0 colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-JOIN[6]: detects none/multiple PK parents. 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.JoinOperator: Setting stats (Num rows: 742 Data size: 5936 Basic stats: COMPLETE Column stats: COMPLETE) on JOIN[6] 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-JOIN[6]: numRows: 742 dataSize: 5936 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[8]: inputSize: 5936 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[8] hashAgg: true 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 3] STATS-GBY[8]: cardinality: 1 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[8] 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[8]: numRows: 1 dataSize: 8 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE) on RS[9] 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[9]: numRows: 1 dataSize: 8 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {VALUE._col0= colName: VALUE._col0 colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[10]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[10] hashAgg: false 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 9] STATS-GBY[10]: cardinality: 1 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[10] 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[10]: numRows: 1 dataSize: 8 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-FS[12]: numRows: 1 dataSize: 8 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[key]]; bucket count: 2 }) on TS[0] 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[key]]; bucket count: 4 }) on TS[2] 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: 2 }) on SEL[1] 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: 4 }) on SEL[3] 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[KEY.reducesinkkey0]]; sort column names: [[KEY.reducesinkkey0]]; bucket count: 2 }) on RS[4] 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[KEY.reducesinkkey0]]; sort column names: [[KEY.reducesinkkey0]]; bucket count: 4 }) on RS[5] 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.JoinOperator: Setting traits ({ bucket column names: [[], []]; sort column names: [[], []]; bucket count: -1 }) on JOIN[6] 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[8] 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[9] 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[10] 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FileSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on FS[12] 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,282 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,282 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Set parallelism for reduce sink RS[4] to: 1 2018-01-05T11:29:10,283 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Set parallelism for reduce sink RS[5] to: 1 2018-01-05T11:29:10,283 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConvertJoinMapJoin: Memory monitor info set to : { isLlap: true executorsPerNode: 4 maxExecutorsOverSubscribeMemory: 3 memoryOverSubscriptionFactor: 0.20000000298023224 memoryCheckInterval: 100000 noConditionalTaskSize: 10000000 adjustedNoConditionalTaskSize: 16000000 hashTableInflationFactor: 2.0 threshold: 32000000 } 2018-01-05T11:29:10,283 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConvertJoinMapJoin: Could not get a valid join position. Defaulting to position 0 2018-01-05T11:29:10,283 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConvertJoinMapJoin: Fallback to common merge join operator 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.CommonMergeJoinOperator: Setting traits ({ bucket column names: [[], []]; sort column names: [[], []]; bucket count: 0 }) on MERGEJOIN[13] 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.CommonMergeJoinOperator: Setting stats (Num rows: 742 Data size: 5936 Basic stats: COMPLETE Column stats: COMPLETE) on MERGEJOIN[13] 2018-01-05T11:29:10,283 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: FS, 12 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: GBY, 10 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 9 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: GBY, 8 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: MERGEJOIN, 13 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 4 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 1 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 5 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 3 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: TS, 2 2018-01-05T11:29:10,283 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Cycle free: true 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: Before SharedWorkOptimizer: TS[0]-SEL[1]-RS[4]-MERGEJOIN[13]-GBY[8]-RS[9]-GBY[10]-FS[12] TS[2]-SEL[3]-RS[5]-MERGEJOIN[13] 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: DPP information stored in the cache: {} 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: Sorted tables by size: [default.tab_part=2000, default.tab=968] 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: After SharedWorkOptimizer: TS[0]-SEL[1]-RS[4]-MERGEJOIN[13]-GBY[8]-RS[9]-GBY[10]-FS[12] TS[2]-SEL[3]-RS[5]-MERGEJOIN[13] 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: int|{a}key,value: string|{a}value,ds: string|{a}ds,BLOCK__OFFSET__INSIDE__FILE: bigint|{a}block__offset__inside__file,INPUT__FILE__NAME: string|{a}input__file__name,ROW__ID: struct|{a}row__id) 2018-01-05T11:29:10,283 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[2] with rs:(key: int|{b}key,value: string|{b}value,ds: string|{b}ds,BLOCK__OFFSET__INSIDE__FILE: bigint|{b}block__offset__inside__file,INPUT__FILE__NAME: string|{b}input__file__name,ROW__ID: struct|{b}row__id) 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[1] with rs:(_col0: int|{$hdt$_0}key) 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[1] 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key]) 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[3] with rs:(_col0: int|{$hdt$_1}key) 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_2 {} 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[3] 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key]) 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[4] with rs:(KEY.reducesinkkey0: int|{$hdt$_0}key) 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_1 {} 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[4] 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[5] with rs:(KEY.reducesinkkey0: int|{$hdt$_1}key) 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_3 {} 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[5] 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:MERGEJOIN[13] with rs:() 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_4 {} 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_5 {} 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator MERGEJOIN[13] 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[8] with rs:(_col0: bigint|{}(tok_functionstar count)) 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op MERGEJOIN_13 {} 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[8] 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[9] with rs:(VALUE._col0: bigint|{}(tok_functionstar count)) 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_8 {} 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[9] 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[10] with rs:(_col0: bigint|{}(tok_functionstar count)) 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_9 {} 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[10] 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[12] with rs:(_col0: bigint|{}) 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_10 {} 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[12] 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: TS[0] 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[4] 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 of table a 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[4] 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: MERGEJOIN[13] 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[9] 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding reduce work (Reducer 2) for MERGEJOIN[13] 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Setting up reduce sink: RS[4] with following reduce work: Reducer 2 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[4] as parent from MERGEJOIN[13] 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[5] as parent from MERGEJOIN[13] 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[9] 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: GBY[10] 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: FS[12] 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding reduce work (Reducer 3) for GBY[10] 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Setting up reduce sink: RS[9] with following reduce work: Reducer 3 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[9] as parent from GBY[10] 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: FS[12] 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: TS[2] 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[5] 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding map work (Map 4) for TS[2] 2018-01-05T11:29:10,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 of table b 2018-01-05T11:29:10,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:10,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Second pass. Leaf operator: RS[5] has common downstream work: org.apache.hadoop.hive.ql.plan.MergeJoinWork@ce0c2b3 2018-01-05T11:29:10,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:29:10,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:29:10,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:10,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:10,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 4 2018-01-05T11:29:10,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:10,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:10,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 1 2018-01-05T11:29:10,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:10,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:10,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 4 2018-01-05T11:29:10,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:10,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:10,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:29:10,285 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Examining input format to see if vectorization is enabled. 2018-01-05T11:29:10,285 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorization is enabled for input format(s) [org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] 2018-01-05T11:29:10,285 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing MapWork... 2018-01-05T11:29:10,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[key], Vectorized Expression = col 0:int 2018-01-05T11:29:10,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator 2018-01-05T11:29:10,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.SelectDesc 2018-01-05T11:29:10,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:int 2018-01-05T11:29:10,285 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator groupby typeName int 2018-01-05T11:29:10,285 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkLongOperator 2018-01-05T11:29:10,285 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:10,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkLongOperator 2018-01-05T11:29:10,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:10,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [key, value, ds, ROW__ID] 2018-01-05T11:29:10,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [int, string, string, struct] 2018-01-05T11:29:10,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE, NONE, NONE, NONE] 2018-01-05T11:29:10,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 1 2018-01-05T11:29:10,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 1 2018-01-05T11:29:10,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:10,285 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:10,285 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Examining input format to see if vectorization is enabled. 2018-01-05T11:29:10,286 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorization is enabled for input format(s) [org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] 2018-01-05T11:29:10,286 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing MapWork... 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[key], Vectorized Expression = col 0:int 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.SelectDesc 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:int 2018-01-05T11:29:10,286 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator groupby typeName int 2018-01-05T11:29:10,286 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkLongOperator 2018-01-05T11:29:10,286 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkLongOperator 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [key, value, ds, ROW__ID] 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [int, string, string, struct] 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE, NONE, NONE, NONE] 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 1 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 1 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Using reduce tag 0 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0] columnTypes=[bigint] 2018-01-05T11:29:10,286 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing ReduceWork... 2018-01-05T11:29:10,286 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vector GROUP BY operator will use processing mode GLOBAL 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[VALUE._col0], Vectorized Expression = col 0:bigint 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.GroupByDesc 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorFileSinkOperator 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.FileSinkDesc 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [VALUE._col0] 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [bigint] 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE] 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 0 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 0 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:29:10,286 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: llap mode: only 2018-01-05T11:29:10,286 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key' 2018-01-05T11:29:10,286 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@4a527b87 can be done in LLAP 2018-01-05T11:29:10,286 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Map 4 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key' 2018-01-05T11:29:10,286 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@6e75747e can be done in LLAP 2018-01-05T11:29:10,286 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Reducer 2 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'count()' 2018-01-05T11:29:10,286 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MergeJoinWork@ce0c2b3 can be done in LLAP 2018-01-05T11:29:10,286 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Reducer 3 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'count(VALUE._col0)' 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:29:10,286 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@3722bf5e can be done in LLAP 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,286 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed plan generation 2018-01-05T11:29:10,286 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: validation start 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:29:10,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,286 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0, type:bigint, comment:null)], properties:null) 2018-01-05T11:29:10,287 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: Initializing operator LIST_SINK[20] 2018-01-05T11:29:10,287 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: Initialization Done 20 LIST_SINK 2018-01-05T11:29:10,287 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: Operator 20 LIST_SINK initialized 2018-01-05T11:29:10,287 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: Initialization Done 20 LIST_SINK done is reset. 2018-01-05T11:29:10,287 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,287 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:10,287 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {getNotNullConstraints_(NotNullConstraintsRequest, )=0, isCompatibleWith_(Configuration, )=0, getAggrColStatsFor_(String, String, List, List, )=6, getTable_(String, String, )=13, flushCache_()=0, getUniqueConstraints_(UniqueConstraintsRequest, )=0, getPrimaryKeys_(PrimaryKeysRequest, )=2, listPartitions_(String, String, short, )=19, getForeignKeys_(ForeignKeysRequest, )=1} 2018-01-05T11:29:10,287 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca); Time taken: 0.131 seconds 2018-01-05T11:29:10,287 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:10,287 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,287 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca): select count(*) from tab a full outer join tab_part b on a.key = b.key 2018-01-05T11:29:10,288 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,288 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: select count(*) from tab a full outer join tab_part b on a.key = b.key 2018-01-05T11:29:10,288 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:29:10,288 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@tab 2018-01-05T11:29:10,288 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@tab@ds=2008-04-08 2018-01-05T11:29:10,288 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@tab_part 2018-01-05T11:29:10,288 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@tab_part@ds=2008-04-08 2018-01-05T11:29:10,288 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/-mr-10001 2018-01-05T11:29:10,288 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,288 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,288 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,288 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Query ID = djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca 2018-01-05T11:29:10,288 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Total jobs = 1 2018-01-05T11:29:10,288 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,288 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:29:10,288 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-1:MAPRED] in serial mode 2018-01-05T11:29:10,298 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:29:10,320 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:29:10,320 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:29:10,320 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:29:10,320 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca 2018-01-05T11:29:10,320 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Session is already open 2018-01-05T11:29:10,320 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:29:10,320 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,320 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Dag name: select count(*) from tab a full oute...b.key (Stage-1) 2018-01-05T11:29:10,320 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: DagInfo: {"context":"Hive","description":"\n\nselect count(*)\nfrom tab a full outer join tab_part b on a.key = b.key"} 2018-01-05T11:29:10,320 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:29:10,320 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,331 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,331 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:29:10,332 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,332 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 3 size: 1.90KB 2018-01-05T11:29:10,348 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,348 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,360 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,360 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:29:10,361 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,361 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 2 size: 1.49KB 2018-01-05T11:29:10,366 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,368 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,371 DEBUG [AMReporterQueueDrainer] impl.AMReporter: Removing am 10.22.8.182:49824 with last associated dag QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=28} from heartbeat with taskCount=0, amFailed=false, isDone=true 2018-01-05T11:29:10,380 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:29:10,380 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,380 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:29:10,381 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,381 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: Serialized plan (via RPC) - name: Map 4 size: 3.07KB 2018-01-05T11:29:10,386 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:10,386 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,388 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,398 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:29:10,399 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,399 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:29:10,399 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,399 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 3.05KB 2018-01-05T11:29:10,405 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:10,405 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,406 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,406 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,406 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=select count(*) from tab a full oute...b.key (Stage-1), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca } 2018-01-05T11:29:10,407 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:10,407 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:10,407 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:10,407 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:10,409 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: select count(*) from tab a full oute...b.key (Stage-1) 2018-01-05T11:29:10,409 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:29:10,409 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_41, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_41.dot 2018-01-05T11:29:10,410 INFO [IPC Server handler 1 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_41 2018-01-05T11:29:10,410 WARN [IPC Server handler 1 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:29:10,410 INFO [IPC Server handler 1 on 49823] app.DAGAppMaster: Running DAG: select count(*) from tab a full oute...b.key (Stage-1), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca } 2018-01-05T11:29:10,431 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_41, submitTime=1515180550409, queueName=null 2018-01-05T11:29:10,432 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_41_00 [Map 4]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_41_00 [Map 4], Default Resources= 2018-01-05T11:29:10,432 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_41_00 [Map 4] 2018-01-05T11:29:10,432 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 4 2018-01-05T11:29:10,432 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_41_00 [Map 4] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:10,432 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_41_01 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_41_01 [Map 1], Default Resources= 2018-01-05T11:29:10,432 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_41_01 [Map 1] 2018-01-05T11:29:10,432 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:29:10,432 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_41_01 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:10,433 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_41_02 [Reducer 2]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_41_02 [Reducer 2], Default Resources= 2018-01-05T11:29:10,433 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 2 2018-01-05T11:29:10,433 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_41_02 [Reducer 2] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:10,433 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_41_03 [Reducer 3]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_41_03 [Reducer 3], Default Resources= 2018-01-05T11:29:10,433 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_41_03 [Reducer 3] 2018-01-05T11:29:10,433 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 3 2018-01-05T11:29:10,433 INFO [IPC Server handler 1 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_41_03 [Reducer 3] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:10,433 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:29:10,433 INFO [IPC Server handler 1 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_41, initTime=1515180550433 2018-01-05T11:29:10,433 INFO [IPC Server handler 1 on 49823] impl.DAGImpl: dag_1515180518813_0001_41 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:29:10,433 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:29:10,433 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_41, startTime=1515180550433 2018-01-05T11:29:10,433 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_41 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:29:10,433 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {a={InputName=a}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:29:10,433 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: a, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:29:10,433 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_41, dagName=select count(*) from tab a full oute...b.key (Stage-1) 2018-01-05T11:29:10,433 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_41_01 [Map 1] 2018-01-05T11:29:10,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,437 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_41_01 [Map 1] 2018-01-05T11:29:10,437 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_41_01 [Map 1] 2018-01-05T11:29:10,437 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:10,437 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_41_01 [Map 1] 2018-01-05T11:29:10,439 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:29:10,439 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@1d4e9598 2018-01-05T11:29:10,439 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/5b41658d-5fab-4895-ab8a-ffc4518906a8/map.xml 2018-01-05T11:29:10,439 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/5b41658d-5fab-4895-ab8a-ffc4518906a8/map.xml 2018-01-05T11:29:10,439 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/5b41658d-5fab-4895-ab8a-ffc4518906a8/map.xml 2018-01-05T11:29:10,439 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:10,439 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:10,439 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:10,440 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:10,440 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.05KB 2018-01-05T11:29:10,440 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_41_01 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:29:10,440 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 4 : {b={InputName=b}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:29:10,440 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: b, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:29:10,440 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_41_00 [Map 4] 2018-01-05T11:29:10,440 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: a on vertex vertex_1515180518813_0001_41_01 [Map 1] 2018-01-05T11:29:10,440 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:29:10,440 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:10,440 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:29:10,440 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:10,440 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/5b41658d-5fab-4895-ab8a-ffc4518906a8/map.xml 2018-01-05T11:29:10,440 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:10,440 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias a 2018-01-05T11:29:10,440 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:10,440 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:10,441 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 0 2018-01-05T11:29:10,441 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = key 2018-01-05T11:29:10,441 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:10,441 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:10,441 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:10,441 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:10,441 DEBUG [InputInitializer {Map 1} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:10,441 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: getSplits started 2018-01-05T11:29:10,441 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: Number of buckets specified by conf file is 2 2018-01-05T11:29:10,441 INFO [InputInitializer {Map 1} #0] orc.OrcInputFormat: ORC pushdown predicate: null 2018-01-05T11:29:10,441 INFO [InputInitializer {Map 1} #0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:10,441 ERROR [ORC_GET_SPLITS #7] io.AcidUtils: Failed to get files with ID; using regular API: Only supported for DFS; got class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:10,441 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: Generate splits schema evolution property true reader schema struct transactional scan property false 2018-01-05T11:29:10,444 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_41_00 [Map 4] 2018-01-05T11:29:10,444 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_41_00 [Map 4] 2018-01-05T11:29:10,444 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:10,444 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_41_00 [Map 4] 2018-01-05T11:29:10,446 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:29:10,447 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@3f1186b6 2018-01-05T11:29:10,447 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/d20ff970-7e5b-402f-9f4b-cba1fc4a55de/map.xml 2018-01-05T11:29:10,447 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/d20ff970-7e5b-402f-9f4b-cba1fc4a55de/map.xml 2018-01-05T11:29:10,447 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/d20ff970-7e5b-402f-9f4b-cba1fc4a55de/map.xml 2018-01-05T11:29:10,447 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:10,447 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:10,447 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:10,447 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:10,447 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 4 size: 3.07KB 2018-01-05T11:29:10,448 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_41_00 [Map 4] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:29:10,449 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to ShuffleVertexManager for vertex_1515180518813_0001_41_02 [Reducer 2] 2018-01-05T11:29:10,449 INFO [InputInitializer {Map 4} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: b on vertex vertex_1515180518813_0001_41_00 [Map 4] 2018-01-05T11:29:10,449 INFO [InputInitializer {Map 4} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:29:10,449 DEBUG [InputInitializer {Map 4} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:10,449 INFO [InputInitializer {Map 4} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:29:10,449 DEBUG [InputInitializer {Map 4} #0] log.PerfLogger: 2018-01-05T11:29:10,450 INFO [InputInitializer {Map 4} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/d20ff970-7e5b-402f-9f4b-cba1fc4a55de/map.xml 2018-01-05T11:29:10,450 DEBUG [InputInitializer {Map 4} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:10,450 INFO [InputInitializer {Map 4} #0] exec.Utilities: Processing alias b 2018-01-05T11:29:10,450 DEBUG [InputInitializer {Map 4} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:10,450 INFO [InputInitializer {Map 4} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:10,450 INFO [InputInitializer {Map 4} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 0 2018-01-05T11:29:10,450 INFO [InputInitializer {Map 4} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = key 2018-01-05T11:29:10,450 INFO [InputInitializer {Map 4} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:10,450 DEBUG [InputInitializer {Map 4} #0] orc.OrcInputFormat: getSplits started 2018-01-05T11:29:10,450 DEBUG [InputInitializer {Map 4} #0] orc.OrcInputFormat: Number of buckets specified by conf file is 4 2018-01-05T11:29:10,450 INFO [InputInitializer {Map 4} #0] orc.OrcInputFormat: ORC pushdown predicate: null 2018-01-05T11:29:10,450 INFO [InputInitializer {Map 4} #0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:10,450 DEBUG [InputInitializer {Map 4} #0] orc.OrcInputFormat: Generate splits schema evolution property true reader schema struct transactional scan property false 2018-01-05T11:29:10,450 ERROR [ORC_GET_SPLITS #6] io.AcidUtils: Failed to get files with ID; using regular API: Only supported for DFS; got class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:10,454 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.05 maxFrac: 0.75 auto: false desiredTaskIput: 104857600 2018-01-05T11:29:10,454 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:29:10,454 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 1 tasks for vertex: vertex_1515180518813_0001_41_02 [Reducer 2] 2018-01-05T11:29:10,454 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_41_02 [Reducer 2] 2018-01-05T11:29:10,454 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_41_02, reconfigureDoneTime=1515180550454, numTasks=1, vertexLocationHint=null, edgeManagersCount=2, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:29:10,454 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:VERTEX_INITIALIZED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_41_02, initRequestedTime=1515180550448, initedTime=1515180550454, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:10,454 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_41_02 [Reducer 2] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:29:10,454 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to ShuffleVertexManager for vertex_1515180518813_0001_41_03 [Reducer 3] 2018-01-05T11:29:10,459 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.05 maxFrac: 0.75 auto: false desiredTaskIput: 104857600 2018-01-05T11:29:10,459 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:29:10,459 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 1 tasks for vertex: vertex_1515180518813_0001_41_03 [Reducer 3] 2018-01-05T11:29:10,459 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_41_03 [Reducer 3] 2018-01-05T11:29:10,459 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_41_03, reconfigureDoneTime=1515180550459, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:29:10,459 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_41_03 [Reducer 3], numAdditionalOutputs=1 2018-01-05T11:29:10,459 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:VERTEX_INITIALIZED]: vertexName=Reducer 3, vertexId=vertex_1515180518813_0001_41_03, initRequestedTime=1515180550454, initedTime=1515180550459, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:10,459 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_41_03 [Reducer 3] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:29:10,470 DEBUG [ORC_GET_SPLITS #7] io.AcidUtils: in directory file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 base = null deltas = 0 2018-01-05T11:29:10,470 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: Split strategy: ETLSplitStrategy strategy for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:10,470 DEBUG [ORC_GET_SPLITS #8] orc.LocalCache: Serialized tail cached for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0 2018-01-05T11:29:10,470 DEBUG [ORC_GET_SPLITS #8] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:10,470 DEBUG [ORC_GET_SPLITS #8] orc.LocalCache: Serialized tail cached for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0 2018-01-05T11:29:10,470 DEBUG [ORC_GET_SPLITS #8] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:10,470 DEBUG [ORC_GET_SPLITS #8] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:10,470 DEBUG [ORC_GET_SPLITS #9] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:10,470 DEBUG [ORC_GET_SPLITS #9] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:29:10,470 DEBUG [ORC_GET_SPLITS #8] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:29:10,471 INFO [InputInitializer {Map 1} #0] orc.OrcInputFormat: FooterCacheHitRatio: 2/2 2018-01-05T11:29:10,471 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0, start=3, length=566, isOriginal=true, fileLength=744, hasFooter=false, hasBase=true, deltas=0] projected_columns_uncompressed_size: 371 2018-01-05T11:29:10,471 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0, start=3, length=583, isOriginal=true, fileLength=767, hasFooter=false, hasBase=true, deltas=0] projected_columns_uncompressed_size: 364 2018-01-05T11:29:10,471 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: getSplits finished 2018-01-05T11:29:10,471 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 2 2018-01-05T11:29:10,471 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:10,471 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 2. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:29:10,471 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/5b41658d-5fab-4895-ab8a-ffc4518906a8/map.xml 2018-01-05T11:29:10,471 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/5b41658d-5fab-4895-ab8a-ffc4518906a8/map.xml 2018-01-05T11:29:10,471 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/5b41658d-5fab-4895-ab8a-ffc4518906a8/map.xml 2018-01-05T11:29:10,471 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:10,471 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:10,471 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:10,471 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:10,472 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.05KB 2018-01-05T11:29:10,472 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0 to src new group? true 2018-01-05T11:29:10,472 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0 to src new group? false 2018-01-05T11:29:10,472 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:29:10,472 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:29:10,472 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:29:10,472 DEBUG [InputInitializer {Map 1} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 364 2018-01-05T11:29:10,472 DEBUG [InputInitializer {Map 1} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 371 2018-01-05T11:29:10,472 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 13 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 735 Original splits: 2 2018-01-05T11:29:10,472 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired numSplits: 1 lengthPerGroup: 735 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 735 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false 2018-01-05T11:29:10,472 DEBUG [InputInitializer {Map 1} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 364 2018-01-05T11:29:10,472 DEBUG [InputInitializer {Map 1} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 371 2018-01-05T11:29:10,472 DEBUG [InputInitializer {Map 1} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 371 2018-01-05T11:29:10,472 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Number of splits desired: 1 created: 1 splitsProcessed: 2 2018-01-05T11:29:10,472 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 2 grouped split count is 1, for bucket: 1 2018-01-05T11:29:10,472 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:29:10,472 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: a on vertex vertex_1515180518813_0001_41_01 [Map 1] 2018-01-05T11:29:10,472 INFO [App Shared Pool - #2] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_41_01 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:10,472 INFO [App Shared Pool - #2] impl.VertexImpl: Got updated RootInputsSpecs: {a=forAllWorkUnits=true, update=[1]} 2018-01-05T11:29:10,472 INFO [App Shared Pool - #2] impl.VertexImpl: Vertex vertex_1515180518813_0001_41_01 [Map 1] parallelism set to 1 2018-01-05T11:29:10,472 INFO [App Shared Pool - #2] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_41_01 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:10,473 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_41_01, initRequestedTime=1515180550433, initedTime=1515180550472, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:10,473 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_41_01 [Map 1] with distanceFromRoot: 0 2018-01-05T11:29:10,473 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_41_01 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:29:10,473 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_41_01, startRequestedTime=1515180550448, startedTime=1515180550473 2018-01-05T11:29:10,473 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_41_01, reconfigureDoneTime=1515180550473, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:29:10,473 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_41_01 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:10,473 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_41_01 for vertex: vertex_1515180518813_0001_41_02 [Reducer 2] numStartedSources: 1 numSources: 2 2018-01-05T11:29:10,473 INFO [Dispatcher thread {Central}] impl.VertexImpl: Cannot start vertex: vertex_1515180518813_0001_41_02 [Reducer 2] numStartedSources: 1 numSources: 2 2018-01-05T11:29:10,473 INFO [App Shared Pool - #2] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 1 2018-01-05T11:29:10,473 INFO [App Shared Pool - #2] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:29:10,473 INFO [App Shared Pool - #2] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_41_01 [Map 1] 2018-01-05T11:29:10,473 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_41_01_000000, scheduledTime=1515180550473, launchTime=1515180550473 2018-01-05T11:29:10,473 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_41_01 [Map 1] 2018-01-05T11:29:10,473 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_41_01_000000_0, priority=14, capability=, hosts=null 2018-01-05T11:29:10,473 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:10,473 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=14,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:10,473 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:10,473 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_41_01_000000_0 on hosts=null 2018-01-05T11:29:10,473 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_41_01_000000_0 on hosts=null 2018-01-05T11:29:10,473 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_41_01_000000_0 2018-01-05T11:29:10,473 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:10,473 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #101, task=TaskInfo{task=attempt_1515180518813_0001_41_01_000000_0, priority=14, startTime=0, containerId=null, uniqueId=100, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000101 2018-01-05T11:29:10,473 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_41_01_000000_0; false 2018-01-05T11:29:10,473 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_41_01_000000_0, priority=14, startTime=311570983, containerId=container_111101111_0001_00_000101assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=100, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:10,473 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000101 for node: 10.22.8.182:49818 2018-01-05T11:29:10,475 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 41, name=select count(*) from tab a full oute...b.key (Stage-1), queryId=djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca 2018-01-05T11:29:10,475 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000101, attempt_1515180518813_0001_41_01_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:10,475 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000101 for node: 10.22.8.182:49818 2018-01-05T11:29:10,476 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000101 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:10,476 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000101, launchTime=1515180550476 2018-01-05T11:29:10,476 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_41_01_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000101, dagName=select count(*) from tab a full oute...b.key (Stage-1), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{a,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{Reducer 2,org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput,1}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180550433, firstAttemptStartTime=1515180550473, currentAttemptStartTime=1515180550476} 2018-01-05T11:29:10,477 INFO [IPC Server handler 0 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000101, QueryId=djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca-41, DagId=41, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180550477 2018-01-05T11:29:10,477 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_41_01_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000101] on NM: [10.22.8.182:49818] 2018-01-05T11:29:10,477 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:10,477 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:10,477 DEBUG [IPC Server handler 0 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=41} with the ShuffleHandler 2018-01-05T11:29:10,477 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_41_01_000000_0, startTime=1515180550476, containerId=container_111101111_0001_00_000101, nodeId=10.22.8.182:49818 2018-01-05T11:29:10,488 INFO [IPC Server handler 0 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:29:10,488 DEBUG [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/41] 2018-01-05T11:29:10,488 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:29:10,489 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_41_01_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:10,489 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_41_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180550473, dagStartTime=1515180550433, withinDagPriority=14, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:29:10,489 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_41_01_000000_0 : ACCEPTED 2018-01-05T11:29:10,489 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_41_01_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:29:10,489 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_41_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180550473, dagStartTime=1515180550433, withinDagPriority=14, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:29:10,489 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_41_01_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:10,489 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_41_01_000000_0 to pre-emption queue 2018-01-05T11:29:10,489 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_41_01_000000_0: true 2018-01-05T11:29:10,489 INFO [TaskCommunicator # 7] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_41_01_000000_0 2018-01-05T11:29:10,489 INFO [TaskCommunicator # 7] GuaranteedTasks: Received response for attempt_1515180518813_0001_41_01_000000_0, true 2018-01-05T11:29:10,489 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: taskOwner hashCode:981898900 2018-01-05T11:29:10,489 INFO [Task-Executor-2] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : select count(*) from tab a full oute...b.key (Stage-1), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_41_01_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=a, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=Reducer 2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ] 2018-01-05T11:29:10,489 INFO [Task-Executor-2] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1023410176, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:10,489 INFO [Task-Executor-2] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_41_01_000000_0 vertexName: Map 1 2018-01-05T11:29:10,489 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000101, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_41_01_000000_0, eventCount=1 } 2018-01-05T11:29:10,490 DEBUG [TezTR-518813_1_41_1_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_41 queryId: djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca instanceType: TaskRunner2Callable 2018-01-05T11:29:10,490 INFO [TezTR-518813_1_41_1_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_41_01_000000_0 2018-01-05T11:29:10,490 DEBUG [TezTR-518813_1_41_1_0_0] log.PerfLogger: 2018-01-05T11:29:10,491 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:10,492 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:10,492 INFO [IPC Server handler 18 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_41_01_000000_0 sent events: (0-1). 2018-01-05T11:29:10,493 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:29:10,493 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_41_01_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:29:10,493 DEBUG [TezTR-518813_1_41_1_0_0] log.PerfLogger: 2018-01-05T11:29:10,493 INFO [TezTR-518813_1_41_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:10,493 INFO [TezTR-518813_1_41_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:10,493 INFO [I/O Setup 0 Initialize: {a}] input.MRInput: a using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:29:10,493 INFO [I/O Setup 0 Initialize: {a}] input.MRInputLegacy: a MRInputLegacy deferring initialization 2018-01-05T11:29:10,493 INFO [TezTR-518813_1_41_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:10,493 INFO [TezTR-518813_1_41_1_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:10,493 INFO [TezTR-518813_1_41_1_0_0] resources.MemoryDistributor: InitialRequests=[a:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [Reducer 2:OUTPUT:10485760:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput] 2018-01-05T11:29:10,493 INFO [TezTR-518813_1_41_1_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:10,493 INFO [TezTR-518813_1_41_1_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:10,493 INFO [TezTR-518813_1_41_1_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 10485760, TotalRequestedScaled: 9679163.076923078, TotalJVMHeap: 1023410176, TotalAvailable: 685684817, TotalRequested/TotalJVMHeap:0.01 2018-01-05T11:29:10,493 INFO [TezTR-518813_1_41_1_0_0] resources.MemoryDistributor: Allocations=[a:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:10485760:10485760] 2018-01-05T11:29:10,493 INFO [TezTR-518813_1_41_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:10,493 INFO [TezTR-518813_1_41_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: a being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:10,493 INFO [I/O Setup 1 Start: {a}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: a 2018-01-05T11:29:10,494 INFO [TezTR-518813_1_41_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:29:10,494 INFO [TezTR-518813_1_41_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:10,494 INFO [TezTR-518813_1_41_1_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:10,494 INFO [TezTR-518813_1_41_1_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_41_01_000000_0 2018-01-05T11:29:10,494 DEBUG [TezTR-518813_1_41_1_0_0] log.PerfLogger: 2018-01-05T11:29:10,494 DEBUG [TezTR-518813_1_41_1_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_41_01_000000_0_10308 2018-01-05T11:29:10,494 DEBUG [TezTR-518813_1_41_1_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:10,494 DEBUG [TezTR-518813_1_41_1_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_41_01_000000_0 2018-01-05T11:29:10,494 INFO [TezTR-518813_1_41_1_0_0] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca 2018-01-05T11:29:10,494 DEBUG [TezTR-518813_1_41_1_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:10,494 DEBUG [TezTR-518813_1_41_1_0_0] log.PerfLogger: 2018-01-05T11:29:10,494 DEBUG [TezTR-518813_1_41_1_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:10,494 DEBUG [TezTR-518813_1_41_1_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:10,494 DEBUG [TezTR-518813_1_41_1_0_0] mr.ObjectCache: Creating djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca_Map 1__MAP_PLAN__ 2018-01-05T11:29:10,494 INFO [TezTR-518813_1_41_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/5b41658d-5fab-4895-ab8a-ffc4518906a8/map.xml 2018-01-05T11:29:10,494 DEBUG [TezTR-518813_1_41_1_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/5b41658d-5fab-4895-ab8a-ffc4518906a8/map.xml 2018-01-05T11:29:10,494 DEBUG [TezTR-518813_1_41_1_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/5b41658d-5fab-4895-ab8a-ffc4518906a8/map.xml 2018-01-05T11:29:10,494 DEBUG [TezTR-518813_1_41_1_0_0] log.PerfLogger: 2018-01-05T11:29:10,494 INFO [TezTR-518813_1_41_1_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:10,494 DEBUG [TezTR-518813_1_41_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:10,495 DEBUG [TezTR-518813_1_41_1_0_0] log.PerfLogger: 2018-01-05T11:29:10,495 INFO [TezTR-518813_1_41_1_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 3.05KB 2018-01-05T11:29:10,495 DEBUG [TezTR-518813_1_41_1_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:10,495 INFO [TezTR-518813_1_41_1_0_0] tez.MapRecordProcessor: The input names are: [a] 2018-01-05T11:29:10,495 INFO [TezTR-518813_1_41_1_0_0] input.MRInput: Initialized MRInput: a 2018-01-05T11:29:10,495 DEBUG [TezTR-518813_1_41_1_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 2 2018-01-05T11:29:10,495 INFO [TezTR-518813_1_41_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/5b41658d-5fab-4895-ab8a-ffc4518906a8/map.xml 2018-01-05T11:29:10,496 DEBUG [TezTR-518813_1_41_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:10,496 DEBUG [TezTR-518813_1_41_1_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180546}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, numFiles=2, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, rawDataSize=22748, partition_columns.types=string, numRows=242, partition_columns=ds, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=1511, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08, transient_lastDdlTime=1515180547}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180546}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, numFiles=2, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, rawDataSize=22748, partition_columns.types=string, numRows=242, partition_columns=ds, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=1511, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08, transient_lastDdlTime=1515180547}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat]} 2018-01-05T11:29:10,496 INFO [TezTR-518813_1_41_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/5b41658d-5fab-4895-ab8a-ffc4518906a8/map.xml 2018-01-05T11:29:10,496 DEBUG [TezTR-518813_1_41_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:10,496 DEBUG [TezTR-518813_1_41_1_0_0] io.HiveInputFormat: Processing org.apache.hadoop.hive.ql.io.orc.OrcInputFormat 2018-01-05T11:29:10,497 DEBUG [TezTR-518813_1_41_1_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:29:10,497 INFO [TezTR-518813_1_41_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/5b41658d-5fab-4895-ab8a-ffc4518906a8/map.xml 2018-01-05T11:29:10,497 DEBUG [TezTR-518813_1_41_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:10,497 INFO [TezTR-518813_1_41_1_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_41_01_000000_0 2018-01-05T11:29:10,497 INFO [TezTR-518813_1_41_1_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:10,497 DEBUG [TezTR-518813_1_41_1_0_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:10,497 DEBUG [TezTR-518813_1_41_1_0_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:10,497 DEBUG [TezTR-518813_1_41_1_0_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:10,497 DEBUG [TezTR-518813_1_41_1_0_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:10,497 DEBUG [TezTR-518813_1_41_1_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:10,497 DEBUG [TezTR-518813_1_41_1_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:10,497 INFO [TezTR-518813_1_41_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/5b41658d-5fab-4895-ab8a-ffc4518906a8/map.xml 2018-01-05T11:29:10,497 DEBUG [TezTR-518813_1_41_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:10,497 DEBUG [TezTR-518813_1_41_1_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0 2018-01-05T11:29:10,497 DEBUG [IO-Elevator-Thread-4] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_41 queryId: djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca instanceType: OrcEncodedDataReader 2018-01-05T11:29:10,498 INFO [IO-Elevator-Thread-4] LlapIoImpl: Processing data for file [-5122461581150239451, 1515180546000, 767]: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0 2018-01-05T11:29:10,498 DEBUG [IO-Elevator-Thread-4] LlapIoOrc: FileSplit {3, 583}; stripes {3, 583}, 2018-01-05T11:29:10,497 INFO [TezTR-518813_1_41_1_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0, start=3, length=583, isOriginal=true, fileLength=767, hasFooter=false, hasBase=true, deltas=0], org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0, start=3, length=566, isOriginal=true, fileLength=744, hasFooter=false, hasBase=true, deltas=0]], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=1149} 2018-01-05T11:29:10,498 INFO [TezTR-518813_1_41_1_0_0] input.MRInput: a initialized RecordReader from event 2018-01-05T11:29:10,498 DEBUG [TezTR-518813_1_41_1_0_0] tez.MapRecordProcessor: Starting Output: Reducer 2 2018-01-05T11:29:10,498 DEBUG [IO-Elevator-Thread-4] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:10,499 DEBUG [IO-Elevator-Thread-4] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:10,499 INFO [IO-Elevator-Thread-4] impl.OrcCodecPool: Got brand-new codec ZLIB 2018-01-05T11:29:10,499 INFO [TezTR-518813_1_41_1_0_0] impl.ExternalSorter: Reducer 2 using: memoryMb=10, keySerializerClass=class org.apache.hadoop.hive.ql.io.HiveKey, valueSerializerClass=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization$TezBytesWritableSerializer@11851ceb, comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator@8602230, partitioner=org.apache.tez.mapreduce.partition.MRPartitioner, serialization=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.hadoop.io.serializer.WritableSerialization, org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization, org.apache.hadoop.io.serializer.avro.AvroReflectSerialization, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:29:10,499 INFO [IO-Elevator-Thread-4] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [-5122461581150239451, 1515180546000, 767]): [{range start: 0 end: 38}] 2018-01-05T11:29:10,499 INFO [TezTR-518813_1_41_1_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 2: , UsingHashComparator=true 2018-01-05T11:29:10,499 INFO [IO-Elevator-Thread-4] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [-5122461581150239451, 1515180546000, 767], base offset 3): [{start: 0 end: 11 cache buffer: 0x317f0390(1)}, {start: 11 end: 38 cache buffer: 0x4252621d(1)}] 2018-01-05T11:29:10,499 INFO [IO-Elevator-Thread-4] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [-5122461581150239451, 1515180546000, 767]): [{range start: 76 end: 230}] 2018-01-05T11:29:10,499 INFO [IO-Elevator-Thread-4] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [-5122461581150239451, 1515180546000, 767], base offset 3): [{start: 76 end: 230 cache buffer: 0x112f2b3b(1)}] 2018-01-05T11:29:10,499 DEBUG [IO-Elevator-Thread-4] encoded.EncodedTreeReaderFactory: Column at 1 2:string has no data 2018-01-05T11:29:10,499 DEBUG [IO-Elevator-Thread-4] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: int streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 164 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:29:10,499 DEBUG [IO-Elevator-Thread-4] LlapIoOrc: Done with decode 2018-01-05T11:29:10,499 DEBUG [IO-Elevator-Thread-4] LlapIoImpl: setDone called; closed false, done false, err null, pending 1 2018-01-05T11:29:10,500 DEBUG [IO-Elevator-Thread-4] impl.StatsRecordingThreadPool: Updated stats: instance: OrcEncodedDataReader thread name: IO-Elevator-Thread-4 thread id: 2194 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:10,500 INFO [TezTR-518813_1_41_1_0_0] impl.PipelinedSorter: Newly allocated block size=10485760, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=10485760, total=10485760 2018-01-05T11:29:10,500 INFO [TezTR-518813_1_41_1_0_0] impl.PipelinedSorter: Pre allocating rest of memory buffers upfront 2018-01-05T11:29:10,500 INFO [TezTR-518813_1_41_1_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 2: , UsingHashComparator=true#blocks=1, maxMemUsage=10485760, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=10485760, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=10 2018-01-05T11:29:10,500 INFO [TezTR-518813_1_41_1_0_0] impl.PipelinedSorter: Reducer 2: reserved.remaining()=10485760, reserved.metasize=5242880 2018-01-05T11:29:10,500 INFO [TezTR-518813_1_41_1_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:29:10,500 INFO [TezTR-518813_1_41_1_0_0] vector.VectorMapOperator: Initializing operator MAP[0] 2018-01-05T11:29:10,500 DEBUG [TezTR-518813_1_41_1_0_0] vector.VectorMapOperator: Initialization Done 0 MAP 2018-01-05T11:29:10,500 DEBUG [TezTR-518813_1_41_1_0_0] vector.VectorMapOperator: Operator 0 MAP initialized 2018-01-05T11:29:10,500 DEBUG [TezTR-518813_1_41_1_0_0] vector.VectorMapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:29:10,501 INFO [TezTR-518813_1_41_1_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:29:10,501 INFO [TezTR-518813_1_41_1_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:10,501 DEBUG [TezTR-518813_1_41_1_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:29:10,501 INFO [TezTR-518813_1_41_1_0_0] exec.TableScanOperator: Initializing operator TS[0] 2018-01-05T11:29:10,501 DEBUG [TezTR-518813_1_41_1_0_0] exec.TableScanOperator: Initialization Done 0 TS 2018-01-05T11:29:10,501 DEBUG [TezTR-518813_1_41_1_0_0] exec.TableScanOperator: Operator 0 TS initialized 2018-01-05T11:29:10,501 DEBUG [TezTR-518813_1_41_1_0_0] exec.TableScanOperator: Initializing children of 0 TS 2018-01-05T11:29:10,501 DEBUG [TezTR-518813_1_41_1_0_0] vector.VectorSelectOperator: Initializing child 14 SEL 2018-01-05T11:29:10,501 INFO [TezTR-518813_1_41_1_0_0] vector.VectorSelectOperator: Initializing operator SEL[14] 2018-01-05T11:29:10,501 DEBUG [TezTR-518813_1_41_1_0_0] vector.VectorSelectOperator: Initialization Done 14 SEL 2018-01-05T11:29:10,501 DEBUG [TezTR-518813_1_41_1_0_0] vector.VectorSelectOperator: Operator 14 SEL initialized 2018-01-05T11:29:10,501 DEBUG [TezTR-518813_1_41_1_0_0] vector.VectorSelectOperator: Initializing children of 14 SEL 2018-01-05T11:29:10,501 DEBUG [TezTR-518813_1_41_1_0_0] reducesink.VectorReduceSinkLongOperator: Initializing child 15 RS 2018-01-05T11:29:10,501 INFO [TezTR-518813_1_41_1_0_0] reducesink.VectorReduceSinkLongOperator: Initializing operator RS[15] 2018-01-05T11:29:10,501 INFO [TezTR-518813_1_41_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/5b41658d-5fab-4895-ab8a-ffc4518906a8/map.xml 2018-01-05T11:29:10,501 DEBUG [TezTR-518813_1_41_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:10,501 INFO [TezTR-518813_1_41_1_0_0] reducesink.VectorReduceSinkCommonOperator: Using tag = 0 2018-01-05T11:29:10,501 DEBUG [TezTR-518813_1_41_1_0_0] reducesink.VectorReduceSinkLongOperator: Initialization Done 15 RS 2018-01-05T11:29:10,501 DEBUG [TezTR-518813_1_41_1_0_0] reducesink.VectorReduceSinkLongOperator: Operator 15 RS initialized 2018-01-05T11:29:10,501 DEBUG [TezTR-518813_1_41_1_0_0] reducesink.VectorReduceSinkLongOperator: Initialization Done 15 RS done is reset. 2018-01-05T11:29:10,501 DEBUG [TezTR-518813_1_41_1_0_0] vector.VectorSelectOperator: Initialization Done 14 SEL done is reset. 2018-01-05T11:29:10,501 DEBUG [TezTR-518813_1_41_1_0_0] exec.TableScanOperator: Initialization Done 0 TS done is reset. 2018-01-05T11:29:10,501 INFO [TezTR-518813_1_41_1_0_0] exec.OperatorUtils: Setting output collector: RS[15] --> Reducer 2 2018-01-05T11:29:10,501 DEBUG [TezTR-518813_1_41_1_0_0] log.PerfLogger: 2018-01-05T11:29:10,501 INFO [TezTR-518813_1_41_1_0_0] reducesink.VectorReduceSinkCommonOperator: RS[15]: records written - 1 2018-01-05T11:29:10,501 INFO [TezTR-518813_1_41_1_0_0] reducesink.VectorReduceSinkCommonOperator: RS[15]: records written - 10 2018-01-05T11:29:10,501 INFO [TezTR-518813_1_41_1_0_0] reducesink.VectorReduceSinkCommonOperator: RS[15]: records written - 100 2018-01-05T11:29:10,501 INFO [TezTR-518813_1_41_1_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:10,501 INFO [TezTR-518813_1_41_1_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, warehouse.tab, file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0 ([-5122461581150239451, 1515180546000, 767]), 0,1]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=1, NUM_ERRORS=0, ROWS_EMITTED=120, METADATA_CACHE_HIT=2, METADATA_CACHE_MISS=0, CACHE_HIT_BYTES=192, CACHE_MISS_BYTES=0, ALLOCATED_BYTES=0, ALLOCATED_USED_BYTES=0, TOTAL_IO_TIME_NS=1903149, DECODE_TIME_NS=240576, HDFS_TIME_NS=395533, CONSUMER_TIME_NS=249176 ] 2018-01-05T11:29:10,501 DEBUG [TezTR-518813_1_41_1_0_0] encoded.OrcEncodedDataReader: Encoded reader is being stopped 2018-01-05T11:29:10,501 DEBUG [TezTR-518813_1_41_1_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 1 of 2 2018-01-05T11:29:10,501 DEBUG [TezTR-518813_1_41_1_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180546}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, numFiles=2, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, rawDataSize=22748, partition_columns.types=string, numRows=242, partition_columns=ds, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=1511, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08, transient_lastDdlTime=1515180547}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180546}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, numFiles=2, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, rawDataSize=22748, partition_columns.types=string, numRows=242, partition_columns=ds, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=1511, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08, transient_lastDdlTime=1515180547}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat]} 2018-01-05T11:29:10,502 INFO [TezTR-518813_1_41_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/5b41658d-5fab-4895-ab8a-ffc4518906a8/map.xml 2018-01-05T11:29:10,502 DEBUG [TezTR-518813_1_41_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:10,502 DEBUG [TezTR-518813_1_41_1_0_0] io.HiveInputFormat: Processing org.apache.hadoop.hive.ql.io.orc.OrcInputFormat 2018-01-05T11:29:10,503 DEBUG [TezTR-518813_1_41_1_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:29:10,503 INFO [TezTR-518813_1_41_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/5b41658d-5fab-4895-ab8a-ffc4518906a8/map.xml 2018-01-05T11:29:10,503 DEBUG [TezTR-518813_1_41_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:10,503 INFO [TezTR-518813_1_41_1_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_41_01_000000_0 2018-01-05T11:29:10,503 INFO [TezTR-518813_1_41_1_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:10,503 DEBUG [TezTR-518813_1_41_1_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:10,503 DEBUG [TezTR-518813_1_41_1_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:10,503 INFO [TezTR-518813_1_41_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/5b41658d-5fab-4895-ab8a-ffc4518906a8/map.xml 2018-01-05T11:29:10,503 DEBUG [TezTR-518813_1_41_1_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:10,504 DEBUG [TezTR-518813_1_41_1_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0 2018-01-05T11:29:10,504 DEBUG [IO-Elevator-Thread-5] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_41 queryId: djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca instanceType: OrcEncodedDataReader 2018-01-05T11:29:10,504 INFO [IO-Elevator-Thread-5] LlapIoImpl: Processing data for file [-5122461581151735180, 1515180546000, 744]: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0 2018-01-05T11:29:10,504 DEBUG [IO-Elevator-Thread-5] LlapIoOrc: FileSplit {3, 566}; stripes {3, 566}, 2018-01-05T11:29:10,504 DEBUG [IO-Elevator-Thread-5] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:10,504 DEBUG [IO-Elevator-Thread-5] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:10,504 INFO [IO-Elevator-Thread-5] impl.OrcCodecPool: Got brand-new codec ZLIB 2018-01-05T11:29:10,504 INFO [IO-Elevator-Thread-5] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [-5122461581151735180, 1515180546000, 744]): [{range start: 0 end: 38}] 2018-01-05T11:29:10,504 INFO [IO-Elevator-Thread-5] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [-5122461581151735180, 1515180546000, 744], base offset 3): [{start: 0 end: 11 cache buffer: 0x4d1ed212(1)}, {start: 11 end: 38 cache buffer: 0x6497a38f(1)}] 2018-01-05T11:29:10,504 INFO [IO-Elevator-Thread-5] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [-5122461581151735180, 1515180546000, 744]): [{range start: 73 end: 204}] 2018-01-05T11:29:10,504 INFO [IO-Elevator-Thread-5] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [-5122461581151735180, 1515180546000, 744], base offset 3): [{start: 73 end: 204 cache buffer: 0x3e73bafa(1)}] 2018-01-05T11:29:10,504 DEBUG [IO-Elevator-Thread-5] encoded.EncodedTreeReaderFactory: Column at 1 2:string has no data 2018-01-05T11:29:10,504 DEBUG [IO-Elevator-Thread-5] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: int streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 136 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:29:10,504 DEBUG [IO-Elevator-Thread-5] LlapIoOrc: Done with decode 2018-01-05T11:29:10,504 DEBUG [IO-Elevator-Thread-5] LlapIoImpl: setDone called; closed false, done false, err null, pending 0 2018-01-05T11:29:10,505 DEBUG [IO-Elevator-Thread-5] impl.StatsRecordingThreadPool: Updated stats: instance: OrcEncodedDataReader thread name: IO-Elevator-Thread-5 thread id: 2196 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:10,505 INFO [TezTR-518813_1_41_1_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:10,505 INFO [TezTR-518813_1_41_1_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, warehouse.tab, file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0 ([-5122461581151735180, 1515180546000, 744]), 0,1]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=1, NUM_ERRORS=0, ROWS_EMITTED=122, METADATA_CACHE_HIT=2, METADATA_CACHE_MISS=0, CACHE_HIT_BYTES=169, CACHE_MISS_BYTES=0, ALLOCATED_BYTES=0, ALLOCATED_USED_BYTES=0, TOTAL_IO_TIME_NS=785294, DECODE_TIME_NS=155583, HDFS_TIME_NS=41190, CONSUMER_TIME_NS=600861 ] 2018-01-05T11:29:10,505 DEBUG [TezTR-518813_1_41_1_0_0] encoded.OrcEncodedDataReader: Encoded reader is being stopped 2018-01-05T11:29:10,505 DEBUG [TezTR-518813_1_41_1_0_0] log.PerfLogger: 2018-01-05T11:29:10,505 DEBUG [TezTR-518813_1_41_1_0_0] mr.ObjectCache: djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:29:10,505 DEBUG [TezTR-518813_1_41_1_0_0] vector.VectorMapOperator: close called for operator MAP[0] 2018-01-05T11:29:10,505 INFO [TezTR-518813_1_41_1_0_0] vector.VectorMapOperator: Closing operator MAP[0] 2018-01-05T11:29:10,505 INFO [TezTR-518813_1_41_1_0_0] vector.VectorMapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_1:242, 2018-01-05T11:29:10,505 DEBUG [TezTR-518813_1_41_1_0_0] vector.VectorMapOperator: Closing child = TS[0] 2018-01-05T11:29:10,505 DEBUG [TezTR-518813_1_41_1_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:29:10,505 INFO [TezTR-518813_1_41_1_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:29:10,505 DEBUG [TezTR-518813_1_41_1_0_0] exec.TableScanOperator: Closing child = SEL[14] 2018-01-05T11:29:10,505 DEBUG [TezTR-518813_1_41_1_0_0] vector.VectorSelectOperator: close called for operator SEL[14] 2018-01-05T11:29:10,505 DEBUG [TezTR-518813_1_41_1_0_0] vector.VectorSelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:10,505 INFO [TezTR-518813_1_41_1_0_0] vector.VectorSelectOperator: Closing operator SEL[14] 2018-01-05T11:29:10,505 DEBUG [TezTR-518813_1_41_1_0_0] vector.VectorSelectOperator: Closing child = RS[15] 2018-01-05T11:29:10,505 DEBUG [TezTR-518813_1_41_1_0_0] reducesink.VectorReduceSinkLongOperator: close called for operator RS[15] 2018-01-05T11:29:10,505 DEBUG [TezTR-518813_1_41_1_0_0] reducesink.VectorReduceSinkLongOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:10,505 INFO [TezTR-518813_1_41_1_0_0] reducesink.VectorReduceSinkLongOperator: Closing operator RS[15] 2018-01-05T11:29:10,505 INFO [TezTR-518813_1_41_1_0_0] reducesink.VectorReduceSinkCommonOperator: RS[15]: records written - 242 2018-01-05T11:29:10,505 INFO [TezTR-518813_1_41_1_0_0] reducesink.VectorReduceSinkLongOperator: RECORDS_OUT_INTERMEDIATE_Map_1:242, 2018-01-05T11:29:10,505 DEBUG [TezTR-518813_1_41_1_0_0] reducesink.VectorReduceSinkLongOperator: 15 Close done 2018-01-05T11:29:10,505 DEBUG [TezTR-518813_1_41_1_0_0] vector.VectorSelectOperator: 14 Close done 2018-01-05T11:29:10,505 DEBUG [TezTR-518813_1_41_1_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:29:10,505 DEBUG [TezTR-518813_1_41_1_0_0] vector.VectorMapOperator: 0 Close done 2018-01-05T11:29:10,505 INFO [TezTR-518813_1_41_1_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_41_01_000000_0 2018-01-05T11:29:10,505 INFO [TezTR-518813_1_41_1_0_0] impl.PipelinedSorter: Reducer 2: Starting flush of map output 2018-01-05T11:29:10,505 INFO [TezTR-518813_1_41_1_0_0] impl.PipelinedSorter: Reducer 2: Span0.length = 242, perItem = 6 2018-01-05T11:29:10,506 INFO [TezTR-518813_1_41_1_0_0] impl.PipelinedSorter: Reducer 2: done sorting span=0, length=242, time=1 2018-01-05T11:29:10,506 INFO [TezTR-518813_1_41_1_0_0] impl.PipelinedSorter: Reducer 2: Heap = SpanIterator<0:241> (span=Span[5242880,1452]), 2018-01-05T11:29:10,512 DEBUG [ORC_GET_SPLITS #6] io.AcidUtils: in directory file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 base = null deltas = 0 2018-01-05T11:29:10,512 DEBUG [InputInitializer {Map 4} #0] orc.OrcInputFormat: Split strategy: ETLSplitStrategy strategy for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:10,513 DEBUG [ORC_GET_SPLITS #1] orc.LocalCache: Serialized tail cached for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0 2018-01-05T11:29:10,513 DEBUG [ORC_GET_SPLITS #1] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:10,513 DEBUG [ORC_GET_SPLITS #1] orc.LocalCache: Serialized tail cached for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0 2018-01-05T11:29:10,513 DEBUG [ORC_GET_SPLITS #1] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:10,513 DEBUG [ORC_GET_SPLITS #1] orc.LocalCache: Serialized tail cached for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0 2018-01-05T11:29:10,513 DEBUG [ORC_GET_SPLITS #1] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:10,513 DEBUG [ORC_GET_SPLITS #1] orc.LocalCache: Serialized tail cached for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0 2018-01-05T11:29:10,513 DEBUG [ORC_GET_SPLITS #1] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:10,513 DEBUG [ORC_GET_SPLITS #1] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:10,513 DEBUG [ORC_GET_SPLITS #3] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:10,513 INFO [ORC_GET_SPLITS #5] impl.OrcCodecPool: Got brand-new codec ZLIB 2018-01-05T11:29:10,513 DEBUG [ORC_GET_SPLITS #3] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:29:10,513 INFO [ORC_GET_SPLITS #2] impl.OrcCodecPool: Got brand-new codec ZLIB 2018-01-05T11:29:10,513 DEBUG [ORC_GET_SPLITS #5] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:29:10,513 DEBUG [ORC_GET_SPLITS #1] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:29:10,514 DEBUG [ORC_GET_SPLITS #2] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:29:10,514 INFO [InputInitializer {Map 4} #0] orc.OrcInputFormat: FooterCacheHitRatio: 4/4 2018-01-05T11:29:10,514 DEBUG [InputInitializer {Map 4} #0] orc.OrcInputFormat: OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0, start=3, length=574, isOriginal=true, fileLength=752, hasFooter=false, hasBase=true, deltas=0] projected_columns_uncompressed_size: 363 2018-01-05T11:29:10,514 DEBUG [InputInitializer {Map 4} #0] orc.OrcInputFormat: OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0, start=3, length=641, isOriginal=true, fileLength=827, hasFooter=false, hasBase=true, deltas=0] projected_columns_uncompressed_size: 427 2018-01-05T11:29:10,514 DEBUG [InputInitializer {Map 4} #0] orc.OrcInputFormat: OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0, start=3, length=641, isOriginal=true, fileLength=823, hasFooter=false, hasBase=true, deltas=0] projected_columns_uncompressed_size: 398 2018-01-05T11:29:10,514 DEBUG [InputInitializer {Map 4} #0] orc.OrcInputFormat: OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0, start=3, length=586, isOriginal=true, fileLength=770, hasFooter=false, hasBase=true, deltas=0] projected_columns_uncompressed_size: 350 2018-01-05T11:29:10,514 DEBUG [InputInitializer {Map 4} #0] orc.OrcInputFormat: getSplits finished 2018-01-05T11:29:10,514 INFO [InputInitializer {Map 4} #0] io.HiveInputFormat: number of splits 4 2018-01-05T11:29:10,514 DEBUG [InputInitializer {Map 4} #0] log.PerfLogger: 2018-01-05T11:29:10,514 INFO [InputInitializer {Map 4} #0] tez.HiveSplitGenerator: Number of input splits: 4. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:29:10,514 INFO [InputInitializer {Map 4} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/d20ff970-7e5b-402f-9f4b-cba1fc4a55de/map.xml 2018-01-05T11:29:10,514 DEBUG [InputInitializer {Map 4} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/d20ff970-7e5b-402f-9f4b-cba1fc4a55de/map.xml 2018-01-05T11:29:10,514 DEBUG [InputInitializer {Map 4} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/d20ff970-7e5b-402f-9f4b-cba1fc4a55de/map.xml 2018-01-05T11:29:10,514 DEBUG [InputInitializer {Map 4} #0] log.PerfLogger: 2018-01-05T11:29:10,514 INFO [InputInitializer {Map 4} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:10,514 DEBUG [InputInitializer {Map 4} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:10,515 DEBUG [InputInitializer {Map 4} #0] log.PerfLogger: 2018-01-05T11:29:10,515 INFO [InputInitializer {Map 4} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 4 size: 3.07KB 2018-01-05T11:29:10,515 DEBUG [InputInitializer {Map 4} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0 to src new group? true 2018-01-05T11:29:10,515 DEBUG [InputInitializer {Map 4} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0 to src new group? false 2018-01-05T11:29:10,515 DEBUG [InputInitializer {Map 4} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0 to src new group? false 2018-01-05T11:29:10,515 DEBUG [InputInitializer {Map 4} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0 to src new group? false 2018-01-05T11:29:10,515 INFO [InputInitializer {Map 4} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:29:10,515 INFO [InputInitializer {Map 4} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:29:10,515 INFO [InputInitializer {Map 4} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:29:10,515 DEBUG [InputInitializer {Map 4} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 427 2018-01-05T11:29:10,515 DEBUG [InputInitializer {Map 4} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 398 2018-01-05T11:29:10,515 DEBUG [InputInitializer {Map 4} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 350 2018-01-05T11:29:10,515 DEBUG [InputInitializer {Map 4} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 363 2018-01-05T11:29:10,515 INFO [InputInitializer {Map 4} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 28 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 1538 Original splits: 4 2018-01-05T11:29:10,515 INFO [InputInitializer {Map 4} #0] grouper.TezSplitGrouper: Desired numSplits: 1 lengthPerGroup: 1538 numLocations: 1 numSplitsPerLocation: 4 numSplitsInGroup: 4 totalLength: 1538 numOriginalSplits: 4 . Grouping by length: true count: false nodeLocalOnly: false 2018-01-05T11:29:10,515 DEBUG [InputInitializer {Map 4} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 427 2018-01-05T11:29:10,515 DEBUG [InputInitializer {Map 4} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 398 2018-01-05T11:29:10,515 DEBUG [InputInitializer {Map 4} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 398 2018-01-05T11:29:10,515 DEBUG [InputInitializer {Map 4} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 350 2018-01-05T11:29:10,515 DEBUG [InputInitializer {Map 4} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 350 2018-01-05T11:29:10,515 DEBUG [InputInitializer {Map 4} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 363 2018-01-05T11:29:10,515 DEBUG [InputInitializer {Map 4} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 363 2018-01-05T11:29:10,515 INFO [InputInitializer {Map 4} #0] grouper.TezSplitGrouper: Number of splits desired: 1 created: 1 splitsProcessed: 4 2018-01-05T11:29:10,515 INFO [InputInitializer {Map 4} #0] tez.SplitGrouper: Original split count is 4 grouped split count is 1, for bucket: 1 2018-01-05T11:29:10,515 INFO [InputInitializer {Map 4} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:29:10,515 INFO [InputInitializer {Map 4} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: b on vertex vertex_1515180518813_0001_41_00 [Map 4] 2018-01-05T11:29:10,515 INFO [App Shared Pool - #2] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_41_00 [Map 4] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:10,515 INFO [App Shared Pool - #2] impl.VertexImpl: Got updated RootInputsSpecs: {b=forAllWorkUnits=true, update=[1]} 2018-01-05T11:29:10,515 INFO [App Shared Pool - #2] impl.VertexImpl: Vertex vertex_1515180518813_0001_41_00 [Map 4] parallelism set to 1 2018-01-05T11:29:10,515 INFO [App Shared Pool - #2] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_41_00 [Map 4] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:10,515 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:VERTEX_INITIALIZED]: vertexName=Map 4, vertexId=vertex_1515180518813_0001_41_00, initRequestedTime=1515180550440, initedTime=1515180550515, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:10,515 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_41_00 [Map 4] with distanceFromRoot: 0 2018-01-05T11:29:10,516 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_41_00 [Map 4] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:29:10,516 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_41_00, startRequestedTime=1515180550448, startedTime=1515180550516 2018-01-05T11:29:10,516 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_41_00, reconfigureDoneTime=1515180550516, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:29:10,516 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_41_00 [Map 4] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:10,516 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_41_00 for vertex: vertex_1515180518813_0001_41_02 [Reducer 2] numStartedSources: 2 numSources: 2 2018-01-05T11:29:10,516 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_41_02 [Reducer 2] with distanceFromRoot: 1 2018-01-05T11:29:10,516 INFO [App Shared Pool - #2] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 4 2018-01-05T11:29:10,516 INFO [App Shared Pool - #2] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 4 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:29:10,516 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_41_02, startRequestedTime=1515180550516, startedTime=1515180550516 2018-01-05T11:29:10,516 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_41_02 [Reducer 2] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:10,516 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_41_02 for vertex: vertex_1515180518813_0001_41_03 [Reducer 3] numStartedSources: 1 numSources: 1 2018-01-05T11:29:10,516 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_41_03 [Reducer 3] with distanceFromRoot: 2 2018-01-05T11:29:10,516 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_41_03, startRequestedTime=1515180550516, startedTime=1515180550516 2018-01-05T11:29:10,516 INFO [App Shared Pool - #2] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_41_00 [Map 4] 2018-01-05T11:29:10,516 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: OnVertexStarted vertex: Reducer 2 with 0 source tasks and 1 pending tasks 2018-01-05T11:29:10,516 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_41_03 [Reducer 3] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:10,516 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:TASK_STARTED]: vertexName=Map 4, taskId=task_1515180518813_0001_41_00_000000, scheduledTime=1515180550516, launchTime=1515180550516 2018-01-05T11:29:10,516 INFO [App Shared Pool - #1] vertexmanager.ShuffleVertexManagerBase: OnVertexStarted vertex: Reducer 3 with 0 source tasks and 1 pending tasks 2018-01-05T11:29:10,516 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_41_00 [Map 4] 2018-01-05T11:29:10,517 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Reducer 2 in vertex: Reducer 3 numBipartiteSourceTasks: 1 2018-01-05T11:29:10,517 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_41_00_000000_0, priority=11, capability=, hosts=null 2018-01-05T11:29:10,517 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=11,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:10,517 INFO [App Shared Pool - #3] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Map 4 in vertex: Reducer 2 numBipartiteSourceTasks: 1 2018-01-05T11:29:10,517 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=11,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:10,517 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:10,517 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Map 1 in vertex: Reducer 2 numBipartiteSourceTasks: 2 2018-01-05T11:29:10,517 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_41_00_000000_0 on hosts=null 2018-01-05T11:29:10,517 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_41_00_000000_0 on hosts=null 2018-01-05T11:29:10,517 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_41_00_000000_0 2018-01-05T11:29:10,517 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:10,517 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #102, task=TaskInfo{task=attempt_1515180518813_0001_41_00_000000_0, priority=11, startTime=0, containerId=null, uniqueId=101, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, to container=container_111101111_0001_00_000102 2018-01-05T11:29:10,518 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_41_00_000000_0; false 2018-01-05T11:29:10,518 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_41_00_000000_0, priority=11, startTime=311571027, containerId=container_111101111_0001_00_000102assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=2, ac=6, commF=false, disabled=false}, uniqueId=101, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:10,518 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000102 for node: 10.22.8.182:49818 2018-01-05T11:29:10,518 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000102, attempt_1515180518813_0001_41_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:10,518 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000102 for node: 10.22.8.182:49818 2018-01-05T11:29:10,518 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000102 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:10,518 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000102, launchTime=1515180550518 2018-01-05T11:29:10,519 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_41_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000102, dagName=select count(*) from tab a full oute...b.key (Stage-1), vertexName=Map 4, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{b,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{Reducer 2,org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput,1}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180550433, firstAttemptStartTime=1515180550516, currentAttemptStartTime=1515180550518} 2018-01-05T11:29:10,519 INFO [IPC Server handler 2 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000102, QueryId=djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca-41, DagId=41, VertexName=Map 4, TaskId=0, TaskAttemptId=0, SubmitTime=1515180550519 2018-01-05T11:29:10,519 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_41_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000102] on NM: [10.22.8.182:49818] 2018-01-05T11:29:10,519 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:10,519 DEBUG [IPC Server handler 2 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=41} with the ShuffleHandler 2018-01-05T11:29:10,519 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:10,519 DEBUG [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/41] 2018-01-05T11:29:10,519 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 4, taskAttemptId=attempt_1515180518813_0001_41_00_000000_0, startTime=1515180550518, containerId=container_111101111_0001_00_000102, nodeId=10.22.8.182:49818 2018-01-05T11:29:10,519 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=3, runningFragmentCount=1 2018-01-05T11:29:10,519 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_41_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:10,519 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_41_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180550516, dagStartTime=1515180550433, withinDagPriority=11, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:29:10,519 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_41_00_000000_0 : ACCEPTED 2018-01-05T11:29:10,519 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_41_00_000000_0, canFinish=true. Current state: preemptionQueueSize=1, numSlotsAvailable=3, waitQueueSize=1 2018-01-05T11:29:10,519 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_41_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180550516, dagStartTime=1515180550433, withinDagPriority=11, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:29:10,519 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_41_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:10,519 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_41_00_000000_0 to pre-emption queue 2018-01-05T11:29:10,519 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_41_00_000000_0: true 2018-01-05T11:29:10,520 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: taskOwner hashCode:992187299 2018-01-05T11:29:10,520 INFO [TaskCommunicator # 8] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_41_00_000000_0 2018-01-05T11:29:10,520 INFO [TaskCommunicator # 8] GuaranteedTasks: Received response for attempt_1515180518813_0001_41_00_000000_0, true 2018-01-05T11:29:10,520 INFO [Task-Executor-1] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : select count(*) from tab a full oute...b.key (Stage-1), VertexName: Map 4, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_41_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=b, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=Reducer 2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ] 2018-01-05T11:29:10,520 INFO [Task-Executor-1] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1023410176, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:10,520 INFO [Task-Executor-1] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_41_00_000000_0 vertexName: Map 4 2018-01-05T11:29:10,520 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000102, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_41_00_000000_0, eventCount=1 } 2018-01-05T11:29:10,520 DEBUG [TezTR-518813_1_41_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_41 queryId: djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca instanceType: TaskRunner2Callable 2018-01-05T11:29:10,520 INFO [TezTR-518813_1_41_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_41_00_000000_0 2018-01-05T11:29:10,521 DEBUG [TezTR-518813_1_41_0_0_0] log.PerfLogger: 2018-01-05T11:29:10,521 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:10,522 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:10,523 INFO [IPC Server handler 24 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_41_00_000000_0 sent events: (0-1). 2018-01-05T11:29:10,523 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:29:10,523 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_41_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:29:10,524 DEBUG [TezTR-518813_1_41_0_0_0] log.PerfLogger: 2018-01-05T11:29:10,524 INFO [TezTR-518813_1_41_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:10,524 INFO [TezTR-518813_1_41_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:10,524 INFO [TezTR-518813_1_41_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:10,525 INFO [I/O Setup 0 Initialize: {b}] input.MRInput: b using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:29:10,525 INFO [I/O Setup 0 Initialize: {b}] input.MRInputLegacy: b MRInputLegacy deferring initialization 2018-01-05T11:29:10,525 INFO [TezTR-518813_1_41_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:10,525 INFO [TezTR-518813_1_41_0_0_0] resources.MemoryDistributor: InitialRequests=[b:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [Reducer 2:OUTPUT:10485760:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput] 2018-01-05T11:29:10,525 INFO [TezTR-518813_1_41_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:10,525 INFO [TezTR-518813_1_41_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:10,525 INFO [TezTR-518813_1_41_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 10485760, TotalRequestedScaled: 9679163.076923078, TotalJVMHeap: 1023410176, TotalAvailable: 685684817, TotalRequested/TotalJVMHeap:0.01 2018-01-05T11:29:10,525 INFO [TezTR-518813_1_41_0_0_0] resources.MemoryDistributor: Allocations=[b:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:10485760:10485760] 2018-01-05T11:29:10,525 INFO [TezTR-518813_1_41_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:10,525 INFO [TezTR-518813_1_41_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: b being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:10,525 INFO [TezTR-518813_1_41_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:29:10,525 INFO [TezTR-518813_1_41_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:10,525 INFO [I/O Setup 1 Start: {b}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: b 2018-01-05T11:29:10,525 INFO [TezTR-518813_1_41_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:10,525 INFO [TezTR-518813_1_41_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_41_00_000000_0 2018-01-05T11:29:10,525 DEBUG [TezTR-518813_1_41_0_0_0] log.PerfLogger: 2018-01-05T11:29:10,525 DEBUG [TezTR-518813_1_41_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_41_00_000000_0_10311 2018-01-05T11:29:10,525 DEBUG [TezTR-518813_1_41_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:10,525 DEBUG [TezTR-518813_1_41_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_41_00_000000_0 2018-01-05T11:29:10,525 DEBUG [TezTR-518813_1_41_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:10,525 DEBUG [TezTR-518813_1_41_0_0_0] log.PerfLogger: 2018-01-05T11:29:10,525 DEBUG [TezTR-518813_1_41_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:10,525 DEBUG [TezTR-518813_1_41_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:10,525 DEBUG [TezTR-518813_1_41_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca_Map 4__MAP_PLAN__ 2018-01-05T11:29:10,525 INFO [TezTR-518813_1_41_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/d20ff970-7e5b-402f-9f4b-cba1fc4a55de/map.xml 2018-01-05T11:29:10,525 DEBUG [TezTR-518813_1_41_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/d20ff970-7e5b-402f-9f4b-cba1fc4a55de/map.xml 2018-01-05T11:29:10,525 DEBUG [TezTR-518813_1_41_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/d20ff970-7e5b-402f-9f4b-cba1fc4a55de/map.xml 2018-01-05T11:29:10,525 DEBUG [TezTR-518813_1_41_0_0_0] log.PerfLogger: 2018-01-05T11:29:10,525 INFO [TezTR-518813_1_41_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:10,526 DEBUG [TezTR-518813_1_41_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:10,526 INFO [TezTR-518813_1_41_1_0_0] impl.PipelinedSorter: Reducer 2: Spilling to /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/41/output/attempt_1515180518813_0001_41_01_000000_0_10310_0/file.out 2018-01-05T11:29:10,526 DEBUG [TezTR-518813_1_41_0_0_0] log.PerfLogger: 2018-01-05T11:29:10,526 INFO [TezTR-518813_1_41_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 4 size: 3.07KB 2018-01-05T11:29:10,527 DEBUG [TezTR-518813_1_41_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:10,527 INFO [TezTR-518813_1_41_0_0_0] tez.MapRecordProcessor: The input names are: [b] 2018-01-05T11:29:10,527 INFO [TezTR-518813_1_41_0_0_0] input.MRInput: Initialized MRInput: b 2018-01-05T11:29:10,527 DEBUG [TezTR-518813_1_41_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 4 2018-01-05T11:29:10,527 INFO [TezTR-518813_1_41_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/d20ff970-7e5b-402f-9f4b-cba1fc4a55de/map.xml 2018-01-05T11:29:10,527 DEBUG [TezTR-518813_1_41_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:10,527 DEBUG [TezTR-518813_1_41_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat]} 2018-01-05T11:29:10,527 INFO [TezTR-518813_1_41_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/d20ff970-7e5b-402f-9f4b-cba1fc4a55de/map.xml 2018-01-05T11:29:10,527 DEBUG [TezTR-518813_1_41_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:10,527 DEBUG [TezTR-518813_1_41_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.hive.ql.io.orc.OrcInputFormat 2018-01-05T11:29:10,528 DEBUG [TezTR-518813_1_41_0_0_0] impl.LlapRecordReader: Initializing for input Map 4 2018-01-05T11:29:10,528 INFO [TezTR-518813_1_41_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/d20ff970-7e5b-402f-9f4b-cba1fc4a55de/map.xml 2018-01-05T11:29:10,528 DEBUG [TezTR-518813_1_41_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:10,528 INFO [TezTR-518813_1_41_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_41_00_000000_0 2018-01-05T11:29:10,528 INFO [TezTR-518813_1_41_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:10,528 DEBUG [TezTR-518813_1_41_0_0_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:10,528 DEBUG [TezTR-518813_1_41_0_0_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:10,528 DEBUG [TezTR-518813_1_41_0_0_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:10,528 DEBUG [TezTR-518813_1_41_0_0_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:10,529 DEBUG [TezTR-518813_1_41_0_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:10,529 DEBUG [TezTR-518813_1_41_0_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:10,529 INFO [TezTR-518813_1_41_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/d20ff970-7e5b-402f-9f4b-cba1fc4a55de/map.xml 2018-01-05T11:29:10,529 DEBUG [TezTR-518813_1_41_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:10,529 DEBUG [IO-Elevator-Thread-6] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_41 queryId: djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca instanceType: OrcEncodedDataReader 2018-01-05T11:29:10,529 DEBUG [TezTR-518813_1_41_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0 2018-01-05T11:29:10,529 INFO [IO-Elevator-Thread-6] LlapIoImpl: Processing data for file [6462268725460710401, 1515180546000, 827]: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0 2018-01-05T11:29:10,529 INFO [TezTR-518813_1_41_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0, start=3, length=641, isOriginal=true, fileLength=827, hasFooter=false, hasBase=true, deltas=0], org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0, start=3, length=641, isOriginal=true, fileLength=823, hasFooter=false, hasBase=true, deltas=0], org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0, start=3, length=586, isOriginal=true, fileLength=770, hasFooter=false, hasBase=true, deltas=0], org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0, start=3, length=574, isOriginal=true, fileLength=752, hasFooter=false, hasBase=true, deltas=0]], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=2442} 2018-01-05T11:29:10,529 DEBUG [IO-Elevator-Thread-6] LlapIoOrc: FileSplit {3, 641}; stripes {3, 641}, 2018-01-05T11:29:10,529 INFO [TezTR-518813_1_41_0_0_0] input.MRInput: b initialized RecordReader from event 2018-01-05T11:29:10,529 DEBUG [TezTR-518813_1_41_0_0_0] tez.MapRecordProcessor: Starting Output: Reducer 2 2018-01-05T11:29:10,530 DEBUG [IO-Elevator-Thread-6] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:10,530 DEBUG [IO-Elevator-Thread-6] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:10,530 INFO [TezTR-518813_1_41_0_0_0] impl.ExternalSorter: Reducer 2 using: memoryMb=10, keySerializerClass=class org.apache.hadoop.hive.ql.io.HiveKey, valueSerializerClass=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization$TezBytesWritableSerializer@5dc3c9fa, comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator@17073086, partitioner=org.apache.tez.mapreduce.partition.MRPartitioner, serialization=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.hadoop.io.serializer.WritableSerialization, org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization, org.apache.hadoop.io.serializer.avro.AvroReflectSerialization, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:29:10,530 DEBUG [IO-Elevator-Thread-6] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:10,530 INFO [TezTR-518813_1_41_0_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 2: , UsingHashComparator=true 2018-01-05T11:29:10,530 INFO [IO-Elevator-Thread-6] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725460710401, 1515180546000, 827]): [{range start: 0 end: 40}] 2018-01-05T11:29:10,531 INFO [IO-Elevator-Thread-6] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [6462268725460710401, 1515180546000, 827], base offset 3): [{start: 0 end: 12 cache buffer: 0x7302251(1)}, {start: 12 end: 40 cache buffer: 0x2cb64d2b(1)}] 2018-01-05T11:29:10,531 INFO [IO-Elevator-Thread-6] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725460710401, 1515180546000, 827]): [{range start: 79 end: 240}] 2018-01-05T11:29:10,531 INFO [IO-Elevator-Thread-6] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [6462268725460710401, 1515180546000, 827], base offset 3): [{start: 79 end: 240 cache buffer: 0x24272ac5(1)}] 2018-01-05T11:29:10,531 DEBUG [IO-Elevator-Thread-6] encoded.EncodedTreeReaderFactory: Column at 1 2:string has no data 2018-01-05T11:29:10,531 DEBUG [IO-Elevator-Thread-6] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: int streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 200 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:29:10,531 DEBUG [IO-Elevator-Thread-6] LlapIoOrc: Done with decode 2018-01-05T11:29:10,531 DEBUG [IO-Elevator-Thread-6] LlapIoImpl: setDone called; closed false, done false, err null, pending 1 2018-01-05T11:29:10,532 DEBUG [IO-Elevator-Thread-6] impl.StatsRecordingThreadPool: Updated stats: instance: OrcEncodedDataReader thread name: IO-Elevator-Thread-6 thread id: 2198 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:10,532 INFO [TezTR-518813_1_41_0_0_0] impl.PipelinedSorter: Newly allocated block size=10485760, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=10485760, total=10485760 2018-01-05T11:29:10,532 INFO [TezTR-518813_1_41_0_0_0] impl.PipelinedSorter: Pre allocating rest of memory buffers upfront 2018-01-05T11:29:10,532 INFO [TezTR-518813_1_41_0_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 2: , UsingHashComparator=true#blocks=1, maxMemUsage=10485760, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=10485760, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=10 2018-01-05T11:29:10,532 INFO [TezTR-518813_1_41_0_0_0] impl.PipelinedSorter: Reducer 2: reserved.remaining()=10485760, reserved.metasize=5242880 2018-01-05T11:29:10,532 INFO [TezTR-518813_1_41_0_0_0] tez.MapRecordProcessor: Main input name is Map 4 2018-01-05T11:29:10,532 INFO [TezTR-518813_1_41_0_0_0] vector.VectorMapOperator: Initializing operator MAP[0] 2018-01-05T11:29:10,532 DEBUG [TezTR-518813_1_41_0_0_0] vector.VectorMapOperator: Initialization Done 0 MAP 2018-01-05T11:29:10,532 DEBUG [TezTR-518813_1_41_0_0_0] vector.VectorMapOperator: Operator 0 MAP initialized 2018-01-05T11:29:10,532 DEBUG [TezTR-518813_1_41_0_0_0] vector.VectorMapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:29:10,533 INFO [TezTR-518813_1_41_0_0_0] tez.MapRecordProcessor: Id =0 Id =2 Id =16 Id =17 <\Children> Id = 16 null<\Parent> <\RS> <\Children> Id = 2 null<\Parent> <\SEL> <\Children> <\Parent> <\TS> <\Children> <\Parent> <\MAP> 2018-01-05T11:29:10,533 INFO [TezTR-518813_1_41_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:10,533 DEBUG [TezTR-518813_1_41_0_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:10,533 INFO [TezTR-518813_1_41_0_0_0] exec.TableScanOperator: Initializing operator TS[2] 2018-01-05T11:29:10,533 DEBUG [TezTR-518813_1_41_0_0_0] exec.TableScanOperator: Initialization Done 2 TS 2018-01-05T11:29:10,533 DEBUG [TezTR-518813_1_41_0_0_0] exec.TableScanOperator: Operator 2 TS initialized 2018-01-05T11:29:10,533 DEBUG [TezTR-518813_1_41_0_0_0] exec.TableScanOperator: Initializing children of 2 TS 2018-01-05T11:29:10,533 DEBUG [TezTR-518813_1_41_0_0_0] vector.VectorSelectOperator: Initializing child 16 SEL 2018-01-05T11:29:10,533 INFO [TezTR-518813_1_41_0_0_0] vector.VectorSelectOperator: Initializing operator SEL[16] 2018-01-05T11:29:10,533 DEBUG [TezTR-518813_1_41_0_0_0] vector.VectorSelectOperator: Initialization Done 16 SEL 2018-01-05T11:29:10,533 DEBUG [TezTR-518813_1_41_0_0_0] vector.VectorSelectOperator: Operator 16 SEL initialized 2018-01-05T11:29:10,533 DEBUG [TezTR-518813_1_41_0_0_0] vector.VectorSelectOperator: Initializing children of 16 SEL 2018-01-05T11:29:10,533 DEBUG [TezTR-518813_1_41_0_0_0] reducesink.VectorReduceSinkLongOperator: Initializing child 17 RS 2018-01-05T11:29:10,533 INFO [TezTR-518813_1_41_0_0_0] reducesink.VectorReduceSinkLongOperator: Initializing operator RS[17] 2018-01-05T11:29:10,533 INFO [TezTR-518813_1_41_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/d20ff970-7e5b-402f-9f4b-cba1fc4a55de/map.xml 2018-01-05T11:29:10,533 DEBUG [TezTR-518813_1_41_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:10,533 INFO [TezTR-518813_1_41_0_0_0] reducesink.VectorReduceSinkCommonOperator: Using tag = 1 2018-01-05T11:29:10,533 DEBUG [TezTR-518813_1_41_0_0_0] reducesink.VectorReduceSinkLongOperator: Initialization Done 17 RS 2018-01-05T11:29:10,533 DEBUG [TezTR-518813_1_41_0_0_0] reducesink.VectorReduceSinkLongOperator: Operator 17 RS initialized 2018-01-05T11:29:10,533 DEBUG [TezTR-518813_1_41_0_0_0] reducesink.VectorReduceSinkLongOperator: Initialization Done 17 RS done is reset. 2018-01-05T11:29:10,533 DEBUG [TezTR-518813_1_41_0_0_0] vector.VectorSelectOperator: Initialization Done 16 SEL done is reset. 2018-01-05T11:29:10,533 DEBUG [TezTR-518813_1_41_0_0_0] exec.TableScanOperator: Initialization Done 2 TS done is reset. 2018-01-05T11:29:10,533 INFO [TezTR-518813_1_41_0_0_0] exec.OperatorUtils: Setting output collector: RS[17] --> Reducer 2 2018-01-05T11:29:10,533 DEBUG [TezTR-518813_1_41_0_0_0] log.PerfLogger: 2018-01-05T11:29:10,533 INFO [TezTR-518813_1_41_0_0_0] reducesink.VectorReduceSinkCommonOperator: RS[17]: records written - 1 2018-01-05T11:29:10,533 INFO [TezTR-518813_1_41_0_0_0] reducesink.VectorReduceSinkCommonOperator: RS[17]: records written - 10 2018-01-05T11:29:10,533 INFO [TezTR-518813_1_41_0_0_0] reducesink.VectorReduceSinkCommonOperator: RS[17]: records written - 100 2018-01-05T11:29:10,533 INFO [TezTR-518813_1_41_0_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:10,533 INFO [TezTR-518813_1_41_0_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, warehouse.tab_part, file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0 ([6462268725460710401, 1515180546000, 827]), 0,1]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=1, NUM_ERRORS=0, ROWS_EMITTED=138, METADATA_CACHE_HIT=2, METADATA_CACHE_MISS=0, CACHE_HIT_BYTES=201, CACHE_MISS_BYTES=0, ALLOCATED_BYTES=0, ALLOCATED_USED_BYTES=0, TOTAL_IO_TIME_NS=2308100, DECODE_TIME_NS=256398, HDFS_TIME_NS=574948, CONSUMER_TIME_NS=342050 ] 2018-01-05T11:29:10,533 DEBUG [TezTR-518813_1_41_0_0_0] encoded.OrcEncodedDataReader: Encoded reader is being stopped 2018-01-05T11:29:10,533 DEBUG [TezTR-518813_1_41_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 1 of 4 2018-01-05T11:29:10,533 DEBUG [TezTR-518813_1_41_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat]} 2018-01-05T11:29:10,534 INFO [TezTR-518813_1_41_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/d20ff970-7e5b-402f-9f4b-cba1fc4a55de/map.xml 2018-01-05T11:29:10,534 DEBUG [TezTR-518813_1_41_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:10,534 DEBUG [TezTR-518813_1_41_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.hive.ql.io.orc.OrcInputFormat 2018-01-05T11:29:10,534 DEBUG [TezTR-518813_1_41_0_0_0] impl.LlapRecordReader: Initializing for input Map 4 2018-01-05T11:29:10,535 INFO [TezTR-518813_1_41_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/d20ff970-7e5b-402f-9f4b-cba1fc4a55de/map.xml 2018-01-05T11:29:10,535 DEBUG [TezTR-518813_1_41_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:10,535 INFO [TezTR-518813_1_41_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_41_00_000000_0 2018-01-05T11:29:10,535 INFO [TezTR-518813_1_41_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:10,535 DEBUG [TezTR-518813_1_41_0_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:10,535 DEBUG [TezTR-518813_1_41_0_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:10,535 INFO [TezTR-518813_1_41_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/d20ff970-7e5b-402f-9f4b-cba1fc4a55de/map.xml 2018-01-05T11:29:10,535 DEBUG [TezTR-518813_1_41_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:10,535 DEBUG [TezTR-518813_1_41_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0 2018-01-05T11:29:10,535 DEBUG [IO-Elevator-Thread-7] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_41 queryId: djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca instanceType: OrcEncodedDataReader 2018-01-05T11:29:10,535 INFO [IO-Elevator-Thread-7] LlapIoImpl: Processing data for file [6462268725462206130, 1515180546000, 823]: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0 2018-01-05T11:29:10,535 DEBUG [IO-Elevator-Thread-7] LlapIoOrc: FileSplit {3, 641}; stripes {3, 641}, 2018-01-05T11:29:10,535 DEBUG [IO-Elevator-Thread-7] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:10,535 DEBUG [IO-Elevator-Thread-7] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:10,535 DEBUG [IO-Elevator-Thread-7] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:10,535 INFO [IO-Elevator-Thread-7] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725462206130, 1515180546000, 823]): [{range start: 0 end: 40}] 2018-01-05T11:29:10,536 INFO [IO-Elevator-Thread-7] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [6462268725462206130, 1515180546000, 823], base offset 3): [{start: 0 end: 12 cache buffer: 0x469c2dcf(1)}, {start: 12 end: 40 cache buffer: 0x2f556695(1)}] 2018-01-05T11:29:10,536 INFO [IO-Elevator-Thread-7] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725462206130, 1515180546000, 823]): [{range start: 76 end: 242}] 2018-01-05T11:29:10,536 INFO [IO-Elevator-Thread-7] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [6462268725462206130, 1515180546000, 823], base offset 3): [{start: 76 end: 242 cache buffer: 0x6e3a2cb3(1)}] 2018-01-05T11:29:10,536 DEBUG [IO-Elevator-Thread-7] encoded.EncodedTreeReaderFactory: Column at 1 2:string has no data 2018-01-05T11:29:10,536 DEBUG [IO-Elevator-Thread-7] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: int streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 200 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:29:10,536 DEBUG [IO-Elevator-Thread-7] LlapIoOrc: Done with decode 2018-01-05T11:29:10,536 DEBUG [IO-Elevator-Thread-7] LlapIoImpl: setDone called; closed false, done false, err null, pending 0 2018-01-05T11:29:10,536 DEBUG [IO-Elevator-Thread-7] impl.StatsRecordingThreadPool: Updated stats: instance: OrcEncodedDataReader thread name: IO-Elevator-Thread-7 thread id: 2200 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:10,537 INFO [TezTR-518813_1_41_0_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:10,537 INFO [TezTR-518813_1_41_0_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, warehouse.tab_part, file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0 ([6462268725462206130, 1515180546000, 823]), 0,1]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=1, NUM_ERRORS=0, ROWS_EMITTED=128, METADATA_CACHE_HIT=2, METADATA_CACHE_MISS=0, CACHE_HIT_BYTES=206, CACHE_MISS_BYTES=0, ALLOCATED_BYTES=0, ALLOCATED_USED_BYTES=0, TOTAL_IO_TIME_NS=693018, DECODE_TIME_NS=162996, HDFS_TIME_NS=36041, CONSUMER_TIME_NS=752488 ] 2018-01-05T11:29:10,537 DEBUG [TezTR-518813_1_41_0_0_0] encoded.OrcEncodedDataReader: Encoded reader is being stopped 2018-01-05T11:29:10,537 DEBUG [TezTR-518813_1_41_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 2 of 4 2018-01-05T11:29:10,537 DEBUG [TezTR-518813_1_41_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat]} 2018-01-05T11:29:10,537 INFO [TezTR-518813_1_41_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/d20ff970-7e5b-402f-9f4b-cba1fc4a55de/map.xml 2018-01-05T11:29:10,537 DEBUG [TezTR-518813_1_41_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:10,537 DEBUG [TezTR-518813_1_41_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.hive.ql.io.orc.OrcInputFormat 2018-01-05T11:29:10,538 DEBUG [TezTR-518813_1_41_0_0_0] impl.LlapRecordReader: Initializing for input Map 4 2018-01-05T11:29:10,538 INFO [TezTR-518813_1_41_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/d20ff970-7e5b-402f-9f4b-cba1fc4a55de/map.xml 2018-01-05T11:29:10,538 DEBUG [TezTR-518813_1_41_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:10,538 INFO [TezTR-518813_1_41_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_41_00_000000_0 2018-01-05T11:29:10,539 INFO [TezTR-518813_1_41_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:10,539 DEBUG [TezTR-518813_1_41_0_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:10,539 DEBUG [TezTR-518813_1_41_0_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:10,539 INFO [TezTR-518813_1_41_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/d20ff970-7e5b-402f-9f4b-cba1fc4a55de/map.xml 2018-01-05T11:29:10,539 DEBUG [TezTR-518813_1_41_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:10,539 DEBUG [TezTR-518813_1_41_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0 2018-01-05T11:29:10,539 DEBUG [IO-Elevator-Thread-8] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_41 queryId: djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca instanceType: OrcEncodedDataReader 2018-01-05T11:29:10,540 INFO [IO-Elevator-Thread-8] LlapIoImpl: Processing data for file [6462268725463701859, 1515180546000, 770]: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0 2018-01-05T11:29:10,540 DEBUG [IO-Elevator-Thread-8] LlapIoOrc: FileSplit {3, 586}; stripes {3, 586}, 2018-01-05T11:29:10,540 DEBUG [IO-Elevator-Thread-8] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:10,540 DEBUG [IO-Elevator-Thread-8] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:10,540 INFO [IO-Elevator-Thread-8] impl.OrcCodecPool: Got brand-new codec ZLIB 2018-01-05T11:29:10,540 INFO [IO-Elevator-Thread-8] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725463701859, 1515180546000, 770]): [{range start: 0 end: 38}] 2018-01-05T11:29:10,540 INFO [IO-Elevator-Thread-8] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [6462268725463701859, 1515180546000, 770], base offset 3): [{start: 0 end: 11 cache buffer: 0x5a722c41(1)}, {start: 11 end: 38 cache buffer: 0x7b40d4ac(1)}] 2018-01-05T11:29:10,540 INFO [IO-Elevator-Thread-8] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725463701859, 1515180546000, 770]): [{range start: 76 end: 223}] 2018-01-05T11:29:10,540 INFO [IO-Elevator-Thread-8] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [6462268725463701859, 1515180546000, 770], base offset 3): [{start: 76 end: 223 cache buffer: 0x1ddca8ea(1)}] 2018-01-05T11:29:10,540 DEBUG [IO-Elevator-Thread-8] encoded.EncodedTreeReaderFactory: Column at 1 2:string has no data 2018-01-05T11:29:10,540 DEBUG [IO-Elevator-Thread-8] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: int streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 171 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:29:10,540 DEBUG [IO-Elevator-Thread-8] LlapIoOrc: Done with decode 2018-01-05T11:29:10,540 DEBUG [IO-Elevator-Thread-8] LlapIoImpl: setDone called; closed false, done false, err null, pending 1 2018-01-05T11:29:10,540 DEBUG [IO-Elevator-Thread-8] impl.StatsRecordingThreadPool: Updated stats: instance: OrcEncodedDataReader thread name: IO-Elevator-Thread-8 thread id: 2387 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:10,541 INFO [TezTR-518813_1_41_0_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:10,541 INFO [TezTR-518813_1_41_0_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, warehouse.tab_part, file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0 ([6462268725463701859, 1515180546000, 770]), 0,1]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=1, NUM_ERRORS=0, ROWS_EMITTED=115, METADATA_CACHE_HIT=2, METADATA_CACHE_MISS=0, CACHE_HIT_BYTES=185, CACHE_MISS_BYTES=0, ALLOCATED_BYTES=0, ALLOCATED_USED_BYTES=0, TOTAL_IO_TIME_NS=708828, DECODE_TIME_NS=162974, HDFS_TIME_NS=36754, CONSUMER_TIME_NS=190668 ] 2018-01-05T11:29:10,541 DEBUG [TezTR-518813_1_41_0_0_0] encoded.OrcEncodedDataReader: Encoded reader is being stopped 2018-01-05T11:29:10,541 DEBUG [TezTR-518813_1_41_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 3 of 4 2018-01-05T11:29:10,541 DEBUG [TezTR-518813_1_41_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180544}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab_part, numFiles=4, bucket_field_name=key, serialization.ddl=struct tab_part { i32 key, string value}, serialization.format=1, rawDataSize=47000, partition_columns.types=string, numRows=500, partition_columns=ds, bucket_count=4, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=3172, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08, transient_lastDdlTime=1515180546}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat]} 2018-01-05T11:29:10,541 INFO [TezTR-518813_1_41_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/d20ff970-7e5b-402f-9f4b-cba1fc4a55de/map.xml 2018-01-05T11:29:10,541 DEBUG [TezTR-518813_1_41_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:10,541 DEBUG [TezTR-518813_1_41_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.hive.ql.io.orc.OrcInputFormat 2018-01-05T11:29:10,546 DEBUG [TezTR-518813_1_41_1_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:10,546 INFO [TezTR-518813_1_41_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:10,546 INFO [TezTR-518813_1_41_1_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_41_01_000000_0, askedToStop=false 2018-01-05T11:29:10,546 DEBUG [TezTR-518813_1_41_0_0_0] impl.LlapRecordReader: Initializing for input Map 4 2018-01-05T11:29:10,546 INFO [TezTR-518813_1_41_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/d20ff970-7e5b-402f-9f4b-cba1fc4a55de/map.xml 2018-01-05T11:29:10,547 DEBUG [TezTR-518813_1_41_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:10,547 INFO [TezTR-518813_1_41_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_41_00_000000_0 2018-01-05T11:29:10,547 INFO [TezTR-518813_1_41_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:10,546 INFO [TezTR-518813_1_41_1_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_41_01_000000_0, stopRequested=false 2018-01-05T11:29:10,547 DEBUG [TezTR-518813_1_41_0_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:10,547 DEBUG [TezTR-518813_1_41_0_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:10,547 INFO [TezTR-518813_1_41_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/d20ff970-7e5b-402f-9f4b-cba1fc4a55de/map.xml 2018-01-05T11:29:10,547 DEBUG [TezTR-518813_1_41_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:10,547 DEBUG [TezTR-518813_1_41_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0 2018-01-05T11:29:10,547 DEBUG [IO-Elevator-Thread-9] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_41 queryId: djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca instanceType: OrcEncodedDataReader 2018-01-05T11:29:10,547 INFO [IO-Elevator-Thread-9] LlapIoImpl: Processing data for file [6462268725459214672, 1515180546000, 752]: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0 2018-01-05T11:29:10,547 DEBUG [IO-Elevator-Thread-9] LlapIoOrc: FileSplit {3, 574}; stripes {3, 574}, 2018-01-05T11:29:10,547 DEBUG [IO-Elevator-Thread-9] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:10,547 DEBUG [IO-Elevator-Thread-9] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:10,548 INFO [IO-Elevator-Thread-9] impl.OrcCodecPool: Got brand-new codec ZLIB 2018-01-05T11:29:10,548 INFO [IO-Elevator-Thread-9] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725459214672, 1515180546000, 752]): [{range start: 0 end: 38}] 2018-01-05T11:29:10,548 INFO [IO-Elevator-Thread-9] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [6462268725459214672, 1515180546000, 752], base offset 3): [{start: 0 end: 11 cache buffer: 0x1c0198da(1)}, {start: 11 end: 38 cache buffer: 0x55974329(1)}] 2018-01-05T11:29:10,548 INFO [IO-Elevator-Thread-9] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [6462268725459214672, 1515180546000, 752]): [{range start: 73 end: 208}] 2018-01-05T11:29:10,548 INFO [IO-Elevator-Thread-9] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [6462268725459214672, 1515180546000, 752], base offset 3): [{start: 73 end: 208 cache buffer: 0x21723a65(1)}] 2018-01-05T11:29:10,548 DEBUG [IO-Elevator-Thread-9] encoded.EncodedTreeReaderFactory: Column at 1 2:string has no data 2018-01-05T11:29:10,548 DEBUG [IO-Elevator-Thread-9] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: int streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 150 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:29:10,548 DEBUG [IO-Elevator-Thread-9] LlapIoOrc: Done with decode 2018-01-05T11:29:10,551 DEBUG [IO-Elevator-Thread-9] LlapIoImpl: setDone called; closed false, done false, err null, pending 1 2018-01-05T11:29:10,551 DEBUG [IO-Elevator-Thread-9] impl.StatsRecordingThreadPool: Updated stats: instance: OrcEncodedDataReader thread name: IO-Elevator-Thread-9 thread id: 2389 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:10,551 INFO [TezTR-518813_1_41_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_41_01_000000_0: Counters: 31 [[File System Counters FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=0, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=242, INPUT_RECORDS_PROCESSED=2, INPUT_SPLIT_LENGTH_BYTES=1149, OUTPUT_RECORDS=242, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=1452, OUTPUT_BYTES_WITH_OVERHEAD=1447, OUTPUT_BYTES_PHYSICAL=1451, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=242, RECORDS_OUT_INTERMEDIATE_Map_1=242][org.apache.hadoop.hive.llap.counters.LlapIOCounters CACHE_HIT_BYTES=361, CACHE_MISS_BYTES=0, CONSUMER_TIME_NS=850037, DECODE_TIME_NS=396159, HDFS_TIME_NS=436723, METADATA_CACHE_HIT=4, NUM_DECODED_BATCHES=2, NUM_VECTOR_BATCHES=2, ROWS_EMITTED=242, SELECTED_ROWGROUPS=2, TOTAL_IO_TIME_NS=2688443]] 2018-01-05T11:29:10,551 DEBUG [TezTR-518813_1_41_1_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_41_1_0_0 thread id: 2846 scheme: file bytesRead: 64 bytesWritten: 1483 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:10,552 INFO [TezTR-518813_1_41_0_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:10,552 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_41_01_000000_0, isTaskDone=true 2018-01-05T11:29:10,552 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000101, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_41_01_000000_0, eventCount=4 } 2018-01-05T11:29:10,552 INFO [TezTR-518813_1_41_0_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, warehouse.tab_part, file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0 ([6462268725459214672, 1515180546000, 752]), 0,1]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=1, NUM_ERRORS=0, ROWS_EMITTED=119, METADATA_CACHE_HIT=2, METADATA_CACHE_MISS=0, CACHE_HIT_BYTES=173, CACHE_MISS_BYTES=0, ALLOCATED_BYTES=0, ALLOCATED_USED_BYTES=0, TOTAL_IO_TIME_NS=3843605, DECODE_TIME_NS=269654, HDFS_TIME_NS=42310, CONSUMER_TIME_NS=241946 ] 2018-01-05T11:29:10,552 DEBUG [TezTR-518813_1_41_0_0_0] encoded.OrcEncodedDataReader: Encoded reader is being stopped 2018-01-05T11:29:10,552 DEBUG [TezTR-518813_1_41_0_0_0] log.PerfLogger: 2018-01-05T11:29:10,552 DEBUG [TezTR-518813_1_41_0_0_0] mr.ObjectCache: djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca_Map 4__MAP_PLAN__ no longer needed 2018-01-05T11:29:10,552 DEBUG [TezTR-518813_1_41_0_0_0] vector.VectorMapOperator: close called for operator MAP[0] 2018-01-05T11:29:10,552 INFO [TezTR-518813_1_41_0_0_0] vector.VectorMapOperator: Closing operator MAP[0] 2018-01-05T11:29:10,552 INFO [TezTR-518813_1_41_0_0_0] vector.VectorMapOperator: RECORDS_IN_Map_4:500, DESERIALIZE_ERRORS:0, 2018-01-05T11:29:10,552 DEBUG [TezTR-518813_1_41_0_0_0] vector.VectorMapOperator: Closing child = TS[2] 2018-01-05T11:29:10,552 DEBUG [TezTR-518813_1_41_0_0_0] exec.TableScanOperator: close called for operator TS[2] 2018-01-05T11:29:10,552 INFO [TezTR-518813_1_41_0_0_0] exec.TableScanOperator: Closing operator TS[2] 2018-01-05T11:29:10,552 DEBUG [TezTR-518813_1_41_0_0_0] exec.TableScanOperator: Closing child = SEL[16] 2018-01-05T11:29:10,552 DEBUG [TezTR-518813_1_41_0_0_0] vector.VectorSelectOperator: close called for operator SEL[16] 2018-01-05T11:29:10,552 DEBUG [TezTR-518813_1_41_0_0_0] vector.VectorSelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:10,552 INFO [TezTR-518813_1_41_0_0_0] vector.VectorSelectOperator: Closing operator SEL[16] 2018-01-05T11:29:10,552 DEBUG [TezTR-518813_1_41_0_0_0] vector.VectorSelectOperator: Closing child = RS[17] 2018-01-05T11:29:10,552 DEBUG [TezTR-518813_1_41_0_0_0] reducesink.VectorReduceSinkLongOperator: close called for operator RS[17] 2018-01-05T11:29:10,552 DEBUG [TezTR-518813_1_41_0_0_0] reducesink.VectorReduceSinkLongOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:10,552 INFO [TezTR-518813_1_41_0_0_0] reducesink.VectorReduceSinkLongOperator: Closing operator RS[17] 2018-01-05T11:29:10,552 INFO [TezTR-518813_1_41_0_0_0] reducesink.VectorReduceSinkCommonOperator: RS[17]: records written - 500 2018-01-05T11:29:10,552 INFO [TezTR-518813_1_41_0_0_0] reducesink.VectorReduceSinkLongOperator: RECORDS_OUT_INTERMEDIATE_Map_4:500, 2018-01-05T11:29:10,552 DEBUG [TezTR-518813_1_41_0_0_0] reducesink.VectorReduceSinkLongOperator: 17 Close done 2018-01-05T11:29:10,552 DEBUG [TezTR-518813_1_41_0_0_0] vector.VectorSelectOperator: 16 Close done 2018-01-05T11:29:10,552 DEBUG [TezTR-518813_1_41_0_0_0] exec.TableScanOperator: 2 Close done 2018-01-05T11:29:10,552 DEBUG [TezTR-518813_1_41_0_0_0] vector.VectorMapOperator: 0 Close done 2018-01-05T11:29:10,552 INFO [TezTR-518813_1_41_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_41_00_000000_0 2018-01-05T11:29:10,552 INFO [TezTR-518813_1_41_0_0_0] impl.PipelinedSorter: Reducer 2: Starting flush of map output 2018-01-05T11:29:10,552 INFO [TezTR-518813_1_41_0_0_0] impl.PipelinedSorter: Reducer 2: Span0.length = 500, perItem = 6 2018-01-05T11:29:10,553 INFO [TezTR-518813_1_41_0_0_0] impl.PipelinedSorter: Reducer 2: done sorting span=0, length=500, time=1 2018-01-05T11:29:10,553 INFO [TezTR-518813_1_41_0_0_0] impl.PipelinedSorter: Reducer 2: Heap = SpanIterator<0:499> (span=Span[5242880,3000]), 2018-01-05T11:29:10,555 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_41_01_000000_0, creationTime=1515180550473, allocationTime=1515180550474, startTime=1515180550476, finishTime=1515180550555, timeTaken=79, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:10,556 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_41_01_000000_0 2018-01-05T11:29:10,556 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_41_01_000000, startTime=1515180550476, finishTime=1515180550556, timeTaken=80, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_41_01_000000_0, diagnostics=, counters=Counters: 32, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=64, FILE_BYTES_WRITTEN=1483, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=242, INPUT_RECORDS_PROCESSED=2, INPUT_SPLIT_LENGTH_BYTES=1149, OUTPUT_RECORDS=242, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=1452, OUTPUT_BYTES_WITH_OVERHEAD=1447, OUTPUT_BYTES_PHYSICAL=1451, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=242, RECORDS_OUT_INTERMEDIATE_Map_1=242, org.apache.hadoop.hive.llap.counters.LlapIOCounters, CACHE_HIT_BYTES=361, CACHE_MISS_BYTES=0, CONSUMER_TIME_NS=850037, DECODE_TIME_NS=396159, HDFS_TIME_NS=436723, METADATA_CACHE_HIT=4, NUM_DECODED_BATCHES=2, NUM_VECTOR_BATCHES=2, ROWS_EMITTED=242, SELECTED_ROWGROUPS=2, TOTAL_IO_TIME_NS=2688443 2018-01-05T11:29:10,556 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_41_01 [Map 1], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:10,556 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_41_01 [Map 1] 2018-01-05T11:29:10,557 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_41_01, initRequestedTime=1515180550433, initedTime=1515180550472, startRequestedTime=1515180550448, startedTime=1515180550473, finishTime=1515180550556, timeTaken=83, status=SUCCEEDED, diagnostics=, counters=Counters: 32, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=64, FILE_BYTES_WRITTEN=1483, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=242, INPUT_RECORDS_PROCESSED=2, INPUT_SPLIT_LENGTH_BYTES=1149, OUTPUT_RECORDS=242, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=1452, OUTPUT_BYTES_WITH_OVERHEAD=1447, OUTPUT_BYTES_PHYSICAL=1451, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=242, RECORDS_OUT_INTERMEDIATE_Map_1=242, org.apache.hadoop.hive.llap.counters.LlapIOCounters, CACHE_HIT_BYTES=361, CACHE_MISS_BYTES=0, CONSUMER_TIME_NS=850037, DECODE_TIME_NS=396159, HDFS_TIME_NS=436723, METADATA_CACHE_HIT=4, NUM_DECODED_BATCHES=2, NUM_VECTOR_BATCHES=2, ROWS_EMITTED=242, SELECTED_ROWGROUPS=2, TOTAL_IO_TIME_NS=2688443, vertexStats=firstTaskStartTime=1515180550476, firstTasksToStart=[ task_1515180518813_0001_41_01_000000 ], lastTaskFinishTime=1515180550555, lastTasksToFinish=[ task_1515180518813_0001_41_01_000000 ], minTaskDuration=79, maxTaskDuration=79, avgTaskDuration=79.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_41_01_000000 ], longestDurationTasks=[ task_1515180518813_0001_41_01_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:10,557 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_41_01 [Map 1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:10,557 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_41_02 [Reducer 2] attempt: attempt_1515180518813_0001_41_01_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:29:10,557 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_41_01 [Map 1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=4 2018-01-05T11:29:10,557 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=4, commitInProgress=0, terminationCause=null 2018-01-05T11:29:10,557 DEBUG [Task-Executor-2] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:29:10,557 INFO [Task-Executor-2] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_41_01_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:10,557 INFO [Task-Executor-2] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_41_01_000000_0 2018-01-05T11:29:10,557 INFO [Task-Executor-2] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000101=68 2018-01-05T11:29:10,557 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_41_01_000000_0: true 2018-01-05T11:29:10,558 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_41_01_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:10,558 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_41_01_000000_0; false 2018-01-05T11:29:10,558 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_41_01_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:10,562 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_41_01_000000_0 2018-01-05T11:29:10,562 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000101, QueryId=djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca-41, Vertex=Map 1, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:10,562 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_41_01_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=3, preemptionQueueSize=1 2018-01-05T11:29:10,562 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_41_01_000000_0 2018-01-05T11:29:10,562 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:10,562 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:10,563 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000101, QueryId=djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca-41, DagId=41, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-2, Succeeded=true, StartTime=1515180550489, EndTime=1515180550563 2018-01-05T11:29:10,563 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000101, stoppedTime=1515180550563, exitStatus=0 2018-01-05T11:29:10,563 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000101 exited with diagnostics set to Container completed. 2018-01-05T11:29:10,563 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000101 2018-01-05T11:29:10,585 INFO [TezTR-518813_1_41_0_0_0] impl.PipelinedSorter: Reducer 2: Spilling to /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/41/output/attempt_1515180518813_0001_41_00_000000_0_10313_0/file.out 2018-01-05T11:29:10,614 DEBUG [TezTR-518813_1_41_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:10,614 INFO [TezTR-518813_1_41_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:10,614 INFO [TezTR-518813_1_41_0_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_41_00_000000_0, askedToStop=false 2018-01-05T11:29:10,614 INFO [TezTR-518813_1_41_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_41_00_000000_0, stopRequested=false 2018-01-05T11:29:10,615 INFO [TezTR-518813_1_41_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_41_00_000000_0: Counters: 31 [[File System Counters FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=0, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=500, INPUT_RECORDS_PROCESSED=4, INPUT_SPLIT_LENGTH_BYTES=2442, OUTPUT_RECORDS=500, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=3000, OUTPUT_BYTES_WITH_OVERHEAD=2941, OUTPUT_BYTES_PHYSICAL=2945, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_4=500, RECORDS_OUT_INTERMEDIATE_Map_4=500][org.apache.hadoop.hive.llap.counters.LlapIOCounters CACHE_HIT_BYTES=765, CACHE_MISS_BYTES=0, CONSUMER_TIME_NS=1527152, DECODE_TIME_NS=852022, HDFS_TIME_NS=690053, METADATA_CACHE_HIT=8, NUM_DECODED_BATCHES=4, NUM_VECTOR_BATCHES=4, ROWS_EMITTED=500, SELECTED_ROWGROUPS=4, TOTAL_IO_TIME_NS=7553551]] 2018-01-05T11:29:10,615 DEBUG [TezTR-518813_1_41_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_41_0_0_0 thread id: 2857 scheme: file bytesRead: 64 bytesWritten: 2977 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:10,615 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_41_00_000000_0, isTaskDone=true 2018-01-05T11:29:10,615 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000102, requestId=2, startIndex=0, preRoutedStartIndex=1, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_41_00_000000_0, eventCount=4 } 2018-01-05T11:29:10,616 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 4, taskAttemptId=attempt_1515180518813_0001_41_00_000000_0, creationTime=1515180550516, allocationTime=1515180550518, startTime=1515180550518, finishTime=1515180550616, timeTaken=98, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:10,616 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_41_00_000000_0 2018-01-05T11:29:10,618 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:TASK_FINISHED]: vertexName=Map 4, taskId=task_1515180518813_0001_41_00_000000, startTime=1515180550518, finishTime=1515180550618, timeTaken=100, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_41_00_000000_0, diagnostics=, counters=Counters: 32, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=64, FILE_BYTES_WRITTEN=2977, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=500, INPUT_RECORDS_PROCESSED=4, INPUT_SPLIT_LENGTH_BYTES=2442, OUTPUT_RECORDS=500, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=3000, OUTPUT_BYTES_WITH_OVERHEAD=2941, OUTPUT_BYTES_PHYSICAL=2945, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_4=500, RECORDS_OUT_INTERMEDIATE_Map_4=500, org.apache.hadoop.hive.llap.counters.LlapIOCounters, CACHE_HIT_BYTES=765, CACHE_MISS_BYTES=0, CONSUMER_TIME_NS=1527152, DECODE_TIME_NS=852022, HDFS_TIME_NS=690053, METADATA_CACHE_HIT=8, NUM_DECODED_BATCHES=4, NUM_VECTOR_BATCHES=4, ROWS_EMITTED=500, SELECTED_ROWGROUPS=4, TOTAL_IO_TIME_NS=7553551 2018-01-05T11:29:10,618 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_41_00 [Map 4], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:10,618 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_41_00 [Map 4] 2018-01-05T11:29:10,618 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:VERTEX_FINISHED]: vertexName=Map 4, vertexId=vertex_1515180518813_0001_41_00, initRequestedTime=1515180550440, initedTime=1515180550515, startRequestedTime=1515180550448, startedTime=1515180550516, finishTime=1515180550618, timeTaken=102, status=SUCCEEDED, diagnostics=, counters=Counters: 32, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=64, FILE_BYTES_WRITTEN=2977, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=500, INPUT_RECORDS_PROCESSED=4, INPUT_SPLIT_LENGTH_BYTES=2442, OUTPUT_RECORDS=500, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=3000, OUTPUT_BYTES_WITH_OVERHEAD=2941, OUTPUT_BYTES_PHYSICAL=2945, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=1, HIVE, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_4=500, RECORDS_OUT_INTERMEDIATE_Map_4=500, org.apache.hadoop.hive.llap.counters.LlapIOCounters, CACHE_HIT_BYTES=765, CACHE_MISS_BYTES=0, CONSUMER_TIME_NS=1527152, DECODE_TIME_NS=852022, HDFS_TIME_NS=690053, METADATA_CACHE_HIT=8, NUM_DECODED_BATCHES=4, NUM_VECTOR_BATCHES=4, ROWS_EMITTED=500, SELECTED_ROWGROUPS=4, TOTAL_IO_TIME_NS=7553551, vertexStats=firstTaskStartTime=1515180550518, firstTasksToStart=[ task_1515180518813_0001_41_00_000000 ], lastTaskFinishTime=1515180550616, lastTasksToFinish=[ task_1515180518813_0001_41_00_000000 ], minTaskDuration=98, maxTaskDuration=98, avgTaskDuration=98.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_41_00_000000 ], longestDurationTasks=[ task_1515180518813_0001_41_00_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:10,619 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_41_00 [Map 4] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:10,619 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_41_02 [Reducer 2] attempt: attempt_1515180518813_0001_41_00_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:29:10,619 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_41_00 [Map 4] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=4 2018-01-05T11:29:10,619 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=4, commitInProgress=0, terminationCause=null 2018-01-05T11:29:10,619 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=0 } 2018-01-05T11:29:10,619 INFO [Task-Executor-1] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_41_00_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:10,619 INFO [Task-Executor-1] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_41_00_000000_0 2018-01-05T11:29:10,619 INFO [Task-Executor-1] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000102=99 2018-01-05T11:29:10,619 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_41_00_000000_0: true 2018-01-05T11:29:10,618 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_41_00_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:10,619 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_41_00_000000_0; false 2018-01-05T11:29:10,619 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_41_00_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:10,620 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_41_00_000000_0 2018-01-05T11:29:10,620 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000102, QueryId=djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca-41, Vertex=Map 4, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:10,620 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_41_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:29:10,620 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_41_00_000000_0 2018-01-05T11:29:10,620 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:10,620 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:10,620 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: All source tasks completed. Ramping up 1 remaining tasks for vertex: Reducer 2 2018-01-05T11:29:10,620 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: Scheduling 1 tasks for vertex: Reducer 2 with totalTasks: 1. 2 source tasks completed out of 2. MinSourceTaskCompletedFraction: 1.0 min: 0.05 max: 0.75 2018-01-05T11:29:10,620 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000102, stoppedTime=1515180550620, exitStatus=0 2018-01-05T11:29:10,620 INFO [App Shared Pool - #2] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Map 1, destinationVertexName=Reducer 2. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:29:10,620 INFO [App Shared Pool - #2] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Map 4, destinationVertexName=Reducer 2. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.OrderedGroupedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:29:10,620 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000102 exited with diagnostics set to Container completed. 2018-01-05T11:29:10,620 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000102 2018-01-05T11:29:10,620 INFO [App Shared Pool - #2] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_41_02 [Reducer 2] 2018-01-05T11:29:10,621 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:TASK_STARTED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_41_02_000000, scheduledTime=1515180550620, launchTime=1515180550620 2018-01-05T11:29:10,621 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_41_02 [Reducer 2] 2018-01-05T11:29:10,621 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_41_02_000000_0, priority=29, capability=, hosts=null 2018-01-05T11:29:10,621 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=29,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:10,621 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=29,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:10,621 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:10,621 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_41_02_000000_0 on hosts=null 2018-01-05T11:29:10,621 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_41_02_000000_0 on hosts=null 2018-01-05T11:29:10,621 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_41_02_000000_0 2018-01-05T11:29:10,621 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:10,621 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #103, task=TaskInfo{task=attempt_1515180518813_0001_41_02_000000_0, priority=29, startTime=0, containerId=null, uniqueId=102, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000103 2018-01-05T11:29:10,621 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_41_02_000000_0; false 2018-01-05T11:29:10,621 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_41_02_000000_0, priority=29, startTime=311571131, containerId=container_111101111_0001_00_000103assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=102, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:10,621 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000103 for node: 10.22.8.182:49818 2018-01-05T11:29:10,622 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000103, attempt_1515180518813_0001_41_02_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:10,622 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000103 for node: 10.22.8.182:49818 2018-01-05T11:29:10,622 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000103 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:10,622 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000103, launchTime=1515180550622 2018-01-05T11:29:10,622 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=41}, sourceName=Map 1, state=S_SUCCEEDED 2018-01-05T11:29:10,623 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=41}, sourceName=Map 4, state=S_SUCCEEDED 2018-01-05T11:29:10,623 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_41_02_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000103] on NM: [10.22.8.182:49818] 2018-01-05T11:29:10,623 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:10,623 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:10,623 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_41_02_000000_0, startTime=1515180550622, containerId=container_111101111_0001_00_000103, nodeId=10.22.8.182:49818 2018-01-05T11:29:10,624 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_41_02_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000103, dagName=select count(*) from tab a full oute...b.key (Stage-1), vertexName=Reducer 2, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=2, numOutputs=1, numGroupedInputs=0, Inputs={{Map 1,org.apache.tez.runtime.library.input.OrderedGroupedKVInput,1}{Map 4,org.apache.tez.runtime.library.input.OrderedGroupedKVInput,1}}, Outputs={{Reducer 3,org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput,1}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=3, completedTaskCount=2, dagStartTime=1515180550433, firstAttemptStartTime=1515180550621, currentAttemptStartTime=1515180550622} 2018-01-05T11:29:10,624 INFO [IPC Server handler 1 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000103, QueryId=djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca-41, DagId=41, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, SubmitTime=1515180550624 2018-01-05T11:29:10,624 DEBUG [IPC Server handler 1 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=41} with the ShuffleHandler 2018-01-05T11:29:10,624 DEBUG [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/41] 2018-01-05T11:29:10,624 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:29:10,624 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_41_02_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:10,624 DEBUG [IPC Server handler 1 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_41_02_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180550621, dagStartTime=1515180550433, withinDagPriority=29, vertexParallelism= 1, selfAndUpstreamParallelism= 3, selfAndUpstreamComplete= 2}] 2018-01-05T11:29:10,624 INFO [IPC Server handler 1 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_41_02_000000_0 : ACCEPTED 2018-01-05T11:29:10,624 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_41_02_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:29:10,624 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_41_02_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180550621, dagStartTime=1515180550433, withinDagPriority=29, vertexParallelism= 1, selfAndUpstreamParallelism= 3, selfAndUpstreamComplete= 2} 2018-01-05T11:29:10,624 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_41_02_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:10,624 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_41_02_000000_0 to pre-emption queue 2018-01-05T11:29:10,624 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_41_02_000000_0: true 2018-01-05T11:29:10,625 INFO [TaskCommunicator # 1] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_41_02_000000_0 2018-01-05T11:29:10,625 INFO [TaskCommunicator # 1] GuaranteedTasks: Received response for attempt_1515180518813_0001_41_02_000000_0, true 2018-01-05T11:29:10,625 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: taskOwner hashCode:981898900 2018-01-05T11:29:10,625 INFO [Task-Executor-1] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : select count(*) from tab a full oute...b.key (Stage-1), VertexName: Reducer 2, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_41_02_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=Map 4, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=Reducer 3, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput }}, ] 2018-01-05T11:29:10,625 INFO [Task-Executor-1] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=2, numOutputs=1, JVM.maxFree=1023410176, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:10,625 INFO [Task-Executor-1] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_41_02_000000_0 vertexName: Reducer 2 2018-01-05T11:29:10,625 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000103, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_41_02_000000_0, eventCount=1 } 2018-01-05T11:29:10,626 DEBUG [TezTR-518813_1_41_2_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_41 queryId: djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca instanceType: TaskRunner2Callable 2018-01-05T11:29:10,626 INFO [TezTR-518813_1_41_2_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_41_02_000000_0 2018-01-05T11:29:10,626 DEBUG [TezTR-518813_1_41_2_0_0] log.PerfLogger: 2018-01-05T11:29:10,626 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:10,626 INFO [I/O Setup 0 Initialize: {Map 1}] orderedgrouped.MergeManager: Initial Memory required for SHUFFLE_BUFFER=131404920 based on INPUT_BUFFER_FACTOR=0.7, for final merged output=0, using factor: 0.0 2018-01-05T11:29:10,627 INFO [I/O Setup 1 Initialize: {Map 4}] orderedgrouped.MergeManager: Initial Memory required for SHUFFLE_BUFFER=131404920 based on INPUT_BUFFER_FACTOR=0.7, for final merged output=0, using factor: 0.0 2018-01-05T11:29:10,627 INFO [I/O Setup 2 Initialize: {Reducer 3}] writers.UnorderedPartitionedKVWriter: Requested BufferSize (tez.runtime.unordered.output.buffer.size-mb) : 100 2018-01-05T11:29:10,627 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:10,628 INFO [IPC Server handler 21 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_41_02_000000_0 2 events [0,2) total 2 vertex_1515180518813_0001_41_02 [Reducer 2] 2018-01-05T11:29:10,628 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=2, nextPreRoutedEventId=0, eventCount=2 } 2018-01-05T11:29:10,628 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_41_02_000000_0, eventCount=2 fromEventId=0 nextFromEventId=2 2018-01-05T11:29:10,630 DEBUG [TezTR-518813_1_41_2_0_0] log.PerfLogger: 2018-01-05T11:29:10,630 INFO [TezTR-518813_1_41_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:10,630 INFO [TezTR-518813_1_41_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 3 initializers to finish 2018-01-05T11:29:10,630 INFO [TezTR-518813_1_41_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:10,630 INFO [TezTR-518813_1_41_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:10,630 INFO [TezTR-518813_1_41_2_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:10,630 INFO [TezTR-518813_1_41_2_0_0] resources.MemoryDistributor: InitialRequests=[Map 1:INPUT:131404920:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [Map 4:INPUT:131404920:org.apache.tez.runtime.library.input.OrderedGroupedKVInput], [Reducer 3:OUTPUT:104857600:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput] 2018-01-05T11:29:10,630 INFO [TezTR-518813_1_41_2_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:10,630 INFO [TezTR-518813_1_41_2_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.045, finalReserveFractionUsed=0.345 2018-01-05T11:29:10,630 INFO [TezTR-518813_1_41_2_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 3, numScaledRequests: 25, TotalRequested: 367667440, TotalRequestedScaled: 1.3034302719999999E8, TotalJVMHeap: 1023410176, TotalAvailable: 670333665, TotalRequested/TotalJVMHeap:0.36 2018-01-05T11:29:10,630 INFO [TezTR-518813_1_41_2_0_0] resources.MemoryDistributor: Allocations=[Map 1:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:131404920:131404920], [Map 4:org.apache.tez.runtime.library.input.OrderedGroupedKVInput:INPUT:131404920:131404920], [Reducer 3:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput:OUTPUT:104857600:21570645] 2018-01-05T11:29:10,630 INFO [TezTR-518813_1_41_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:10,630 INFO [TezTR-518813_1_41_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: Map 1 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:10,630 INFO [TezTR-518813_1_41_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: Map 4 being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:10,630 INFO [TezTR-518813_1_41_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 2 2018-01-05T11:29:10,630 INFO [TezTR-518813_1_41_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 IOs to start 2018-01-05T11:29:10,631 DEBUG [I/O Setup 0 Start: {Map 1}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:10,631 DEBUG [I/O Setup 1 Start: {Map 4}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:10,631 DEBUG [I/O Setup 0 Start: {Map 1}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:10,631 DEBUG [I/O Setup 1 Start: {Map 4}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:10,631 DEBUG [I/O Setup 0 Start: {Map 1}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:10,631 DEBUG [I/O Setup 1 Start: {Map 4}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:10,631 DEBUG [I/O Setup 0 Start: {Map 1}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:10,631 DEBUG [I/O Setup 1 Start: {Map 4}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:10,631 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.Shuffle: Map_1: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true 2018-01-05T11:29:10,631 INFO [I/O Setup 1 Start: {Map 4}] orderedgrouped.Shuffle: Map_4: Shuffle assigned with 1 inputs, codec: None, ifileReadAhead: true 2018-01-05T11:29:10,631 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.MergeManager: Map 1: MergerManager: memoryLimit=131404920, maxSingleShuffleLimit=32851230, mergeThreshold=86727248, ioSortFactor=10, postMergeMem=0, memToMemMergeOutputsThreshold=10 2018-01-05T11:29:10,631 INFO [I/O Setup 1 Start: {Map 4}] orderedgrouped.MergeManager: Map 4: MergerManager: memoryLimit=131404920, maxSingleShuffleLimit=32851230, mergeThreshold=86727248, ioSortFactor=10, postMergeMem=0, memToMemMergeOutputsThreshold=10 2018-01-05T11:29:10,632 INFO [I/O Setup 0 Start: {Map 1}] orderedgrouped.ShuffleScheduler: ShuffleScheduler running for sourceVertex: Map 1 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true 2018-01-05T11:29:10,632 INFO [I/O Setup 0 Start: {Map 1}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: Map 1 2018-01-05T11:29:10,632 INFO [TezTR-518813_1_41_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:10,633 INFO [I/O Setup 1 Start: {Map 4}] orderedgrouped.ShuffleScheduler: ShuffleScheduler running for sourceVertex: Map 4 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=15, maxTaskOutputAtOnce=20, numFetchers=1, hostFailureFraction=0.2, minFailurePerHost=4, maxAllowedFailedFetchFraction=0.5, maxStallTimeFraction=0.5, minReqProgressFraction=0.5, checkFailedFetchSinceLastCompletion=true 2018-01-05T11:29:10,635 INFO [I/O Setup 1 Start: {Map 4}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: Map 4 2018-01-05T11:29:10,635 INFO [TezTR-518813_1_41_2_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:10,635 INFO [TezTR-518813_1_41_2_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_41_02_000000_0 2018-01-05T11:29:10,635 DEBUG [TezTR-518813_1_41_2_0_0] log.PerfLogger: 2018-01-05T11:29:10,635 DEBUG [TezTR-518813_1_41_2_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_41_02_000000_0_10314 2018-01-05T11:29:10,635 DEBUG [TezTR-518813_1_41_2_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:10,635 DEBUG [TezTR-518813_1_41_2_0_0] mr.ObjectCache: Creating djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca_Reducer 2__REDUCE_PLAN__ 2018-01-05T11:29:10,636 INFO [TezTR-518813_1_41_2_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/e9021bae-7b3b-4529-8ca0-a45e46516d5c/reduce.xml 2018-01-05T11:29:10,636 DEBUG [TezTR-518813_1_41_2_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/e9021bae-7b3b-4529-8ca0-a45e46516d5c/reduce.xml 2018-01-05T11:29:10,636 DEBUG [TezTR-518813_1_41_2_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/e9021bae-7b3b-4529-8ca0-a45e46516d5c/reduce.xml 2018-01-05T11:29:10,636 INFO [TezTaskEventRouter{attempt_1515180518813_0001_41_02_000000_0}] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:29:10,636 DEBUG [TezTR-518813_1_41_2_0_0] log.PerfLogger: 2018-01-05T11:29:10,636 INFO [TezTR-518813_1_41_2_0_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:29:10,636 DEBUG [TezTR-518813_1_41_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:10,636 INFO [TezTaskEventRouter{attempt_1515180518813_0001_41_02_000000_0}] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 4: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:29:10,636 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=41&reduce=0&map=attempt_1515180518813_0001_41_01_000000_0_10310 mapId: [attempt_1515180518813_0001_41_01_000000_0_10310] reduceId: [0] jobId: [job_1515180518813_0001] dagId: [41] keepAlive: false 2018-01-05T11:29:10,637 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=41&reduce=0&map=attempt_1515180518813_0001_41_01_000000_0_10310; hash=...0n8ASqywJEeoU 2018-01-05T11:29:10,637 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=41&reduce=0&map=attempt_1515180518813_0001_41_01_000000_0_10310;reply=a2pbxvMrZg8qs 2018-01-05T11:29:10,637 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=41, user='djaiswal', attemptId='attempt_1515180518813_0001_41_01_000000_0_10310'} via loader 2018-01-05T11:29:10,637 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=41, user='djaiswal', attemptId='attempt_1515180518813_0001_41_01_000000_0_10310'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:29:10,637 DEBUG [New I/O worker #1] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_41_01_000000_0_10310 not found 2018-01-05T11:29:10,637 DEBUG [TezTR-518813_1_41_2_0_0] log.PerfLogger: 2018-01-05T11:29:10,637 INFO [TezTR-518813_1_41_2_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 2 size: 1.49KB 2018-01-05T11:29:10,637 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_41_01_000000_0_10310,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/41/output/attempt_1515180518813_0001_41_01_000000_0_10310/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/41/output/attempt_1515180518813_0001_41_01_000000_0_10310/file.out.index 2018-01-05T11:29:10,637 DEBUG [New I/O worker #1] shufflehandler.ShuffleHandler: Content Length in shuffle : 1506 2018-01-05T11:29:10,637 DEBUG [TezTR-518813_1_41_2_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:10,637 DEBUG [TezTR-518813_1_41_2_0_0] log.PerfLogger: 2018-01-05T11:29:10,637 DEBUG [TezTR-518813_1_41_2_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:10,637 DEBUG [TezTR-518813_1_41_2_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:10,638 DEBUG [TezTR-518813_1_41_2_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:10,638 INFO [Fetcher_O {Map_1} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=41&reduce=0&map=attempt_1515180518813_0001_41_01_000000_0_10310 sent hash and receievd reply 0 ms 2018-01-05T11:29:10,639 INFO [TezTR-518813_1_41_2_0_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:29:10,639 INFO [Fetcher_O {Map_1} #0] orderedgrouped.MergeManager: CloseInMemoryFile. Current state: inMemoryMapOutputs.size=1, commitMemory=0, usedMemory=1447. Since last log: count=1, min=1447, max=1447, total=1447, avg=1447.0 2018-01-05T11:29:10,639 INFO [Fetcher_O {Map_1} #0] ShuffleScheduler.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_41_01_000000_0_10310} to MEMORY, csize=1451, dsize=1447, EndTime=1515180550639, TimeTaken=0, Rate=0.00 MB/s 2018-01-05T11:29:10,639 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=41&reduce=0&map=attempt_1515180518813_0001_41_00_000000_0_10313 mapId: [attempt_1515180518813_0001_41_00_000000_0_10313] reduceId: [0] jobId: [job_1515180518813_0001] dagId: [41] keepAlive: false 2018-01-05T11:29:10,639 INFO [Fetcher_O {Map_1} #0] orderedgrouped.ShuffleScheduler: All inputs fetched for input vertex : Map 1 2018-01-05T11:29:10,639 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=41&reduce=0&map=attempt_1515180518813_0001_41_00_000000_0_10313; hash=...XEV3hckMKWp6Y 2018-01-05T11:29:10,639 INFO [Fetcher_O {Map_1} #0] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:10,639 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=41&reduce=0&map=attempt_1515180518813_0001_41_00_000000_0_10313;reply=W0e87Rt3FZdYA 2018-01-05T11:29:10,639 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down FetchScheduler for input: Map_1, wasInterrupted=false 2018-01-05T11:29:10,639 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:10,640 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.ShuffleScheduler: Shutting down fetchers for input: Map_1, shutdown timetaken: 0 ms, hasFetcherExecutorStopped: true 2018-01-05T11:29:10,640 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: TotalInMemFetchStats: count=1, totalSize=1447, min=1447, max=1447, avg=1.0 2018-01-05T11:29:10,640 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.MergeManager: finalMerge with #inMemoryOutputs=1, size=1447 and #onDiskOutputs=0, size=0 2018-01-05T11:29:10,640 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=41, user='djaiswal', attemptId='attempt_1515180518813_0001_41_00_000000_0_10313'} via loader 2018-01-05T11:29:10,640 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=41, user='djaiswal', attemptId='attempt_1515180518813_0001_41_00_000000_0_10313'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:29:10,640 DEBUG [New I/O worker #3] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_41_00_000000_0_10313 not found 2018-01-05T11:29:10,640 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_41_00_000000_0_10313,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/41/output/attempt_1515180518813_0001_41_00_000000_0_10313/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/41/output/attempt_1515180518813_0001_41_00_000000_0_10313/file.out.index 2018-01-05T11:29:10,640 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Content Length in shuffle : 3000 2018-01-05T11:29:10,640 INFO [Fetcher_O {Map_4} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=41&reduce=0&map=attempt_1515180518813_0001_41_00_000000_0_10313 sent hash and receievd reply 0 ms 2018-01-05T11:29:10,641 INFO [Fetcher_O {Map_4} #0] orderedgrouped.MergeManager: CloseInMemoryFile. Current state: inMemoryMapOutputs.size=1, commitMemory=0, usedMemory=2941. Since last log: count=1, min=2941, max=2941, total=2941, avg=2941.0 2018-01-05T11:29:10,641 INFO [Fetcher_O {Map_4} #0] ShuffleScheduler.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_41_00_000000_0_10313} to MEMORY, csize=2945, dsize=2941, EndTime=1515180550641, TimeTaken=1, Rate=2.80 MB/s 2018-01-05T11:29:10,641 INFO [Fetcher_O {Map_4} #0] orderedgrouped.ShuffleScheduler: All inputs fetched for input vertex : Map 4 2018-01-05T11:29:10,641 INFO [Fetcher_O {Map_4} #0] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:10,641 INFO [ShuffleAndMergeRunner {Map_4}] orderedgrouped.ShuffleScheduler: Shutting down FetchScheduler for input: Map_4, wasInterrupted=false 2018-01-05T11:29:10,641 INFO [ShuffleAndMergeRunner {Map_4}] orderedgrouped.ShuffleScheduler: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:10,641 INFO [ShuffleAndMergeRunner {Map_4}] orderedgrouped.ShuffleScheduler: Shutting down fetchers for input: Map_4, shutdown timetaken: 0 ms, hasFetcherExecutorStopped: true 2018-01-05T11:29:10,641 INFO [ShuffleAndMergeRunner {Map_4}] orderedgrouped.MergeManager: TotalInMemFetchStats: count=1, totalSize=2941, min=2941, max=2941, avg=1.0 2018-01-05T11:29:10,641 INFO [ShuffleAndMergeRunner {Map_4}] orderedgrouped.MergeManager: finalMerge with #inMemoryOutputs=1, size=2941 and #onDiskOutputs=0, size=0 2018-01-05T11:29:10,665 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,665 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: Status: Running (Executing on YARN cluster with App id application_1515180518813_0001) 2018-01-05T11:29:10,665 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: merge complete for input vertex : Map_1 2018-01-05T11:29:10,665 INFO [ShuffleAndMergeRunner {Map_1}] orderedgrouped.Shuffle: Map_1: Shuffle Runner thread complete 2018-01-05T11:29:10,667 INFO [ShuffleAndMergeRunner {Map_4}] orderedgrouped.Shuffle: merge complete for input vertex : Map_4 2018-01-05T11:29:10,667 INFO [TezTR-518813_1_41_2_0_0] tez.ReduceRecordProcessor: Main work is Reducer 2 2018-01-05T11:29:10,667 INFO [ShuffleAndMergeRunner {Map_4}] orderedgrouped.Shuffle: Map_4: Shuffle Runner thread complete 2018-01-05T11:29:10,667 INFO [TezTR-518813_1_41_2_0_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:10,667 DEBUG [TezTR-518813_1_41_2_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:10,667 INFO [TezTR-518813_1_41_2_0_0] input.OrderedGroupedKVInput: Map 1: creating ValuesIterator with comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator, keyClass=org.apache.hadoop.hive.ql.io.HiveKey, valClass=org.apache.hadoop.io.BytesWritable 2018-01-05T11:29:10,667 DEBUG [TezTR-518813_1_41_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:10,667 DEBUG [TezTR-518813_1_41_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:10,667 DEBUG [TezTR-518813_1_41_2_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[] columnTypes=[] 2018-01-05T11:29:10,667 DEBUG [TezTR-518813_1_41_2_0_0] log.PerfLogger: 2018-01-05T11:29:10,667 INFO [TezTR-518813_1_41_2_0_0] input.OrderedGroupedKVInput: Map 4: creating ValuesIterator with comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator, keyClass=org.apache.hadoop.hive.ql.io.HiveKey, valClass=org.apache.hadoop.io.BytesWritable 2018-01-05T11:29:10,667 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,667 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,667 DEBUG [TezTR-518813_1_41_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:10,667 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,667 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,667 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,667 DEBUG [TezTR-518813_1_41_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:10,667 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 Map 4: 1/1 Reducer 2: 0(+1)/1 Reducer 3: 0/1 2018-01-05T11:29:10,667 DEBUG [TezTR-518813_1_41_2_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[] columnTypes=[] 2018-01-05T11:29:10,667 DEBUG [TezTR-518813_1_41_2_0_0] log.PerfLogger: 2018-01-05T11:29:10,667 INFO [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Initializing operator MERGEJOIN[13] 2018-01-05T11:29:10,667 INFO [TezTR-518813_1_41_2_0_0] exec.CommonJoinOperator: JOIN struct<> totalsz = 0 2018-01-05T11:29:10,668 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Initialization Done 13 MERGEJOIN 2018-01-05T11:29:10,668 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Operator 13 MERGEJOIN initialized 2018-01-05T11:29:10,668 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Initializing children of 13 MERGEJOIN 2018-01-05T11:29:10,668 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Initializing child 8 GBY 2018-01-05T11:29:10,668 INFO [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Initializing operator GBY[8] 2018-01-05T11:29:10,668 DEBUG [TezTR-518813_1_41_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:10,668 INFO [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Max hash table memory: 93860656 bytes 2018-01-05T11:29:10,668 INFO [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: isTez: true isLlap: true numExecutors: 4 maxMemory: 187721312 2018-01-05T11:29:10,668 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Initialization Done 8 GBY 2018-01-05T11:29:10,668 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Operator 8 GBY initialized 2018-01-05T11:29:10,668 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Initializing children of 8 GBY 2018-01-05T11:29:10,668 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Initializing child 9 RS 2018-01-05T11:29:10,668 INFO [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Initializing operator RS[9] 2018-01-05T11:29:10,668 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: keys size is 0 2018-01-05T11:29:10,668 INFO [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Using tag = -1 2018-01-05T11:29:10,668 DEBUG [TezTR-518813_1_41_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:10,668 DEBUG [TezTR-518813_1_41_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:10,668 DEBUG [TezTR-518813_1_41_2_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0] columnTypes=[bigint] 2018-01-05T11:29:10,668 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Initialization Done 9 RS 2018-01-05T11:29:10,668 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Operator 9 RS initialized 2018-01-05T11:29:10,668 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Initialization Done 9 RS done is reset. 2018-01-05T11:29:10,668 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Initialization Done 8 GBY done is reset. 2018-01-05T11:29:10,668 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Initialization Done 13 MERGEJOIN done is reset. 2018-01-05T11:29:10,668 INFO [TezTR-518813_1_41_2_0_0] tez.ReduceRecordProcessor: Id =13 Id =8 Id =9 <\Children> Id = 8 null<\Parent> <\RS> <\Children> Id = 13 null<\Parent> <\GBY> <\Children> <\Parent> <\MERGEJOIN> 2018-01-05T11:29:10,668 INFO [TezTR-518813_1_41_2_0_0] exec.OperatorUtils: Setting output collector: RS[9] --> Reducer 3 2018-01-05T11:29:10,668 DEBUG [TezTR-518813_1_41_2_0_0] log.PerfLogger: 2018-01-05T11:29:10,668 INFO [TezTR-518813_1_41_2_0_0] tez.ReduceRecordProcessor: Starting Output: Reducer 3 2018-01-05T11:29:10,669 INFO [TezTR-518813_1_41_2_0_0] writers.BaseUnorderedPartitionedKVWriter: Instantiating Partitioner: [org.apache.tez.mapreduce.partition.MRPartitioner] 2018-01-05T11:29:10,677 DEBUG [AMReporterQueueDrainer] impl.AMReporter: Removing am 10.22.8.182:49824 with last associated dag QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=29} from heartbeat with taskCount=0, amFailed=false, isDone=true 2018-01-05T11:29:10,694 INFO [TezTR-518813_1_41_2_0_0] writers.UnorderedPartitionedKVWriter: Reducer_3: numBuffers=2, sizePerBuffer=10785320, skipBuffers=true, numPartitions=1, availableMemory=21570645, maxSingleBufferSizeBytes=2147483647, pipelinedShuffle=false, isFinalMergeEnabled=true, numPartitions=1, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:29:10,694 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,694 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,694 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,694 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,694 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,694 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,694 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,694 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,694 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,697 INFO [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,697 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,698 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,699 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,700 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,701 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,702 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,703 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,704 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,705 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,706 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,707 INFO [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,707 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,708 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,709 INFO [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,709 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,710 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,711 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,712 INFO [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,712 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,713 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,714 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,715 INFO [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: We are emitting rows since we hit the join emit interval of 2 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Ending group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Ending group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Ending group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: End group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: End group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: End group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Starting group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Starting group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Starting group for children: 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Start group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Start group Done 2018-01-05T11:29:10,715 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Start group Done 2018-01-05T11:29:10,716 DEBUG [TezTR-518813_1_41_2_0_0] log.PerfLogger: 2018-01-05T11:29:10,716 DEBUG [TezTR-518813_1_41_2_0_0] mr.ObjectCache: djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca_Reducer 2__REDUCE_PLAN__ no longer needed 2018-01-05T11:29:10,716 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: close called for operator MERGEJOIN[13] 2018-01-05T11:29:10,716 INFO [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Closing operator MERGEJOIN[13] 2018-01-05T11:29:10,716 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Cleaning up the operator state 2018-01-05T11:29:10,716 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: Closing child = GBY[8] 2018-01-05T11:29:10,716 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: close called for operator GBY[8] 2018-01-05T11:29:10,716 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:10,716 INFO [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Closing operator GBY[8] 2018-01-05T11:29:10,716 INFO [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Begin Hash Table flush: size = 1 2018-01-05T11:29:10,716 INFO [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: keys are [] num distributions: 0 2018-01-05T11:29:10,716 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: Closing child = RS[9] 2018-01-05T11:29:10,716 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: close called for operator RS[9] 2018-01-05T11:29:10,716 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:10,716 INFO [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: Closing operator RS[9] 2018-01-05T11:29:10,716 INFO [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: RECORDS_OUT_INTERMEDIATE_Reducer_2:1, 2018-01-05T11:29:10,716 DEBUG [TezTR-518813_1_41_2_0_0] exec.ReduceSinkOperator: 9 Close done 2018-01-05T11:29:10,716 DEBUG [TezTR-518813_1_41_2_0_0] exec.GroupByOperator: 8 Close done 2018-01-05T11:29:10,716 DEBUG [TezTR-518813_1_41_2_0_0] exec.CommonMergeJoinOperator: 13 Close done 2018-01-05T11:29:10,716 INFO [TezTR-518813_1_41_2_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_41_02_000000_0 2018-01-05T11:29:10,716 INFO [TezTR-518813_1_41_2_0_0] orderedgrouped.Shuffle: Shutting down Shuffle for source: Map_1 2018-01-05T11:29:10,716 INFO [TezTR-518813_1_41_2_0_0] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 1: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:29:10,716 INFO [TezTR-518813_1_41_2_0_0] orderedgrouped.Shuffle: Shutting down Shuffle for source: Map_4 2018-01-05T11:29:10,716 INFO [TezTR-518813_1_41_2_0_0] orderedgrouped.ShuffleInputEventHandlerOrderedGrouped: Map 4: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:29:10,716 INFO [TezTR-518813_1_41_2_0_0] writers.UnorderedPartitionedKVWriter: Reducer_3: Waiting for all spills to complete : Pending : 0 2018-01-05T11:29:10,716 INFO [TezTR-518813_1_41_2_0_0] writers.UnorderedPartitionedKVWriter: Reducer_3: All spills complete 2018-01-05T11:29:10,723 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000102, QueryId=djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca-41, DagId=41, VertexName=Map 4, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-1, Succeeded=true, StartTime=1515180550519, EndTime=1515180550723 2018-01-05T11:29:10,733 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000103, requestId=2, startIndex=2, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_41_02_000000_0, eventCount=1 } 2018-01-05T11:29:10,734 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=2, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:10,735 DEBUG [TezTR-518813_1_41_2_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:10,735 INFO [TezTR-518813_1_41_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:10,735 INFO [TezTR-518813_1_41_2_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_41_02_000000_0, askedToStop=false 2018-01-05T11:29:10,735 INFO [TezTR-518813_1_41_2_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_41_02_000000_0, stopRequested=false 2018-01-05T11:29:10,735 INFO [TezTR-518813_1_41_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_41_02_000000_0: Counters: 34 [[org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=462, REDUCE_INPUT_RECORDS=742, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=742, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=5, OUTPUT_BYTES_WITH_OVERHEAD=13, OUTPUT_BYTES_PHYSICAL=41, ADDITIONAL_SPILLS_BYTES_WRITTEN=4396, ADDITIONAL_SPILLS_BYTES_READ=4396, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=4396, SHUFFLE_BYTES_DECOMPRESSED=4388, SHUFFLE_BYTES_TO_MEM=4396, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=19, MERGE_PHASE_TIME=70, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10][HIVE RECORDS_OUT_INTERMEDIATE_Reducer_2=1][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0]] 2018-01-05T11:29:10,735 DEBUG [TezTR-518813_1_41_2_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_41_2_0_0 thread id: 2868 scheme: file bytesRead: 0 bytesWritten: 49 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:10,735 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_41_02_000000_0, isTaskDone=true 2018-01-05T11:29:10,735 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000103, requestId=3, startIndex=2, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_41_02_000000_0, eventCount=4 } 2018-01-05T11:29:10,736 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 2, taskAttemptId=attempt_1515180518813_0001_41_02_000000_0, creationTime=1515180550621, allocationTime=1515180550621, startTime=1515180550622, finishTime=1515180550736, timeTaken=114, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:10,736 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_41_02_000000_0 2018-01-05T11:29:10,736 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=3, shouldDie=false, nextFromEventId=2, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:10,736 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_41_02_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:10,736 INFO [Task-Executor-1] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_41_02_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:10,736 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_41_02_000000_0; false 2018-01-05T11:29:10,736 INFO [Task-Executor-1] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_41_02_000000_0 2018-01-05T11:29:10,736 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_41_02_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:10,736 INFO [Task-Executor-1] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000103=111 2018-01-05T11:29:10,736 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_41_02_000000_0: true 2018-01-05T11:29:10,736 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_41_02_000000, startTime=1515180550622, finishTime=1515180550736, timeTaken=114, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_41_02_000000_0, diagnostics=, counters=Counters: 39, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=49, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=462, REDUCE_INPUT_RECORDS=742, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=742, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=5, OUTPUT_BYTES_WITH_OVERHEAD=13, OUTPUT_BYTES_PHYSICAL=41, ADDITIONAL_SPILLS_BYTES_WRITTEN=4396, ADDITIONAL_SPILLS_BYTES_READ=4396, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=4396, SHUFFLE_BYTES_DECOMPRESSED=4388, SHUFFLE_BYTES_TO_MEM=4396, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=19, MERGE_PHASE_TIME=70, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10, HIVE, RECORDS_OUT_INTERMEDIATE_Reducer_2=1, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0 2018-01-05T11:29:10,736 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:10,736 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_41_02_000000_0 2018-01-05T11:29:10,736 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:10,737 INFO [ExecutionCompletionThread #0] impl.TaskExecutorService: AppId=application_1515180518813_0001, containerId=container_111101111_0001_00_000103, QueryId=djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca-41, Vertex=Reducer 2, FragmentNum=0, Attempt=0 request SUCCESS! Removed from preemption list. 2018-01-05T11:29:10,737 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_41_02_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:29:10,737 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_41_02_000000_0 2018-01-05T11:29:10,737 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_41_02 [Reducer 2], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:10,737 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_41_02 [Reducer 2] 2018-01-05T11:29:10,737 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:VERTEX_FINISHED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_41_02, initRequestedTime=1515180550448, initedTime=1515180550454, startRequestedTime=1515180550516, startedTime=1515180550516, finishTime=1515180550737, timeTaken=221, status=SUCCEEDED, diagnostics=, counters=Counters: 39, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=49, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=462, REDUCE_INPUT_RECORDS=742, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=742, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, OUTPUT_RECORDS=1, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=5, OUTPUT_BYTES_WITH_OVERHEAD=13, OUTPUT_BYTES_PHYSICAL=41, ADDITIONAL_SPILLS_BYTES_WRITTEN=4396, ADDITIONAL_SPILLS_BYTES_READ=4396, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=4396, SHUFFLE_BYTES_DECOMPRESSED=4388, SHUFFLE_BYTES_TO_MEM=4396, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=19, MERGE_PHASE_TIME=70, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10, HIVE, RECORDS_OUT_INTERMEDIATE_Reducer_2=1, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, vertexStats=firstTaskStartTime=1515180550622, firstTasksToStart=[ task_1515180518813_0001_41_02_000000 ], lastTaskFinishTime=1515180550736, lastTasksToFinish=[ task_1515180518813_0001_41_02_000000 ], minTaskDuration=114, maxTaskDuration=114, avgTaskDuration=114.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_41_02_000000 ], longestDurationTasks=[ task_1515180518813_0001_41_02_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:10,737 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_41_02 [Reducer 2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:10,737 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000103, stoppedTime=1515180550737, exitStatus=0 2018-01-05T11:29:10,737 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000103 exited with diagnostics set to Container completed. 2018-01-05T11:29:10,737 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source task attempt completed for vertex: vertex_1515180518813_0001_41_03 [Reducer 3] attempt: attempt_1515180518813_0001_41_02_000000_0 with state: SUCCEEDED vertexState: RUNNING 2018-01-05T11:29:10,737 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_41_02 [Reducer 2] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=4 2018-01-05T11:29:10,737 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=4, commitInProgress=0, terminationCause=null 2018-01-05T11:29:10,737 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000103 2018-01-05T11:29:10,737 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: All source tasks completed. Ramping up 1 remaining tasks for vertex: Reducer 3 2018-01-05T11:29:10,737 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: Scheduling 1 tasks for vertex: Reducer 3 with totalTasks: 1. 1 source tasks completed out of 1. MinSourceTaskCompletedFraction: 1.0 min: 0.05 max: 0.75 2018-01-05T11:29:10,737 INFO [App Shared Pool - #2] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Reducer 2, destinationVertexName=Reducer 3. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.UnorderedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:29:10,737 INFO [App Shared Pool - #2] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_41_03 [Reducer 3] 2018-01-05T11:29:10,737 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:TASK_STARTED]: vertexName=Reducer 3, taskId=task_1515180518813_0001_41_03_000000, scheduledTime=1515180550737, launchTime=1515180550737 2018-01-05T11:29:10,737 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_41_03 [Reducer 3] 2018-01-05T11:29:10,738 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_41_03_000000_0, priority=44, capability=, hosts=null 2018-01-05T11:29:10,738 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=44,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:10,738 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=44,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:10,738 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:10,738 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_41_03_000000_0 on hosts=null 2018-01-05T11:29:10,738 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_41_03_000000_0 on hosts=null 2018-01-05T11:29:10,738 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_41_03_000000_0 2018-01-05T11:29:10,738 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:10,738 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #104, task=TaskInfo{task=attempt_1515180518813_0001_41_03_000000_0, priority=44, startTime=0, containerId=null, uniqueId=103, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000104 2018-01-05T11:29:10,738 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_41_03_000000_0; false 2018-01-05T11:29:10,738 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_41_03_000000_0, priority=44, startTime=311571247, containerId=container_111101111_0001_00_000104assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=103, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:10,738 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000104 for node: 10.22.8.182:49818 2018-01-05T11:29:10,738 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000104, attempt_1515180518813_0001_41_03_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:10,738 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000104 for node: 10.22.8.182:49818 2018-01-05T11:29:10,739 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000104 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:10,739 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Processing state update: queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=41}, sourceName=Reducer 2, state=S_SUCCEEDED 2018-01-05T11:29:10,739 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000104, launchTime=1515180550739 2018-01-05T11:29:10,739 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_41_03_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000104] on NM: [10.22.8.182:49818] 2018-01-05T11:29:10,739 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_41_03_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000104, dagName=select count(*) from tab a full oute...b.key (Stage-1), vertexName=Reducer 3, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Reducer 2,org.apache.tez.runtime.library.input.UnorderedKVInput,1}}, Outputs={{out_Reducer 3,org.apache.tez.mapreduce.output.MROutput,0}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=4, completedTaskCount=3, dagStartTime=1515180550433, firstAttemptStartTime=1515180550738, currentAttemptStartTime=1515180550738} 2018-01-05T11:29:10,739 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:10,739 INFO [IPC Server handler 2 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000104, QueryId=djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca-41, DagId=41, VertexName=Reducer 3, TaskId=0, TaskAttemptId=0, SubmitTime=1515180550739 2018-01-05T11:29:10,739 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:10,739 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 3, taskAttemptId=attempt_1515180518813_0001_41_03_000000_0, startTime=1515180550739, containerId=container_111101111_0001_00_000104, nodeId=10.22.8.182:49818 2018-01-05T11:29:10,739 DEBUG [IPC Server handler 2 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=41} with the ShuffleHandler 2018-01-05T11:29:10,739 DEBUG [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/41] 2018-01-05T11:29:10,740 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:29:10,740 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_41_03_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:10,740 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_41_03_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180550738, dagStartTime=1515180550433, withinDagPriority=44, vertexParallelism= 1, selfAndUpstreamParallelism= 4, selfAndUpstreamComplete= 3}] 2018-01-05T11:29:10,740 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_41_03_000000_0 : ACCEPTED 2018-01-05T11:29:10,740 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_41_03_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:29:10,740 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_41_03_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180550738, dagStartTime=1515180550433, withinDagPriority=44, vertexParallelism= 1, selfAndUpstreamParallelism= 4, selfAndUpstreamComplete= 3} 2018-01-05T11:29:10,740 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_41_03_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:10,740 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_41_03_000000_0 to pre-emption queue 2018-01-05T11:29:10,740 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_41_03_000000_0: true 2018-01-05T11:29:10,740 INFO [TaskCommunicator # 2] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_41_03_000000_0 2018-01-05T11:29:10,740 INFO [TaskCommunicator # 2] GuaranteedTasks: Received response for attempt_1515180518813_0001_41_03_000000_0, true 2018-01-05T11:29:10,740 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: taskOwner hashCode:992187299 2018-01-05T11:29:10,741 INFO [Task-Executor-1] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : select count(*) from tab a full oute...b.key (Stage-1), VertexName: Reducer 3, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_41_03_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Reducer 2, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }}, ], outputSpecList=[{{ destinationVertexName=out_Reducer 3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ] 2018-01-05T11:29:10,741 INFO [Task-Executor-1] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1023410176, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:10,741 INFO [Task-Executor-1] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_41_03_000000_0 vertexName: Reducer 3 2018-01-05T11:29:10,741 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000104, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_41_03_000000_0, eventCount=1 } 2018-01-05T11:29:10,741 DEBUG [TezTR-518813_1_41_3_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_41 queryId: djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca instanceType: TaskRunner2Callable 2018-01-05T11:29:10,741 INFO [TezTR-518813_1_41_3_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_41_03_000000_0 2018-01-05T11:29:10,741 DEBUG [TezTR-518813_1_41_3_0_0] log.PerfLogger: 2018-01-05T11:29:10,742 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:10,742 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:10,743 INFO [IPC Server handler 17 on 49824] impl.VertexImpl: Sending attempt_1515180518813_0001_41_03_000000_0 1 events [0,1) total 1 vertex_1515180518813_0001_41_03 [Reducer 3] 2018-01-05T11:29:10,743 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=1 } 2018-01-05T11:29:10,743 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_41_03_000000_0, eventCount=1 fromEventId=0 nextFromEventId=1 2018-01-05T11:29:10,745 DEBUG [TezTR-518813_1_41_3_0_0] log.PerfLogger: 2018-01-05T11:29:10,745 INFO [TezTR-518813_1_41_3_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:10,745 INFO [TezTR-518813_1_41_3_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:10,745 INFO [TezTR-518813_1_41_3_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:10,745 INFO [I/O Setup 1 Initialize: {out_Reducer 3}] output.MROutput: out_Reducer 3: outputFormat=org.apache.hadoop.hive.ql.io.HiveOutputFormatImpl, using newmapreduce API=false 2018-01-05T11:29:10,745 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:10,745 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:10,745 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:10,745 DEBUG [I/O Setup 1 Initialize: {out_Reducer 3}] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:10,745 INFO [TezTR-518813_1_41_3_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:10,745 INFO [TezTR-518813_1_41_3_0_0] resources.MemoryDistributor: InitialRequests=[out_Reducer 3:OUTPUT:0:org.apache.tez.mapreduce.output.MROutput], [Reducer 2:INPUT:131404920:org.apache.tez.runtime.library.input.UnorderedKVInput] 2018-01-05T11:29:10,745 INFO [TezTR-518813_1_41_3_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:10,745 INFO [TezTR-518813_1_41_3_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:10,745 INFO [TezTR-518813_1_41_3_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 131404920, TotalRequestedScaled: 6.570246E7, TotalJVMHeap: 1023410176, TotalAvailable: 685684817, TotalRequested/TotalJVMHeap:0.13 2018-01-05T11:29:10,745 INFO [TezTR-518813_1_41_3_0_0] resources.MemoryDistributor: Allocations=[out_Reducer 3:org.apache.tez.mapreduce.output.MROutput:OUTPUT:0:0], [Reducer 2:org.apache.tez.runtime.library.input.UnorderedKVInput:INPUT:131404920:131404920] 2018-01-05T11:29:10,745 INFO [TezTR-518813_1_41_3_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:10,745 INFO [TezTR-518813_1_41_3_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 0 2018-01-05T11:29:10,745 INFO [TezTR-518813_1_41_3_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:10,745 INFO [TezTR-518813_1_41_3_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_41_03_000000_0 2018-01-05T11:29:10,745 DEBUG [TezTR-518813_1_41_3_0_0] log.PerfLogger: 2018-01-05T11:29:10,745 DEBUG [TezTR-518813_1_41_3_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_41_03_000000_0_10318 2018-01-05T11:29:10,745 DEBUG [TezTR-518813_1_41_3_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:10,745 DEBUG [TezTR-518813_1_41_3_0_0] mr.ObjectCache: Creating djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca_Reducer 3__REDUCE_PLAN__ 2018-01-05T11:29:10,746 INFO [TezTR-518813_1_41_3_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/0e573408-4348-441a-a17c-ff08c5a32b50/reduce.xml 2018-01-05T11:29:10,746 DEBUG [TezTR-518813_1_41_3_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/0e573408-4348-441a-a17c-ff08c5a32b50/reduce.xml 2018-01-05T11:29:10,746 DEBUG [TezTR-518813_1_41_3_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/djaiswal/_tez_scratch_dir/0e573408-4348-441a-a17c-ff08c5a32b50/reduce.xml 2018-01-05T11:29:10,746 DEBUG [TezTR-518813_1_41_3_0_0] log.PerfLogger: 2018-01-05T11:29:10,746 INFO [TezTR-518813_1_41_3_0_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:29:10,746 DEBUG [TezTR-518813_1_41_3_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:10,747 DEBUG [TezTR-518813_1_41_3_0_0] log.PerfLogger: 2018-01-05T11:29:10,747 INFO [TezTR-518813_1_41_3_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 3 size: 1.90KB 2018-01-05T11:29:10,747 DEBUG [TezTR-518813_1_41_3_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:10,747 DEBUG [TezTR-518813_1_41_3_0_0] log.PerfLogger: 2018-01-05T11:29:10,747 DEBUG [TezTR-518813_1_41_3_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:10,747 DEBUG [TezTR-518813_1_41_3_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:10,747 DEBUG [TezTR-518813_1_41_3_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:10,747 INFO [TezTR-518813_1_41_3_0_0] impl.SimpleFetchedInputAllocator: Reducer_2: RequestedMemory=131404920, AssignedMemory=131404920, maxSingleShuffleLimit=32851230 2018-01-05T11:29:10,747 INFO [TezTR-518813_1_41_3_0_0] impl.ShuffleManager: Reducer_2: numInputs=1, compressionCodec=NoCompressionCodec, numFetchers=1, ifileBufferSize=4096, ifileReadAheadEnabled=true, ifileReadAheadLength=4194304, localDiskFetchEnabled=false, sharedFetchEnabled=false, keepAlive=false, keepAliveMaxConnections=20, connectionTimeout=180000, readTimeout=180000, bufferSize=8192, bufferSize=8192, maxTaskOutputAtOnce=20 2018-01-05T11:29:10,748 INFO [TezTR-518813_1_41_3_0_0] impl.ShuffleInputEventHandlerImpl: Reducer 2: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0 2018-01-05T11:29:10,748 INFO [TezTR-518813_1_41_3_0_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:29:10,749 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: RECV: /mapOutput?job=job_1515180518813_0001&dag=41&reduce=0&map=attempt_1515180518813_0001_41_02_000000_0_10317 mapId: [attempt_1515180518813_0001_41_02_000000_0_10317] reduceId: [0] jobId: [job_1515180518813_0001] dagId: [41] keepAlive: false 2018-01-05T11:29:10,749 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: verifying request. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=41&reduce=0&map=attempt_1515180518813_0001_41_02_000000_0_10317; hash=...GhuliN9PCc7Qw 2018-01-05T11:29:10,749 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Fetcher request verified. enc_str=49816/mapOutput?job=job_1515180518813_0001&dag=41&reduce=0&map=attempt_1515180518813_0001_41_02_000000_0_10317;reply=2VKbhNxA2gTcQ 2018-01-05T11:29:10,749 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Loaded : AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=41, user='djaiswal', attemptId='attempt_1515180518813_0001_41_02_000000_0_10317'} via loader 2018-01-05T11:29:10,749 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Retrieved pathInfo for AttemptPathIdentifier{jobId='job_1515180518813_0001', dagId=41, user='djaiswal', attemptId='attempt_1515180518813_0001_41_02_000000_0_10317'} check for corresponding loaded messages to determine whether it was loaded or cached 2018-01-05T11:29:10,749 DEBUG [New I/O worker #3] shufflehandler.IndexCache: IndexCache MISS: MapId attempt_1515180518813_0001_41_02_000000_0_10317 not found 2018-01-05T11:29:10,749 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: jobId=job_1515180518813_0001, mapId=attempt_1515180518813_0001_41_02_000000_0_10317,dataFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/41/output/attempt_1515180518813_0001_41_02_000000_0_10317/file.out, indexFile=/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/41/output/attempt_1515180518813_0001_41_02_000000_0_10317/file.out.index 2018-01-05T11:29:10,749 DEBUG [New I/O worker #3] shufflehandler.ShuffleHandler: Content Length in shuffle : 68 2018-01-05T11:29:10,749 INFO [Fetcher_B {Reducer_2} #0] HttpConnection.url: for url=http://10.22.8.182:49816/mapOutput?job=job_1515180518813_0001&dag=41&reduce=0&map=attempt_1515180518813_0001_41_02_000000_0_10317 sent hash and receievd reply 0 ms 2018-01-05T11:29:10,750 INFO [Fetcher_B {Reducer_2} #0] ShuffleManager.fetch: Completed fetch for attempt: {0, 0, attempt_1515180518813_0001_41_02_000000_0_10317} to MEMORY, csize=17, dsize=13, EndTime=1515180550750, TimeTaken=0, Rate=0.00 MB/s 2018-01-05T11:29:10,750 INFO [Fetcher_B {Reducer_2} #0] impl.ShuffleManager: All inputs fetched for input vertex : Reducer 2 2018-01-05T11:29:10,750 INFO [TezTR-518813_1_41_3_0_0] tez.ReduceRecordProcessor: Main work is Reducer 3 2018-01-05T11:29:10,750 INFO [Fetcher_B {Reducer_2} #0] impl.ShuffleManager: copy(1 (spillsFetched=1) of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s) 2018-01-05T11:29:10,750 INFO [TezTR-518813_1_41_3_0_0] tez.ReduceRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:10,750 INFO [ShuffleRunner {Reducer_2}] impl.ShuffleManager: Reducer_2: Shutting down FetchScheduler, Was Interrupted: false 2018-01-05T11:29:10,750 DEBUG [TezTR-518813_1_41_3_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:10,750 INFO [ShuffleRunner {Reducer_2}] impl.ShuffleManager: Reducer_2: Scheduler thread completed 2018-01-05T11:29:10,750 DEBUG [TezTR-518813_1_41_3_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:10,750 DEBUG [TezTR-518813_1_41_3_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:10,750 DEBUG [TezTR-518813_1_41_3_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0] columnTypes=[bigint] 2018-01-05T11:29:10,750 DEBUG [TezTR-518813_1_41_3_0_0] log.PerfLogger: 2018-01-05T11:29:10,750 INFO [TezTR-518813_1_41_3_0_0] vector.VectorGroupByOperator: Initializing operator GBY[18] 2018-01-05T11:29:10,750 INFO [TezTR-518813_1_41_3_0_0] vector.VectorGroupByOperator: using global aggregation processing mode 2018-01-05T11:29:10,750 DEBUG [TezTR-518813_1_41_3_0_0] vector.VectorGroupByOperator: Initialization Done 18 GBY 2018-01-05T11:29:10,750 DEBUG [TezTR-518813_1_41_3_0_0] vector.VectorGroupByOperator: Operator 18 GBY initialized 2018-01-05T11:29:10,750 DEBUG [TezTR-518813_1_41_3_0_0] vector.VectorGroupByOperator: Initializing children of 18 GBY 2018-01-05T11:29:10,750 DEBUG [TezTR-518813_1_41_3_0_0] vector.VectorFileSinkOperator: Initializing child 19 FS 2018-01-05T11:29:10,750 INFO [TezTR-518813_1_41_3_0_0] vector.VectorFileSinkOperator: Initializing operator FS[19] 2018-01-05T11:29:10,750 DEBUG [TezTR-518813_1_41_3_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:10,750 INFO [TezTR-518813_1_41_3_0_0] exec.FileSinkOperator: Using serializer : class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe[[[B@772fd8a7]:[_col0]:[bigint]] and formatter : org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat@1a39f540 2018-01-05T11:29:10,750 DEBUG [TezTR-518813_1_41_3_0_0] exec.FileSinkOperator: Created slots for 1 2018-01-05T11:29:10,750 DEBUG [TezTR-518813_1_41_3_0_0] vector.VectorFileSinkOperator: Initialization Done 19 FS 2018-01-05T11:29:10,750 DEBUG [TezTR-518813_1_41_3_0_0] vector.VectorFileSinkOperator: Operator 19 FS initialized 2018-01-05T11:29:10,750 DEBUG [TezTR-518813_1_41_3_0_0] vector.VectorFileSinkOperator: Initialization Done 19 FS done is reset. 2018-01-05T11:29:10,750 DEBUG [TezTR-518813_1_41_3_0_0] vector.VectorGroupByOperator: Initialization Done 18 GBY done is reset. 2018-01-05T11:29:10,750 INFO [TezTR-518813_1_41_3_0_0] tez.ReduceRecordProcessor: Id =18 Id =19 <\Children> Id = 18 null<\Parent> <\FS> <\Children> <\Parent> <\GBY> 2018-01-05T11:29:10,750 DEBUG [TezTR-518813_1_41_3_0_0] log.PerfLogger: 2018-01-05T11:29:10,750 INFO [TezTR-518813_1_41_3_0_0] tez.ReduceRecordProcessor: Starting Output: out_Reducer 3 2018-01-05T11:29:10,751 INFO [TezTR-518813_1_41_3_0_0] readers.UnorderedKVReader: Num Records read: 1 2018-01-05T11:29:10,751 DEBUG [TezTR-518813_1_41_3_0_0] log.PerfLogger: 2018-01-05T11:29:10,751 DEBUG [TezTR-518813_1_41_3_0_0] mr.ObjectCache: djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca_Reducer 3__REDUCE_PLAN__ no longer needed 2018-01-05T11:29:10,751 DEBUG [TezTR-518813_1_41_3_0_0] vector.VectorGroupByOperator: close called for operator GBY[18] 2018-01-05T11:29:10,751 INFO [TezTR-518813_1_41_3_0_0] vector.VectorGroupByOperator: Closing operator GBY[18] 2018-01-05T11:29:10,751 INFO [TezTR-518813_1_41_3_0_0] exec.FileSinkOperator: Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-10_156_6921175463455711345-1/_tmp.-ext-10002/000000_0 2018-01-05T11:29:10,751 INFO [TezTR-518813_1_41_3_0_0] exec.FileSinkOperator: Writing to temp file: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-10_156_6921175463455711345-1/_task_tmp.-ext-10002/_tmp.000000_0 2018-01-05T11:29:10,751 INFO [TezTR-518813_1_41_3_0_0] exec.FileSinkOperator: New Final Path: FS file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-10_156_6921175463455711345-1/_tmp.-ext-10002/000000_0 2018-01-05T11:29:10,800 INFO [TezTR-518813_1_41_3_0_0] exec.FileSinkOperator: FS[19]: records written - 1 2018-01-05T11:29:10,800 DEBUG [TezTR-518813_1_41_3_0_0] vector.VectorGroupByOperator: Closing child = FS[19] 2018-01-05T11:29:10,800 DEBUG [TezTR-518813_1_41_3_0_0] vector.VectorFileSinkOperator: close called for operator FS[19] 2018-01-05T11:29:10,800 DEBUG [TezTR-518813_1_41_3_0_0] vector.VectorFileSinkOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:10,800 INFO [TezTR-518813_1_41_3_0_0] vector.VectorFileSinkOperator: Closing operator FS[19] 2018-01-05T11:29:10,800 INFO [TezTR-518813_1_41_3_0_0] exec.FileSinkOperator: FS[19]: records written - 1 2018-01-05T11:29:10,801 INFO [TezTR-518813_1_41_3_0_0] vector.VectorFileSinkOperator: RECORDS_OUT_0:1, 2018-01-05T11:29:10,801 DEBUG [TezTR-518813_1_41_3_0_0] vector.VectorFileSinkOperator: 19 Close done 2018-01-05T11:29:10,801 DEBUG [TezTR-518813_1_41_3_0_0] vector.VectorGroupByOperator: 18 Close done 2018-01-05T11:29:10,801 INFO [TezTR-518813_1_41_3_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_41_03_000000_0 2018-01-05T11:29:10,801 INFO [TezTR-518813_1_41_3_0_0] impl.ShuffleInputEventHandlerImpl: Reducer 2: numDmeEventsSeen=1, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:29:10,801 INFO [TezTR-518813_1_41_3_0_0] impl.ShuffleManager: Shutting down pending fetchers on sourceReducer_2: 0 2018-01-05T11:29:10,801 INFO [TezTR-518813_1_41_3_0_0] output.MROutput: out_Reducer 3 closed 2018-01-05T11:29:10,801 DEBUG [TezTR-518813_1_41_3_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:10,801 INFO [TezTR-518813_1_41_3_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:10,801 INFO [TezTR-518813_1_41_3_0_0] task.TaskRunner2Callable: Task completed, taskAttemptId=attempt_1515180518813_0001_41_03_000000_0, askedToStop=false 2018-01-05T11:29:10,801 INFO [TezTR-518813_1_41_3_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_41_03_000000_0, stopRequested=false 2018-01-05T11:29:10,801 INFO [TezTR-518813_1_41_3_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_41_03_000000_0: Counters: 14 [[org.apache.tez.common.counters.TaskCounter NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=17, SHUFFLE_BYTES_DECOMPRESSED=13, SHUFFLE_BYTES_TO_MEM=17, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=3, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1][HIVE CREATED_FILES=1, RECORDS_OUT_0=1]] 2018-01-05T11:29:10,801 DEBUG [TezTR-518813_1_41_3_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_41_3_0_0 thread id: 2891 scheme: file bytesRead: 0 bytesWritten: 115 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:10,801 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Invoking OOB heartbeat for successful attempt: attempt_1515180518813_0001_41_03_000000_0, isTaskDone=true 2018-01-05T11:29:10,801 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000104, requestId=2, startIndex=1, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_41_03_000000_0, eventCount=2 } 2018-01-05T11:29:10,802 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 3, taskAttemptId=attempt_1515180518813_0001_41_03_000000_0, creationTime=1515180550737, allocationTime=1515180550738, startTime=1515180550739, finishTime=1515180550802, timeTaken=63, status=SUCCEEDED, diagnostics= 2018-01-05T11:29:10,802 INFO [Dispatcher thread {Central}] impl.TaskImpl: Task succeeded with attempt attempt_1515180518813_0001_41_03_000000_0 2018-01-05T11:29:10,802 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_41_03_000000_0, taskSucceeded=true, endReason=null 2018-01-05T11:29:10,802 DEBUG [Task-Executor-1] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=2, shouldDie=false, nextFromEventId=1, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:10,802 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_41_03_000000_0; false 2018-01-05T11:29:10,802 INFO [Task-Executor-1] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_41_03_000000_0 : TaskRunner2Result{endReason=SUCCESS, containerShutdownRequested=false} 2018-01-05T11:29:10,802 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_41_03_000000_0, state=ASSIGNED, endReason=null 2018-01-05T11:29:10,802 INFO [Task-Executor-1] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_41_03_000000_0 2018-01-05T11:29:10,802 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:10,802 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:TASK_FINISHED]: vertexName=Reducer 3, taskId=task_1515180518813_0001_41_03_000000, startTime=1515180550739, finishTime=1515180550802, timeTaken=63, status=SUCCEEDED, successfulAttemptID=attempt_1515180518813_0001_41_03_000000_0, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=115, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=17, SHUFFLE_BYTES_DECOMPRESSED=13, SHUFFLE_BYTES_TO_MEM=17, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=3, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1 2018-01-05T11:29:10,802 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:10,803 INFO [Task-Executor-1] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000104=62 2018-01-05T11:29:10,803 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_41_03_000000_0: true 2018-01-05T11:29:10,803 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_41_03 [Reducer 3], tasks=1, failed=0, killed=0, success=1, completed=1, commits=0, err=null 2018-01-05T11:29:10,803 INFO [Dispatcher thread {Central}] impl.VertexImpl: All tasks have succeeded, vertex:vertex_1515180518813_0001_41_03 [Reducer 3] 2018-01-05T11:29:10,803 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:VERTEX_FINISHED]: vertexName=Reducer 3, vertexId=vertex_1515180518813_0001_41_03, initRequestedTime=1515180550454, initedTime=1515180550459, startRequestedTime=1515180550516, startedTime=1515180550516, finishTime=1515180550803, timeTaken=287, status=SUCCEEDED, diagnostics=, counters=Counters: 19, File System Counters, FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=115, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, NUM_SHUFFLED_INPUTS=1, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=1, OUTPUT_RECORDS=0, SHUFFLE_BYTES=17, SHUFFLE_BYTES_DECOMPRESSED=13, SHUFFLE_BYTES_TO_MEM=17, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=3, FIRST_EVENT_RECEIVED=1, LAST_EVENT_RECEIVED=1, HIVE, CREATED_FILES=1, RECORDS_OUT_0=1, vertexStats=firstTaskStartTime=1515180550739, firstTasksToStart=[ task_1515180518813_0001_41_03_000000 ], lastTaskFinishTime=1515180550802, lastTasksToFinish=[ task_1515180518813_0001_41_03_000000 ], minTaskDuration=63, maxTaskDuration=63, avgTaskDuration=63.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1515180518813_0001_41_03_000000 ], longestDurationTasks=[ task_1515180518813_0001_41_03_000000 ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=1, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:10,804 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_41_03 [Reducer 3] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED 2018-01-05T11:29:10,804 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000104, stoppedTime=1515180550804, exitStatus=0 2018-01-05T11:29:10,804 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000104 exited with diagnostics set to Container completed. 2018-01-05T11:29:10,804 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_41_03 [Reducer 3] completed., numCompletedVertices=4, numSuccessfulVertices=4, numFailedVertices=0, numKilledVertices=0, numVertices=4 2018-01-05T11:29:10,804 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=4, numSuccessfulVertices=4, numFailedVertices=0, numKilledVertices=0, numVertices=4, commitInProgress=0, terminationCause=null 2018-01-05T11:29:10,804 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_41_01 [Map 1] 2018-01-05T11:29:10,804 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_41_00 [Map 4] 2018-01-05T11:29:10,804 INFO [Dispatcher thread {Central}] impl.DAGImpl: No exclusive output committers for vertex: vertex_1515180518813_0001_41_03 [Reducer 3] 2018-01-05T11:29:10,804 INFO [Dispatcher thread {Central}] impl.DAGImpl: No output committers for vertex: vertex_1515180518813_0001_41_02 [Reducer 2] 2018-01-05T11:29:10,804 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_41, queueSize=0 2018-01-05T11:29:10,804 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_41][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_41, startTime=1515180550433, finishTime=1515180550804, timeTaken=371, status=SUCCEEDED, diagnostics=, counters=Counters: 65, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=4, TOTAL_LAUNCHED_TASKS=4, OTHER_LOCAL_TASKS=2, AM_CPU_MILLISECONDS=944, AM_GC_TIME_MILLIS=0, File System Counters, FILE_BYTES_READ=128, FILE_BYTES_WRITTEN=4624, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=462, REDUCE_INPUT_RECORDS=742, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=1484, NUM_SHUFFLED_INPUTS=3, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, INPUT_RECORDS_PROCESSED=7, INPUT_SPLIT_LENGTH_BYTES=3591, OUTPUT_RECORDS=743, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=4457, OUTPUT_BYTES_WITH_OVERHEAD=4401, OUTPUT_BYTES_PHYSICAL=4437, ADDITIONAL_SPILLS_BYTES_WRITTEN=4396, ADDITIONAL_SPILLS_BYTES_READ=4396, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=2, SHUFFLE_BYTES=4413, SHUFFLE_BYTES_DECOMPRESSED=4401, SHUFFLE_BYTES_TO_MEM=4413, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=22, MERGE_PHASE_TIME=70, FIRST_EVENT_RECEIVED=11, LAST_EVENT_RECEIVED=11, HIVE, CREATED_FILES=1, DESERIALIZE_ERRORS=0, RECORDS_IN_Map_1=242, RECORDS_IN_Map_4=500, RECORDS_OUT_0=1, RECORDS_OUT_INTERMEDIATE_Map_1=242, RECORDS_OUT_INTERMEDIATE_Map_4=500, RECORDS_OUT_INTERMEDIATE_Reducer_2=1, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, org.apache.hadoop.hive.llap.counters.LlapIOCounters, CACHE_HIT_BYTES=1126, CACHE_MISS_BYTES=0, CONSUMER_TIME_NS=2377189, DECODE_TIME_NS=1248181, HDFS_TIME_NS=1126776, METADATA_CACHE_HIT=12, NUM_DECODED_BATCHES=6, NUM_VECTOR_BATCHES=6, ROWS_EMITTED=742, SELECTED_ROWGROUPS=6, TOTAL_IO_TIME_NS=10241994 2018-01-05T11:29:10,805 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_41 finished with state: SUCCEEDED 2018-01-05T11:29:10,805 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_41 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:29:10,805 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000104 2018-01-05T11:29:10,805 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_41, dagState=SUCCEEDED 2018-01-05T11:29:10,805 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_41_post 2018-01-05T11:29:10,805 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:29:10,805 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 0 2018-01-05T11:29:10,805 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:29:10,805 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 41 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=4, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=4,NumRequestsWithoutLocation=4, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=4} 2018-01-05T11:29:10,805 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:29:10,805 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=select count(*) from tab a full oute...b.key (Stage-1), with id=dag_1515180518813_0001_41 2018-01-05T11:29:10,805 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 41, to 10.22.8.182:49818 2018-01-05T11:29:10,805 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=4, Remaining=0 2018-01-05T11:29:10,805 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=select count(*) from tab a full oute...b.key (Stage-1), with id=dag_1515180518813_0001_41 2018-01-05T11:29:10,805 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=41} 2018-01-05T11:29:10,805 INFO [IPC Server handler 4 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=41}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:29:10,805 INFO [IPC Server handler 4 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/41 after 300 seconds 2018-01-05T11:29:10,805 ERROR [IPC Server handler 4 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca, dag_1515180518813_0001_41 2018-01-05T11:29:10,805 INFO [IPC Server handler 4 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca 2018-01-05T11:29:10,805 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=41} = 1 2018-01-05T11:29:10,805 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Issuing killFragment for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=41} attempt_1515180518813_0001_41_03_000000_0 2018-01-05T11:29:10,805 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Removing attempt_1515180518813_0001_41_03_000000_0 from preemptionQueue 2018-01-05T11:29:10,805 INFO [IPC Server handler 4 on 49818] impl.TaskRunnerCallable: Ignoring kill request for task attempt_1515180518813_0001_41_03_000000_0 since it's already complete 2018-01-05T11:29:10,805 DEBUG [IPC Server handler 4 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=41} 2018-01-05T11:29:10,805 DEBUG [IPC Server handler 4 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:29:10,805 DEBUG [IPC Server handler 4 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=41} 2018-01-05T11:29:10,807 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,807 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,807 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,807 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 1/1 Map 4: 1/1 Reducer 2: 1/1 Reducer 3: 1/1 2018-01-05T11:29:10,807 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,809 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,809 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: TaskId for 000000_0 = 000000 2018-01-05T11:29:10,809 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,809 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,809 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,810 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,810 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,810 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: select count(*) from tab a full outer join tab_part b on a.key = b.key 2018-01-05T11:29:10,810 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:29:10,810 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@tab 2018-01-05T11:29:10,810 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@tab@ds=2008-04-08 2018-01-05T11:29:10,810 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@tab_part 2018-01-05T11:29:10,810 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@tab_part@ds=2008-04-08 2018-01-05T11:29:10,810 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/-mr-10001 2018-01-05T11:29:10,810 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,810 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,810 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:10,810 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {} 2018-01-05T11:29:10,810 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca); Time taken: 0.523 seconds 2018-01-05T11:29:10,810 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:10,810 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,810 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:10,810 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,810 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query select count(*) from tab a full outer join tab_part b on a.key = b.key 2018-01-05T11:29:10,829 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] mapred.FileInputFormat: Time taken to get FileStatuses: 18 2018-01-05T11:29:10,829 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:29:10,829 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 18 2018-01-05T11:29:10,829 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FetchOperator: Creating fetchTask with deserializer typeinfo: struct<_col0:bigint> 2018-01-05T11:29:10,829 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FetchOperator: deserializer properties: table properties: {columns=_col0, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=bigint, escape.delim=\} partition properties: {columns=_col0, serialization.escape.crlf=true, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, hive.serialization.extend.additional.nesting.levels=true, serialization.format=1, columns.types=bigint, escape.delim=\} 2018-01-05T11:29:10,830 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: close called for operator LIST_SINK[20] 2018-01-05T11:29:10,830 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: Closing operator LIST_SINK[20] 2018-01-05T11:29:10,830 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: 20 Close done 2018-01-05T11:29:10,832 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting result dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/-mr-10001 2018-01-05T11:29:10,834 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-10_156_6921175463455711345-1 2018-01-05T11:29:10,834 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_156_6921175463455711345-1 2018-01-05T11:29:10,835 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.654 seconds, Fetched: 1 row(s) 2018-01-05T11:29:10,835 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:10,835 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:10,835 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:10,835 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:10,836 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,836 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:10,836 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,836 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,836 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,836 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: explain vectorization detail select count(*) from tab a join tab_part b on a.key = b.key join src1 c on a.value = c.value 2018-01-05T11:29:10,836 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112910_b90e584b-fd25-4300-aa00-d604e47c88c2): explain vectorization detail select count(*) from tab a join tab_part b on a.key = b.key join src1 c on a.value = c.value 2018-01-05T11:29:10,836 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,836 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: explain vectorization detail select count(*) from tab a join tab_part b on a.key = b.key join src1 c on a.value = c.value 2018-01-05T11:29:10,836 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:10,836 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,836 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,837 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,837 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,837 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:10,837 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000103, QueryId=djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca-41, DagId=41, VertexName=Reducer 2, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-1, Succeeded=true, StartTime=1515180550624, EndTime=1515180550837 2018-01-05T11:29:10,837 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_41_03_000000_0 2018-01-05T11:29:10,837 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:29:10,837 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_41_03_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:29:10,837 DEBUG [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Successfully finished attempt_1515180518813_0001_41_03_000000_0 2018-01-05T11:29:10,837 INFO [ExecutionCompletionThread #0] impl.QueryTracker: Ignoring fragmentComplete message for unknown query: QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=41} 2018-01-05T11:29:10,837 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:10,837 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed phase 1 of Semantic Analysis 2018-01-05T11:29:10,837 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:10,837 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,837 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab 2018-01-05T11:29:10,837 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab 2018-01-05T11:29:10,841 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,841 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,841 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,841 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab_part 2018-01-05T11:29:10,841 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab_part 2018-01-05T11:29:10,843 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,844 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,844 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,844 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src1 2018-01-05T11:29:10,844 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src1 2018-01-05T11:29:10,848 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,848 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,849 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:10,849 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:10,849 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed getting MetaData in Semantic Analysis 2018-01-05T11:29:10,850 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,850 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,850 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_not_null_constraints : db=default tbl=tab 2018-01-05T11:29:10,850 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_not_null_constraints : db=default tbl=tab 2018-01-05T11:29:10,851 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,851 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,851 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,851 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab 2018-01-05T11:29:10,851 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab 2018-01-05T11:29:10,851 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,851 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,851 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,851 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab 2018-01-05T11:29:10,851 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab 2018-01-05T11:29:10,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,852 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,852 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_unique_constraints : db=default tbl=tab 2018-01-05T11:29:10,852 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_unique_constraints : db=default tbl=tab 2018-01-05T11:29:10,853 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,853 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,853 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,853 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab 2018-01-05T11:29:10,853 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab 2018-01-05T11:29:10,853 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,853 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,854 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,854 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_not_null_constraints : db=default tbl=tab_part 2018-01-05T11:29:10,854 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_not_null_constraints : db=default tbl=tab_part 2018-01-05T11:29:10,854 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,854 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,854 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,854 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:10,854 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:10,854 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,854 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,854 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,854 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:10,854 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:10,854 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,854 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,855 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,855 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_unique_constraints : db=default tbl=tab_part 2018-01-05T11:29:10,855 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_unique_constraints : db=default tbl=tab_part 2018-01-05T11:29:10,855 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,855 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,855 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,855 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab_part 2018-01-05T11:29:10,855 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab_part 2018-01-05T11:29:10,855 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,855 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,855 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,855 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_not_null_constraints : db=default tbl=src1 2018-01-05T11:29:10,855 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_not_null_constraints : db=default tbl=src1 2018-01-05T11:29:10,856 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,856 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,856 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,856 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=src1 2018-01-05T11:29:10,856 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=src1 2018-01-05T11:29:10,856 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,856 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,856 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,856 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=src1 2018-01-05T11:29:10,856 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=src1 2018-01-05T11:29:10,856 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,856 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,856 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,856 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_unique_constraints : db=default tbl=src1 2018-01-05T11:29:10,856 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_unique_constraints : db=default tbl=src1 2018-01-05T11:29:10,857 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,857 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,857 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,857 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=src1 2018-01-05T11:29:10,857 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=src1 2018-01-05T11:29:10,857 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,857 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,858 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:10,858 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:10,859 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:10,859 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:29:10,859 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,872 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.sql2rel: Plan after trimming unused fields HiveAggregate(group=[{}], agg#0=[count()]) HiveProject(DUMMY=[0]) HiveJoin(condition=[=($1, $3)], joinType=[inner], algorithm=[none], cost=[not available]) HiveJoin(condition=[=($0, $2)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) HiveProject(value=[$1]) HiveTableScan(table=[[default.src1]], table:alias=[c]) 2018-01-05T11:29:10,872 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan before removing subquery: HiveProject(_o__c0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveProject($f0=[$0]) HiveJoin(condition=[=($1, $13)], joinType=[inner], algorithm=[none], cost=[not available]) HiveJoin(condition=[=($0, $6)], joinType=[inner], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) HiveTableScan(table=[[default.src1]], table:alias=[c]) 2018-01-05T11:29:10,872 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan just after removing subquery: HiveProject(_o__c0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveProject($f0=[$0]) HiveJoin(condition=[=($1, $13)], joinType=[inner], algorithm=[none], cost=[not available]) HiveJoin(condition=[=($0, $6)], joinType=[inner], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) HiveTableScan(table=[[default.src1]], table:alias=[c]) 2018-01-05T11:29:10,872 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan after decorrelation: HiveProject(_o__c0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveProject($f0=[$0]) HiveJoin(condition=[=($1, $13)], joinType=[inner], algorithm=[none], cost=[not available]) HiveJoin(condition=[=($0, $6)], joinType=[inner], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) HiveTableScan(table=[[default.src1]], table:alias=[c]) 2018-01-05T11:29:10,872 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,873 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,873 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,873 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,873 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,873 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,873 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,873 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,874 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,874 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,874 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,905 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,905 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,906 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,906 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,906 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partitions : db=default tbl=tab 2018-01-05T11:29:10,906 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions : db=default tbl=tab 2018-01-05T11:29:10,909 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,910 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.195229ms + 0.003834ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? ] 2018-01-05T11:29:10,910 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.164139ms + 0.052097ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (15) order by "PART_NAME" asc] 2018-01-05T11:29:10,910 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.084763ms + 0.060603ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (15) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:29:10,910 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.070328ms + 0.019408ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (15) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:10,910 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.064066ms + 0.007103ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (43) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:29:10,911 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.059254ms + 0.020562ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (43) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:10,911 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.057915ms + 0.016171ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (43) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:10,911 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.052518ms + 0.005697ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (43) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:10,911 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.084155ms + 0.0237ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (28) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:10,911 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.066468ms + 0.020464ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (43) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:29:10,911 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,914 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,914 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,914 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,914 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,914 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,914 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partitions : db=default tbl=tab_part 2018-01-05T11:29:10,914 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions : db=default tbl=tab_part 2018-01-05T11:29:10,917 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,917 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.144597ms + 0.002742ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? ] 2018-01-05T11:29:10,918 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.123443ms + 0.044776ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (14) order by "PART_NAME" asc] 2018-01-05T11:29:10,918 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.072387ms + 0.05522ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (14) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:29:10,918 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.063029ms + 0.016897ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (14) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:10,918 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.056479ms + 0.005886ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (41) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:29:10,918 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.064236ms + 0.019722ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (41) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:10,918 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.058191ms + 0.015737ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (41) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:10,918 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.051519ms + 0.005363ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (41) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:10,918 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.07965ms + 0.020255ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (26) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:10,918 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.062567ms + 0.018882ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (41) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:29:10,918 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,921 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,921 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,921 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,921 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,921 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,921 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,921 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,930 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Estimated average row size: 104 2018-01-05T11:29:10,930 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Stats for column key in table default@tab could not be retrieved from cache 2018-01-05T11:29:10,930 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Stats for column value in table default@tab could not be retrieved from cache 2018-01-05T11:29:10,930 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,930 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_aggr_stats_for: db=default table=tab 2018-01-05T11:29:10,930 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_aggr_stats_for: db=default table=tab 2018-01-05T11:29:10,932 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.AggregateStatsCache: Returning aggregate stats from the cache; total hits: 19, total misses: 0, hit ratio: 1.0 2018-01-05T11:29:10,932 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.AggregateStatsCache: Returning aggregate stats from the cache; total hits: 20, total misses: 0, hit ratio: 1.0 2018-01-05T11:29:10,932 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: useDensityFunctionForNDVEstimation = false partsFound = 1 ColumnStatisticsObj = [ColumnStatisticsObj(colName:key, colType:int, statsData:), ColumnStatisticsObj(colName:value, colType:string, statsData:)] 2018-01-05T11:29:10,933 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,934 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,934 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.RelOptHiveTable: Stats for column key in table tab stored in cache 2018-01-05T11:29:10,935 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.RelOptHiveTable: colName: key colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false 2018-01-05T11:29:10,935 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.RelOptHiveTable: Stats for column value in table tab stored in cache 2018-01-05T11:29:10,935 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.RelOptHiveTable: colName: value colType: string countDistincts: 153 numNulls: 0 avgColLen: 6.801652892561983 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false 2018-01-05T11:29:10,935 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] rules.HiveRelFieldTrimmer: Got col stats for [0, 1] in [default.tab] 2018-01-05T11:29:10,936 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Estimated average row size: 4 2018-01-05T11:29:10,936 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Stats for column key in table default@tab_part could not be retrieved from cache 2018-01-05T11:29:10,936 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,936 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_aggr_stats_for: db=default table=tab_part 2018-01-05T11:29:10,936 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_aggr_stats_for: db=default table=tab_part 2018-01-05T11:29:10,938 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.AggregateStatsCache: Returning aggregate stats from the cache; total hits: 21, total misses: 0, hit ratio: 1.0 2018-01-05T11:29:10,938 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: useDensityFunctionForNDVEstimation = false partsFound = 1 ColumnStatisticsObj = [ColumnStatisticsObj(colName:key, colType:int, statsData:)] 2018-01-05T11:29:10,939 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000104, QueryId=djaiswal_20180105112910_89ba3661-36c7-4c13-a597-b291424706ca-41, DagId=41, VertexName=Reducer 3, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-1, Succeeded=true, StartTime=1515180550740, EndTime=1515180550939 2018-01-05T11:29:10,939 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,940 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,940 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.RelOptHiveTable: Stats for column key in table tab_part stored in cache 2018-01-05T11:29:10,940 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.RelOptHiveTable: colName: key colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false 2018-01-05T11:29:10,940 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] rules.HiveRelFieldTrimmer: Got col stats for [0] in [default.tab_part] 2018-01-05T11:29:10,940 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,940 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=src1 2018-01-05T11:29:10,940 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=src1 2018-01-05T11:29:10,945 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 3.55461ms + 0.003103ms, the query is [select "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "TAB_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (...)] 2018-01-05T11:29:10,946 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,955 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:10,956 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:10,956 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.RelOptHiveTable: Stats for column value in table src1 stored in cache 2018-01-05T11:29:10,956 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.RelOptHiveTable: colName: value colType: string countDistincts: 19 numNulls: 0 avgColLen: 4.92 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false 2018-01-05T11:29:10,956 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] rules.HiveRelFieldTrimmer: Got col stats for [1] in [default.src1] 2018-01-05T11:29:10,963 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.sql2rel: Plan after trimming unused fields HiveAggregate(group=[{}], agg#0=[count()]) HiveProject(DUMMY=[0]) HiveJoin(condition=[=($1, $3)], joinType=[inner], algorithm=[none], cost=[not available]) HiveJoin(condition=[=($0, $2)], joinType=[inner], algorithm=[none], cost=[not available]) HiveFilter(condition=[AND(IS NOT NULL($0), IS NOT NULL($1))]) HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveFilter(condition=[IS NOT NULL($0)]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) HiveFilter(condition=[IS NOT NULL($0)]) HiveProject(value=[$1]) HiveTableScan(table=[[default.src1]], table:alias=[c]) 2018-01-05T11:29:10,963 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,963 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,964 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,964 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,965 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,965 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:10,994 DEBUG [AMReporterQueueDrainer] impl.AMReporter: Removing am 10.22.8.182:49824 with last associated dag QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=30} from heartbeat with taskCount=0, amFailed=false, isDone=true 2018-01-05T11:29:11,052 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,052 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,053 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,053 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,054 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,054 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,054 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,054 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,055 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,055 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,055 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,056 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Original plan for PlanModifier HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($1, $0)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) HiveJoin(condition=[=($1, $2)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1]) HiveFilter(condition=[AND(IS NOT NULL($0), IS NOT NULL($1))]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(value=[$1]) HiveFilter(condition=[IS NOT NULL($1)]) HiveTableScan(table=[[default.src1]], table:alias=[c]) 2018-01-05T11:29:11,056 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after top-level introduceDerivedTable HiveProject($f0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($1, $0)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) HiveJoin(condition=[=($1, $2)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1]) HiveFilter(condition=[AND(IS NOT NULL($0), IS NOT NULL($1))]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(value=[$1]) HiveFilter(condition=[IS NOT NULL($1)]) HiveTableScan(table=[[default.src1]], table:alias=[c]) 2018-01-05T11:29:11,056 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after nested convertOpTree HiveProject($f0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($1, $0)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) HiveJoin(condition=[=($1, $2)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1]) HiveFilter(condition=[AND(IS NOT NULL($0), IS NOT NULL($1))]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(value=[$1]) HiveFilter(condition=[IS NOT NULL($1)]) HiveTableScan(table=[[default.src1]], table:alias=[c]) 2018-01-05T11:29:11,056 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after propagating order HiveProject($f0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($1, $0)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) HiveJoin(condition=[=($1, $2)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1]) HiveFilter(condition=[AND(IS NOT NULL($0), IS NOT NULL($1))]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(value=[$1]) HiveFilter(condition=[IS NOT NULL($1)]) HiveTableScan(table=[[default.src1]], table:alias=[c]) 2018-01-05T11:29:11,056 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after fixTopOBSchema HiveProject($f0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($1, $0)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) HiveJoin(condition=[=($1, $2)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1]) HiveFilter(condition=[AND(IS NOT NULL($0), IS NOT NULL($1))]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(value=[$1]) HiveFilter(condition=[IS NOT NULL($1)]) HiveTableScan(table=[[default.src1]], table:alias=[c]) 2018-01-05T11:29:11,056 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Final plan after modifier HiveProject(_c0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($1, $0)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) HiveJoin(condition=[=($1, $2)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1]) HiveFilter(condition=[AND(IS NOT NULL($0), IS NOT NULL($1))]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(value=[$1]) HiveFilter(condition=[IS NOT NULL($1)]) HiveTableScan(table=[[default.src1]], table:alias=[c]) 2018-01-05T11:29:11,057 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:11,057 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:11,057 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:11,057 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:11,057 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,057 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab 2018-01-05T11:29:11,057 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab 2018-01-05T11:29:11,060 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,061 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,061 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:11,061 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:11,061 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:11,061 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,061 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src1 2018-01-05T11:29:11,061 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src1 2018-01-05T11:29:11,063 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,063 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,063 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:11,063 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:11,063 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:11,063 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,064 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab_part 2018-01-05T11:29:11,064 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab_part 2018-01-05T11:29:11,066 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,066 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,066 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:11,066 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:11,066 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:11,066 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Table Plan for a TS[0] 2018-01-05T11:29:11,067 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Filter Plan for $hdt$_1 row schema: a{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:11,067 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB a{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB a{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:11,067 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (. (tok_table_or_col a) key) key) (tok_selexpr (. (tok_table_or_col a) value) value)) 2018-01-05T11:29:11,067 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = a{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} starRr = null 2018-01-05T11:29:11,067 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(key,_col0: int)(value,_col1: string)} 2018-01-05T11:29:11,067 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:11,067 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block $hdt$_1 2018-01-05T11:29:11,067 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block $hdt$_1 2018-01-05T11:29:11,067 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Table Plan for c TS[3] 2018-01-05T11:29:11,067 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Filter Plan for $hdt$_2 row schema: c{(key,key: string)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:11,067 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB c{(key,key: string)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB c{(key,key: string)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:11,067 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (. (tok_table_or_col c) value) value)) 2018-01-05T11:29:11,067 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = c{(key,key: string)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} starRr = null 2018-01-05T11:29:11,067 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(value,_col0: string)} 2018-01-05T11:29:11,067 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:11,067 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block $hdt$_2 2018-01-05T11:29:11,067 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block $hdt$_2 2018-01-05T11:29:11,067 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Table Plan for b TS[6] 2018-01-05T11:29:11,068 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Filter Plan for $hdt$_0 row schema: b{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:11,068 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB b{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB b{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:11,068 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (. (tok_table_or_col b) key) key)) 2018-01-05T11:29:11,068 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = b{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} starRr = null 2018-01-05T11:29:11,068 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(key,_col0: int)} 2018-01-05T11:29:11,068 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:11,068 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block $hdt$_0 2018-01-05T11:29:11,068 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block $hdt$_0 2018-01-05T11:29:11,068 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:11,068 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB $hdt$_1{(key,_col0: int)(value,_col1: string)} $hdt$_2{(value,_col2: string)} $hdt$_0{(key,_col3: int)} after GB {((tok_functionstar count),_col0: bigint)} 2018-01-05T11:29:11,068 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (tok_functionstar count) _c0)) 2018-01-05T11:29:11,068 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = {((tok_functionstar count),_col0: bigint)} starRr = $hdt$_1{(key,_col0: int)(value,_col1: string)} $hdt$_2{(value,_col2: string)} $hdt$_0{(key,_col3: int)} 2018-01-05T11:29:11,069 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(_c0,_col0: bigint)} 2018-01-05T11:29:11,069 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:11,069 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_836_4139621204064528450-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-10_836_4139621204064528450-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_836_4139621204064528450-1/-mr-10001 2018-01-05T11:29:11,069 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_836_4139621204064528450-1/-mr-10003 2018-01-05T11:29:11,069 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_836_4139621204064528450-1/-mr-10001 row schema: null{(_c0,_col0: bigint)} 2018-01-05T11:29:11,069 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block null 2018-01-05T11:29:11,069 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:29:11,069 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: CBO Succeeded; optimized logical plan. 2018-01-05T11:29:11,069 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Before logical optimization TS[0]-FIL[1]-SEL[2]-RS[9]-JOIN[11]-RS[12]-JOIN[14]-SEL[15]-GBY[16]-RS[17]-GBY[18]-SEL[19]-FS[20] TS[3]-FIL[4]-SEL[5]-RS[10]-JOIN[11] TS[6]-FIL[7]-SEL[8]-RS[13]-JOIN[14] 2018-01-05T11:29:11,069 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,069 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,069 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,070 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] lineage.Generator: Time taken for lineage transform=1 2018-01-05T11:29:11,070 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,070 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,070 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.PartitionColumnsSeparator: Partition columns not separated for null, is not IN operator : 2018-01-05T11:29:11,070 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.PartitionColumnsSeparator: Partition columns not separated for null, is not IN operator : 2018-01-05T11:29:11,070 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.PartitionColumnsSeparator: Partition columns not separated for null, is not IN operator : 2018-01-05T11:29:11,070 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.PartitionColumnsSeparator: Partition columns not separated for null, is not IN operator : 2018-01-05T11:29:11,070 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.PartitionColumnsSeparator: Partition columns not separated for null, is not IN operator : 2018-01-05T11:29:11,070 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,070 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,070 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SyntheticJoinPredicate: Synthetic predicate: 0 --> 1 2018-01-05T11:29:11,071 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SyntheticJoinPredicate: Synthetic predicate: 0 --> 1 2018-01-05T11:29:11,071 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SyntheticJoinPredicate: Synthetic predicate: 1 --> 0 2018-01-05T11:29:11,071 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SyntheticJoinPredicate: Synthetic predicate: 0 --> 1 2018-01-05T11:29:11,071 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SyntheticJoinPredicate: Synthetic predicate: 1 --> 0 2018-01-05T11:29:11,071 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,071 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,071 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FS(20) 2018-01-05T11:29:11,071 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(19) 2018-01-05T11:29:11,071 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for GBY(18) 2018-01-05T11:29:11,071 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(17) 2018-01-05T11:29:11,071 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for GBY(16) 2018-01-05T11:29:11,071 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(15) 2018-01-05T11:29:11,071 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for JOIN(14) 2018-01-05T11:29:11,071 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(12) 2018-01-05T11:29:11,071 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(24) 2018-01-05T11:29:11,071 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias $hdt$_1: (_col0) IN (RS[13]) 2018-01-05T11:29:11,071 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(22) 2018-01-05T11:29:11,071 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias $hdt$_1: (_col0) IN (RS[13]) 2018-01-05T11:29:11,071 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias $hdt$_1: (_col0) IN (RS[13]) 2018-01-05T11:29:11,071 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for JOIN(11) 2018-01-05T11:29:11,071 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of JOIN for alias $hdt$_1: (VALUE._col0) IN (RS[13]) 2018-01-05T11:29:11,071 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(9) 2018-01-05T11:29:11,071 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of RS for alias $hdt$_1: (_col0) IN (RS[13]) 2018-01-05T11:29:11,071 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(21) 2018-01-05T11:29:11,072 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias $hdt$_1: (_col1) IN (RS[10]) 2018-01-05T11:29:11,072 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias $hdt$_1: (_col0) IN (RS[13]) 2018-01-05T11:29:11,072 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(2) 2018-01-05T11:29:11,072 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of SEL for alias $hdt$_1: (value) IN (RS[10]); (key) IN (RS[13]) 2018-01-05T11:29:11,072 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(1) 2018-01-05T11:29:11,072 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias a: key is not null; value is not null 2018-01-05T11:29:11,072 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias a: (value) IN (RS[10]); (key) IN (RS[13]) 2018-01-05T11:29:11,072 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:29:11,072 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of TS for alias a: key is not null; value is not null; (value) IN (RS[10]); (key) IN (RS[13]) 2018-01-05T11:29:11,072 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(10) 2018-01-05T11:29:11,072 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(23) 2018-01-05T11:29:11,072 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias $hdt$_2: (_col0) IN (RS[9]) 2018-01-05T11:29:11,072 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(5) 2018-01-05T11:29:11,072 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of SEL for alias $hdt$_2: (value) IN (RS[9]) 2018-01-05T11:29:11,072 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(4) 2018-01-05T11:29:11,072 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias c: value is not null 2018-01-05T11:29:11,072 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias c: (value) IN (RS[9]) 2018-01-05T11:29:11,072 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for TS(3) 2018-01-05T11:29:11,072 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of TS for alias c: value is not null; (value) IN (RS[9]) 2018-01-05T11:29:11,073 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(13) 2018-01-05T11:29:11,073 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(25) 2018-01-05T11:29:11,073 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias $hdt$_0: (_col0) IN (RS[12]) 2018-01-05T11:29:11,073 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(8) 2018-01-05T11:29:11,073 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of SEL for alias $hdt$_0: (key) IN (RS[12]) 2018-01-05T11:29:11,073 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(7) 2018-01-05T11:29:11,073 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias b: key is not null 2018-01-05T11:29:11,073 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias b: (key) IN (RS[12]) 2018-01-05T11:29:11,073 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for TS(6) 2018-01-05T11:29:11,073 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of TS for alias b: key is not null; (key) IN (RS[12]) 2018-01-05T11:29:11,073 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SimplePredicatePushDown: After PPD: TS[0]-FIL[26]-SEL[2]-RS[9]-JOIN[11]-RS[12]-JOIN[14]-SEL[15]-GBY[16]-RS[17]-GBY[18]-SEL[19]-FS[20] TS[3]-FIL[27]-SEL[5]-RS[10]-JOIN[11] TS[6]-FIL[28]-SEL[8]-RS[13]-JOIN[14] 2018-01-05T11:29:11,073 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,073 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,073 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,073 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,073 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,073 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,073 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,073 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,073 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pcr.PcrExprProcFactory: Processing key is not null org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNotNull outputs {UNKNOWN, Column[key]}, 2018-01-05T11:29:11,073 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pcr.PcrExprProcFactory: Processing value is not null org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNotNull outputs {UNKNOWN, Column[value]}, 2018-01-05T11:29:11,073 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pcr.PcrExprProcFactory: Processing (value) IN (RS[10]) org.apache.hadoop.hive.ql.udf.generic.GenericUDFIn outputs {UNKNOWN, Column[value]}, {UNKNOWN, RS[10]}, 2018-01-05T11:29:11,073 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pcr.PcrExprProcFactory: Processing (key) IN (RS[13]) org.apache.hadoop.hive.ql.udf.generic.GenericUDFIn outputs {UNKNOWN, Column[key]}, {UNKNOWN, RS[13]}, 2018-01-05T11:29:11,073 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pcr.PcrExprProcFactory: Processing (key is not null and value is not null and (value) IN (RS[10]) and (key) IN (RS[13])) org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPAnd outputs {UNKNOWN, GenericUDFOPNotNull(Column[key])}, {UNKNOWN, GenericUDFOPNotNull(Column[value])}, {UNKNOWN, GenericUDFIn(Column[value], RS[10])}, {UNKNOWN, GenericUDFIn(Column[key], RS[13])}, 2018-01-05T11:29:11,073 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pcr.PcrExprProcFactory: Processing key is not null org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNotNull outputs {UNKNOWN, Column[key]}, 2018-01-05T11:29:11,073 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pcr.PcrExprProcFactory: Processing (key) IN (RS[12]) org.apache.hadoop.hive.ql.udf.generic.GenericUDFIn outputs {UNKNOWN, Column[key]}, {UNKNOWN, RS[12]}, 2018-01-05T11:29:11,073 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pcr.PcrExprProcFactory: Processing (key is not null and (key) IN (RS[12])) org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPAnd outputs {UNKNOWN, GenericUDFOPNotNull(Column[key])}, {UNKNOWN, GenericUDFIn(Column[key], RS[12])}, 2018-01-05T11:29:11,073 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,073 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,073 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,073 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,074 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 17 key:[] 2018-01-05T11:29:11,074 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 17 oldColExprMap: {VALUE._col0=Column[_col0]} 2018-01-05T11:29:11,074 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 17 newColExprMap: {VALUE._col0=Column[_col0]} 2018-01-05T11:29:11,074 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: JOIN 14 oldExprs: {0=[Column[KEY.reducesinkkey0], Column[VALUE._col0], Column[VALUE._col1]], 1=[Column[KEY.reducesinkkey0]]} 2018-01-05T11:29:11,074 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: JOIN 14 newExprs: {0=[], 1=[]} 2018-01-05T11:29:11,074 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 12 key:[Column[_col0]] 2018-01-05T11:29:11,074 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 12 oldColExprMap: {VALUE._col0=Column[_col1], VALUE._col1=Column[_col2], KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:11,074 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 12 newColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:11,074 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: JOIN 11 oldExprs: {0=[Column[VALUE._col0], Column[KEY.reducesinkkey0]], 1=[Column[KEY.reducesinkkey0]]} 2018-01-05T11:29:11,074 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 12 oldColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:11,074 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 12 newColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:11,074 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: JOIN 11 newExprs: {0=[Column[VALUE._col0]], 1=[]} 2018-01-05T11:29:11,074 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 13 key:[Column[_col0]] 2018-01-05T11:29:11,074 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 13 oldColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:11,074 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 13 newColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:11,074 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 9 key:[Column[_col1]] 2018-01-05T11:29:11,074 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 9 oldColExprMap: {VALUE._col0=Column[_col0], KEY.reducesinkkey0=Column[_col1]} 2018-01-05T11:29:11,074 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 9 newColExprMap: {VALUE._col0=Column[_col0], KEY.reducesinkkey0=Column[_col1]} 2018-01-05T11:29:11,074 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 10 key:[Column[_col0]] 2018-01-05T11:29:11,074 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 10 oldColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:11,074 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 10 newColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:11,074 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,074 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,074 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,074 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,074 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,074 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,074 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,074 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,074 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,074 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,074 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,074 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,074 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,074 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,074 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,074 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,074 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.IdentityProjectRemover: Identity project remover optimization removed : SEL[19] 2018-01-05T11:29:11,074 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.IdentityProjectRemover: Identity project remover optimization removed : SEL[15] 2018-01-05T11:29:11,075 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,075 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,075 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,075 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,075 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,075 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,075 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,075 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: After logical optimization TS[0]-FIL[26]-SEL[2]-RS[9]-JOIN[11]-RS[12]-JOIN[14]-GBY[16]-RS[17]-GBY[18]-FS[20] TS[3]-FIL[27]-SEL[5]-RS[10]-JOIN[11] TS[6]-FIL[28]-SEL[8]-RS[13]-JOIN[14] 2018-01-05T11:29:11,075 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,075 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Parent: TS[0] 2018-01-05T11:29:11,075 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Filter: (key is not null and value is not null and (value) IN (RS[10]) and (key) IN (RS[13])) (type: boolean) 2018-01-05T11:29:11,075 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: TableScan: TS[0] 2018-01-05T11:29:11,075 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Column value is not a partition column 2018-01-05T11:29:11,075 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Initiate semijoin reduction for value ((key is not null and value is not null and (value) IN (RS[10]) and (key) IN (RS[13])) 2018-01-05T11:29:11,075 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:11,075 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:11,075 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:11,075 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:11,075 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:11,075 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:11,075 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: DynamicSemiJoinPushdown: Saving RS to TS mapping: RS[33]: TS[0] 2018-01-05T11:29:11,075 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Column key is not a partition column 2018-01-05T11:29:11,075 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Initiate semijoin reduction for key ((key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter))) and (key) IN (RS[13])) 2018-01-05T11:29:11,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:11,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:11,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:11,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:11,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:11,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:11,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: DynamicSemiJoinPushdown: Saving RS to TS mapping: RS[38]: TS[0] 2018-01-05T11:29:11,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Parent: TS[3] 2018-01-05T11:29:11,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Filter: (value is not null and (value) IN (RS[9])) (type: boolean) 2018-01-05T11:29:11,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: TableScan: TS[3] 2018-01-05T11:29:11,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Column value is not a partition column 2018-01-05T11:29:11,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Initiate semijoin reduction for value ((value is not null and (value) IN (RS[9])) 2018-01-05T11:29:11,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:11,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:11,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:11,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:11,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:11,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:11,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: DynamicSemiJoinPushdown: Saving RS to TS mapping: RS[43]: TS[3] 2018-01-05T11:29:11,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Parent: TS[6] 2018-01-05T11:29:11,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Filter: (key is not null and (key) IN (RS[12])) (type: boolean) 2018-01-05T11:29:11,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: TableScan: TS[6] 2018-01-05T11:29:11,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Column key is not a partition column 2018-01-05T11:29:11,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Initiate semijoin reduction for key ((key is not null and (key) IN (RS[12])) 2018-01-05T11:29:11,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:11,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:11,076 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:11,077 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:11,077 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:11,077 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:11,077 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: DynamicSemiJoinPushdown: Saving RS to TS mapping: RS[48]: TS[6] 2018-01-05T11:29:11,077 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,077 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,077 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Estimated average row size: 104 2018-01-05T11:29:11,077 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Stats for column key in table default@tab retrieved from cache 2018-01-05T11:29:11,077 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Stats for column value in table default@tab retrieved from cache 2018-01-05T11:29:11,077 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting stats (Num rows: 242 Data size: 22990 Basic stats: COMPLETE Column stats: COMPLETE) on TS[0] 2018-01-05T11:29:11,077 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (tab): numRows: 242 dataSize: 22990 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {value= colName: value colType: string countDistincts: 153 numNulls: 0 avgColLen: 6.801652892561983 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, key= colName: key colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,077 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,077 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=src1 2018-01-05T11:29:11,077 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=src1 2018-01-05T11:29:11,080 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,082 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Stats for column value in table default@src1 retrieved from cache 2018-01-05T11:29:11,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting stats (Num rows: 25 Data size: 2225 Basic stats: COMPLETE Column stats: COMPLETE) on TS[3] 2018-01-05T11:29:11,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-TS[3] (src1): numRows: 25 dataSize: 2225 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {value= colName: value colType: string countDistincts: 19 numNulls: 0 avgColLen: 4.92 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Estimated average row size: 4 2018-01-05T11:29:11,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Stats for column key in table default@tab_part retrieved from cache 2018-01-05T11:29:11,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on TS[6] 2018-01-05T11:29:11,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-TS[6] (tab_part): numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {key= colName: key colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFBetween(Const boolean false, Column[value], DynamicValue(RS_10_c_value_min), DynamicValue(RS_10_c_value_max)) Original num rows: 242 New num rows: 242 2018-01-05T11:29:11,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[value], DynamicValue(RS_10_c_value_min), DynamicValue(RS_10_c_value_max)), GenericUDFInBloomFilter(Column[value], DynamicValue(RS_10_c_value_bloom_filter))) Original num rows: 242 New num rows: 242 2018-01-05T11:29:11,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_13_b_key_min), DynamicValue(RS_13_b_key_max)) Original num rows: 242 New num rows: 242 2018-01-05T11:29:11,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_13_b_key_min), DynamicValue(RS_13_b_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_13_b_key_bloom_filter))) Original num rows: 242 New num rows: 242 2018-01-05T11:29:11,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPNotNull(Column[value]), GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[value], DynamicValue(RS_10_c_value_min), DynamicValue(RS_10_c_value_max)), GenericUDFInBloomFilter(Column[value], DynamicValue(RS_10_c_value_bloom_filter))), GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_13_b_key_min), DynamicValue(RS_13_b_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_13_b_key_bloom_filter)))) Original num rows: 242 New num rows: 242 2018-01-05T11:29:11,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-FIL[26]: numRows: 242 dataSize: 22990 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {value= colName: value colType: string countDistincts: 153 numNulls: 0 avgColLen: 6.801652892561983 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, key= colName: key colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting stats (Num rows: 242 Data size: 22990 Basic stats: COMPLETE Column stats: COMPLETE) on FIL[26] 2018-01-05T11:29:11,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFBetween(Const boolean false, Column[value], DynamicValue(RS_9_a_value_min), DynamicValue(RS_9_a_value_max)) Original num rows: 25 New num rows: 25 2018-01-05T11:29:11,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[value], DynamicValue(RS_9_a_value_min), DynamicValue(RS_9_a_value_max)), GenericUDFInBloomFilter(Column[value], DynamicValue(RS_9_a_value_bloom_filter))) Original num rows: 25 New num rows: 25 2018-01-05T11:29:11,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFOPAnd(GenericUDFOPNotNull(Column[value]), GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[value], DynamicValue(RS_9_a_value_min), DynamicValue(RS_9_a_value_max)), GenericUDFInBloomFilter(Column[value], DynamicValue(RS_9_a_value_bloom_filter)))) Original num rows: 25 New num rows: 25 2018-01-05T11:29:11,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-FIL[27]: numRows: 25 dataSize: 2225 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {value= colName: value colType: string countDistincts: 19 numNulls: 0 avgColLen: 4.92 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting stats (Num rows: 25 Data size: 2225 Basic stats: COMPLETE Column stats: COMPLETE) on FIL[27] 2018-01-05T11:29:11,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_12_a_key_min), DynamicValue(RS_12_a_key_max)) Original num rows: 500 New num rows: 500 2018-01-05T11:29:11,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_12_a_key_min), DynamicValue(RS_12_a_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_12_a_key_bloom_filter))) Original num rows: 500 New num rows: 500 2018-01-05T11:29:11,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_12_a_key_min), DynamicValue(RS_12_a_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_12_a_key_bloom_filter)))) Original num rows: 500 New num rows: 500 2018-01-05T11:29:11,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-FIL[28]: numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {key= colName: key colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on FIL[28] 2018-01-05T11:29:11,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 242 Data size: 22990 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[2] 2018-01-05T11:29:11,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[2]: numRows: 242 dataSize: 22990 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: string countDistincts: 153 numNulls: 0 avgColLen: 6.801652892561983 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 25 Data size: 2225 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[5] 2018-01-05T11:29:11,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[5]: numRows: 25 dataSize: 2225 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: string countDistincts: 19 numNulls: 0 avgColLen: 4.92 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[8] 2018-01-05T11:29:11,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[8]: numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 242 Data size: 22990 Basic stats: COMPLETE Column stats: COMPLETE) on RS[9] 2018-01-05T11:29:11,083 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[9]: numRows: 242 dataSize: 22990 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {VALUE._col0= colName: VALUE._col0 colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false, KEY.reducesinkkey0= colName: KEY.reducesinkkey0 colType: string countDistincts: 153 numNulls: 0 avgColLen: 6.801652892561983 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 242 Data size: 22022 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[39] 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[39]: numRows: 242 dataSize: 22022 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col1= colName: _col1 colType: string countDistincts: 153 numNulls: 0 avgColLen: 6.801652892561983 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 25 Data size: 2225 Basic stats: COMPLETE Column stats: COMPLETE) on RS[10] 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[10]: numRows: 25 dataSize: 2225 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {KEY.reducesinkkey0= colName: KEY.reducesinkkey0 colType: string countDistincts: 19 numNulls: 0 avgColLen: 4.92 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 25 Data size: 2225 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[29] 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[29]: numRows: 25 dataSize: 2225 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: string countDistincts: 19 numNulls: 0 avgColLen: 4.92 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on RS[13] 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[13]: numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {KEY.reducesinkkey0= colName: KEY.reducesinkkey0 colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[34] 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[34]: numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-JOIN[11]: detects none/multiple PK parents. 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.JoinOperator: Setting stats (Num rows: 39 Data size: 156 Basic stats: COMPLETE Column stats: COMPLETE) on JOIN[11] 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-JOIN[11]: numRows: 39 dataSize: 156 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 25 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[40]: inputSize: 22990 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[40] hashAgg: true 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 3] STATS-GBY[40]: cardinality: 1 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 552 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[40] 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[40]: numRows: 1 dataSize: 552 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[30]: inputSize: 2225 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[30] hashAgg: true 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 3] STATS-GBY[30]: cardinality: 1 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 552 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[30] 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[30]: numRows: 1 dataSize: 552 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[35]: inputSize: 2000 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[35] hashAgg: true 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 3] STATS-GBY[35]: cardinality: 1 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[35] 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[35]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 39 Data size: 156 Basic stats: COMPLETE Column stats: COMPLETE) on RS[12] 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[12]: numRows: 39 dataSize: 156 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {KEY.reducesinkkey0= colName: KEY.reducesinkkey0 colType: int countDistincts: 25 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 39 Data size: 156 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[44] 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[44]: numRows: 39 dataSize: 156 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 25 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 552 Basic stats: COMPLETE Column stats: COMPLETE) on RS[41] 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[41]: numRows: 1 dataSize: 552 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 552 Basic stats: COMPLETE Column stats: COMPLETE) on RS[31] 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[31]: numRows: 1 dataSize: 552 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on RS[36] 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[36]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-JOIN[14]: detects none/multiple PK parents. 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.JoinOperator: Setting stats (Num rows: 63 Data size: 504 Basic stats: COMPLETE Column stats: COMPLETE) on JOIN[14] 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-JOIN[14]: numRows: 63 dataSize: 504 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[45]: inputSize: 156 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[45] hashAgg: true 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 3] STATS-GBY[45]: cardinality: 1 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[45] 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[45]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[42]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[42] hashAgg: false 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 9] STATS-GBY[42]: cardinality: 1 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 552 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[42] 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[42]: numRows: 1 dataSize: 552 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[32]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[32] hashAgg: false 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 9] STATS-GBY[32]: cardinality: 1 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 552 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[32] 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[32]: numRows: 1 dataSize: 552 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[37]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[37] hashAgg: false 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 9] STATS-GBY[37]: cardinality: 1 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[37] 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[37]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[16]: inputSize: 504 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[16] hashAgg: true 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 3] STATS-GBY[16]: cardinality: 1 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[16] 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[16]: numRows: 1 dataSize: 8 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on RS[46] 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[46]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:11,084 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 552 Basic stats: COMPLETE Column stats: COMPLETE) on RS[43] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[43]: numRows: 1 dataSize: 552 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 552 Basic stats: COMPLETE Column stats: COMPLETE) on RS[33] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[33]: numRows: 1 dataSize: 552 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on RS[38] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[38]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE) on RS[17] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[17]: numRows: 1 dataSize: 8 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {VALUE._col0= colName: VALUE._col0 colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[47]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[47] hashAgg: false 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 9] STATS-GBY[47]: cardinality: 1 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[47] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[47]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[18]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[18] hashAgg: false 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 9] STATS-GBY[18]: cardinality: 1 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[18] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[18]: numRows: 1 dataSize: 8 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on RS[48] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[48]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-FS[20]: numRows: 1 dataSize: 8 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[key]]; bucket count: 2 }) on TS[0] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on TS[3] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[key]]; bucket count: 4 }) on TS[6] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[key]]; bucket count: 2 }) on FIL[26] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on FIL[27] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[key]]; bucket count: 4 }) on FIL[28] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: 2 }) on SEL[2] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on SEL[5] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: 4 }) on SEL[8] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[VALUE._col0]]; sort column names: [[VALUE._col0]]; bucket count: 2 }) on RS[9] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: 2 }) on SEL[39] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[KEY.reducesinkkey0]]; sort column names: [[KEY.reducesinkkey0]]; bucket count: -1 }) on RS[10] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on SEL[29] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[KEY.reducesinkkey0]]; sort column names: [[KEY.reducesinkkey0]]; bucket count: 4 }) on RS[13] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: 4 }) on SEL[34] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.JoinOperator: Setting traits ({ bucket column names: [[_col0], []]; sort column names: [[_col0], []]; bucket count: -1 }) on JOIN[11] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[40] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[30] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[35] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[KEY.reducesinkkey0]]; sort column names: [[KEY.reducesinkkey0]]; bucket count: -1 }) on RS[12] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0], []]; sort column names: [[_col0], []]; bucket count: -1 }) on SEL[44] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[41] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[31] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[36] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.JoinOperator: Setting traits ({ bucket column names: [[], []]; sort column names: [[], []]; bucket count: -1 }) on JOIN[14] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[45] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[42] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[32] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[37] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[16] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[46] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[43] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[33] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[38] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[17] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[47] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[18] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[48] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FileSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on FS[20] 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,085 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Set parallelism for reduce sink RS[9] to: 1 2018-01-05T11:29:11,085 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Set parallelism for reduce sink RS[10] to: 1 2018-01-05T11:29:11,085 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Set parallelism for reduce sink RS[13] to: 1 2018-01-05T11:29:11,085 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConvertJoinMapJoin: Memory monitor info set to : { isLlap: true executorsPerNode: 4 maxExecutorsOverSubscribeMemory: 3 memoryOverSubscriptionFactor: 0.20000000298023224 memoryCheckInterval: 100000 noConditionalTaskSize: 10000000 adjustedNoConditionalTaskSize: 16000000 hashTableInflationFactor: 2.0 threshold: 32000000 } 2018-01-05T11:29:11,085 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConvertJoinMapJoin: Fallback to common merge join operator 2018-01-05T11:29:11,085 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.CommonMergeJoinOperator: Setting traits ({ bucket column names: [[_col0], []]; sort column names: [[_col0], []]; bucket count: 0 }) on MERGEJOIN[49] 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.CommonMergeJoinOperator: Setting stats (Num rows: 39 Data size: 156 Basic stats: COMPLETE Column stats: COMPLETE) on MERGEJOIN[49] 2018-01-05T11:29:11,086 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Set parallelism for reduce sink RS[12] to: 1 2018-01-05T11:29:11,086 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConvertJoinMapJoin: Memory monitor info set to : { isLlap: true executorsPerNode: 4 maxExecutorsOverSubscribeMemory: 3 memoryOverSubscriptionFactor: 0.20000000298023224 memoryCheckInterval: 100000 noConditionalTaskSize: 10000000 adjustedNoConditionalTaskSize: 16000000 hashTableInflationFactor: 2.0 threshold: 32000000 } 2018-01-05T11:29:11,086 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConvertJoinMapJoin: Fallback to common merge join operator 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.CommonMergeJoinOperator: Setting traits ({ bucket column names: [[], []]; sort column names: [[], []]; bucket count: 0 }) on MERGEJOIN[50] 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.CommonMergeJoinOperator: Setting stats (Num rows: 63 Data size: 504 Basic stats: COMPLETE Column stats: COMPLETE) on MERGEJOIN[50] 2018-01-05T11:29:11,086 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:11,086 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:11,086 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:11,086 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:11,086 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:11,086 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:11,086 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:11,086 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:11,086 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing BloomFilter cost/benefit for RS[48] - TS[6] (b) (Column[key]) 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing key domain cardinality, keyDomainCardinality=461, semiJoinKeyIsPK=false, selColStat= colName: _col0 colType: int countDistincts: 25 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false, selColSourceStat= colName: key colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false, tsColStat= colName: key colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit for Column[_col0] to Column[key], selKeyCardinality=25, tsKeyCardinality=308, tsRows=500, keyDomainCardinality=461 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: SemiJoin key selectivity=0.05422993492407809, benefit=472.8850325379609 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit=472.8850325379609, cost=39.0, tsDataSize=500, netBenefit=433.8850325379609 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: netBenefit=0.8677700650759218 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing BloomFilter cost/benefit for RS[33] - TS[0] (a) (Column[value]) 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing key domain cardinality, keyDomainCardinality=172, semiJoinKeyIsPK=false, selColStat= colName: _col0 colType: string countDistincts: 19 numNulls: 0 avgColLen: 4.92 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, selColSourceStat= colName: value colType: string countDistincts: 19 numNulls: 0 avgColLen: 4.92 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, tsColStat= colName: value colType: string countDistincts: 153 numNulls: 0 avgColLen: 6.801652892561983 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit for Column[_col0] to Column[value], selKeyCardinality=19, tsKeyCardinality=153, tsRows=242, keyDomainCardinality=172 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: SemiJoin key selectivity=0.11046511627906977, benefit=215.2674418604651 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit=215.2674418604651, cost=25.0, tsDataSize=242, netBenefit=190.2674418604651 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: netBenefit=0.7862290985969633 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing BloomFilter cost/benefit for RS[43] - TS[3] (c) (Column[value]) 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing key domain cardinality, keyDomainCardinality=172, semiJoinKeyIsPK=false, selColStat= colName: _col1 colType: string countDistincts: 153 numNulls: 0 avgColLen: 6.801652892561983 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, selColSourceStat= colName: value colType: string countDistincts: 153 numNulls: 0 avgColLen: 6.801652892561983 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, tsColStat= colName: value colType: string countDistincts: 19 numNulls: 0 avgColLen: 4.92 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit for Column[_col1] to Column[value], selKeyCardinality=153, tsKeyCardinality=19, tsRows=25, keyDomainCardinality=172 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: SemiJoin key selectivity=0.8895348837209303, benefit=2.7616279069767433 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit=2.7616279069767433, cost=242.0, tsDataSize=25, netBenefit=-239.23837209302326 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: netBenefit=-9.56953488372093 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing BloomFilter cost/benefit for RS[38] - TS[0] (a) (Column[key]) 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing key domain cardinality, keyDomainCardinality=461, semiJoinKeyIsPK=false, selColStat= colName: _col0 colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false, selColSourceStat= colName: key colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false, tsColStat= colName: key colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit for Column[_col0] to Column[key], selKeyCardinality=308, tsKeyCardinality=153, tsRows=242, keyDomainCardinality=461 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: SemiJoin key selectivity=0.6681127982646421, benefit=80.31670281995662 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit=80.31670281995662, cost=500.0, tsDataSize=242, netBenefit=-419.6832971800434 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: netBenefit=-1.7342285007439808 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Reduction factor not satisfied for RS[43]-TS[3] (c). Removing semijoin optimization. 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Removing ReduceSink RS[43] and TableScan TS[3] 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Reduction factor not satisfied for RS[38]-TS[0] (a). Removing semijoin optimization. 2018-01-05T11:29:11,086 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Removing ReduceSink RS[38] and TableScan TS[0] 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Adding special edge: RS --> TS[6] 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Adding special edge: RS --> TS[0] 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: FS, 20 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: GBY, 18 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 17 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: GBY, 16 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: MERGEJOIN, 50 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 12 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 13 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 8 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: FIL, 28 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: TS, 6 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 48 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: GBY, 47 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 46 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: GBY, 45 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 44 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: MERGEJOIN, 49 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 9 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 2 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: FIL, 26 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 10 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 33 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: GBY, 32 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 31 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: GBY, 30 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 29 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 5 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: FIL, 27 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: TS, 3 2018-01-05T11:29:11,087 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Cycle free: true 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: Before SharedWorkOptimizer: TS[0]-FIL[26]-SEL[2]-RS[9]-MERGEJOIN[49]-RS[12]-MERGEJOIN[50]-GBY[16]-RS[17]-GBY[18]-FS[20] -SEL[44]-GBY[45]-RS[46]-GBY[47]-RS[48] TS[3]-FIL[27]-SEL[5]-RS[10]-MERGEJOIN[49] -SEL[29]-GBY[30]-RS[31]-GBY[32]-RS[33] TS[6]-FIL[28]-SEL[8]-RS[13]-MERGEJOIN[50] 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: DPP information stored in the cache: {TS[0]=[RS[33]], TS[6]=[RS[48]]} 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: Sorted tables by size: [default.tab=22990, default.src1=2225, default.tab_part=2000] 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: After SharedWorkOptimizer: TS[0]-FIL[26]-SEL[2]-RS[9]-MERGEJOIN[49]-RS[12]-MERGEJOIN[50]-GBY[16]-RS[17]-GBY[18]-FS[20] -SEL[44]-GBY[45]-RS[46]-GBY[47]-RS[48] TS[3]-FIL[27]-SEL[5]-RS[10]-MERGEJOIN[49] -SEL[29]-GBY[30]-RS[31]-GBY[32]-RS[33] TS[6]-FIL[28]-SEL[8]-RS[13]-MERGEJOIN[50] 2018-01-05T11:29:11,087 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: int|{a}key,value: string|{a}value,ds: string|{a}ds,BLOCK__OFFSET__INSIDE__FILE: bigint|{a}block__offset__inside__file,INPUT__FILE__NAME: string|{a}input__file__name,ROW__ID: struct|{a}row__id) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(Const boolean true, Const boolean true) -> Const boolean true 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[3] with rs:(key: string|{c}key,value: string|{c}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{c}block__offset__inside__file,INPUT__FILE__NAME: string|{c}input__file__name,ROW__ID: struct|{c}row__id) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(Const boolean true, Const boolean true) -> Const boolean true 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(GenericUDFOPNotNull(Column[value]), GenericUDFOPAnd(Const boolean true, Const boolean true)) -> GenericUDFOPNotNull(Column[value]) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[6] with rs:(key: int|{b}key,value: string|{b}value,ds: string|{b}ds,BLOCK__OFFSET__INSIDE__FILE: bigint|{b}block__offset__inside__file,INPUT__FILE__NAME: string|{b}input__file__name,ROW__ID: struct|{b}row__id) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FIL[26] with rs:(key: int|{a}key,value: string|{a}value) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FIL[26] 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Old filter FIL[26] conditions:(key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter)))) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New filter FIL[26] conditions:(key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter)))) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FIL[27] with rs:(value: string|{c}value) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_3 {} 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FIL[27] 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Old filter FIL[27] conditions:(value is not null and (true and true)) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(Const boolean true, Const boolean true) -> Const boolean true 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(GenericUDFOPNotNull(Column[value]), GenericUDFOPAnd(Const boolean true, Const boolean true)) -> GenericUDFOPNotNull(Column[value]) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New filter FIL[27] conditions:value is not null 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FIL[28] with rs:(key: int|{b}key) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_6 {} 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FIL[28] 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Old filter FIL[28] conditions:(key is not null and (key BETWEEN DynamicValue(RS_12_a_key_min) AND DynamicValue(RS_12_a_key_max) and in_bloom_filter(key, DynamicValue(RS_12_a_key_bloom_filter)))) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New filter FIL[28] conditions:(key is not null and (key BETWEEN DynamicValue(RS_12_a_key_min) AND DynamicValue(RS_12_a_key_max) and in_bloom_filter(key, DynamicValue(RS_12_a_key_bloom_filter)))) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[2] with rs:(_col0: int|{$hdt$_1}key,_col1: string|{$hdt$_1}value) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op FIL_26 {} 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[2] 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value]) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: string|{$hdt$_2}value) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op FIL_27 {} 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[value]) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[8] with rs:(_col0: int|{$hdt$_0}key) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op FIL_28 {} 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[8] 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key]) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[9] with rs:(VALUE._col0: int|{$hdt$_1}key,KEY.reducesinkkey0: string|{$hdt$_1}value) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_2 {} 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[9] 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[10] with rs:(KEY.reducesinkkey0: string|{$hdt$_2}value) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[10] 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[29] with rs:(_col0: string|{$hdt$_2}value) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[29] 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0]) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[13] with rs:(KEY.reducesinkkey0: int|{$hdt$_0}key) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_8 {} 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[13] 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:MERGEJOIN[49] with rs:(_col0: int|{$hdt$_1}key) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_9 {} 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_10 {} 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator MERGEJOIN[49] 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[30] with rs:(_col0: string|{},_col1: string|{},_col2: string|{}) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_29 {} 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[30] 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[12] with rs:(KEY.reducesinkkey0: int|{$hdt$_1}key) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op MERGEJOIN_49 {} 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[12] 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[13] with rs:(KEY.reducesinkkey0: int|{$hdt$_0}key) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_8 {} 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[13] 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[44] with rs:(_col0: int|{$hdt$_1}key) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op MERGEJOIN_49 {} 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[44] 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0]) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[31] with rs:(_col0: string|{},_col1: string|{},_col2: string|{}) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_30 {} 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[31] 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:MERGEJOIN[50] with rs:() 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_12 {} 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_13 {} 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator MERGEJOIN[50] 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[45] with rs:(_col0: int|{},_col1: int|{},_col2: int|{}) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_44 {} 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[45] 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[32] with rs:(_col0: string|{},_col1: string|{},_col2: string|{}) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_31 {} 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[32] 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[16] with rs:(_col0: bigint|{}(tok_functionstar count)) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op MERGEJOIN_50 {} 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[16] 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[46] with rs:(_col0: int|{},_col1: int|{},_col2: int|{}) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_45 {} 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[46] 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[33] with rs:(_col0: string|{},_col1: string|{},_col2: string|{}) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_32 {} 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[33] 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[17] with rs:(VALUE._col0: bigint|{}(tok_functionstar count)) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_16 {} 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[17] 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[47] with rs:(_col0: int|{},_col1: int|{},_col2: int|{}) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_46 {} 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[47] 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[18] with rs:(_col0: bigint|{}(tok_functionstar count)) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_17 {} 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[18] 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[48] with rs:(_col0: int|{},_col1: int|{},_col2: int|{}) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_47 {} 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[48] 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[20] with rs:(_col0: bigint|{}) 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_18 {} 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[20] 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: TS[0] 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[9] 2018-01-05T11:29:11,088 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 of table a 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[9] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: MERGEJOIN[49] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[12] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding reduce work (Reducer 2) for MERGEJOIN[49] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Setting up reduce sink: RS[9] with following reduce work: Reducer 2 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[9] as parent from MERGEJOIN[49] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[10] as parent from MERGEJOIN[49] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[12] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: MERGEJOIN[50] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[17] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding reduce work (Reducer 3) for MERGEJOIN[50] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Setting up reduce sink: RS[12] with following reduce work: Reducer 3 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[12] as parent from MERGEJOIN[50] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[13] as parent from MERGEJOIN[50] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[17] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: GBY[18] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: FS[20] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding reduce work (Reducer 4) for GBY[18] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Setting up reduce sink: RS[17] with following reduce work: Reducer 4 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[17] as parent from GBY[18] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: FS[20] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: MERGEJOIN[49] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[46] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[46] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: GBY[47] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[48] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding reduce work (Reducer 5) for GBY[47] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Setting up reduce sink: RS[46] with following reduce work: Reducer 5 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[46] as parent from GBY[47] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[48] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: TS[3] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[10] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding map work (Map 6) for TS[3] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1 of table c 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Second pass. Leaf operator: RS[10] has common downstream work: org.apache.hadoop.hive.ql.plan.MergeJoinWork@500bc4cd 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: TS[3] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[31] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[31] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: GBY[32] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[33] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding reduce work (Reducer 7) for GBY[32] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Setting up reduce sink: RS[31] with following reduce work: Reducer 7 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[31] as parent from GBY[32] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[33] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: TS[6] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[13] 2018-01-05T11:29:11,089 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding map work (Map 8) for TS[6] 2018-01-05T11:29:11,090 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 of table b 2018-01-05T11:29:11,090 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:11,090 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Second pass. Leaf operator: RS[13] has common downstream work: org.apache.hadoop.hive.ql.plan.MergeJoinWork@79031858 2018-01-05T11:29:11,090 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: ResduceSink RS[48] to TableScan TS[6] 2018-01-05T11:29:11,090 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Connecting Baswork - Reducer 5 to Map 8 2018-01-05T11:29:11,090 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: ResduceSink RS[33] to TableScan TS[0] 2018-01-05T11:29:11,090 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Connecting Baswork - Reducer 7 to Map 1 2018-01-05T11:29:11,090 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:29:11,090 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,090 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,090 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 6 2018-01-05T11:29:11,090 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:11,090 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:11,090 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 6 2018-01-05T11:29:11,090 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:11,090 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:11,090 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:29:11,090 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Examining input format to see if vectorization is enabled. 2018-01-05T11:29:11,090 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorization is enabled for input format(s) [org.apache.hadoop.mapred.TextInputFormat] 2018-01-05T11:29:11,090 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing MapWork... 2018-01-05T11:29:11,090 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFOPNotNull(Column[value]), Vectorized Expression = SelectColumnIsNotNull(col 1:string) 2018-01-05T11:29:11,090 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator 2018-01-05T11:29:11,090 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.FilterDesc 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[value], Vectorized Expression = col 1:string 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.SelectDesc 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 1:string 2018-01-05T11:29:11,091 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator groupby typeName string 2018-01-05T11:29:11,091 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkStringOperator 2018-01-05T11:29:11,091 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkStringOperator 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 1:string 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.SelectDesc 2018-01-05T11:29:11,091 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vector GROUP BY operator will use processing mode HASH 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 1:string 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 1:string 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 1:string 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.GroupByDesc 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:string 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col1], Vectorized Expression = col 1:string 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col2], Vectorized Expression = col 2:binary 2018-01-05T11:29:11,091 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkEmptyKeyOperator 2018-01-05T11:29:11,091 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:11,091 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkEmptyKeyOperator: VectorReduceSinkEmptyKeyOperator constructor vectorReduceSinkInfo org.apache.hadoop.hive.ql.plan.VectorReduceSinkInfo@40a56901 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkEmptyKeyOperator 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [key, value, ROW__ID] 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [string, string, struct] 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE, NONE, NONE] 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 0 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 1 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Using reduce tag 0 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1, _col2] columnTypes=[string, string, binary] 2018-01-05T11:29:11,091 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing ReduceWork... 2018-01-05T11:29:11,091 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vector GROUP BY operator will use processing mode STREAMING 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[VALUE._col0], Vectorized Expression = col 0:string 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[VALUE._col1], Vectorized Expression = col 1:string 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[VALUE._col2], Vectorized Expression = col 2:binary 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.GroupByDesc 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:string 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col1], Vectorized Expression = col 1:string 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col2], Vectorized Expression = col 2:binary 2018-01-05T11:29:11,091 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkEmptyKeyOperator 2018-01-05T11:29:11,091 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:11,091 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkEmptyKeyOperator: VectorReduceSinkEmptyKeyOperator constructor vectorReduceSinkInfo org.apache.hadoop.hive.ql.plan.VectorReduceSinkInfo@2e224cd5 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkEmptyKeyOperator 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [VALUE._col0, VALUE._col1, VALUE._col2] 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [string, string, binary] 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE, NONE, NONE] 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 0 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 0 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:11,091 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:11,091 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Examining input format to see if vectorization is enabled. 2018-01-05T11:29:11,092 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorization is enabled for input format(s) [org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] 2018-01-05T11:29:11,092 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing MapWork... 2018-01-05T11:29:11,092 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFOPNotNull(Column[key]), Vectorized Expression = SelectColumnIsNotNull(col 0:int) 2018-01-05T11:29:11,092 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFOPNotNull(Column[value]), Vectorized Expression = SelectColumnIsNotNull(col 1:string) 2018-01-05T11:29:11,092 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFBetween(Const boolean false, Column[value], DynamicValue(RS_10_c_value_min), DynamicValue(RS_10_c_value_max)), Vectorized Expression = FilterStringColumnBetweenDynamicValue(col 1:string, left NULL, right NULL) 2018-01-05T11:29:11,092 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFInBloomFilter(Column[value], DynamicValue(RS_10_c_value_bloom_filter)), Vectorized Expression = VectorInBloomFilterColDynamicValue 2018-01-05T11:29:11,092 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[value], DynamicValue(RS_10_c_value_min), DynamicValue(RS_10_c_value_max)), GenericUDFInBloomFilter(Column[value], DynamicValue(RS_10_c_value_bloom_filter))), Vectorized Expression = FilterExprAndExpr(children: FilterStringColumnBetweenDynamicValue(col 1:string, left NULL, right NULL), VectorInBloomFilterColDynamicValue) 2018-01-05T11:29:11,092 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPNotNull(Column[value]), GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[value], DynamicValue(RS_10_c_value_min), DynamicValue(RS_10_c_value_max)), GenericUDFInBloomFilter(Column[value], DynamicValue(RS_10_c_value_bloom_filter)))), Vectorized Expression = FilterExprAndExpr(children: SelectColumnIsNotNull(col 0:int), SelectColumnIsNotNull(col 1:string), FilterExprAndExpr(children: FilterStringColumnBetweenDynamicValue(col 1:string, left NULL, right NULL), VectorInBloomFilterColDynamicValue)) 2018-01-05T11:29:11,092 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator 2018-01-05T11:29:11,092 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.FilterDesc 2018-01-05T11:29:11,092 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[key], Vectorized Expression = col 0:int 2018-01-05T11:29:11,092 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[value], Vectorized Expression = col 1:string 2018-01-05T11:29:11,092 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator 2018-01-05T11:29:11,092 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.SelectDesc 2018-01-05T11:29:11,092 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col1], Vectorized Expression = col 1:string 2018-01-05T11:29:11,092 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:int 2018-01-05T11:29:11,092 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator groupby typeName string 2018-01-05T11:29:11,092 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkStringOperator 2018-01-05T11:29:11,092 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:11,092 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkStringOperator 2018-01-05T11:29:11,092 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:11,092 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [key, value, ds, ROW__ID] 2018-01-05T11:29:11,092 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [int, string, string, struct] 2018-01-05T11:29:11,092 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE, NONE, NONE, NONE] 2018-01-05T11:29:11,092 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 1 2018-01-05T11:29:11,092 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 1 2018-01-05T11:29:11,092 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:11,092 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:11,092 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Using reduce tag 0 2018-01-05T11:29:11,092 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1, _col2] columnTypes=[int, int, binary] 2018-01-05T11:29:11,092 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing ReduceWork... 2018-01-05T11:29:11,092 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vector GROUP BY operator will use processing mode STREAMING 2018-01-05T11:29:11,092 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[VALUE._col0], Vectorized Expression = col 0:int 2018-01-05T11:29:11,092 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[VALUE._col1], Vectorized Expression = col 1:int 2018-01-05T11:29:11,092 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[VALUE._col2], Vectorized Expression = col 2:binary 2018-01-05T11:29:11,092 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator 2018-01-05T11:29:11,092 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.GroupByDesc 2018-01-05T11:29:11,092 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:int 2018-01-05T11:29:11,092 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col1], Vectorized Expression = col 1:int 2018-01-05T11:29:11,093 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col2], Vectorized Expression = col 2:binary 2018-01-05T11:29:11,093 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkEmptyKeyOperator 2018-01-05T11:29:11,093 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:11,093 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkEmptyKeyOperator: VectorReduceSinkEmptyKeyOperator constructor vectorReduceSinkInfo org.apache.hadoop.hive.ql.plan.VectorReduceSinkInfo@17aa498e 2018-01-05T11:29:11,093 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkEmptyKeyOperator 2018-01-05T11:29:11,093 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:11,093 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [VALUE._col0, VALUE._col1, VALUE._col2] 2018-01-05T11:29:11,093 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [int, int, binary] 2018-01-05T11:29:11,093 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE, NONE, NONE] 2018-01-05T11:29:11,093 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 0 2018-01-05T11:29:11,093 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 0 2018-01-05T11:29:11,093 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:11,093 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:11,093 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Examining input format to see if vectorization is enabled. 2018-01-05T11:29:11,093 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorization is enabled for input format(s) [org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] 2018-01-05T11:29:11,093 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing MapWork... 2018-01-05T11:29:11,093 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFOPNotNull(Column[key]), Vectorized Expression = SelectColumnIsNotNull(col 0:int) 2018-01-05T11:29:11,095 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_12_a_key_min), DynamicValue(RS_12_a_key_max)), Vectorized Expression = FilterLongColumnBetweenDynamicValue(col 0:int, left 0, right 0) 2018-01-05T11:29:11,095 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFInBloomFilter(Column[key], DynamicValue(RS_12_a_key_bloom_filter)), Vectorized Expression = VectorInBloomFilterColDynamicValue 2018-01-05T11:29:11,095 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_12_a_key_min), DynamicValue(RS_12_a_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_12_a_key_bloom_filter))), Vectorized Expression = FilterExprAndExpr(children: FilterLongColumnBetweenDynamicValue(col 0:int, left 0, right 0), VectorInBloomFilterColDynamicValue) 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_12_a_key_min), DynamicValue(RS_12_a_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_12_a_key_bloom_filter)))), Vectorized Expression = FilterExprAndExpr(children: SelectColumnIsNotNull(col 0:int), FilterExprAndExpr(children: FilterLongColumnBetweenDynamicValue(col 0:int, left 0, right 0), VectorInBloomFilterColDynamicValue)) 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.FilterDesc 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[key], Vectorized Expression = col 0:int 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.SelectDesc 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:int 2018-01-05T11:29:11,096 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator groupby typeName int 2018-01-05T11:29:11,096 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkLongOperator 2018-01-05T11:29:11,096 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkLongOperator 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [key, value, ds, ROW__ID] 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [int, string, string, struct] 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE, NONE, NONE, NONE] 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 1 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 1 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Using reduce tag 0 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0] columnTypes=[bigint] 2018-01-05T11:29:11,096 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing ReduceWork... 2018-01-05T11:29:11,096 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vector GROUP BY operator will use processing mode GLOBAL 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[VALUE._col0], Vectorized Expression = col 0:bigint 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.GroupByDesc 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorFileSinkOperator 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.FileSinkDesc 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [VALUE._col0] 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [bigint] 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE] 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 0 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 0 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:29:11,096 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: llap mode: only 2018-01-05T11:29:11,096 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Map 6 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'min(_col0)' 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '_col0' 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'max(_col0)' 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '_col0' 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'bloom_filter(_col0, expectedEntries=19)' 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '_col0' 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '_col0' 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'value' 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'value is not null' 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'value is not null' 2018-01-05T11:29:11,096 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@26e05464 can be done in LLAP 2018-01-05T11:29:11,096 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Reducer 7 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'min(VALUE._col0)' 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'max(VALUE._col1)' 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'VALUE._col1' 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'bloom_filter(VALUE._col2, expectedEntries=19)' 2018-01-05T11:29:11,096 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'VALUE._col2' 2018-01-05T11:29:11,096 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@509dc16a can be done in LLAP 2018-01-05T11:29:11,097 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'value' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter))))' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter))))' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter))))' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter))))' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter))))' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter))))' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter))))' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter))))' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter))))' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter))))' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter))))' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter))))' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter))))' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter))))' 2018-01-05T11:29:11,097 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@230cc272 can be done in LLAP 2018-01-05T11:29:11,097 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Reducer 2 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'min(_col0)' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '_col0' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'max(_col0)' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '_col0' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'bloom_filter(_col0, expectedEntries=25)' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '_col0' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '_col0' 2018-01-05T11:29:11,097 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MergeJoinWork@500bc4cd can be done in LLAP 2018-01-05T11:29:11,097 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Reducer 5 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'min(VALUE._col0)' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'max(VALUE._col1)' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'VALUE._col1' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'bloom_filter(VALUE._col2, expectedEntries=25)' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'VALUE._col2' 2018-01-05T11:29:11,097 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@47c0451c can be done in LLAP 2018-01-05T11:29:11,097 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Map 8 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_12_a_key_min) AND DynamicValue(RS_12_a_key_max) and in_bloom_filter(key, DynamicValue(RS_12_a_key_bloom_filter))))' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_12_a_key_min) AND DynamicValue(RS_12_a_key_max) and in_bloom_filter(key, DynamicValue(RS_12_a_key_bloom_filter))))' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_12_a_key_min) AND DynamicValue(RS_12_a_key_max) and in_bloom_filter(key, DynamicValue(RS_12_a_key_bloom_filter))))' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_12_a_key_min) AND DynamicValue(RS_12_a_key_max) and in_bloom_filter(key, DynamicValue(RS_12_a_key_bloom_filter))))' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_12_a_key_min) AND DynamicValue(RS_12_a_key_max) and in_bloom_filter(key, DynamicValue(RS_12_a_key_bloom_filter))))' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_12_a_key_min) AND DynamicValue(RS_12_a_key_max) and in_bloom_filter(key, DynamicValue(RS_12_a_key_bloom_filter))))' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_12_a_key_min) AND DynamicValue(RS_12_a_key_max) and in_bloom_filter(key, DynamicValue(RS_12_a_key_bloom_filter))))' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_12_a_key_min) AND DynamicValue(RS_12_a_key_max) and in_bloom_filter(key, DynamicValue(RS_12_a_key_bloom_filter))))' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_12_a_key_min) AND DynamicValue(RS_12_a_key_max) and in_bloom_filter(key, DynamicValue(RS_12_a_key_bloom_filter))))' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_12_a_key_min) AND DynamicValue(RS_12_a_key_max) and in_bloom_filter(key, DynamicValue(RS_12_a_key_bloom_filter))))' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_12_a_key_min) AND DynamicValue(RS_12_a_key_max) and in_bloom_filter(key, DynamicValue(RS_12_a_key_bloom_filter))))' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_12_a_key_min) AND DynamicValue(RS_12_a_key_max) and in_bloom_filter(key, DynamicValue(RS_12_a_key_bloom_filter))))' 2018-01-05T11:29:11,097 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@34660887 can be done in LLAP 2018-01-05T11:29:11,097 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Reducer 3 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'count()' 2018-01-05T11:29:11,097 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MergeJoinWork@79031858 can be done in LLAP 2018-01-05T11:29:11,097 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Reducer 4 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'count(VALUE._col0)' 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:29:11,097 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@327adcb2 can be done in LLAP 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.SerializeFilter: Serializing: value is not null 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.SerializeFilter: Serializing: (key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter)))) 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.SerializeFilter: Serializing: (key is not null and (key BETWEEN DynamicValue(RS_12_a_key_min) AND DynamicValue(RS_12_a_key_max) and in_bloom_filter(key, DynamicValue(RS_12_a_key_bloom_filter)))) 2018-01-05T11:29:11,097 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,098 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed plan generation 2018-01-05T11:29:11,098 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: validation start 2018-01-05T11:29:11,098 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:29:11,098 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:11,098 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,098 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:Explain, type:string, comment:null)], properties:null) 2018-01-05T11:29:11,098 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,098 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:11,098 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {getNotNullConstraints_(NotNullConstraintsRequest, )=3, isCompatibleWith_(Configuration, )=0, getAggrColStatsFor_(String, String, List, List, )=8, getTable_(String, String, )=20, flushCache_()=0, getUniqueConstraints_(UniqueConstraintsRequest, )=2, getPrimaryKeys_(PrimaryKeysRequest, )=2, listPartitions_(String, String, short, )=15, getTableColumnStatistics_(String, String, List, )=22, getForeignKeys_(ForeignKeysRequest, )=0} 2018-01-05T11:29:11,098 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112910_b90e584b-fd25-4300-aa00-d604e47c88c2); Time taken: 0.262 seconds 2018-01-05T11:29:11,098 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:11,098 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,098 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112910_b90e584b-fd25-4300-aa00-d604e47c88c2): explain vectorization detail select count(*) from tab a join tab_part b on a.key = b.key join src1 c on a.value = c.value 2018-01-05T11:29:11,099 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,099 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: explain vectorization detail select count(*) from tab a join tab_part b on a.key = b.key join src1 c on a.value = c.value 2018-01-05T11:29:11,099 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:29:11,099 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,099 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,099 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,099 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,099 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-3:EXPLAIN] in serial mode 2018-01-05T11:29:11,147 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,147 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,147 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: explain vectorization detail select count(*) from tab a join tab_part b on a.key = b.key join src1 c on a.value = c.value 2018-01-05T11:29:11,147 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: QUERY 2018-01-05T11:29:11,147 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,147 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,147 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:11,147 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {} 2018-01-05T11:29:11,147 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112910_b90e584b-fd25-4300-aa00-d604e47c88c2); Time taken: 0.049 seconds 2018-01-05T11:29:11,147 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:11,147 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,147 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:11,147 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,147 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query explain vectorization detail select count(*) from tab a join tab_part b on a.key = b.key join src1 c on a.value = c.value 2018-01-05T11:29:11,149 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting result file: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_836_4139621204064528450-1/-local-10004 2018-01-05T11:29:11,150 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_836_4139621204064528450-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-10_836_4139621204064528450-1 2018-01-05T11:29:11,150 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-10_836_4139621204064528450-1 2018-01-05T11:29:11,151 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.311 seconds, Fetched: 381 row(s) 2018-01-05T11:29:11,151 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:11,151 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:11,151 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:11,151 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:11,151 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,151 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:11,151 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,151 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,151 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,151 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: select count(*) from tab a join tab_part b on a.key = b.key join src1 c on a.value = c.value 2018-01-05T11:29:11,151 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112911_d60e4252-4a13-4247-a28a-f07830bbf022): select count(*) from tab a join tab_part b on a.key = b.key join src1 c on a.value = c.value 2018-01-05T11:29:11,151 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,151 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: select count(*) from tab a join tab_part b on a.key = b.key join src1 c on a.value = c.value 2018-01-05T11:29:11,152 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:11,152 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,152 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,152 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,152 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,152 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:11,152 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Starting Semantic Analysis 2018-01-05T11:29:11,152 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:11,153 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed phase 1 of Semantic Analysis 2018-01-05T11:29:11,153 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:11,153 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,153 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab 2018-01-05T11:29:11,153 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab 2018-01-05T11:29:11,157 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,158 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,159 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,159 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab_part 2018-01-05T11:29:11,159 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab_part 2018-01-05T11:29:11,162 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,163 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,163 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,163 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src1 2018-01-05T11:29:11,163 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src1 2018-01-05T11:29:11,165 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,166 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,166 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:11,166 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:11,179 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed getting MetaData in Semantic Analysis 2018-01-05T11:29:11,180 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,180 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,180 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_not_null_constraints : db=default tbl=tab 2018-01-05T11:29:11,180 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_not_null_constraints : db=default tbl=tab 2018-01-05T11:29:11,180 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,180 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,180 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,180 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab 2018-01-05T11:29:11,180 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab 2018-01-05T11:29:11,181 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,181 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,181 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,181 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab 2018-01-05T11:29:11,181 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab 2018-01-05T11:29:11,181 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,181 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,181 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,181 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_unique_constraints : db=default tbl=tab 2018-01-05T11:29:11,181 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_unique_constraints : db=default tbl=tab 2018-01-05T11:29:11,181 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,181 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,181 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,181 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab 2018-01-05T11:29:11,181 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab 2018-01-05T11:29:11,182 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,182 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,182 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,182 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_not_null_constraints : db=default tbl=tab_part 2018-01-05T11:29:11,182 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_not_null_constraints : db=default tbl=tab_part 2018-01-05T11:29:11,182 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,182 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,182 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,182 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:11,182 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:11,182 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,183 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,183 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,183 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:11,183 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=tab_part 2018-01-05T11:29:11,183 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,183 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,183 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,183 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_unique_constraints : db=default tbl=tab_part 2018-01-05T11:29:11,183 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_unique_constraints : db=default tbl=tab_part 2018-01-05T11:29:11,183 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,183 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,183 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,183 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab_part 2018-01-05T11:29:11,183 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=tab_part 2018-01-05T11:29:11,183 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,183 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,184 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,184 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_not_null_constraints : db=default tbl=src1 2018-01-05T11:29:11,184 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_not_null_constraints : db=default tbl=src1 2018-01-05T11:29:11,184 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,184 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,184 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,184 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=src1 2018-01-05T11:29:11,184 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=src1 2018-01-05T11:29:11,184 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,184 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,184 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,184 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_primary_keys : db=default tbl=src1 2018-01-05T11:29:11,184 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_primary_keys : db=default tbl=src1 2018-01-05T11:29:11,184 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,185 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,185 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,185 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_unique_constraints : db=default tbl=src1 2018-01-05T11:29:11,185 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_unique_constraints : db=default tbl=src1 2018-01-05T11:29:11,185 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,185 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,185 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,185 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=src1 2018-01-05T11:29:11,185 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default foreigntbl=src1 2018-01-05T11:29:11,185 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,185 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,186 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:11,186 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:11,186 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:11,186 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:29:11,186 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,188 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.sql2rel: Plan after trimming unused fields HiveAggregate(group=[{}], agg#0=[count()]) HiveProject(DUMMY=[0]) HiveJoin(condition=[=($1, $3)], joinType=[inner], algorithm=[none], cost=[not available]) HiveJoin(condition=[=($0, $2)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) HiveProject(value=[$1]) HiveTableScan(table=[[default.src1]], table:alias=[c]) 2018-01-05T11:29:11,188 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan before removing subquery: HiveProject(_o__c0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveProject($f0=[$0]) HiveJoin(condition=[=($1, $13)], joinType=[inner], algorithm=[none], cost=[not available]) HiveJoin(condition=[=($0, $6)], joinType=[inner], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) HiveTableScan(table=[[default.src1]], table:alias=[c]) 2018-01-05T11:29:11,189 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan just after removing subquery: HiveProject(_o__c0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveProject($f0=[$0]) HiveJoin(condition=[=($1, $13)], joinType=[inner], algorithm=[none], cost=[not available]) HiveJoin(condition=[=($0, $6)], joinType=[inner], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) HiveTableScan(table=[[default.src1]], table:alias=[c]) 2018-01-05T11:29:11,189 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Plan after decorrelation: HiveProject(_o__c0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveProject($f0=[$0]) HiveJoin(condition=[=($1, $13)], joinType=[inner], algorithm=[none], cost=[not available]) HiveJoin(condition=[=($0, $6)], joinType=[inner], algorithm=[none], cost=[not available]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) HiveTableScan(table=[[default.src1]], table:alias=[c]) 2018-01-05T11:29:11,189 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,189 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,189 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,189 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,189 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,190 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,190 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,190 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,190 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,190 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,190 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,205 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,205 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,206 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,206 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,206 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partitions : db=default tbl=tab 2018-01-05T11:29:11,206 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions : db=default tbl=tab 2018-01-05T11:29:11,209 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,210 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.167037ms + 0.004359ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? ] 2018-01-05T11:29:11,210 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.138266ms + 0.050011ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (15) order by "PART_NAME" asc] 2018-01-05T11:29:11,210 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.080625ms + 0.07471ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (15) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:29:11,210 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.070737ms + 0.018952ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (15) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:11,211 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.061226ms + 0.006473ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (43) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:29:11,211 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.059365ms + 0.019978ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (43) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:11,211 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.057593ms + 0.016143ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (43) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:11,211 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.063998ms + 0.008252ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (43) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:11,211 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.098034ms + 0.021564ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (28) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:11,211 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.072589ms + 0.023598ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (43) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:29:11,211 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,214 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,214 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,214 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,214 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,214 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,214 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_partitions : db=default tbl=tab_part 2018-01-05T11:29:11,214 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_partitions : db=default tbl=tab_part 2018-01-05T11:29:11,217 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,217 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.145742ms + 0.002785ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? ] 2018-01-05T11:29:11,217 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.124768ms + 0.043736ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (14) order by "PART_NAME" asc] 2018-01-05T11:29:11,218 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.136505ms + 0.068449ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (14) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:29:11,218 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.069399ms + 0.018731ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (14) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:11,218 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.058808ms + 0.005978ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (41) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:29:11,218 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.061568ms + 0.019785ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (41) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:11,218 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.056319ms + 0.015823ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (41) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:11,218 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.05245ms + 0.005462ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (41) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:11,218 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.079878ms + 0.020209ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (26) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:11,218 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.062403ms + 0.018935ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (41) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:29:11,218 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,221 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,221 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,221 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,221 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,221 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,221 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,221 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,231 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Estimated average row size: 104 2018-01-05T11:29:11,231 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Stats for column key in table default@tab could not be retrieved from cache 2018-01-05T11:29:11,231 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Stats for column value in table default@tab could not be retrieved from cache 2018-01-05T11:29:11,231 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,231 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_aggr_stats_for: db=default table=tab 2018-01-05T11:29:11,231 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_aggr_stats_for: db=default table=tab 2018-01-05T11:29:11,233 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.AggregateStatsCache: Returning aggregate stats from the cache; total hits: 22, total misses: 0, hit ratio: 1.0 2018-01-05T11:29:11,233 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.AggregateStatsCache: Returning aggregate stats from the cache; total hits: 23, total misses: 0, hit ratio: 1.0 2018-01-05T11:29:11,233 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: useDensityFunctionForNDVEstimation = false partsFound = 1 ColumnStatisticsObj = [ColumnStatisticsObj(colName:key, colType:int, statsData:), ColumnStatisticsObj(colName:value, colType:string, statsData:)] 2018-01-05T11:29:11,234 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,234 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,234 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.RelOptHiveTable: Stats for column key in table tab stored in cache 2018-01-05T11:29:11,234 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.RelOptHiveTable: colName: key colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false 2018-01-05T11:29:11,234 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.RelOptHiveTable: Stats for column value in table tab stored in cache 2018-01-05T11:29:11,234 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.RelOptHiveTable: colName: value colType: string countDistincts: 153 numNulls: 0 avgColLen: 6.801652892561983 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false 2018-01-05T11:29:11,234 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] rules.HiveRelFieldTrimmer: Got col stats for [0, 1] in [default.tab] 2018-01-05T11:29:11,235 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Estimated average row size: 4 2018-01-05T11:29:11,235 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Stats for column key in table default@tab_part could not be retrieved from cache 2018-01-05T11:29:11,235 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,235 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_aggr_stats_for: db=default table=tab_part 2018-01-05T11:29:11,235 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_aggr_stats_for: db=default table=tab_part 2018-01-05T11:29:11,237 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.AggregateStatsCache: Returning aggregate stats from the cache; total hits: 24, total misses: 0, hit ratio: 1.0 2018-01-05T11:29:11,237 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: useDensityFunctionForNDVEstimation = false partsFound = 1 ColumnStatisticsObj = [ColumnStatisticsObj(colName:key, colType:int, statsData:)] 2018-01-05T11:29:11,238 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,239 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,239 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.RelOptHiveTable: Stats for column key in table tab_part stored in cache 2018-01-05T11:29:11,239 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.RelOptHiveTable: colName: key colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false 2018-01-05T11:29:11,239 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] rules.HiveRelFieldTrimmer: Got col stats for [0] in [default.tab_part] 2018-01-05T11:29:11,240 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,240 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=src1 2018-01-05T11:29:11,240 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=src1 2018-01-05T11:29:11,242 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.12574ms + 0.003203ms, the query is [select "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", "BIT_VECTOR", "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "TAB_COL_STATS" where "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (...)] 2018-01-05T11:29:11,243 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,246 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,246 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,246 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.RelOptHiveTable: Stats for column value in table src1 stored in cache 2018-01-05T11:29:11,246 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.RelOptHiveTable: colName: value colType: string countDistincts: 19 numNulls: 0 avgColLen: 4.92 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false 2018-01-05T11:29:11,246 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] rules.HiveRelFieldTrimmer: Got col stats for [1] in [default.src1] 2018-01-05T11:29:11,254 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] calcite.sql2rel: Plan after trimming unused fields HiveAggregate(group=[{}], agg#0=[count()]) HiveProject(DUMMY=[0]) HiveJoin(condition=[=($1, $3)], joinType=[inner], algorithm=[none], cost=[not available]) HiveJoin(condition=[=($0, $2)], joinType=[inner], algorithm=[none], cost=[not available]) HiveFilter(condition=[AND(IS NOT NULL($0), IS NOT NULL($1))]) HiveProject(key=[$0], value=[$1]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveFilter(condition=[IS NOT NULL($0)]) HiveProject(key=[$0]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) HiveFilter(condition=[IS NOT NULL($0)]) HiveProject(value=[$1]) HiveTableScan(table=[[default.src1]], table:alias=[c]) 2018-01-05T11:29:11,254 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,254 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,255 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,255 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,256 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,256 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,285 DEBUG [AMReporterQueueDrainer] impl.AMReporter: Removing am 10.22.8.182:49824 with last associated dag QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=31} from heartbeat with taskCount=0, amFailed=false, isDone=true 2018-01-05T11:29:11,315 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,315 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,316 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,316 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,316 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,316 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,317 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,317 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,317 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,317 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,317 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,318 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Original plan for PlanModifier HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($1, $0)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) HiveJoin(condition=[=($1, $2)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1]) HiveFilter(condition=[AND(IS NOT NULL($0), IS NOT NULL($1))]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(value=[$1]) HiveFilter(condition=[IS NOT NULL($1)]) HiveTableScan(table=[[default.src1]], table:alias=[c]) 2018-01-05T11:29:11,318 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after top-level introduceDerivedTable HiveProject($f0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($1, $0)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) HiveJoin(condition=[=($1, $2)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1]) HiveFilter(condition=[AND(IS NOT NULL($0), IS NOT NULL($1))]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(value=[$1]) HiveFilter(condition=[IS NOT NULL($1)]) HiveTableScan(table=[[default.src1]], table:alias=[c]) 2018-01-05T11:29:11,318 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after nested convertOpTree HiveProject($f0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($1, $0)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) HiveJoin(condition=[=($1, $2)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1]) HiveFilter(condition=[AND(IS NOT NULL($0), IS NOT NULL($1))]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(value=[$1]) HiveFilter(condition=[IS NOT NULL($1)]) HiveTableScan(table=[[default.src1]], table:alias=[c]) 2018-01-05T11:29:11,318 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after propagating order HiveProject($f0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($1, $0)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) HiveJoin(condition=[=($1, $2)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1]) HiveFilter(condition=[AND(IS NOT NULL($0), IS NOT NULL($1))]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(value=[$1]) HiveFilter(condition=[IS NOT NULL($1)]) HiveTableScan(table=[[default.src1]], table:alias=[c]) 2018-01-05T11:29:11,319 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Plan after fixTopOBSchema HiveProject($f0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($1, $0)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) HiveJoin(condition=[=($1, $2)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1]) HiveFilter(condition=[AND(IS NOT NULL($0), IS NOT NULL($1))]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(value=[$1]) HiveFilter(condition=[IS NOT NULL($1)]) HiveTableScan(table=[[default.src1]], table:alias=[c]) 2018-01-05T11:29:11,319 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] translator.PlanModifierForASTConv: Final plan after modifier HiveProject(_c0=[$0]) HiveAggregate(group=[{}], agg#0=[count()]) HiveJoin(condition=[=($1, $0)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0]) HiveFilter(condition=[IS NOT NULL($0)]) HiveTableScan(table=[[default.tab_part]], table:alias=[b]) HiveJoin(condition=[=($1, $2)], joinType=[inner], algorithm=[none], cost=[not available]) HiveProject(key=[$0], value=[$1]) HiveFilter(condition=[AND(IS NOT NULL($0), IS NOT NULL($1))]) HiveTableScan(table=[[default.tab]], table:alias=[a]) HiveProject(value=[$1]) HiveFilter(condition=[IS NOT NULL($1)]) HiveTableScan(table=[[default.src1]], table:alias=[c]) 2018-01-05T11:29:11,319 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:11,319 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:11,319 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:11,319 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:11,319 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,319 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab 2018-01-05T11:29:11,319 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab 2018-01-05T11:29:11,322 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,322 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,322 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:11,322 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:11,322 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:11,322 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,322 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src1 2018-01-05T11:29:11,322 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src1 2018-01-05T11:29:11,325 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,325 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,325 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:11,325 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:11,325 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for source tables 2018-01-05T11:29:11,325 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,325 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab_part 2018-01-05T11:29:11,325 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab_part 2018-01-05T11:29:11,328 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,329 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,329 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for subqueries 2018-01-05T11:29:11,329 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:11,329 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Get metadata for destination tables 2018-01-05T11:29:11,330 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Table Plan for a TS[0] 2018-01-05T11:29:11,330 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Filter Plan for $hdt$_1 row schema: a{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:11,330 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB a{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB a{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:11,330 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (. (tok_table_or_col a) key) key) (tok_selexpr (. (tok_table_or_col a) value) value)) 2018-01-05T11:29:11,330 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = a{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} starRr = null 2018-01-05T11:29:11,330 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(key,_col0: int)(value,_col1: string)} 2018-01-05T11:29:11,330 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:11,330 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block $hdt$_1 2018-01-05T11:29:11,330 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block $hdt$_1 2018-01-05T11:29:11,330 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Table Plan for c TS[3] 2018-01-05T11:29:11,330 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Filter Plan for $hdt$_2 row schema: c{(key,key: string)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:11,330 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB c{(key,key: string)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB c{(key,key: string)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:11,330 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (. (tok_table_or_col c) value) value)) 2018-01-05T11:29:11,330 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = c{(key,key: string)(value,value: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} starRr = null 2018-01-05T11:29:11,330 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(value,_col0: string)} 2018-01-05T11:29:11,330 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:11,330 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block $hdt$_2 2018-01-05T11:29:11,330 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block $hdt$_2 2018-01-05T11:29:11,331 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Table Plan for b TS[6] 2018-01-05T11:29:11,331 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Filter Plan for $hdt$_0 row schema: b{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:11,331 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB b{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} after GB b{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} 2018-01-05T11:29:11,331 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (. (tok_table_or_col b) key) key)) 2018-01-05T11:29:11,331 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = b{(key,key: int)(value,value: string)(ds,ds: string)(block__offset__inside__file,BLOCK__OFFSET__INSIDE__FILE: bigint)(input__file__name,INPUT__FILE__NAME: string)(row__id,ROW__ID: struct)} starRr = null 2018-01-05T11:29:11,331 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(key,_col0: int)} 2018-01-05T11:29:11,331 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:11,331 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block $hdt$_0 2018-01-05T11:29:11,331 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block $hdt$_0 2018-01-05T11:29:11,332 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: count 2018-01-05T11:29:11,332 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: RR before GB $hdt$_1{(key,_col0: int)(value,_col1: string)} $hdt$_2{(value,_col2: string)} $hdt$_0{(key,_col3: int)} after GB {((tok_functionstar count),_col0: bigint)} 2018-01-05T11:29:11,332 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: tree: (tok_select (tok_selexpr (tok_functionstar count) _c0)) 2018-01-05T11:29:11,332 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: genSelectPlan: input = {((tok_functionstar count),_col0: bigint)} starRr = $hdt$_1{(key,_col0: int)(value,_col1: string)} $hdt$_2{(value,_col2: string)} $hdt$_0{(key,_col3: int)} 2018-01-05T11:29:11,332 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan row schema: null{(_c0,_col0: bigint)} 2018-01-05T11:29:11,332 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Select Plan for clause: insclause-0 2018-01-05T11:29:11,332 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Created staging dir = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-11_151_1086992364664144340-1 for path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/-mr-10001 2018-01-05T11:29:11,332 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.FileUtils: Creating directory if it doesn't exist: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-11_151_1086992364664144340-1 2018-01-05T11:29:11,357 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Set stats collection dir : file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/-mr-10003 2018-01-05T11:29:11,357 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created FileSink Plan for clause: insclause-0dest_path: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/-mr-10001 row schema: null{(_c0,_col0: bigint)} 2018-01-05T11:29:11,357 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Body Plan for Query Block null 2018-01-05T11:29:11,357 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Created Plan for Query Block null 2018-01-05T11:29:11,357 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: CBO Succeeded; optimized logical plan. 2018-01-05T11:29:11,357 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Before logical optimization TS[0]-FIL[1]-SEL[2]-RS[9]-JOIN[11]-RS[12]-JOIN[14]-SEL[15]-GBY[16]-RS[17]-GBY[18]-SEL[19]-FS[20] TS[3]-FIL[4]-SEL[5]-RS[10]-JOIN[11] TS[6]-FIL[7]-SEL[8]-RS[13]-JOIN[14] 2018-01-05T11:29:11,357 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,357 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,357 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,359 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] lineage.Generator: Time taken for lineage transform=2 2018-01-05T11:29:11,359 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,359 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,359 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.PartitionColumnsSeparator: Partition columns not separated for null, is not IN operator : 2018-01-05T11:29:11,359 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.PartitionColumnsSeparator: Partition columns not separated for null, is not IN operator : 2018-01-05T11:29:11,359 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.PartitionColumnsSeparator: Partition columns not separated for null, is not IN operator : 2018-01-05T11:29:11,359 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.PartitionColumnsSeparator: Partition columns not separated for null, is not IN operator : 2018-01-05T11:29:11,359 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.PartitionColumnsSeparator: Partition columns not separated for null, is not IN operator : 2018-01-05T11:29:11,359 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,359 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,359 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SyntheticJoinPredicate: Synthetic predicate: 0 --> 1 2018-01-05T11:29:11,359 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SyntheticJoinPredicate: Synthetic predicate: 0 --> 1 2018-01-05T11:29:11,359 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SyntheticJoinPredicate: Synthetic predicate: 1 --> 0 2018-01-05T11:29:11,359 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SyntheticJoinPredicate: Synthetic predicate: 0 --> 1 2018-01-05T11:29:11,359 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SyntheticJoinPredicate: Synthetic predicate: 1 --> 0 2018-01-05T11:29:11,359 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,360 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,360 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FS(20) 2018-01-05T11:29:11,360 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(19) 2018-01-05T11:29:11,360 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for GBY(18) 2018-01-05T11:29:11,360 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(17) 2018-01-05T11:29:11,360 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for GBY(16) 2018-01-05T11:29:11,360 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(15) 2018-01-05T11:29:11,360 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for JOIN(14) 2018-01-05T11:29:11,360 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(12) 2018-01-05T11:29:11,360 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(24) 2018-01-05T11:29:11,360 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias $hdt$_1: (_col0) IN (RS[13]) 2018-01-05T11:29:11,360 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(22) 2018-01-05T11:29:11,360 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias $hdt$_1: (_col0) IN (RS[13]) 2018-01-05T11:29:11,360 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias $hdt$_1: (_col0) IN (RS[13]) 2018-01-05T11:29:11,360 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for JOIN(11) 2018-01-05T11:29:11,360 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of JOIN for alias $hdt$_1: (VALUE._col0) IN (RS[13]) 2018-01-05T11:29:11,360 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(9) 2018-01-05T11:29:11,360 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of RS for alias $hdt$_1: (_col0) IN (RS[13]) 2018-01-05T11:29:11,360 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(21) 2018-01-05T11:29:11,360 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias $hdt$_1: (_col1) IN (RS[10]) 2018-01-05T11:29:11,360 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias $hdt$_1: (_col0) IN (RS[13]) 2018-01-05T11:29:11,360 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(2) 2018-01-05T11:29:11,360 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of SEL for alias $hdt$_1: (value) IN (RS[10]); (key) IN (RS[13]) 2018-01-05T11:29:11,360 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(1) 2018-01-05T11:29:11,360 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias a: key is not null; value is not null 2018-01-05T11:29:11,360 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias a: (value) IN (RS[10]); (key) IN (RS[13]) 2018-01-05T11:29:11,360 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for TS(0) 2018-01-05T11:29:11,360 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of TS for alias a: key is not null; value is not null; (value) IN (RS[10]); (key) IN (RS[13]) 2018-01-05T11:29:11,361 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(10) 2018-01-05T11:29:11,361 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(23) 2018-01-05T11:29:11,361 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias $hdt$_2: (_col0) IN (RS[9]) 2018-01-05T11:29:11,361 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(5) 2018-01-05T11:29:11,361 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of SEL for alias $hdt$_2: (value) IN (RS[9]) 2018-01-05T11:29:11,361 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(4) 2018-01-05T11:29:11,361 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias c: value is not null 2018-01-05T11:29:11,361 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias c: (value) IN (RS[9]) 2018-01-05T11:29:11,361 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for TS(3) 2018-01-05T11:29:11,361 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of TS for alias c: value is not null; (value) IN (RS[9]) 2018-01-05T11:29:11,361 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for RS(13) 2018-01-05T11:29:11,361 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(25) 2018-01-05T11:29:11,361 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias $hdt$_0: (_col0) IN (RS[12]) 2018-01-05T11:29:11,361 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for SEL(8) 2018-01-05T11:29:11,361 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of SEL for alias $hdt$_0: (key) IN (RS[12]) 2018-01-05T11:29:11,361 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for FIL(7) 2018-01-05T11:29:11,361 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias b: key is not null 2018-01-05T11:29:11,361 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of FIL for alias b: (key) IN (RS[12]) 2018-01-05T11:29:11,361 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Processing for TS(6) 2018-01-05T11:29:11,361 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.OpProcFactory: Pushdown predicates of TS for alias b: key is not null; (key) IN (RS[12]) 2018-01-05T11:29:11,362 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ppd.SimplePredicatePushDown: After PPD: TS[0]-FIL[26]-SEL[2]-RS[9]-JOIN[11]-RS[12]-JOIN[14]-SEL[15]-GBY[16]-RS[17]-GBY[18]-SEL[19]-FS[20] TS[3]-FIL[27]-SEL[5]-RS[10]-JOIN[11] TS[6]-FIL[28]-SEL[8]-RS[13]-JOIN[14] 2018-01-05T11:29:11,362 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,362 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,362 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,362 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,362 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,362 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,362 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,362 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,362 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pcr.PcrExprProcFactory: Processing key is not null org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNotNull outputs {UNKNOWN, Column[key]}, 2018-01-05T11:29:11,362 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pcr.PcrExprProcFactory: Processing value is not null org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNotNull outputs {UNKNOWN, Column[value]}, 2018-01-05T11:29:11,362 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pcr.PcrExprProcFactory: Processing (value) IN (RS[10]) org.apache.hadoop.hive.ql.udf.generic.GenericUDFIn outputs {UNKNOWN, Column[value]}, {UNKNOWN, RS[10]}, 2018-01-05T11:29:11,362 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pcr.PcrExprProcFactory: Processing (key) IN (RS[13]) org.apache.hadoop.hive.ql.udf.generic.GenericUDFIn outputs {UNKNOWN, Column[key]}, {UNKNOWN, RS[13]}, 2018-01-05T11:29:11,362 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pcr.PcrExprProcFactory: Processing (key is not null and value is not null and (value) IN (RS[10]) and (key) IN (RS[13])) org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPAnd outputs {UNKNOWN, GenericUDFOPNotNull(Column[key])}, {UNKNOWN, GenericUDFOPNotNull(Column[value])}, {UNKNOWN, GenericUDFIn(Column[value], RS[10])}, {UNKNOWN, GenericUDFIn(Column[key], RS[13])}, 2018-01-05T11:29:11,362 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pcr.PcrExprProcFactory: Processing key is not null org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPNotNull outputs {UNKNOWN, Column[key]}, 2018-01-05T11:29:11,362 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pcr.PcrExprProcFactory: Processing (key) IN (RS[12]) org.apache.hadoop.hive.ql.udf.generic.GenericUDFIn outputs {UNKNOWN, Column[key]}, {UNKNOWN, RS[12]}, 2018-01-05T11:29:11,362 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pcr.PcrExprProcFactory: Processing (key is not null and (key) IN (RS[12])) org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPAnd outputs {UNKNOWN, GenericUDFOPNotNull(Column[key])}, {UNKNOWN, GenericUDFIn(Column[key], RS[12])}, 2018-01-05T11:29:11,362 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,362 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,362 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,362 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,362 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 17 key:[] 2018-01-05T11:29:11,362 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 17 oldColExprMap: {VALUE._col0=Column[_col0]} 2018-01-05T11:29:11,362 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 17 newColExprMap: {VALUE._col0=Column[_col0]} 2018-01-05T11:29:11,362 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: JOIN 14 oldExprs: {0=[Column[KEY.reducesinkkey0], Column[VALUE._col0], Column[VALUE._col1]], 1=[Column[KEY.reducesinkkey0]]} 2018-01-05T11:29:11,363 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: JOIN 14 newExprs: {0=[], 1=[]} 2018-01-05T11:29:11,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 12 key:[Column[_col0]] 2018-01-05T11:29:11,363 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 12 oldColExprMap: {VALUE._col0=Column[_col1], VALUE._col1=Column[_col2], KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:11,363 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 12 newColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:11,363 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: JOIN 11 oldExprs: {0=[Column[VALUE._col0], Column[KEY.reducesinkkey0]], 1=[Column[KEY.reducesinkkey0]]} 2018-01-05T11:29:11,363 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 12 oldColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:11,363 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 12 newColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:11,363 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: JOIN 11 newExprs: {0=[Column[VALUE._col0]], 1=[]} 2018-01-05T11:29:11,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 13 key:[Column[_col0]] 2018-01-05T11:29:11,363 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 13 oldColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:11,363 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 13 newColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:11,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 9 key:[Column[_col1]] 2018-01-05T11:29:11,363 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 9 oldColExprMap: {VALUE._col0=Column[_col0], KEY.reducesinkkey0=Column[_col1]} 2018-01-05T11:29:11,363 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 9 newColExprMap: {VALUE._col0=Column[_col0], KEY.reducesinkkey0=Column[_col1]} 2018-01-05T11:29:11,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: Reduce Sink Operator 10 key:[Column[_col0]] 2018-01-05T11:29:11,363 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 10 oldColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:11,363 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ColumnPrunerProcFactory: RS 10 newColExprMap: {KEY.reducesinkkey0=Column[_col0]} 2018-01-05T11:29:11,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.IdentityProjectRemover: Identity project remover optimization removed : SEL[19] 2018-01-05T11:29:11,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.IdentityProjectRemover: Identity project remover optimization removed : SEL[15] 2018-01-05T11:29:11,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: After logical optimization TS[0]-FIL[26]-SEL[2]-RS[9]-JOIN[11]-RS[12]-JOIN[14]-GBY[16]-RS[17]-GBY[18]-FS[20] TS[3]-FIL[27]-SEL[5]-RS[10]-JOIN[11] TS[6]-FIL[28]-SEL[8]-RS[13]-JOIN[14] 2018-01-05T11:29:11,363 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,364 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Parent: TS[0] 2018-01-05T11:29:11,364 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Filter: (key is not null and value is not null and (value) IN (RS[10]) and (key) IN (RS[13])) (type: boolean) 2018-01-05T11:29:11,364 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: TableScan: TS[0] 2018-01-05T11:29:11,364 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Column value is not a partition column 2018-01-05T11:29:11,364 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Initiate semijoin reduction for value ((key is not null and value is not null and (value) IN (RS[10]) and (key) IN (RS[13])) 2018-01-05T11:29:11,364 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:11,364 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:11,364 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:11,364 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:11,364 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:11,364 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:11,364 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: DynamicSemiJoinPushdown: Saving RS to TS mapping: RS[33]: TS[0] 2018-01-05T11:29:11,364 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Column key is not a partition column 2018-01-05T11:29:11,364 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Initiate semijoin reduction for key ((key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter))) and (key) IN (RS[13])) 2018-01-05T11:29:11,364 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:11,364 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:11,364 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:11,364 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:11,364 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:11,364 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:11,364 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: DynamicSemiJoinPushdown: Saving RS to TS mapping: RS[38]: TS[0] 2018-01-05T11:29:11,364 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Parent: TS[3] 2018-01-05T11:29:11,364 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Filter: (value is not null and (value) IN (RS[9])) (type: boolean) 2018-01-05T11:29:11,364 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: TableScan: TS[3] 2018-01-05T11:29:11,364 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Column value is not a partition column 2018-01-05T11:29:11,364 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Initiate semijoin reduction for value ((value is not null and (value) IN (RS[9])) 2018-01-05T11:29:11,364 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:11,364 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:11,364 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:11,365 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:11,365 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:11,365 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:11,365 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: DynamicSemiJoinPushdown: Saving RS to TS mapping: RS[43]: TS[3] 2018-01-05T11:29:11,365 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Parent: TS[6] 2018-01-05T11:29:11,365 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Filter: (key is not null and (key) IN (RS[12])) (type: boolean) 2018-01-05T11:29:11,365 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: TableScan: TS[6] 2018-01-05T11:29:11,365 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Column key is not a partition column 2018-01-05T11:29:11,365 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: Initiate semijoin reduction for key ((key is not null and (key) IN (RS[12])) 2018-01-05T11:29:11,365 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:11,365 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:11,365 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:11,365 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: min 2018-01-05T11:29:11,365 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: max 2018-01-05T11:29:11,365 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FunctionRegistry: Looking up GenericUDAF: bloom_filter 2018-01-05T11:29:11,365 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.DynamicPartitionPruningOptimization: DynamicSemiJoinPushdown: Saving RS to TS mapping: RS[48]: TS[6] 2018-01-05T11:29:11,365 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,365 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,365 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Estimated average row size: 104 2018-01-05T11:29:11,365 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Stats for column key in table default@tab retrieved from cache 2018-01-05T11:29:11,365 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Stats for column value in table default@tab retrieved from cache 2018-01-05T11:29:11,365 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting stats (Num rows: 242 Data size: 22990 Basic stats: COMPLETE Column stats: COMPLETE) on TS[0] 2018-01-05T11:29:11,365 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-TS[0] (tab): numRows: 242 dataSize: 22990 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {value= colName: value colType: string countDistincts: 153 numNulls: 0 avgColLen: 6.801652892561983 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, key= colName: key colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,366 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,366 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table_statistics_req: db=default table=src1 2018-01-05T11:29:11,366 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table_statistics_req: db=default table=src1 2018-01-05T11:29:11,368 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,371 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,371 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:11,371 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Stats for column value in table default@src1 retrieved from cache 2018-01-05T11:29:11,371 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting stats (Num rows: 25 Data size: 2225 Basic stats: COMPLETE Column stats: COMPLETE) on TS[3] 2018-01-05T11:29:11,371 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-TS[3] (src1): numRows: 25 dataSize: 2225 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {value= colName: value colType: string countDistincts: 19 numNulls: 0 avgColLen: 4.92 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,371 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Estimated average row size: 4 2018-01-05T11:29:11,371 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] stats.StatsUtils: Stats for column key in table default@tab_part retrieved from cache 2018-01-05T11:29:11,371 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on TS[6] 2018-01-05T11:29:11,371 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-TS[6] (tab_part): numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {key= colName: key colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,371 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFBetween(Const boolean false, Column[value], DynamicValue(RS_10_c_value_min), DynamicValue(RS_10_c_value_max)) Original num rows: 242 New num rows: 242 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[value], DynamicValue(RS_10_c_value_min), DynamicValue(RS_10_c_value_max)), GenericUDFInBloomFilter(Column[value], DynamicValue(RS_10_c_value_bloom_filter))) Original num rows: 242 New num rows: 242 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_13_b_key_min), DynamicValue(RS_13_b_key_max)) Original num rows: 242 New num rows: 242 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_13_b_key_min), DynamicValue(RS_13_b_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_13_b_key_bloom_filter))) Original num rows: 242 New num rows: 242 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPNotNull(Column[value]), GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[value], DynamicValue(RS_10_c_value_min), DynamicValue(RS_10_c_value_max)), GenericUDFInBloomFilter(Column[value], DynamicValue(RS_10_c_value_bloom_filter))), GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_13_b_key_min), DynamicValue(RS_13_b_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_13_b_key_bloom_filter)))) Original num rows: 242 New num rows: 242 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-FIL[26]: numRows: 242 dataSize: 22990 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {value= colName: value colType: string countDistincts: 153 numNulls: 0 avgColLen: 6.801652892561983 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, key= colName: key colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting stats (Num rows: 242 Data size: 22990 Basic stats: COMPLETE Column stats: COMPLETE) on FIL[26] 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFBetween(Const boolean false, Column[value], DynamicValue(RS_9_a_value_min), DynamicValue(RS_9_a_value_max)) Original num rows: 25 New num rows: 25 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[value], DynamicValue(RS_9_a_value_min), DynamicValue(RS_9_a_value_max)), GenericUDFInBloomFilter(Column[value], DynamicValue(RS_9_a_value_bloom_filter))) Original num rows: 25 New num rows: 25 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFOPAnd(GenericUDFOPNotNull(Column[value]), GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[value], DynamicValue(RS_9_a_value_min), DynamicValue(RS_9_a_value_max)), GenericUDFInBloomFilter(Column[value], DynamicValue(RS_9_a_value_bloom_filter)))) Original num rows: 25 New num rows: 25 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-FIL[27]: numRows: 25 dataSize: 2225 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {value= colName: value colType: string countDistincts: 19 numNulls: 0 avgColLen: 4.92 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting stats (Num rows: 25 Data size: 2225 Basic stats: COMPLETE Column stats: COMPLETE) on FIL[27] 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_12_a_key_min), DynamicValue(RS_12_a_key_max)) Original num rows: 500 New num rows: 500 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_12_a_key_min), DynamicValue(RS_12_a_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_12_a_key_bloom_filter))) Original num rows: 500 New num rows: 500 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: Estimating row count for GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_12_a_key_min), DynamicValue(RS_12_a_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_12_a_key_bloom_filter)))) Original num rows: 500 New num rows: 500 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-FIL[28]: numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {key= colName: key colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on FIL[28] 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 242 Data size: 22990 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[2] 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[2]: numRows: 242 dataSize: 22990 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: string countDistincts: 153 numNulls: 0 avgColLen: 6.801652892561983 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 25 Data size: 2225 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[5] 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[5]: numRows: 25 dataSize: 2225 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: string countDistincts: 19 numNulls: 0 avgColLen: 4.92 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[8] 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[8]: numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 242 Data size: 22990 Basic stats: COMPLETE Column stats: COMPLETE) on RS[9] 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[9]: numRows: 242 dataSize: 22990 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {VALUE._col0= colName: VALUE._col0 colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false, KEY.reducesinkkey0= colName: KEY.reducesinkkey0 colType: string countDistincts: 153 numNulls: 0 avgColLen: 6.801652892561983 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 242 Data size: 22022 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[39] 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[39]: numRows: 242 dataSize: 22022 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col1= colName: _col1 colType: string countDistincts: 153 numNulls: 0 avgColLen: 6.801652892561983 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 25 Data size: 2225 Basic stats: COMPLETE Column stats: COMPLETE) on RS[10] 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[10]: numRows: 25 dataSize: 2225 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {KEY.reducesinkkey0= colName: KEY.reducesinkkey0 colType: string countDistincts: 19 numNulls: 0 avgColLen: 4.92 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 25 Data size: 2225 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[29] 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[29]: numRows: 25 dataSize: 2225 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: string countDistincts: 19 numNulls: 0 avgColLen: 4.92 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on RS[13] 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[13]: numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {KEY.reducesinkkey0= colName: KEY.reducesinkkey0 colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[34] 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[34]: numRows: 500 dataSize: 2000 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-JOIN[11]: detects none/multiple PK parents. 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.JoinOperator: Setting stats (Num rows: 39 Data size: 156 Basic stats: COMPLETE Column stats: COMPLETE) on JOIN[11] 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-JOIN[11]: numRows: 39 dataSize: 156 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 25 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[40]: inputSize: 22990 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[40] hashAgg: true 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 3] STATS-GBY[40]: cardinality: 1 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 552 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[40] 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[40]: numRows: 1 dataSize: 552 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[30]: inputSize: 2225 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[30] hashAgg: true 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 3] STATS-GBY[30]: cardinality: 1 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 552 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[30] 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[30]: numRows: 1 dataSize: 552 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[35]: inputSize: 2000 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[35] hashAgg: true 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 3] STATS-GBY[35]: cardinality: 1 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[35] 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[35]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 39 Data size: 156 Basic stats: COMPLETE Column stats: COMPLETE) on RS[12] 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[12]: numRows: 39 dataSize: 156 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {KEY.reducesinkkey0= colName: KEY.reducesinkkey0 colType: int countDistincts: 25 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting stats (Num rows: 39 Data size: 156 Basic stats: COMPLETE Column stats: COMPLETE) on SEL[44] 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-SEL[44]: numRows: 39 dataSize: 156 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 25 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 552 Basic stats: COMPLETE Column stats: COMPLETE) on RS[41] 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[41]: numRows: 1 dataSize: 552 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 552 Basic stats: COMPLETE Column stats: COMPLETE) on RS[31] 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[31]: numRows: 1 dataSize: 552 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on RS[36] 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[36]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-JOIN[14]: detects none/multiple PK parents. 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.JoinOperator: Setting stats (Num rows: 63 Data size: 504 Basic stats: COMPLETE Column stats: COMPLETE) on JOIN[14] 2018-01-05T11:29:11,372 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-JOIN[14]: numRows: 63 dataSize: 504 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[45]: inputSize: 156 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[45] hashAgg: true 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 3] STATS-GBY[45]: cardinality: 1 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[45] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[45]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[42]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[42] hashAgg: false 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 9] STATS-GBY[42]: cardinality: 1 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 552 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[42] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[42]: numRows: 1 dataSize: 552 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[32]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[32] hashAgg: false 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 9] STATS-GBY[32]: cardinality: 1 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 552 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[32] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[32]: numRows: 1 dataSize: 552 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: string countDistincts: 1 numNulls: 0 avgColLen: 100.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[37]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[37] hashAgg: false 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 9] STATS-GBY[37]: cardinality: 1 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[37] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[37]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[16]: inputSize: 504 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[16] hashAgg: true 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 3] STATS-GBY[16]: cardinality: 1 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[16] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[16]: numRows: 1 dataSize: 8 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on RS[46] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[46]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 552 Basic stats: COMPLETE Column stats: COMPLETE) on RS[43] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[43]: numRows: 1 dataSize: 552 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 552 Basic stats: COMPLETE Column stats: COMPLETE) on RS[33] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[33]: numRows: 1 dataSize: 552 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on RS[38] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[38]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE) on RS[17] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[17]: numRows: 1 dataSize: 8 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {VALUE._col0= colName: VALUE._col0 colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[47]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[47] hashAgg: false 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 9] STATS-GBY[47]: cardinality: 1 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[47] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[47]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col2= colName: _col2 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, _col1= colName: _col1 colType: int countDistincts: 1 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[18]: inputSize: 1 maxSplitSize: 256000000 parallelism: 1 containsGroupingSet: false sizeOfGroupingSet: 1 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: STATS-GBY[18] hashAgg: false 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [Case 9] STATS-GBY[18]: cardinality: 1 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting stats (Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE) on GBY[18] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-GBY[18]: numRows: 1 dataSize: 8 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting stats (Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE) on RS[48] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-RS[48]: numRows: 1 dataSize: 12 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {} 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] annotation.StatsRulesProcFactory: [0] STATS-FS[20]: numRows: 1 dataSize: 8 basicStatsState: COMPLETE colStatsState: COMPLETE colStats: {_col0= colName: _col0 colType: bigint countDistincts: 1 numNulls: 0 avgColLen: 8.0 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false} 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[key]]; bucket count: 2 }) on TS[0] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on TS[3] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.TableScanOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[key]]; bucket count: 4 }) on TS[6] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[key]]; bucket count: 2 }) on FIL[26] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on FIL[27] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FilterOperator: Setting traits ({ bucket column names: [[key]]; sort column names: [[key]]; bucket count: 4 }) on FIL[28] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: 2 }) on SEL[2] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on SEL[5] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: 4 }) on SEL[8] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[VALUE._col0]]; sort column names: [[VALUE._col0]]; bucket count: 2 }) on RS[9] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: 2 }) on SEL[39] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[KEY.reducesinkkey0]]; sort column names: [[KEY.reducesinkkey0]]; bucket count: -1 }) on RS[10] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: []; sort column names: []; bucket count: -1 }) on SEL[29] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[KEY.reducesinkkey0]]; sort column names: [[KEY.reducesinkkey0]]; bucket count: 4 }) on RS[13] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0]]; sort column names: [[_col0]]; bucket count: 4 }) on SEL[34] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.JoinOperator: Setting traits ({ bucket column names: [[_col0], []]; sort column names: [[_col0], []]; bucket count: -1 }) on JOIN[11] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[40] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[30] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[35] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[KEY.reducesinkkey0]]; sort column names: [[KEY.reducesinkkey0]]; bucket count: -1 }) on RS[12] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SelectOperator: Setting traits ({ bucket column names: [[_col0], []]; sort column names: [[_col0], []]; bucket count: -1 }) on SEL[44] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[41] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[31] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[36] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.JoinOperator: Setting traits ({ bucket column names: [[], []]; sort column names: [[], []]; bucket count: -1 }) on JOIN[14] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[45] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[42] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[32] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[37] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[16] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[46] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[43] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[33] 2018-01-05T11:29:11,373 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[38] 2018-01-05T11:29:11,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[17] 2018-01-05T11:29:11,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[47] 2018-01-05T11:29:11,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.GroupByOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on GBY[18] 2018-01-05T11:29:11,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ReduceSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on RS[48] 2018-01-05T11:29:11,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.FileSinkOperator: Setting traits ({ bucket column names: [[]]; sort column names: [[]]; bucket count: -1 }) on FS[20] 2018-01-05T11:29:11,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,374 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Set parallelism for reduce sink RS[9] to: 1 2018-01-05T11:29:11,374 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Set parallelism for reduce sink RS[10] to: 1 2018-01-05T11:29:11,374 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Set parallelism for reduce sink RS[13] to: 1 2018-01-05T11:29:11,374 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConvertJoinMapJoin: Memory monitor info set to : { isLlap: true executorsPerNode: 4 maxExecutorsOverSubscribeMemory: 3 memoryOverSubscriptionFactor: 0.20000000298023224 memoryCheckInterval: 100000 noConditionalTaskSize: 10000000 adjustedNoConditionalTaskSize: 16000000 hashTableInflationFactor: 2.0 threshold: 32000000 } 2018-01-05T11:29:11,374 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConvertJoinMapJoin: Fallback to common merge join operator 2018-01-05T11:29:11,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.CommonMergeJoinOperator: Setting traits ({ bucket column names: [[_col0], []]; sort column names: [[_col0], []]; bucket count: 0 }) on MERGEJOIN[49] 2018-01-05T11:29:11,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.CommonMergeJoinOperator: Setting stats (Num rows: 39 Data size: 156 Basic stats: COMPLETE Column stats: COMPLETE) on MERGEJOIN[49] 2018-01-05T11:29:11,374 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Set parallelism for reduce sink RS[12] to: 1 2018-01-05T11:29:11,374 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConvertJoinMapJoin: Memory monitor info set to : { isLlap: true executorsPerNode: 4 maxExecutorsOverSubscribeMemory: 3 memoryOverSubscriptionFactor: 0.20000000298023224 memoryCheckInterval: 100000 noConditionalTaskSize: 10000000 adjustedNoConditionalTaskSize: 16000000 hashTableInflationFactor: 2.0 threshold: 32000000 } 2018-01-05T11:29:11,374 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConvertJoinMapJoin: Fallback to common merge join operator 2018-01-05T11:29:11,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.CommonMergeJoinOperator: Setting traits ({ bucket column names: [[], []]; sort column names: [[], []]; bucket count: 0 }) on MERGEJOIN[50] 2018-01-05T11:29:11,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.CommonMergeJoinOperator: Setting stats (Num rows: 63 Data size: 504 Basic stats: COMPLETE Column stats: COMPLETE) on MERGEJOIN[50] 2018-01-05T11:29:11,374 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:11,374 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:11,374 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:11,374 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:11,374 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:11,374 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:11,374 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:11,374 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:11,374 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SetReducerParallelism: Number of reducers determined to be: 1 2018-01-05T11:29:11,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing BloomFilter cost/benefit for RS[48] - TS[6] (b) (Column[key]) 2018-01-05T11:29:11,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing key domain cardinality, keyDomainCardinality=461, semiJoinKeyIsPK=false, selColStat= colName: _col0 colType: int countDistincts: 25 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false, selColSourceStat= colName: key colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false, tsColStat= colName: key colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false 2018-01-05T11:29:11,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit for Column[_col0] to Column[key], selKeyCardinality=25, tsKeyCardinality=308, tsRows=500, keyDomainCardinality=461 2018-01-05T11:29:11,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: SemiJoin key selectivity=0.05422993492407809, benefit=472.8850325379609 2018-01-05T11:29:11,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit=472.8850325379609, cost=39.0, tsDataSize=500, netBenefit=433.8850325379609 2018-01-05T11:29:11,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: netBenefit=0.8677700650759218 2018-01-05T11:29:11,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing BloomFilter cost/benefit for RS[43] - TS[3] (c) (Column[value]) 2018-01-05T11:29:11,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing key domain cardinality, keyDomainCardinality=172, semiJoinKeyIsPK=false, selColStat= colName: _col1 colType: string countDistincts: 153 numNulls: 0 avgColLen: 6.801652892561983 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, selColSourceStat= colName: value colType: string countDistincts: 153 numNulls: 0 avgColLen: 6.801652892561983 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, tsColStat= colName: value colType: string countDistincts: 19 numNulls: 0 avgColLen: 4.92 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false 2018-01-05T11:29:11,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit for Column[_col1] to Column[value], selKeyCardinality=153, tsKeyCardinality=19, tsRows=25, keyDomainCardinality=172 2018-01-05T11:29:11,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: SemiJoin key selectivity=0.8895348837209303, benefit=2.7616279069767433 2018-01-05T11:29:11,374 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit=2.7616279069767433, cost=242.0, tsDataSize=25, netBenefit=-239.23837209302326 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: netBenefit=-9.56953488372093 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing BloomFilter cost/benefit for RS[33] - TS[0] (a) (Column[value]) 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing key domain cardinality, keyDomainCardinality=172, semiJoinKeyIsPK=false, selColStat= colName: _col0 colType: string countDistincts: 19 numNulls: 0 avgColLen: 4.92 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, selColSourceStat= colName: value colType: string countDistincts: 19 numNulls: 0 avgColLen: 4.92 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false, tsColStat= colName: value colType: string countDistincts: 153 numNulls: 0 avgColLen: 6.801652892561983 numTrues: 0 numFalses: 0 isPrimaryKey: false isEstimated: false 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit for Column[_col0] to Column[value], selKeyCardinality=19, tsKeyCardinality=153, tsRows=242, keyDomainCardinality=172 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: SemiJoin key selectivity=0.11046511627906977, benefit=215.2674418604651 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit=215.2674418604651, cost=25.0, tsDataSize=242, netBenefit=190.2674418604651 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: netBenefit=0.7862290985969633 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing BloomFilter cost/benefit for RS[38] - TS[0] (a) (Column[key]) 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Computing key domain cardinality, keyDomainCardinality=461, semiJoinKeyIsPK=false, selColStat= colName: _col0 colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false, selColSourceStat= colName: key colType: int countDistincts: 308 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 498 ] isPrimaryKey: false isEstimated: false, tsColStat= colName: key colType: int countDistincts: 153 numNulls: 0 avgColLen: 4.0 numTrues: 0 numFalses: 0 Range: [ min: 0 max: 497 ] isPrimaryKey: false isEstimated: false 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit for Column[_col0] to Column[key], selKeyCardinality=308, tsKeyCardinality=153, tsRows=242, keyDomainCardinality=461 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: SemiJoin key selectivity=0.6681127982646421, benefit=80.31670281995662 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: BloomFilter benefit=80.31670281995662, cost=500.0, tsDataSize=242, netBenefit=-419.6832971800434 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: netBenefit=-1.7342285007439808 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Reduction factor not satisfied for RS[43]-TS[3] (c). Removing semijoin optimization. 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Removing ReduceSink RS[43] and TableScan TS[3] 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Reduction factor not satisfied for RS[38]-TS[0] (a). Removing semijoin optimization. 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Removing ReduceSink RS[38] and TableScan TS[0] 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Adding special edge: RS --> TS[6] 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Adding special edge: RS --> TS[0] 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: FS, 20 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: GBY, 18 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 17 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: GBY, 16 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: MERGEJOIN, 50 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 12 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 13 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 8 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: FIL, 28 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: TS, 6 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 48 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: GBY, 47 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 46 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: GBY, 45 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 44 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: MERGEJOIN, 49 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 9 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 2 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: FIL, 26 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: TS, 0 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 10 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 33 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: GBY, 32 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: RS, 31 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: GBY, 30 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 29 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: SEL, 5 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: FIL, 27 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Component: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Operator: TS, 3 2018-01-05T11:29:11,375 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Cycle free: true 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: Before SharedWorkOptimizer: TS[0]-FIL[26]-SEL[2]-RS[9]-MERGEJOIN[49]-RS[12]-MERGEJOIN[50]-GBY[16]-RS[17]-GBY[18]-FS[20] -SEL[44]-GBY[45]-RS[46]-GBY[47]-RS[48] TS[3]-FIL[27]-SEL[5]-RS[10]-MERGEJOIN[49] -SEL[29]-GBY[30]-RS[31]-GBY[32]-RS[33] TS[6]-FIL[28]-SEL[8]-RS[13]-MERGEJOIN[50] 2018-01-05T11:29:11,375 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: DPP information stored in the cache: {TS[0]=[RS[33]], TS[6]=[RS[48]]} 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: Sorted tables by size: [default.tab=22990, default.src1=2225, default.tab_part=2000] 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.SharedWorkOptimizer: After SharedWorkOptimizer: TS[0]-FIL[26]-SEL[2]-RS[9]-MERGEJOIN[49]-RS[12]-MERGEJOIN[50]-GBY[16]-RS[17]-GBY[18]-FS[20] -SEL[44]-GBY[45]-RS[46]-GBY[47]-RS[48] TS[3]-FIL[27]-SEL[5]-RS[10]-MERGEJOIN[49] -SEL[29]-GBY[30]-RS[31]-GBY[32]-RS[33] TS[6]-FIL[28]-SEL[8]-RS[13]-MERGEJOIN[50] 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[0] with rs:(key: int|{a}key,value: string|{a}value,ds: string|{a}ds,BLOCK__OFFSET__INSIDE__FILE: bigint|{a}block__offset__inside__file,INPUT__FILE__NAME: string|{a}input__file__name,ROW__ID: struct|{a}row__id) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(Const boolean true, Const boolean true) -> Const boolean true 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[3] with rs:(key: string|{c}key,value: string|{c}value,BLOCK__OFFSET__INSIDE__FILE: bigint|{c}block__offset__inside__file,INPUT__FILE__NAME: string|{c}input__file__name,ROW__ID: struct|{c}row__id) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(Const boolean true, Const boolean true) -> Const boolean true 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(GenericUDFOPNotNull(Column[value]), GenericUDFOPAnd(Const boolean true, Const boolean true)) -> GenericUDFOPNotNull(Column[value]) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:TS[6] with rs:(key: int|{b}key,value: string|{b}value,ds: string|{b}ds,BLOCK__OFFSET__INSIDE__FILE: bigint|{b}block__offset__inside__file,INPUT__FILE__NAME: string|{b}input__file__name,ROW__ID: struct|{b}row__id) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FIL[26] with rs:(key: int|{a}key,value: string|{a}value) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_0 {} 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FIL[26] 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Old filter FIL[26] conditions:(key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter)))) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New filter FIL[26] conditions:(key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter)))) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FIL[27] with rs:(value: string|{c}value) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_3 {} 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FIL[27] 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Old filter FIL[27] conditions:(value is not null and (true and true)) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(Const boolean true, Const boolean true) -> Const boolean true 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Folding expression:GenericUDFOPAnd(GenericUDFOPNotNull(Column[value]), GenericUDFOPAnd(Const boolean true, Const boolean true)) -> GenericUDFOPNotNull(Column[value]) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New filter FIL[27] conditions:value is not null 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FIL[28] with rs:(key: int|{b}key) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op TS_6 {} 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FIL[28] 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: Old filter FIL[28] conditions:(key is not null and (key BETWEEN DynamicValue(RS_12_a_key_min) AND DynamicValue(RS_12_a_key_max) and in_bloom_filter(key, DynamicValue(RS_12_a_key_bloom_filter)))) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New filter FIL[28] conditions:(key is not null and (key BETWEEN DynamicValue(RS_12_a_key_min) AND DynamicValue(RS_12_a_key_max) and in_bloom_filter(key, DynamicValue(RS_12_a_key_bloom_filter)))) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[2] with rs:(_col0: int|{$hdt$_1}key,_col1: string|{$hdt$_1}value) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op FIL_26 {} 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[2] 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key] Column[value]) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[5] with rs:(_col0: string|{$hdt$_2}value) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op FIL_27 {} 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[5] 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[value]) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[8] with rs:(_col0: int|{$hdt$_0}key) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op FIL_28 {} 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[8] 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[key]) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[9] with rs:(VALUE._col0: int|{$hdt$_1}key,KEY.reducesinkkey0: string|{$hdt$_1}value) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_2 {} 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[9] 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[10] with rs:(KEY.reducesinkkey0: string|{$hdt$_2}value) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[10] 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[29] with rs:(_col0: string|{$hdt$_2}value) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_5 {} 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[29] 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0]) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[13] with rs:(KEY.reducesinkkey0: int|{$hdt$_0}key) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_8 {} 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[13] 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:MERGEJOIN[49] with rs:(_col0: int|{$hdt$_1}key) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_9 {} 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_10 {} 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator MERGEJOIN[49] 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[30] with rs:(_col0: string|{},_col1: string|{},_col2: string|{}) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_29 {} 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[30] 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[12] with rs:(KEY.reducesinkkey0: int|{$hdt$_1}key) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op MERGEJOIN_49 {} 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[12] 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[13] with rs:(KEY.reducesinkkey0: int|{$hdt$_0}key) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_8 {} 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[13] 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:SEL[44] with rs:(_col0: int|{$hdt$_1}key) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op MERGEJOIN_49 {} 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator SEL[44] 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcFactory: New column list:(Column[_col0]) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[31] with rs:(_col0: string|{},_col1: string|{},_col2: string|{}) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_30 {} 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[31] 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:MERGEJOIN[50] with rs:() 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_12 {} 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_13 {} 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator MERGEJOIN[50] 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[45] with rs:(_col0: int|{},_col1: int|{},_col2: int|{}) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op SEL_44 {} 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[45] 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[32] with rs:(_col0: string|{},_col1: string|{},_col2: string|{}) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_31 {} 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[32] 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[16] with rs:(_col0: bigint|{}(tok_functionstar count)) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op MERGEJOIN_50 {} 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[16] 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[46] with rs:(_col0: int|{},_col1: int|{},_col2: int|{}) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_45 {} 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[46] 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[33] with rs:(_col0: string|{},_col1: string|{},_col2: string|{}) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_32 {} 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[33] 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[17] with rs:(VALUE._col0: bigint|{}(tok_functionstar count)) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_16 {} 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[17] 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[47] with rs:(_col0: int|{},_col1: int|{},_col2: int|{}) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_46 {} 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[47] 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:GBY[18] with rs:(_col0: bigint|{}(tok_functionstar count)) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op RS_17 {} 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator GBY[18] 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:RS[48] with rs:(_col0: int|{},_col1: int|{},_col2: int|{}) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_47 {} 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator RS[48] 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Getting constants of op:FS[20] with rs:(_col0: bigint|{}) 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Constant of Op GBY_18 {} 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.ConstantPropagateProcCtx: Offering constants [] to operator FS[20] 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: TS[0] 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[9] 2018-01-05T11:29:11,376 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding map work (Map 1) for TS[0] 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 of table a 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[9] 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: MERGEJOIN[49] 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[12] 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding reduce work (Reducer 2) for MERGEJOIN[49] 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Setting up reduce sink: RS[9] with following reduce work: Reducer 2 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[9] as parent from MERGEJOIN[49] 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[10] as parent from MERGEJOIN[49] 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[12] 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: MERGEJOIN[50] 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[17] 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding reduce work (Reducer 3) for MERGEJOIN[50] 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Setting up reduce sink: RS[12] with following reduce work: Reducer 3 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[12] as parent from MERGEJOIN[50] 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[13] as parent from MERGEJOIN[50] 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[17] 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: GBY[18] 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: FS[20] 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding reduce work (Reducer 4) for GBY[18] 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Setting up reduce sink: RS[17] with following reduce work: Reducer 4 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[17] as parent from GBY[18] 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: FS[20] 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: MERGEJOIN[49] 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[46] 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[46] 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: GBY[47] 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[48] 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding reduce work (Reducer 5) for GBY[47] 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Setting up reduce sink: RS[46] with following reduce work: Reducer 5 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[46] as parent from GBY[47] 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[48] 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: TS[3] 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[10] 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding map work (Map 6) for TS[3] 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1 of table c 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Second pass. Leaf operator: RS[10] has common downstream work: org.apache.hadoop.hive.ql.plan.MergeJoinWork@3bc4c105 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: TS[3] 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[31] 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[31] 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: GBY[32] 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[33] 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding reduce work (Reducer 7) for GBY[32] 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Setting up reduce sink: RS[31] with following reduce work: Reducer 7 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Removing RS[31] as parent from GBY[32] 2018-01-05T11:29:11,377 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: First pass. Leaf operator: RS[33] 2018-01-05T11:29:11,378 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Root operator: TS[6] 2018-01-05T11:29:11,378 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Leaf operator: RS[13] 2018-01-05T11:29:11,378 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Adding map work (Map 8) for TS[6] 2018-01-05T11:29:11,378 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Adding file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 of table b 2018-01-05T11:29:11,378 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] optimizer.GenMapRedUtils: Information added for path file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:11,378 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezWork: Second pass. Leaf operator: RS[13] has common downstream work: org.apache.hadoop.hive.ql.plan.MergeJoinWork@7288d563 2018-01-05T11:29:11,378 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: ResduceSink RS[48] to TableScan TS[6] 2018-01-05T11:29:11,378 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Connecting Baswork - Reducer 5 to Map 8 2018-01-05T11:29:11,378 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: ResduceSink RS[33] to TableScan TS[0] 2018-01-05T11:29:11,378 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.GenTezUtils: Connecting Baswork - Reducer 7 to Map 1 2018-01-05T11:29:11,378 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: There are 0 app master events. 2018-01-05T11:29:11,378 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,378 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,378 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 6 2018-01-05T11:29:11,378 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:11,378 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:11,378 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking at: Map 6 2018-01-05T11:29:11,378 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 2018-01-05T11:29:11,378 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.NullScanTaskDispatcher: Found 0 null table scans 2018-01-05T11:29:11,378 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Skipping metadata only query optimization 2018-01-05T11:29:11,378 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Examining input format to see if vectorization is enabled. 2018-01-05T11:29:11,378 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorization is enabled for input format(s) [org.apache.hadoop.mapred.TextInputFormat] 2018-01-05T11:29:11,378 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing MapWork... 2018-01-05T11:29:11,379 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFOPNotNull(Column[value]), Vectorized Expression = SelectColumnIsNotNull(col 1:string) 2018-01-05T11:29:11,379 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator 2018-01-05T11:29:11,379 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.FilterDesc 2018-01-05T11:29:11,379 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[value], Vectorized Expression = col 1:string 2018-01-05T11:29:11,379 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator 2018-01-05T11:29:11,379 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.SelectDesc 2018-01-05T11:29:11,379 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 1:string 2018-01-05T11:29:11,379 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator groupby typeName string 2018-01-05T11:29:11,379 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkStringOperator 2018-01-05T11:29:11,379 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:11,379 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkStringOperator 2018-01-05T11:29:11,379 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:11,379 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 1:string 2018-01-05T11:29:11,379 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator 2018-01-05T11:29:11,379 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.SelectDesc 2018-01-05T11:29:11,379 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vector GROUP BY operator will use processing mode HASH 2018-01-05T11:29:11,379 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 1:string 2018-01-05T11:29:11,379 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 1:string 2018-01-05T11:29:11,379 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 1:string 2018-01-05T11:29:11,379 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator 2018-01-05T11:29:11,379 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.GroupByDesc 2018-01-05T11:29:11,379 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:string 2018-01-05T11:29:11,379 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col1], Vectorized Expression = col 1:string 2018-01-05T11:29:11,379 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col2], Vectorized Expression = col 2:binary 2018-01-05T11:29:11,379 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkEmptyKeyOperator 2018-01-05T11:29:11,379 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:11,379 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkEmptyKeyOperator: VectorReduceSinkEmptyKeyOperator constructor vectorReduceSinkInfo org.apache.hadoop.hive.ql.plan.VectorReduceSinkInfo@3479facb 2018-01-05T11:29:11,379 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkEmptyKeyOperator 2018-01-05T11:29:11,379 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:11,380 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [key, value, ROW__ID] 2018-01-05T11:29:11,380 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [string, string, struct] 2018-01-05T11:29:11,380 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE, NONE, NONE] 2018-01-05T11:29:11,380 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 0 2018-01-05T11:29:11,380 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 1 2018-01-05T11:29:11,380 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:11,380 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:11,380 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Using reduce tag 0 2018-01-05T11:29:11,380 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1, _col2] columnTypes=[string, string, binary] 2018-01-05T11:29:11,380 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing ReduceWork... 2018-01-05T11:29:11,380 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vector GROUP BY operator will use processing mode STREAMING 2018-01-05T11:29:11,380 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[VALUE._col0], Vectorized Expression = col 0:string 2018-01-05T11:29:11,380 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[VALUE._col1], Vectorized Expression = col 1:string 2018-01-05T11:29:11,380 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[VALUE._col2], Vectorized Expression = col 2:binary 2018-01-05T11:29:11,380 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator 2018-01-05T11:29:11,380 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.GroupByDesc 2018-01-05T11:29:11,380 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:string 2018-01-05T11:29:11,380 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col1], Vectorized Expression = col 1:string 2018-01-05T11:29:11,380 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col2], Vectorized Expression = col 2:binary 2018-01-05T11:29:11,380 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkEmptyKeyOperator 2018-01-05T11:29:11,380 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:11,380 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkEmptyKeyOperator: VectorReduceSinkEmptyKeyOperator constructor vectorReduceSinkInfo org.apache.hadoop.hive.ql.plan.VectorReduceSinkInfo@3a3026b0 2018-01-05T11:29:11,380 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkEmptyKeyOperator 2018-01-05T11:29:11,380 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:11,380 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [VALUE._col0, VALUE._col1, VALUE._col2] 2018-01-05T11:29:11,380 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [string, string, binary] 2018-01-05T11:29:11,380 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE, NONE, NONE] 2018-01-05T11:29:11,380 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 0 2018-01-05T11:29:11,380 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 0 2018-01-05T11:29:11,380 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:11,380 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:11,380 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Examining input format to see if vectorization is enabled. 2018-01-05T11:29:11,380 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorization is enabled for input format(s) [org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] 2018-01-05T11:29:11,380 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing MapWork... 2018-01-05T11:29:11,381 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFOPNotNull(Column[key]), Vectorized Expression = SelectColumnIsNotNull(col 0:int) 2018-01-05T11:29:11,381 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFOPNotNull(Column[value]), Vectorized Expression = SelectColumnIsNotNull(col 1:string) 2018-01-05T11:29:11,381 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFBetween(Const boolean false, Column[value], DynamicValue(RS_10_c_value_min), DynamicValue(RS_10_c_value_max)), Vectorized Expression = FilterStringColumnBetweenDynamicValue(col 1:string, left NULL, right NULL) 2018-01-05T11:29:11,381 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFInBloomFilter(Column[value], DynamicValue(RS_10_c_value_bloom_filter)), Vectorized Expression = VectorInBloomFilterColDynamicValue 2018-01-05T11:29:11,381 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[value], DynamicValue(RS_10_c_value_min), DynamicValue(RS_10_c_value_max)), GenericUDFInBloomFilter(Column[value], DynamicValue(RS_10_c_value_bloom_filter))), Vectorized Expression = FilterExprAndExpr(children: FilterStringColumnBetweenDynamicValue(col 1:string, left NULL, right NULL), VectorInBloomFilterColDynamicValue) 2018-01-05T11:29:11,381 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPNotNull(Column[value]), GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[value], DynamicValue(RS_10_c_value_min), DynamicValue(RS_10_c_value_max)), GenericUDFInBloomFilter(Column[value], DynamicValue(RS_10_c_value_bloom_filter)))), Vectorized Expression = FilterExprAndExpr(children: SelectColumnIsNotNull(col 0:int), SelectColumnIsNotNull(col 1:string), FilterExprAndExpr(children: FilterStringColumnBetweenDynamicValue(col 1:string, left NULL, right NULL), VectorInBloomFilterColDynamicValue)) 2018-01-05T11:29:11,381 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator 2018-01-05T11:29:11,381 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.FilterDesc 2018-01-05T11:29:11,381 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[key], Vectorized Expression = col 0:int 2018-01-05T11:29:11,381 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[value], Vectorized Expression = col 1:string 2018-01-05T11:29:11,381 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator 2018-01-05T11:29:11,381 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.SelectDesc 2018-01-05T11:29:11,381 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col1], Vectorized Expression = col 1:string 2018-01-05T11:29:11,382 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:int 2018-01-05T11:29:11,382 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator groupby typeName string 2018-01-05T11:29:11,382 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkStringOperator 2018-01-05T11:29:11,382 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:11,382 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkStringOperator 2018-01-05T11:29:11,382 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:11,382 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [key, value, ds, ROW__ID] 2018-01-05T11:29:11,382 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [int, string, string, struct] 2018-01-05T11:29:11,382 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE, NONE, NONE, NONE] 2018-01-05T11:29:11,382 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 1 2018-01-05T11:29:11,382 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 1 2018-01-05T11:29:11,382 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:11,382 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:11,382 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Using reduce tag 0 2018-01-05T11:29:11,382 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1, _col2] columnTypes=[int, int, binary] 2018-01-05T11:29:11,382 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing ReduceWork... 2018-01-05T11:29:11,382 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vector GROUP BY operator will use processing mode STREAMING 2018-01-05T11:29:11,382 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[VALUE._col0], Vectorized Expression = col 0:int 2018-01-05T11:29:11,382 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[VALUE._col1], Vectorized Expression = col 1:int 2018-01-05T11:29:11,382 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[VALUE._col2], Vectorized Expression = col 2:binary 2018-01-05T11:29:11,382 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator 2018-01-05T11:29:11,382 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.GroupByDesc 2018-01-05T11:29:11,382 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:int 2018-01-05T11:29:11,382 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col1], Vectorized Expression = col 1:int 2018-01-05T11:29:11,382 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col2], Vectorized Expression = col 2:binary 2018-01-05T11:29:11,382 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkEmptyKeyOperator 2018-01-05T11:29:11,382 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:11,382 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkEmptyKeyOperator: VectorReduceSinkEmptyKeyOperator constructor vectorReduceSinkInfo org.apache.hadoop.hive.ql.plan.VectorReduceSinkInfo@3dec0924 2018-01-05T11:29:11,382 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkEmptyKeyOperator 2018-01-05T11:29:11,382 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:11,382 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [VALUE._col0, VALUE._col1, VALUE._col2] 2018-01-05T11:29:11,382 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [int, int, binary] 2018-01-05T11:29:11,382 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE, NONE, NONE] 2018-01-05T11:29:11,382 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 0 2018-01-05T11:29:11,382 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 0 2018-01-05T11:29:11,382 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:11,382 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:11,382 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Examining input format to see if vectorization is enabled. 2018-01-05T11:29:11,382 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorization is enabled for input format(s) [org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] 2018-01-05T11:29:11,382 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing MapWork... 2018-01-05T11:29:11,382 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFOPNotNull(Column[key]), Vectorized Expression = SelectColumnIsNotNull(col 0:int) 2018-01-05T11:29:11,382 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_12_a_key_min), DynamicValue(RS_12_a_key_max)), Vectorized Expression = FilterLongColumnBetweenDynamicValue(col 0:int, left 0, right 0) 2018-01-05T11:29:11,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFInBloomFilter(Column[key], DynamicValue(RS_12_a_key_bloom_filter)), Vectorized Expression = VectorInBloomFilterColDynamicValue 2018-01-05T11:29:11,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_12_a_key_min), DynamicValue(RS_12_a_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_12_a_key_bloom_filter))), Vectorized Expression = FilterExprAndExpr(children: FilterLongColumnBetweenDynamicValue(col 0:int, left 0, right 0), VectorInBloomFilterColDynamicValue) 2018-01-05T11:29:11,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_12_a_key_min), DynamicValue(RS_12_a_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_12_a_key_bloom_filter)))), Vectorized Expression = FilterExprAndExpr(children: SelectColumnIsNotNull(col 0:int), FilterExprAndExpr(children: FilterLongColumnBetweenDynamicValue(col 0:int, left 0, right 0), VectorInBloomFilterColDynamicValue)) 2018-01-05T11:29:11,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator 2018-01-05T11:29:11,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.FilterDesc 2018-01-05T11:29:11,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[key], Vectorized Expression = col 0:int 2018-01-05T11:29:11,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator 2018-01-05T11:29:11,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.SelectDesc 2018-01-05T11:29:11,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[_col0], Vectorized Expression = col 0:int 2018-01-05T11:29:11,383 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator groupby typeName int 2018-01-05T11:29:11,383 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vectorizer vectorizeOperator reduce sink class VectorReduceSinkLongOperator 2018-01-05T11:29:11,383 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] reducesink.VectorReduceSinkCommonOperator: VectorReduceSinkCommonOperator constructor 2018-01-05T11:29:11,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkLongOperator 2018-01-05T11:29:11,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.ReduceSinkDesc 2018-01-05T11:29:11,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [key, value, ds, ROW__ID] 2018-01-05T11:29:11,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [int, string, string, struct] 2018-01-05T11:29:11,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE, NONE, NONE, NONE] 2018-01-05T11:29:11,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 1 2018-01-05T11:29:11,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 1 2018-01-05T11:29:11,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:11,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:11,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Using reduce tag 0 2018-01-05T11:29:11,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0] columnTypes=[bigint] 2018-01-05T11:29:11,383 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Validating and vectorizing ReduceWork... 2018-01-05T11:29:11,383 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: Vector GROUP BY operator will use processing mode GLOBAL 2018-01-05T11:29:11,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] vector.VectorizationContext: Input Expression = Column[VALUE._col0], Vectorized Expression = col 0:bigint 2018-01-05T11:29:11,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator 2018-01-05T11:29:11,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.GroupByDesc 2018-01-05T11:29:11,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.exec.vector.VectorFileSinkOperator 2018-01-05T11:29:11,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: vectorizeOperator org.apache.hadoop.hive.ql.plan.FileSinkDesc 2018-01-05T11:29:11,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnNames [VALUE._col0] 2018-01-05T11:29:11,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowColumnTypeInfos [bigint] 2018-01-05T11:29:11,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo rowDataTypePhysicalVariations [NONE] 2018-01-05T11:29:11,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo partitionColumnCount 0 2018-01-05T11:29:11,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo virtualColumnCount 0 2018-01-05T11:29:11,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchColumnTypeNames [] 2018-01-05T11:29:11,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.Vectorizer: debugDisplayVertexInfo scratchdataTypePhysicalVariations [] 2018-01-05T11:29:11,383 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.TezCompiler: Skipping stage id rearranger 2018-01-05T11:29:11,383 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: llap mode: only 2018-01-05T11:29:11,384 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Map 6 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'min(_col0)' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '_col0' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'max(_col0)' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '_col0' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'bloom_filter(_col0, expectedEntries=19)' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '_col0' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '_col0' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'value' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'value is not null' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'value is not null' 2018-01-05T11:29:11,384 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@c362045 can be done in LLAP 2018-01-05T11:29:11,384 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Reducer 7 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'min(VALUE._col0)' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'max(VALUE._col1)' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'VALUE._col1' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'bloom_filter(VALUE._col2, expectedEntries=19)' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'VALUE._col2' 2018-01-05T11:29:11,384 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@5549a58b can be done in LLAP 2018-01-05T11:29:11,384 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Map 1 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'value' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter))))' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter))))' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter))))' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter))))' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter))))' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter))))' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter))))' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter))))' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter))))' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter))))' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter))))' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter))))' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter))))' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter))))' 2018-01-05T11:29:11,384 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@723ac4fc can be done in LLAP 2018-01-05T11:29:11,384 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Reducer 2 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'min(_col0)' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '_col0' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'max(_col0)' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '_col0' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'bloom_filter(_col0, expectedEntries=25)' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '_col0' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '_col0' 2018-01-05T11:29:11,384 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MergeJoinWork@3bc4c105 can be done in LLAP 2018-01-05T11:29:11,384 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Reducer 5 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'min(VALUE._col0)' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'max(VALUE._col1)' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'VALUE._col1' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'bloom_filter(VALUE._col2, expectedEntries=25)' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'VALUE._col2' 2018-01-05T11:29:11,384 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@38a1d9ff can be done in LLAP 2018-01-05T11:29:11,384 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Map 8 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'key' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_12_a_key_min) AND DynamicValue(RS_12_a_key_max) and in_bloom_filter(key, DynamicValue(RS_12_a_key_bloom_filter))))' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_12_a_key_min) AND DynamicValue(RS_12_a_key_max) and in_bloom_filter(key, DynamicValue(RS_12_a_key_bloom_filter))))' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_12_a_key_min) AND DynamicValue(RS_12_a_key_max) and in_bloom_filter(key, DynamicValue(RS_12_a_key_bloom_filter))))' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_12_a_key_min) AND DynamicValue(RS_12_a_key_max) and in_bloom_filter(key, DynamicValue(RS_12_a_key_bloom_filter))))' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_12_a_key_min) AND DynamicValue(RS_12_a_key_max) and in_bloom_filter(key, DynamicValue(RS_12_a_key_bloom_filter))))' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_12_a_key_min) AND DynamicValue(RS_12_a_key_max) and in_bloom_filter(key, DynamicValue(RS_12_a_key_bloom_filter))))' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_12_a_key_min) AND DynamicValue(RS_12_a_key_max) and in_bloom_filter(key, DynamicValue(RS_12_a_key_bloom_filter))))' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_12_a_key_min) AND DynamicValue(RS_12_a_key_max) and in_bloom_filter(key, DynamicValue(RS_12_a_key_bloom_filter))))' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_12_a_key_min) AND DynamicValue(RS_12_a_key_max) and in_bloom_filter(key, DynamicValue(RS_12_a_key_bloom_filter))))' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_12_a_key_min) AND DynamicValue(RS_12_a_key_max) and in_bloom_filter(key, DynamicValue(RS_12_a_key_bloom_filter))))' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_12_a_key_min) AND DynamicValue(RS_12_a_key_max) and in_bloom_filter(key, DynamicValue(RS_12_a_key_bloom_filter))))' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking '(key is not null and (key BETWEEN DynamicValue(RS_12_a_key_min) AND DynamicValue(RS_12_a_key_max) and in_bloom_filter(key, DynamicValue(RS_12_a_key_bloom_filter))))' 2018-01-05T11:29:11,384 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MapWork@188aecbf can be done in LLAP 2018-01-05T11:29:11,384 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Reducer 3 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'count()' 2018-01-05T11:29:11,384 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.MergeJoinWork@7288d563 can be done in LLAP 2018-01-05T11:29:11,384 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Evaluating work item: Reducer 4 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'count(VALUE._col0)' 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Checking 'VALUE._col0' 2018-01-05T11:29:11,384 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: LLAP mode set to 'only' so can convert any work. 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.LlapDecider: Work org.apache.hadoop.hive.ql.plan.ReduceWork@4b60cdd0 can be done in LLAP 2018-01-05T11:29:11,384 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.SerializeFilter: Serializing: value is not null 2018-01-05T11:29:11,385 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.SerializeFilter: Serializing: (key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter)))) 2018-01-05T11:29:11,385 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] physical.SerializeFilter: Serializing: (key is not null and (key BETWEEN DynamicValue(RS_12_a_key_min) AND DynamicValue(RS_12_a_key_max) and in_bloom_filter(key, DynamicValue(RS_12_a_key_bloom_filter)))) 2018-01-05T11:29:11,385 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,385 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: Completed plan generation 2018-01-05T11:29:11,385 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:11,385 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: validation start 2018-01-05T11:29:11,385 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.CalcitePlanner: not validating writeEntity, because entity is neither table nor partition 2018-01-05T11:29:11,385 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,385 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0, type:bigint, comment:null)], properties:null) 2018-01-05T11:29:11,385 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: Initializing operator LIST_SINK[69] 2018-01-05T11:29:11,385 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: Initialization Done 69 LIST_SINK 2018-01-05T11:29:11,385 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: Operator 69 LIST_SINK initialized 2018-01-05T11:29:11,385 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: Initialization Done 69 LIST_SINK done is reset. 2018-01-05T11:29:11,385 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,385 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:11,386 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {getNotNullConstraints_(NotNullConstraintsRequest, )=0, isCompatibleWith_(Configuration, )=0, getAggrColStatsFor_(String, String, List, List, )=7, getTable_(String, String, )=23, flushCache_()=0, getUniqueConstraints_(UniqueConstraintsRequest, )=0, getPrimaryKeys_(PrimaryKeysRequest, )=3, listPartitions_(String, String, short, )=15, getTableColumnStatistics_(String, String, List, )=11, getForeignKeys_(ForeignKeysRequest, )=1} 2018-01-05T11:29:11,386 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112911_d60e4252-4a13-4247-a28a-f07830bbf022); Time taken: 0.234 seconds 2018-01-05T11:29:11,386 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:11,386 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,386 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112911_d60e4252-4a13-4247-a28a-f07830bbf022): select count(*) from tab a join tab_part b on a.key = b.key join src1 c on a.value = c.value 2018-01-05T11:29:11,386 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,386 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: select count(*) from tab a join tab_part b on a.key = b.key join src1 c on a.value = c.value 2018-01-05T11:29:11,386 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: QUERY 2018-01-05T11:29:11,387 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@src1 2018-01-05T11:29:11,387 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@tab 2018-01-05T11:29:11,387 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@tab@ds=2008-04-08 2018-01-05T11:29:11,387 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@tab_part 2018-01-05T11:29:11,387 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@tab_part@ds=2008-04-08 2018-01-05T11:29:11,387 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/-mr-10001 2018-01-05T11:29:11,387 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,387 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,387 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,387 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Query ID = djaiswal_20180105112911_d60e4252-4a13-4247-a28a-f07830bbf022 2018-01-05T11:29:11,387 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Total jobs = 1 2018-01-05T11:29:11,387 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,387 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Launching Job 1 out of 1 2018-01-05T11:29:11,387 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-1:MAPRED] in serial mode 2018-01-05T11:29:11,397 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: TezDir path set file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/djaiswal/_tez_scratch_dir for user: djaiswal 2018-01-05T11:29:11,420 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.WorkloadManagerFederation: Using unmanaged session - WM is not initialized 2018-01-05T11:29:11,420 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.TezSessionPoolManager: The current user: djaiswal, session user: djaiswal 2018-01-05T11:29:11,420 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.TezSessionPoolManager: Current queue name is null incoming queue name is null 2018-01-05T11:29:11,420 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Subscribed to counters: [] for queryId: djaiswal_20180105112911_d60e4252-4a13-4247-a28a-f07830bbf022 2018-01-05T11:29:11,420 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Session is already open 2018-01-05T11:29:11,420 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: No local resources to process (other than hive-exec) 2018-01-05T11:29:11,420 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,420 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Dag name: select count(*) from tab a join ta...c.value (Stage-1) 2018-01-05T11:29:11,420 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: DagInfo: {"context":"Hive","description":"\nselect count(*) from tab a join tab_part b on a.key = b.key join src1 c on a.value = c.value"} 2018-01-05T11:29:11,420 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Task: Setting Tez DAG access for queryId=djaiswal_20180105112911_d60e4252-4a13-4247-a28a-f07830bbf022 with viewAclString=hive_test_user,djaiswal, modifyStr=hive_test_user,djaiswal 2018-01-05T11:29:11,420 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,432 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,432 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:29:11,433 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,433 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 4 size: 1.91KB 2018-01-05T11:29:11,449 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,449 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,461 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,461 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:29:11,463 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,463 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 3 size: 1.49KB 2018-01-05T11:29:11,465 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,467 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,479 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:29:11,479 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,479 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:29:11,480 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,480 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: Serialized plan (via RPC) - name: Map 8 size: 5.29KB 2018-01-05T11:29:11,486 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:11,486 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,487 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,499 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,499 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:29:11,500 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,500 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 5 size: 3.14KB 2018-01-05T11:29:11,503 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,504 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,517 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,517 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:29:11,518 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,518 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 2 size: 2.06KB 2018-01-05T11:29:11,521 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,524 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,536 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:29:11,536 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,536 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:29:11,538 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,539 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: Serialized plan (via RPC) - name: Map 1 size: 5.49KB 2018-01-05T11:29:11,545 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:11,545 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,547 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,559 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,559 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SerializationUtilities: Serializing ReduceWork using kryo 2018-01-05T11:29:11,561 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,561 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: Serialized plan (via RPC) - name: Reducer 7 size: 4.89KB 2018-01-05T11:29:11,563 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,565 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,577 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: Vertex has custom input? false 2018-01-05T11:29:11,578 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,578 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.SerializationUtilities: Serializing MapWork using kryo 2018-01-05T11:29:11,579 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,579 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.Utilities: Serialized plan (via RPC) - name: Map 6 size: 5.16KB 2018-01-05T11:29:11,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] tez.DagUtils: Marking URI as needing credentials: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1 2018-01-05T11:29:11,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,605 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,605 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,605 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] client.TezClient: Submitting dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagName=select count(*) from tab a join ta...c.value (Stage-1), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112911_d60e4252-4a13-4247-a28a-f07830bbf022 } 2018-01-05T11:29:11,605 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:11,605 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:11,606 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:11,606 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:11,606 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:11,606 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:11,606 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:11,606 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] common.TezYARNUtils: Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true 2018-01-05T11:29:11,610 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Starting DAG submitted via RPC: select count(*) from tab a join ta...c.value (Stage-1) 2018-01-05T11:29:11,610 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Merging AM credentials into DAG credentials 2018-01-05T11:29:11,610 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Generating DAG graphviz file, dagId=dag_1515180518813_0001_42, filePath=/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001_wd/localmode-log-dir/dag_1515180518813_0001_42.dot 2018-01-05T11:29:11,611 INFO [IPC Server handler 0 on 49823] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_42 2018-01-05T11:29:11,611 WARN [IPC Server handler 0 on 49823] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:29:11,611 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Running DAG: select count(*) from tab a join ta...c.value (Stage-1), callerContext={ context=HIVE, callerType=HIVE_QUERY_ID, callerId=djaiswal_20180105112911_d60e4252-4a13-4247-a28a-f07830bbf022 } 2018-01-05T11:29:11,635 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:DAG_SUBMITTED]: dagID=dag_1515180518813_0001_42, submitTime=1515180551610, queueName=null 2018-01-05T11:29:11,635 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_42_00 [Map 6]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_42_00 [Map 6], Default Resources= 2018-01-05T11:29:11,635 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_42_00 [Map 6] 2018-01-05T11:29:11,635 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 6 2018-01-05T11:29:11,635 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_42_00 [Map 6] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:11,636 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_42_01 [Reducer 7]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_42_01 [Reducer 7], Default Resources= 2018-01-05T11:29:11,636 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 7 2018-01-05T11:29:11,636 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_42_01 [Reducer 7] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:11,636 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_42_02 [Map 1]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_42_02 [Map 1], Default Resources= 2018-01-05T11:29:11,636 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_42_02 [Map 1] 2018-01-05T11:29:11,636 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 1 2018-01-05T11:29:11,636 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_42_02 [Map 1] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:11,637 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_42_03 [Reducer 2]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_42_03 [Reducer 2], Default Resources= 2018-01-05T11:29:11,637 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 2 2018-01-05T11:29:11,637 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_42_03 [Reducer 2] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:11,637 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_42_04 [Reducer 5]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_42_04 [Reducer 5], Default Resources= 2018-01-05T11:29:11,637 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 5 2018-01-05T11:29:11,637 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_42_04 [Reducer 5] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:11,637 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_42_05 [Map 8]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_42_05 [Map 8], Default Resources= 2018-01-05T11:29:11,637 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional inputs for vertexvertex_1515180518813_0001_42_05 [Map 8] 2018-01-05T11:29:11,637 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Map 8 2018-01-05T11:29:11,637 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_42_05 [Map 8] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:11,638 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_42_06 [Reducer 3]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_42_06 [Reducer 3], Default Resources= 2018-01-05T11:29:11,638 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 3 2018-01-05T11:29:11,638 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_42_06 [Reducer 3] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:11,638 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Default container context for vertex_1515180518813_0001_42_07 [Reducer 4]=LocalResources: [[ name=tez-conf.pb, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001/tez-conf.pb" } size: 191039 timestamp: 1515180518000 type: FILE visibility: APPLICATION],[ name=hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar, value=resource { scheme: "file" port: -1 file: "/Users/djaiswal/hive/hive/itests/qtest/target/localfs/jar/hive-exec-3.0.0-SNAPSHOT-ddd79d9b61ac5c1a44d377bc92bc04a715e8a64794287fda68fa4e7b043f31a4.jar" } size: 37724412 timestamp: 1515180518000 type: FILE visibility: PRIVATE],], environment: [[ SHELL=/bin/bash ],[ LD_LIBRARY_PATH=$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$PWD:$HADOOP_COMMON_HOME/lib/native:$PWD:$HADOOP_COMMON_HOME/lib/native/ ],[ HADOOP_ROOT_LOGGER=INFO,CLA ],[ CLASSPATH=$PWD:$PWD/*: ],], credentials(token kinds): [tez.job,], javaOpts: -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Dlog4j.configurationFile=tez-container-log4j2.properties -Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir= -Dtez.root.logger=INFO,CLA , vertex: vertex_1515180518813_0001_42_07 [Reducer 4], Default Resources= 2018-01-05T11:29:11,638 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Setting 1 additional outputs for vertex vertex_1515180518813_0001_42_07 [Reducer 4] 2018-01-05T11:29:11,638 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Using ExecutionContext from Vertex for Vertex Reducer 4 2018-01-05T11:29:11,638 INFO [IPC Server handler 0 on 49823] impl.VertexImpl: Running vertex: vertex_1515180518813_0001_42_07 [Reducer 4] : TaskScheduler=1:LLAP, ContainerLauncher=1:LLAP, TaskCommunicator=1:LLAP 2018-01-05T11:29:11,638 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled 2018-01-05T11:29:11,638 INFO [IPC Server handler 0 on 49823] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:DAG_INITIALIZED]: dagID=dag_1515180518813_0001_42, initTime=1515180551638 2018-01-05T11:29:11,638 INFO [IPC Server handler 0 on 49823] impl.DAGImpl: dag_1515180518813_0001_42 transitioned from NEW to INITED due to event DAG_INIT 2018-01-05T11:29:11,638 INFO [Dispatcher thread {Central}] impl.DAGImpl: Added additional resources : [[]] to classpath 2018-01-05T11:29:11,638 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:DAG_STARTED]: dagID=dag_1515180518813_0001_42, startTime=1515180551638 2018-01-05T11:29:11,638 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_42 transitioned from INITED to RUNNING due to event DAG_START 2018-01-05T11:29:11,638 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 6 : {c={InputName=c}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:29:11,638 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: c, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:29:11,638 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_42_00 [Map 6] 2018-01-05T11:29:11,638 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] client.TezClient: Submitted dag to TezSession, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001, dagId=dag_1515180518813_0001_42, dagName=select count(*) from tab a join ta...c.value (Stage-1) 2018-01-05T11:29:11,638 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,638 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,638 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,642 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_42_00 [Map 6] 2018-01-05T11:29:11,642 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_42_00 [Map 6] 2018-01-05T11:29:11,642 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:11,642 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_42_00 [Map 6] 2018-01-05T11:29:11,644 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:29:11,645 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@1fce21de 2018-01-05T11:29:11,645 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/djaiswal/_tez_scratch_dir/9493e99b-e6bc-44b7-b894-124305abb6f5/map.xml 2018-01-05T11:29:11,645 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/djaiswal/_tez_scratch_dir/9493e99b-e6bc-44b7-b894-124305abb6f5/map.xml 2018-01-05T11:29:11,645 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/djaiswal/_tez_scratch_dir/9493e99b-e6bc-44b7-b894-124305abb6f5/map.xml 2018-01-05T11:29:11,645 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:11,645 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:11,645 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:11,647 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:11,647 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 6 size: 5.16KB 2018-01-05T11:29:11,647 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_42_00 [Map 6] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:29:11,647 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting user vertex manager plugin: org.apache.tez.dag.library.vertexmanager.ShuffleVertexManager on vertex: vertex_1515180518813_0001_42_01 [Reducer 7] 2018-01-05T11:29:11,648 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.0 maxFrac: 0.0 auto: false desiredTaskIput: 104857600 2018-01-05T11:29:11,648 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:29:11,648 INFO [InputInitializer {Map 6} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: c on vertex vertex_1515180518813_0001_42_00 [Map 6] 2018-01-05T11:29:11,648 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 1 tasks for vertex: vertex_1515180518813_0001_42_01 [Reducer 7] 2018-01-05T11:29:11,648 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_42_01 [Reducer 7] 2018-01-05T11:29:11,648 INFO [InputInitializer {Map 6} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:29:11,648 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_42_01, reconfigureDoneTime=1515180551648, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:29:11,648 DEBUG [InputInitializer {Map 6} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:11,648 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:VERTEX_INITIALIZED]: vertexName=Reducer 7, vertexId=vertex_1515180518813_0001_42_01, initRequestedTime=1515180551647, initedTime=1515180551648, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:11,648 INFO [InputInitializer {Map 6} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:29:11,648 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_42_01 [Reducer 7] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:29:11,648 DEBUG [InputInitializer {Map 6} #0] log.PerfLogger: 2018-01-05T11:29:11,648 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 1 : {a={InputName=a}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:29:11,648 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: a, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:29:11,648 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_42_02 [Map 1] 2018-01-05T11:29:11,648 INFO [InputInitializer {Map 6} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/djaiswal/_tez_scratch_dir/9493e99b-e6bc-44b7-b894-124305abb6f5/map.xml 2018-01-05T11:29:11,648 DEBUG [InputInitializer {Map 6} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:11,648 INFO [InputInitializer {Map 6} #0] exec.Utilities: Processing alias c 2018-01-05T11:29:11,648 DEBUG [InputInitializer {Map 6} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1 2018-01-05T11:29:11,648 INFO [InputInitializer {Map 6} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1 2018-01-05T11:29:11,648 DEBUG [InputInitializer {Map 6} #0] io.HiveInputFormat: Pushdown initiated with filterText = value is not null, filterExpr = GenericUDFOPNotNull(Column[value]), serializedFilterExpr = AQEAamF2YS51dGlsLkFycmF5TGlz9AEBAQFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVDb2x1bW5EZXPjAQF2YWx15QAAAYJjAQJvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnNlcmRlMi50eXBlaW5mby5QcmltaXRpdmVUeXBlSW5m7wEBc3RyaW7nAQNvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnVkZi5nZW5lcmljLkdlbmVyaWNVREZPUE5vdE51bOwBAQRvcmcuYXBhY2hlLmhhZG9vcC5pby5Cb29sZWFuV3JpdGFibOUBAAABAgEBYm9vbGVh7g== (desc) 2018-01-05T11:29:11,648 INFO [InputInitializer {Map 6} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 1 2018-01-05T11:29:11,648 INFO [InputInitializer {Map 6} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = value 2018-01-05T11:29:11,648 INFO [InputInitializer {Map 6} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1 2018-01-05T11:29:11,648 DEBUG [InputInitializer {Map 6} #0] io.HiveInputFormat: Pushdown initiated with filterText = value is not null, filterExpr = GenericUDFOPNotNull(Column[value]), serializedFilterExpr = AQEAamF2YS51dGlsLkFycmF5TGlz9AEBAQFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVDb2x1bW5EZXPjAQF2YWx15QAAAYJjAQJvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnNlcmRlMi50eXBlaW5mby5QcmltaXRpdmVUeXBlSW5m7wEBc3RyaW7nAQNvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnVkZi5nZW5lcmljLkdlbmVyaWNVREZPUE5vdE51bOwBAQRvcmcuYXBhY2hlLmhhZG9vcC5pby5Cb29sZWFuV3JpdGFibOUBAAABAgEBYm9vbGVh7g== (desc) 2018-01-05T11:29:11,648 DEBUG [InputInitializer {Map 6} #0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:11,648 DEBUG [InputInitializer {Map 6} #0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:11,648 DEBUG [InputInitializer {Map 6} #0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:11,648 DEBUG [InputInitializer {Map 6} #0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:11,651 DEBUG [AMReporterQueueDrainer] impl.AMReporter: Removing am 10.22.8.182:49824 with last associated dag QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=32} from heartbeat with taskCount=0, amFailed=false, isDone=true 2018-01-05T11:29:11,653 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_42_02 [Map 1] 2018-01-05T11:29:11,653 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_42_02 [Map 1] 2018-01-05T11:29:11,653 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:11,653 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_42_02 [Map 1] 2018-01-05T11:29:11,655 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:29:11,655 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@9724072 2018-01-05T11:29:11,655 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/djaiswal/_tez_scratch_dir/79b99006-4f31-4e98-bc9a-2de4e79e98fe/map.xml 2018-01-05T11:29:11,655 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/djaiswal/_tez_scratch_dir/79b99006-4f31-4e98-bc9a-2de4e79e98fe/map.xml 2018-01-05T11:29:11,655 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/djaiswal/_tez_scratch_dir/79b99006-4f31-4e98-bc9a-2de4e79e98fe/map.xml 2018-01-05T11:29:11,655 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:11,655 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:11,655 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:11,656 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:11,656 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 5.49KB 2018-01-05T11:29:11,656 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_42_02 [Map 1] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:29:11,656 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to ShuffleVertexManager for vertex_1515180518813_0001_42_03 [Reducer 2] 2018-01-05T11:29:11,656 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: a on vertex vertex_1515180518813_0001_42_02 [Map 1] 2018-01-05T11:29:11,656 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:29:11,656 DEBUG [InputInitializer {Map 1} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:11,656 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:29:11,656 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:11,656 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/djaiswal/_tez_scratch_dir/79b99006-4f31-4e98-bc9a-2de4e79e98fe/map.xml 2018-01-05T11:29:11,656 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:11,656 INFO [InputInitializer {Map 1} #0] exec.Utilities: Processing alias a 2018-01-05T11:29:11,656 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:11,656 INFO [InputInitializer {Map 1} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:11,657 DEBUG [InputInitializer {Map 1} #0] io.HiveInputFormat: Pushdown initiated with filterText = (key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter)))), filterExpr = GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPNotNull(Column[value]), GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[value], DynamicValue(RS_10_c_value_min), DynamicValue(RS_10_c_value_max)), GenericUDFInBloomFilter(Column[value], DynamicValue(RS_10_c_value_bloom_filter)))), serializedFilterExpr = AQEAamF2YS51dGlsLkFycmF5TGlz9AEDAQFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVHZW5lcmljRnVuY0Rlc+MBAQABAQECb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC5wbGFuLkV4cHJOb2RlQ29sdW1uRGVz4wEBa2X5AAABgmEBA29yZy5hcGFjaGUuaGFkb29wLmhpdmUuc2VyZGUyLnR5cGVpbmZvLlByaW1pdGl2ZVR5cGVJbmbvAQFpbvQBBG9yZy5hcGFjaGUuaGFkb29wLmhpdmUucWwudWRmLmdlbmVyaWMuR2VuZXJpY1VERk9QTm90TnVs7AEBBW9yZy5hcGFjaGUuaGFkb29wLmlvLkJvb2xlYW5Xcml0YWJs5QEAAAEDAQFib29sZWHuAQEBAQABAQECAQF2YWx15QAACAEDAQFzdHJpbucBBAEBBQEAAAEDDQEBAQEAAQIBAQEBAAEEAQZvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVDb25zdGFudERlc+MBAQMNBQABAgESAAAIAQMTAQdvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVEeW5hbWljVmFsdWVEZXPjAQEIb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC5wbGFuLkR5bmFtaWNWYWx15QEBUlNfMTBfY192YWx1ZV9tae4BCdYBb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5zZXJkZTIub2JqZWN0aW5zcGVjdG9yLnByaW1pdGl2ZS5Xcml0YWJsZVN0cmluZ09iamVjdEluc3BlY3RvcgEBAxMBAxMBAxMBBwEBCAEBUlNfMTBfY192YWx1ZV9tYfgBCSABAxMBAxMBCm9yZy5hcGFjaGUuaGFkb29wLmhpdmUucWwudWRmLmdlbmVyaWMuR2VuZXJpY1VERkJldHdlZe4BAQvBAVtMb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5zZXJkZTIub2JqZWN0aW5zcGVjdG9yLk9iamVjdEluc3BlY3RvcjsBBQEM3wFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnNlcmRlMi5vYmplY3RpbnNwZWN0b3IucHJpbWl0aXZlLldyaXRhYmxlQ29uc3RhbnRCb29sZWFuT2JqZWN0SW5zcGVjdG9yAQEDDQEFAQABCSABCSABCSABDcUBb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGT1BFcXVhbE9yR3JlYXRlclRoYW4BAAECAT69AUVRVUFMIE9SIEdSRUFURVIgVEhBzgEFAQABDsIBb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGT1BFcXVhbE9yTGVzc1RoYW4BACkBPL0BRVFVQUwgT1IgTEVTUyBUSEHOAQUBAAEFAQAAAQMNAQEBAQABAgECHAEHAQEIAQFSU18xMF9jX3ZhbHVlX2Jsb29tX2ZpbHRl8gEP1gFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnNlcmRlMi5vYmplY3RpbnNwZWN0b3IucHJpbWl0aXZlLldyaXRhYmxlQmluYXJ5T2JqZWN0SW5zcGVjdG9yAQEDAQFiaW5hcvkBAzgBAzgBEG9yZy5hcGFjaGUuaGFkb29wLmhpdmUucWwudWRmLmdlbmVyaWMuR2VuZXJpY1VERkluQmxvb21GaWx0ZfIBAAEDDQERb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGT1BBbuQBAQUBAAABAw0BEQEBBQEAAAEDDQ== (desc) 2018-01-05T11:29:11,657 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 0,1 2018-01-05T11:29:11,657 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = key,value 2018-01-05T11:29:11,657 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:11,657 DEBUG [InputInitializer {Map 1} #0] io.HiveInputFormat: Pushdown initiated with filterText = (key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter)))), filterExpr = GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPNotNull(Column[value]), GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[value], DynamicValue(RS_10_c_value_min), DynamicValue(RS_10_c_value_max)), GenericUDFInBloomFilter(Column[value], DynamicValue(RS_10_c_value_bloom_filter)))), serializedFilterExpr = AQEAamF2YS51dGlsLkFycmF5TGlz9AEDAQFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVHZW5lcmljRnVuY0Rlc+MBAQABAQECb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC5wbGFuLkV4cHJOb2RlQ29sdW1uRGVz4wEBa2X5AAABgmEBA29yZy5hcGFjaGUuaGFkb29wLmhpdmUuc2VyZGUyLnR5cGVpbmZvLlByaW1pdGl2ZVR5cGVJbmbvAQFpbvQBBG9yZy5hcGFjaGUuaGFkb29wLmhpdmUucWwudWRmLmdlbmVyaWMuR2VuZXJpY1VERk9QTm90TnVs7AEBBW9yZy5hcGFjaGUuaGFkb29wLmlvLkJvb2xlYW5Xcml0YWJs5QEAAAEDAQFib29sZWHuAQEBAQABAQECAQF2YWx15QAACAEDAQFzdHJpbucBBAEBBQEAAAEDDQEBAQEAAQIBAQEBAAEEAQZvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVDb25zdGFudERlc+MBAQMNBQABAgESAAAIAQMTAQdvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVEeW5hbWljVmFsdWVEZXPjAQEIb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC5wbGFuLkR5bmFtaWNWYWx15QEBUlNfMTBfY192YWx1ZV9tae4BCdYBb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5zZXJkZTIub2JqZWN0aW5zcGVjdG9yLnByaW1pdGl2ZS5Xcml0YWJsZVN0cmluZ09iamVjdEluc3BlY3RvcgEBAxMBAxMBAxMBBwEBCAEBUlNfMTBfY192YWx1ZV9tYfgBCSABAxMBAxMBCm9yZy5hcGFjaGUuaGFkb29wLmhpdmUucWwudWRmLmdlbmVyaWMuR2VuZXJpY1VERkJldHdlZe4BAQvBAVtMb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5zZXJkZTIub2JqZWN0aW5zcGVjdG9yLk9iamVjdEluc3BlY3RvcjsBBQEM3wFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnNlcmRlMi5vYmplY3RpbnNwZWN0b3IucHJpbWl0aXZlLldyaXRhYmxlQ29uc3RhbnRCb29sZWFuT2JqZWN0SW5zcGVjdG9yAQEDDQEFAQABCSABCSABCSABDcUBb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGT1BFcXVhbE9yR3JlYXRlclRoYW4BAAECAT69AUVRVUFMIE9SIEdSRUFURVIgVEhBzgEFAQABDsIBb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGT1BFcXVhbE9yTGVzc1RoYW4BACkBPL0BRVFVQUwgT1IgTEVTUyBUSEHOAQUBAAEFAQAAAQMNAQEBAQABAgECHAEHAQEIAQFSU18xMF9jX3ZhbHVlX2Jsb29tX2ZpbHRl8gEP1gFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnNlcmRlMi5vYmplY3RpbnNwZWN0b3IucHJpbWl0aXZlLldyaXRhYmxlQmluYXJ5T2JqZWN0SW5zcGVjdG9yAQEDAQFiaW5hcvkBAzgBAzgBEG9yZy5hcGFjaGUuaGFkb29wLmhpdmUucWwudWRmLmdlbmVyaWMuR2VuZXJpY1VERkluQmxvb21GaWx0ZfIBAAEDDQERb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGT1BBbuQBAQUBAAABAw0BEQEBBQEAAAEDDQ== (desc) 2018-01-05T11:29:11,657 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: getSplits started 2018-01-05T11:29:11,658 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: Number of buckets specified by conf file is 2 2018-01-05T11:29:11,658 INFO [InputInitializer {Map 1} #0] orc.OrcInputFormat: ORC pushdown predicate: leaf-0 = (IS_NULL key), leaf-1 = (IS_NULL value), leaf-2 = (BETWEEN value DynamicValue(RS_10_c_value_min) DynamicValue(RS_10_c_value_max)), expr = (and (not leaf-0) (not leaf-1) leaf-2) 2018-01-05T11:29:11,658 INFO [InputInitializer {Map 1} #0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:11,658 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: Generate splits schema evolution property true reader schema struct transactional scan property false 2018-01-05T11:29:11,658 ERROR [ORC_GET_SPLITS #7] io.AcidUtils: Failed to get files with ID; using regular API: Only supported for DFS; got class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:11,661 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.05 maxFrac: 0.75 auto: false desiredTaskIput: 104857600 2018-01-05T11:29:11,661 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:29:11,661 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 1 tasks for vertex: vertex_1515180518813_0001_42_03 [Reducer 2] 2018-01-05T11:29:11,661 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_42_03 [Reducer 2] 2018-01-05T11:29:11,661 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_42_03, reconfigureDoneTime=1515180551661, numTasks=1, vertexLocationHint=null, edgeManagersCount=2, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:29:11,661 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:VERTEX_INITIALIZED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_42_03, initRequestedTime=1515180551656, initedTime=1515180551661, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:11,662 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_42_03 [Reducer 2] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:29:11,662 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting user vertex manager plugin: org.apache.tez.dag.library.vertexmanager.ShuffleVertexManager on vertex: vertex_1515180518813_0001_42_04 [Reducer 5] 2018-01-05T11:29:11,662 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.0 maxFrac: 0.0 auto: false desiredTaskIput: 104857600 2018-01-05T11:29:11,662 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:29:11,662 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 1 tasks for vertex: vertex_1515180518813_0001_42_04 [Reducer 5] 2018-01-05T11:29:11,662 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_42_04 [Reducer 5] 2018-01-05T11:29:11,662 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_42_04, reconfigureDoneTime=1515180551662, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:29:11,662 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:VERTEX_INITIALIZED]: vertexName=Reducer 5, vertexId=vertex_1515180518813_0001_42_04, initRequestedTime=1515180551662, initedTime=1515180551662, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:11,662 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_42_04 [Reducer 5] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:29:11,662 INFO [Dispatcher thread {Central}] impl.VertexImpl: Root Inputs exist for Vertex: Map 8 : {b={InputName=b}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInputLegacy, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator, hasPayload=false}} 2018-01-05T11:29:11,662 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting root input initializer for input: b, with class: [org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator] 2018-01-05T11:29:11,662 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to RootInputVertexManager for vertex_1515180518813_0001_42_05 [Map 8] 2018-01-05T11:29:11,665 DEBUG [InputInitializer {Map 6} #0] mapred.FileInputFormat: Time taken to get FileStatuses: 16 2018-01-05T11:29:11,665 INFO [InputInitializer {Map 6} #0] mapred.FileInputFormat: Total input files to process : 1 2018-01-05T11:29:11,665 DEBUG [InputInitializer {Map 6} #0] mapred.FileInputFormat: Total # of splits generated by getSplits: 1, TimeTaken: 16 2018-01-05T11:29:11,665 INFO [InputInitializer {Map 6} #0] io.HiveInputFormat: number of splits 1 2018-01-05T11:29:11,665 DEBUG [InputInitializer {Map 6} #0] log.PerfLogger: 2018-01-05T11:29:11,665 INFO [InputInitializer {Map 6} #0] tez.HiveSplitGenerator: Number of input splits: 1. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:29:11,665 INFO [InputInitializer {Map 6} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/djaiswal/_tez_scratch_dir/9493e99b-e6bc-44b7-b894-124305abb6f5/map.xml 2018-01-05T11:29:11,665 DEBUG [InputInitializer {Map 6} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/djaiswal/_tez_scratch_dir/9493e99b-e6bc-44b7-b894-124305abb6f5/map.xml 2018-01-05T11:29:11,665 DEBUG [InputInitializer {Map 6} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/djaiswal/_tez_scratch_dir/9493e99b-e6bc-44b7-b894-124305abb6f5/map.xml 2018-01-05T11:29:11,665 DEBUG [InputInitializer {Map 6} #0] log.PerfLogger: 2018-01-05T11:29:11,665 INFO [InputInitializer {Map 6} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:11,665 DEBUG [InputInitializer {Map 6} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:11,666 INFO [Dispatcher thread {Central}] impl.VertexImpl: Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1515180518813_0001_42_05 [Map 8] 2018-01-05T11:29:11,666 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex will initialize from input initializer. vertex_1515180518813_0001_42_05 [Map 8] 2018-01-05T11:29:11,666 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:11,666 INFO [Dispatcher thread {Central}] impl.VertexImpl: Starting 1 inputInitializers for vertex vertex_1515180518813_0001_42_05 [Map 8] 2018-01-05T11:29:11,667 DEBUG [InputInitializer {Map 6} #0] log.PerfLogger: 2018-01-05T11:29:11,667 INFO [InputInitializer {Map 6} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 6 size: 5.16KB 2018-01-05T11:29:11,667 DEBUG [InputInitializer {Map 6} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1/kv3.txt to src new group? true 2018-01-05T11:29:11,667 INFO [InputInitializer {Map 6} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:29:11,667 INFO [InputInitializer {Map 6} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:29:11,667 INFO [InputInitializer {Map 6} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:29:11,667 INFO [InputInitializer {Map 6} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 4 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 216 Original splits: 1 2018-01-05T11:29:11,667 INFO [InputInitializer {Map 6} #0] grouper.TezSplitGrouper: Using original number of splits: 1 desired splits: 1 2018-01-05T11:29:11,667 INFO [InputInitializer {Map 6} #0] tez.SplitGrouper: Original split count is 1 grouped split count is 1, for bucket: 1 2018-01-05T11:29:11,667 INFO [InputInitializer {Map 6} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:29:11,667 INFO [InputInitializer {Map 6} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: c on vertex vertex_1515180518813_0001_42_00 [Map 6] 2018-01-05T11:29:11,669 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitGenerator using llap affinitized locations: false 2018-01-05T11:29:11,669 INFO [Dispatcher thread {Central}] tez.HiveSplitGenerator: SplitLocationProvider: org.apache.hadoop.hive.ql.exec.tez.Utils$1@56ac78e3 2018-01-05T11:29:11,669 INFO [Dispatcher thread {Central}] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/djaiswal/_tez_scratch_dir/bd86f586-febc-478a-827e-ba77f0e7e2c1/map.xml 2018-01-05T11:29:11,669 DEBUG [Dispatcher thread {Central}] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/djaiswal/_tez_scratch_dir/bd86f586-febc-478a-827e-ba77f0e7e2c1/map.xml 2018-01-05T11:29:11,669 DEBUG [Dispatcher thread {Central}] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/djaiswal/_tez_scratch_dir/bd86f586-febc-478a-827e-ba77f0e7e2c1/map.xml 2018-01-05T11:29:11,669 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:11,669 INFO [Dispatcher thread {Central}] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:11,669 DEBUG [Dispatcher thread {Central}] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:11,671 DEBUG [Dispatcher thread {Central}] log.PerfLogger: 2018-01-05T11:29:11,671 INFO [Dispatcher thread {Central}] exec.Utilities: Deserialized plan (via RPC) - name: Map 8 size: 5.29KB 2018-01-05T11:29:11,671 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_42_05 [Map 8] transitioned from NEW to INITIALIZING due to event V_INIT 2018-01-05T11:29:11,671 INFO [InputInitializer {Map 8} #0] dag.RootInputInitializerManager: Starting InputInitializer for Input: b on vertex vertex_1515180518813_0001_42_05 [Map 8] 2018-01-05T11:29:11,671 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to ShuffleVertexManager for vertex_1515180518813_0001_42_06 [Reducer 3] 2018-01-05T11:29:11,671 INFO [App Shared Pool - #2] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_42_00 [Map 6] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:11,671 INFO [InputInitializer {Map 8} #0] tez.HiveSplitGenerator: GenerateConsistentSplitsInHive=true 2018-01-05T11:29:11,671 INFO [App Shared Pool - #2] impl.VertexImpl: Got updated RootInputsSpecs: {c=forAllWorkUnits=true, update=[1]} 2018-01-05T11:29:11,671 DEBUG [InputInitializer {Map 8} #0] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:11,671 INFO [App Shared Pool - #2] impl.VertexImpl: Vertex vertex_1515180518813_0001_42_00 [Map 6] parallelism set to 1 2018-01-05T11:29:11,671 INFO [InputInitializer {Map 8} #0] tez.HiveSplitGenerator: The preferred split size is 52428800 2018-01-05T11:29:11,671 INFO [App Shared Pool - #2] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_42_00 [Map 6] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:11,671 DEBUG [InputInitializer {Map 8} #0] log.PerfLogger: 2018-01-05T11:29:11,671 INFO [InputInitializer {Map 8} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/djaiswal/_tez_scratch_dir/bd86f586-febc-478a-827e-ba77f0e7e2c1/map.xml 2018-01-05T11:29:11,671 DEBUG [InputInitializer {Map 8} #0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:11,671 INFO [InputInitializer {Map 8} #0] exec.Utilities: Processing alias b 2018-01-05T11:29:11,671 DEBUG [InputInitializer {Map 8} #0] exec.Utilities: Adding input file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:11,671 INFO [InputInitializer {Map 8} #0] exec.Utilities: Adding 1 inputs; the first input is file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:11,672 DEBUG [InputInitializer {Map 8} #0] io.HiveInputFormat: Pushdown initiated with filterText = (key is not null and (key BETWEEN DynamicValue(RS_12_a_key_min) AND DynamicValue(RS_12_a_key_max) and in_bloom_filter(key, DynamicValue(RS_12_a_key_bloom_filter)))), filterExpr = GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_12_a_key_min), DynamicValue(RS_12_a_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_12_a_key_bloom_filter)))), serializedFilterExpr = AQEAamF2YS51dGlsLkFycmF5TGlz9AECAQFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVHZW5lcmljRnVuY0Rlc+MBAQABAQECb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC5wbGFuLkV4cHJOb2RlQ29sdW1uRGVz4wEBa2X5AAABgmIBA29yZy5hcGFjaGUuaGFkb29wLmhpdmUuc2VyZGUyLnR5cGVpbmZvLlByaW1pdGl2ZVR5cGVJbmbvAQFpbvQBBG9yZy5hcGFjaGUuaGFkb29wLmhpdmUucWwudWRmLmdlbmVyaWMuR2VuZXJpY1VERk9QTm90TnVs7AEBBW9yZy5hcGFjaGUuaGFkb29wLmlvLkJvb2xlYW5Xcml0YWJs5QEAAAEDAQFib29sZWHuAQEBAQABAgEBAQEAAQQBBm9yZy5hcGFjaGUuaGFkb29wLmhpdmUucWwucGxhbi5FeHByTm9kZUNvbnN0YW50RGVz4wEBAw0FAAECAQcAAAgBAwkBB29yZy5hcGFjaGUuaGFkb29wLmhpdmUucWwucGxhbi5FeHByTm9kZUR5bmFtaWNWYWx1ZURlc+MBAQhvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRHluYW1pY1ZhbHXlAQFSU18xMl9hX2tleV9tae4BCdMBb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5zZXJkZTIub2JqZWN0aW5zcGVjdG9yLnByaW1pdGl2ZS5Xcml0YWJsZUludE9iamVjdEluc3BlY3RvcgEBAwkBAwkBAwkBBwEBCAEBUlNfMTJfYV9rZXlfbWH4AQkYAQMJAQMJAQpvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnVkZi5nZW5lcmljLkdlbmVyaWNVREZCZXR3ZWXuAQELwQFbTG9yZy5hcGFjaGUuaGFkb29wLmhpdmUuc2VyZGUyLm9iamVjdGluc3BlY3Rvci5PYmplY3RJbnNwZWN0b3I7AQUBDN8Bb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5zZXJkZTIub2JqZWN0aW5zcGVjdG9yLnByaW1pdGl2ZS5Xcml0YWJsZUNvbnN0YW50Qm9vbGVhbk9iamVjdEluc3BlY3RvcgEBAw0BBQEAAQkYAQkYAQkYAQ3FAW9yZy5hcGFjaGUuaGFkb29wLmhpdmUucWwudWRmLmdlbmVyaWMuR2VuZXJpY1VERk9QRXF1YWxPckdyZWF0ZXJUaGFuAQABAwE+vQFFUVVBTCBPUiBHUkVBVEVSIFRIQc4BBQEAAQ7CAW9yZy5hcGFjaGUuaGFkb29wLmhpdmUucWwudWRmLmdlbmVyaWMuR2VuZXJpY1VERk9QRXF1YWxPckxlc3NUaGFuAQAhATy9AUVRVUFMIE9SIExFU1MgVEhBzgEFAQABBQEAAAEDDQEBAQEAAQIBAhQBBwEBCAEBUlNfMTJfYV9rZXlfYmxvb21fZmlsdGXyAQ/WAW9yZy5hcGFjaGUuaGFkb29wLmhpdmUuc2VyZGUyLm9iamVjdGluc3BlY3Rvci5wcmltaXRpdmUuV3JpdGFibGVCaW5hcnlPYmplY3RJbnNwZWN0b3IBAQMBAWJpbmFy+QEDMAEDMAEQb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGSW5CbG9vbUZpbHRl8gEAAQMNARFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnVkZi5nZW5lcmljLkdlbmVyaWNVREZPUEFu5AEBBQEAAAEDDQERAQEFAQAAAQMN (desc) 2018-01-05T11:29:11,672 INFO [InputInitializer {Map 8} #0] io.HiveInputFormat: hive.io.file.readcolumn.ids = 0 2018-01-05T11:29:11,672 INFO [InputInitializer {Map 8} #0] io.HiveInputFormat: hive.io.file.readcolumn.names = key 2018-01-05T11:29:11,672 INFO [InputInitializer {Map 8} #0] io.HiveInputFormat: Generating splits for dirs: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:11,672 DEBUG [InputInitializer {Map 8} #0] io.HiveInputFormat: Pushdown initiated with filterText = (key is not null and (key BETWEEN DynamicValue(RS_12_a_key_min) AND DynamicValue(RS_12_a_key_max) and in_bloom_filter(key, DynamicValue(RS_12_a_key_bloom_filter)))), filterExpr = GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[key], DynamicValue(RS_12_a_key_min), DynamicValue(RS_12_a_key_max)), GenericUDFInBloomFilter(Column[key], DynamicValue(RS_12_a_key_bloom_filter)))), serializedFilterExpr = AQEAamF2YS51dGlsLkFycmF5TGlz9AECAQFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVHZW5lcmljRnVuY0Rlc+MBAQABAQECb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC5wbGFuLkV4cHJOb2RlQ29sdW1uRGVz4wEBa2X5AAABgmIBA29yZy5hcGFjaGUuaGFkb29wLmhpdmUuc2VyZGUyLnR5cGVpbmZvLlByaW1pdGl2ZVR5cGVJbmbvAQFpbvQBBG9yZy5hcGFjaGUuaGFkb29wLmhpdmUucWwudWRmLmdlbmVyaWMuR2VuZXJpY1VERk9QTm90TnVs7AEBBW9yZy5hcGFjaGUuaGFkb29wLmlvLkJvb2xlYW5Xcml0YWJs5QEAAAEDAQFib29sZWHuAQEBAQABAgEBAQEAAQQBBm9yZy5hcGFjaGUuaGFkb29wLmhpdmUucWwucGxhbi5FeHByTm9kZUNvbnN0YW50RGVz4wEBAw0FAAECAQcAAAgBAwkBB29yZy5hcGFjaGUuaGFkb29wLmhpdmUucWwucGxhbi5FeHByTm9kZUR5bmFtaWNWYWx1ZURlc+MBAQhvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRHluYW1pY1ZhbHXlAQFSU18xMl9hX2tleV9tae4BCdMBb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5zZXJkZTIub2JqZWN0aW5zcGVjdG9yLnByaW1pdGl2ZS5Xcml0YWJsZUludE9iamVjdEluc3BlY3RvcgEBAwkBAwkBAwkBBwEBCAEBUlNfMTJfYV9rZXlfbWH4AQkYAQMJAQMJAQpvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnVkZi5nZW5lcmljLkdlbmVyaWNVREZCZXR3ZWXuAQELwQFbTG9yZy5hcGFjaGUuaGFkb29wLmhpdmUuc2VyZGUyLm9iamVjdGluc3BlY3Rvci5PYmplY3RJbnNwZWN0b3I7AQUBDN8Bb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5zZXJkZTIub2JqZWN0aW5zcGVjdG9yLnByaW1pdGl2ZS5Xcml0YWJsZUNvbnN0YW50Qm9vbGVhbk9iamVjdEluc3BlY3RvcgEBAw0BBQEAAQkYAQkYAQkYAQ3FAW9yZy5hcGFjaGUuaGFkb29wLmhpdmUucWwudWRmLmdlbmVyaWMuR2VuZXJpY1VERk9QRXF1YWxPckdyZWF0ZXJUaGFuAQABAwE+vQFFUVVBTCBPUiBHUkVBVEVSIFRIQc4BBQEAAQ7CAW9yZy5hcGFjaGUuaGFkb29wLmhpdmUucWwudWRmLmdlbmVyaWMuR2VuZXJpY1VERk9QRXF1YWxPckxlc3NUaGFuAQAhATy9AUVRVUFMIE9SIExFU1MgVEhBzgEFAQABBQEAAAEDDQEBAQEAAQIBAhQBBwEBCAEBUlNfMTJfYV9rZXlfYmxvb21fZmlsdGXyAQ/WAW9yZy5hcGFjaGUuaGFkb29wLmhpdmUuc2VyZGUyLm9iamVjdGluc3BlY3Rvci5wcmltaXRpdmUuV3JpdGFibGVCaW5hcnlPYmplY3RJbnNwZWN0b3IBAQMBAWJpbmFy+QEDMAEDMAEQb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGSW5CbG9vbUZpbHRl8gEAAQMNARFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnVkZi5nZW5lcmljLkdlbmVyaWNVREZPUEFu5AEBBQEAAAEDDQERAQEFAQAAAQMN (desc) 2018-01-05T11:29:11,672 DEBUG [InputInitializer {Map 8} #0] orc.OrcInputFormat: getSplits started 2018-01-05T11:29:11,672 DEBUG [InputInitializer {Map 8} #0] orc.OrcInputFormat: Number of buckets specified by conf file is 4 2018-01-05T11:29:11,672 INFO [InputInitializer {Map 8} #0] orc.OrcInputFormat: ORC pushdown predicate: leaf-0 = (IS_NULL key), leaf-1 = (BETWEEN key DynamicValue(RS_12_a_key_min) DynamicValue(RS_12_a_key_max)), expr = (and (not leaf-0) leaf-1) 2018-01-05T11:29:11,672 INFO [InputInitializer {Map 8} #0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:11,672 DEBUG [InputInitializer {Map 8} #0] orc.OrcInputFormat: Generate splits schema evolution property true reader schema struct transactional scan property false 2018-01-05T11:29:11,672 ERROR [ORC_GET_SPLITS #0] io.AcidUtils: Failed to get files with ID; using regular API: Only supported for DFS; got class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:11,676 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.05 maxFrac: 0.75 auto: false desiredTaskIput: 104857600 2018-01-05T11:29:11,676 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:29:11,676 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 1 tasks for vertex: vertex_1515180518813_0001_42_06 [Reducer 3] 2018-01-05T11:29:11,676 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_42_06 [Reducer 3] 2018-01-05T11:29:11,676 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_42_06, reconfigureDoneTime=1515180551676, numTasks=1, vertexLocationHint=null, edgeManagersCount=2, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:29:11,676 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:VERTEX_INITIALIZED]: vertexName=Reducer 3, vertexId=vertex_1515180518813_0001_42_06, initRequestedTime=1515180551671, initedTime=1515180551676, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:11,676 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_42_06 [Reducer 3] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:29:11,676 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:VERTEX_INITIALIZED]: vertexName=Map 6, vertexId=vertex_1515180518813_0001_42_00, initRequestedTime=1515180551638, initedTime=1515180551676, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:11,676 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_42_00 [Map 6] with distanceFromRoot: 0 2018-01-05T11:29:11,676 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_42_00 [Map 6] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:29:11,676 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting vertexManager to ShuffleVertexManager for vertex_1515180518813_0001_42_07 [Reducer 4] 2018-01-05T11:29:11,680 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManagerBase: Settings minFrac: 0.05 maxFrac: 0.75 auto: false desiredTaskIput: 104857600 2018-01-05T11:29:11,680 INFO [Dispatcher thread {Central}] vertexmanager.ShuffleVertexManager: minTaskParallelism 1 2018-01-05T11:29:11,680 INFO [Dispatcher thread {Central}] impl.VertexImpl: Creating 1 tasks for vertex: vertex_1515180518813_0001_42_07 [Reducer 4] 2018-01-05T11:29:11,680 INFO [Dispatcher thread {Central}] impl.VertexImpl: Directly initializing vertex: vertex_1515180518813_0001_42_07 [Reducer 4] 2018-01-05T11:29:11,680 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_42_07, reconfigureDoneTime=1515180551680, numTasks=1, vertexLocationHint=null, edgeManagersCount=1, rootInputSpecUpdateCount=0, setParallelismCalledFlag=false 2018-01-05T11:29:11,680 INFO [Dispatcher thread {Central}] impl.VertexImpl: Setting up committers for vertex vertex_1515180518813_0001_42_07 [Reducer 4], numAdditionalOutputs=1 2018-01-05T11:29:11,680 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:VERTEX_INITIALIZED]: vertexName=Reducer 4, vertexId=vertex_1515180518813_0001_42_07, initRequestedTime=1515180551676, initedTime=1515180551680, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, additionalInputsCount=0, initGeneratedEventsCount=0, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:11,680 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_42_07 [Reducer 4] transitioned from NEW to INITED due to event V_INIT 2018-01-05T11:29:11,680 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_42_00, startRequestedTime=1515180551647, startedTime=1515180551680 2018-01-05T11:29:11,680 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_42_00, reconfigureDoneTime=1515180551680, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=0, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:29:11,680 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_42_00 [Map 6] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:11,680 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_42_00 for vertex: vertex_1515180518813_0001_42_01 [Reducer 7] numStartedSources: 1 numSources: 1 2018-01-05T11:29:11,680 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_42_01 [Reducer 7] with distanceFromRoot: 1 2018-01-05T11:29:11,680 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_42_00 for vertex: vertex_1515180518813_0001_42_03 [Reducer 2] numStartedSources: 1 numSources: 2 2018-01-05T11:29:11,680 INFO [Dispatcher thread {Central}] impl.VertexImpl: Cannot start vertex: vertex_1515180518813_0001_42_03 [Reducer 2] numStartedSources: 1 numSources: 2 2018-01-05T11:29:11,680 INFO [App Shared Pool - #2] impl.RootInputVertexManager: All source tasks completed. Ramping up 1 remaining tasks for vertex: Map 6 2018-01-05T11:29:11,680 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_42_01, startRequestedTime=1515180551680, startedTime=1515180551680 2018-01-05T11:29:11,680 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_42_01 [Reducer 7] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:11,680 INFO [App Shared Pool - #2] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 6 with totalTasks: 1. 0 source tasks completed out of 0. MinSourceTaskCompletedFraction: 1.0 min: 0.0 max: 0.0 2018-01-05T11:29:11,681 INFO [App Shared Pool - #2] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_42_00 [Map 6] 2018-01-05T11:29:11,681 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_42_01 for vertex: vertex_1515180518813_0001_42_02 [Map 1] numStartedSources: 1 numSources: 1 2018-01-05T11:29:11,681 INFO [Dispatcher thread {Central}] impl.VertexImpl: Cannot start vertex. Not in inited state. vertex_1515180518813_0001_42_02 [Map 1] . VertesState: INITIALIZING numTasks: -1 Num uninitialized edges: 0 2018-01-05T11:29:11,681 INFO [App Shared Pool - #3] vertexmanager.ShuffleVertexManagerBase: OnVertexStarted vertex: Reducer 7 with 0 source tasks and 1 pending tasks 2018-01-05T11:29:11,681 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:TASK_STARTED]: vertexName=Map 6, taskId=task_1515180518813_0001_42_00_000000, scheduledTime=1515180551681, launchTime=1515180551681 2018-01-05T11:29:11,681 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_42_00 [Map 6] 2018-01-05T11:29:11,681 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Map 6 in vertex: Reducer 7 numBipartiteSourceTasks: 1 2018-01-05T11:29:11,681 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: Scheduling 1 tasks for vertex: Reducer 7 with totalTasks: 1. 0 source tasks completed out of 1. MinSourceTaskCompletedFraction: 0.0 min: 0.0 max: 0.0 2018-01-05T11:29:11,681 INFO [App Shared Pool - #2] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Map 6, destinationVertexName=Reducer 7. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.UnorderedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:29:11,681 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_42_00_000000_0, priority=23, capability=, hosts=null 2018-01-05T11:29:11,681 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=23,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:11,681 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:TASK_STARTED]: vertexName=Reducer 7, taskId=task_1515180518813_0001_42_01_000000, scheduledTime=1515180551681, launchTime=1515180551681 2018-01-05T11:29:11,681 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=23,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:11,681 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:11,681 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_42_00_000000_0 on hosts=null 2018-01-05T11:29:11,682 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_42_00_000000_0 on hosts=null 2018-01-05T11:29:11,682 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_42_01 [Reducer 7] 2018-01-05T11:29:11,682 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_42_00_000000_0 2018-01-05T11:29:11,682 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:11,682 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_42_01_000000_0, priority=50, capability=, hosts=null 2018-01-05T11:29:11,682 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #105, task=TaskInfo{task=attempt_1515180518813_0001_42_00_000000_0, priority=23, startTime=0, containerId=null, uniqueId=104, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=0, ac=8, commF=false, disabled=false}, to container=container_111101111_0001_00_000105 2018-01-05T11:29:11,682 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_42_00_000000_0; false 2018-01-05T11:29:11,682 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_42_00_000000_0, priority=23, startTime=311572191, containerId=container_111101111_0001_00_000105assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, uniqueId=104, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:11,682 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=50,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:11,682 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=50,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:11,682 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:11,682 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_42_01_000000_0 on hosts=null 2018-01-05T11:29:11,682 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_42_01_000000_0 on hosts=null 2018-01-05T11:29:11,682 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_42_01_000000_0 2018-01-05T11:29:11,682 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:11,682 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #106, task=TaskInfo{task=attempt_1515180518813_0001_42_01_000000_0, priority=50, startTime=0, containerId=null, uniqueId=105, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=1, ac=7, commF=false, disabled=false}, to container=container_111101111_0001_00_000106 2018-01-05T11:29:11,682 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_42_01_000000_0; false 2018-01-05T11:29:11,682 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_42_01_000000_0, priority=50, startTime=311572192, containerId=container_111101111_0001_00_000106assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=2, ac=6, commF=false, disabled=false}, uniqueId=105, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:11,682 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000105 for node: 10.22.8.182:49818 2018-01-05T11:29:11,684 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: CurrentDagId set to: 42, name=select count(*) from tab a join ta...c.value (Stage-1), queryId=djaiswal_20180105112911_d60e4252-4a13-4247-a28a-f07830bbf022 2018-01-05T11:29:11,684 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000105, attempt_1515180518813_0001_42_00_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:11,684 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000105 for node: 10.22.8.182:49818 2018-01-05T11:29:11,684 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000106 for node: 10.22.8.182:49818 2018-01-05T11:29:11,685 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000106, attempt_1515180518813_0001_42_01_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:11,685 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000106 for node: 10.22.8.182:49818 2018-01-05T11:29:11,685 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000105 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:11,685 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000105, launchTime=1515180551685 2018-01-05T11:29:11,685 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_42_00_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112911_d60e4252-4a13-4247-a28a-f07830bbf022, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000105, dagName=select count(*) from tab a join ta...c.value (Stage-1), vertexName=Map 6, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=1, numOutputs=2, numGroupedInputs=0, Inputs={{c,org.apache.tez.mapreduce.input.MRInputLegacy,1}}, Outputs={{Reducer 7,org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput,1}{Reducer 2,org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput,1}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=1, completedTaskCount=0, dagStartTime=1515180551638, firstAttemptStartTime=1515180551681, currentAttemptStartTime=1515180551684} 2018-01-05T11:29:11,685 INFO [IPC Server handler 0 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000105, QueryId=djaiswal_20180105112911_d60e4252-4a13-4247-a28a-f07830bbf022-42, DagId=42, VertexName=Map 6, TaskId=0, TaskAttemptId=0, SubmitTime=1515180551685 2018-01-05T11:29:11,685 DEBUG [IPC Server handler 0 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=42} with the ShuffleHandler 2018-01-05T11:29:11,685 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_42_00_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000105] on NM: [10.22.8.182:49818] 2018-01-05T11:29:11,685 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:11,685 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:11,686 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 6, taskAttemptId=attempt_1515180518813_0001_42_00_000000_0, startTime=1515180551685, containerId=container_111101111_0001_00_000105, nodeId=10.22.8.182:49818 2018-01-05T11:29:11,686 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000106 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:11,686 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000106, launchTime=1515180551686 2018-01-05T11:29:11,686 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_42_01_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000106] on NM: [10.22.8.182:49818] 2018-01-05T11:29:11,686 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:11,686 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:11,686 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:TASK_ATTEMPT_STARTED]: vertexName=Reducer 7, taskAttemptId=attempt_1515180518813_0001_42_01_000000_0, startTime=1515180551686, containerId=container_111101111_0001_00_000106, nodeId=10.22.8.182:49818 2018-01-05T11:29:11,688 DEBUG [ORC_GET_SPLITS #7] io.AcidUtils: in directory file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 base = null deltas = 0 2018-01-05T11:29:11,688 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: Split strategy: ETLSplitStrategy strategy for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08 2018-01-05T11:29:11,688 DEBUG [ORC_GET_SPLITS #9] orc.LocalCache: Serialized tail cached for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0 2018-01-05T11:29:11,688 DEBUG [ORC_GET_SPLITS #9] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:11,688 DEBUG [ORC_GET_SPLITS #9] orc.LocalCache: Serialized tail cached for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0 2018-01-05T11:29:11,688 DEBUG [ORC_GET_SPLITS #9] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:11,688 DEBUG [ORC_GET_SPLITS #9] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:11,688 DEBUG [ORC_GET_SPLITS #8] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:11,688 DEBUG [ORC_GET_SPLITS #9] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:29:11,688 DEBUG [ORC_GET_SPLITS #8] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:29:11,689 INFO [ORC_GET_SPLITS #9] exec.ObjectCacheFactory: Created object cache for djaiswal_20180105112911_d60e4252-4a13-4247-a28a-f07830bbf022 2018-01-05T11:29:11,689 DEBUG [ORC_GET_SPLITS #8] orc.OrcInputFormat: Dynamic values are not available here DynamicValueRegistry not available 2018-01-05T11:29:11,689 DEBUG [ORC_GET_SPLITS #9] orc.OrcInputFormat: Dynamic values are not available here DynamicValueRegistry not available 2018-01-05T11:29:11,689 INFO [InputInitializer {Map 1} #0] orc.OrcInputFormat: FooterCacheHitRatio: 2/2 2018-01-05T11:29:11,689 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0, start=3, length=566, isOriginal=true, fileLength=744, hasFooter=false, hasBase=true, deltas=0] projected_columns_uncompressed_size: 8724 2018-01-05T11:29:11,689 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0, start=3, length=583, isOriginal=true, fileLength=767, hasFooter=false, hasBase=true, deltas=0] projected_columns_uncompressed_size: 8573 2018-01-05T11:29:11,689 DEBUG [InputInitializer {Map 1} #0] orc.OrcInputFormat: getSplits finished 2018-01-05T11:29:11,689 INFO [InputInitializer {Map 1} #0] io.HiveInputFormat: number of splits 2 2018-01-05T11:29:11,689 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:11,689 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of input splits: 2. 32 available slots, 1.7 waves. Input format is: org.apache.hadoop.hive.ql.io.HiveInputFormat 2018-01-05T11:29:11,689 INFO [InputInitializer {Map 1} #0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/djaiswal/_tez_scratch_dir/79b99006-4f31-4e98-bc9a-2de4e79e98fe/map.xml 2018-01-05T11:29:11,689 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/djaiswal/_tez_scratch_dir/79b99006-4f31-4e98-bc9a-2de4e79e98fe/map.xml 2018-01-05T11:29:11,689 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/djaiswal/_tez_scratch_dir/79b99006-4f31-4e98-bc9a-2de4e79e98fe/map.xml 2018-01-05T11:29:11,689 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:11,689 INFO [InputInitializer {Map 1} #0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:11,689 DEBUG [InputInitializer {Map 1} #0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:11,691 DEBUG [InputInitializer {Map 1} #0] log.PerfLogger: 2018-01-05T11:29:11,691 INFO [InputInitializer {Map 1} #0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 5.49KB 2018-01-05T11:29:11,691 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0 to src new group? true 2018-01-05T11:29:11,691 DEBUG [InputInitializer {Map 1} #0] tez.SplitGrouper: Adding split file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0 to src new group? false 2018-01-05T11:29:11,691 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: # Src groups for split generation: 2 2018-01-05T11:29:11,691 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Estimated number of tasks: 54 for bucket 1 2018-01-05T11:29:11,691 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Grouping splits in Tez 2018-01-05T11:29:11,691 DEBUG [InputInitializer {Map 1} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 8573 2018-01-05T11:29:11,691 DEBUG [InputInitializer {Map 1} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 8724 2018-01-05T11:29:11,691 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired splits: 1 too large. Desired splitLength: 320 Min splitLength: 52428800 New desired splits: 1 Final desired splits: 1 All splits have localhost: false Total length: 17297 Original splits: 2 2018-01-05T11:29:11,691 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Desired numSplits: 1 lengthPerGroup: 17297 numLocations: 1 numSplitsPerLocation: 2 numSplitsInGroup: 2 totalLength: 17297 numOriginalSplits: 2 . Grouping by length: true count: false nodeLocalOnly: false 2018-01-05T11:29:11,691 DEBUG [InputInitializer {Map 1} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 8573 2018-01-05T11:29:11,691 DEBUG [InputInitializer {Map 1} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 8724 2018-01-05T11:29:11,691 DEBUG [InputInitializer {Map 1} #0] tez.ColumnarSplitSizeEstimator: Estimated column projection size: 8724 2018-01-05T11:29:11,691 INFO [InputInitializer {Map 1} #0] grouper.TezSplitGrouper: Number of splits desired: 1 created: 1 splitsProcessed: 2 2018-01-05T11:29:11,691 INFO [InputInitializer {Map 1} #0] tez.SplitGrouper: Original split count is 2 grouped split count is 1, for bucket: 1 2018-01-05T11:29:11,691 INFO [InputInitializer {Map 1} #0] tez.HiveSplitGenerator: Number of split groups: 1 2018-01-05T11:29:11,692 INFO [InputInitializer {Map 1} #0] dag.RootInputInitializerManager: Succeeded InputInitializer for Input: a on vertex vertex_1515180518813_0001_42_02 [Map 1] 2018-01-05T11:29:11,692 INFO [App Shared Pool - #2] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_42_02 [Map 1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:11,692 INFO [App Shared Pool - #2] impl.VertexImpl: Got updated RootInputsSpecs: {a=forAllWorkUnits=true, update=[1]} 2018-01-05T11:29:11,692 INFO [App Shared Pool - #2] impl.VertexImpl: Vertex vertex_1515180518813_0001_42_02 [Map 1] parallelism set to 1 2018-01-05T11:29:11,692 INFO [App Shared Pool - #2] impl.VertexImpl: Cannot init vertex: vertex_1515180518813_0001_42_02 [Map 1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers: true 2018-01-05T11:29:11,692 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:VERTEX_INITIALIZED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_42_02, initRequestedTime=1515180551648, initedTime=1515180551692, numTasks=1, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, additionalInputsCount=1, initGeneratedEventsCount=1, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:11,692 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_42_02 [Map 1] with distanceFromRoot: 2 2018-01-05T11:29:11,692 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_42_02 [Map 1] transitioned from INITIALIZING to INITED due to event V_INPUT_DATA_INFORMATION 2018-01-05T11:29:11,692 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_42_02, startRequestedTime=1515180551681, startedTime=1515180551692 2018-01-05T11:29:11,692 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:VERTEX_CONFIGURE_DONE]: vertexId=vertex_1515180518813_0001_42_02, reconfigureDoneTime=1515180551692, numTasks=1, vertexLocationHint=org.apache.tez.dag.api.VertexLocationHint@6855, edgeManagersCount=1, rootInputSpecUpdateCount=1, setParallelismCalledFlag=true 2018-01-05T11:29:11,692 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_42_02 [Map 1] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:11,692 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_42_02 for vertex: vertex_1515180518813_0001_42_03 [Reducer 2] numStartedSources: 2 numSources: 2 2018-01-05T11:29:11,692 INFO [App Shared Pool - #2] impl.RootInputVertexManager: Task count in Reducer 7: 1 2018-01-05T11:29:11,692 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_42_03 [Reducer 2] with distanceFromRoot: 3 2018-01-05T11:29:11,692 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_42_03, startRequestedTime=1515180551692, startedTime=1515180551692 2018-01-05T11:29:11,692 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_42_03 [Reducer 2] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:11,692 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_42_03 for vertex: vertex_1515180518813_0001_42_06 [Reducer 3] numStartedSources: 1 numSources: 2 2018-01-05T11:29:11,692 INFO [Dispatcher thread {Central}] impl.VertexImpl: Cannot start vertex: vertex_1515180518813_0001_42_06 [Reducer 3] numStartedSources: 1 numSources: 2 2018-01-05T11:29:11,692 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_42_03 for vertex: vertex_1515180518813_0001_42_04 [Reducer 5] numStartedSources: 1 numSources: 1 2018-01-05T11:29:11,692 INFO [Dispatcher thread {Central}] impl.VertexImpl: Triggering start event for vertex: vertex_1515180518813_0001_42_04 [Reducer 5] with distanceFromRoot: 4 2018-01-05T11:29:11,692 INFO [App Shared Pool - #2] impl.RootInputVertexManager: Received configured notification : CONFIGURED for vertex: Reducer 7 in vertex: Map 1 numjourceTasks: 1 2018-01-05T11:29:11,692 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:VERTEX_STARTED]: vertexId=vertex_1515180518813_0001_42_04, startRequestedTime=1515180551692, startedTime=1515180551692 2018-01-05T11:29:11,692 INFO [App Shared Pool - #2] impl.RootInputVertexManager: Scheduling 1 tasks for vertex: Map 1 with totalTasks: 1. 0 source tasks completed out of 1. MinSourceTaskCompletedFraction: 0.0 min: 0.0 max: 0.0 2018-01-05T11:29:11,692 INFO [App Shared Pool - #2] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Reducer 7, destinationVertexName=Map 1. EdgeProperty: { BROADCAST : org.apache.tez.runtime.library.input.UnorderedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.UnorderedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:29:11,692 INFO [App Shared Pool - #2] impl.VertexImpl: Routing pending task events for vertex: vertex_1515180518813_0001_42_02 [Map 1] 2018-01-05T11:29:11,692 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_42_04 [Reducer 5] transitioned from INITED to RUNNING due to event V_START 2018-01-05T11:29:11,692 INFO [App Shared Pool - #3] vertexmanager.ShuffleVertexManagerBase: OnVertexStarted vertex: Reducer 2 with 0 source tasks and 1 pending tasks 2018-01-05T11:29:11,692 INFO [Dispatcher thread {Central}] impl.VertexImpl: Source vertex started: vertex_1515180518813_0001_42_04 for vertex: vertex_1515180518813_0001_42_05 [Map 8] numStartedSources: 1 numSources: 1 2018-01-05T11:29:11,692 INFO [Dispatcher thread {Central}] impl.VertexImpl: Cannot start vertex. Not in inited state. vertex_1515180518813_0001_42_05 [Map 8] . VertesState: INITIALIZING numTasks: -1 Num uninitialized edges: 0 2018-01-05T11:29:11,692 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: OnVertexStarted vertex: Reducer 5 with 0 source tasks and 1 pending tasks 2018-01-05T11:29:11,693 INFO [App Shared Pool - #2] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Map 6 in vertex: Reducer 2 numBipartiteSourceTasks: 1 2018-01-05T11:29:11,693 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:TASK_STARTED]: vertexName=Map 1, taskId=task_1515180518813_0001_42_02_000000, scheduledTime=1515180551692, launchTime=1515180551692 2018-01-05T11:29:11,693 INFO [App Shared Pool - #3] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Reducer 2 in vertex: Reducer 5 numBipartiteSourceTasks: 1 2018-01-05T11:29:11,693 INFO [App Shared Pool - #0] vertexmanager.ShuffleVertexManagerBase: Received configured notification : CONFIGURED for vertex: Map 1 in vertex: Reducer 2 numBipartiteSourceTasks: 2 2018-01-05T11:29:11,693 INFO [App Shared Pool - #3] vertexmanager.ShuffleVertexManagerBase: Scheduling 1 tasks for vertex: Reducer 5 with totalTasks: 1. 0 source tasks completed out of 1. MinSourceTaskCompletedFraction: 0.0 min: 0.0 max: 0.0 2018-01-05T11:29:11,693 INFO [Dispatcher thread {Central}] impl.DAGSchedulerNaturalOrderControlled: Scheduled vertex: vertex_1515180518813_0001_42_02 [Map 1] 2018-01-05T11:29:11,693 INFO [App Shared Pool - #3] impl.Edge: Routing to begin for edge: EdgeInfo: sourceVertexName=Reducer 2, destinationVertexName=Reducer 5. EdgeProperty: { SCATTER_GATHER : org.apache.tez.runtime.library.input.UnorderedKVInput >> PERSISTED >> org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput >> NullEdgeManager } onDemandRouting: true 2018-01-05T11:29:11,693 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:TASK_STARTED]: vertexName=Reducer 5, taskId=task_1515180518813_0001_42_04_000000, scheduledTime=1515180551693, launchTime=1515180551693 2018-01-05T11:29:11,693 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Received allocateRequest. task=attempt_1515180518813_0001_42_02_000000_0, priority=77, capability=, hosts=null 2018-01-05T11:29:11,693 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: PendingTasksInfo=numPriorityLevels=1. [p=77,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:11,693 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=1. [p=77,c=1]. totalPendingTasks=1. delayedTaskQueueSize=0 2018-01-05T11:29:11,693 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:11,693 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: selectingHost for task=attempt_1515180518813_0001_42_02_000000_0 on hosts=null 2018-01-05T11:29:11,693 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ShouldDelayForLocality=false for task=attempt_1515180518813_0001_42_02_000000_0 on hosts=null 2018-01-05T11:29:11,694 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: No-locality requested. Selecting a random host for task=attempt_1515180518813_0001_42_02_000000_0 2018-01-05T11:29:11,694 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigning {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=2, ac=6, commF=false, disabled=false} when looking for any host, from #hosts=1, requestedHosts=null 2018-01-05T11:29:11,694 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Assigned #107, task=TaskInfo{task=attempt_1515180518813_0001_42_02_000000_0, priority=77, startTime=0, containerId=null, uniqueId=106, localityDelayTimeout=0} on node={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=2, ac=6, commF=false, disabled=false}, to container=container_111101111_0001_00_000107 2018-01-05T11:29:11,694 INFO [LlapScheduler] GuaranteedTasks: Registering attempt_1515180518813_0001_42_02_000000_0; false 2018-01-05T11:29:11,694 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleResult for Task: TaskInfo{task=attempt_1515180518813_0001_42_02_000000_0, priority=77, startTime=311572203, containerId=container_111101111_0001_00_000107assignedNode={10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=true, st=3, ac=5, commF=false, disabled=false}, uniqueId=106, localityDelayTimeout=0} = SCHEDULED 2018-01-05T11:29:11,694 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000107 for node: 10.22.8.182:49818 2018-01-05T11:29:11,694 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000107, attempt_1515180518813_0001_42_02_000000_0 for node: 10.22.8.182:49818 2018-01-05T11:29:11,694 DEBUG [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Registering container_111101111_0001_00_000107 for node: 10.22.8.182:49818 2018-01-05T11:29:11,694 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op launch for container: container_111101111_0001_00_000107 succeeded on host: 10.22.8.182:49818 2018-01-05T11:29:11,694 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:N/A][Event:CONTAINER_LAUNCHED]: containerId=container_111101111_0001_00_000107, launchTime=1515180551694 2018-01-05T11:29:11,695 INFO [Dispatcher thread {Central}] impl.TaskAttemptImpl: TaskAttempt: [attempt_1515180518813_0001_42_02_000000_0] submitted. Is using containerId: [container_111101111_0001_00_000107] on NM: [10.22.8.182:49818] 2018-01-05T11:29:11,695 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:11,695 DEBUG [Dispatcher thread {Central}] impl.ZkRegistryBase: Returning 1 hosts for locality allocation on 10.22.8.182 2018-01-05T11:29:11,695 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:TASK_ATTEMPT_STARTED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_42_02_000000_0, startTime=1515180551694, containerId=container_111101111_0001_00_000107, nodeId=10.22.8.182:49818 2018-01-05T11:29:11,697 INFO [IPC Server handler 0 on 49818] shufflehandler.ShuffleHandler: Added token for job_1515180518813_0001 2018-01-05T11:29:11,697 DEBUG [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/42] 2018-01-05T11:29:11,698 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=4, runningFragmentCount=0 2018-01-05T11:29:11,698 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_42_00_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:11,698 DEBUG [IPC Server handler 0 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_42_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180551681, dagStartTime=1515180551638, withinDagPriority=23, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0}] 2018-01-05T11:29:11,698 INFO [IPC Server handler 0 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_42_00_000000_0 : ACCEPTED 2018-01-05T11:29:11,698 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to schedule task attempt_1515180518813_0001_42_00_000000_0, canFinish=true. Current state: preemptionQueueSize=0, numSlotsAvailable=4, waitQueueSize=1 2018-01-05T11:29:11,698 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_42_00_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=true, canFinish(in queue)=true, isGuaranteed=false, firstAttemptStartTime=1515180551681, dagStartTime=1515180551638, withinDagPriority=23, vertexParallelism= 1, selfAndUpstreamParallelism= 1, selfAndUpstreamComplete= 0} 2018-01-05T11:29:11,698 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_42_00_000000_0 scheduled for execution. canFinish=true, isGuaranteed=false 2018-01-05T11:29:11,698 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_42_00_000000_0 to pre-emption queue 2018-01-05T11:29:11,698 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_42_00_000000_0: true 2018-01-05T11:29:11,698 INFO [TaskCommunicator # 5] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_42_00_000000_0 2018-01-05T11:29:11,698 INFO [TaskCommunicator # 5] GuaranteedTasks: Received response for attempt_1515180518813_0001_42_00_000000_0, true 2018-01-05T11:29:11,698 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: taskOwner hashCode:1596571858 2018-01-05T11:29:11,698 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_42_01_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112911_d60e4252-4a13-4247-a28a-f07830bbf022, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000106, dagName=select count(*) from tab a join ta...c.value (Stage-1), vertexName=Reducer 7, processor=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, numInputs=1, numOutputs=1, numGroupedInputs=0, Inputs={{Map 6,org.apache.tez.runtime.library.input.UnorderedKVInput,1}}, Outputs={{Map 1,org.apache.tez.runtime.library.output.UnorderedKVOutput,1}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=2, completedTaskCount=0, dagStartTime=1515180551638, firstAttemptStartTime=1515180551682, currentAttemptStartTime=1515180551685} 2018-01-05T11:29:11,698 INFO [IPC Server handler 2 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000106, QueryId=djaiswal_20180105112911_d60e4252-4a13-4247-a28a-f07830bbf022-42, DagId=42, VertexName=Reducer 7, TaskId=0, TaskAttemptId=0, SubmitTime=1515180551698 2018-01-05T11:29:11,699 DEBUG [IPC Server handler 2 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=42} with the ShuffleHandler 2018-01-05T11:29:11,699 DEBUG [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/42] 2018-01-05T11:29:11,699 INFO [Task-Executor-1] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : select count(*) from tab a join ta...c.value (Stage-1), VertexName: Map 6, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_42_00_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=1, outputSpecListSize=2, inputSpecList=[{{ sourceVertexName=c, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, ], outputSpecList=[{{ destinationVertexName=Reducer 7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput }}, {{ destinationVertexName=Reducer 2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ] 2018-01-05T11:29:11,699 INFO [Task-Executor-1] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=2, JVM.maxFree=1023934464, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:11,699 INFO [Task-Executor-1] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_42_00_000000_0 vertexName: Map 6 2018-01-05T11:29:11,699 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000105, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_42_00_000000_0, eventCount=1 } 2018-01-05T11:29:11,699 DEBUG [TezTR-518813_1_42_0_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_42 queryId: djaiswal_20180105112911_d60e4252-4a13-4247-a28a-f07830bbf022 instanceType: TaskRunner2Callable 2018-01-05T11:29:11,699 INFO [TezTR-518813_1_42_0_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_42_00_000000_0 2018-01-05T11:29:11,699 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=3, runningFragmentCount=1 2018-01-05T11:29:11,699 DEBUG [IPC Server handler 2 on 49818] impl.QueryFragmentInfo: Cannot finish due to source: Map 6 2018-01-05T11:29:11,699 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_42_01_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:11,699 DEBUG [IPC Server handler 2 on 49818] impl.QueryFragmentInfo: Cannot finish due to source: Map 6 2018-01-05T11:29:11,699 DEBUG [IPC Server handler 2 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_42_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=false, canFinish(in queue)=false, isGuaranteed=false, firstAttemptStartTime=1515180551682, dagStartTime=1515180551638, withinDagPriority=50, vertexParallelism= 1, selfAndUpstreamParallelism= 2, selfAndUpstreamComplete= 0}] 2018-01-05T11:29:11,699 INFO [IPC Server handler 2 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_42_01_000000_0 : ACCEPTED 2018-01-05T11:29:11,699 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_42_01_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=false, canFinish(in queue)=false, isGuaranteed=false, firstAttemptStartTime=1515180551682, dagStartTime=1515180551638, withinDagPriority=50, vertexParallelism= 1, selfAndUpstreamParallelism= 2, selfAndUpstreamComplete= 0} 2018-01-05T11:29:11,699 DEBUG [Wait-Queue-Scheduler-0] impl.QueryFragmentInfo: Cannot finish due to source: Map 6 2018-01-05T11:29:11,699 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_42_01_000000_0 scheduled for execution. canFinish=false, isGuaranteed=false 2018-01-05T11:29:11,699 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_42_01_000000_0 to pre-emption queue 2018-01-05T11:29:11,700 DEBUG [TezTR-518813_1_42_0_0_0] log.PerfLogger: 2018-01-05T11:29:11,700 INFO [TaskCommunicator # 6] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_42_01_000000_0 2018-01-05T11:29:11,700 INFO [TaskCommunicator # 6] GuaranteedTasks: Received response for attempt_1515180518813_0001_42_01_000000_0, true 2018-01-05T11:29:11,700 DEBUG [Task-Executor-2] impl.QueryFragmentInfo: Cannot finish due to source: Map 6 2018-01-05T11:29:11,700 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_42_01_000000_0: false 2018-01-05T11:29:11,700 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: taskOwner hashCode:1702363224 2018-01-05T11:29:11,700 INFO [I/O Setup 1 Initialize: {Reducer 7}] writers.UnorderedPartitionedKVWriter: Requested BufferSize (tez.runtime.unordered.output.buffer.size-mb) : 100 2018-01-05T11:29:11,700 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:11,700 INFO [Task-Executor-2] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : select count(*) from tab a join ta...c.value (Stage-1), VertexName: Reducer 7, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_42_01_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.ReduceTezProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=Map 6, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }}, ], outputSpecList=[{{ destinationVertexName=Map 1, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.UnorderedKVOutput }}, ] 2018-01-05T11:29:11,700 INFO [Task-Executor-2] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=1023934464, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:11,700 INFO [Task-Executor-2] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_42_01_000000_0 vertexName: Reducer 7 2018-01-05T11:29:11,701 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Queueing container for execution: fragemendId=attempt_1515180518813_0001_42_02_000000_0, am_details=10.22.8.182:49824, taskInfo= fragment 0 attempt 0, user=djaiswal, queryId=djaiswal_20180105112911_d60e4252-4a13-4247-a28a-f07830bbf022, appIdString=application_1515180518813_0001, appAttemptNum=0, containerIdString=container_111101111_0001_00_000107, dagName=select count(*) from tab a join ta...c.value (Stage-1), vertexName=Map 1, processor=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, numInputs=2, numOutputs=1, numGroupedInputs=0, Inputs={{a,org.apache.tez.mapreduce.input.MRInputLegacy,1}{Reducer 7,org.apache.tez.runtime.library.input.UnorderedKVInput,1}}, Outputs={{Reducer 2,org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput,1}}, GroupedInputs={}, FragmentRuntimeInfo={taskCount=3, completedTaskCount=0, dagStartTime=1515180551638, firstAttemptStartTime=1515180551693, currentAttemptStartTime=1515180551694} 2018-01-05T11:29:11,701 INFO [IPC Server handler 4 on 49818] daemon.HistoryLogger: Event=FRAGMENT_START, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000107, QueryId=djaiswal_20180105112911_d60e4252-4a13-4247-a28a-f07830bbf022-42, DagId=42, VertexName=Map 1, TaskId=0, TaskAttemptId=0, SubmitTime=1515180551701 2018-01-05T11:29:11,701 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000106, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_42_01_000000_0, eventCount=1 } 2018-01-05T11:29:11,701 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:11,701 DEBUG [IPC Server handler 4 on 49818] impl.QueryTracker: Registering request for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=42} with the ShuffleHandler 2018-01-05T11:29:11,701 DEBUG [TezTR-518813_1_42_1_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_42 queryId: djaiswal_20180105112911_d60e4252-4a13-4247-a28a-f07830bbf022 instanceType: TaskRunner2Callable 2018-01-05T11:29:11,701 DEBUG [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: Dirs are: [/Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/42] 2018-01-05T11:29:11,701 INFO [TezTR-518813_1_42_1_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_42_01_000000_0 2018-01-05T11:29:11,701 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Offering to wait queue with: waitQueueSize=0, numSlotsAvailable=2, runningFragmentCount=2 2018-01-05T11:29:11,701 DEBUG [IPC Server handler 4 on 49818] impl.QueryFragmentInfo: Cannot finish due to source: Reducer 7 2018-01-05T11:29:11,701 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: attempt_1515180518813_0001_42_02_000000_0 added to wait queue. Current wait queue size=1 2018-01-05T11:29:11,701 DEBUG [IPC Server handler 4 on 49818] impl.QueryFragmentInfo: Cannot finish due to source: Reducer 7 2018-01-05T11:29:11,701 DEBUG [IPC Server handler 4 on 49818] impl.TaskExecutorService: Wait Queue: currentSize=1, queue=[TaskWrapper{task=attempt_1515180518813_0001_42_02_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=false, canFinish(in queue)=false, isGuaranteed=false, firstAttemptStartTime=1515180551693, dagStartTime=1515180551638, withinDagPriority=77, vertexParallelism= 1, selfAndUpstreamParallelism= 3, selfAndUpstreamComplete= 0}] 2018-01-05T11:29:11,701 INFO [IPC Server handler 4 on 49818] impl.ContainerRunnerImpl: SubmissionState for attempt_1515180518813_0001_42_02_000000_0 : ACCEPTED 2018-01-05T11:29:11,701 INFO [IPC Server handler 7 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_42_00_000000_0 sent events: (0-1). 2018-01-05T11:29:11,702 DEBUG [TezTR-518813_1_42_1_0_0] log.PerfLogger: 2018-01-05T11:29:11,702 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:11,702 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Attempting to execute TaskWrapper{task=attempt_1515180518813_0001_42_02_000000_0, inWaitQueue=true, inPreemptionQueue=false, registeredForNotifications=true, canFinish=false, canFinish(in queue)=false, isGuaranteed=false, firstAttemptStartTime=1515180551693, dagStartTime=1515180551638, withinDagPriority=77, vertexParallelism= 1, selfAndUpstreamParallelism= 3, selfAndUpstreamComplete= 0} 2018-01-05T11:29:11,702 DEBUG [Wait-Queue-Scheduler-0] impl.QueryFragmentInfo: Cannot finish due to source: Reducer 7 2018-01-05T11:29:11,702 DEBUG [Wait-Queue-Scheduler-0] impl.TaskExecutorService: attempt_1515180518813_0001_42_02_000000_0 scheduled for execution. canFinish=false, isGuaranteed=false 2018-01-05T11:29:11,702 INFO [Wait-Queue-Scheduler-0] impl.TaskExecutorService: Adding attempt_1515180518813_0001_42_02_000000_0 to pre-emption queue 2018-01-05T11:29:11,702 INFO [TaskCommunicator # 7] tezplugins.LlapTaskCommunicator: Successfully launched task: attempt_1515180518813_0001_42_02_000000_0 2018-01-05T11:29:11,702 INFO [TaskCommunicator # 7] GuaranteedTasks: Received response for attempt_1515180518813_0001_42_02_000000_0, true 2018-01-05T11:29:11,702 DEBUG [Task-Executor-3] impl.QueryFragmentInfo: Cannot finish due to source: Reducer 7 2018-01-05T11:29:11,702 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish: attempt_1515180518813_0001_42_02_000000_0: false 2018-01-05T11:29:11,702 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: taskOwner hashCode:1393672602 2018-01-05T11:29:11,703 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:29:11,703 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_42_00_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:29:11,703 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:11,703 INFO [Task-Executor-3] runtime.LogicalIOProcessorRuntimeTask: Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : select count(*) from tab a join ta...c.value (Stage-1), VertexName: Map 1, VertexParallelism: 1, TaskAttemptID:attempt_1515180518813_0001_42_02_000000_0, processorName=org.apache.hadoop.hive.ql.exec.tez.MapTezProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=a, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInputLegacy }}, {{ sourceVertexName=Reducer 7, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.UnorderedKVInput }}, ], outputSpecList=[{{ destinationVertexName=Reducer 2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ] 2018-01-05T11:29:11,703 INFO [Task-Executor-3] resources.MemoryDistributor: InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=2, numOutputs=1, JVM.maxFree=1023934464, allocatorClassName=org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor 2018-01-05T11:29:11,703 INFO [Task-Executor-3] impl.LlapTaskReporter: Registered counters for fragment: 1515180518813_0001_42_02_000000_0 vertexName: Map 1 2018-01-05T11:29:11,704 DEBUG [TezTR-518813_1_42_2_0_0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_42 queryId: djaiswal_20180105112911_d60e4252-4a13-4247-a28a-f07830bbf022 instanceType: TaskRunner2Callable 2018-01-05T11:29:11,704 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Sending heartbeat to AM, request={ containerId=container_111101111_0001_00_000107, requestId=1, startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, taskAttemptId=attempt_1515180518813_0001_42_02_000000_0, eventCount=1 } 2018-01-05T11:29:11,704 INFO [TezTR-518813_1_42_2_0_0] task.TaskRunner2Callable: Initializing task, taskAttemptId=attempt_1515180518813_0001_42_02_000000_0 2018-01-05T11:29:11,704 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=0, eventCount=0 } 2018-01-05T11:29:11,704 DEBUG [TezTR-518813_1_42_0_0_0] log.PerfLogger: 2018-01-05T11:29:11,704 INFO [TezTR-518813_1_42_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:11,704 INFO [TezTR-518813_1_42_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 3 initializers to finish 2018-01-05T11:29:11,704 INFO [TezTR-518813_1_42_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:11,704 INFO [TezTR-518813_1_42_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:11,704 INFO [I/O Setup 0 Initialize: {c}] input.MRInput: c using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:29:11,704 INFO [I/O Setup 0 Initialize: {c}] input.MRInputLegacy: c MRInputLegacy deferring initialization 2018-01-05T11:29:11,705 INFO [TezTR-518813_1_42_0_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:11,705 INFO [TezTR-518813_1_42_0_0_0] resources.MemoryDistributor: InitialRequests=[c:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [Reducer 7:OUTPUT:104857600:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput], [Reducer 2:OUTPUT:10485760:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput] 2018-01-05T11:29:11,705 INFO [TezTR-518813_1_42_0_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:11,705 INFO [TezTR-518813_1_42_0_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.045, finalReserveFractionUsed=0.345 2018-01-05T11:29:11,705 INFO [TezTR-518813_1_42_0_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 3, numScaledRequests: 14, TotalRequested: 115343360, TotalRequestedScaled: 1.6477622857142856E7, TotalJVMHeap: 1023934464, TotalAvailable: 670677073, TotalRequested/TotalJVMHeap:0.11 2018-01-05T11:29:11,705 INFO [TezTR-518813_1_42_0_0_0] resources.MemoryDistributor: Allocations=[c:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [Reducer 7:org.apache.tez.runtime.library.output.UnorderedPartitionedKVOutput:OUTPUT:104857600:104857600], [Reducer 2:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:10485760:10485760] 2018-01-05T11:29:11,705 INFO [TezTR-518813_1_42_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:11,705 DEBUG [TaskHeartbeatThread] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:11,705 DEBUG [TezTR-518813_1_42_2_0_0] log.PerfLogger: 2018-01-05T11:29:11,706 INFO [TezTR-518813_1_42_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: c being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:11,706 INFO [TezTR-518813_1_42_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:29:11,706 INFO [TezTR-518813_1_42_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:11,707 INFO [I/O Setup 1 Start: {c}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: c 2018-01-05T11:29:11,707 DEBUG [TezTR-518813_1_42_1_0_0] log.PerfLogger: 2018-01-05T11:29:11,707 INFO [TezTR-518813_1_42_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:11,707 INFO [TezTR-518813_1_42_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:11,707 INFO [TezTR-518813_1_42_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:11,707 INFO [TezTR-518813_1_42_1_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:11,707 INFO [TezTR-518813_1_42_1_0_0] resources.MemoryDistributor: InitialRequests=[Map 6:INPUT:131404920:org.apache.tez.runtime.library.input.UnorderedKVInput], [Map 1:OUTPUT:0:org.apache.tez.runtime.library.output.UnorderedKVOutput] 2018-01-05T11:29:11,707 INFO [TezTR-518813_1_42_0_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:11,707 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:11,707 INFO [TezTR-518813_1_42_1_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:11,707 INFO [TezTR-518813_1_42_1_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996 2018-01-05T11:29:11,707 INFO [TezTR-518813_1_42_0_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_42_00_000000_0 2018-01-05T11:29:11,707 DEBUG [TezTR-518813_1_42_0_0_0] log.PerfLogger: 2018-01-05T11:29:11,707 DEBUG [TezTR-518813_1_42_0_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_42_00_000000_0_10321 2018-01-05T11:29:11,707 DEBUG [TezTR-518813_1_42_0_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:11,707 DEBUG [TezTR-518813_1_42_0_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_42_00_000000_0 2018-01-05T11:29:11,707 INFO [TezTR-518813_1_42_1_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 131404920, TotalRequestedScaled: 6.570246E7, TotalJVMHeap: 1023934464, TotalAvailable: 686036090, TotalRequested/TotalJVMHeap:0.13 2018-01-05T11:29:11,707 INFO [TezTR-518813_1_42_1_0_0] resources.MemoryDistributor: Allocations=[Map 6:org.apache.tez.runtime.library.input.UnorderedKVInput:INPUT:131404920:131404920], [Map 1:org.apache.tez.runtime.library.output.UnorderedKVOutput:OUTPUT:0:0] 2018-01-05T11:29:11,707 INFO [TezTR-518813_1_42_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:11,707 INFO [TezTR-518813_1_42_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 0 2018-01-05T11:29:11,707 INFO [TezTR-518813_1_42_1_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:11,708 INFO [TezTR-518813_1_42_1_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_42_01_000000_0 2018-01-05T11:29:11,708 DEBUG [TezTR-518813_1_42_1_0_0] log.PerfLogger: 2018-01-05T11:29:11,708 DEBUG [TezTR-518813_1_42_1_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_42_01_000000_0_10325 2018-01-05T11:29:11,708 DEBUG [TezTR-518813_1_42_1_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:11,708 DEBUG [TezTR-518813_1_42_1_0_0] mr.ObjectCache: Creating djaiswal_20180105112911_d60e4252-4a13-4247-a28a-f07830bbf022_Reducer 7__REDUCE_PLAN__ 2018-01-05T11:29:11,708 INFO [TezTR-518813_1_42_1_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/djaiswal/_tez_scratch_dir/9803cf99-0767-46d6-ba68-bd52aec6494e/reduce.xml 2018-01-05T11:29:11,708 DEBUG [TezTR-518813_1_42_1_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/djaiswal/_tez_scratch_dir/9803cf99-0767-46d6-ba68-bd52aec6494e/reduce.xml 2018-01-05T11:29:11,708 DEBUG [TezTR-518813_1_42_1_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/djaiswal/_tez_scratch_dir/9803cf99-0767-46d6-ba68-bd52aec6494e/reduce.xml 2018-01-05T11:29:11,708 DEBUG [TezTR-518813_1_42_1_0_0] log.PerfLogger: 2018-01-05T11:29:11,708 INFO [TezTR-518813_1_42_1_0_0] exec.SerializationUtilities: Deserializing ReduceWork using kryo 2018-01-05T11:29:11,708 DEBUG [TezTR-518813_1_42_1_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:11,708 DEBUG [TezTR-518813_1_42_0_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:11,708 DEBUG [TezTR-518813_1_42_0_0_0] log.PerfLogger: 2018-01-05T11:29:11,708 DEBUG [TezTR-518813_1_42_0_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:11,708 DEBUG [TezTR-518813_1_42_0_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:11,708 DEBUG [TezTR-518813_1_42_0_0_0] mr.ObjectCache: Creating djaiswal_20180105112911_d60e4252-4a13-4247-a28a-f07830bbf022_Map 6__MAP_PLAN__ 2018-01-05T11:29:11,708 INFO [TezTR-518813_1_42_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/djaiswal/_tez_scratch_dir/9493e99b-e6bc-44b7-b894-124305abb6f5/map.xml 2018-01-05T11:29:11,708 DEBUG [TezTR-518813_1_42_0_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/djaiswal/_tez_scratch_dir/9493e99b-e6bc-44b7-b894-124305abb6f5/map.xml 2018-01-05T11:29:11,708 DEBUG [TezTR-518813_1_42_0_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/djaiswal/_tez_scratch_dir/9493e99b-e6bc-44b7-b894-124305abb6f5/map.xml 2018-01-05T11:29:11,708 INFO [IPC Server handler 17 on 49824] impl.TaskImpl: TaskAttempt:attempt_1515180518813_0001_42_02_000000_0 sent events: (0-1). 2018-01-05T11:29:11,708 DEBUG [TezTR-518813_1_42_0_0_0] log.PerfLogger: 2018-01-05T11:29:11,708 INFO [TezTR-518813_1_42_0_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:11,708 DEBUG [TezTR-518813_1_42_0_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:11,708 INFO [I/O Setup 0 Initialize: {a}] input.MRInput: a using newmapreduce API=false, split via event=true, numPhysicalInputs=1 2018-01-05T11:29:11,708 INFO [I/O Setup 0 Initialize: {a}] input.MRInputLegacy: a MRInputLegacy deferring initialization 2018-01-05T11:29:11,708 DEBUG [TaskHeartbeatThread] impl.LlapTaskReporter: Received heartbeat response from AM, response={ lastRequestId=1, shouldDie=false, nextFromEventId=0, nextPreRoutedEventId=1, eventCount=1 } 2018-01-05T11:29:11,708 INFO [TaskHeartbeatThread] impl.LlapTaskReporter: Routing events from heartbeat response to task, currentTaskAttemptId=attempt_1515180518813_0001_42_02_000000_0, eventCount=1 fromEventId=0 nextFromEventId=0 2018-01-05T11:29:11,709 DEBUG [TezTR-518813_1_42_2_0_0] log.PerfLogger: 2018-01-05T11:29:11,709 INFO [TezTR-518813_1_42_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Initialized processor 2018-01-05T11:29:11,709 INFO [TezTR-518813_1_42_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 3 initializers to finish 2018-01-05T11:29:11,709 INFO [TezTR-518813_1_42_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 2 initializers to finish 2018-01-05T11:29:11,709 INFO [TezTR-518813_1_42_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 initializers to finish 2018-01-05T11:29:11,709 INFO [TezTR-518813_1_42_2_0_0] runtime.LogicalIOProcessorRuntimeTask: All initializers finished 2018-01-05T11:29:11,709 INFO [TezTR-518813_1_42_2_0_0] resources.MemoryDistributor: InitialRequests=[a:INPUT:0:org.apache.tez.mapreduce.input.MRInputLegacy], [Reducer 7:INPUT:131404920:org.apache.tez.runtime.library.input.UnorderedKVInput], [Reducer 2:OUTPUT:10485760:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput] 2018-01-05T11:29:11,709 INFO [TezTR-518813_1_42_2_0_0] resources.WeightedScalingMemoryDistributor: ScaleRatiosUsed=[PARTITIONED_UNSORTED_OUTPUT:1][UNSORTED_OUTPUT:1][UNSORTED_INPUT:1][SORTED_OUTPUT:12][SORTED_MERGED_INPUT:12][PROCESSOR:1][OTHER:1] 2018-01-05T11:29:11,709 INFO [TezTR-518813_1_42_2_0_0] resources.WeightedScalingMemoryDistributor: InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.045, finalReserveFractionUsed=0.345 2018-01-05T11:29:11,709 INFO [TezTR-518813_1_42_2_0_0] resources.WeightedScalingMemoryDistributor: Scaling Requests. NumRequests: 3, numScaledRequests: 14, TotalRequested: 141890680, TotalRequestedScaled: 1.837386E7, TotalJVMHeap: 1023934464, TotalAvailable: 670677073, TotalRequested/TotalJVMHeap:0.14 2018-01-05T11:29:11,709 INFO [TezTR-518813_1_42_2_0_0] resources.MemoryDistributor: Allocations=[a:org.apache.tez.mapreduce.input.MRInputLegacy:INPUT:0:0], [Reducer 7:org.apache.tez.runtime.library.input.UnorderedKVInput:INPUT:131404920:131404920], [Reducer 2:org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput:OUTPUT:10485760:10485760] 2018-01-05T11:29:11,709 INFO [TezTR-518813_1_42_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Starting Inputs/Outputs 2018-01-05T11:29:11,709 INFO [TezTR-518813_1_42_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Input: a being auto started by the framework. Subsequent instances will not be auto-started 2018-01-05T11:29:11,709 INFO [TezTR-518813_1_42_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Num IOs determined for AutoStart: 1 2018-01-05T11:29:11,709 INFO [TezTR-518813_1_42_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Waiting for 1 IOs to start 2018-01-05T11:29:11,709 INFO [I/O Setup 1 Start: {a}] runtime.LogicalIOProcessorRuntimeTask: Started Input with src edge: a 2018-01-05T11:29:11,709 INFO [TezTR-518813_1_42_2_0_0] runtime.LogicalIOProcessorRuntimeTask: AutoStartComplete 2018-01-05T11:29:11,709 INFO [TezTR-518813_1_42_2_0_0] task.TaskRunner2Callable: Running task, taskAttemptId=attempt_1515180518813_0001_42_02_000000_0 2018-01-05T11:29:11,709 DEBUG [TezTR-518813_1_42_2_0_0] log.PerfLogger: 2018-01-05T11:29:11,709 DEBUG [TezTR-518813_1_42_2_0_0] tez.TezProcessor: Running task: attempt_1515180518813_0001_42_02_000000_0_10328 2018-01-05T11:29:11,709 DEBUG [TezTR-518813_1_42_2_0_0] tez.TezProcessor: ProgressHelper initialized! 2018-01-05T11:29:11,709 DEBUG [TezTR-518813_1_42_2_0_0] tez.MapRecordProcessor: Setting the LLAP fragment ID for OF to attempt_1515180518813_0001_42_02_000000_0 2018-01-05T11:29:11,709 DEBUG [TezTR-518813_1_42_0_0_0] log.PerfLogger: 2018-01-05T11:29:11,709 DEBUG [TezTR-518813_1_42_2_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:11,709 DEBUG [TezTR-518813_1_42_2_0_0] log.PerfLogger: 2018-01-05T11:29:11,709 INFO [TezTR-518813_1_42_0_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 6 size: 5.16KB 2018-01-05T11:29:11,709 DEBUG [TezTR-518813_1_42_2_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:11,709 DEBUG [TezTR-518813_1_42_2_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:11,709 DEBUG [TezTR-518813_1_42_2_0_0] mr.ObjectCache: Creating djaiswal_20180105112911_d60e4252-4a13-4247-a28a-f07830bbf022_Map 1__MAP_PLAN__ 2018-01-05T11:29:11,709 INFO [TezTR-518813_1_42_2_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/djaiswal/_tez_scratch_dir/79b99006-4f31-4e98-bc9a-2de4e79e98fe/map.xml 2018-01-05T11:29:11,709 DEBUG [TezTR-518813_1_42_2_0_0] exec.Utilities: local path = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/djaiswal/_tez_scratch_dir/79b99006-4f31-4e98-bc9a-2de4e79e98fe/map.xml 2018-01-05T11:29:11,709 DEBUG [TezTR-518813_1_42_2_0_0] exec.Utilities: Loading plan from string: /Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/djaiswal/_tez_scratch_dir/79b99006-4f31-4e98-bc9a-2de4e79e98fe/map.xml 2018-01-05T11:29:11,709 DEBUG [TezTR-518813_1_42_2_0_0] log.PerfLogger: 2018-01-05T11:29:11,709 INFO [TezTR-518813_1_42_2_0_0] exec.SerializationUtilities: Deserializing MapWork using kryo 2018-01-05T11:29:11,709 DEBUG [TezTR-518813_1_42_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:11,710 DEBUG [TezTR-518813_1_42_0_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:11,710 INFO [TezTR-518813_1_42_0_0_0] tez.MapRecordProcessor: The input names are: [c] 2018-01-05T11:29:11,710 INFO [TezTR-518813_1_42_0_0_0] input.MRInput: Initialized MRInput: c 2018-01-05T11:29:11,710 DEBUG [TezTR-518813_1_42_1_0_0] log.PerfLogger: 2018-01-05T11:29:11,710 INFO [TezTR-518813_1_42_1_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Reducer 7 size: 4.89KB 2018-01-05T11:29:11,710 DEBUG [TezTR-518813_1_42_0_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 1 2018-01-05T11:29:11,710 INFO [TezTR-518813_1_42_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/djaiswal/_tez_scratch_dir/9493e99b-e6bc-44b7-b894-124305abb6f5/map.xml 2018-01-05T11:29:11,710 DEBUG [TezTR-518813_1_42_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:11,710 DEBUG [TezTR-518813_1_42_1_0_0] tez.TezProcessor: scheduleProgressTaskService() called! 2018-01-05T11:29:11,710 DEBUG [TezTR-518813_1_42_0_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1/kv3.txt PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=25, numFiles=1, serialization.ddl=struct src1 { string key, string value}, transient_lastDdlTime=1515180522, rawDataSize=191, columns.comments=defaultdefault, totalSize=216, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.src1}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=25, numFiles=1, serialization.ddl=struct src1 { string key, string value}, transient_lastDdlTime=1515180522, rawDataSize=191, columns.comments=defaultdefault, totalSize=216, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.src1}, baseFileName=src1, vectorPartitionDesc=vector map operator read type VECTOR_DESERIALIZE, input file format class name org.apache.hadoop.mapred.TextInputFormat, deserialize type LAZY_SIMPLE] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=25, numFiles=1, serialization.ddl=struct src1 { string key, string value}, transient_lastDdlTime=1515180522, rawDataSize=191, columns.comments=defaultdefault, totalSize=216, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.src1}, jobProperties=null], partSpec={}, inputFileFormatClass=class org.apache.hadoop.mapred.TextInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, properties={columns.types=string:string, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1, columns=key,value, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, serialization.format=1, numRows=25, numFiles=1, serialization.ddl=struct src1 { string key, string value}, transient_lastDdlTime=1515180522, rawDataSize=191, columns.comments=defaultdefault, totalSize=216, bucket_count=-1, file.outputformat=org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat, serialization.lib=org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, file.inputformat=org.apache.hadoop.mapred.TextInputFormat, column.name.delimiter=,, name=default.src1}, baseFileName=src1, vectorPartitionDesc=vector map operator read type VECTOR_DESERIALIZE, input file format class name org.apache.hadoop.mapred.TextInputFormat, deserialize type LAZY_SIMPLE]} 2018-01-05T11:29:11,710 DEBUG [TezTR-518813_1_42_1_0_0] log.PerfLogger: 2018-01-05T11:29:11,710 DEBUG [TezTR-518813_1_42_2_0_0] log.PerfLogger: 2018-01-05T11:29:11,710 DEBUG [TezTR-518813_1_42_1_0_0] tez.RecordProcessor: conf classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:11,710 DEBUG [TezTR-518813_1_42_1_0_0] tez.RecordProcessor: thread classpath = [file:/Users/djaiswal/hive/hive/itests/qtest/target/surefire/surefirebooter6575813209148907678.jar] 2018-01-05T11:29:11,710 INFO [TezTR-518813_1_42_2_0_0] exec.Utilities: Deserialized plan (via RPC) - name: Map 1 size: 5.49KB 2018-01-05T11:29:11,710 DEBUG [TezTR-518813_1_42_0_0_0] io.HiveInputFormat: Pushdown initiated with filterText = value is not null, filterExpr = GenericUDFOPNotNull(Column[value]), serializedFilterExpr = AQEAamF2YS51dGlsLkFycmF5TGlz9AEBAQFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVDb2x1bW5EZXPjAQF2YWx15QAAAYJjAQJvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnNlcmRlMi50eXBlaW5mby5QcmltaXRpdmVUeXBlSW5m7wEBc3RyaW7nAQNvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnVkZi5nZW5lcmljLkdlbmVyaWNVREZPUE5vdE51bOwBAQRvcmcuYXBhY2hlLmhhZG9vcC5pby5Cb29sZWFuV3JpdGFibOUBAAABAgEBYm9vbGVh7g== (desc) 2018-01-05T11:29:11,710 INFO [TezTR-518813_1_42_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/djaiswal/_tez_scratch_dir/9493e99b-e6bc-44b7-b894-124305abb6f5/map.xml 2018-01-05T11:29:11,710 DEBUG [TezTR-518813_1_42_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:11,710 DEBUG [TezTR-518813_1_42_0_0_0] io.HiveInputFormat: Checking org.apache.hadoop.mapred.TextInputFormat against org.apache.hadoop.mapred.TextInputFormat, 2018-01-05T11:29:11,710 INFO [TezTR-518813_1_42_0_0_0] io.HiveInputFormat: Using SerDe-based LLAP reader for org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:29:11,710 DEBUG [TezTR-518813_1_42_0_0_0] io.HiveInputFormat: Processing org.apache.hadoop.mapred.TextInputFormat 2018-01-05T11:29:11,710 INFO [TezTR-518813_1_42_0_0_0] io.HiveInputFormat: Resetting VectorMapOperatorReadType from VECTOR_DESERIALIZE for partition default.src1 {} 2018-01-05T11:29:11,711 DEBUG [TezTR-518813_1_42_2_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:11,711 INFO [TezTR-518813_1_42_2_0_0] tez.MapRecordProcessor: The input names are: [a, Reducer 7] 2018-01-05T11:29:11,711 INFO [TezTR-518813_1_42_2_0_0] input.MRInput: Initialized MRInput: a 2018-01-05T11:29:11,711 DEBUG [TezTR-518813_1_42_1_0_0] MapredContext: MapredContext initialized. 2018-01-05T11:29:11,711 INFO [TezTR-518813_1_42_1_0_0] impl.SimpleFetchedInputAllocator: Map_6: RequestedMemory=131404920, AssignedMemory=131404920, maxSingleShuffleLimit=32851230 2018-01-05T11:29:11,712 DEBUG [TezTR-518813_1_42_0_0_0] impl.LlapRecordReader: Initializing for input Map 6 2018-01-05T11:29:11,712 DEBUG [TezTR-518813_1_42_2_0_0] split.TezGroupedSplitsInputFormat: Init record reader for index 0 of 2 2018-01-05T11:29:11,712 INFO [TezTR-518813_1_42_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/djaiswal/_tez_scratch_dir/9493e99b-e6bc-44b7-b894-124305abb6f5/map.xml 2018-01-05T11:29:11,712 DEBUG [TezTR-518813_1_42_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:11,712 INFO [TezTR-518813_1_42_2_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/djaiswal/_tez_scratch_dir/79b99006-4f31-4e98-bc9a-2de4e79e98fe/map.xml 2018-01-05T11:29:11,712 DEBUG [TezTR-518813_1_42_2_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:11,712 DEBUG [TezTR-518813_1_42_1_0_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:11,712 DEBUG [TezTR-518813_1_42_1_0_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:11,712 DEBUG [TezTR-518813_1_42_1_0_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:11,712 DEBUG [TezTR-518813_1_42_1_0_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:11,712 DEBUG [TezTR-518813_1_42_2_0_0] io.HiveInputFormat: Found spec for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0 PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180546}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, numFiles=2, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, rawDataSize=22748, partition_columns.types=string, numRows=242, partition_columns=ds, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=1511, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08, transient_lastDdlTime=1515180547}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat] from {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08=PartitionDesc [tableDesc=TableDesc [inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, columns.types=int:string, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, columns=key,value, columns.comments=, partition_columns.types=string, partition_columns=ds, SORTBUCKETCOLSPREFIX=TRUE, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, column.name.delimiter=,, transient_lastDdlTime=1515180546}, jobProperties=null], partSpec={ds=2008-04-08}, inputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFileFormatClass=class org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, properties={name=default.tab, numFiles=2, bucket_field_name=key, serialization.ddl=struct tab { i32 key, string value}, serialization.format=1, rawDataSize=22748, partition_columns.types=string, numRows=242, partition_columns=ds, bucket_count=2, serialization.lib=org.apache.hadoop.hive.ql.io.orc.OrcSerde, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}, file.inputformat=org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, totalSize=1511, file.outputformat=org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, location=file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08, transient_lastDdlTime=1515180547}, baseFileName=ds=2008-04-08, vectorPartitionDesc=vector map operator read type VECTORIZED_INPUT_FILE_FORMAT, input file format class name org.apache.hadoop.hive.ql.io.orc.OrcInputFormat]} 2018-01-05T11:29:11,712 INFO [TezTR-518813_1_42_0_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_42_00_000000_0 2018-01-05T11:29:11,712 INFO [TezTR-518813_1_42_0_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [string, string] (isAcidRead false) 2018-01-05T11:29:11,712 DEBUG [TezTR-518813_1_42_2_0_0] io.HiveInputFormat: Pushdown initiated with filterText = (key is not null and value is not null and (value BETWEEN DynamicValue(RS_10_c_value_min) AND DynamicValue(RS_10_c_value_max) and in_bloom_filter(value, DynamicValue(RS_10_c_value_bloom_filter)))), filterExpr = GenericUDFOPAnd(GenericUDFOPNotNull(Column[key]), GenericUDFOPNotNull(Column[value]), GenericUDFOPAnd(GenericUDFBetween(Const boolean false, Column[value], DynamicValue(RS_10_c_value_min), DynamicValue(RS_10_c_value_max)), GenericUDFInBloomFilter(Column[value], DynamicValue(RS_10_c_value_bloom_filter)))), serializedFilterExpr = AQEAamF2YS51dGlsLkFycmF5TGlz9AEDAQFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVHZW5lcmljRnVuY0Rlc+MBAQABAQECb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC5wbGFuLkV4cHJOb2RlQ29sdW1uRGVz4wEBa2X5AAABgmEBA29yZy5hcGFjaGUuaGFkb29wLmhpdmUuc2VyZGUyLnR5cGVpbmZvLlByaW1pdGl2ZVR5cGVJbmbvAQFpbvQBBG9yZy5hcGFjaGUuaGFkb29wLmhpdmUucWwudWRmLmdlbmVyaWMuR2VuZXJpY1VERk9QTm90TnVs7AEBBW9yZy5hcGFjaGUuaGFkb29wLmlvLkJvb2xlYW5Xcml0YWJs5QEAAAEDAQFib29sZWHuAQEBAQABAQECAQF2YWx15QAACAEDAQFzdHJpbucBBAEBBQEAAAEDDQEBAQEAAQIBAQEBAAEEAQZvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVDb25zdGFudERlc+MBAQMNBQABAgESAAAIAQMTAQdvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnFsLnBsYW4uRXhwck5vZGVEeW5hbWljVmFsdWVEZXPjAQEIb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC5wbGFuLkR5bmFtaWNWYWx15QEBUlNfMTBfY192YWx1ZV9tae4BCdYBb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5zZXJkZTIub2JqZWN0aW5zcGVjdG9yLnByaW1pdGl2ZS5Xcml0YWJsZVN0cmluZ09iamVjdEluc3BlY3RvcgEBAxMBAxMBAxMBBwEBCAEBUlNfMTBfY192YWx1ZV9tYfgBCSABAxMBAxMBCm9yZy5hcGFjaGUuaGFkb29wLmhpdmUucWwudWRmLmdlbmVyaWMuR2VuZXJpY1VERkJldHdlZe4BAQvBAVtMb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5zZXJkZTIub2JqZWN0aW5zcGVjdG9yLk9iamVjdEluc3BlY3RvcjsBBQEM3wFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnNlcmRlMi5vYmplY3RpbnNwZWN0b3IucHJpbWl0aXZlLldyaXRhYmxlQ29uc3RhbnRCb29sZWFuT2JqZWN0SW5zcGVjdG9yAQEDDQEFAQABCSABCSABCSABDcUBb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGT1BFcXVhbE9yR3JlYXRlclRoYW4BAAECAT69AUVRVUFMIE9SIEdSRUFURVIgVEhBzgEFAQABDsIBb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGT1BFcXVhbE9yTGVzc1RoYW4BACkBPL0BRVFVQUwgT1IgTEVTUyBUSEHOAQUBAAEFAQAAAQMNAQEBAQABAgECHAEHAQEIAQFSU18xMF9jX3ZhbHVlX2Jsb29tX2ZpbHRl8gEP1gFvcmcuYXBhY2hlLmhhZG9vcC5oaXZlLnNlcmRlMi5vYmplY3RpbnNwZWN0b3IucHJpbWl0aXZlLldyaXRhYmxlQmluYXJ5T2JqZWN0SW5zcGVjdG9yAQEDAQFiaW5hcvkBAzgBAzgBEG9yZy5hcGFjaGUuaGFkb29wLmhpdmUucWwudWRmLmdlbmVyaWMuR2VuZXJpY1VERkluQmxvb21GaWx0ZfIBAAEDDQERb3JnLmFwYWNoZS5oYWRvb3AuaGl2ZS5xbC51ZGYuZ2VuZXJpYy5HZW5lcmljVURGT1BBbuQBAQUBAAABAw0BEQEBBQEAAAEDDQ== (desc) 2018-01-05T11:29:11,712 INFO [TezTR-518813_1_42_2_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/djaiswal/_tez_scratch_dir/79b99006-4f31-4e98-bc9a-2de4e79e98fe/map.xml 2018-01-05T11:29:11,712 DEBUG [TezTR-518813_1_42_2_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:11,712 DEBUG [TezTR-518813_1_42_2_0_0] io.HiveInputFormat: Processing org.apache.hadoop.hive.ql.io.orc.OrcInputFormat 2018-01-05T11:29:11,712 DEBUG [TezTR-518813_1_42_0_0_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:11,712 DEBUG [TezTR-518813_1_42_0_0_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:11,712 DEBUG [TezTR-518813_1_42_0_0_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:11,712 DEBUG [TezTR-518813_1_42_0_0_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:11,712 WARN [TezTR-518813_1_42_0_0_0] LlapIoImpl: Split for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1/kv3.txt (class org.apache.hadoop.mapred.FileSplit) does not have file ID 2018-01-05T11:29:11,712 DEBUG [TezTR-518813_1_42_0_0_0] LlapIoImpl: Ignoring schema: struct 2018-01-05T11:29:11,712 INFO [TezTR-518813_1_42_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/djaiswal/_tez_scratch_dir/9493e99b-e6bc-44b7-b894-124305abb6f5/map.xml 2018-01-05T11:29:11,712 DEBUG [TezTR-518813_1_42_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:11,712 DEBUG [IO-Elevator-Thread-0] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_42 queryId: djaiswal_20180105112911_d60e4252-4a13-4247-a28a-f07830bbf022 instanceType: SerDeEncodedDataReader 2018-01-05T11:29:11,712 DEBUG [TezTR-518813_1_42_0_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1/kv3.txt 2018-01-05T11:29:11,712 INFO [TezTR-518813_1_42_0_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.mapred.TextInputFormat:file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1/kv3.txt:0+216], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=216} 2018-01-05T11:29:11,712 INFO [IO-Elevator-Thread-0] LlapIoImpl: Processing data for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1/kv3.txt 2018-01-05T11:29:11,713 INFO [TezTR-518813_1_42_0_0_0] input.MRInput: c initialized RecordReader from event 2018-01-05T11:29:11,713 DEBUG [TezTR-518813_1_42_0_0_0] tez.MapRecordProcessor: Starting Output: Reducer 7 2018-01-05T11:29:11,713 DEBUG [TezTR-518813_1_42_2_0_0] impl.LlapRecordReader: Initializing for input Map 1 2018-01-05T11:29:11,713 DEBUG [IO-Elevator-Thread-0] LlapIoCache: Looking for data between 0 and 216 2018-01-05T11:29:11,713 INFO [TezTR-518813_1_42_2_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/djaiswal/_tez_scratch_dir/79b99006-4f31-4e98-bc9a-2de4e79e98fe/map.xml 2018-01-05T11:29:11,713 DEBUG [TezTR-518813_1_42_2_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:11,713 DEBUG [IO-Elevator-Thread-0] LlapIoCache: Determined stripe indexes 0, 1 2018-01-05T11:29:11,713 INFO [TezTR-518813_1_42_0_0_0] writers.BaseUnorderedPartitionedKVWriter: Instantiating Partitioner: [org.apache.tez.mapreduce.partition.MRPartitioner] 2018-01-05T11:29:11,713 INFO [TezTR-518813_1_42_2_0_0] impl.LlapRecordReader: Received fragment id: 1515180518813_0001_42_02_000000_0 2018-01-05T11:29:11,713 INFO [TezTR-518813_1_42_2_0_0] orc.OrcInputFormat: Using schema evolution configuration variables schema.evolution.columns [key, value] / schema.evolution.columns.types [int, string] (isAcidRead false) 2018-01-05T11:29:11,713 INFO [IO-Elevator-Thread-0] LlapIoImpl: Processing one split {file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1/kv3.txt, 0, 214} 2018-01-05T11:29:11,713 DEBUG [TezTR-518813_1_42_2_0_0] fs.FileSystem: Looking for FS supporting file 2018-01-05T11:29:11,713 DEBUG [TezTR-518813_1_42_2_0_0] fs.FileSystem: looking for configuration option fs.file.impl 2018-01-05T11:29:11,713 DEBUG [TezTR-518813_1_42_2_0_0] fs.FileSystem: Looking in service filesystems for implementation class 2018-01-05T11:29:11,713 DEBUG [TezTR-518813_1_42_2_0_0] fs.FileSystem: FS for file is class org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem 2018-01-05T11:29:11,713 DEBUG [IO-Elevator-Thread-0] LlapIoImpl: Processing slice #0 stripe kts 0 from 0 to [214, 216); has cache data; has no disk data 2018-01-05T11:29:11,713 DEBUG [TezTR-518813_1_42_2_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:11,713 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: Column at 0 1:string has no data 2018-01-05T11:29:11,714 DEBUG [TezTR-518813_1_42_2_0_0] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:11,714 INFO [TezTR-518813_1_42_2_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/djaiswal/_tez_scratch_dir/79b99006-4f31-4e98-bc9a-2de4e79e98fe/map.xml 2018-01-05T11:29:11,714 DEBUG [TezTR-518813_1_42_2_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:11,714 DEBUG [TezTR-518813_1_42_2_0_0] io.HiveContextAwareRecordReader: Processing file file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0 2018-01-05T11:29:11,714 DEBUG [IO-Elevator-Thread-1] impl.StatsRecordingThreadPool: Received dagId: dag_1515180518813_0001_42 queryId: djaiswal_20180105112911_d60e4252-4a13-4247-a28a-f07830bbf022 instanceType: OrcEncodedDataReader 2018-01-05T11:29:11,714 INFO [TezTR-518813_1_42_2_0_0] lib.MRReaderMapred: Processing split: TezGroupedSplit{wrappedSplits=[org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0, start=3, length=583, isOriginal=true, fileLength=767, hasFooter=false, hasBase=true, deltas=0], org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit [file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000000_0, start=3, length=566, isOriginal=true, fileLength=744, hasFooter=false, hasBase=true, deltas=0]], wrappedInputFormatName='org.apache.hadoop.hive.ql.io.HiveInputFormat', locations=null, rack='null', length=1149} 2018-01-05T11:29:11,714 INFO [IO-Elevator-Thread-1] LlapIoImpl: Processing data for file [-5122461581150239451, 1515180546000, 767]: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0 2018-01-05T11:29:11,714 INFO [TezTR-518813_1_42_2_0_0] input.MRInput: a initialized RecordReader from event 2018-01-05T11:29:11,714 DEBUG [TezTR-518813_1_42_2_0_0] tez.MapRecordProcessor: Starting Output: Reducer 2 2018-01-05T11:29:11,714 DEBUG [IO-Elevator-Thread-1] LlapIoOrc: FileSplit {3, 583}; stripes {3, 583}, 2018-01-05T11:29:11,719 INFO [TezTR-518813_1_42_1_0_0] impl.ShuffleManager: Map_6: numInputs=1, compressionCodec=NoCompressionCodec, numFetchers=1, ifileBufferSize=4096, ifileReadAheadEnabled=true, ifileReadAheadLength=4194304, localDiskFetchEnabled=false, sharedFetchEnabled=false, keepAlive=false, keepAliveMaxConnections=20, connectionTimeout=180000, readTimeout=180000, bufferSize=8192, bufferSize=8192, maxTaskOutputAtOnce=20 2018-01-05T11:29:11,719 INFO [TezTR-518813_1_42_2_0_0] impl.ExternalSorter: Reducer 2 using: memoryMb=10, keySerializerClass=class org.apache.hadoop.hive.ql.io.HiveKey, valueSerializerClass=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization$TezBytesWritableSerializer@1c8b2924, comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator@60d55fc7, partitioner=org.apache.tez.mapreduce.partition.MRPartitioner, serialization=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.hadoop.io.serializer.WritableSerialization, org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization, org.apache.hadoop.io.serializer.avro.AvroReflectSerialization, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:29:11,719 INFO [TezTR-518813_1_42_2_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 2: , UsingHashComparator=true 2018-01-05T11:29:11,721 INFO [TezTR-518813_1_42_2_0_0] impl.PipelinedSorter: Newly allocated block size=10485760, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=10485760, total=10485760 2018-01-05T11:29:11,721 INFO [TezTR-518813_1_42_2_0_0] impl.PipelinedSorter: Pre allocating rest of memory buffers upfront 2018-01-05T11:29:11,721 INFO [TezTR-518813_1_42_2_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 2: , UsingHashComparator=true#blocks=1, maxMemUsage=10485760, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=10485760, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=10 2018-01-05T11:29:11,721 DEBUG [IO-Elevator-Thread-1] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:11,713 DEBUG [IO-Elevator-Thread-0] encoded.EncodedTreeReaderFactory: columnIndex: 2 columnType: string streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DICTIONARY_V2 dictionarySize: 19 present: false data: ColumnStreamData [cacheBuffers=[LlapSerDeDataBuffer with 16384 bytes, ], indexBaseOffset=0] dictionary: true lengths: true secondary: false tz: US/Pacific 2018-01-05T11:29:11,721 INFO [TezTR-518813_1_42_1_0_0] tez.ReduceRecordProcessor: Waiting for ShuffleInputs to become ready 2018-01-05T11:29:11,721 DEBUG [IO-Elevator-Thread-1] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:11,721 INFO [TezTR-518813_1_42_2_0_0] impl.PipelinedSorter: Reducer 2: reserved.remaining()=10485760, reserved.metasize=5242880 2018-01-05T11:29:11,721 INFO [IO-Elevator-Thread-1] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [-5122461581150239451, 1515180546000, 767]): [{range start: 0 end: 76}] 2018-01-05T11:29:11,721 DEBUG [IO-Elevator-Thread-0] LlapIoOrc: Done with decode 2018-01-05T11:29:11,721 INFO [IO-Elevator-Thread-1] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [-5122461581150239451, 1515180546000, 767], base offset 3): [{start: 0 end: 11 cache buffer: 0x317f0390(1)}, {start: 11 end: 38 cache buffer: 0x4252621d(1)}, {start: 38 end: 76 cache buffer: 0x479a13fd(1)}] 2018-01-05T11:29:11,721 INFO [TezTR-518813_1_42_2_0_0] tez.MapRecordProcessor: Main input name is Map 1 2018-01-05T11:29:11,721 DEBUG [IO-Elevator-Thread-0] LlapIoImpl: setDone called; closed false, done false, err null, pending 1 2018-01-05T11:29:11,722 DEBUG [IO-Elevator-Thread-0] impl.StatsRecordingThreadPool: Updated stats: instance: SerDeEncodedDataReader thread name: IO-Elevator-Thread-0 thread id: 1252 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:11,722 INFO [TezTR-518813_1_42_2_0_0] vector.VectorMapOperator: Initializing operator MAP[0] 2018-01-05T11:29:11,722 DEBUG [TezTR-518813_1_42_2_0_0] vector.VectorMapOperator: Initialization Done 0 MAP 2018-01-05T11:29:11,722 DEBUG [TezTR-518813_1_42_2_0_0] vector.VectorMapOperator: Operator 0 MAP initialized 2018-01-05T11:29:11,722 DEBUG [TezTR-518813_1_42_2_0_0] vector.VectorMapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:29:11,722 INFO [TezTR-518813_1_42_2_0_0] tez.MapRecordProcessor: Id =0 null <\Children> <\Parent> <\MAP> 2018-01-05T11:29:11,722 INFO [TezTR-518813_1_42_2_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:11,722 DEBUG [TezTR-518813_1_42_2_0_0] tez.LlapObjectCache: Caching new object for key: DynamicValueRegistry 2018-01-05T11:29:11,722 INFO [TezTR-518813_1_42_2_0_0] tez.DynamicValueRegistryTez: Runtime value source: Reducer 7 2018-01-05T11:29:11,722 DEBUG [TezTR-518813_1_42_2_0_0] exec.Utilities: Hive Conf not found or Session not initiated, use thread based class loader instead 2018-01-05T11:29:11,722 DEBUG [TezTR-518813_1_42_2_0_0] lazybinary.LazyBinarySerDe: LazyBinarySerDe initialized with: columnNames=[_col0, _col1, _col2] columnTypes=[string, string, binary] 2018-01-05T11:29:11,722 INFO [TezTR-518813_1_42_2_0_0] impl.SimpleFetchedInputAllocator: Reducer_7: RequestedMemory=131404920, AssignedMemory=131404920, maxSingleShuffleLimit=32851230 2018-01-05T11:29:11,723 INFO [TezTR-518813_1_42_2_0_0] impl.ShuffleManager: Reducer_7: numInputs=1, compressionCodec=NoCompressionCodec, numFetchers=1, ifileBufferSize=4096, ifileReadAheadEnabled=true, ifileReadAheadLength=4194304, localDiskFetchEnabled=false, sharedFetchEnabled=false, keepAlive=false, keepAliveMaxConnections=20, connectionTimeout=180000, readTimeout=180000, bufferSize=8192, bufferSize=8192, maxTaskOutputAtOnce=20 2018-01-05T11:29:11,727 WARN [IO-Elevator-Thread-1] impl.RecordReaderImpl: NoDynamicValuesException when evaluating predicate. Skipping ORC PPD. Stats: numberOfValues: 120 stringStatistics { minimum: "val_103" maximum: "val_97" sum: 820 } hasNull: false Predicate: (BETWEEN value DynamicValue(RS_10_c_value_min) DynamicValue(RS_10_c_value_max)) org.apache.hadoop.hive.ql.plan.DynamicValue$NoDynamicValuesException: DynamicValueRegistry not available at org.apache.hadoop.hive.ql.plan.DynamicValue.getValue(DynamicValue.java:135) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.plan.DynamicValue.getJavaValue(DynamicValue.java:97) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.plan.DynamicValue.getLiteral(DynamicValue.java:93) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.io.sarg.SearchArgumentImpl$PredicateLeafImpl.getLiteralList(SearchArgumentImpl.java:120) ~[hive-storage-api-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.orc.impl.RecordReaderImpl.evaluatePredicateMinMax(RecordReaderImpl.java:553) ~[orc-core-1.4.1.jar:1.4.1] at org.apache.orc.impl.RecordReaderImpl.evaluatePredicateRange(RecordReaderImpl.java:463) ~[orc-core-1.4.1.jar:1.4.1] at org.apache.orc.impl.RecordReaderImpl.evaluatePredicateProto(RecordReaderImpl.java:423) ~[orc-core-1.4.1.jar:1.4.1] at org.apache.orc.impl.RecordReaderImpl$SargApplier.pickRowGroups(RecordReaderImpl.java:848) [orc-core-1.4.1.jar:1.4.1] at org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.determineRgsToRead(OrcEncodedDataReader.java:825) [hive-llap-server-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.performDataRead(OrcEncodedDataReader.java:332) [hive-llap-server-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader$4.run(OrcEncodedDataReader.java:273) [hive-llap-server-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader$4.run(OrcEncodedDataReader.java:270) [hive-llap-server-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_101] at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_101] at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962) [hadoop-common-3.0.0-beta1.jar:?] at org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:270) [hive-llap-server-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:110) [hive-llap-server-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) [tez-common-0.9.1-SNAPSHOT.jar:0.9.1-SNAPSHOT] at org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:110) [hive-llap-server-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_101] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_101] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_101] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_101] 2018-01-05T11:29:11,727 DEBUG [IO-Elevator-Thread-1] impl.RecordReaderImpl: Row group 0 to 9999 is included. 2018-01-05T11:29:11,727 INFO [IO-Elevator-Thread-1] impl.OrcCodecPool: Got brand-new codec ZLIB 2018-01-05T11:29:11,727 INFO [IO-Elevator-Thread-1] encoded.EncodedReaderImpl: Resulting disk ranges to read (file [-5122461581150239451, 1515180546000, 767]): [{range start: 76 end: 523}] 2018-01-05T11:29:11,728 INFO [IO-Elevator-Thread-1] encoded.EncodedReaderImpl: Disk ranges after cache (found everything true; file [-5122461581150239451, 1515180546000, 767], base offset 3): [{start: 76 end: 230 cache buffer: 0x112f2b3b(1)}, {start: 230 end: 330 cache buffer: 0x28b0a379(1)}, {start: 330 end: 361 cache buffer: 0xf056330(1)}, {start: 361 end: 523 cache buffer: 0x67fff5bf(1)}] 2018-01-05T11:29:11,728 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 1 columnType: int streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DIRECT_V2 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 164 bytes, ], indexBaseOffset=0] dictionary: false lengths: false secondary: false tz: 2018-01-05T11:29:11,728 DEBUG [IO-Elevator-Thread-1] encoded.EncodedTreeReaderFactory: columnIndex: 2 columnType: string streamBuffers.length: 6 vectors: 0 columnEncoding: kind: DICTIONARY_V2 dictionarySize: 73 present: false data: ColumnStreamData [cacheBuffers=[LlapDataBuffer with 97 bytes, ], indexBaseOffset=0] dictionary: true lengths: true secondary: false tz: 2018-01-05T11:29:11,728 DEBUG [IO-Elevator-Thread-1] LlapIoOrc: Done with decode 2018-01-05T11:29:11,728 DEBUG [IO-Elevator-Thread-1] LlapIoImpl: setDone called; closed false, done false, err null, pending 1 2018-01-05T11:29:11,728 DEBUG [IO-Elevator-Thread-1] impl.StatsRecordingThreadPool: Updated stats: instance: OrcEncodedDataReader thread name: IO-Elevator-Thread-1 thread id: 1309 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:11,734 INFO [TezTR-518813_1_42_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_7: numBuffers=2, sizePerBuffer=52428800, skipBuffers=true, numPartitions=1, availableMemory=104857600, maxSingleBufferSizeBytes=2147483647, pipelinedShuffle=false, isFinalMergeEnabled=true, numPartitions=1, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:29:11,734 DEBUG [TezTR-518813_1_42_0_0_0] tez.MapRecordProcessor: Starting Output: Reducer 2 2018-01-05T11:29:11,735 INFO [TezTR-518813_1_42_0_0_0] impl.ExternalSorter: Reducer 2 using: memoryMb=10, keySerializerClass=class org.apache.hadoop.hive.ql.io.HiveKey, valueSerializerClass=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization$TezBytesWritableSerializer@2ca8e51, comparator=org.apache.tez.runtime.library.common.comparator.TezBytesComparator@539cc98e, partitioner=org.apache.tez.mapreduce.partition.MRPartitioner, serialization=org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.tez.runtime.library.common.serializer.TezBytesWritableSerialization,org.apache.hadoop.io.serializer.WritableSerialization, org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization, org.apache.hadoop.io.serializer.avro.AvroReflectSerialization, reportPartitionStats=MEMORY_OPTIMIZED 2018-01-05T11:29:11,735 INFO [TezTR-518813_1_42_0_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 2: , UsingHashComparator=true 2018-01-05T11:29:11,736 DEBUG [ORC_GET_SPLITS #0] io.AcidUtils: in directory file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 base = null deltas = 0 2018-01-05T11:29:11,736 DEBUG [InputInitializer {Map 8} #0] orc.OrcInputFormat: Split strategy: ETLSplitStrategy strategy for file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08 2018-01-05T11:29:11,736 DEBUG [ORC_GET_SPLITS #4] orc.LocalCache: Serialized tail cached for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000000_0 2018-01-05T11:29:11,736 DEBUG [ORC_GET_SPLITS #4] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:11,736 DEBUG [ORC_GET_SPLITS #4] orc.LocalCache: Serialized tail cached for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000001_0 2018-01-05T11:29:11,736 DEBUG [ORC_GET_SPLITS #4] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:11,736 DEBUG [ORC_GET_SPLITS #4] orc.LocalCache: Serialized tail cached for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000002_0 2018-01-05T11:29:11,736 DEBUG [ORC_GET_SPLITS #4] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:11,736 INFO [TezTR-518813_1_42_0_0_0] impl.PipelinedSorter: Newly allocated block size=10485760, index=0, Number of buffers=1, currentAllocatableMemory=0, currentBufferSize=10485760, total=10485760 2018-01-05T11:29:11,736 INFO [TezTR-518813_1_42_0_0_0] impl.PipelinedSorter: Pre allocating rest of memory buffers upfront 2018-01-05T11:29:11,736 DEBUG [ORC_GET_SPLITS #4] orc.LocalCache: Serialized tail cached for path: file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part/ds=2008-04-08/000003_0 2018-01-05T11:29:11,736 INFO [TezTR-518813_1_42_0_0_0] impl.PipelinedSorter: Setting up PipelinedSorter for Reducer 2: , UsingHashComparator=true#blocks=1, maxMemUsage=10485760, lazyAllocateMem=false, minBlockSize=2097152000, initial BLOCK_SIZE=10485760, finalMergeEnabled=true, pipelinedShuffle=false, sendEmptyPartitions=true, tez.runtime.io.sort.mb=10 2018-01-05T11:29:11,736 DEBUG [ORC_GET_SPLITS #4] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:11,736 INFO [TezTR-518813_1_42_0_0_0] impl.PipelinedSorter: Reducer 2: reserved.remaining()=10485760, reserved.metasize=5242880 2018-01-05T11:29:11,736 INFO [TezTR-518813_1_42_0_0_0] tez.MapRecordProcessor: Main input name is Map 6 2018-01-05T11:29:11,736 DEBUG [ORC_GET_SPLITS #4] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:11,736 INFO [TezTR-518813_1_42_0_0_0] vector.VectorMapOperator: Initializing operator MAP[0] 2018-01-05T11:29:11,736 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorMapOperator: Initialization Done 0 MAP 2018-01-05T11:29:11,736 DEBUG [ORC_GET_SPLITS #3] impl.OrcCodecPool: Got recycled codec 2018-01-05T11:29:11,736 INFO [ORC_GET_SPLITS #1] impl.OrcCodecPool: Got brand-new codec ZLIB 2018-01-05T11:29:11,736 INFO [ORC_GET_SPLITS #5] impl.OrcCodecPool: Got brand-new codec ZLIB 2018-01-05T11:29:11,736 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorMapOperator: Operator 0 MAP initialized 2018-01-05T11:29:11,737 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorMapOperator: Initialization Done 0 MAP done is reset. 2018-01-05T11:29:11,737 DEBUG [ORC_GET_SPLITS #4] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:29:11,737 DEBUG [ORC_GET_SPLITS #3] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:29:11,737 DEBUG [ORC_GET_SPLITS #1] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:29:11,737 DEBUG [ORC_GET_SPLITS #5] impl.ReaderImpl: Unknown primitive category: STRUCT 2018-01-05T11:29:11,737 DEBUG [ORC_GET_SPLITS #4] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:11,737 DEBUG [ORC_GET_SPLITS #3] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:11,737 DEBUG [ORC_GET_SPLITS #1] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:11,737 DEBUG [ORC_GET_SPLITS #5] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:11,737 INFO [TezTR-518813_1_42_0_0_0] tez.MapRecordProcessor: Id =0 Id =3 Id =51 Id =52 Id =53 <\Children> Id = 52 null<\Parent> <\RS> Id =54 Id =55 Id =56 <\Children> Id = 55 null<\Parent> <\RS> <\Children> Id = 54 null<\Parent> <\GBY> <\Children> Id = 52 null<\Parent> <\SEL> <\Children> Id = 51 null<\Parent> <\SEL> <\Children> Id = 3 null<\Parent> <\FIL> <\Children> <\Parent> <\TS> <\Children> <\Parent> <\MAP> 2018-01-05T11:29:11,737 INFO [TezTR-518813_1_42_0_0_0] tez.MapRecordProcessor: Memory available for operators set to 179.02MB 2018-01-05T11:29:11,737 DEBUG [TezTR-518813_1_42_0_0_0] tez.LlapObjectCache: Found DynamicValueRegistry in cache 2018-01-05T11:29:11,737 INFO [TezTR-518813_1_42_0_0_0] exec.TableScanOperator: Initializing operator TS[3] 2018-01-05T11:29:11,737 DEBUG [TezTR-518813_1_42_0_0_0] exec.TableScanOperator: Initialization Done 3 TS 2018-01-05T11:29:11,737 DEBUG [TezTR-518813_1_42_0_0_0] exec.TableScanOperator: Operator 3 TS initialized 2018-01-05T11:29:11,737 DEBUG [TezTR-518813_1_42_0_0_0] exec.TableScanOperator: Initializing children of 3 TS 2018-01-05T11:29:11,737 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorFilterOperator: Initializing child 51 FIL 2018-01-05T11:29:11,737 INFO [TezTR-518813_1_42_0_0_0] vector.VectorFilterOperator: Initializing operator FIL[51] 2018-01-05T11:29:11,737 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorFilterOperator: Initialization Done 51 FIL 2018-01-05T11:29:11,737 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorFilterOperator: Operator 51 FIL initialized 2018-01-05T11:29:11,737 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorFilterOperator: Initializing children of 51 FIL 2018-01-05T11:29:11,737 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorSelectOperator: Initializing child 52 SEL 2018-01-05T11:29:11,737 INFO [TezTR-518813_1_42_0_0_0] vector.VectorSelectOperator: Initializing operator SEL[52] 2018-01-05T11:29:11,737 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorSelectOperator: Initialization Done 52 SEL 2018-01-05T11:29:11,737 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorSelectOperator: Operator 52 SEL initialized 2018-01-05T11:29:11,737 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorSelectOperator: Initializing children of 52 SEL 2018-01-05T11:29:11,737 DEBUG [TezTR-518813_1_42_0_0_0] reducesink.VectorReduceSinkStringOperator: Initializing child 53 RS 2018-01-05T11:29:11,737 INFO [TezTR-518813_1_42_0_0_0] reducesink.VectorReduceSinkStringOperator: Initializing operator RS[53] 2018-01-05T11:29:11,737 INFO [TezTR-518813_1_42_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/djaiswal/_tez_scratch_dir/9493e99b-e6bc-44b7-b894-124305abb6f5/map.xml 2018-01-05T11:29:11,737 DEBUG [TezTR-518813_1_42_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:11,737 INFO [TezTR-518813_1_42_0_0_0] reducesink.VectorReduceSinkCommonOperator: Using tag = 1 2018-01-05T11:29:11,737 DEBUG [TezTR-518813_1_42_0_0_0] reducesink.VectorReduceSinkStringOperator: Initialization Done 53 RS 2018-01-05T11:29:11,737 DEBUG [TezTR-518813_1_42_0_0_0] reducesink.VectorReduceSinkStringOperator: Operator 53 RS initialized 2018-01-05T11:29:11,737 DEBUG [TezTR-518813_1_42_0_0_0] reducesink.VectorReduceSinkStringOperator: Initialization Done 53 RS done is reset. 2018-01-05T11:29:11,737 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorSelectOperator: Initializing child 54 SEL 2018-01-05T11:29:11,737 INFO [TezTR-518813_1_42_0_0_0] vector.VectorSelectOperator: Initializing operator SEL[54] 2018-01-05T11:29:11,737 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorSelectOperator: Initialization Done 54 SEL 2018-01-05T11:29:11,737 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorSelectOperator: Operator 54 SEL initialized 2018-01-05T11:29:11,737 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorSelectOperator: Initializing children of 54 SEL 2018-01-05T11:29:11,737 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorGroupByOperator: Initializing child 55 GBY 2018-01-05T11:29:11,738 INFO [TezTR-518813_1_42_0_0_0] vector.VectorGroupByOperator: Initializing operator GBY[55] 2018-01-05T11:29:11,738 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorGroupByOperator: maxMemory:878Mb (976 * 0.900000) fixSize:884 (key:456 agg:384) 2018-01-05T11:29:11,738 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorGroupByOperator: using hash aggregation processing mode 2018-01-05T11:29:11,738 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorGroupByOperator: Initialization Done 55 GBY 2018-01-05T11:29:11,738 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorGroupByOperator: Operator 55 GBY initialized 2018-01-05T11:29:11,738 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorGroupByOperator: Initializing children of 55 GBY 2018-01-05T11:29:11,738 DEBUG [TezTR-518813_1_42_0_0_0] reducesink.VectorReduceSinkEmptyKeyOperator: Initializing child 56 RS 2018-01-05T11:29:11,738 INFO [TezTR-518813_1_42_0_0_0] reducesink.VectorReduceSinkEmptyKeyOperator: Initializing operator RS[56] 2018-01-05T11:29:11,738 INFO [TezTR-518813_1_42_0_0_0] exec.Utilities: PLAN PATH = file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/djaiswal/_tez_scratch_dir/9493e99b-e6bc-44b7-b894-124305abb6f5/map.xml 2018-01-05T11:29:11,738 DEBUG [TezTR-518813_1_42_0_0_0] exec.Utilities: Found plan in cache for name: map.xml 2018-01-05T11:29:11,738 INFO [TezTR-518813_1_42_0_0_0] reducesink.VectorReduceSinkCommonOperator: Using tag = -1 2018-01-05T11:29:11,738 DEBUG [TezTR-518813_1_42_0_0_0] reducesink.VectorReduceSinkEmptyKeyOperator: Initialization Done 56 RS 2018-01-05T11:29:11,738 DEBUG [TezTR-518813_1_42_0_0_0] reducesink.VectorReduceSinkEmptyKeyOperator: Operator 56 RS initialized 2018-01-05T11:29:11,738 DEBUG [TezTR-518813_1_42_0_0_0] reducesink.VectorReduceSinkEmptyKeyOperator: Initialization Done 56 RS done is reset. 2018-01-05T11:29:11,738 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorGroupByOperator: Initialization Done 55 GBY done is reset. 2018-01-05T11:29:11,738 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorSelectOperator: Initialization Done 54 SEL done is reset. 2018-01-05T11:29:11,738 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorSelectOperator: Initialization Done 52 SEL done is reset. 2018-01-05T11:29:11,738 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorFilterOperator: Initialization Done 51 FIL done is reset. 2018-01-05T11:29:11,738 DEBUG [TezTR-518813_1_42_0_0_0] exec.TableScanOperator: Initialization Done 3 TS done is reset. 2018-01-05T11:29:11,738 INFO [TezTR-518813_1_42_0_0_0] exec.OperatorUtils: Setting output collector: RS[53] --> Reducer 2 2018-01-05T11:29:11,738 INFO [TezTR-518813_1_42_0_0_0] exec.OperatorUtils: Setting output collector: RS[56] --> Reducer 7 2018-01-05T11:29:11,738 DEBUG [TezTR-518813_1_42_0_0_0] log.PerfLogger: 2018-01-05T11:29:11,738 INFO [InputInitializer {Map 8} #0] dag.RootInputInitializerManager: Failed InputInitializer for Input: b on vertex vertex_1515180518813_0001_42_05 [Map 8] 2018-01-05T11:29:11,738 INFO [TezTR-518813_1_42_0_0_0] reducesink.VectorReduceSinkCommonOperator: RS[53]: records written - 1 2018-01-05T11:29:11,738 INFO [TezTR-518813_1_42_0_0_0] reducesink.VectorReduceSinkCommonOperator: RS[53]: records written - 10 2018-01-05T11:29:11,739 INFO [TezTR-518813_1_42_0_0_0] LlapIoImpl: Maximum queue length observed 1 2018-01-05T11:29:11,739 INFO [TezTR-518813_1_42_0_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, , , ]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=0, NUM_ERRORS=0, ROWS_EMITTED=25, METADATA_CACHE_HIT=0, METADATA_CACHE_MISS=0, CACHE_HIT_BYTES=216, CACHE_MISS_BYTES=0, ALLOCATED_BYTES=0, ALLOCATED_USED_BYTES=0, TOTAL_IO_TIME_NS=8967856, DECODE_TIME_NS=7813156, HDFS_TIME_NS=0, CONSUMER_TIME_NS=260097 ] 2018-01-05T11:29:11,739 DEBUG [TezTR-518813_1_42_0_0_0] LlapIoImpl: Encoded reader is being stopped 2018-01-05T11:29:11,739 DEBUG [TezTR-518813_1_42_0_0_0] log.PerfLogger: 2018-01-05T11:29:11,739 DEBUG [TezTR-518813_1_42_0_0_0] mr.ObjectCache: djaiswal_20180105112911_d60e4252-4a13-4247-a28a-f07830bbf022_Map 6__MAP_PLAN__ no longer needed 2018-01-05T11:29:11,739 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorMapOperator: close called for operator MAP[0] 2018-01-05T11:29:11,739 INFO [TezTR-518813_1_42_0_0_0] vector.VectorMapOperator: Closing operator MAP[0] 2018-01-05T11:29:11,739 INFO [TezTR-518813_1_42_0_0_0] vector.VectorMapOperator: DESERIALIZE_ERRORS:0, RECORDS_IN_Map_6:25, 2018-01-05T11:29:11,739 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorMapOperator: Closing child = TS[3] 2018-01-05T11:29:11,739 DEBUG [TezTR-518813_1_42_0_0_0] exec.TableScanOperator: close called for operator TS[3] 2018-01-05T11:29:11,739 INFO [TezTR-518813_1_42_0_0_0] exec.TableScanOperator: Closing operator TS[3] 2018-01-05T11:29:11,739 DEBUG [TezTR-518813_1_42_0_0_0] exec.TableScanOperator: Closing child = FIL[51] 2018-01-05T11:29:11,739 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorFilterOperator: close called for operator FIL[51] 2018-01-05T11:29:11,739 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorFilterOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:11,739 INFO [TezTR-518813_1_42_0_0_0] vector.VectorFilterOperator: Closing operator FIL[51] 2018-01-05T11:29:11,739 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorFilterOperator: Closing child = SEL[52] 2018-01-05T11:29:11,739 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorSelectOperator: close called for operator SEL[52] 2018-01-05T11:29:11,739 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorSelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:11,739 INFO [TezTR-518813_1_42_0_0_0] vector.VectorSelectOperator: Closing operator SEL[52] 2018-01-05T11:29:11,739 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorSelectOperator: Closing child = RS[53] 2018-01-05T11:29:11,739 DEBUG [TezTR-518813_1_42_0_0_0] reducesink.VectorReduceSinkStringOperator: close called for operator RS[53] 2018-01-05T11:29:11,739 DEBUG [TezTR-518813_1_42_0_0_0] reducesink.VectorReduceSinkStringOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:11,739 INFO [TezTR-518813_1_42_0_0_0] reducesink.VectorReduceSinkStringOperator: Closing operator RS[53] 2018-01-05T11:29:11,739 INFO [TezTR-518813_1_42_0_0_0] reducesink.VectorReduceSinkCommonOperator: RS[53]: records written - 25 2018-01-05T11:29:11,739 INFO [TezTR-518813_1_42_0_0_0] reducesink.VectorReduceSinkStringOperator: RECORDS_OUT_INTERMEDIATE_Map_6:25, 2018-01-05T11:29:11,739 DEBUG [TezTR-518813_1_42_0_0_0] reducesink.VectorReduceSinkStringOperator: 53 Close done 2018-01-05T11:29:11,739 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorSelectOperator: Closing child = SEL[54] 2018-01-05T11:29:11,739 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorSelectOperator: close called for operator SEL[54] 2018-01-05T11:29:11,739 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorSelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:11,739 INFO [TezTR-518813_1_42_0_0_0] vector.VectorSelectOperator: Closing operator SEL[54] 2018-01-05T11:29:11,739 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorSelectOperator: Closing child = GBY[55] 2018-01-05T11:29:11,739 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorGroupByOperator: close called for operator GBY[55] 2018-01-05T11:29:11,739 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorGroupByOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:11,739 INFO [TezTR-518813_1_42_0_0_0] vector.VectorGroupByOperator: Closing operator GBY[55] 2018-01-05T11:29:11,739 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorGroupByOperator: Flush 1 (all) entries:1 fixed:884 variable:112 (used:0Mb max:878Mb) gcCanary:alive 2018-01-05T11:29:11,739 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorGroupByOperator: GC canary caused 0 flushes 2018-01-05T11:29:11,739 INFO [TezTR-518813_1_42_0_0_0] reducesink.VectorReduceSinkCommonOperator: RS[56]: records written - 1 2018-01-05T11:29:11,739 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorGroupByOperator: Closing child = RS[56] 2018-01-05T11:29:11,739 DEBUG [TezTR-518813_1_42_0_0_0] reducesink.VectorReduceSinkEmptyKeyOperator: close called for operator RS[56] 2018-01-05T11:29:11,739 DEBUG [TezTR-518813_1_42_0_0_0] reducesink.VectorReduceSinkEmptyKeyOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:11,739 INFO [TezTR-518813_1_42_0_0_0] reducesink.VectorReduceSinkEmptyKeyOperator: Closing operator RS[56] 2018-01-05T11:29:11,739 INFO [TezTR-518813_1_42_0_0_0] reducesink.VectorReduceSinkCommonOperator: RS[56]: records written - 1 2018-01-05T11:29:11,739 INFO [TezTR-518813_1_42_0_0_0] reducesink.VectorReduceSinkEmptyKeyOperator: RECORDS_OUT_INTERMEDIATE_Map_6:1, 2018-01-05T11:29:11,739 DEBUG [TezTR-518813_1_42_0_0_0] reducesink.VectorReduceSinkEmptyKeyOperator: 56 Close done 2018-01-05T11:29:11,739 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorGroupByOperator: 55 Close done 2018-01-05T11:29:11,739 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorSelectOperator: 54 Close done 2018-01-05T11:29:11,739 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorSelectOperator: 52 Close done 2018-01-05T11:29:11,739 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorFilterOperator: 51 Close done 2018-01-05T11:29:11,739 DEBUG [TezTR-518813_1_42_0_0_0] exec.TableScanOperator: 3 Close done 2018-01-05T11:29:11,739 DEBUG [TezTR-518813_1_42_0_0_0] vector.VectorMapOperator: 0 Close done 2018-01-05T11:29:11,739 INFO [TezTR-518813_1_42_0_0_0] task.TaskRunner2Callable: Closing task, taskAttemptId=attempt_1515180518813_0001_42_00_000000_0 2018-01-05T11:29:11,739 INFO [TezTR-518813_1_42_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_7: Waiting for all spills to complete : Pending : 0 2018-01-05T11:29:11,739 INFO [TezTR-518813_1_42_0_0_0] writers.UnorderedPartitionedKVWriter: Reducer_7: All spills complete 2018-01-05T11:29:11,740 ERROR [Dispatcher thread {Central}] impl.VertexImpl: Vertex Input: b initializer failed, vertex=vertex_1515180518813_0001_42_05 [Map 8] org.apache.tez.dag.app.dag.impl.AMUserCodeException: java.lang.RuntimeException: ORC split generation failed with exception: java.lang.IllegalStateException: Failed to retrieve dynamic value for RS_12_a_key_min at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallback.onFailure(RootInputInitializerManager.java:328) ~[tez-dag-0.9.1-SNAPSHOT.jar:0.9.1-SNAPSHOT] at com.google.common.util.concurrent.Futures$6.run(Futures.java:1764) ~[guava-19.0.jar:?] at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:456) ~[guava-19.0.jar:?] at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:817) ~[guava-19.0.jar:?] at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:753) ~[guava-19.0.jar:?] at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:634) ~[guava-19.0.jar:?] at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:110) ~[guava-19.0.jar:?] at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41) ~[guava-19.0.jar:?] at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77) ~[guava-19.0.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_101] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_101] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_101] Caused by: java.lang.RuntimeException: ORC split generation failed with exception: java.lang.IllegalStateException: Failed to retrieve dynamic value for RS_12_a_key_min at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:1784) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getSplits(OrcInputFormat.java:1872) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.io.HiveInputFormat.addSplitsForGroup(HiveInputFormat.java:499) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.io.HiveInputFormat.getSplits(HiveInputFormat.java:684) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:196) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:278) ~[tez-dag-0.9.1-SNAPSHOT.jar:0.9.1-SNAPSHOT] at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:269) ~[tez-dag-0.9.1-SNAPSHOT.jar:0.9.1-SNAPSHOT] at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_101] at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_101] at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962) ~[hadoop-common-3.0.0-beta1.jar:?] at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:269) ~[tez-dag-0.9.1-SNAPSHOT.jar:0.9.1-SNAPSHOT] at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:253) ~[tez-dag-0.9.1-SNAPSHOT.jar:0.9.1-SNAPSHOT] at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108) ~[guava-19.0.jar:?] ... 5 more Caused by: java.util.concurrent.ExecutionException: java.lang.IllegalStateException: Failed to retrieve dynamic value for RS_12_a_key_min at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[?:1.8.0_101] at java.util.concurrent.FutureTask.get(FutureTask.java:192) ~[?:1.8.0_101] at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:1778) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getSplits(OrcInputFormat.java:1872) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.io.HiveInputFormat.addSplitsForGroup(HiveInputFormat.java:499) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.io.HiveInputFormat.getSplits(HiveInputFormat.java:684) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:196) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:278) ~[tez-dag-0.9.1-SNAPSHOT.jar:0.9.1-SNAPSHOT] at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:269) ~[tez-dag-0.9.1-SNAPSHOT.jar:0.9.1-SNAPSHOT] at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_101] at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_101] at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962) ~[hadoop-common-3.0.0-beta1.jar:?] at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:269) ~[tez-dag-0.9.1-SNAPSHOT.jar:0.9.1-SNAPSHOT] at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:253) ~[tez-dag-0.9.1-SNAPSHOT.jar:0.9.1-SNAPSHOT] at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108) ~[guava-19.0.jar:?] ... 5 more Caused by: java.lang.IllegalStateException: Failed to retrieve dynamic value for RS_12_a_key_min at org.apache.hadoop.hive.ql.plan.DynamicValue.getValue(DynamicValue.java:142) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.plan.DynamicValue.getJavaValue(DynamicValue.java:97) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.plan.DynamicValue.getLiteral(DynamicValue.java:93) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.io.sarg.SearchArgumentImpl$PredicateLeafImpl.getLiteralList(SearchArgumentImpl.java:120) ~[hive-storage-api-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.orc.impl.RecordReaderImpl.evaluatePredicateMinMax(RecordReaderImpl.java:553) ~[orc-core-1.4.1.jar:1.4.1] at org.apache.orc.impl.RecordReaderImpl.evaluatePredicateRange(RecordReaderImpl.java:463) ~[orc-core-1.4.1.jar:1.4.1] at org.apache.orc.impl.RecordReaderImpl.evaluatePredicate(RecordReaderImpl.java:440) ~[orc-core-1.4.1.jar:1.4.1] at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.isStripeSatisfyPredicate(OrcInputFormat.java:2163) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.pickStripesInternal(OrcInputFormat.java:2140) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.pickStripes(OrcInputFormat.java:2131) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.access$3000(OrcInputFormat.java:157) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.callInternal(OrcInputFormat.java:1476) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.access$2700(OrcInputFormat.java:1261) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator$1.run(OrcInputFormat.java:1445) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator$1.run(OrcInputFormat.java:1442) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_101] at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_101] at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962) ~[hadoop-common-3.0.0-beta1.jar:?] at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.call(OrcInputFormat.java:1442) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.call(OrcInputFormat.java:1261) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_101] ... 3 more Caused by: java.lang.IllegalStateException: Value does not exist in registry: RS_12_a_key_min at org.apache.hadoop.hive.ql.exec.tez.DynamicValueRegistryTez.getValue(DynamicValueRegistryTez.java:76) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.plan.DynamicValue.getValue(DynamicValue.java:137) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.plan.DynamicValue.getJavaValue(DynamicValue.java:97) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.plan.DynamicValue.getLiteral(DynamicValue.java:93) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.io.sarg.SearchArgumentImpl$PredicateLeafImpl.getLiteralList(SearchArgumentImpl.java:120) ~[hive-storage-api-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.orc.impl.RecordReaderImpl.evaluatePredicateMinMax(RecordReaderImpl.java:553) ~[orc-core-1.4.1.jar:1.4.1] at org.apache.orc.impl.RecordReaderImpl.evaluatePredicateRange(RecordReaderImpl.java:463) ~[orc-core-1.4.1.jar:1.4.1] at org.apache.orc.impl.RecordReaderImpl.evaluatePredicate(RecordReaderImpl.java:440) ~[orc-core-1.4.1.jar:1.4.1] at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.isStripeSatisfyPredicate(OrcInputFormat.java:2163) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.pickStripesInternal(OrcInputFormat.java:2140) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.pickStripes(OrcInputFormat.java:2131) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.access$3000(OrcInputFormat.java:157) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.callInternal(OrcInputFormat.java:1476) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.access$2700(OrcInputFormat.java:1261) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator$1.run(OrcInputFormat.java:1445) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator$1.run(OrcInputFormat.java:1442) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_101] at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_101] at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962) ~[hadoop-common-3.0.0-beta1.jar:?] at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.call(OrcInputFormat.java:1442) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.call(OrcInputFormat.java:1261) ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_101] ... 3 more 2018-01-05T11:29:11,745 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:VERTEX_FINISHED]: vertexName=Map 8, vertexId=vertex_1515180518813_0001_42_05, initRequestedTime=1515180551662, initedTime=0, startRequestedTime=1515180551692, startedTime=0, finishTime=1515180551745, timeTaken=1515180551745, status=FAILED, diagnostics=Vertex vertex_1515180518813_0001_42_05 [Map 8] killed/failed due to:ROOT_INPUT_INIT_FAILURE Vertex Input: b initializer failed, vertex=vertex_1515180518813_0001_42_05 [Map 8], java.lang.RuntimeException: ORC split generation failed with exception: java.lang.IllegalStateException: Failed to retrieve dynamic value for RS_12_a_key_min at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:1784) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getSplits(OrcInputFormat.java:1872) at org.apache.hadoop.hive.ql.io.HiveInputFormat.addSplitsForGroup(HiveInputFormat.java:499) at org.apache.hadoop.hive.ql.io.HiveInputFormat.getSplits(HiveInputFormat.java:684) at org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:196) at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:278) at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:269) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962) at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:269) at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:253) at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108) at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41) at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.util.concurrent.ExecutionException: java.lang.IllegalStateException: Failed to retrieve dynamic value for RS_12_a_key_min at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:1778) ... 17 more Caused by: java.lang.IllegalStateException: Failed to retrieve dynamic value for RS_12_a_key_min at org.apache.hadoop.hive.ql.plan.DynamicValue.getValue(DynamicValue.java:142) at org.apache.hadoop.hive.ql.plan.DynamicValue.getJavaValue(DynamicValue.java:97) at org.apache.hadoop.hive.ql.plan.DynamicValue.getLiteral(DynamicValue.java:93) at org.apache.hadoop.hive.ql.io.sarg.SearchArgumentImpl$PredicateLeafImpl.getLiteralList(SearchArgumentImpl.java:120) at org.apache.orc.impl.RecordReaderImpl.evaluatePredicateMinMax(RecordReaderImpl.java:553) at org.apache.orc.impl.RecordReaderImpl.evaluatePredicateRange(RecordReaderImpl.java:463) at org.apache.orc.impl.RecordReaderImpl.evaluatePredicate(RecordReaderImpl.java:440) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.isStripeSatisfyPredicate(OrcInputFormat.java:2163) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.pickStripesInternal(OrcInputFormat.java:2140) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.pickStripes(OrcInputFormat.java:2131) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.access$3000(OrcInputFormat.java:157) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.callInternal(OrcInputFormat.java:1476) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.access$2700(OrcInputFormat.java:1261) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator$1.run(OrcInputFormat.java:1445) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator$1.run(OrcInputFormat.java:1442) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.call(OrcInputFormat.java:1442) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.call(OrcInputFormat.java:1261) at java.util.concurrent.FutureTask.run(FutureTask.java:266) ... 3 more Caused by: java.lang.IllegalStateException: Value does not exist in registry: RS_12_a_key_min at org.apache.hadoop.hive.ql.exec.tez.DynamicValueRegistryTez.getValue(DynamicValueRegistryTez.java:76) at org.apache.hadoop.hive.ql.plan.DynamicValue.getValue(DynamicValue.java:137) ... 23 more , counters=Counters: 0, vertexStats=firstTaskStartTime=-1, firstTasksToStart=[ ], lastTaskFinishTime=-1, lastTasksToFinish=[ ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=0, numSucceededTasks=0, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:11,745 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_42_05 [Map 8] transitioned from INITIALIZING to FAILED due to event V_ROOT_INPUT_FAILED 2018-01-05T11:29:11,746 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_42_05 [Map 8] completed., numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=8 2018-01-05T11:29:11,746 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=1, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=0, numVertices=8, commitInProgress=0, terminationCause=VERTEX_FAILURE 2018-01-05T11:29:11,746 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_42 transitioned from RUNNING to TERMINATING due to event DAG_VERTEX_COMPLETED 2018-01-05T11:29:11,747 INFO [Dispatcher thread {Central}] impl.VertexImpl: Killing tasks in vertex: vertex_1515180518813_0001_42_04 [Reducer 5] due to trigger: OTHER_VERTEX_FAILURE 2018-01-05T11:29:11,748 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_42_04 [Reducer 5] transitioned from RUNNING to TERMINATING due to event V_TERMINATE 2018-01-05T11:29:11,748 INFO [Dispatcher thread {Central}] impl.VertexImpl: Killing tasks in vertex: vertex_1515180518813_0001_42_03 [Reducer 2] due to trigger: OTHER_VERTEX_FAILURE 2018-01-05T11:29:11,748 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_42_03 [Reducer 2] transitioned from RUNNING to TERMINATING due to event V_TERMINATE 2018-01-05T11:29:11,748 INFO [Dispatcher thread {Central}] impl.VertexImpl: Killing tasks in vertex: vertex_1515180518813_0001_42_02 [Map 1] due to trigger: OTHER_VERTEX_FAILURE 2018-01-05T11:29:11,748 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_42_02 [Map 1] transitioned from RUNNING to TERMINATING due to event V_TERMINATE 2018-01-05T11:29:11,748 INFO [Dispatcher thread {Central}] impl.VertexImpl: Killing tasks in vertex: vertex_1515180518813_0001_42_01 [Reducer 7] due to trigger: OTHER_VERTEX_FAILURE 2018-01-05T11:29:11,748 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_42_01 [Reducer 7] transitioned from RUNNING to TERMINATING due to event V_TERMINATE 2018-01-05T11:29:11,748 INFO [Dispatcher thread {Central}] impl.VertexImpl: Invoking committer abort for vertex, vertexId=vertex_1515180518813_0001_42_07 [Reducer 4] 2018-01-05T11:29:11,749 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:VERTEX_FINISHED]: vertexName=Reducer 4, vertexId=vertex_1515180518813_0001_42_07, initRequestedTime=1515180551676, initedTime=1515180551680, startRequestedTime=0, startedTime=0, finishTime=1515180551749, timeTaken=1515180551749, status=KILLED, diagnostics=Vertex received Kill in INITED state. Vertex vertex_1515180518813_0001_42_07 [Reducer 4] killed/failed due to:OTHER_VERTEX_FAILURE, counters=Counters: 0, vertexStats=firstTaskStartTime=0, firstTasksToStart=[ task_1515180518813_0001_42_07_000000 ], lastTaskFinishTime=-1, lastTasksToFinish=[ ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=0, numSucceededTasks=0, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:11,749 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_42_07 [Reducer 4] transitioned from INITED to KILLED due to event V_TERMINATE 2018-01-05T11:29:11,749 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:VERTEX_FINISHED]: vertexName=Reducer 3, vertexId=vertex_1515180518813_0001_42_06, initRequestedTime=1515180551671, initedTime=1515180551676, startRequestedTime=1515180551692, startedTime=0, finishTime=1515180551749, timeTaken=1515180551749, status=KILLED, diagnostics=Vertex received Kill in INITED state. Vertex vertex_1515180518813_0001_42_06 [Reducer 3] killed/failed due to:OTHER_VERTEX_FAILURE, counters=Counters: 0, vertexStats=firstTaskStartTime=0, firstTasksToStart=[ task_1515180518813_0001_42_06_000000 ], lastTaskFinishTime=-1, lastTasksToFinish=[ ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=0, numSucceededTasks=0, numKilledTasks=0, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:11,749 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_42_06 [Reducer 3] transitioned from INITED to KILLED due to event V_TERMINATE 2018-01-05T11:29:11,749 INFO [Dispatcher thread {Central}] impl.VertexImpl: Killing tasks in vertex: vertex_1515180518813_0001_42_00 [Map 6] due to trigger: OTHER_VERTEX_FAILURE 2018-01-05T11:29:11,749 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_42_00 [Map 6] transitioned from RUNNING to TERMINATING due to event V_TERMINATE 2018-01-05T11:29:11,750 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:TASK_FINISHED]: vertexName=Reducer 2, taskId=task_1515180518813_0001_42_03_000000, startTime=0, finishTime=1515180551750, timeTaken=1515180551750, status=KILLED, successfulAttemptID=null, diagnostics=Killing tasks in vertex: vertex_1515180518813_0001_42_03 [Reducer 2] due to trigger: OTHER_VERTEX_FAILURE, counters=Counters: 0 2018-01-05T11:29:11,750 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_42_07 [Reducer 4] completed., numCompletedVertices=2, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=1, numVertices=8 2018-01-05T11:29:11,750 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=2, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=1, numVertices=8, commitInProgress=0, terminationCause=VERTEX_FAILURE 2018-01-05T11:29:11,750 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_42_06 [Reducer 3] completed., numCompletedVertices=3, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=2, numVertices=8 2018-01-05T11:29:11,750 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=3, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=2, numVertices=8, commitInProgress=0, terminationCause=VERTEX_FAILURE 2018-01-05T11:29:11,750 INFO [TezTR-518813_1_42_0_0_0] impl.PipelinedSorter: Reducer 2: Starting flush of map output 2018-01-05T11:29:11,750 INFO [TezTR-518813_1_42_0_0_0] impl.PipelinedSorter: Reducer 2: Span0.length = 25, perItem = 7 2018-01-05T11:29:11,750 INFO [TezTR-518813_1_42_0_0_0] impl.PipelinedSorter: Reducer 2: done sorting span=0, length=25, time=0 2018-01-05T11:29:11,750 INFO [TezTR-518813_1_42_0_0_0] impl.PipelinedSorter: Reducer 2: Heap = SpanIterator<0:24> (span=Span[5242880,198]), 2018-01-05T11:29:11,751 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 5, taskAttemptId=attempt_1515180518813_0001_42_04_000000_0, creationTime=1515180551693, allocationTime=0, startTime=0, finishTime=1515180551751, timeTaken=1515180551751, status=KILLED, errorEnum=TERMINATED_AT_SHUTDOWN, diagnostics=Task KILL is received. Killing attempt. Diagnostics: Killing tasks in vertex: vertex_1515180518813_0001_42_04 [Reducer 5] due to trigger: OTHER_VERTEX_FAILURE, counters=Counters: 0 2018-01-05T11:29:11,751 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_42_03 [Reducer 2], tasks=1, failed=0, killed=1, success=0, completed=1, commits=0, err=OTHER_VERTEX_FAILURE 2018-01-05T11:29:11,751 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1 2018-01-05T11:29:11,751 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:VERTEX_FINISHED]: vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_42_03, initRequestedTime=1515180551656, initedTime=1515180551661, startRequestedTime=1515180551692, startedTime=1515180551692, finishTime=1515180551751, timeTaken=59, status=KILLED, diagnostics=Vertex received Kill while in RUNNING state. Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1 Vertex vertex_1515180518813_0001_42_03 [Reducer 2] killed/failed due to:OTHER_VERTEX_FAILURE, counters=Counters: 0, vertexStats=firstTaskStartTime=0, firstTasksToStart=[ task_1515180518813_0001_42_03_000000 ], lastTaskFinishTime=-1, lastTasksToFinish=[ ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=0, numKilledTasks=1, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:11,751 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_42_03 [Reducer 2] transitioned from TERMINATING to KILLED due to event V_TASK_COMPLETED 2018-01-05T11:29:11,751 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 1, taskAttemptId=attempt_1515180518813_0001_42_02_000000_0, creationTime=1515180551693, allocationTime=1515180551694, startTime=1515180551694, finishTime=1515180551751, timeTaken=57, status=KILLED, errorEnum=TERMINATED_AT_SHUTDOWN, diagnostics=Task KILL is received. Killing attempt. Diagnostics: Killing tasks in vertex: vertex_1515180518813_0001_42_02 [Map 1] due to trigger: OTHER_VERTEX_FAILURE, nodeHttpAddress=http://10.22.8.182:15002, counters=Counters: 1, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1 2018-01-05T11:29:11,752 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:TASK_ATTEMPT_FINISHED]: vertexName=Reducer 7, taskAttemptId=attempt_1515180518813_0001_42_01_000000_0, creationTime=1515180551681, allocationTime=1515180551684, startTime=1515180551686, finishTime=1515180551752, timeTaken=66, status=KILLED, errorEnum=TERMINATED_AT_SHUTDOWN, diagnostics=Task KILL is received. Killing attempt. Diagnostics: Killing tasks in vertex: vertex_1515180518813_0001_42_01 [Reducer 7] due to trigger: OTHER_VERTEX_FAILURE, nodeHttpAddress=http://10.22.8.182:15002, counters=Counters: 0 2018-01-05T11:29:11,753 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:TASK_ATTEMPT_FINISHED]: vertexName=Map 6, taskAttemptId=attempt_1515180518813_0001_42_00_000000_0, creationTime=1515180551681, allocationTime=1515180551682, startTime=1515180551685, finishTime=1515180551753, timeTaken=68, status=KILLED, errorEnum=TERMINATED_AT_SHUTDOWN, diagnostics=Task KILL is received. Killing attempt. Diagnostics: Killing tasks in vertex: vertex_1515180518813_0001_42_00 [Map 6] due to trigger: OTHER_VERTEX_FAILURE, nodeHttpAddress=http://10.22.8.182:15002, counters=Counters: 1, org.apache.tez.common.counters.DAGCounter, OTHER_LOCAL_TASKS=1 2018-01-05T11:29:11,753 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:TASK_FINISHED]: vertexName=Reducer 5, taskId=task_1515180518813_0001_42_04_000000, startTime=1515180551693, finishTime=1515180551753, timeTaken=60, status=KILLED, successfulAttemptID=null, diagnostics=Killing tasks in vertex: vertex_1515180518813_0001_42_04 [Reducer 5] due to trigger: OTHER_VERTEX_FAILURE, counters=Counters: 0 2018-01-05T11:29:11,753 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_42_03 [Reducer 2] completed., numCompletedVertices=4, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=3, numVertices=8 2018-01-05T11:29:11,753 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=4, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=3, numVertices=8, commitInProgress=0, terminationCause=VERTEX_FAILURE 2018-01-05T11:29:11,753 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1515180518813_0001_42_02_000000, startTime=1515180551694, finishTime=1515180551753, timeTaken=59, status=KILLED, successfulAttemptID=null, diagnostics=Killing tasks in vertex: vertex_1515180518813_0001_42_02 [Map 1] due to trigger: OTHER_VERTEX_FAILURE, counters=Counters: 0 2018-01-05T11:29:11,753 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_42_02_000000_0, taskSucceeded=false, endReason=OTHER 2018-01-05T11:29:11,753 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_42_02_000000_0; false 2018-01-05T11:29:11,753 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_42_02_000000_0, state=ASSIGNED, endReason=OTHER 2018-01-05T11:29:11,753 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:TASK_FINISHED]: vertexName=Reducer 7, taskId=task_1515180518813_0001_42_01_000000, startTime=1515180551686, finishTime=1515180551753, timeTaken=67, status=KILLED, successfulAttemptID=null, diagnostics=Killing tasks in vertex: vertex_1515180518813_0001_42_01 [Reducer 7] due to trigger: OTHER_VERTEX_FAILURE, counters=Counters: 0 2018-01-05T11:29:11,754 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:TASK_FINISHED]: vertexName=Map 6, taskId=task_1515180518813_0001_42_00_000000, startTime=1515180551685, finishTime=1515180551754, timeTaken=69, status=KILLED, successfulAttemptID=null, diagnostics=Killing tasks in vertex: vertex_1515180518813_0001_42_00 [Map 6] due to trigger: OTHER_VERTEX_FAILURE, counters=Counters: 0 2018-01-05T11:29:11,754 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Disabling instance {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=false, st=2, ac=6, commF=false, disabled=true} for 200 milli-seconds. commFailure=false 2018-01-05T11:29:11,754 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_42_04 [Reducer 5], tasks=1, failed=0, killed=1, success=0, completed=1, commits=0, err=OTHER_VERTEX_FAILURE 2018-01-05T11:29:11,754 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1 2018-01-05T11:29:11,754 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:11,754 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:11,754 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:VERTEX_FINISHED]: vertexName=Reducer 5, vertexId=vertex_1515180518813_0001_42_04, initRequestedTime=1515180551662, initedTime=1515180551662, startRequestedTime=1515180551692, startedTime=1515180551692, finishTime=1515180551754, timeTaken=62, status=KILLED, diagnostics=Vertex received Kill while in RUNNING state. Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1 Vertex vertex_1515180518813_0001_42_04 [Reducer 5] killed/failed due to:OTHER_VERTEX_FAILURE, counters=Counters: 0, vertexStats=firstTaskStartTime=1515180551693, firstTasksToStart=[ task_1515180518813_0001_42_04_000000 ], lastTaskFinishTime=-1, lastTasksToFinish=[ ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=0, numKilledTasks=1, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:11,754 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_42_04 [Reducer 5] transitioned from TERMINATING to KILLED due to event V_TASK_COMPLETED 2018-01-05T11:29:11,754 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_42_02 [Map 1], tasks=1, failed=0, killed=1, success=0, completed=1, commits=0, err=OTHER_VERTEX_FAILURE 2018-01-05T11:29:11,754 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1 2018-01-05T11:29:11,754 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:VERTEX_FINISHED]: vertexName=Map 1, vertexId=vertex_1515180518813_0001_42_02, initRequestedTime=1515180551648, initedTime=1515180551692, startRequestedTime=1515180551681, startedTime=1515180551692, finishTime=1515180551754, timeTaken=62, status=KILLED, diagnostics=Vertex received Kill while in RUNNING state. Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1 Vertex vertex_1515180518813_0001_42_02 [Map 1] killed/failed due to:OTHER_VERTEX_FAILURE, counters=Counters: 0, vertexStats=firstTaskStartTime=1515180551694, firstTasksToStart=[ task_1515180518813_0001_42_02_000000 ], lastTaskFinishTime=1515180551751, lastTasksToFinish=[ task_1515180518813_0001_42_02_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=0, numKilledTasks=1, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:11,754 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_42_02 [Map 1] transitioned from TERMINATING to KILLED due to event V_TASK_COMPLETED 2018-01-05T11:29:11,754 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_42_01 [Reducer 7], tasks=1, failed=0, killed=1, success=0, completed=1, commits=0, err=OTHER_VERTEX_FAILURE 2018-01-05T11:29:11,754 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1 2018-01-05T11:29:11,754 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:VERTEX_FINISHED]: vertexName=Reducer 7, vertexId=vertex_1515180518813_0001_42_01, initRequestedTime=1515180551647, initedTime=1515180551648, startRequestedTime=1515180551680, startedTime=1515180551680, finishTime=1515180551754, timeTaken=74, status=KILLED, diagnostics=Vertex received Kill while in RUNNING state. Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1 Vertex vertex_1515180518813_0001_42_01 [Reducer 7] killed/failed due to:OTHER_VERTEX_FAILURE, counters=Counters: 0, vertexStats=firstTaskStartTime=1515180551686, firstTasksToStart=[ task_1515180518813_0001_42_01_000000 ], lastTaskFinishTime=1515180551752, lastTasksToFinish=[ task_1515180518813_0001_42_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=0, numKilledTasks=1, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:11,754 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_42_01 [Reducer 7] transitioned from TERMINATING to KILLED due to event V_TASK_COMPLETED 2018-01-05T11:29:11,754 INFO [Dispatcher thread {Central}] impl.VertexImpl: Task Completion: vertex_1515180518813_0001_42_00 [Map 6], tasks=1, failed=0, killed=1, success=0, completed=1, commits=0, err=OTHER_VERTEX_FAILURE 2018-01-05T11:29:11,754 INFO [Dispatcher thread {Central}] impl.VertexImpl: Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1 2018-01-05T11:29:11,754 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:VERTEX_FINISHED]: vertexName=Map 6, vertexId=vertex_1515180518813_0001_42_00, initRequestedTime=1515180551638, initedTime=1515180551676, startRequestedTime=1515180551647, startedTime=1515180551680, finishTime=1515180551754, timeTaken=74, status=KILLED, diagnostics=Vertex received Kill while in RUNNING state. Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1 Vertex vertex_1515180518813_0001_42_00 [Map 6] killed/failed due to:OTHER_VERTEX_FAILURE, counters=Counters: 0, vertexStats=firstTaskStartTime=1515180551685, firstTasksToStart=[ task_1515180518813_0001_42_00_000000 ], lastTaskFinishTime=1515180551753, lastTasksToFinish=[ task_1515180518813_0001_42_00_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTaskAttempts=0, numKilledTaskAttempts=0, numCompletedTasks=1, numSucceededTasks=0, numKilledTasks=1, numFailedTasks=0}, servicePluginInfo=ServicePluginInfo {containerLauncherName=LLAP, taskSchedulerName=LLAP, taskCommunicatorName=LLAP, containerLauncherClassName=org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher, taskSchedulerClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService, taskCommunicatorClassName=org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator } 2018-01-05T11:29:11,754 INFO [Dispatcher thread {Central}] impl.VertexImpl: vertex_1515180518813_0001_42_00 [Map 6] transitioned from TERMINATING to KILLED due to event V_TASK_COMPLETED 2018-01-05T11:29:11,754 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_42_04 [Reducer 5] completed., numCompletedVertices=5, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=4, numVertices=8 2018-01-05T11:29:11,754 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=5, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=4, numVertices=8, commitInProgress=0, terminationCause=VERTEX_FAILURE 2018-01-05T11:29:11,754 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_42_01_000000_0, taskSucceeded=false, endReason=OTHER 2018-01-05T11:29:11,755 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_42_01_000000_0; false 2018-01-05T11:29:11,755 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_42_01_000000_0, state=ASSIGNED, endReason=OTHER 2018-01-05T11:29:11,755 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Node: {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=false, st=1, ac=7, commF=false, disabled=true} already disabled, or invalid. Not doing anything. 2018-01-05T11:29:11,755 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing deallocateTask for task=attempt_1515180518813_0001_42_00_000000_0, taskSucceeded=false, endReason=OTHER 2018-01-05T11:29:11,755 INFO [TaskSchedulerEventHandlerThread] GuaranteedTasks: Unregistering attempt_1515180518813_0001_42_00_000000_0; false 2018-01-05T11:29:11,755 INFO [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Processing de-allocate request for task=attempt_1515180518813_0001_42_00_000000_0, state=ASSIGNED, endReason=OTHER 2018-01-05T11:29:11,755 DEBUG [TaskSchedulerEventHandlerThread] tezplugins.LlapTaskSchedulerService: Node: {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=false, st=0, ac=8, commF=false, disabled=true} already disabled, or invalid. Not doing anything. 2018-01-05T11:29:11,755 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000107, stoppedTime=1515180551755, exitStatus=-1000 2018-01-05T11:29:11,755 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_42_02 [Map 1] completed., numCompletedVertices=6, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=5, numVertices=8 2018-01-05T11:29:11,755 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=6, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=5, numVertices=8, commitInProgress=0, terminationCause=VERTEX_FAILURE 2018-01-05T11:29:11,755 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_42_01 [Reducer 7] completed., numCompletedVertices=7, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=6, numVertices=8 2018-01-05T11:29:11,755 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=7, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=6, numVertices=8, commitInProgress=0, terminationCause=VERTEX_FAILURE 2018-01-05T11:29:11,755 INFO [Dispatcher thread {Central}] impl.DAGImpl: Vertex vertex_1515180518813_0001_42_00 [Map 6] completed., numCompletedVertices=8, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=7, numVertices=8 2018-01-05T11:29:11,755 INFO [Dispatcher thread {Central}] impl.DAGImpl: Checking vertices for DAG completion, numCompletedVertices=8, numSuccessfulVertices=0, numFailedVertices=1, numKilledVertices=7, numVertices=8, commitInProgress=0, terminationCause=VERTEX_FAILURE 2018-01-05T11:29:11,755 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:7 2018-01-05T11:29:11,755 INFO [Dispatcher thread {Central}] recovery.RecoveryService: DAG completed, dagId=dag_1515180518813_0001_42, queueSize=0 2018-01-05T11:29:11,755 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:DAG_FINISHED]: dagId=dag_1515180518813_0001_42, startTime=1515180551638, finishTime=1515180551755, timeTaken=117, status=FAILED, diagnostics=Vertex failed, vertexName=Map 8, vertexId=vertex_1515180518813_0001_42_05, diagnostics=[Vertex vertex_1515180518813_0001_42_05 [Map 8] killed/failed due to:ROOT_INPUT_INIT_FAILURE, Vertex Input: b initializer failed, vertex=vertex_1515180518813_0001_42_05 [Map 8], java.lang.RuntimeException: ORC split generation failed with exception: java.lang.IllegalStateException: Failed to retrieve dynamic value for RS_12_a_key_min at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:1784) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getSplits(OrcInputFormat.java:1872) at org.apache.hadoop.hive.ql.io.HiveInputFormat.addSplitsForGroup(HiveInputFormat.java:499) at org.apache.hadoop.hive.ql.io.HiveInputFormat.getSplits(HiveInputFormat.java:684) at org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:196) at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:278) at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:269) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962) at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:269) at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:253) at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108) at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41) at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.util.concurrent.ExecutionException: java.lang.IllegalStateException: Failed to retrieve dynamic value for RS_12_a_key_min at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:1778) ... 17 more Caused by: java.lang.IllegalStateException: Failed to retrieve dynamic value for RS_12_a_key_min at org.apache.hadoop.hive.ql.plan.DynamicValue.getValue(DynamicValue.java:142) at org.apache.hadoop.hive.ql.plan.DynamicValue.getJavaValue(DynamicValue.java:97) at org.apache.hadoop.hive.ql.plan.DynamicValue.getLiteral(DynamicValue.java:93) at org.apache.hadoop.hive.ql.io.sarg.SearchArgumentImpl$PredicateLeafImpl.getLiteralList(SearchArgumentImpl.java:120) at org.apache.orc.impl.RecordReaderImpl.evaluatePredicateMinMax(RecordReaderImpl.java:553) at org.apache.orc.impl.RecordReaderImpl.evaluatePredicateRange(RecordReaderImpl.java:463) at org.apache.orc.impl.RecordReaderImpl.evaluatePredicate(RecordReaderImpl.java:440) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.isStripeSatisfyPredicate(OrcInputFormat.java:2163) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.pickStripesInternal(OrcInputFormat.java:2140) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.pickStripes(OrcInputFormat.java:2131) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.access$3000(OrcInputFormat.java:157) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.callInternal(OrcInputFormat.java:1476) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.access$2700(OrcInputFormat.java:1261) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator$1.run(OrcInputFormat.java:1445) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator$1.run(OrcInputFormat.java:1442) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.call(OrcInputFormat.java:1442) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.call(OrcInputFormat.java:1261) at java.util.concurrent.FutureTask.run(FutureTask.java:266) ... 3 more Caused by: java.lang.IllegalStateException: Value does not exist in registry: RS_12_a_key_min at org.apache.hadoop.hive.ql.exec.tez.DynamicValueRegistryTez.getValue(DynamicValueRegistryTez.java:76) at org.apache.hadoop.hive.ql.plan.DynamicValue.getValue(DynamicValue.java:137) ... 23 more ] Vertex killed, vertexName=Reducer 4, vertexId=vertex_1515180518813_0001_42_07, diagnostics=[Vertex received Kill in INITED state., Vertex vertex_1515180518813_0001_42_07 [Reducer 4] killed/failed due to:OTHER_VERTEX_FAILURE] Vertex killed, vertexName=Reducer 3, vertexId=vertex_1515180518813_0001_42_06, diagnostics=[Vertex received Kill in INITED state., Vertex vertex_1515180518813_0001_42_06 [Reducer 3] killed/failed due to:OTHER_VERTEX_FAILURE] Vertex killed, vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_42_03, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1, Vertex vertex_1515180518813_0001_42_03 [Reducer 2] killed/failed due to:OTHER_VERTEX_FAILURE] Vertex killed, vertexName=Reducer 5, vertexId=vertex_1515180518813_0001_42_04, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1, Vertex vertex_1515180518813_0001_42_04 [Reducer 5] killed/failed due to:OTHER_VERTEX_FAILURE] Vertex killed, vertexName=Map 1, vertexId=vertex_1515180518813_0001_42_02, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1, Vertex vertex_1515180518813_0001_42_02 [Map 1] killed/failed due to:OTHER_VERTEX_FAILURE] Vertex killed, vertexName=Reducer 7, vertexId=vertex_1515180518813_0001_42_01, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1, Vertex vertex_1515180518813_0001_42_01 [Reducer 7] killed/failed due to:OTHER_VERTEX_FAILURE] Vertex killed, vertexName=Map 6, vertexId=vertex_1515180518813_0001_42_00, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1, Vertex vertex_1515180518813_0001_42_00 [Map 6] killed/failed due to:OTHER_VERTEX_FAILURE] DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:7, counters=Counters: 4, org.apache.tez.common.counters.DAGCounter, NUM_KILLED_TASKS=4, TOTAL_LAUNCHED_TASKS=3, AM_CPU_MILLISECONDS=312, AM_GC_TIME_MILLIS=0 2018-01-05T11:29:11,756 INFO [Dispatcher thread {Central}] impl.VertexImpl: Ignoring multiple aborts for vertex: vertex_1515180518813_0001_42_04 [Reducer 5] 2018-01-05T11:29:11,756 INFO [Dispatcher thread {Central}] impl.VertexImpl: Ignoring multiple aborts for vertex: vertex_1515180518813_0001_42_03 [Reducer 2] 2018-01-05T11:29:11,756 INFO [Dispatcher thread {Central}] impl.VertexImpl: Ignoring multiple aborts for vertex: vertex_1515180518813_0001_42_02 [Map 1] 2018-01-05T11:29:11,756 INFO [Dispatcher thread {Central}] impl.VertexImpl: Ignoring multiple aborts for vertex: vertex_1515180518813_0001_42_01 [Reducer 7] 2018-01-05T11:29:11,756 INFO [Dispatcher thread {Central}] impl.VertexImpl: Ignoring multiple aborts for vertex: vertex_1515180518813_0001_42_07 [Reducer 4] 2018-01-05T11:29:11,756 INFO [Dispatcher thread {Central}] impl.VertexImpl: Ignoring multiple aborts for vertex: vertex_1515180518813_0001_42_06 [Reducer 3] 2018-01-05T11:29:11,756 INFO [Dispatcher thread {Central}] impl.VertexImpl: Ignoring multiple aborts for vertex: vertex_1515180518813_0001_42_05 [Map 8] 2018-01-05T11:29:11,756 INFO [Dispatcher thread {Central}] impl.VertexImpl: Ignoring multiple aborts for vertex: vertex_1515180518813_0001_42_00 [Map 6] 2018-01-05T11:29:11,756 INFO [Dispatcher thread {Central}] impl.DAGImpl: DAG: dag_1515180518813_0001_42 finished with state: FAILED 2018-01-05T11:29:11,756 INFO [Dispatcher thread {Central}] impl.DAGImpl: dag_1515180518813_0001_42 transitioned from TERMINATING to FAILED due to event DAG_VERTEX_COMPLETED 2018-01-05T11:29:11,756 INFO [Dispatcher thread {Central}] node.AMNodeImpl: Attempt killedon node: 10.22.8.182:49818 TA: attempt_1515180518813_0001_42_02_000000_0, container: container_111101111_0001_00_000107, numFailedTAs: 0 2018-01-05T11:29:11,756 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000106, stoppedTime=1515180551756, exitStatus=-1000 2018-01-05T11:29:11,756 INFO [Dispatcher thread {Central}] node.AMNodeImpl: Attempt killedon node: 10.22.8.182:49818 TA: attempt_1515180518813_0001_42_01_000000_0, container: container_111101111_0001_00_000106, numFailedTAs: 0 2018-01-05T11:29:11,756 INFO [Dispatcher thread {Central}] HistoryEventHandler.criticalEvents: [HISTORY][DAG:dag_1515180518813_0001_42][Event:CONTAINER_STOPPED]: containerId=container_111101111_0001_00_000105, stoppedTime=1515180551756, exitStatus=-1000 2018-01-05T11:29:11,756 INFO [Dispatcher thread {Central}] node.AMNodeImpl: Attempt killedon node: 10.22.8.182:49818 TA: attempt_1515180518813_0001_42_00_000000_0, container: container_111101111_0001_00_000105, numFailedTAs: 0 2018-01-05T11:29:11,756 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000107 exited with diagnostics set to Container completed. 2018-01-05T11:29:11,756 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000106 exited with diagnostics set to Container completed. 2018-01-05T11:29:11,756 INFO [Dispatcher thread {Central}] container.AMContainerImpl: Container container_111101111_0001_00_000105 exited with diagnostics set to Container completed. 2018-01-05T11:29:11,756 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000107 2018-01-05T11:29:11,756 INFO [Dispatcher thread {Central}] app.DAGAppMaster: DAG completed, dagId=dag_1515180518813_0001_42, dagState=FAILED 2018-01-05T11:29:11,756 INFO [Dispatcher thread {Central}] common.TezUtilsInternal: Redirecting log file based on addend: dag_1515180518813_0001_42_post 2018-01-05T11:29:11,756 WARN [Dispatcher thread {Central}] common.TezUtilsInternal: Not configured with appender named: CLA. Cannot reconfigure logger output 2018-01-05T11:29:11,756 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Central Dispatcher queue size after DAG completion, before cleanup: 9 2018-01-05T11:29:11,756 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Waiting for next DAG to be submitted. 2018-01-05T11:29:11,756 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG: 42 completed. Scheduling stats: NumPreemptedTasks=0, NumRequestedAllocations=3, NumRequestsWithlocation=0, NumLocalAllocations=0,NumNonLocalAllocations=0,NumTotalAllocations=3,NumRequestsWithoutLocation=3, NumRejectedTasks=0, NumCommFailures=0, NumDelayedAllocations=0, LocalityBasedAllocationsPerHost={}, NumAllocationsPerHost={10.22.8.182=3} 2018-01-05T11:29:11,756 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskSchedulerService: DAG reset. Current knownTaskCount=0, pendingTaskCount=0, runningTaskCount=0 2018-01-05T11:29:11,756 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000106 2018-01-05T11:29:11,756 DEBUG [Dispatcher thread {Central}] tezplugins.LlapContainerLauncher: No-op stopContainer invocation for containerId=container_111101111_0001_00_000105 2018-01-05T11:29:11,756 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Cleaning up DAG: name=select count(*) from tab a join ta...c.value (Stage-1), with id=dag_1515180518813_0001_42 2018-01-05T11:29:11,756 INFO [Dispatcher thread {Central}] tezplugins.LlapTaskCommunicator: Sending dagComplete message for 42, to 10.22.8.182:49818 2018-01-05T11:29:11,756 INFO [Dispatcher thread {Central}] container.AMContainerMap: Cleaned up completed containers on dagComplete. Removed=3, Remaining=0 2018-01-05T11:29:11,756 INFO [Dispatcher thread {Central}] app.DAGAppMaster: Completed cleanup for DAG: name=select count(*) from tab a join ta...c.value (Stage-1), with id=dag_1515180518813_0001_42 2018-01-05T11:29:11,757 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Processing queryComplete notification for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=42} 2018-01-05T11:29:11,757 INFO [IPC Server handler 3 on 49818] impl.QueryTracker: Processing queryComplete for queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=42}, isExternalQuery=false, with deleteDelay=300 seconds 2018-01-05T11:29:11,757 INFO [IPC Server handler 3 on 49818] impl.QueryTracker: Scheduling deletion of /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/42 after 300 seconds 2018-01-05T11:29:11,757 ERROR [IPC Server handler 3 on 49818] impl.QueryTracker: Ignore this. Log line to interact with logger. Query complete: djaiswal_20180105112911_d60e4252-4a13-4247-a28a-f07830bbf022, dag_1515180518813_0001_42 2018-01-05T11:29:11,757 INFO [IPC Server handler 3 on 49818] exec.ObjectCacheFactory: Removing object cache for djaiswal_20180105112911_d60e4252-4a13-4247-a28a-f07830bbf022 2018-01-05T11:29:11,757 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: DBG: Pending fragment count for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=42} = 3 2018-01-05T11:29:11,757 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Issuing killFragment for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=42} attempt_1515180518813_0001_42_02_000000_0 2018-01-05T11:29:11,757 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Removing attempt_1515180518813_0001_42_02_000000_0 from preemptionQueue 2018-01-05T11:29:11,757 INFO [IPC Server handler 3 on 49818] impl.TaskRunnerCallable: Kill task requested for id=attempt_1515180518813_0001_42_02_000000_0, taskRunnerSetup=true 2018-01-05T11:29:11,757 INFO [IPC Server handler 3 on 49818] impl.TaskRunnerCallable: Issuing kill to task attempt_1515180518813_0001_42_02_000000_0 2018-01-05T11:29:11,758 INFO [IPC Server handler 3 on 49818] task.TezTaskRunner2: Attempting to abort attempt_1515180518813_0001_42_02_000000_0 due to an invocation of killTask 2018-01-05T11:29:11,758 INFO [IPC Server handler 3 on 49818] tez.TezProcessor: Received abort 2018-01-05T11:29:11,758 INFO [IPC Server handler 3 on 49818] tez.TezProcessor: Forwarding abort to RecordProcessor 2018-01-05T11:29:11,758 INFO [IPC Server handler 3 on 49818] tez.MapRecordProcessor: Forwarding abort to mapOp: {} MAP 2018-01-05T11:29:11,758 INFO [IPC Server handler 3 on 49818] vector.VectorMapOperator: Received abort in operator: MAP 2018-01-05T11:29:11,758 INFO [IPC Server handler 3 on 49818] impl.TaskRunnerCallable: Kill request for task attempt_1515180518813_0001_42_02_000000_0 completed. Informing AM 2018-01-05T11:29:11,758 INFO [TezTR-518813_1_42_2_0_0] tez.MapRecordProcessor: Hit an interrupt while initializing MapRecordProcessor. Message=null 2018-01-05T11:29:11,760 DEBUG [TezTR-518813_1_42_2_0_0] mr.ObjectCache: djaiswal_20180105112911_d60e4252-4a13-4247-a28a-f07830bbf022_Map 1__MAP_PLAN__ no longer needed 2018-01-05T11:29:11,760 DEBUG [TezTR-518813_1_42_2_0_0] vector.VectorMapOperator: close called for operator MAP[0] 2018-01-05T11:29:11,760 INFO [TezTR-518813_1_42_2_0_0] vector.VectorMapOperator: Closing operator MAP[0] 2018-01-05T11:29:11,760 DEBUG [TezTR-518813_1_42_2_0_0] vector.VectorMapOperator: Closing child = TS[0] 2018-01-05T11:29:11,760 DEBUG [TezTR-518813_1_42_2_0_0] exec.TableScanOperator: close called for operator TS[0] 2018-01-05T11:29:11,760 INFO [TezTR-518813_1_42_2_0_0] exec.TableScanOperator: Closing operator TS[0] 2018-01-05T11:29:11,760 DEBUG [TezTR-518813_1_42_2_0_0] exec.TableScanOperator: Closing child = FIL[59] 2018-01-05T11:29:11,760 DEBUG [TezTR-518813_1_42_2_0_0] vector.VectorFilterOperator: close called for operator FIL[59] 2018-01-05T11:29:11,760 DEBUG [TezTR-518813_1_42_2_0_0] vector.VectorFilterOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:11,760 INFO [TezTR-518813_1_42_2_0_0] vector.VectorFilterOperator: Closing operator FIL[59] 2018-01-05T11:29:11,760 DEBUG [TezTR-518813_1_42_2_0_0] vector.VectorFilterOperator: Closing child = SEL[60] 2018-01-05T11:29:11,760 DEBUG [TezTR-518813_1_42_2_0_0] vector.VectorSelectOperator: close called for operator SEL[60] 2018-01-05T11:29:11,760 DEBUG [TezTR-518813_1_42_2_0_0] vector.VectorSelectOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:11,760 INFO [TezTR-518813_1_42_2_0_0] vector.VectorSelectOperator: Closing operator SEL[60] 2018-01-05T11:29:11,760 DEBUG [TezTR-518813_1_42_2_0_0] vector.VectorSelectOperator: Closing child = RS[61] 2018-01-05T11:29:11,760 DEBUG [TezTR-518813_1_42_2_0_0] reducesink.VectorReduceSinkStringOperator: close called for operator RS[61] 2018-01-05T11:29:11,760 DEBUG [TezTR-518813_1_42_2_0_0] reducesink.VectorReduceSinkStringOperator: allInitializedParentsAreClosed? parent.state = CLOSE 2018-01-05T11:29:11,760 INFO [TezTR-518813_1_42_2_0_0] reducesink.VectorReduceSinkStringOperator: Closing operator RS[61] 2018-01-05T11:29:11,760 INFO [TezTR-518813_1_42_2_0_0] reducesink.VectorReduceSinkCommonOperator: RS[61]: records written - 0 2018-01-05T11:29:11,760 DEBUG [TezTR-518813_1_42_2_0_0] reducesink.VectorReduceSinkStringOperator: 61 Close done 2018-01-05T11:29:11,760 DEBUG [TezTR-518813_1_42_2_0_0] vector.VectorSelectOperator: 60 Close done 2018-01-05T11:29:11,760 DEBUG [TezTR-518813_1_42_2_0_0] vector.VectorFilterOperator: 59 Close done 2018-01-05T11:29:11,760 DEBUG [TezTR-518813_1_42_2_0_0] exec.TableScanOperator: 0 Close done 2018-01-05T11:29:11,760 DEBUG [TezTR-518813_1_42_2_0_0] vector.VectorMapOperator: 0 Close done 2018-01-05T11:29:11,760 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Issuing killFragment for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=42} attempt_1515180518813_0001_42_01_000000_0 2018-01-05T11:29:11,760 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Removing attempt_1515180518813_0001_42_01_000000_0 from preemptionQueue 2018-01-05T11:29:11,761 INFO [IPC Server handler 3 on 49818] impl.TaskRunnerCallable: Kill task requested for id=attempt_1515180518813_0001_42_01_000000_0, taskRunnerSetup=true 2018-01-05T11:29:11,761 INFO [IPC Server handler 3 on 49818] impl.TaskRunnerCallable: Issuing kill to task attempt_1515180518813_0001_42_01_000000_0 2018-01-05T11:29:11,761 INFO [IPC Server handler 3 on 49818] task.TezTaskRunner2: Attempting to abort attempt_1515180518813_0001_42_01_000000_0 due to an invocation of killTask 2018-01-05T11:29:11,761 INFO [IPC Server handler 3 on 49818] tez.TezProcessor: Received abort 2018-01-05T11:29:11,761 INFO [IPC Server handler 3 on 49818] tez.TezProcessor: Forwarding abort to RecordProcessor 2018-01-05T11:29:11,761 INFO [IPC Server handler 3 on 49818] tez.ReduceRecordProcessor: reducer not setup yet. abort not being forwarded 2018-01-05T11:29:11,761 INFO [IPC Server handler 3 on 49818] impl.TaskRunnerCallable: Kill request for task attempt_1515180518813_0001_42_01_000000_0 completed. Informing AM 2018-01-05T11:29:11,761 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] monitoring.RenderStrategy$LogToFileFunction: Map 1: 0/1 Map 6: 0/1 Map 8: -/- Reducer 2: 0/1 Reducer 3: 0/1 Reducer 4: 0/1 Reducer 5: 0/1 Reducer 7: 0/1 2018-01-05T11:29:11,762 DEBUG [TezTR-518813_1_42_1_0_0] mr.ObjectCache: djaiswal_20180105112911_d60e4252-4a13-4247-a28a-f07830bbf022_Reducer 7__REDUCE_PLAN__ no longer needed 2018-01-05T11:29:11,762 ERROR [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: Status: Failed 2018-01-05T11:29:11,762 ERROR [TezTR-518813_1_42_2_0_0] tez.TezProcessor: java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) at org.apache.tez.runtime.InputReadyTracker$InputReadyMonitor.awaitCondition(InputReadyTracker.java:147) at org.apache.tez.runtime.InputReadyTracker.waitForAllInputsReady(InputReadyTracker.java:107) at org.apache.tez.runtime.api.impl.TezProcessorContextImpl.waitForAllInputsReady(TezProcessorContextImpl.java:141) at org.apache.tez.runtime.api.impl.TezProcessorContextImpl.waitForAllInputsReady(TezProcessorContextImpl.java:136) at org.apache.hadoop.hive.ql.exec.tez.DynamicValueRegistryTez.init(DynamicValueRegistryTez.java:111) at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:313) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:253) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:237) at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374) at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73) at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962) at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61) at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37) at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) at org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:110) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 2018-01-05T11:29:11,762 INFO [IPC Server handler 3 on 49818] impl.ContainerRunnerImpl: Issuing killFragment for completed query QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=42} attempt_1515180518813_0001_42_00_000000_0 2018-01-05T11:29:11,762 DEBUG [IPC Server handler 3 on 49818] impl.TaskExecutorService: Removing attempt_1515180518813_0001_42_00_000000_0 from preemptionQueue 2018-01-05T11:29:11,762 INFO [IPC Server handler 3 on 49818] impl.TaskRunnerCallable: Kill task requested for id=attempt_1515180518813_0001_42_00_000000_0, taskRunnerSetup=true 2018-01-05T11:29:11,762 INFO [IPC Server handler 3 on 49818] impl.TaskRunnerCallable: Issuing kill to task attempt_1515180518813_0001_42_00_000000_0 2018-01-05T11:29:11,762 INFO [IPC Server handler 3 on 49818] task.TezTaskRunner2: Attempting to abort attempt_1515180518813_0001_42_00_000000_0 due to an invocation of killTask 2018-01-05T11:29:11,763 INFO [IPC Server handler 3 on 49818] tez.TezProcessor: Received abort 2018-01-05T11:29:11,763 INFO [IPC Server handler 3 on 49818] tez.TezProcessor: Forwarding abort to RecordProcessor 2018-01-05T11:29:11,763 INFO [IPC Server handler 3 on 49818] tez.MapRecordProcessor: Forwarding abort to mapOp: {} MAP 2018-01-05T11:29:11,763 INFO [IPC Server handler 3 on 49818] vector.VectorMapOperator: Received abort in operator: MAP 2018-01-05T11:29:11,763 INFO [IPC Server handler 3 on 49818] impl.TaskRunnerCallable: Kill request for task attempt_1515180518813_0001_42_00_000000_0 completed. Informing AM 2018-01-05T11:29:11,762 ERROR [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: Vertex failed, vertexName=Map 8, vertexId=vertex_1515180518813_0001_42_05, diagnostics=[Vertex vertex_1515180518813_0001_42_05 [Map 8] killed/failed due to:ROOT_INPUT_INIT_FAILURE, Vertex Input: b initializer failed, vertex=vertex_1515180518813_0001_42_05 [Map 8], java.lang.RuntimeException: ORC split generation failed with exception: java.lang.IllegalStateException: Failed to retrieve dynamic value for RS_12_a_key_min at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:1784) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getSplits(OrcInputFormat.java:1872) at org.apache.hadoop.hive.ql.io.HiveInputFormat.addSplitsForGroup(HiveInputFormat.java:499) at org.apache.hadoop.hive.ql.io.HiveInputFormat.getSplits(HiveInputFormat.java:684) at org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:196) at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:278) at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:269) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962) at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:269) at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:253) at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108) at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41) at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.util.concurrent.ExecutionException: java.lang.IllegalStateException: Failed to retrieve dynamic value for RS_12_a_key_min at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:1778) ... 17 more Caused by: java.lang.IllegalStateException: Failed to retrieve dynamic value for RS_12_a_key_min at org.apache.hadoop.hive.ql.plan.DynamicValue.getValue(DynamicValue.java:142) at org.apache.hadoop.hive.ql.plan.DynamicValue.getJavaValue(DynamicValue.java:97) at org.apache.hadoop.hive.ql.plan.DynamicValue.getLiteral(DynamicValue.java:93) at org.apache.hadoop.hive.ql.io.sarg.SearchArgumentImpl$PredicateLeafImpl.getLiteralList(SearchArgumentImpl.java:120) at org.apache.orc.impl.RecordReaderImpl.evaluatePredicateMinMax(RecordReaderImpl.java:553) at org.apache.orc.impl.RecordReaderImpl.evaluatePredicateRange(RecordReaderImpl.java:463) at org.apache.orc.impl.RecordReaderImpl.evaluatePredicate(RecordReaderImpl.java:440) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.isStripeSatisfyPredicate(OrcInputFormat.java:2163) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.pickStripesInternal(OrcInputFormat.java:2140) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.pickStripes(OrcInputFormat.java:2131) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.access$3000(OrcInputFormat.java:157) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.callInternal(OrcInputFormat.java:1476) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.access$2700(OrcInputFormat.java:1261) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator$1.run(OrcInputFormat.java:1445) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator$1.run(OrcInputFormat.java:1442) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.call(OrcInputFormat.java:1442) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.call(OrcInputFormat.java:1261) at java.util.concurrent.FutureTask.run(FutureTask.java:266) ... 3 more Caused by: java.lang.IllegalStateException: Value does not exist in registry: RS_12_a_key_min at org.apache.hadoop.hive.ql.exec.tez.DynamicValueRegistryTez.getValue(DynamicValueRegistryTez.java:76) at org.apache.hadoop.hive.ql.plan.DynamicValue.getValue(DynamicValue.java:137) ... 23 more ] 2018-01-05T11:29:11,762 INFO [TezTR-518813_1_42_2_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_42_02_000000_0, stopRequested=true 2018-01-05T11:29:11,763 ERROR [TezTR-518813_1_42_1_0_0] tez.TezProcessor: java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) at org.apache.tez.runtime.InputReadyTracker$InputReadyMonitor.awaitCondition(InputReadyTracker.java:147) at org.apache.tez.runtime.InputReadyTracker.waitForAllInputsReady(InputReadyTracker.java:107) at org.apache.tez.runtime.api.impl.TezProcessorContextImpl.waitForAllInputsReady(TezProcessorContextImpl.java:141) at org.apache.tez.runtime.api.impl.TezProcessorContextImpl.waitForAllInputsReady(TezProcessorContextImpl.java:136) at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.init(ReduceRecordProcessor.java:122) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:253) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:237) at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374) at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73) at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962) at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61) at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37) at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) at org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:110) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 2018-01-05T11:29:11,763 ERROR [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: Vertex killed, vertexName=Reducer 4, vertexId=vertex_1515180518813_0001_42_07, diagnostics=[Vertex received Kill in INITED state., Vertex vertex_1515180518813_0001_42_07 [Reducer 4] killed/failed due to:OTHER_VERTEX_FAILURE] 2018-01-05T11:29:11,763 DEBUG [IPC Server handler 3 on 49818] impl.AMReporter: Query complete received for QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=42} 2018-01-05T11:29:11,763 ERROR [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: Vertex killed, vertexName=Reducer 3, vertexId=vertex_1515180518813_0001_42_06, diagnostics=[Vertex received Kill in INITED state., Vertex vertex_1515180518813_0001_42_06 [Reducer 3] killed/failed due to:OTHER_VERTEX_FAILURE] 2018-01-05T11:29:11,763 INFO [TezTR-518813_1_42_1_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_42_01_000000_0, stopRequested=true 2018-01-05T11:29:11,763 DEBUG [IPC Server handler 3 on 49818] impl.AMReporter: Removed following AMs due to query complete: 2018-01-05T11:29:11,763 ERROR [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: Vertex killed, vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_42_03, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1, Vertex vertex_1515180518813_0001_42_03 [Reducer 2] killed/failed due to:OTHER_VERTEX_FAILURE] 2018-01-05T11:29:11,763 DEBUG [IPC Server handler 3 on 49818] impl.AMReporter: AMInfo: 10.22.8.182:49824, taskCount=0, queryIdentifier=QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=42} 2018-01-05T11:29:11,763 ERROR [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: Vertex killed, vertexName=Reducer 5, vertexId=vertex_1515180518813_0001_42_04, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1, Vertex vertex_1515180518813_0001_42_04 [Reducer 5] killed/failed due to:OTHER_VERTEX_FAILURE] 2018-01-05T11:29:11,763 INFO [TezTR-518813_1_42_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_42_01_000000_0: Counters: 11 [[org.apache.tez.common.counters.TaskCounter NUM_SHUFFLED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=0, SHUFFLE_BYTES=0, SHUFFLE_BYTES_DECOMPRESSED=0, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=0, FIRST_EVENT_RECEIVED=0, LAST_EVENT_RECEIVED=0]] 2018-01-05T11:29:11,763 ERROR [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: Vertex killed, vertexName=Map 1, vertexId=vertex_1515180518813_0001_42_02, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1, Vertex vertex_1515180518813_0001_42_02 [Map 1] killed/failed due to:OTHER_VERTEX_FAILURE] 2018-01-05T11:29:11,763 INFO [TezTR-518813_1_42_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:11,763 ERROR [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: Vertex killed, vertexName=Reducer 7, vertexId=vertex_1515180518813_0001_42_01, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1, Vertex vertex_1515180518813_0001_42_01 [Reducer 7] killed/failed due to:OTHER_VERTEX_FAILURE] 2018-01-05T11:29:11,763 INFO [TezTR-518813_1_42_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_42_02_000000_0: Counters: 36 [[File System Counters FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=0, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, INPUT_RECORDS_PROCESSED=0, INPUT_SPLIT_LENGTH_BYTES=1149, OUTPUT_RECORDS=0, OUTPUT_BYTES=0, OUTPUT_BYTES_WITH_OVERHEAD=0, OUTPUT_BYTES_PHYSICAL=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=0, SHUFFLE_BYTES=0, SHUFFLE_BYTES_DECOMPRESSED=0, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=0, SHUFFLE_PHASE_TIME=0, FIRST_EVENT_RECEIVED=0, LAST_EVENT_RECEIVED=0][org.apache.hadoop.hive.llap.counters.LlapIOCounters CACHE_HIT_BYTES=523, CACHE_MISS_BYTES=0, DECODE_TIME_NS=210116, HDFS_TIME_NS=114111, METADATA_CACHE_HIT=2, NUM_DECODED_BATCHES=1, NUM_VECTOR_BATCHES=1, ROWS_EMITTED=120, SELECTED_ROWGROUPS=1, TOTAL_IO_TIME_NS=13896309]] 2018-01-05T11:29:11,763 INFO [TezTR-518813_1_42_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:11,763 INFO [TezTR-518813_1_42_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Ignoring interrupt while waiting for the router thread to die 2018-01-05T11:29:11,763 INFO [TezTR-518813_1_42_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Ignoring interrupt while waiting for the router thread to die 2018-01-05T11:29:11,763 ERROR [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: Vertex killed, vertexName=Map 6, vertexId=vertex_1515180518813_0001_42_00, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1, Vertex vertex_1515180518813_0001_42_00 [Map 6] killed/failed due to:OTHER_VERTEX_FAILURE] 2018-01-05T11:29:11,763 ERROR [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:7 2018-01-05T11:29:11,763 DEBUG [AMReporter 0] endpoint.LlapPluginSecurityInfo: Trying to get TokenInfo for interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol 2018-01-05T11:29:11,763 INFO [TezTR-518813_1_42_2_0_0] LlapIoImpl: Maximum queue length observed 0 2018-01-05T11:29:11,763 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,763 INFO [TezTR-518813_1_42_1_0_0] impl.ShuffleInputEventHandlerImpl: Map 6: numDmeEventsSeen=0, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:29:11,763 INFO [TezTR-518813_1_42_2_0_0] LlapIoImpl: Llap counters: Fragment counters for [hw13500.local/10.22.8.182, warehouse.tab, file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab/ds=2008-04-08/000001_0 ([-5122461581150239451, 1515180546000, 767]), 0,1]: [ NUM_VECTOR_BATCHES=1, NUM_DECODED_BATCHES=1, SELECTED_ROWGROUPS=1, NUM_ERRORS=0, ROWS_EMITTED=120, METADATA_CACHE_HIT=2, METADATA_CACHE_MISS=0, CACHE_HIT_BYTES=523, CACHE_MISS_BYTES=0, ALLOCATED_BYTES=0, ALLOCATED_USED_BYTES=0, TOTAL_IO_TIME_NS=13896309, DECODE_TIME_NS=210116, HDFS_TIME_NS=114111, CONSUMER_TIME_NS=0 ] 2018-01-05T11:29:11,763 DEBUG [TezTR-518813_1_42_2_0_0] encoded.OrcEncodedDataReader: Encoded reader is being stopped 2018-01-05T11:29:11,763 INFO [TezTR-518813_1_42_1_0_0] impl.ShuffleManager: Map_6: Thread interrupted. Need to cleanup the local dirs 2018-01-05T11:29:11,763 INFO [TezTR-518813_1_42_1_0_0] impl.ShuffleManager: Shutting down pending fetchers on sourceMap_6: 0 2018-01-05T11:29:11,763 INFO [TezTR-518813_1_42_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Closed input for vertex=Reducer 7, sourceVertex=Map 6, interruptedStatus=true 2018-01-05T11:29:11,763 INFO [TezTR-518813_1_42_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Closed input for vertex=Map 1, sourceVertex=a, interruptedStatus=true 2018-01-05T11:29:11,763 INFO [TezTR-518813_1_42_2_0_0] impl.ShuffleInputEventHandlerImpl: Reducer 7: numDmeEventsSeen=0, numDmeEventsSeenWithNoData=0, numObsoletionEventsSeen=0, updateOnClose 2018-01-05T11:29:11,764 INFO [TezTR-518813_1_42_2_0_0] impl.ShuffleManager: Reducer_7: Thread interrupted. Need to cleanup the local dirs 2018-01-05T11:29:11,764 INFO [TezTR-518813_1_42_2_0_0] impl.ShuffleManager: Shutting down pending fetchers on sourceReducer_7: 0 2018-01-05T11:29:11,764 INFO [TezTR-518813_1_42_0_0_0] impl.PipelinedSorter: Reducer 2: Spilling to /Users/djaiswal/hive/hive/itests/qtest/target/llap_MiniLlapCluster/localDir/usercache/djaiswal/appcache/application_1515180518813_0001/42/output/attempt_1515180518813_0001_42_00_000000_0_10324_0/file.out 2018-01-05T11:29:11,764 WARN [TezTR-518813_1_42_1_0_0] output.UnorderedKVOutput: Map 1: Attempting to close output Map 1 of type UnorderedKVOutput before it was started. Generating empty events 2018-01-05T11:29:11,764 INFO [TezTR-518813_1_42_0_0_0] impl.PipelinedSorter: Reducer 2: Thread interrupted, cleaned up stale data, sorter threads shutdown=true, terminated=true 2018-01-05T11:29:11,764 INFO [TezTR-518813_1_42_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Closed input for vertex=Map 1, sourceVertex=Reducer 7, interruptedStatus=true 2018-01-05T11:29:11,764 INFO [TezTR-518813_1_42_2_0_0] impl.PipelinedSorter: Reducer 2: Thread interrupted, cleaned up stale data, sorter threads shutdown=true, terminated=true 2018-01-05T11:29:11,764 INFO [TezTR-518813_1_42_1_0_0] shuffle.ShuffleUtils: Setting all 1 partitions as empty for non-started output 2018-01-05T11:29:11,764 INFO [TezTR-518813_1_42_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Closed input for vertex=Reducer 7, sourceVertex=Map 1, interruptedStatus=true 2018-01-05T11:29:11,764 DEBUG [TezTR-518813_1_42_1_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:11,764 INFO [TezTR-518813_1_42_1_0_0] runtime.LogicalIOProcessorRuntimeTask: Closed processor for vertex=Reducer 7, index=1, interruptedStatus=true 2018-01-05T11:29:11,764 INFO [TezTR-518813_1_42_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Joining on EventRouter 2018-01-05T11:29:11,764 DEBUG [TezTR-518813_1_42_1_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_42_1_0_0 thread id: 2945 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:11,764 INFO [Socket Reader #1 for port 49824] ipc.Server: Auth successful for application_1515180518813_0001 (auth:SIMPLE) 2018-01-05T11:29:11,764 WARN [TezTR-518813_1_42_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Ignoring exception when closing output Reducer 2(cleanup). Exception class=java.lang.NullPointerException, message=null 2018-01-05T11:29:11,764 INFO [TezTR-518813_1_42_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Closed input for vertex=Map 1, sourceVertex=Reducer 2, interruptedStatus=true 2018-01-05T11:29:11,764 INFO [Task-Executor-2] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_42_01_000000_0 : TaskRunner2Result{endReason=KILL_REQUESTED, containerShutdownRequested=false, failureType=null, error=null} 2018-01-05T11:29:11,764 INFO [Task-Executor-2] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_42_01_000000_0 2018-01-05T11:29:11,764 INFO [TezTR-518813_1_42_0_0_0] task.TaskRunner2Callable: Cleaning up task attempt_1515180518813_0001_42_00_000000_0, stopRequested=true 2018-01-05T11:29:11,764 INFO [Task-Executor-2] task.TezTaskRunner2: Time taken to interrupt task=3 2018-01-05T11:29:11,764 DEBUG [TezTR-518813_1_42_2_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:11,764 INFO [TezTR-518813_1_42_2_0_0] runtime.LogicalIOProcessorRuntimeTask: Closed processor for vertex=Map 1, index=2, interruptedStatus=true 2018-01-05T11:29:11,764 INFO [Task-Executor-2] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000106=64 2018-01-05T11:29:11,764 DEBUG [Task-Executor-2] impl.QueryFragmentInfo: Cannot finish due to source: Map 6 2018-01-05T11:29:11,764 DEBUG [Task-Executor-2] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_42_01_000000_0: false 2018-01-05T11:29:11,764 INFO [TezTR-518813_1_42_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Final Counters for attempt_1515180518813_0001_42_00_000000_0: Counters: 27 [[File System Counters FILE_BYTES_READ=0, FILE_BYTES_WRITTEN=0, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=0, INPUT_RECORDS_PROCESSED=1, INPUT_SPLIT_LENGTH_BYTES=216, OUTPUT_RECORDS=26, OUTPUT_LARGE_RECORDS=0, OUTPUT_BYTES=278, OUTPUT_BYTES_WITH_OVERHEAD=88, OUTPUT_BYTES_PHYSICAL=116, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_CHUNK_COUNT=0][HIVE DESERIALIZE_ERRORS=0, RECORDS_IN_Map_6=25, RECORDS_OUT_INTERMEDIATE_Map_6=26][org.apache.hadoop.hive.llap.counters.LlapIOCounters CACHE_HIT_BYTES=216, CONSUMER_TIME_NS=260097, DECODE_TIME_NS=7813156, NUM_DECODED_BATCHES=1, NUM_VECTOR_BATCHES=1, ROWS_EMITTED=25, TOTAL_IO_TIME_NS=8967856]] 2018-01-05T11:29:11,764 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_42_01_000000_0 2018-01-05T11:29:11,764 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_42_01_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=2, preemptionQueueSize=0 2018-01-05T11:29:11,764 DEBUG [TezTR-518813_1_42_0_0_0] tez.TezProcessor: shutDownProgressTaskService() called! 2018-01-05T11:29:11,764 DEBUG [TezTR-518813_1_42_2_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_42_2_0_0 thread id: 2951 scheme: file bytesRead: 0 bytesWritten: 0 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:11,764 INFO [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Killed task attempt_1515180518813_0001_42_01_000000_0 2018-01-05T11:29:11,764 INFO [TezTR-518813_1_42_0_0_0] runtime.LogicalIOProcessorRuntimeTask: Closed processor for vertex=Map 6, index=0, interruptedStatus=true 2018-01-05T11:29:11,764 INFO [Task-Executor-3] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_42_02_000000_0 : TaskRunner2Result{endReason=KILL_REQUESTED, containerShutdownRequested=false, failureType=null, error=null} 2018-01-05T11:29:11,765 INFO [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Time to die for task 3 2018-01-05T11:29:11,765 DEBUG [TezTR-518813_1_42_0_0_0] impl.StatsRecordingThreadPool: Updated stats: instance: TaskRunner2Callable thread name: TezTR-518813_1_42_0_0_0 thread id: 2938 scheme: file bytesRead: 0 bytesWritten: 124 readOps: 0 largeReadOps: 0 writeOps: 0 2018-01-05T11:29:11,765 INFO [ExecutionCompletionThread #0] impl.QueryTracker: Ignoring fragmentComplete message for unknown query: QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=42} 2018-01-05T11:29:11,765 INFO [Task-Executor-3] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_42_02_000000_0 2018-01-05T11:29:11,765 INFO [Task-Executor-1] task.TezTaskRunner2: TaskRunnerResult for attempt_1515180518813_0001_42_00_000000_0 : TaskRunner2Result{endReason=KILL_REQUESTED, containerShutdownRequested=false, failureType=null, error=null} 2018-01-05T11:29:11,765 INFO [Task-Executor-3] task.TezTaskRunner2: Time taken to interrupt task=7 2018-01-05T11:29:11,765 INFO [Task-Executor-1] impl.LlapTaskReporter: Unregistered counters for fragment: 1515180518813_0001_42_00_000000_0 2018-01-05T11:29:11,765 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000106, QueryId=djaiswal_20180105112911_d60e4252-4a13-4247-a28a-f07830bbf022-42, DagId=42, VertexName=Reducer 7, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-2, Succeeded=true, StartTime=1515180551700, EndTime=1515180551765 2018-01-05T11:29:11,765 INFO [Task-Executor-3] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000107=62 2018-01-05T11:29:11,765 WARN [TaskHeartbeatThread] impl.LlapTaskReporter: Exiting TaskReporter thread with pending queue size=2 2018-01-05T11:29:11,765 INFO [Task-Executor-1] task.TezTaskRunner2: Time taken to interrupt task=3 2018-01-05T11:29:11,765 DEBUG [Task-Executor-3] impl.QueryFragmentInfo: Cannot finish due to source: Reducer 7 2018-01-05T11:29:11,765 DEBUG [Task-Executor-3] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_42_02_000000_0: false 2018-01-05T11:29:11,765 INFO [Task-Executor-1] impl.TaskRunnerCallable: ExecutionTime for Container: container_111101111_0001_00_000105=66 2018-01-05T11:29:11,765 DEBUG [Task-Executor-1] impl.TaskRunnerCallable: canFinish post completion: attempt_1515180518813_0001_42_00_000000_0: true 2018-01-05T11:29:11,765 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_42_02_000000_0 2018-01-05T11:29:11,765 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_42_02_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=3, preemptionQueueSize=0 2018-01-05T11:29:11,765 INFO [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Killed task attempt_1515180518813_0001_42_02_000000_0 2018-01-05T11:29:11,765 INFO [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Time to die for task 7 2018-01-05T11:29:11,765 INFO [ExecutionCompletionThread #0] impl.QueryTracker: Ignoring fragmentComplete message for unknown query: QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=42} 2018-01-05T11:29:11,765 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000107, QueryId=djaiswal_20180105112911_d60e4252-4a13-4247-a28a-f07830bbf022-42, DagId=42, VertexName=Map 1, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-3, Succeeded=true, StartTime=1515180551702, EndTime=1515180551765 2018-01-05T11:29:11,765 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Received successful completion for: attempt_1515180518813_0001_42_00_000000_0 2018-01-05T11:29:11,765 DEBUG [ExecutionCompletionThread #0] impl.TaskExecutorService: Task attempt_1515180518813_0001_42_00_000000_0 complete. WaitQueueSize=0, numSlotsAvailable=4, preemptionQueueSize=0 2018-01-05T11:29:11,765 INFO [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Killed task attempt_1515180518813_0001_42_00_000000_0 2018-01-05T11:29:11,765 INFO [ExecutionCompletionThread #0] impl.TaskRunnerCallable: Time to die for task 2 2018-01-05T11:29:11,765 INFO [ExecutionCompletionThread #0] impl.QueryTracker: Ignoring fragmentComplete message for unknown query: QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=42} 2018-01-05T11:29:11,765 INFO [ExecutionCompletionThread #0] daemon.HistoryLogger: Event=FRAGMENT_END, HostName=10.22.8.182, ApplicationId=application_1515180518813_0001, ContainerId=container_111101111_0001_00_000105, QueryId=djaiswal_20180105112911_d60e4252-4a13-4247-a28a-f07830bbf022-42, DagId=42, VertexName=Map 6, TaskId=0, TaskAttemptId=0, ThreadName=Task-Executor-1, Succeeded=true, StartTime=1515180551698, EndTime=1515180551765 2018-01-05T11:29:11,765 ERROR [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 8, vertexId=vertex_1515180518813_0001_42_05, diagnostics=[Vertex vertex_1515180518813_0001_42_05 [Map 8] killed/failed due to:ROOT_INPUT_INIT_FAILURE, Vertex Input: b initializer failed, vertex=vertex_1515180518813_0001_42_05 [Map 8], java.lang.RuntimeException: ORC split generation failed with exception: java.lang.IllegalStateException: Failed to retrieve dynamic value for RS_12_a_key_min at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:1784) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getSplits(OrcInputFormat.java:1872) at org.apache.hadoop.hive.ql.io.HiveInputFormat.addSplitsForGroup(HiveInputFormat.java:499) at org.apache.hadoop.hive.ql.io.HiveInputFormat.getSplits(HiveInputFormat.java:684) at org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:196) at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:278) at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:269) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962) at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:269) at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:253) at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108) at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41) at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.util.concurrent.ExecutionException: java.lang.IllegalStateException: Failed to retrieve dynamic value for RS_12_a_key_min at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:1778) ... 17 more Caused by: java.lang.IllegalStateException: Failed to retrieve dynamic value for RS_12_a_key_min at org.apache.hadoop.hive.ql.plan.DynamicValue.getValue(DynamicValue.java:142) at org.apache.hadoop.hive.ql.plan.DynamicValue.getJavaValue(DynamicValue.java:97) at org.apache.hadoop.hive.ql.plan.DynamicValue.getLiteral(DynamicValue.java:93) at org.apache.hadoop.hive.ql.io.sarg.SearchArgumentImpl$PredicateLeafImpl.getLiteralList(SearchArgumentImpl.java:120) at org.apache.orc.impl.RecordReaderImpl.evaluatePredicateMinMax(RecordReaderImpl.java:553) at org.apache.orc.impl.RecordReaderImpl.evaluatePredicateRange(RecordReaderImpl.java:463) at org.apache.orc.impl.RecordReaderImpl.evaluatePredicate(RecordReaderImpl.java:440) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.isStripeSatisfyPredicate(OrcInputFormat.java:2163) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.pickStripesInternal(OrcInputFormat.java:2140) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.pickStripes(OrcInputFormat.java:2131) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.access$3000(OrcInputFormat.java:157) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.callInternal(OrcInputFormat.java:1476) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.access$2700(OrcInputFormat.java:1261) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator$1.run(OrcInputFormat.java:1445) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator$1.run(OrcInputFormat.java:1442) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.call(OrcInputFormat.java:1442) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.call(OrcInputFormat.java:1261) at java.util.concurrent.FutureTask.run(FutureTask.java:266) ... 3 more Caused by: java.lang.IllegalStateException: Value does not exist in registry: RS_12_a_key_min at org.apache.hadoop.hive.ql.exec.tez.DynamicValueRegistryTez.getValue(DynamicValueRegistryTez.java:76) at org.apache.hadoop.hive.ql.plan.DynamicValue.getValue(DynamicValue.java:137) ... 23 more ]Vertex killed, vertexName=Reducer 4, vertexId=vertex_1515180518813_0001_42_07, diagnostics=[Vertex received Kill in INITED state., Vertex vertex_1515180518813_0001_42_07 [Reducer 4] killed/failed due to:OTHER_VERTEX_FAILURE]Vertex killed, vertexName=Reducer 3, vertexId=vertex_1515180518813_0001_42_06, diagnostics=[Vertex received Kill in INITED state., Vertex vertex_1515180518813_0001_42_06 [Reducer 3] killed/failed due to:OTHER_VERTEX_FAILURE]Vertex killed, vertexName=Reducer 2, vertexId=vertex_1515180518813_0001_42_03, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1, Vertex vertex_1515180518813_0001_42_03 [Reducer 2] killed/failed due to:OTHER_VERTEX_FAILURE]Vertex killed, vertexName=Reducer 5, vertexId=vertex_1515180518813_0001_42_04, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1, Vertex vertex_1515180518813_0001_42_04 [Reducer 5] killed/failed due to:OTHER_VERTEX_FAILURE]Vertex killed, vertexName=Map 1, vertexId=vertex_1515180518813_0001_42_02, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1, Vertex vertex_1515180518813_0001_42_02 [Map 1] killed/failed due to:OTHER_VERTEX_FAILURE]Vertex killed, vertexName=Reducer 7, vertexId=vertex_1515180518813_0001_42_01, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1, Vertex vertex_1515180518813_0001_42_01 [Reducer 7] killed/failed due to:OTHER_VERTEX_FAILURE]Vertex killed, vertexName=Map 6, vertexId=vertex_1515180518813_0001_42_00, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1, Vertex vertex_1515180518813_0001_42_00 [Map 6] killed/failed due to:OTHER_VERTEX_FAILURE]DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:7 2018-01-05T11:29:11,765 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query select count(*) from tab a join tab_part b on a.key = b.key join src1 c on a.value = c.value 2018-01-05T11:29:11,765 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,765 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:11,765 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {} 2018-01-05T11:29:11,765 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112911_d60e4252-4a13-4247-a28a-f07830bbf022); Time taken: 0.379 seconds 2018-01-05T11:29:11,765 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:11,765 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:11,765 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query select count(*) from tab a join tab_part b on a.key = b.key join src1 c on a.value = c.value 2018-01-05T11:29:11,766 INFO [AMReporter 1] impl.AMReporter: Sent taskKilled for attempt_1515180518813_0001_42_01_000000_0 2018-01-05T11:29:11,766 INFO [AMReporter 2] impl.AMReporter: Sent taskKilled for attempt_1515180518813_0001_42_00_000000_0 2018-01-05T11:29:11,766 INFO [AMReporter 0] impl.AMReporter: Sent taskKilled for attempt_1515180518813_0001_42_02_000000_0 2018-01-05T11:29:11,766 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: close called for operator LIST_SINK[69] 2018-01-05T11:29:11,766 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: Closing operator LIST_SINK[69] 2018-01-05T11:29:11,766 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.ListSinkOperator: 69 Close done 2018-01-05T11:29:11,766 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting result dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/-mr-10001 2018-01-05T11:29:11,767 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1/-mr-10001/.hive-staging_hive_2018-01-05_11-29-11_151_1086992364664144340-1 2018-01-05T11:29:11,767 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Context: Deleting scratch dir: file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/localscratchdir/dfdd8629-6c16-4f61-9218-1f1e0156f79c/hive_2018-01-05_11-29-11_151_1086992364664144340-1 2018-01-05T11:29:11,768 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:11,768 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:11,768 ERROR [main] QTestUtil: Client execution failed with error code = 2 running " select count(*) from tab a join tab_part b on a.key = b.key join src1 c on a.value = c.value" fname=mergejoin.q See ./ql/target/tmp/log/hive.log or ./itests/qtest/target/tmp/log/hive.log, or check ./ql/target/surefire-reports or ./itests/qtest/target/surefire-reports/ for specific test cases logs. 2018-01-05T11:29:11,768 INFO [main] control.CoreCliDriver: Done querymergejoin.q. succeeded=false, skipped=false. ElapsedTime(ms)=9173 2018-01-05T11:29:11,769 INFO [ProcessThread(sid:0 cport:-1):] server.PrepRequestProcessor: Processed session termination for sessionid: 0x160c7cbc1600005 2018-01-05T11:29:11,769 INFO [main] zookeeper.ZooKeeper: Session: 0x160c7cbc1600005 closed 2018-01-05T11:29:11,774 INFO [main] control.CoreCliDriver: PerTestTearDown done. ElapsedTime(ms)=5 2018-01-05T11:29:11,780 INFO [main] metastore.HiveMetaStoreClient: Mestastore configuration metastore.filter.hook changed from org.apache.hadoop.hive.ql.security.authorization.plugin.AuthorizationMetaStoreFilterHook to org.apache.hadoop.hive.metastore.DefaultMetaStoreFilterHookImpl 2018-01-05T11:29:11,780 DEBUG [main] metadata.Hive: Creating new db. db = org.apache.hadoop.hive.ql.metadata.Hive@e316d60, needsRefresh = false, db.isCurrentUserOwner = true 2018-01-05T11:29:11,780 DEBUG [main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:11,780 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,780 INFO [main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:29:11,780 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:29:11,780 INFO [main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:11,780 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:11,780 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,780 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,780 INFO [main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:11,780 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:11,780 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,780 DEBUG [main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:11,780 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,780 INFO [main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:11,780 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:11,780 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,781 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,781 INFO [main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.VerifyingObjectStore 2018-01-05T11:29:11,781 WARN [main] metastore.VerifyingObjectStore: VerifyingObjectStore is being used - test run 2018-01-05T11:29:11,781 WARN [main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2018-01-05T11:29:11,781 INFO [main] metastore.ObjectStore: ObjectStore, initialize called 2018-01-05T11:29:11,781 DEBUG [main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,781 DEBUG [main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,781 DEBUG [main] pool.PoolBase: HikariPool-2 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@2136347897 (XID = 169), (SESSIONID = 3), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,794 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,795 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,795 INFO [main] metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 2018-01-05T11:29:11,795 INFO [main] metastore.ObjectStore: Initialized ObjectStore 2018-01-05T11:29:11,795 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,795 INFO [main] metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=djaiswal (auth:SIMPLE) retries=1 delay=1 lifetime=0 2018-01-05T11:29:11,795 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,795 INFO [main] metastore.HiveMetaStore: 0: get_all_databases 2018-01-05T11:29:11,795 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_all_databases 2018-01-05T11:29:11,796 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,796 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,796 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,796 INFO [main] metastore.HiveMetaStore: 0: get_tables: db=default pat=.* 2018-01-05T11:29:11,796 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_tables: db=default pat=.* 2018-01-05T11:29:11,798 WARN [main] metastore.ObjectStore: Falling back to ORM path due to direct SQL failure (this is not an error): java.lang.String cannot be cast to [Ljava.lang.Object; at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.getTables(MetaStoreDirectSql.java:411) at org.apache.hadoop.hive.metastore.ObjectStore$2.getSqlResult(ObjectStore.java:1313) 2018-01-05T11:29:11,798 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,801 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,802 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,802 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,802 INFO [main] metastore.HiveMetaStore: 0: get_indexes : db=default tbl=alltypesorc 2018-01-05T11:29:11,802 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_indexes : db=default tbl=alltypesorc 2018-01-05T11:29:11,806 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,806 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,806 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,806 INFO [main] metastore.HiveMetaStore: 0: get_indexes : db=default tbl=alltypesparquet 2018-01-05T11:29:11,806 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_indexes : db=default tbl=alltypesparquet 2018-01-05T11:29:11,806 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,806 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,806 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,806 INFO [main] metastore.HiveMetaStore: 0: get_indexes : db=default tbl=cbo_t1 2018-01-05T11:29:11,807 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_indexes : db=default tbl=cbo_t1 2018-01-05T11:29:11,807 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,807 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,807 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,807 INFO [main] metastore.HiveMetaStore: 0: get_indexes : db=default tbl=cbo_t2 2018-01-05T11:29:11,807 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_indexes : db=default tbl=cbo_t2 2018-01-05T11:29:11,807 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,807 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,807 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,807 INFO [main] metastore.HiveMetaStore: 0: get_indexes : db=default tbl=cbo_t3 2018-01-05T11:29:11,807 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_indexes : db=default tbl=cbo_t3 2018-01-05T11:29:11,807 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,807 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,807 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,807 INFO [main] metastore.HiveMetaStore: 0: get_indexes : db=default tbl=lineitem 2018-01-05T11:29:11,807 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_indexes : db=default tbl=lineitem 2018-01-05T11:29:11,808 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,808 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,808 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,808 INFO [main] metastore.HiveMetaStore: 0: get_indexes : db=default tbl=part 2018-01-05T11:29:11,808 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_indexes : db=default tbl=part 2018-01-05T11:29:11,808 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,808 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,808 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,808 INFO [main] metastore.HiveMetaStore: 0: get_indexes : db=default tbl=src 2018-01-05T11:29:11,808 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_indexes : db=default tbl=src 2018-01-05T11:29:11,808 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,808 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,808 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,808 INFO [main] metastore.HiveMetaStore: 0: get_indexes : db=default tbl=src1 2018-01-05T11:29:11,808 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_indexes : db=default tbl=src1 2018-01-05T11:29:11,808 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,808 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,808 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,808 INFO [main] metastore.HiveMetaStore: 0: get_indexes : db=default tbl=src_cbo 2018-01-05T11:29:11,808 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_indexes : db=default tbl=src_cbo 2018-01-05T11:29:11,809 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,809 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,809 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,809 INFO [main] metastore.HiveMetaStore: 0: get_indexes : db=default tbl=src_json 2018-01-05T11:29:11,809 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_indexes : db=default tbl=src_json 2018-01-05T11:29:11,809 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,809 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,809 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,809 INFO [main] metastore.HiveMetaStore: 0: get_indexes : db=default tbl=src_sequencefile 2018-01-05T11:29:11,809 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_indexes : db=default tbl=src_sequencefile 2018-01-05T11:29:11,809 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,809 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,809 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,809 INFO [main] metastore.HiveMetaStore: 0: get_indexes : db=default tbl=src_thrift 2018-01-05T11:29:11,809 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_indexes : db=default tbl=src_thrift 2018-01-05T11:29:11,809 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,809 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,809 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,809 INFO [main] metastore.HiveMetaStore: 0: get_indexes : db=default tbl=srcbucket 2018-01-05T11:29:11,809 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_indexes : db=default tbl=srcbucket 2018-01-05T11:29:11,810 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,810 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,810 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,810 INFO [main] metastore.HiveMetaStore: 0: get_indexes : db=default tbl=srcbucket2 2018-01-05T11:29:11,810 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_indexes : db=default tbl=srcbucket2 2018-01-05T11:29:11,810 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,810 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,810 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,810 INFO [main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:11,810 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:11,812 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,813 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,813 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,813 INFO [main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:11,813 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:11,815 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,816 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,816 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,816 INFO [main] metastore.HiveMetaStore: 0: drop_table : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:11,816 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=drop_table : db=default tbl=srcbucket_mapjoin 2018-01-05T11:29:11,821 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 2.754408ms + 0.008744ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc] 2018-01-05T11:29:11,821 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 0.126906ms + 0.047208ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (12) order by "PART_NAME" asc] 2018-01-05T11:29:11,821 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 0.073566ms + 0.042922ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (12) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:29:11,821 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 0.058997ms + 0.015818ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (12) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:11,821 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 0.053645ms + 0.005409ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (39) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:29:11,821 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 0.050524ms + 0.005169ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (39) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:11,821 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 0.04995ms + 0.014988ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (39) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:11,821 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 0.047451ms + 0.004943ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (39) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:11,822 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 0.075735ms + 0.019076ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (25) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:11,822 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 0.057294ms + 0.017295ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (39) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:29:11,877 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 0.581987ms + 0.002941ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc] 2018-01-05T11:29:11,930 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,930 DEBUG [main] utils.FileUtils: deleting file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin 2018-01-05T11:29:11,931 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,931 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,931 INFO [main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:11,931 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:11,958 INFO [LlapSchedulerNodeEnabler] tezplugins.LlapTaskSchedulerService: Attempting to re-enable node: {10.22.8.182:49818, id=fb153ec0-2938-4de8-ad1c-4b4dfba96f4f, canAcceptTask=false, st=0, ac=8, commF=false, disabled=true} 2018-01-05T11:29:11,958 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: ScheduleRun: numPriorityLevels=0. . totalPendingTasks=0. delayedTaskQueueSize=0 2018-01-05T11:29:11,958 DEBUG [LlapScheduler] tezplugins.LlapTaskSchedulerService: GetTotalResources: numInstancesFound=1, totalMem=4096, totalVcores=4 2018-01-05T11:29:11,964 DEBUG [AMReporterQueueDrainer] impl.AMReporter: Removing am 10.22.8.182:49824 with last associated dag QueryIdentifier{appIdentifier='application_1515180518813_0001', dagIdentifier=33} from heartbeat with taskCount=0, amFailed=false, isDone=true 2018-01-05T11:29:11,966 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,966 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,966 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,966 INFO [main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:11,966 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:11,969 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:11,970 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,970 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:11,970 INFO [main] metastore.HiveMetaStore: 0: drop_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:11,970 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=drop_table : db=default tbl=srcbucket_mapjoin_part 2018-01-05T11:29:11,973 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 0.140165ms + 0.00339ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc] 2018-01-05T11:29:11,976 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 2.858243ms + 0.060978ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (13) order by "PART_NAME" asc] 2018-01-05T11:29:11,978 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 1.910469ms + 0.07171ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (13) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:29:11,979 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 1.582121ms + 0.03578ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (13) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:11,981 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 1.528014ms + 0.013731ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (40) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:29:11,982 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 1.4055ms + 0.014334ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (40) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:11,984 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 1.543221ms + 0.041917ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (40) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:11,986 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 1.393888ms + 0.014688ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (40) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:11,987 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 1.602364ms + 0.038133ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (27) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:11,989 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 2.228603ms + 0.046481ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (40) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:29:12,001 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 0.2754ms + 0.005321ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc] 2018-01-05T11:29:12,010 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,010 DEBUG [main] utils.FileUtils: deleting file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket_mapjoin_part 2018-01-05T11:29:12,011 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:12,011 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:12,011 INFO [main] metastore.HiveMetaStore: 0: get_indexes : db=default tbl=srcpart 2018-01-05T11:29:12,011 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_indexes : db=default tbl=srcpart 2018-01-05T11:29:12,012 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,012 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:12,012 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:12,012 INFO [main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab 2018-01-05T11:29:12,012 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab 2018-01-05T11:29:12,014 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,014 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:12,015 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:12,015 INFO [main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab 2018-01-05T11:29:12,015 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab 2018-01-05T11:29:12,017 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,018 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:12,018 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:12,018 INFO [main] metastore.HiveMetaStore: 0: drop_table : db=default tbl=tab 2018-01-05T11:29:12,018 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=drop_table : db=default tbl=tab 2018-01-05T11:29:12,020 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 0.132816ms + 0.003168ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc] 2018-01-05T11:29:12,023 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 2.487351ms + 0.051752ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (15) order by "PART_NAME" asc] 2018-01-05T11:29:12,024 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 1.805918ms + 0.073638ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (15) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:29:12,026 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 1.480435ms + 0.033626ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (15) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:12,028 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 1.493295ms + 0.012512ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (43) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:29:12,029 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 1.424402ms + 0.037163ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (43) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:12,030 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 1.348419ms + 0.032412ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (43) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:12,032 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 1.309028ms + 0.015293ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (43) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:12,034 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 1.818036ms + 0.056013ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (28) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:12,036 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 1.783255ms + 0.038314ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (43) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:29:12,045 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 0.244963ms + 0.002769ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc] 2018-01-05T11:29:12,053 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,054 DEBUG [main] utils.FileUtils: deleting file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab 2018-01-05T11:29:12,054 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:12,054 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:12,054 INFO [main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab_part 2018-01-05T11:29:12,054 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab_part 2018-01-05T11:29:12,057 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,058 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:12,058 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:12,058 INFO [main] metastore.HiveMetaStore: 0: get_table : db=default tbl=tab_part 2018-01-05T11:29:12,058 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=tab_part 2018-01-05T11:29:12,060 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,061 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:12,061 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:12,061 INFO [main] metastore.HiveMetaStore: 0: drop_table : db=default tbl=tab_part 2018-01-05T11:29:12,061 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=drop_table : db=default tbl=tab_part 2018-01-05T11:29:12,063 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 0.12396ms + 0.002913ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc] 2018-01-05T11:29:12,066 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 2.819895ms + 0.054641ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (14) order by "PART_NAME" asc] 2018-01-05T11:29:12,068 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 1.80032ms + 0.076301ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (14) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:29:12,069 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 1.435182ms + 0.035842ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (14) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:12,071 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 1.511232ms + 0.01336ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (41) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:29:12,073 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 2.326259ms + 0.041825ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (41) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:12,075 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 1.413076ms + 0.034201ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (41) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:12,076 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 1.279964ms + 0.012252ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (41) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:12,078 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 1.469081ms + 0.033656ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (26) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:12,079 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 1.510338ms + 0.038284ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (41) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:29:12,084 DEBUG [main] metastore.MetaStoreDirectSql: Direct SQL query in 0.225757ms + 0.002558ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc] 2018-01-05T11:29:12,091 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,091 DEBUG [main] utils.FileUtils: deleting file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/tab_part 2018-01-05T11:29:12,092 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:12,093 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:12,095 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,095 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:12,095 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:12,095 INFO [main] metastore.HiveMetaStore: 0: get_functions: db=default pat=.* 2018-01-05T11:29:12,095 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_functions: db=default pat=.* 2018-01-05T11:29:12,099 DEBUG [main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,099 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:12,099 INFO [main] QTestUtil: Cleanup (/Users/djaiswal/hive/hive/itests/qtest/../..//data/scripts/q_test_cleanup.sql): DROP TABLE IF EXISTS src; DROP TABLE IF EXISTS src1; DROP TABLE IF EXISTS src_json; DROP TABLE IF EXISTS src_sequencefile; DROP TABLE IF EXISTS src_thrift; DROP TABLE IF EXISTS srcbucket; DROP TABLE IF EXISTS srcbucket2; DROP TABLE IF EXISTS srcpart; DROP TABLE IF EXISTS primitives; DROP TABLE IF EXISTS dest1; DROP TABLE IF EXISTS dest2; DROP TABLE IF EXISTS dest3; DROP TABLE IF EXISTS dest4; DROP TABLE IF EXISTS dest4_sequencefile; DROP TABLE IF EXISTS dest_j1; DROP TABLE IF EXISTS dest_g1; DROP TABLE IF EXISTS dest_g2; DROP TABLE IF EXISTS fetchtask_ioexception; DROP TABLE IF EXISTS alltypesorc; DROP TABLE IF EXISTS alltypesparquet; DROP TABLE IF EXISTS cbo_t1; DROP TABLE IF EXISTS cbo_t2; DROP TABLE IF EXISTS cbo_t3; DROP TABLE IF EXISTS src_cbo; DROP TABLE IF EXISTS part; DROP TABLE IF EXISTS lineitem; 2018-01-05T11:29:12,100 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,100 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:12,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:12,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS src 2018-01-05T11:29:12,100 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112912_09375be7-85df-412f-a470-4bd2ad5a411d): DROP TABLE IF EXISTS src 2018-01-05T11:29:12,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS src 2018-01-05T11:29:12,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:12,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,100 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,101 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,101 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,101 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src 2018-01-05T11:29:12,101 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src 2018-01-05T11:29:12,103 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,104 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:12,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,104 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:29:12,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,104 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:12,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTables_(String, String, )=6, dropTable_(String, String, boolean, boolean, boolean, )=237, getTable_(String, String, )=47, getAllDatabases_()=1, flushCache_()=0, listIndexes_(String, String, short, )=9, listRoleNames_()=2, getFunctions_(String, String, )=4} 2018-01-05T11:29:12,104 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112912_09375be7-85df-412f-a470-4bd2ad5a411d); Time taken: 0.004 seconds 2018-01-05T11:29:12,104 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,104 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,104 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112912_09375be7-85df-412f-a470-4bd2ad5a411d): DROP TABLE IF EXISTS src 2018-01-05T11:29:12,105 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,105 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS src 2018-01-05T11:29:12,105 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:29:12,105 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@src 2018-01-05T11:29:12,105 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: default@src 2018-01-05T11:29:12,105 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,105 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,105 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,105 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,105 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:29:12,105 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,105 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src 2018-01-05T11:29:12,105 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src 2018-01-05T11:29:12,108 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,108 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,108 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,108 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src 2018-01-05T11:29:12,108 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src 2018-01-05T11:29:12,110 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,111 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,111 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,111 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: drop_table : db=default tbl=src 2018-01-05T11:29:12,111 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=drop_table : db=default tbl=src 2018-01-05T11:29:12,116 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.126827ms + 0.002441ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc] 2018-01-05T11:29:12,129 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,129 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] utils.FileUtils: deleting file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src 2018-01-05T11:29:12,130 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,130 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.DDLTask: Ignoring request to add WriteEntity(default@src) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@src) Type=TABLE WriteType=DDL_EXCLUSIVE is present 2018-01-05T11:29:12,130 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,130 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,130 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS src 2018-01-05T11:29:12,130 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:29:12,130 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@src 2018-01-05T11:29:12,130 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: default@src 2018-01-05T11:29:12,130 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,130 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,130 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:12,130 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=22, getTable_(String, String, )=3} 2018-01-05T11:29:12,130 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112912_09375be7-85df-412f-a470-4bd2ad5a411d); Time taken: 0.026 seconds 2018-01-05T11:29:12,130 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:12,130 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,130 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,130 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,130 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query DROP TABLE IF EXISTS src 2018-01-05T11:29:12,130 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.03 seconds 2018-01-05T11:29:12,130 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,130 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:12,131 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,131 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:12,131 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,131 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:12,131 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,131 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,131 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,131 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS src1 2018-01-05T11:29:12,131 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112912_6fe8c75c-f654-45ef-b515-ddd5e63dade5): DROP TABLE IF EXISTS src1 2018-01-05T11:29:12,131 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,131 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS src1 2018-01-05T11:29:12,131 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:12,131 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,131 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,131 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,131 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,131 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,131 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,131 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src1 2018-01-05T11:29:12,131 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src1 2018-01-05T11:29:12,134 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,134 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,134 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:12,134 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,134 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:29:12,134 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,134 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:12,134 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=3, flushCache_()=0} 2018-01-05T11:29:12,134 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112912_6fe8c75c-f654-45ef-b515-ddd5e63dade5); Time taken: 0.003 seconds 2018-01-05T11:29:12,134 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,134 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,134 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112912_6fe8c75c-f654-45ef-b515-ddd5e63dade5): DROP TABLE IF EXISTS src1 2018-01-05T11:29:12,135 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,135 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS src1 2018-01-05T11:29:12,135 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:29:12,135 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@src1 2018-01-05T11:29:12,135 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: default@src1 2018-01-05T11:29:12,135 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,135 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,135 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,135 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,135 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:29:12,135 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,136 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src1 2018-01-05T11:29:12,136 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src1 2018-01-05T11:29:12,138 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,139 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,139 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,139 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src1 2018-01-05T11:29:12,139 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src1 2018-01-05T11:29:12,142 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,142 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,142 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,143 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: drop_table : db=default tbl=src1 2018-01-05T11:29:12,143 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=drop_table : db=default tbl=src1 2018-01-05T11:29:12,145 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.143691ms + 0.00361ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc] 2018-01-05T11:29:12,151 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,152 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] utils.FileUtils: deleting file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src1 2018-01-05T11:29:12,152 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,152 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.DDLTask: Ignoring request to add WriteEntity(default@src1) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@src1) Type=TABLE WriteType=DDL_EXCLUSIVE is present 2018-01-05T11:29:12,152 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,152 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,152 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS src1 2018-01-05T11:29:12,152 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:29:12,152 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@src1 2018-01-05T11:29:12,152 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: default@src1 2018-01-05T11:29:12,152 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,152 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,152 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:12,153 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=13, getTable_(String, String, )=4} 2018-01-05T11:29:12,153 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112912_6fe8c75c-f654-45ef-b515-ddd5e63dade5); Time taken: 0.018 seconds 2018-01-05T11:29:12,153 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:12,153 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,153 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,153 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,153 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query DROP TABLE IF EXISTS src1 2018-01-05T11:29:12,153 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.022 seconds 2018-01-05T11:29:12,153 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,153 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:12,153 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,153 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:12,153 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,153 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:12,153 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,153 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,153 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,153 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS src_json 2018-01-05T11:29:12,153 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112912_bb4d9894-0bc5-4333-9fd5-f596dd06a719): DROP TABLE IF EXISTS src_json 2018-01-05T11:29:12,153 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,153 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS src_json 2018-01-05T11:29:12,153 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:12,153 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,153 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,153 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,154 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,154 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,154 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,154 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_json 2018-01-05T11:29:12,154 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_json 2018-01-05T11:29:12,157 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,157 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,157 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:12,157 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,157 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:29:12,157 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,157 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:12,157 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=3, flushCache_()=1} 2018-01-05T11:29:12,157 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112912_bb4d9894-0bc5-4333-9fd5-f596dd06a719); Time taken: 0.004 seconds 2018-01-05T11:29:12,157 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,157 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,157 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112912_bb4d9894-0bc5-4333-9fd5-f596dd06a719): DROP TABLE IF EXISTS src_json 2018-01-05T11:29:12,158 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,158 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS src_json 2018-01-05T11:29:12,158 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:29:12,158 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@src_json 2018-01-05T11:29:12,158 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: default@src_json 2018-01-05T11:29:12,158 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,158 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,158 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,158 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,158 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:29:12,158 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,158 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_json 2018-01-05T11:29:12,158 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_json 2018-01-05T11:29:12,161 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,161 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,161 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,161 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_json 2018-01-05T11:29:12,161 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_json 2018-01-05T11:29:12,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,164 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,165 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,165 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: drop_table : db=default tbl=src_json 2018-01-05T11:29:12,165 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=drop_table : db=default tbl=src_json 2018-01-05T11:29:12,166 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.133344ms + 0.003106ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc] 2018-01-05T11:29:12,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,173 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] utils.FileUtils: deleting file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_json 2018-01-05T11:29:12,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.DDLTask: Ignoring request to add WriteEntity(default@src_json) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@src_json) Type=TABLE WriteType=DDL_EXCLUSIVE is present 2018-01-05T11:29:12,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,174 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS src_json 2018-01-05T11:29:12,174 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:29:12,174 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@src_json 2018-01-05T11:29:12,174 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: default@src_json 2018-01-05T11:29:12,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,174 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:12,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=13, getTable_(String, String, )=3} 2018-01-05T11:29:12,174 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112912_bb4d9894-0bc5-4333-9fd5-f596dd06a719); Time taken: 0.017 seconds 2018-01-05T11:29:12,174 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:12,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,174 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query DROP TABLE IF EXISTS src_json 2018-01-05T11:29:12,174 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.021 seconds 2018-01-05T11:29:12,174 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,174 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:12,174 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,174 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:12,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:12,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS src_sequencefile 2018-01-05T11:29:12,174 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112912_4dbe0400-1d2e-4374-b7ea-349d41bdb14e): DROP TABLE IF EXISTS src_sequencefile 2018-01-05T11:29:12,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS src_sequencefile 2018-01-05T11:29:12,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:12,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,174 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,174 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,174 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_sequencefile 2018-01-05T11:29:12,174 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_sequencefile 2018-01-05T11:29:12,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,177 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:12,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,177 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:29:12,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,177 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:12,177 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=3, flushCache_()=0} 2018-01-05T11:29:12,177 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112912_4dbe0400-1d2e-4374-b7ea-349d41bdb14e); Time taken: 0.003 seconds 2018-01-05T11:29:12,178 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,178 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,178 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112912_4dbe0400-1d2e-4374-b7ea-349d41bdb14e): DROP TABLE IF EXISTS src_sequencefile 2018-01-05T11:29:12,179 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,179 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS src_sequencefile 2018-01-05T11:29:12,179 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:29:12,179 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@src_sequencefile 2018-01-05T11:29:12,179 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: default@src_sequencefile 2018-01-05T11:29:12,179 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,179 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,179 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,179 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,179 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:29:12,179 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,179 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_sequencefile 2018-01-05T11:29:12,179 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_sequencefile 2018-01-05T11:29:12,181 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,182 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,182 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,182 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_sequencefile 2018-01-05T11:29:12,182 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_sequencefile 2018-01-05T11:29:12,184 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,185 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,185 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,185 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: drop_table : db=default tbl=src_sequencefile 2018-01-05T11:29:12,185 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=drop_table : db=default tbl=src_sequencefile 2018-01-05T11:29:12,188 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.128152ms + 0.002638ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc] 2018-01-05T11:29:12,194 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,194 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] utils.FileUtils: deleting file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_sequencefile 2018-01-05T11:29:12,195 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,195 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.DDLTask: Ignoring request to add WriteEntity(default@src_sequencefile) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@src_sequencefile) Type=TABLE WriteType=DDL_EXCLUSIVE is present 2018-01-05T11:29:12,195 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,195 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,195 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS src_sequencefile 2018-01-05T11:29:12,195 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:29:12,195 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@src_sequencefile 2018-01-05T11:29:12,195 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: default@src_sequencefile 2018-01-05T11:29:12,195 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,195 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,195 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:12,195 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=13, getTable_(String, String, )=3} 2018-01-05T11:29:12,195 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112912_4dbe0400-1d2e-4374-b7ea-349d41bdb14e); Time taken: 0.017 seconds 2018-01-05T11:29:12,195 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:12,195 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,195 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,196 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,196 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query DROP TABLE IF EXISTS src_sequencefile 2018-01-05T11:29:12,196 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.022 seconds 2018-01-05T11:29:12,196 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,196 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:12,196 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,196 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:12,196 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,196 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:12,196 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,196 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,196 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,196 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS src_thrift 2018-01-05T11:29:12,196 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112912_86531e60-545c-4e21-887c-3cef67763966): DROP TABLE IF EXISTS src_thrift 2018-01-05T11:29:12,196 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,196 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS src_thrift 2018-01-05T11:29:12,196 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:12,196 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,196 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,196 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,196 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,196 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,196 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,196 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_thrift 2018-01-05T11:29:12,196 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_thrift 2018-01-05T11:29:12,199 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,199 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,199 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:12,199 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,199 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:29:12,199 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,199 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:12,199 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=3, flushCache_()=0} 2018-01-05T11:29:12,199 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112912_86531e60-545c-4e21-887c-3cef67763966); Time taken: 0.003 seconds 2018-01-05T11:29:12,199 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,199 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,199 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112912_86531e60-545c-4e21-887c-3cef67763966): DROP TABLE IF EXISTS src_thrift 2018-01-05T11:29:12,200 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,200 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS src_thrift 2018-01-05T11:29:12,200 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:29:12,200 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@src_thrift 2018-01-05T11:29:12,200 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: default@src_thrift 2018-01-05T11:29:12,200 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,200 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,200 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,201 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,201 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:29:12,201 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,201 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_thrift 2018-01-05T11:29:12,201 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_thrift 2018-01-05T11:29:12,203 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,204 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,204 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,204 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_thrift 2018-01-05T11:29:12,204 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_thrift 2018-01-05T11:29:12,206 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,207 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,207 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,207 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: drop_table : db=default tbl=src_thrift 2018-01-05T11:29:12,207 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=drop_table : db=default tbl=src_thrift 2018-01-05T11:29:12,210 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.146026ms + 0.003106ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc] 2018-01-05T11:29:12,216 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,216 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] utils.FileUtils: deleting file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_thrift 2018-01-05T11:29:12,217 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,217 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.DDLTask: Ignoring request to add WriteEntity(default@src_thrift) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@src_thrift) Type=TABLE WriteType=DDL_EXCLUSIVE is present 2018-01-05T11:29:12,217 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,217 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,217 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS src_thrift 2018-01-05T11:29:12,217 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:29:12,217 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@src_thrift 2018-01-05T11:29:12,217 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: default@src_thrift 2018-01-05T11:29:12,217 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,217 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,217 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:12,217 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=13, getTable_(String, String, )=3} 2018-01-05T11:29:12,217 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112912_86531e60-545c-4e21-887c-3cef67763966); Time taken: 0.018 seconds 2018-01-05T11:29:12,217 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:12,217 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,217 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,217 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,217 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query DROP TABLE IF EXISTS src_thrift 2018-01-05T11:29:12,217 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.021 seconds 2018-01-05T11:29:12,217 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,217 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:12,217 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,217 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:12,217 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,217 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:12,217 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,217 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,217 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,217 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS srcbucket 2018-01-05T11:29:12,217 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112912_4cd592b0-7ba1-4c29-a16a-e21461aa1dc3): DROP TABLE IF EXISTS srcbucket 2018-01-05T11:29:12,217 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,218 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS srcbucket 2018-01-05T11:29:12,218 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:12,218 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,218 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,218 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,218 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,218 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,218 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,218 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket 2018-01-05T11:29:12,218 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket 2018-01-05T11:29:12,221 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,221 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,221 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:12,221 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,221 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:29:12,222 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,222 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:12,222 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=3, flushCache_()=0} 2018-01-05T11:29:12,222 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112912_4cd592b0-7ba1-4c29-a16a-e21461aa1dc3); Time taken: 0.005 seconds 2018-01-05T11:29:12,222 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,222 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,222 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112912_4cd592b0-7ba1-4c29-a16a-e21461aa1dc3): DROP TABLE IF EXISTS srcbucket 2018-01-05T11:29:12,223 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,223 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS srcbucket 2018-01-05T11:29:12,223 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:29:12,223 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@srcbucket 2018-01-05T11:29:12,223 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: default@srcbucket 2018-01-05T11:29:12,223 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,223 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,223 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,223 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,223 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:29:12,223 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,223 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket 2018-01-05T11:29:12,223 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket 2018-01-05T11:29:12,226 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,227 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,227 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,227 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket 2018-01-05T11:29:12,227 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket 2018-01-05T11:29:12,229 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,230 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,230 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,230 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: drop_table : db=default tbl=srcbucket 2018-01-05T11:29:12,230 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=drop_table : db=default tbl=srcbucket 2018-01-05T11:29:12,232 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.137164ms + 0.003131ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc] 2018-01-05T11:29:12,239 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,239 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] utils.FileUtils: deleting file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket 2018-01-05T11:29:12,240 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,240 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.DDLTask: Ignoring request to add WriteEntity(default@srcbucket) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@srcbucket) Type=TABLE WriteType=DDL_EXCLUSIVE is present 2018-01-05T11:29:12,240 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,240 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,240 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS srcbucket 2018-01-05T11:29:12,240 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:29:12,240 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@srcbucket 2018-01-05T11:29:12,240 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: default@srcbucket 2018-01-05T11:29:12,240 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,240 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,240 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:12,240 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=13, getTable_(String, String, )=4} 2018-01-05T11:29:12,240 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112912_4cd592b0-7ba1-4c29-a16a-e21461aa1dc3); Time taken: 0.018 seconds 2018-01-05T11:29:12,240 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:12,240 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,240 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,240 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,240 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query DROP TABLE IF EXISTS srcbucket 2018-01-05T11:29:12,240 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.023 seconds 2018-01-05T11:29:12,240 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,240 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:12,240 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,240 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:12,241 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,241 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:12,241 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,241 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,241 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,241 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS srcbucket2 2018-01-05T11:29:12,241 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112912_6ff62654-5635-455b-a028-14290a4bea03): DROP TABLE IF EXISTS srcbucket2 2018-01-05T11:29:12,241 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,241 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS srcbucket2 2018-01-05T11:29:12,241 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:12,241 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,241 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,241 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,241 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,241 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,241 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,241 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket2 2018-01-05T11:29:12,241 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket2 2018-01-05T11:29:12,244 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,244 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,245 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:12,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,245 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:29:12,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,245 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:12,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=3, flushCache_()=0} 2018-01-05T11:29:12,245 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112912_6ff62654-5635-455b-a028-14290a4bea03); Time taken: 0.004 seconds 2018-01-05T11:29:12,245 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,245 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,245 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112912_6ff62654-5635-455b-a028-14290a4bea03): DROP TABLE IF EXISTS srcbucket2 2018-01-05T11:29:12,246 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,246 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS srcbucket2 2018-01-05T11:29:12,246 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:29:12,246 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@srcbucket2 2018-01-05T11:29:12,246 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: default@srcbucket2 2018-01-05T11:29:12,246 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,246 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,246 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,246 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,246 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:29:12,246 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,246 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket2 2018-01-05T11:29:12,246 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket2 2018-01-05T11:29:12,248 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,249 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,249 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,249 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcbucket2 2018-01-05T11:29:12,249 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcbucket2 2018-01-05T11:29:12,252 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,253 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,253 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,253 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: drop_table : db=default tbl=srcbucket2 2018-01-05T11:29:12,253 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=drop_table : db=default tbl=srcbucket2 2018-01-05T11:29:12,255 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.12717ms + 0.00274ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc] 2018-01-05T11:29:12,263 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,263 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] utils.FileUtils: deleting file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcbucket2 2018-01-05T11:29:12,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.DDLTask: Ignoring request to add WriteEntity(default@srcbucket2) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@srcbucket2) Type=TABLE WriteType=DDL_EXCLUSIVE is present 2018-01-05T11:29:12,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,264 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,264 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS srcbucket2 2018-01-05T11:29:12,265 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:29:12,265 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@srcbucket2 2018-01-05T11:29:12,265 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: default@srcbucket2 2018-01-05T11:29:12,265 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,265 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,265 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:12,265 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=15, getTable_(String, String, )=3} 2018-01-05T11:29:12,265 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112912_6ff62654-5635-455b-a028-14290a4bea03); Time taken: 0.02 seconds 2018-01-05T11:29:12,265 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:12,265 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,265 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,265 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,265 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query DROP TABLE IF EXISTS srcbucket2 2018-01-05T11:29:12,265 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.024 seconds 2018-01-05T11:29:12,265 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,265 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:12,265 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,265 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:12,265 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,265 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:12,265 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,265 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,265 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,265 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS srcpart 2018-01-05T11:29:12,265 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112912_d0cb232f-fd51-4023-8409-ce6e53ff5d92): DROP TABLE IF EXISTS srcpart 2018-01-05T11:29:12,265 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,265 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS srcpart 2018-01-05T11:29:12,265 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:12,265 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,265 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,266 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,266 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,266 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,266 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,266 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcpart 2018-01-05T11:29:12,266 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcpart 2018-01-05T11:29:12,269 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,269 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,269 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:12,269 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,269 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:29:12,269 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,269 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:12,269 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=3, flushCache_()=0} 2018-01-05T11:29:12,269 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112912_d0cb232f-fd51-4023-8409-ce6e53ff5d92); Time taken: 0.004 seconds 2018-01-05T11:29:12,269 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,269 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,269 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112912_d0cb232f-fd51-4023-8409-ce6e53ff5d92): DROP TABLE IF EXISTS srcpart 2018-01-05T11:29:12,271 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,271 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS srcpart 2018-01-05T11:29:12,271 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:29:12,271 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@srcpart 2018-01-05T11:29:12,271 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: default@srcpart 2018-01-05T11:29:12,271 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,271 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,271 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,271 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,271 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:29:12,271 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,271 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcpart 2018-01-05T11:29:12,271 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcpart 2018-01-05T11:29:12,274 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,275 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,275 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,275 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=srcpart 2018-01-05T11:29:12,275 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=srcpart 2018-01-05T11:29:12,277 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,278 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,278 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,278 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: drop_table : db=default tbl=srcpart 2018-01-05T11:29:12,278 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=drop_table : db=default tbl=srcpart 2018-01-05T11:29:12,280 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.50821ms + 0.002983ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc] 2018-01-05T11:29:12,284 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 3.966119ms + 0.106689ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (1,2,3,4) order by "PART_NAME" asc] 2018-01-05T11:29:12,286 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.666328ms + 0.180534ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (1,2,3,4) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:29:12,288 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.644095ms + 0.050776ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (1,2,3,4) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:12,290 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 2.177499ms + 0.01492ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (11,12,13,14) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:29:12,292 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.823641ms + 0.014011ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (11,12,13,14) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:12,294 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.496049ms + 0.014032ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (11,12,13,14) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:12,295 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.519063ms + 0.018146ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (11,12,13,14) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:12,298 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 2.268773ms + 0.047484ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (10) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:12,300 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 2.117359ms + 0.110587ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (11,12,13,14) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:29:12,354 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 3.291107ms + 0.002959ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc] 2018-01-05T11:29:12,388 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,388 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] utils.FileUtils: deleting file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/srcpart 2018-01-05T11:29:12,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.DDLTask: Ignoring request to add WriteEntity(default@srcpart) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@srcpart) Type=TABLE WriteType=DDL_EXCLUSIVE is present 2018-01-05T11:29:12,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,390 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS srcpart 2018-01-05T11:29:12,390 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:29:12,390 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@srcpart 2018-01-05T11:29:12,390 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: default@srcpart 2018-01-05T11:29:12,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,390 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:12,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=115, getTable_(String, String, )=4} 2018-01-05T11:29:12,390 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112912_d0cb232f-fd51-4023-8409-ce6e53ff5d92); Time taken: 0.121 seconds 2018-01-05T11:29:12,390 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:12,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,390 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query DROP TABLE IF EXISTS srcpart 2018-01-05T11:29:12,390 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.125 seconds 2018-01-05T11:29:12,390 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,390 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:12,390 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,390 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:12,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:12,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS primitives 2018-01-05T11:29:12,390 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112912_91176148-c47f-4f0c-94d6-7659dbea3ae2): DROP TABLE IF EXISTS primitives 2018-01-05T11:29:12,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,390 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS primitives 2018-01-05T11:29:12,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:12,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,391 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,391 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=primitives 2018-01-05T11:29:12,391 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=primitives 2018-01-05T11:29:12,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,391 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:12,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,391 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:29:12,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,391 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:12,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:29:12,391 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112912_91176148-c47f-4f0c-94d6-7659dbea3ae2); Time taken: 0.001 seconds 2018-01-05T11:29:12,391 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,391 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,391 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112912_91176148-c47f-4f0c-94d6-7659dbea3ae2): DROP TABLE IF EXISTS primitives 2018-01-05T11:29:12,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,392 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS primitives 2018-01-05T11:29:12,392 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:29:12,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,392 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,392 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:29:12,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,393 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=primitives 2018-01-05T11:29:12,393 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=primitives 2018-01-05T11:29:12,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,393 ERROR [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Table primitives not found: default.primitives table not found 2018-01-05T11:29:12,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,393 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=primitives 2018-01-05T11:29:12,393 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=primitives 2018-01-05T11:29:12,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,393 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,394 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS primitives 2018-01-05T11:29:12,394 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:29:12,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,394 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:12,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=1, dropTable_(String, String, boolean, boolean, boolean, )=1} 2018-01-05T11:29:12,394 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112912_91176148-c47f-4f0c-94d6-7659dbea3ae2); Time taken: 0.003 seconds 2018-01-05T11:29:12,394 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:12,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,394 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query DROP TABLE IF EXISTS primitives 2018-01-05T11:29:12,394 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.004 seconds 2018-01-05T11:29:12,394 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,394 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:12,394 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,394 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:12,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:12,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS dest1 2018-01-05T11:29:12,394 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112912_494c69ce-ae03-4848-8096-987246143516): DROP TABLE IF EXISTS dest1 2018-01-05T11:29:12,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS dest1 2018-01-05T11:29:12,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:12,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,394 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,395 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,395 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,395 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,395 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,395 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest1 2018-01-05T11:29:12,395 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest1 2018-01-05T11:29:12,395 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,395 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,395 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:12,395 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,395 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:29:12,395 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,395 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:12,395 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=1} 2018-01-05T11:29:12,395 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112912_494c69ce-ae03-4848-8096-987246143516); Time taken: 0.001 seconds 2018-01-05T11:29:12,395 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,395 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,395 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112912_494c69ce-ae03-4848-8096-987246143516): DROP TABLE IF EXISTS dest1 2018-01-05T11:29:12,396 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,396 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS dest1 2018-01-05T11:29:12,396 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:29:12,396 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,396 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,396 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,396 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,396 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:29:12,396 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,396 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest1 2018-01-05T11:29:12,396 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest1 2018-01-05T11:29:12,396 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,396 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,396 ERROR [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Table dest1 not found: default.dest1 table not found 2018-01-05T11:29:12,396 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,396 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest1 2018-01-05T11:29:12,397 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest1 2018-01-05T11:29:12,397 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,397 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,397 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,397 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,397 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS dest1 2018-01-05T11:29:12,397 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:29:12,397 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,397 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,397 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:12,397 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=1} 2018-01-05T11:29:12,397 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112912_494c69ce-ae03-4848-8096-987246143516); Time taken: 0.002 seconds 2018-01-05T11:29:12,397 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:12,397 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,397 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,397 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,397 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query DROP TABLE IF EXISTS dest1 2018-01-05T11:29:12,397 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.003 seconds 2018-01-05T11:29:12,397 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,397 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:12,397 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,397 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:12,397 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,397 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:12,397 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,397 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,397 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,397 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS dest2 2018-01-05T11:29:12,397 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112912_a2f286b6-2f92-487b-98af-4d2e2c826559): DROP TABLE IF EXISTS dest2 2018-01-05T11:29:12,397 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,397 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS dest2 2018-01-05T11:29:12,397 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:12,397 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,397 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,397 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,397 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,397 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,397 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,397 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest2 2018-01-05T11:29:12,397 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest2 2018-01-05T11:29:12,398 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,398 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,398 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:12,398 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,398 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:29:12,398 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,398 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:12,398 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:29:12,398 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112912_a2f286b6-2f92-487b-98af-4d2e2c826559); Time taken: 0.001 seconds 2018-01-05T11:29:12,398 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,398 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,398 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112912_a2f286b6-2f92-487b-98af-4d2e2c826559): DROP TABLE IF EXISTS dest2 2018-01-05T11:29:12,399 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,399 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS dest2 2018-01-05T11:29:12,399 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:29:12,399 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,399 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,399 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,399 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,399 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:29:12,399 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,399 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest2 2018-01-05T11:29:12,399 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest2 2018-01-05T11:29:12,399 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,399 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,399 ERROR [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Table dest2 not found: default.dest2 table not found 2018-01-05T11:29:12,399 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,399 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest2 2018-01-05T11:29:12,399 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest2 2018-01-05T11:29:12,399 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,399 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,399 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,400 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS dest2 2018-01-05T11:29:12,400 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:29:12,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,400 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:12,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=0} 2018-01-05T11:29:12,400 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112912_a2f286b6-2f92-487b-98af-4d2e2c826559); Time taken: 0.002 seconds 2018-01-05T11:29:12,400 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:12,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,400 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query DROP TABLE IF EXISTS dest2 2018-01-05T11:29:12,400 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.003 seconds 2018-01-05T11:29:12,400 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,400 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:12,400 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,400 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:12,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:12,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS dest3 2018-01-05T11:29:12,400 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112912_e8470b19-8af3-422e-917c-9a8ef995f986): DROP TABLE IF EXISTS dest3 2018-01-05T11:29:12,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS dest3 2018-01-05T11:29:12,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:12,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,400 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,400 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,400 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest3 2018-01-05T11:29:12,400 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest3 2018-01-05T11:29:12,401 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,401 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,401 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:12,401 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,401 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:29:12,401 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,401 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:12,401 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:29:12,401 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112912_e8470b19-8af3-422e-917c-9a8ef995f986); Time taken: 0.001 seconds 2018-01-05T11:29:12,401 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,401 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,401 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112912_e8470b19-8af3-422e-917c-9a8ef995f986): DROP TABLE IF EXISTS dest3 2018-01-05T11:29:12,402 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,402 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS dest3 2018-01-05T11:29:12,402 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:29:12,402 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,402 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,402 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,402 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,402 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:29:12,402 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,402 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest3 2018-01-05T11:29:12,402 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest3 2018-01-05T11:29:12,402 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,402 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,402 ERROR [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Table dest3 not found: default.dest3 table not found 2018-01-05T11:29:12,402 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,402 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest3 2018-01-05T11:29:12,403 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest3 2018-01-05T11:29:12,403 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,403 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,403 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,403 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,403 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS dest3 2018-01-05T11:29:12,403 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:29:12,403 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,403 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,403 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:12,403 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=1} 2018-01-05T11:29:12,403 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112912_e8470b19-8af3-422e-917c-9a8ef995f986); Time taken: 0.002 seconds 2018-01-05T11:29:12,403 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:12,403 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,403 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,403 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,403 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query DROP TABLE IF EXISTS dest3 2018-01-05T11:29:12,403 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.003 seconds 2018-01-05T11:29:12,403 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,403 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:12,403 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,403 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:12,403 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,403 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:12,403 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,403 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,403 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,403 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS dest4 2018-01-05T11:29:12,403 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112912_2c7052d8-0345-4752-aeb5-2af6805d6e95): DROP TABLE IF EXISTS dest4 2018-01-05T11:29:12,403 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,403 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS dest4 2018-01-05T11:29:12,403 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:12,403 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,403 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,404 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,404 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,404 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,404 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,404 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest4 2018-01-05T11:29:12,404 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest4 2018-01-05T11:29:12,404 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,404 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,404 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:12,404 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,404 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:29:12,404 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,404 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:12,404 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=1, flushCache_()=0} 2018-01-05T11:29:12,404 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112912_2c7052d8-0345-4752-aeb5-2af6805d6e95); Time taken: 0.001 seconds 2018-01-05T11:29:12,404 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,404 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,404 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112912_2c7052d8-0345-4752-aeb5-2af6805d6e95): DROP TABLE IF EXISTS dest4 2018-01-05T11:29:12,405 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,405 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS dest4 2018-01-05T11:29:12,405 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:29:12,405 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,405 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,405 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,405 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,405 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:29:12,405 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,405 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest4 2018-01-05T11:29:12,405 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest4 2018-01-05T11:29:12,405 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,405 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,405 ERROR [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Table dest4 not found: default.dest4 table not found 2018-01-05T11:29:12,405 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,405 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest4 2018-01-05T11:29:12,405 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest4 2018-01-05T11:29:12,406 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,406 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,406 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,406 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,406 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS dest4 2018-01-05T11:29:12,406 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:29:12,406 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,406 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,406 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:12,406 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=1} 2018-01-05T11:29:12,406 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112912_2c7052d8-0345-4752-aeb5-2af6805d6e95); Time taken: 0.002 seconds 2018-01-05T11:29:12,406 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:12,406 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,406 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,406 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,406 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query DROP TABLE IF EXISTS dest4 2018-01-05T11:29:12,406 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.003 seconds 2018-01-05T11:29:12,406 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,406 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:12,406 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,406 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:12,406 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,406 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:12,406 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,406 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,406 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,406 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS dest4_sequencefile 2018-01-05T11:29:12,406 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112912_060f4792-4765-4160-a7e4-879b7b451391): DROP TABLE IF EXISTS dest4_sequencefile 2018-01-05T11:29:12,406 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,406 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS dest4_sequencefile 2018-01-05T11:29:12,406 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:12,406 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,406 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,407 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,407 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,407 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,407 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,407 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest4_sequencefile 2018-01-05T11:29:12,407 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest4_sequencefile 2018-01-05T11:29:12,407 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,407 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,407 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:12,407 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,407 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:29:12,407 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,407 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:12,407 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=1, flushCache_()=0} 2018-01-05T11:29:12,407 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112912_060f4792-4765-4160-a7e4-879b7b451391); Time taken: 0.001 seconds 2018-01-05T11:29:12,407 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,407 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,407 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112912_060f4792-4765-4160-a7e4-879b7b451391): DROP TABLE IF EXISTS dest4_sequencefile 2018-01-05T11:29:12,408 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,408 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS dest4_sequencefile 2018-01-05T11:29:12,408 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:29:12,408 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,408 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,408 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,408 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,408 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:29:12,409 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,409 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest4_sequencefile 2018-01-05T11:29:12,409 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest4_sequencefile 2018-01-05T11:29:12,409 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,409 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,409 ERROR [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Table dest4_sequencefile not found: default.dest4_sequencefile table not found 2018-01-05T11:29:12,409 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,409 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest4_sequencefile 2018-01-05T11:29:12,409 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest4_sequencefile 2018-01-05T11:29:12,409 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,409 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,410 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,410 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,410 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS dest4_sequencefile 2018-01-05T11:29:12,410 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:29:12,410 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,410 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,410 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:12,410 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=1, dropTable_(String, String, boolean, boolean, boolean, )=0} 2018-01-05T11:29:12,410 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112912_060f4792-4765-4160-a7e4-879b7b451391); Time taken: 0.003 seconds 2018-01-05T11:29:12,410 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:12,410 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,410 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,410 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,410 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query DROP TABLE IF EXISTS dest4_sequencefile 2018-01-05T11:29:12,410 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.004 seconds 2018-01-05T11:29:12,410 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,410 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:12,410 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,410 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:12,410 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,410 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:12,410 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,410 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,410 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,410 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS dest_j1 2018-01-05T11:29:12,410 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112912_1946d836-ea6f-450b-befe-43896a8965af): DROP TABLE IF EXISTS dest_j1 2018-01-05T11:29:12,410 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,410 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS dest_j1 2018-01-05T11:29:12,410 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:12,410 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,410 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,410 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,410 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,410 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,410 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,410 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest_j1 2018-01-05T11:29:12,411 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest_j1 2018-01-05T11:29:12,411 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,411 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,411 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:12,411 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,411 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:29:12,411 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,411 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:12,411 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:29:12,411 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112912_1946d836-ea6f-450b-befe-43896a8965af); Time taken: 0.001 seconds 2018-01-05T11:29:12,411 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,411 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,411 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112912_1946d836-ea6f-450b-befe-43896a8965af): DROP TABLE IF EXISTS dest_j1 2018-01-05T11:29:12,412 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,412 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS dest_j1 2018-01-05T11:29:12,412 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:29:12,412 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,412 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,412 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,412 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,412 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:29:12,412 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,412 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest_j1 2018-01-05T11:29:12,412 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest_j1 2018-01-05T11:29:12,412 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,412 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,412 ERROR [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Table dest_j1 not found: default.dest_j1 table not found 2018-01-05T11:29:12,413 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,413 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest_j1 2018-01-05T11:29:12,413 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest_j1 2018-01-05T11:29:12,413 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,413 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,413 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,413 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,413 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS dest_j1 2018-01-05T11:29:12,413 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:29:12,413 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,413 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,413 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:12,413 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=0} 2018-01-05T11:29:12,413 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112912_1946d836-ea6f-450b-befe-43896a8965af); Time taken: 0.002 seconds 2018-01-05T11:29:12,413 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:12,413 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,413 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,413 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,413 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query DROP TABLE IF EXISTS dest_j1 2018-01-05T11:29:12,413 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.003 seconds 2018-01-05T11:29:12,413 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,413 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:12,413 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,413 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:12,413 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,413 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:12,413 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,413 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,413 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,413 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS dest_g1 2018-01-05T11:29:12,413 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112912_072701a6-21af-4667-bbd0-e6ae2b3cd13f): DROP TABLE IF EXISTS dest_g1 2018-01-05T11:29:12,413 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,413 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS dest_g1 2018-01-05T11:29:12,413 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:12,413 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,413 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,413 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,413 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,413 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,413 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,413 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest_g1 2018-01-05T11:29:12,414 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest_g1 2018-01-05T11:29:12,414 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,414 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,414 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:12,414 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,414 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:29:12,414 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,414 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:12,414 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:29:12,414 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112912_072701a6-21af-4667-bbd0-e6ae2b3cd13f); Time taken: 0.001 seconds 2018-01-05T11:29:12,414 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,414 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,414 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112912_072701a6-21af-4667-bbd0-e6ae2b3cd13f): DROP TABLE IF EXISTS dest_g1 2018-01-05T11:29:12,415 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,415 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS dest_g1 2018-01-05T11:29:12,415 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:29:12,415 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,415 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,415 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,415 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,415 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:29:12,415 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,415 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest_g1 2018-01-05T11:29:12,415 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest_g1 2018-01-05T11:29:12,415 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,415 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,415 ERROR [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Table dest_g1 not found: default.dest_g1 table not found 2018-01-05T11:29:12,415 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,415 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest_g1 2018-01-05T11:29:12,415 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest_g1 2018-01-05T11:29:12,416 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,416 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,416 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,416 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,416 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS dest_g1 2018-01-05T11:29:12,416 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:29:12,416 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,416 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,416 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:12,416 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=1} 2018-01-05T11:29:12,416 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112912_072701a6-21af-4667-bbd0-e6ae2b3cd13f); Time taken: 0.002 seconds 2018-01-05T11:29:12,416 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:12,416 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,416 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,416 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,416 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query DROP TABLE IF EXISTS dest_g1 2018-01-05T11:29:12,416 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.003 seconds 2018-01-05T11:29:12,416 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,416 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:12,416 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,416 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:12,416 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,416 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:12,416 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,416 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,416 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,416 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS dest_g2 2018-01-05T11:29:12,416 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112912_5a467536-f189-4ee1-b225-c20c988838c4): DROP TABLE IF EXISTS dest_g2 2018-01-05T11:29:12,416 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,416 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS dest_g2 2018-01-05T11:29:12,416 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:12,416 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,416 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,416 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,416 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,416 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,416 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,416 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest_g2 2018-01-05T11:29:12,416 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest_g2 2018-01-05T11:29:12,417 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,417 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,417 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:12,417 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,417 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:29:12,417 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,417 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:12,417 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:29:12,417 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112912_5a467536-f189-4ee1-b225-c20c988838c4); Time taken: 0.001 seconds 2018-01-05T11:29:12,417 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,417 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,417 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112912_5a467536-f189-4ee1-b225-c20c988838c4): DROP TABLE IF EXISTS dest_g2 2018-01-05T11:29:12,418 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,418 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS dest_g2 2018-01-05T11:29:12,418 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:29:12,418 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,418 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,418 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,418 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,418 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:29:12,418 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,418 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest_g2 2018-01-05T11:29:12,418 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest_g2 2018-01-05T11:29:12,419 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,419 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,419 ERROR [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Table dest_g2 not found: default.dest_g2 table not found 2018-01-05T11:29:12,419 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,419 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=dest_g2 2018-01-05T11:29:12,419 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=dest_g2 2018-01-05T11:29:12,419 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,419 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,419 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,419 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,419 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS dest_g2 2018-01-05T11:29:12,419 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:29:12,419 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,419 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,419 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:12,419 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=0} 2018-01-05T11:29:12,419 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112912_5a467536-f189-4ee1-b225-c20c988838c4); Time taken: 0.002 seconds 2018-01-05T11:29:12,419 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:12,419 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,419 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,419 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,419 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query DROP TABLE IF EXISTS dest_g2 2018-01-05T11:29:12,419 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.003 seconds 2018-01-05T11:29:12,419 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,419 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:12,419 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,419 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:12,419 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,419 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:12,419 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,419 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,419 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,419 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS fetchtask_ioexception 2018-01-05T11:29:12,419 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112912_a6eb9c72-03b1-4bb2-8360-3d91ff0b1e02): DROP TABLE IF EXISTS fetchtask_ioexception 2018-01-05T11:29:12,419 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,419 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS fetchtask_ioexception 2018-01-05T11:29:12,420 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:12,420 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,420 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,420 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,420 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,420 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,420 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,420 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=fetchtask_ioexception 2018-01-05T11:29:12,420 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=fetchtask_ioexception 2018-01-05T11:29:12,420 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,420 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,420 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:12,420 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,420 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:29:12,420 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,420 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:12,420 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, flushCache_()=0} 2018-01-05T11:29:12,420 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112912_a6eb9c72-03b1-4bb2-8360-3d91ff0b1e02); Time taken: 0.001 seconds 2018-01-05T11:29:12,420 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,420 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,420 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112912_a6eb9c72-03b1-4bb2-8360-3d91ff0b1e02): DROP TABLE IF EXISTS fetchtask_ioexception 2018-01-05T11:29:12,421 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,421 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS fetchtask_ioexception 2018-01-05T11:29:12,421 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:29:12,421 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,421 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,421 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,421 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,421 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:29:12,421 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,421 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=fetchtask_ioexception 2018-01-05T11:29:12,421 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=fetchtask_ioexception 2018-01-05T11:29:12,421 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,421 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,422 ERROR [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Table fetchtask_ioexception not found: default.fetchtask_ioexception table not found 2018-01-05T11:29:12,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,422 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=fetchtask_ioexception 2018-01-05T11:29:12,422 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=fetchtask_ioexception 2018-01-05T11:29:12,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,422 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS fetchtask_ioexception 2018-01-05T11:29:12,422 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:29:12,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,422 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:12,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=0} 2018-01-05T11:29:12,422 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112912_a6eb9c72-03b1-4bb2-8360-3d91ff0b1e02); Time taken: 0.002 seconds 2018-01-05T11:29:12,422 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:12,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,422 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query DROP TABLE IF EXISTS fetchtask_ioexception 2018-01-05T11:29:12,422 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.003 seconds 2018-01-05T11:29:12,422 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,422 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:12,422 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,422 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:12,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:12,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS alltypesorc 2018-01-05T11:29:12,422 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112912_7b00d47a-418d-4aa7-bc9d-71fec3868a42): DROP TABLE IF EXISTS alltypesorc 2018-01-05T11:29:12,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS alltypesorc 2018-01-05T11:29:12,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:12,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,422 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,422 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,422 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesorc 2018-01-05T11:29:12,422 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesorc 2018-01-05T11:29:12,434 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,434 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,434 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:12,434 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,435 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:29:12,435 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,435 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:12,435 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=12, flushCache_()=0} 2018-01-05T11:29:12,435 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112912_7b00d47a-418d-4aa7-bc9d-71fec3868a42); Time taken: 0.013 seconds 2018-01-05T11:29:12,435 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,435 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,435 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112912_7b00d47a-418d-4aa7-bc9d-71fec3868a42): DROP TABLE IF EXISTS alltypesorc 2018-01-05T11:29:12,436 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,436 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS alltypesorc 2018-01-05T11:29:12,436 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:29:12,436 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@alltypesorc 2018-01-05T11:29:12,436 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: default@alltypesorc 2018-01-05T11:29:12,436 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,436 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,436 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,436 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,436 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:29:12,436 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,436 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesorc 2018-01-05T11:29:12,436 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesorc 2018-01-05T11:29:12,438 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,439 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,439 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,439 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesorc 2018-01-05T11:29:12,439 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesorc 2018-01-05T11:29:12,441 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,442 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,442 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,442 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: drop_table : db=default tbl=alltypesorc 2018-01-05T11:29:12,442 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=drop_table : db=default tbl=alltypesorc 2018-01-05T11:29:12,447 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.119732ms + 0.002719ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc] 2018-01-05T11:29:12,463 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,464 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] utils.FileUtils: deleting file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesorc 2018-01-05T11:29:12,466 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,466 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.DDLTask: Ignoring request to add WriteEntity(default@alltypesorc) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@alltypesorc) Type=TABLE WriteType=DDL_EXCLUSIVE is present 2018-01-05T11:29:12,466 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,466 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,466 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS alltypesorc 2018-01-05T11:29:12,466 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:29:12,466 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@alltypesorc 2018-01-05T11:29:12,466 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: default@alltypesorc 2018-01-05T11:29:12,466 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,466 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,466 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:12,466 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=27, getTable_(String, String, )=3} 2018-01-05T11:29:12,466 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112912_7b00d47a-418d-4aa7-bc9d-71fec3868a42); Time taken: 0.031 seconds 2018-01-05T11:29:12,466 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:12,466 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,466 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,466 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,466 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query DROP TABLE IF EXISTS alltypesorc 2018-01-05T11:29:12,466 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.044 seconds 2018-01-05T11:29:12,466 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,466 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:12,466 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,466 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:12,466 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,466 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:12,466 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,466 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,466 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,466 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS alltypesparquet 2018-01-05T11:29:12,466 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112912_c5868e78-66af-45d4-bb25-0fb7729daa33): DROP TABLE IF EXISTS alltypesparquet 2018-01-05T11:29:12,466 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,466 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS alltypesparquet 2018-01-05T11:29:12,466 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:12,466 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,466 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,467 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,467 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,467 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,467 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,467 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesparquet 2018-01-05T11:29:12,467 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesparquet 2018-01-05T11:29:12,470 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,470 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,470 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:12,470 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,470 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:29:12,470 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,470 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:12,470 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=3, flushCache_()=0} 2018-01-05T11:29:12,470 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112912_c5868e78-66af-45d4-bb25-0fb7729daa33); Time taken: 0.004 seconds 2018-01-05T11:29:12,470 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,470 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,470 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112912_c5868e78-66af-45d4-bb25-0fb7729daa33): DROP TABLE IF EXISTS alltypesparquet 2018-01-05T11:29:12,471 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,471 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS alltypesparquet 2018-01-05T11:29:12,471 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:29:12,471 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@alltypesparquet 2018-01-05T11:29:12,471 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: default@alltypesparquet 2018-01-05T11:29:12,471 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,471 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,471 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,471 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,471 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:29:12,472 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,472 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesparquet 2018-01-05T11:29:12,472 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesparquet 2018-01-05T11:29:12,474 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,475 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,475 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,475 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=alltypesparquet 2018-01-05T11:29:12,475 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=alltypesparquet 2018-01-05T11:29:12,478 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,479 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,479 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,479 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: drop_table : db=default tbl=alltypesparquet 2018-01-05T11:29:12,479 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=drop_table : db=default tbl=alltypesparquet 2018-01-05T11:29:12,481 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.205071ms + 0.003862ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc] 2018-01-05T11:29:12,492 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,493 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] utils.FileUtils: deleting file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/alltypesparquet 2018-01-05T11:29:12,493 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,493 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.DDLTask: Ignoring request to add WriteEntity(default@alltypesparquet) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@alltypesparquet) Type=TABLE WriteType=DDL_EXCLUSIVE is present 2018-01-05T11:29:12,493 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,493 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,493 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS alltypesparquet 2018-01-05T11:29:12,493 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:29:12,493 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@alltypesparquet 2018-01-05T11:29:12,493 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: default@alltypesparquet 2018-01-05T11:29:12,493 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,493 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,493 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:12,493 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=1, dropTable_(String, String, boolean, boolean, boolean, )=18, getTable_(String, String, )=3} 2018-01-05T11:29:12,493 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112912_c5868e78-66af-45d4-bb25-0fb7729daa33); Time taken: 0.023 seconds 2018-01-05T11:29:12,493 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:12,493 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,493 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,493 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,493 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query DROP TABLE IF EXISTS alltypesparquet 2018-01-05T11:29:12,493 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.027 seconds 2018-01-05T11:29:12,493 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,494 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:12,494 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,494 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:12,494 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,494 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:12,494 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,494 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,494 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,494 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS cbo_t1 2018-01-05T11:29:12,494 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112912_554f289b-ad29-49e3-8735-d3981ccc0fd1): DROP TABLE IF EXISTS cbo_t1 2018-01-05T11:29:12,494 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,494 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS cbo_t1 2018-01-05T11:29:12,494 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:12,494 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,494 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,494 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,494 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,494 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,494 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,494 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t1 2018-01-05T11:29:12,494 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t1 2018-01-05T11:29:12,497 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,497 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,497 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:12,497 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,497 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:29:12,498 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,498 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:12,498 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=3, flushCache_()=0} 2018-01-05T11:29:12,498 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112912_554f289b-ad29-49e3-8735-d3981ccc0fd1); Time taken: 0.004 seconds 2018-01-05T11:29:12,498 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,498 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,498 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112912_554f289b-ad29-49e3-8735-d3981ccc0fd1): DROP TABLE IF EXISTS cbo_t1 2018-01-05T11:29:12,499 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,499 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS cbo_t1 2018-01-05T11:29:12,499 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:29:12,499 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@cbo_t1 2018-01-05T11:29:12,499 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: default@cbo_t1 2018-01-05T11:29:12,499 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,499 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,499 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,499 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,499 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:29:12,499 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,499 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t1 2018-01-05T11:29:12,499 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t1 2018-01-05T11:29:12,502 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,503 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,503 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,503 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t1 2018-01-05T11:29:12,503 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t1 2018-01-05T11:29:12,506 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,507 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,507 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,507 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: drop_table : db=default tbl=cbo_t1 2018-01-05T11:29:12,507 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=drop_table : db=default tbl=cbo_t1 2018-01-05T11:29:12,509 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.181144ms + 0.003104ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc] 2018-01-05T11:29:12,513 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 3.255505ms + 0.078595ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (10) order by "PART_NAME" asc] 2018-01-05T11:29:12,515 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 2.214661ms + 0.111252ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (10) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:29:12,517 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.557986ms + 0.035136ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (10) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:12,518 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.522944ms + 0.013702ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (31) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:29:12,520 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.374376ms + 0.013386ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (31) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:12,521 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.579697ms + 0.019322ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (31) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:12,523 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.745724ms + 0.014977ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (31) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:12,525 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.696267ms + 0.048835ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (19) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:12,527 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.803274ms + 0.052176ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (31) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:29:12,553 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.393536ms + 0.002922ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc] 2018-01-05T11:29:12,561 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,561 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] utils.FileUtils: deleting file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t1 2018-01-05T11:29:12,562 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,562 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.DDLTask: Ignoring request to add WriteEntity(default@cbo_t1) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@cbo_t1) Type=TABLE WriteType=DDL_EXCLUSIVE is present 2018-01-05T11:29:12,562 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,562 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,563 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS cbo_t1 2018-01-05T11:29:12,563 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:29:12,563 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@cbo_t1 2018-01-05T11:29:12,563 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: default@cbo_t1 2018-01-05T11:29:12,563 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,563 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,563 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:12,563 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=59, getTable_(String, String, )=4} 2018-01-05T11:29:12,563 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112912_554f289b-ad29-49e3-8735-d3981ccc0fd1); Time taken: 0.065 seconds 2018-01-05T11:29:12,563 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:12,563 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,563 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,563 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,563 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query DROP TABLE IF EXISTS cbo_t1 2018-01-05T11:29:12,563 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.069 seconds 2018-01-05T11:29:12,563 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,563 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:12,563 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,563 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:12,563 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,563 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:12,563 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,563 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,563 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,563 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS cbo_t2 2018-01-05T11:29:12,563 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112912_16e17748-5bdb-49ba-b9fe-6babc1b7e03d): DROP TABLE IF EXISTS cbo_t2 2018-01-05T11:29:12,563 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,563 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS cbo_t2 2018-01-05T11:29:12,563 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:12,563 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,563 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,564 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,564 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,564 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,564 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,564 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t2 2018-01-05T11:29:12,564 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t2 2018-01-05T11:29:12,567 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,567 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,568 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:12,568 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,568 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:29:12,568 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,568 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:12,568 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=1, getTable_(String, String, )=4, flushCache_()=0} 2018-01-05T11:29:12,568 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112912_16e17748-5bdb-49ba-b9fe-6babc1b7e03d); Time taken: 0.005 seconds 2018-01-05T11:29:12,568 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,568 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,568 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112912_16e17748-5bdb-49ba-b9fe-6babc1b7e03d): DROP TABLE IF EXISTS cbo_t2 2018-01-05T11:29:12,569 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,569 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS cbo_t2 2018-01-05T11:29:12,569 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:29:12,569 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@cbo_t2 2018-01-05T11:29:12,569 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: default@cbo_t2 2018-01-05T11:29:12,569 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,569 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,569 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,569 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,569 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:29:12,569 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,569 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t2 2018-01-05T11:29:12,569 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t2 2018-01-05T11:29:12,572 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,573 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,573 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t2 2018-01-05T11:29:12,573 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t2 2018-01-05T11:29:12,575 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,576 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,576 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,576 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: drop_table : db=default tbl=cbo_t2 2018-01-05T11:29:12,576 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=drop_table : db=default tbl=cbo_t2 2018-01-05T11:29:12,580 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.143652ms + 0.002924ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc] 2018-01-05T11:29:12,583 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 2.996527ms + 0.077157ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (11) order by "PART_NAME" asc] 2018-01-05T11:29:12,585 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.850214ms + 0.080656ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (11) and "PARAM_KEY" is not null order by "PART_ID" asc] 2018-01-05T11:29:12,587 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.835301ms + 0.045599ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (11) order by "PART_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:12,589 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.562737ms + 0.013197ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (32) and "PARAM_KEY" is not null order by "SD_ID" asc] 2018-01-05T11:29:12,590 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.539057ms + 0.012516ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (32) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:12,591 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.241567ms + 0.012115ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (32) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:12,593 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.683655ms + 0.013564ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (32) order by "SD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:12,595 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.560062ms + 0.041398ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (20) order by "CD_ID" asc, "INTEGER_IDX" asc] 2018-01-05T11:29:12,596 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 1.539768ms + 0.049208ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (32) and "PARAM_KEY" is not null order by "SERDE_ID" asc] 2018-01-05T11:29:12,602 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.358528ms + 0.002927ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc] 2018-01-05T11:29:12,610 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,610 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] utils.FileUtils: deleting file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t2 2018-01-05T11:29:12,611 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,611 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.DDLTask: Ignoring request to add WriteEntity(default@cbo_t2) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@cbo_t2) Type=TABLE WriteType=DDL_EXCLUSIVE is present 2018-01-05T11:29:12,611 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,611 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,611 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS cbo_t2 2018-01-05T11:29:12,611 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:29:12,611 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@cbo_t2 2018-01-05T11:29:12,611 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: default@cbo_t2 2018-01-05T11:29:12,611 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,611 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,611 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:12,611 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=38, getTable_(String, String, )=4} 2018-01-05T11:29:12,611 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112912_16e17748-5bdb-49ba-b9fe-6babc1b7e03d); Time taken: 0.043 seconds 2018-01-05T11:29:12,611 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:12,611 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,611 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,611 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,611 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query DROP TABLE IF EXISTS cbo_t2 2018-01-05T11:29:12,611 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.048 seconds 2018-01-05T11:29:12,611 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,611 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:12,611 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,611 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:12,611 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,611 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:12,611 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,611 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,611 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,611 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS cbo_t3 2018-01-05T11:29:12,611 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112912_ba63243e-eb45-4c70-94e7-4b11e41772ad): DROP TABLE IF EXISTS cbo_t3 2018-01-05T11:29:12,611 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,611 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS cbo_t3 2018-01-05T11:29:12,611 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:12,611 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,612 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,612 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,612 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,612 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,612 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,612 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t3 2018-01-05T11:29:12,612 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t3 2018-01-05T11:29:12,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,615 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:12,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,615 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:29:12,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,615 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:12,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=3, flushCache_()=0} 2018-01-05T11:29:12,615 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112912_ba63243e-eb45-4c70-94e7-4b11e41772ad); Time taken: 0.004 seconds 2018-01-05T11:29:12,615 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,615 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,616 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112912_ba63243e-eb45-4c70-94e7-4b11e41772ad): DROP TABLE IF EXISTS cbo_t3 2018-01-05T11:29:12,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,617 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS cbo_t3 2018-01-05T11:29:12,617 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:29:12,617 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@cbo_t3 2018-01-05T11:29:12,617 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: default@cbo_t3 2018-01-05T11:29:12,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,617 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:29:12,617 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,617 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t3 2018-01-05T11:29:12,617 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t3 2018-01-05T11:29:12,619 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,620 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,620 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=cbo_t3 2018-01-05T11:29:12,620 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=cbo_t3 2018-01-05T11:29:12,622 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,623 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,623 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,623 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: drop_table : db=default tbl=cbo_t3 2018-01-05T11:29:12,623 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=drop_table : db=default tbl=cbo_t3 2018-01-05T11:29:12,625 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.119106ms + 0.002822ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc] 2018-01-05T11:29:12,631 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,632 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] utils.FileUtils: deleting file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/cbo_t3 2018-01-05T11:29:12,632 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,632 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.DDLTask: Ignoring request to add WriteEntity(default@cbo_t3) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@cbo_t3) Type=TABLE WriteType=DDL_EXCLUSIVE is present 2018-01-05T11:29:12,632 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,632 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,632 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS cbo_t3 2018-01-05T11:29:12,632 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:29:12,632 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@cbo_t3 2018-01-05T11:29:12,632 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: default@cbo_t3 2018-01-05T11:29:12,632 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,632 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,632 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:12,632 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=12, getTable_(String, String, )=3} 2018-01-05T11:29:12,632 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112912_ba63243e-eb45-4c70-94e7-4b11e41772ad); Time taken: 0.017 seconds 2018-01-05T11:29:12,632 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:12,632 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,632 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,632 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,632 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query DROP TABLE IF EXISTS cbo_t3 2018-01-05T11:29:12,632 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.021 seconds 2018-01-05T11:29:12,632 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,632 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:12,632 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,632 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:12,633 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,633 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:12,633 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,633 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,633 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,633 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS src_cbo 2018-01-05T11:29:12,633 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112912_3398cb2d-f5b9-4293-b703-05654c1be80b): DROP TABLE IF EXISTS src_cbo 2018-01-05T11:29:12,633 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,633 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS src_cbo 2018-01-05T11:29:12,633 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:12,633 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,633 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,633 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,633 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,633 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,633 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,633 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_cbo 2018-01-05T11:29:12,633 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_cbo 2018-01-05T11:29:12,636 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,637 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,637 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:12,637 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,637 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:29:12,637 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,637 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:12,637 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=4, flushCache_()=0} 2018-01-05T11:29:12,637 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112912_3398cb2d-f5b9-4293-b703-05654c1be80b); Time taken: 0.004 seconds 2018-01-05T11:29:12,637 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,637 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,637 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112912_3398cb2d-f5b9-4293-b703-05654c1be80b): DROP TABLE IF EXISTS src_cbo 2018-01-05T11:29:12,638 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,638 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS src_cbo 2018-01-05T11:29:12,638 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:29:12,638 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@src_cbo 2018-01-05T11:29:12,638 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: default@src_cbo 2018-01-05T11:29:12,638 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,638 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,638 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,638 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,638 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:29:12,638 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,638 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_cbo 2018-01-05T11:29:12,638 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_cbo 2018-01-05T11:29:12,641 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,641 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,641 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,641 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=src_cbo 2018-01-05T11:29:12,641 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=src_cbo 2018-01-05T11:29:12,643 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,644 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,644 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,644 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: drop_table : db=default tbl=src_cbo 2018-01-05T11:29:12,644 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=drop_table : db=default tbl=src_cbo 2018-01-05T11:29:12,646 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.12ms + 0.002432ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc] 2018-01-05T11:29:12,654 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,654 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] utils.FileUtils: deleting file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/src_cbo 2018-01-05T11:29:12,654 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,654 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.DDLTask: Ignoring request to add WriteEntity(default@src_cbo) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@src_cbo) Type=TABLE WriteType=DDL_EXCLUSIVE is present 2018-01-05T11:29:12,655 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,655 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,655 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS src_cbo 2018-01-05T11:29:12,655 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:29:12,655 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@src_cbo 2018-01-05T11:29:12,655 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: default@src_cbo 2018-01-05T11:29:12,655 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,655 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,655 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:12,655 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=13, getTable_(String, String, )=3} 2018-01-05T11:29:12,655 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112912_3398cb2d-f5b9-4293-b703-05654c1be80b); Time taken: 0.018 seconds 2018-01-05T11:29:12,655 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:12,655 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,655 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,655 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,655 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query DROP TABLE IF EXISTS src_cbo 2018-01-05T11:29:12,655 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.022 seconds 2018-01-05T11:29:12,655 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,655 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:12,655 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,655 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:12,655 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,655 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:12,655 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,655 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,655 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,655 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS part 2018-01-05T11:29:12,655 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112912_786d3f31-053d-453f-8172-29768a1ca538): DROP TABLE IF EXISTS part 2018-01-05T11:29:12,655 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,655 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS part 2018-01-05T11:29:12,655 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:12,655 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,655 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,655 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,655 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,655 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,655 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,655 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=part 2018-01-05T11:29:12,655 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=part 2018-01-05T11:29:12,658 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,658 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,658 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:12,658 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,658 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:29:12,658 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,658 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:12,658 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=3, flushCache_()=0} 2018-01-05T11:29:12,658 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112912_786d3f31-053d-453f-8172-29768a1ca538); Time taken: 0.003 seconds 2018-01-05T11:29:12,659 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,659 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,659 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112912_786d3f31-053d-453f-8172-29768a1ca538): DROP TABLE IF EXISTS part 2018-01-05T11:29:12,659 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,660 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS part 2018-01-05T11:29:12,660 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:29:12,660 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@part 2018-01-05T11:29:12,660 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: default@part 2018-01-05T11:29:12,660 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,660 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,660 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,660 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,660 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:29:12,660 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,660 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=part 2018-01-05T11:29:12,660 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=part 2018-01-05T11:29:12,662 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,663 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,663 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,663 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=part 2018-01-05T11:29:12,663 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=part 2018-01-05T11:29:12,665 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,665 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,666 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,666 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: drop_table : db=default tbl=part 2018-01-05T11:29:12,666 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=drop_table : db=default tbl=part 2018-01-05T11:29:12,690 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.138197ms + 0.003619ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc] 2018-01-05T11:29:12,701 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,701 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] utils.FileUtils: deleting file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/part 2018-01-05T11:29:12,702 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,702 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.DDLTask: Ignoring request to add WriteEntity(default@part) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@part) Type=TABLE WriteType=DDL_EXCLUSIVE is present 2018-01-05T11:29:12,702 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,702 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,702 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS part 2018-01-05T11:29:12,702 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:29:12,702 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@part 2018-01-05T11:29:12,702 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: default@part 2018-01-05T11:29:12,702 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,702 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,702 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:12,702 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=39, getTable_(String, String, )=3} 2018-01-05T11:29:12,702 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112912_786d3f31-053d-453f-8172-29768a1ca538); Time taken: 0.043 seconds 2018-01-05T11:29:12,702 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:12,702 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,702 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,702 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,702 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query DROP TABLE IF EXISTS part 2018-01-05T11:29:12,702 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.047 seconds 2018-01-05T11:29:12,702 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,702 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:12,702 INFO [main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,702 INFO [main] session.SessionState: Updating thread name to dfdd8629-6c16-4f61-9218-1f1e0156f79c main 2018-01-05T11:29:12,702 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,702 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Acquired the compile lock. 2018-01-05T11:29:12,702 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,702 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,702 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,702 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.VariableSubstitution: Substitution is on: DROP TABLE IF EXISTS lineitem 2018-01-05T11:29:12,702 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Compiling command(queryId=djaiswal_20180105112912_b5836788-50c9-48d8-8038-9d7af0ac9c27): DROP TABLE IF EXISTS lineitem 2018-01-05T11:29:12,703 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,703 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parsing command: DROP TABLE IF EXISTS lineitem 2018-01-05T11:29:12,703 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] parse.ParseDriver: Parse Completed 2018-01-05T11:29:12,703 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,703 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,703 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,703 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,703 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,703 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,703 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=lineitem 2018-01-05T11:29:12,703 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=lineitem 2018-01-05T11:29:12,706 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,706 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,706 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Semantic Analysis Completed 2018-01-05T11:29:12,706 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,706 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2018-01-05T11:29:12,706 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,706 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase 2018-01-05T11:29:12,706 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, getTable_(String, String, )=3, flushCache_()=0} 2018-01-05T11:29:12,706 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed compiling command(queryId=djaiswal_20180105112912_b5836788-50c9-48d8-8038-9d7af0ac9c27); Time taken: 0.004 seconds 2018-01-05T11:29:12,706 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,706 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,706 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Executing command(queryId=djaiswal_20180105112912_b5836788-50c9-48d8-8038-9d7af0ac9c27): DROP TABLE IF EXISTS lineitem 2018-01-05T11:29:12,708 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,708 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: query: DROP TABLE IF EXISTS lineitem 2018-01-05T11:29:12,708 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: type: DROPTABLE 2018-01-05T11:29:12,708 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Input: default@lineitem 2018-01-05T11:29:12,708 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: PREHOOK: Output: default@lineitem 2018-01-05T11:29:12,708 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,708 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,708 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,708 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,708 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Starting task [Stage-0:DDL] in serial mode 2018-01-05T11:29:12,708 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,708 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=lineitem 2018-01-05T11:29:12,708 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=lineitem 2018-01-05T11:29:12,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,713 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,713 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: get_table : db=default tbl=lineitem 2018-01-05T11:29:12,714 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=get_table : db=default tbl=lineitem 2018-01-05T11:29:12,716 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,716 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,717 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,717 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.HiveMetaStore: 0: drop_table : db=default tbl=lineitem 2018-01-05T11:29:12,717 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=drop_table : db=default tbl=lineitem 2018-01-05T11:29:12,720 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metastore.MetaStoreDirectSql: Direct SQL query in 0.191372ms + 0.003358ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? order by "PART_NAME" asc] 2018-01-05T11:29:12,731 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] pool.PoolBase: HikariPool-1 - Reset (autoCommit) on connection org.apache.derby.impl.jdbc.EmbedConnection@1181041291 (XID = 167), (SESSIONID = 1), (DATABASE = /Users/djaiswal/hive/hive/itests/qtest/target/tmp/junit_metastore_db), (DRDAID = null) 2018-01-05T11:29:12,731 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] utils.FileUtils: deleting file:/Users/djaiswal/hive/hive/itests/qtest/target/localfs/warehouse/lineitem 2018-01-05T11:29:12,732 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metrics.PerfLogger: 2018-01-05T11:29:12,732 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] exec.DDLTask: Ignoring request to add WriteEntity(default@lineitem) Type=TABLE WriteType=DDL_NO_LOCK because WriteEntity(default@lineitem) Type=TABLE WriteType=DDL_EXCLUSIVE is present 2018-01-05T11:29:12,732 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,732 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,732 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: query: DROP TABLE IF EXISTS lineitem 2018-01-05T11:29:12,732 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: type: DROPTABLE 2018-01-05T11:29:12,732 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Input: default@lineitem 2018-01-05T11:29:12,732 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] SessionState: POSTHOOK: Output: default@lineitem 2018-01-05T11:29:12,732 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,732 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,732 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Dumping metastore api call timing information for : execution phase 2018-01-05T11:29:12,732 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(Configuration, )=0, dropTable_(String, String, boolean, boolean, boolean, )=19, getTable_(String, String, )=5} 2018-01-05T11:29:12,732 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Completed executing command(queryId=djaiswal_20180105112912_b5836788-50c9-48d8-8038-9d7af0ac9c27); Time taken: 0.026 seconds 2018-01-05T11:29:12,732 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: OK 2018-01-05T11:29:12,732 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,732 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager 2018-01-05T11:29:12,732 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] log.PerfLogger: 2018-01-05T11:29:12,732 DEBUG [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] ql.Driver: Shutting down query DROP TABLE IF EXISTS lineitem 2018-01-05T11:29:12,732 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] CliDriver: Time taken: 0.03 seconds 2018-01-05T11:29:12,732 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] conf.HiveConf: Using the default value passed in for log id: dfdd8629-6c16-4f61-9218-1f1e0156f79c 2018-01-05T11:29:12,732 INFO [dfdd8629-6c16-4f61-9218-1f1e0156f79c main] session.SessionState: Resetting thread name to main 2018-01-05T11:29:12,733 WARN [main] tez.TezSessionPoolManager: We are closing a non-default session because of retry failure. 2018-01-05T11:29:12,733 INFO [main] tez.TezSessionState: Closing Tez Session 2018-01-05T11:29:12,733 INFO [main] client.TezClient: Shutting down Tez Session, sessionName=HIVE-9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b, applicationId=application_1515180518813_0001 2018-01-05T11:29:12,737 INFO [IPC Server handler 0 on 49823] client.DAGClientHandler: Received message to shutdown AM from djaiswal (auth:SIMPLE) at 10.22.8.182 2018-01-05T11:29:12,737 INFO [IPC Server handler 0 on 49823] rm.TaskSchedulerManager: TaskScheduler notified that it should unregister from RM 2018-01-05T11:29:12,737 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: No current running DAG, shutting down the AM 2018-01-05T11:29:12,737 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: DAGAppMasterShutdownHandler invoked 2018-01-05T11:29:12,737 INFO [IPC Server handler 0 on 49823] app.DAGAppMaster: Handling DAGAppMaster shutdown 2018-01-05T11:29:12,738 INFO [AMShutdownThread] app.DAGAppMaster: Sleeping for 5000 ms before shutting down 2018-01-05T11:29:12,741 DEBUG [main] service.AbstractService: Service: org.apache.tez.dag.app.DAGAppMaster entered state STOPPED 2018-01-05T11:29:12,742 DEBUG [main] service.AbstractService: Service: org.apache.tez.dag.history.HistoryEventHandler entered state STOPPED 2018-01-05T11:29:12,742 INFO [main] history.HistoryEventHandler: Stopping HistoryEventHandler 2018-01-05T11:29:12,742 DEBUG [main] service.CompositeService: org.apache.tez.dag.history.HistoryEventHandler: stopping services, size=2 2018-01-05T11:29:12,743 DEBUG [main] service.CompositeService: Stopping service #1: Service org.apache.tez.dag.history.recovery.RecoveryService in state org.apache.tez.dag.history.recovery.RecoveryService: STARTED 2018-01-05T11:29:12,743 DEBUG [main] service.AbstractService: Service: org.apache.tez.dag.history.recovery.RecoveryService entered state STOPPED 2018-01-05T11:29:12,743 INFO [main] recovery.RecoveryService: Stopping RecoveryService 2018-01-05T11:29:12,743 INFO [main] recovery.RecoveryService: Handle the remaining events in queue, queue size=0 2018-01-05T11:29:12,743 INFO [RecoveryEventHandlingThread] recovery.RecoveryService: EventQueue take interrupted. Returning 2018-01-05T11:29:12,743 INFO [main] recovery.RecoveryService: Closing Summary Stream 2018-01-05T11:29:12,743 DEBUG [main] service.CompositeService: Stopping service #0: Service org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService in state org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService: STARTED 2018-01-05T11:29:12,743 DEBUG [main] service.AbstractService: Service: org.apache.tez.dag.history.logging.impl.SimpleHistoryLoggingService entered state STOPPED 2018-01-05T11:29:12,743 INFO [main] impl.SimpleHistoryLoggingService: Stopping SimpleHistoryLoggingService, eventQueueBacklog=0 2018-01-05T11:29:12,743 INFO [HistoryEventHandlingThread] impl.SimpleHistoryLoggingService: EventQueue take interrupted. Returning 2018-01-05T11:29:12,743 DEBUG [main] service.AbstractService: Service: org.apache.tez.dag.app.launcher.ContainerLauncherManager entered state STOPPED 2018-01-05T11:29:12,743 DEBUG [main] service.AbstractService: Service: org.apache.tez.dag.app.launcher.LocalContainerLauncher entered state STOPPED 2018-01-05T11:29:12,743 DEBUG [main] service.AbstractService: Service: org.apache.hadoop.hive.llap.tezplugins.LlapContainerLauncher entered state STOPPED 2018-01-05T11:29:12,743 DEBUG [main] service.AbstractService: Service: org.apache.tez.dag.app.rm.TaskSchedulerManager entered state STOPPED 2018-01-05T11:29:12,744 DEBUG [main] service.AbstractService: Service: org.apache.tez.dag.app.rm.LocalTaskSchedulerService entered state STOPPED 2018-01-05T11:29:12,745 DEBUG [main] service.AbstractService: Service: org.apache.hadoop.hive.llap.tezplugins.LlapTaskSchedulerService entered state STOPPED 2018-01-05T11:29:12,745 INFO [LlapSchedulerNodeEnabler] tezplugins.LlapTaskSchedulerService: NodeEnabler thread interrupted after shutdown 2018-01-05T11:29:12,745 INFO [main] tezplugins.LlapTaskSchedulerService: NodeEnablerThread was cancelled 2018-01-05T11:29:12,745 INFO [main] tezplugins.LlapTaskSchedulerService: DelayedTaskSchedulerThread was cancelled 2018-01-05T11:29:12,745 INFO [LlapSchedulerDelayedTaskHandler] tezplugins.LlapTaskSchedulerService: DelayedTaskScheduler thread interrupted after shutdown 2018-01-05T11:29:12,745 INFO [main] tezplugins.LlapTaskSchedulerService: SchedulerThread was cancelled 2018-01-05T11:29:12,745 INFO [LlapScheduler] tezplugins.LlapTaskSchedulerService: Scheduler thread interrupted after shutdown 2018-01-05T11:29:12,745 DEBUG [main] service.AbstractService: Service: LlapRegistryService entered state STOPPED 2018-01-05T11:29:12,750 DEBUG [main] imps.CuratorFrameworkImpl: Closing 2018-01-05T11:29:12,751 INFO [Curator-Framework-0] imps.CuratorFrameworkImpl: backgroundOperationsLoop exiting 2018-01-05T11:29:12,751 DEBUG [main] curator.CuratorZookeeperClient: Closing 2018-01-05T11:29:12,751 DEBUG [main] curator.ConnectionState: Closing 2018-01-05T11:29:12,752 INFO [ProcessThread(sid:0 cport:-1):] server.PrepRequestProcessor: Processed session termination for sessionid: 0x160c7cbc1600003 2018-01-05T11:29:12,752 INFO [main] zookeeper.ZooKeeper: Session: 0x160c7cbc1600003 closed 2018-01-05T11:29:12,758 DEBUG [main] imps.CuratorFrameworkImpl: Closing 2018-01-05T11:29:12,758 INFO [Curator-Framework-0] imps.CuratorFrameworkImpl: backgroundOperationsLoop exiting 2018-01-05T11:29:12,758 DEBUG [main] curator.CuratorZookeeperClient: Closing 2018-01-05T11:29:12,758 DEBUG [main] curator.ConnectionState: Closing 2018-01-05T11:29:12,758 INFO [ProcessThread(sid:0 cport:-1):] server.PrepRequestProcessor: Processed session termination for sessionid: 0x160c7cbc1600004 2018-01-05T11:29:12,758 INFO [main] zookeeper.ZooKeeper: Session: 0x160c7cbc1600004 closed 2018-01-05T11:29:12,759 INFO [main] rm.TaskSchedulerManager: Shutting down AppCallbackExecutor 2018-01-05T11:29:12,759 DEBUG [main] service.AbstractService: Service: AMNodeTracker entered state STOPPED 2018-01-05T11:29:12,759 DEBUG [main] service.AbstractService: Service: AMContainerMaps entered state STOPPED 2018-01-05T11:29:12,759 DEBUG [main] service.AbstractService: Service: org.apache.tez.dag.app.TaskCommunicatorManager entered state STOPPED 2018-01-05T11:29:12,760 DEBUG [main] service.AbstractService: Service: org.apache.tez.dag.app.TezLocalTaskCommunicatorImpl entered state STOPPED 2018-01-05T11:29:12,761 DEBUG [main] service.AbstractService: Service: org.apache.hadoop.hive.llap.tezplugins.LlapTaskCommunicator entered state STOPPED 2018-01-05T11:29:12,766 DEBUG [main] service.AbstractService: Service: LlapProtocolClientProxy entered state STOPPED 2018-01-05T11:29:12,768 INFO [RequestManagerExecutor] llap.AsyncPbRpcProxy: CallScheduler loop exiting 2018-01-05T11:29:12,768 DEBUG [main] service.AbstractService: Service: ContainerHeartbeatHandler entered state STOPPED 2018-01-05T11:29:12,768 DEBUG [main] service.AbstractService: Service: TaskHeartbeatHandler entered state STOPPED 2018-01-05T11:29:12,768 DEBUG [main] service.AbstractService: Service: DAGClientRPCServer entered state STOPPED 2018-01-05T11:29:12,769 DEBUG [main] service.AbstractService: Service: Central entered state STOPPED 2018-01-05T11:29:12,769 DEBUG [main] service.CompositeService: Central: stopping services, size=1 2018-01-05T11:29:12,769 DEBUG [main] service.CompositeService: Stopping service #0: Service Speculator in state Speculator: STARTED 2018-01-05T11:29:12,769 DEBUG [main] service.AbstractService: Service: Speculator entered state STOPPED 2018-01-05T11:29:12,770 DEBUG [main] service.CompositeService: Speculator: stopping services, size=0 2018-01-05T11:29:12,781 INFO [main] app.DAGAppMaster: Completed deletion of tez scratch data dir, path=file:/Users/djaiswal/hive/hive/itests/qtest/target/tmp/scratchdir/djaiswal/_tez_session_dir/9e67dc2b-fe37-45fa-8f55-fbdc9bf3346b/.tez/application_1515180518813_0001 2018-01-05T11:29:12,787 DEBUG [main] tez.TezSessionPoolManager: Closed a pool session [org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager@751df544] 2018-01-05T11:29:12,789 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:49384] server.NIOServerCnxnFactory: NIOServerCnxn factory exited run method 2018-01-05T11:29:12,789 INFO [main] server.ZooKeeperServer: shutting down 2018-01-05T11:29:12,789 INFO [main] server.SessionTrackerImpl: Shutting down 2018-01-05T11:29:12,789 INFO [main] server.PrepRequestProcessor: Shutting down 2018-01-05T11:29:12,789 INFO [main] server.SyncRequestProcessor: Shutting down 2018-01-05T11:29:12,789 INFO [ProcessThread(sid:0 cport:-1):] server.PrepRequestProcessor: PrepRequestProcessor exited loop! 2018-01-05T11:29:12,789 INFO [SyncThread:0] server.SyncRequestProcessor: SyncRequestProcessor exited! 2018-01-05T11:29:12,789 INFO [main] server.FinalRequestProcessor: shutdown of request processor complete 2018-01-05T11:29:12,790 INFO [main] zookeeper.MiniZooKeeperCluster: Shutdown MiniZK cluster with all ZK servers 2018-01-05T11:29:12,792 DEBUG [main] metadata.Hive: Closing current thread's connection to Hive Metastore. 2018-01-05T11:29:12,792 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:12,792 INFO [main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore... 2018-01-05T11:29:12,792 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Cleaning up thread local RawStore... 2018-01-05T11:29:12,792 INFO [main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore 2018-01-05T11:29:12,792 INFO [main] HiveMetaStore.audit: ugi=djaiswal ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore 2018-01-05T11:29:12,792 DEBUG [main] metrics.PerfLogger: 2018-01-05T11:29:12,793 INFO [main] control.CoreCliDriver: Teardown done. ElapsedTime(ms)=1012 2018-01-05T11:29:12,836 DEBUG [Thread-2] util.ShutdownHookManager: ShutdownHookManger complete shutdown.